Although scientists need to be utterly committed to their work, chance often plays a major role in important discoveries.

A case in point, according to Telmo Pievani, professor of philosophy of biological sciences at the University of Padua, was the Japanese marine biologist Osamu Shimomura, who was “studying jellyfish in the Atlantic Ocean and, out of pure curiosity, produced a taxonomy of the proteins found in their luminescence”. His accidental discovery of aequorin, “used today, in every biological department in the world, to mark cells, gene expression and so on”, netted him a Nobel Prize in Chemistry in 2008.

Professor Pievani’s new book, Serendipity: The Unexpected in Science, offers a range of other striking examples. Antibacterials have been found on the skin of an African frog and Pluto’s elusive moons were spotted on a single anomalous photograph that a researcher decided not to discard. Our understanding of diabetes was transformed by “the sweet pee of a dog”, when scientists removed its pancreas and happened to notice that its sugary urine was attracting flies. Although a trial made clear that Viagra was ineffective in treating heart problems, its “side-effects” meant that men were reluctant to return any unused pills.

Given the vital role of serendipity in science, Professor Pievani’s book goes on to argue, it is often “the new, little-known scientist” who can “break free of the chains of established knowledge”. Meanwhile, the “giants” in their fields can remain “trapped within the framework of established habits” and so are “less inclined to listen to the unexpected”.

����Vlog

So, what are the implications of such arguments for how we should organise and fund science?

There is clearly a place for “big science”, relying on “big data, lots of money, great technicians and so on”, Professor Pievani told ����Vlog, since this is essential in fields such as theoretical physics and brain research. Yet it also remained crucial to “create ecological niches for small science and slow – or, at least, slower – science, without the anxiety of having to publish a new paper every month which is just a minor addition to the previous one”.

����Vlog

In dealing with funding foundations, charities and even the scientific committee of his own department at Padua – the highest-ranked department of biology in Italy – Professor Pievani said that he always argued: “If we have €100 million to invest in science, let us use €80 million for mainstream research, where we need results and are sure to make progress. But please let us invest 10 or 20 per cent of the funds on high-risk research by young researchers who are just asking good questions, new questions.”

Although the risk of failure was great, he went on, “pure research questions” often led to valuable applications in the long run. The idea of “looking at cancer as an evolutionary system”, for example, was driven by pure intellectual curiosity, but it turned out that “if you apply this metaphor to cancer, you can develop better theories about the behaviour of tumours. So please also fund curiosity!”

In his own department, reported Professor Pievani, they put out “a specific call each year for young researchers willing to ask good questions. So, each unit can get more money if it doesn’t just work in the mainstream but proposes other lines of research – which benefits not only the young researcher but the whole unit and its principal investigator”. In at least three cases, the university funded the early stages of projects that went on to gain grants from the European Research Council.

Another key to promoting serendipity is interdisciplinarity. In studying human evolution, Professor Pievani pointed out, “we have to bring together geneticists, palaeontologists, archaeologists, experts in linguistics, palaeo-ecologists and others to produce something new. If you are focused on a scientific problem and someone else, with another analytical language or perspective, is focusing on the same problem, he or she may suggest something you are unable to see for yourself.”

����Vlog

Looking globally, Professor Pievani noted that China had “a planned, authoritarian approach to science” and was producing vast amounts of data-heavy research in fields such as immunotherapy, although this often built on fundamental ideas “produced in Western countries on the basis of pure curiosity”. Rather than competing head on, the more open societies of Europe and North America should perhaps focus their research on the big fundamental questions where serendipity played a central role.

A country such as Italy, reflected Professor Pievani, has “a wonderful history of craftsmanship, so in manufacturing we focus on the quality of our products. We cannot compete with China in terms of investment or quantity. That also applies to the manufacturing of ideas.”

Serendipity: The Unexpected in Science is published by MIT Press on 3 September.