Informator uniwersytecki

Nowadays, academics are under the constant scrutiny of our ‘performance’. In fact, every year or every 2 years. This has been promoted as the only way to ensure better and more productive scientists. But, if the goal is to take science to the top, then the goal should be to tackle the most difficult problems, and those are the hardest and more time-consuming to solve, as well as the ones that involve many researchers and disciplines. Therefore, why to make short-term evaluations? Should not be the other way around and give time to really take on those challenges?

The big question is HOW to evaluate researchers?
On one hand there is the “field”, where some fields have millions of scientists, some others have very few. Examples: cancer research vs quantum physics.

One could agree that a field-adjusted, centralised and quantitative assessment of the works of scientists seems a way to go. This is then approached by a classification of journals per field. Yet, the system is flawed for a variety of reasons. First, because the categorisation of journals is usually based on the journal’s impact factor (IF). And IFs are (over)used as the only measuring stick. A practice that has been proven deficient by multiple agencies e.g. the Declaration on Research Assessment (DoRA) (https://sfdora.org/read/) and the Leiden Manifesto (https://www.nature.com/articles/520429a).

But let’s go a little deeper into the actual problem:
Imagine we have 2 researchers within the very same ‘field’: molecular reproduction. One of them works with Drosophila (fruit flies) while the other researcher works with African elephants (Loxodonta africana). Drosophila has a fast development, 10 days from fertilisation to adulthood, while elephants have a gestation (pregnancy) alone of 18-22 months and several years to reach adulthood.

These 2 researchers are anyway likely to publish in the same journals (reproduction, molecular reproduction, endocrinology, etc), nevertheless their time scale for experimentation cannot be compared. Not to mention the risks, ethical issues, costs and difficulties to work with large animals vs just fruit flies. Yet, these 2 researchers would be evaluated within the same criteria every single time.

You might now think this is an odd example, an exaggeration, but it is not. Many researchers work with cell lines, thus they can have daily tests, while others use animal models that take weeks to months to years. Some research only involves questionnaires or datamining, followed by statistics, while other involves creating novel things, optimising them, and eventually place them into a context – a process that takes from months to years. All of these researchers are doing science, and none is more important than others, in fact they are all needed. The question here is: How can any numerical system account for these differences in time and effort?

Let’s pick some real examples now:
Frederick Sanger, twice Nobel laureate in Chemistry for deciphering proteins and later for sequencing RNA and DNA, who in between those discoveries spent periods of more than 10 years without a single publication (https://www.science.org/doi/full/10.1126/science.1249912). He would be deemed “unproductive” and “lazy” by any evaluation system today.

A more modern example, Katalin Karikó – Nobel laureate for Medicine in 2023. She could not find financial support for her research and eventually was fired from university. Her work was considered not innovative or useful. Fortunately, she, among many others, did not give up, and >20 years of unrecognised work suddenly became salvation for billions when the COVID pandemic hit (you can listen this from herself: https://twitter.com/i/broadcasts/1rmxPMlMzODKN).

Regardless of these examples, and there are a lot more, both the Ministry of Mumsipi and the local authorities believe that they know-how and that they can bring forward a system that is fair, and, more importantly, able to promote the development of national/local science. The result is: the national and local evaluating system for academics, which has degenerated into a travesty similar to Las Vegas – hence the title of this series of commentaries.

In my next commentaries I will discuss the other troubles of the current evaluation system. Although I have multiple times before mentioned other issues associated with the strategies to improve science at the local and national levels (see ‘The decorative serviette‘ , ‘curva peligrosa’, ‘innovation killer’ and Functional stupidity’).


Until then!


Wishing you all a great end of 2024 and a great 2025.
Adolek

A gambling addict to barnacle racing and pitch dropping.

drawing by Adolek