Inadequate science evaluation: a call for change, also in research culture
Report on the Conference “Beyond impact factor, h-Index and university rankings”
This international conference held on 21 November 2018 in Bern highlighted the limitations of current metrics in capturing scientific quality and introduced elements of alternative assessment approaches.
The Swiss Academy of Sciences (SCNAT) organised the conference on behalf of the Swiss Academies of Arts and Sciences, as part of its “We Scientists Shape Science” initiative. Renowned experts highlighted the limitations of metrics in capturing scientific quality and the resulting pressure on the quality of scientific output and presented approaches that challenge conventional metrics. The implications for the Swiss science landscape long-term were the subject of a stakeholder roundtable and discussion with the audience.
The following points emerged from the presentations and discussion:
- Current metrics do not measure and therefore do not reflect scientific quality
- The h-Index and Journal Impact Factor are pseudo-objective and unhealthy for research
- Broader “portfolio” approaches are needed that also take into account criteria such as outreach, managerial, mentoring or teaching skills
- Defining meaningful and discipline-specific indicators requires co-design by producers and users of evaluation results
- Enhancing science assessments needs changes in research culture
- Renewing the social contract between science and society needs an agreement on the goals to be achieved by science and on the indicators to measure its contribution
- Changing metrics requires concerted global action by science stakeholders, namely in relation to the private sector providers of metric and ranking systems
The Swiss Academies will continue their work on these issues as part of their focus area on research culture.