Open Science Online Course

What is responsible researcher evaluation?

Responsible researcher evaluation is a key element of responsible science. The policy is supported by our commitment to both international and national declarations on research and researcher assessment, such as the Declaration on Research Assessment (DORA), the international Agreement on Reforming Research Assessment (CoARA) and the national recommendation on responsible researcher evaluation in Finland (PDF).

The principles of responsible assessment are transparency, integrity, equity, competence and diversity.

The National recommendation on responsible publication metrics, which was published in Finnish in 2020, lists nine recommendations for the responsible use of publication metrics. The recommendation is based on international recommendations.

1. Quantitative indicators may be used to support qualitative expert evaluation in the evaluation of scientific activities. When evaluating an individual researcher, expert evaluation should be used as the main assessment method.
2. Publication metrics shall be based on data relevant to the subject of the evaluation. Known limitations of the data to be used should be disclosed.
3. The data, analytical methods and results used to produce publication metrics shall be as open and transparent as possible. It should be possible for the subject of the evaluation to check the data used as the basis for the analysis, as well as the results of the analysis if possible.
4. The use of publication metrics takes into account the differences and multidisciplinary nature of the disciplines.
5. The publication metric indicators used in the evaluation should be chosen so that they can meet the objectives of the evaluation.
6. The results should be reported with the accuracy of the indicator values relevant to the subject of the assessment, the methods and the data. Inappropriate indicators should be excluded from reporting.
7. Expertise in the field should be used in the production and utilization of publication metrics.
8. An organization committed to the recommendation must ensure that the prerequisites and expertise of publication metrics are adequately resourced. The organization must provide its staff with training in the responsible use of publication metrics.
9. Organizations committed to the recommendation define the body responsible for their own organization to contact if publishing metrics are used irresponsibly.

Sources: 

https://www.aka.fi/en/research-funding/responsible-science/responsible-researcher-evaluation/

Rosti, T., Sandgren, T., Sassali, J., Sipola, T., Suojärvi, E.-M., n.d. Suositukset vastuulliseen julkaisumetriikan käyttöön. [Recommendations for responsible use of publication metrics.]

The evaluation of research and researchers holds significant weight as it directs the conduct of research. Responsible evaluation is a crucial element in responsible research.

LUT University has signed the DORA declaration. This declaration provides guidelines for conducting research for organizations, researchers, and other stakeholders in the research field. DORA, among other things, guides universities to openly and clearly define the criteria for recruitment, career paths, and promotion decisions, and to consider all research outputs, their value, and impact in research evaluation, in addition to research publications.

The national recommendation for responsible researcher evaluation is part of the open science declaration, and LUT University is automatically committed to it. The general principles of researcher evaluation listed in the national recommendation are:

Transparency: Evaluation is conducted as transparently as possible and carefully documented.

Integrity: Evaluation work follows the recognized practices of the research community.

Fairness: Evaluatees are treated equally and impartially.

Competence: Evaluators have the necessary expertise for the evaluation.

Diversity: Evaluation considers the diversity of research and outputs.

The recommendation targets the evaluation of individual researchers, but the same principles apply when evaluating larger units. As part of the national recommendation, there is also a recommendation for the responsible use of publication metrics.

LUT University has also joined CoARA (Coalition for Advancing Research Assessment), a European coalition for reforming research evaluation. Each organization that signs CoARA creates its own plan for achieving the goals. The agreement focuses on actions that promote the diversity, openness, and non-discrimination of research work and its evaluation, thereby enhancing the quality and impact of science and research.