Science, Technologies, Innovations №4(32) 2024, 110-122 р.

http://doi.org/10.35668/2520-6524-2024-4-12

Vavilina N. I. — Senior Researcher, State Scientific Institution “Ukrainian Institute of Scientific and Technical Expertise and Information”, 180, Antonovycha Str., Kyiv, Ukraine, 03150; +38 (044) 521-00-29; vavilina@uintei.kiev.ua; ORCID: 0000-0002-4861-2810

Charkina O. O. — M. Sc., Kyiv, Ukraine, +38 (098) 298-46-99; charkina.alezandra@gmail.com; ORCID: 0009-0006-6641-227х

EVALUATION OF THE QUALITY OF SCIENTIFIC RESEARCH: WORLD EXPERIENCE

Abstract. New approaches and principles of the quality and effectiveness of science evaluation in accordance with the San Francisco Declaration on Research Evaluation (DORA) were studied. It was summarized the experience of the world countries in search for the new multiple forms and methods of evaluating the quality of research in conditions of departure from scientometric indicators. Active participation in the search for forms and ways of improving the evaluation of the quality of scientific research and the career growth of scientists is accepted by many movements for evaluation reforms. They offer a variety of evaluation options, resulting in a new field of research that examines the effectiveness of the proposed approaches. The lack of standardized approaches and indicators for evaluating the effectiveness of science was determined: almost developed countries that carry out full expert evaluation of scientific activity have certain differences. They depend on the level of economic, geographical and historical development, mental characteristics, etc. It was noted that certain elements of this experience are interesting for the domestic practice of science assessment: it is expedient to carry out an examination of the impact of scientific research on the socio-economic development of Ukraine. The conditions and principles of conducting an objective assessment of the results of scientific research in Ukraine have been determined. One of the main places among them is the principle of academic integrity.

Keywords: Quality of scientific research, assessment reform, impact indicators, expert assessment of quality, effectiveness of science, effectiveness of assessment methods.

REFERENCES

  1. 1. Ozgur, C., & Brown, J. R. (2018). Assessment of Research Quality. Retrieved from: https://www.researchgate.net/publication/325654373_Assessment_of_Research_Quality5_21.
  2. Bornmann, L, Butz, A., & Wohlrabe, K. (2018). What are the top five journals in economics? A new meta-ranking. Applied Economics. 50 (6), 659–675. DOI: https://www.tandfonline.com/doi/full/10.1080/00036846.2017.1332753.
  3. Harzing, A.-W. (2017). Bibliometrics in the Arts, Humanities, and Social Sciences. HARZING.COM. Retrieved from: https://harzing.com/blog/2017/03/bibliometrics-in-the-arts-humanities-and-social-sciences.
  4. Horovyi, V. M. (2015). Kryterii yakosti naukovykh doslidzhen u konteksti zabezpechennia natsionalnykh interesiv [Criteria for the quality of scientific research in the context of ensuring national interests]. Visnyk Natsionalnoi akademii nauk Ukrainy [Bulletin of the National Academy of Sciences of Ukraine]. 6, 74–80. Retrieved from: http://dspace.nbuv.gov.ua/handle/123456789/84897. [in Ukr.].
  5. Malytskyi, B. A. (2017). Kto y kak dolzhen otsenyvat nauchnыe rezultatы uchenoho [Who and how should evaluate the scientific results of a scientist]. Nauka ta naukoznavstvo [Science and science]. 3, 34–53. [in Ukr.].
  6. Drobnokhod, M. I. Do kontseptsii reformuvannia naukovoi sfery Ukrainy [To the concept of reforming the scientific sphere of Ukraine]. Akademiia nauk vyshchoi shkoly Ukrainy. Retrieved from: https://anvsu.org.ua/drobnohod-m-%d1%96-do-koncepc%d1%96%d1%97-reformuv/. [in Ukr.].
  7. Odotiuk, I. V. (2012). Otsinka rezultativ naukovoi diialnosti v Ukraini: normatyvno-pravovyi aspekt [Evaluation of the results of scientific activity in Ukraine: regulatory and legal aspect]. Problemy innovatsiino-investytsiinoho rozvytku [Problems of innovation and investment development]. 3, 38–42. [in Ukr.].
  8. Shablysta, L. M. (2019). Pokaznyky yakosti rezultativ naukovykh doslidzhen [Indicators of the quality of the results of scientific research]. Statystyka Ukrainy [Statistics of Ukraine]. 4 (87), 89–106. DOI: https://doi.org/10.31767/su.4(87)2019.04.10. [in Ukr.].
  9. Pavliuk, K. (2019). Problemy otsiniuvannia naukovoi diialnosti [Problems of evaluating scientific activity]. Naukovi pratsi NDFI [Scientific works of NDFI]. 4, 5–19. DOI: https://doi.org/10.33763/npndfi2019.04.005. [in Ukr.].
  10. Yehorov, I. Yu. (2014). Otsinky rezultativ naukovoi diialnosti: tradytsiini pidkhody ta novi vyklyky [Evaluation of the results of scientific activity: traditional approaches and new challenges]. Nauka ta naukoznavstvo [Science and science]. 3, 42–47. [in Ukr.].
  11. Pysarenko, T. V., & Kvasha, T. K. (2022). Krytychni tekhnolohii: rezultaty forsaitnoho doslidzhennia v Ukraini u 2021 rotsi [Critical technologies: results of a foresight study in Ukraine in 2021]. Nauka, tekhnolohii, innovatsii [Science, technology, innovation]. 1 (21), 38–45. DOI: http://doi.org/10.35668/2520-6524-2022-1-06. [in Ukr.].
  12. Musina, L. A., & Kulak, A. O. (2024). Analiz ta otsinka yakosti ekonomichnoho zrostannia v Ukraini na osnovi mizhnarodnykh reitynhiv [Analysis and assessment of the quality of economic growth in Ukraine based on international ratings]. Nauka, tekhnolohii, innovatsii [Science, technology, innovation]. 1 (29), 8–20. DOI: http://doi.org/10.35668/2520-6524-2024-1-02. [in Ukr.].
  13. Kuranda, T. K., Shved, N. Yu., Osadcha, A. B., & Vavilina, N. I. (2023). Analiz svitovykh tendentsii rozvytku nauky za napriamamy morskykh doslidzhen [Global trends analysis of science development in the areas of marine research]. Nauka, tekhnolohii, innovatsii [Science, technology, innovation]. 3 (27), 3–15. DOI: http://doi.org/10.35668/2520-6524-2023-3-01 [in Ukr.].
  14. Glaser, J., & Oltersdorf, J. (2019). Persistent Problems for a Bibliometrics of Social Sciences and Humanities and How to Overcome Them. 17th International Conference On Scientometrics & Informetrics, August. Retrieved from: https://www.researchgate.net/publication/335490767_Persistent_Problems_for_a_Bibliometrics_of_Social_Sciences_and_Humanities_and_How_to_Overcome_Them.
  15. Belter, C. W. (2015). Bibliometric indicators: opportunities and limits. Journal of the Medical Library Association. 103 (4), 219–221. DOI: 10.3163/1536-5050.103.4.014.
  16. Bornmann, L., & Wohlrabe, K. (2017). Normalization of Citation Impact in Economics. CESifo Working Paper. Munich. No. 6592, 60 p. Retrieved from: https://www.econstor.eu/bitstream/10419/171056/1/cesifo1_wp6592.pdf.
  17. Pain, E. (2023). How academia is exploring new approaches for evaluating researchers. Science Adveser. Retrieved from: https://www.science.org/content/article/how-academia-is-exploring-new-approaches-for-evaluating-researchers.
  18. (2021). NOR-CAM – A toolbox for recognition and rewards in academic careers. Universities Norway. Edited 05.12.2022. Retrieved from: https://www.uhr.no/en/news-from-uhr/nor-cam-a-toolbox-for-recognition-and-rewards-in-academic-areers.5780.aspx.
  19. Career path and evaluation policy for Professorial Staff (ZAP). Ghent University. Retrieved from: https://www.ugent.be/en/work/mobility-career/career-aspects/professorial-staff.
  20. Academic Career Framework and Promotions Processes. UCL. Retrieved from: https://www.ucl.ac.uk/human-resources/policies-advice/academic-career-framework-and-promotions-processes.
  21. Research Assessment at EMBL. EMBL. Retrieved from: https://www.embl.org/about/research-assessment/.
  22. Benedictus, R., Miedema, F., & Ferguson, M. W. J. (2016). Fewer numbers, better science. Nature. 538, 453–455. Retrieved from: https://www.nature.com/articles/538453a#auth-Rinze-Benedictus-Aff1.
  23. San Francisco Declaration on Research Assessment. DORA. Retrieved from: https://sfdora.org/read/.
  24. (2014). Criteria for the evaluation of research units: the HCERES standards. HCERES. 32 p. Retrieved from: https://www.hceres.fr/sites/default/files/media/downloads/Hceres_campaign_2016_2017_Criteria_Research_Units.pdf.
  25. (2021). Référentiel d’évaluation des organismes de recherche. Hcéres, Département d’évaluation des organisms. 12 p. Retrieved from: https://www.hceres.fr/sites/default/files/media/downloads/referentiel-devaluation-des-organismes-de-recherche_1.pdf.
  26. (2016). An independent review of university research funding by Lord Nicholas Stern. Independent report. Research Excellence Framework review. Last updated 31 December 2018. Retrieved from: https://www.gov.uk/government/publications/research-excellence-framework-review. DOI: 10.1038/520429a.
  27. (2020). Framework for Research Assessment Exercise (RAE). 26 p. Retrieved from: https://www.ugc.edu.hk/doc/eng/ugc/rae/2020/framework.pdf.
  28. Research Excellence Framework 2028: Summary of stakeholder engagements leading to initial decisions on the Research Excellence Framework 2028 (REF2028/23/02). 38 p. Retrieved from: https://repository.jisc.ac.uk/9181/1/REF2028-research-excellence-framework-2028.pdf.
  29. The Strategy Evaluation Protocol 2021–2027. VSNU, KNAW, NWO. 48 p. Retrieved from: https://storage.knaw.nl/2022-06/SEP_2021-2027.pdf.
  30. (2014). Standard Evaluation Protocol 2015–2021. Protocol for Research Assessments in the Netherlands. Voorburg. 32 p. Retrieved from: https://medialibrary.uantwerpen.be/files/12115/2461924c-a2c0-4a6c-9b4f-baa64b716a6b.pdf.
  31. Yehorchenko, I. (2022). Yak otsiniuvaty ukrainsku nauku: chotyry vazhlyvi rechi [How to evaluate Ukrainian science: four important things]. Dzerkalo tyzhnia [Mirror of the week]. Retrieved from: https://zn.ua/ukr/science/jak-otsinjuvati-ukrajinsku-nauku-chotiri-vazhlivi-rechi.html [in Ukr.].