Mistrust, Dissonance, And Noise: The Assessment of Technological Activities in Argentina’s Research Career System
DOI:
https://doi.org/10.4067/s0718-2724202600024942Keywords:
research evaulation, Research careers, technology transfer, ArgentinaAbstract
Despite the growing emphasis on technology transfer in university missions, academic career evaluation systems continue to privilege a narrow set of research outputs, leaving many technology transfer products and activities at the margins of academic recognition. This article examines how technological activities and products (TAPs) are evaluated within Argentina’s CONICET research career system. Drawing on 421 peer review reports in applied fields, we combine lexicometric and qualitative analysis to explore the place of TAPs in evaluative discourse and the difficulties they generate. TAPs rarely occupy a central position: relative to publications, training and prospective research, they form a weak lexical cluster. When reviewers do discuss TAPs, they repeatedly point to (i) insufficient and uneven documentation that undermines comparability; (ii) “dissonance” between information sources that forces reviewers to reconstruct transfer trajectories from fragmented records; and (iii) uncertainty and mistrust about whether reported activities genuinely represent innovation, their relevance, and the applicant’s specific contribution. We interpret these dynamics through the distinction between productive and unproductive variability in peer review. Some disagreement reflects legitimate plural standards for judging heterogeneous transfer outputs. Yet procedural shortcomings and mismatches between documentation sources also generate evaluative noise: fragmented evidence and uneven reporting formats invite over-reliance on heuristics and amplify reviewer’s personal judgments.
Downloads
References
Abma-Schouten, R., Gijbels, J., Reijmerink, W., & Meijer, I. (2023). Evaluation of research proposals by peer review panels: Broader panels for broader assessments? Science and Public Policy, 50(4), 619-632. https://doi.org/10.1093/scipol/scad009
Aczel, B., Barwich, A.-S., Diekman, A. B., Fishbach, A., Goldstone, R. L., Gomez, P., Gundersen, O. E., von Hippel, P. T., Holcombe, A. O., Lewandowsky, S., Nozari, N., Pestilli, F., & Ioannidis, J. P. A. (2025). The present and future of peer review: Ideas, interventions, and evidence. Proceedings of the National Academy of Sciences, 122(5), e2401232121. https://doi.org/10.1073/pnas.2401232121
Alperin, J. P., Muñoz Nieves, C., Schimanski, L. A., Fischman, G. E., Niles, M. T., & McKiernan, E. C. (2019). How significant are the public dimensions of faculty work in review, promotion and tenure documents? eLife, 8, e42254. https://doi.org/10.7554/eLife.42254
Arboleda, G., & Plazas Tenorio, A. (2024). Innovation management in university research groups. Journal of Technology Management and Innovation, 19(2), 16-27. https://doi.org/10.4067/S0718-27242024000200016
Benneworth, P., & Olmos-Peñuela, J. (2019). Ex ante evaluation of societal impact in research: Towards a conceptually-based reflection [Working paper]. https://hvlopen.brage.unit.no/hvlopen-xmlui/handle/11250/2631353
Bornmann, L. (2013). What is societal impact of research and how can it be assessed? A literature survey. Journal of the American Society for Information Science and Technology, 64(2), 217-233. https://doi.org/10.1002/asi.22803
Bouwma-Gearhart, J., Lenhart, C., Carter, R., Mundorff, K., Cho, H., & Knoch, J. (2021). Inclusively recognizing faculty innovation and entrepreneurship impact within promotion and tenure considerations. Journal of Open Innovation: Technology, Market, and Complexity, 7(3), 182. https://doi.org/10.3390/joitmc7030182
Bruhn, J. G. (2008). Value dissonance and ethics failure in academia: A causal connection? Journal of Academic Ethics, 6(1), 17-32. https://doi.org/10.1007/s10805-008-9054-z
Calderón-Altamirano, E., & Rodriguez, J. C. (2023). Transferencia de tecnología universidad-industria en las universidades públicas estatales de México: Un análisis configuracional. Journal of Technology Management and Innovation, 18(4), 18-30. https://doi.org/10.4067/S0718-27242023000400018
Cañibano, C., Woolley, R., Iversen, E. J., & Corona-Sobrino, C. (2024). The state-of-the-art of research on science research careers. In A. Oancea, G. E. Derrick, N. Nuseibeh, & X. Xu (Eds.), Handbook of metaresearch (pp. 293-308). Edward Elgar Publishing. https://doi.org/10.4337/9781839105722.00032
Consejo Latinoamericano de Ciencias Sociales. (2022). Declaración: Una nueva evaluación académica y científica para una ciencia con relevancia social en América Latina y el Caribe. Foro Latinoamericano de Evaluación Científica (FOLEC). https://biblioteca-repositorio.clacso.edu.ar/handle/CLACSO/169563
Consejo Nacional de Investigaciones Científicas y Técnicas. (2024). CONICET en cifras. https://cifras.conicet.gov.ar/publica/grupografico/show-publico/36
Davies, C., & Ingram, H. (2025). Sceptics and champions: Participant insights on the use of partial randomization to allocate research culture funding. Research Evaluation, 34, rvaf006. https://doi.org/10.1093/reseval/rvaf006
Derrick, G. E., & Samuel, G. (2016). The evaluation scale: Exploring decisions about societal impact in peer review panels. Minerva, 54(1), 75-97. https://doi.org/10.1007/s11024-016-9290-0
Derrick, G. E., & Samuel, G. (2017). The future of societal impact assessment using peer review: Pre-evaluation training, consensus building and inter-reviewer reliability. Palgrave Communications, 3, 17040. https://doi.org/10.1057/palcomms.2017.40
D'Este, P., & Patel, P. (2007). University-industry linkages in the UK: What are the factors underlying the variety of interactions with industry? Research Policy, 36(9), 1295-1313. https://doi.org/10.1016/j.respol.2007.05.002
Donovan, C. (2007). The qualitative future of research evaluation. Science and Public Policy, 34(8), 585-597. https://doi.org/10.3152/030234207X256538
Donovan, C. (2011). State of the art in assessing research impact: Introduction to a special issue. Research Evaluation, 20(3), 175-179. https://doi.org/10.3152/095820211X13118583635918
Dutrénit, G., & Arza, V. (2010). Channels and benefits of interactions between public research organisations and industry: Comparing four Latin American countries. Science and Public Policy, 37(7), 541-553. https://doi.org/10.3152/030234210X512043
Fecher, B., & Hebing, M. (2021). How do researchers approach societal impact? PLOS ONE, 16(7), e0254006. https://doi.org/10.1371/journal.pone.0254006
Friesike, S., Dobusch, L., & Heimstädt, M. (2022). Striving for societal impact as an early-career researcher: Reflections on five common concerns. In A. A. Gümüsay, E. Marti, H. Trittin-Ulbrich, & C. Wickert (Eds.), Organizing for societal grand challenges (pp. 239-255). Emerald Publishing. https://doi.org/10.1108/S0733-558X20220000079022
Genshaft, J., Wickert, J., Gray-Little, B., Hanson, K., Marchase, R., Schiffer, P. E., & Tanner, M. (2016). Consideration of technology transfer in tenure and promotion. Technology & Innovation, 17(4), 197-204. https://doi.org/10.3727/194982416X14520374943103
Hicks, D. (2012). Performance-based university research funding systems. Research Policy, 41(2), 251-261. https://doi.org/10.1016/j.respol.2011.09.007
Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015). Bibliometrics: The Leiden Manifesto for research metrics. Nature, 520(7548), 429-431. https://doi.org/10.1038/520429a
Kolarz, P., Vingre, A., Vinnik, A., Neto, A., Vergara, C., Obando Rodriguez, C., Nielsen, K., & Sutinen, L. (2023). Review of peer review: Final report. UK Research and Innovation. https://www.ukri.org/publications/review-of-peer-review/review-of-peer-review-june-2023/
Langfeldt, L., Nedeva, M., Sörlin, S., & Thomas, D. A. (2020). Co-existing notions of research quality: A framework to study context-specific understandings of good research. Minerva, 58(1), 115-137. https://doi.org/10.1007/s11024-019-09385-2
Liaudat, S., Zukerfeld, M., & Terlizzi, M. S. (2025). Barriers to local use of publicly funded knowledge: Cognitive appropriation and technology transfer in Argentina. Journal of Technology Management and Innovation, 20(4), 60-69. https://www.jotmi.org/index.php/GT/article/view/4859
Lockett, A., & Wright, M. (2005). Resources, capabilities, risk capital and the creation of university spin-out companies. Research Policy, 34(7), 1043-1057. https://doi.org/10.1016/j.respol.2005.05.006
Luo, J., Ma, L., & Shankar, K. (2021). Does the inclusion of non-academic reviewers make any difference for grant impact panels? Science and Public Policy, 48(6), 763-775. https://doi.org/10.1093/scipol/scab046
Ma, L., & Agnew, R. (2022). Deconstructing impact: A framework for impact evaluation in grant applications. Science and Public Policy, 49(2), 289-301. https://doi.org/10.1093/scipol/scab080
Ma, L., Luo, J., Feliciani, T., & Shankar, K. (2020). How to evaluate ex ante impact of funding proposals? An analysis of reviewers' comments on impact statements. Research Evaluation, 29(4), 431-440. https://doi.org/10.1093/reseval/rvaa022
Mathias de Lima, B. B., de Oliveira Lacerda, R. T., & Becker, M. (2023). Academic spin-off management: A bibliometric study. Journal of Technology Management and Innovation, 18(2), 69-80. https://doi.org/10.4067/S0718-27242023000200069
Mayring, P. (2022). Qualitative content analysis: A step-by-step guide. SAGE Publications. https://doi.org/10.4135/9781036231798
Melkers, J., Woolley, R., & Kreth, Q. (2023). Research funding and scientific careers. In B. Lepori, B. Jongbloed, & D. Hicks (Eds.), Handbook on the public funding of research (pp. 301-321). Edward Elgar Publishing.
Moher, D., Bouter, L., Kleinert, S., Glasziou, P., Barbour, V., Coriat, A.-M., Sham, M. H., Foeger, N., & Dirnagl, U. (2020). The Hong Kong Principles for assessing researchers: Fostering research integrity. PLOS Biology, 18(7), e3000737. https://doi.org/10.1371/journal.pbio.3000737
Molas-Gallart, J., Salter, A., Patel, P., Scott, A., & Duran, X. (2002). Measuring third stream activities: Final report to the Russell Group of Universities. SPRU, University of Sussex.
Morgan-Thomas, A., Tsoukas, S., Dudau, A., & Gąska, P. (2024). Beyond declarations: Metrics, rankings and responsible assessment. Research Policy, 53(10), 105093. https://doi.org/10.1016/j.respol.2024.105093
Naidorf, J., Vasen, F., Alonso, M., & Cuschnir, M. (2020). De evaluar diferente a orientar como siempre: Burocratización e inercias institucionales en la implementación de una política científica orientada al desarrollo tecnológico y social. Revista Iberoamericana de Ciencia, Tecnología y Sociedad - CTS, 15(45), 131-161. https://ojs.revistacts.net/index.php/CTS/article/view/178
Niembro, A., & Svampa, F. (2025). Territorial inequalities and (de)concentration of public investment in science: A study on CONICET (Argentina) and the tensions between academic excellence and equity. Minerva, 63(3), 499-533. https://doi.org/10.1007/s11024-024-09550-2
Norn, M. T., Aagaard, K., Bjørnholm, J., & Stage, A. K. (2024). Funder strategies for promoting research addressing societal challenges: Thematic, impact, and collaboration targeting. Science and Public Policy, 51(5), 910-922. https://doi.org/10.1093/scipol/scae033
Oxley, K., & Gulbrandsen, M. (2025). Variability and negligence: Grant peer review panels evaluating impact ex ante. Science and Public Policy, 52(2), 254-268. https://doi.org/10.1093/scipol/scae081
Pekkola, E., & Siekkinen, T. (Eds.). (2024). Tenure tracks in European universities: Managing careers in academia. Edward Elgar Publishing. https://doi.org/10.4337/9781035302451
Peruginelli, G., & Pölönen, J. (2023). The legal foundation of responsible research assessment: An overview on European Union and Italy. Research Evaluation, 32(4), 670-682. https://doi.org/10.1093/reseval/rvad035
Pinheiro, R., Benneworth, P., & Jones, G. A. (Eds.). (2012). Universities and regional development: A critical assessment of tensions and contradictions. Routledge.
Pontika, N., Klebel, T., Correia, A., Metzler, H., Knoth, P., & Ross-Hellauer, T. (2022). Indicators of research quality, quantity, openness, and responsibility in institutional review, promotion, and tenure policies across seven countries. Quantitative Science Studies, 3(4), 888-911. https://doi.org/10.1162/qss_a_00224
Power, M. (2015). How accounting begins: Object formation and the accretion of infrastructure. Accounting, Organizations and Society, 47, 43-55. https://doi.org/10.1016/j.aos.2015.10.005
Ramos Vielba, I., Thomas, D., & Aagaard, K. (2022). Societal targeting in researcher funding: An exploratory approach. Research Evaluation, 31(2), 202-213. https://doi.org/10.1093/reseval/rvab044
Reale, E., Avramov, D., Canhial, K., Donovan, C., Flecha, R., Holm, P., Larkin, C., Lepori, B., Mosoni-Fried, J., Oliver, E., Primeri, E., Puigvert, L., Scharnhorst, A., Schubert, A., Soler, M., Soós, S., Sordé, T., Travis, C., & van Horik, R. (2018). A review of literature on evaluating the scientific, social and political impact of social sciences and humanities research. Research Evaluation, 27(4), 298-308. https://doi.org/10.1093/reseval/rvx025
Reinert, M. (1986). Un logiciel d'analyse lexicale: ALCESTE. Les Cahiers de l'Analyse des Données, 11(4), 471-484.
Reinert, M. (1990). Alceste: Une méthodologie d'analyse des données textuelles et une application: Aurélia de Gérard de Nerval. Bulletin of Sociological Methodology/Bulletin de Méthodologie Sociologique, 26(1), 24-54. https://doi.org/10.1177/075910639002600103
Repiso, R., & Montero, J. (2024). Sale el Sexenio de Transferencia (mal, pero sale). Anuario ThinkEPI, 18. https://doi.org/10.3145/thinkepi.2024.e18a14
Rice, D. B., Raffoul, H., Ioannidis, J. P. A., & Moher, D. (2020). Academic criteria for promotion and tenure in biomedical sciences faculties: Cross sectional analysis of international sample of universities. BMJ, 369, m2081. https://doi.org/10.1136/bmj.m2081
Romano, S., Car, V., & Locher, V. (2020). Reflexiones metodológicas acerca del uso de estadísticas de datos textuales para el análisis de los discursos. RevCom, 10. http://sedici.unlp.edu.ar/handle/10915/107660
Ross-Hellauer, T., Klebel, T., Knoth, P., & Pontika, N. (2024). Value dissonance in research(er) assessment: Individual and perceived institutional priorities in review, promotion, and tenure. Science and Public Policy, 51(3), 337-351. https://doi.org/10.1093/scipol/scad073
Roumbanis, L. (2019). Peer review or lottery? A critical analysis of two different forms of decision-making mechanisms for allocation of research grants. Science, Technology, & Human Values, 44(6), 994-1019. https://doi.org/10.1177/0162243918822744
Rovelli, L. (2024). Debates actuales y reformas en curso en la evaluación responsable de la investigación en América Latina y el Caribe. Oficina Regional de UNESCO en Montevideo y CLACSO.
Rushforth, A. (2025a). Beyond impact factors? Lessons from the Dutch attempt to transform academic research assessment. SocArXiv. https://doi.org/10.31235/osf.io/hza3c.v3
Rushforth, A. (2025b). Research assessment reform as collective action problem: Contested framings of research system transformation. Minerva, 63, 1-21. https://doi.org/10.1007/s11024-025-09573-3
San Francisco Declaration on Research Assessment. (2013). San Francisco Declaration on Research Assessment. https://sfdora.org/read/
Schweiger, G., Barnett, A., van den Besselaar, P., Bornmann, L., De Block, A., Ioannidis, J. P. A., Sandström, U., & Conix, S. (2024). The costs of competition in distributing scarce research funds. Proceedings of the National Academy of Sciences, 121(50), e2407644121. https://doi.org/10.1073/pnas.2407644121
Seeber, M., Svege, I., & Hesselberg, J. O. (2024). Improving the efficiency of research proposals evaluation: A two-stage procedure. Research Evaluation, rvae020. https://doi.org/10.1093/reseval/rvae020
Smit, J. P., & Hessels, L. K. (2021). The production of scientific and societal value in research evaluation: A review of societal impact assessment methods. Research Evaluation, 30(3), 323-335. https://doi.org/10.1093/reseval/rvab002
UNESCO. (2021). Recomendación sobre la ciencia abierta. https://unesdoc.unesco.org/ark:/48223/pf0000378841_spa
Urbanc, M., & de Jong, S. (2025). Navigating societal impact: Strategic management in Horizon 2020 SSH projects. Minerva. https://doi.org/10.1007/s11024-024-09566-8
Vasen, F. (2018). The "ivory tower" as safe bet: Science policies and research evaluation in Mexico. Education Policy Analysis Archives, 26(95). https://doi.org/10.14507/epaa.26.3594
Vasen, F. (2025). A delicate balance: Responsible evaluation, tenure review, and the pursuit of institutional innovation. Science and Public Policy, 52(4), 545-556. https://doi.org/10.1093/scipol/scaf003
Vasen, F., & Sierra Pereiro, M. (2022). "The hardest task": Peer review and the evaluation of technological activities. Minerva, 60(3), 375-395. https://doi.org/10.1007/s11024-022-09461-0
Williams, K., & Grant, J. (2018). A comparative review of how the policy and procedures to assess research impact evolved in Australia and the UK. Research Evaluation, 27(2), 93-105. https://doi.org/10.1093/reseval/rvx042
Wilsdon, J. (2016). The metric tide: Independent review of the role of metrics in research assessment and management. SAGE. https://doi.org/10.4135/9781473978782
Wróblewska, M. N. (2021). Research impact evaluation and academic discourse. Humanities and Social Sciences Communications, 8(1), 58. https://doi.org/10.1057/s41599-021-00727-8
Wróblewska, M. N. (2025). One size fits all? A comparative review of policy-making in the area of research impact evaluation in the UK, Poland and Norway. Research Evaluation, 34, rvaf010. https://doi.org/10.1093/reseval/rvaf010
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Federico Vasen, Nerina Sarthou, Silvina A. Romano
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
