University research governance and the Colombian scientific journal index “PUBLINDEX." Understanding the tensions

The logic of international rankings standards has favoured scientometric indicators for measuring research, focusing mainly on evaluating scientific journals’ citations and impact factors. This focus has led national science policy makers to revise their research evaluation models to comply with international standards, prompting changes in scientific journals that have influenced local scientific production ecosystems.

This doctoral research examines the cascading effect produced by these international demands, which have prompted research-policy changes in Colombia’s National Scientific Journal Index and Policy Instrument, Publindex, influencing the incentives to promote knowledge production and dissemination, sometimes with adverse effects. These internationalization-driven changes in national policy have created tensions that must be managed by university research governance and its actors. This study sets out to understand how university research governance actors respond to the tensions produced by Publindex policy changes to determine the points that should be emphasized to alleviate them.

To this end, this quantitative and qualitative study followed a timeline of Publindex’s most significant events from 1994 to 2021. Literature reviews were conducted in different years, examining existing literature, policy, institutional documents, periodical discussions, and information gathered during academic events. In addition, qualitative data were obtained from 52 interviews, two focus groups, and a survey with different level actors in the scientific journal publishing activity. Furthermore, quantitative data analysed the scientific production data from each case study. Finally, the results were analysed by the organization to understand the institutional changes.

This process revealed the tensions affecting research governance, divided into two central tensions with their associated sub-tensions. Tension 1 involves assessment factors, including internationalization, indicators as a starting point for incentives design, and evaluation mechanisms. Tension 2 involves index journal effects, affecting epistemic communities and autonomy in research agenda and encouraging the last sub-tension in this category, ethical misconduct.

The methodology used followed an embedded case study approach in multi-level environments and cases. One public and two private universities, active within Publindex processes, were used as settings to examine institutional change. The measurable components used to explain institutional change were a) Governance structures, b) Institutional logic, and c) Types of actors, the latter entailing an actors’ role analysis. The cases were subsequently compared to yield the factors that should be considered to mitigate the tensions. The comparative cases showed that developed best practices in term of evaluation mechanism supporting to ethical and integrity instrument can help to mitigate the adverse effects generated for the indicators as a starting point for incentive design. New evaluation mechanism using qualitative and quantitative methods to support epistemic communities as responsible metrics and the diverse manifesto’s recommendations. To alleviate the adverse effects, the key points are developing a coherence narrative at institutional level to develop intrinsic knowledge in the internal governances and institutional logics who responds to the actor’s practices.

Listening to the actors’ voices enables understanding the importance of inclusive evaluation mechanisms, modernizing incentives, and developing ethical policies to mitigate negative consequences. New technologies allow new spaces, formats, and models to produce, disseminate, measure, and regulate knowledge production. More importantly, finding practical solutions to the tensions created by external demands should focus, first and foremost, on understanding how the actors, at different levels, manage and respond to them; this will ensure that the solutions are viable, given the institutional and local contexts.

Keywords: Scientific production policy, knowledge production, Publindex Journal Indexing System, scientific journal policy, university research governance, institutional change.