The Performance Gap of Policy Information Systems: A Knowledge Infrastructure Assessment Framework

Abstract

Digital technologies’ development and their ubiquity has significantly changed the way information is collected and shared. These changes are also observed in the ways that knowledge and information used to design and implement Science, Technology, and Innovation (STI) policies are increasingly made accessible via digital platforms. Lacking, however, are evaluation frameworks to measure the performance and effectiveness of public information systems used for STI policy work. This limits what we know about what aspects work for whom, when, and why. As a response to this gap limiting our collective capacity for improving their utility, this paper presents an assessment framework tool for STI policy-focused digital knowledge platforms. Our proposition is informed by theoretical lessons from the areas of work on Knowledge Infrastructures (KIs) and Next Generation Repositories (NGRs), and practical experiences from policy professionals working in STI policy domains. The tool’s architecture is structured around three interdependent thematic pillars of performance in the production, aggregation, distribution, and maintenance of knowledge in digital information platforms: the communities pillar, the technical systems pillar, and the sustainability pillar. To test the design and utility of our proposed evaluation framework, we applied it on a digital platform of STI policy instruments maintained by the United Nations Educational, Scientific, and Cultural Organization (UNESCO), the Global Observatory for Science, Technology, and Innovation Policy (GO-SPIN) platform. We conclude with reflections on future areas for evaluation framework development.