Overview
For over 20 years, several European digital policies have measured progress in the European Member States through monitoring activities such as benchmarking or maturity assessments. However, legal and technical requirements in the EU initiatives related to the government's digital transformation and interoperability require a holistic approach that integrates knowledge and reduces the administrative burden that its reporting entails to the essential minimum. With this in mind, the study analyses the 'ecosystem' of indicators in several well-established EU monitoring activities involving digital public services, interoperability, digital rights and multi-level governance aspects. It also uncovers gaps and inefficiencies in the various stages of the monitoring exercise lifecycle. The study reveals several areas for improvement, such as the need to improve monitoring asset documentation and their availability in timely and adequate formats to encourage their reuse. These actions can help streamline the monitoring efforts of the EU transitioning to a data-driven approach more aligned with the European digital decade and its 2030 targets.
Quick guide
- Section 1 Rationale, scope and objectives in the larger policy context of digital transformation of the public sector in the EU
- Section 2 the definition of monitoring and the underlying policy context
- Section 3 The emerging landscape of relevant monitoring schemes and the monitoring schemes investigated:
- the European Interoperability Framework (EIF) monitoring,
- Berlin Declaration monitoring,
- the Digital Economy and Society Index (DESI)
- and the eGovernment Benchmark (eGov).
- Section 4 a detailed explanation of the selected monitoring approaches in terms of indicator documentation and the content of 6 indicators.
- Section 5 provides the results of interviews with key stakeholders from the EC and the first set of issues emerging from the Member States.
- Section 6 summarises the main burdens, benefits, and gaps found by the work based on stakeholder inputs and desk research
- Section 7 Steps towards and potential solutions
- Section 8 concludes this report with a summary of the work.
The findings presented in this document are the basis for the subsequent report entitled “Identifying opportunities for streamlining European monitoring of digital policies”, which provides strategic analysis, baseline, and recommendations for stakeholders to take forward in the subsequent phases.
Key messages
Digital monitoring in the public sector is in a good state. Collaborations are mature, but there is a favourable moment to explore alternative approaches to keep relevance and foster efficiencies to improve the collective understanding of the digital landscape for the public sector across Europe.
The EU policy landscape of monitoring interoperability in public administrations and the digital transformation of government is broad and detailed but with insufficient coordination. This situation has led to added reporting burdens and inefficiencies, resulting in requests from the MS to streamline and better organise monitoring and reporting.
The maturity of individual monitoring schemes led by the Directorate-General for Informatics (DG DIGIT) and the Directorate General for Communication Networks, Content and Technology (DG CONNECT) can be recognised by their regular engagement with stakeholders in the MS, including the periodic review of indicators and validation of outputs, often pointing to areas of improvement that would aid the MS in their digital developments, as well as their standing in the monitoring schemes’ metrics.
The challenges in this context are both policy-related and technical, including the increasing amount of emerging digital policy relevant to the study’s context, including local and regional developments and data-sharing contexts like dataspaces. Other stakeholders are also engaged in digital topics, such as the Directorate-General for Structural Reform Support (DG REFORM), considering this from the point-of-view of public sector 5 modernisation. Technology’s pace of development is also a notable concern, where the current approaches are not agile enough to rapidly assess potential benefits and impacts, including for any policy response.
Methodologically, the approach to assessment is questioned by the EC and the MS, where issues of triangulation of results are emerging, as well as requests for a reduction of scope or increasing the priority of certain topics. In some cases, the items monitored are becoming redundant. The latter can be seen by leading MS having maximum achievable scores leading to a form of saturation in the results and an inability to move into new topics. In addition, monitoring schemes are used to assess technical and policy solution uptake for past priorities. Results are almost wholly achieved in all MS but are still being measured.
Such issues point to a need to adopt interoperable approaches to manage monitoring schemes’ assets, including their documentation, metadata on their indicators and how data is exchanged and reused. Such an approach implies improved coordination between activities, including reviewing the scope of activities and the timing of data collection and delivery.
It is timely, therefore, to consider the methodological approaches needed to establish a minimised burden on stakeholders (e.g., increased re-use of existing data flows and the potential of automated monitoring approaches) while being able to assess progress on digital across MS' public sector actors and to consider the measurement of impacts and outcomes in digital policy.
Given issues of trust in the activities’ results, it is important to adopt collaborative multi-stakeholder approaches. Piloting and co-creation should incrementally explore alternatives agilely, which should be a focus for the next phase of this work, including ensuring such an approach is sustainable based on the draft options presented in this report and any others that stakeholders would like to consider.
Keywords
Monitoring, indicator, public sector, interoperability, digital transformation of government, digital government, Digital Decade, European digital policies, EIF, NIFO, DESI, eGovernment Benchmark, Berlin Declaration.
How to cite
Hernandez Quiros, L., S. Smith, R. and Schade, S., Landscaping the monitoring of interoperability and digital transformation, Publications Office of the European Union, Luxembourg, 2023, doi:10.2760/350656, JRC133628.