Uddannelses- og Forskningsudvalget 2016-17
UFU Alm.del Bilag 26
Offentligt
1689070_0001.png
Links between research policy and national academic
performance
A comparative study of Denmark, Sweden and the Netherlands
Main Report
Project partners:
CFA:
The Danish Centre for Studies in Research and Research Policy, Department of Political Science,
Aarhus BSS, Aarhus University.
Technopolis Group:
represented by Faugert & Co Utvärdering AB (Technopolis Sweden) and Technopolis
BV (Technopolis Netherlands)
NIFU:
Nordisk institutt for studier av innovasjon, forskning og utdanning
1
 UFU, Alm.del - 2016-17 - Bilag 26: Henvendelse af 17/11-16 fra Danmark Forsknings- og Innovationspolitiske råd
Table of Contents
Executive summary .............................................................................................................................................................................. 3
1.
Introduction ................................................................................................................................................................................... 5
1.1.
1.2.
2.
3.
Research policy and academic performance: The study in context..................................................... 5
Objectives and main questions ........................................................................................................................... 6
Factors influencing national academic research performance ................................................................................. 9
Research policy and academic performance: Case studies ...................................................................................... 13
3.1.
3.2.
3.3.
Denmark .................................................................................................................................................................... 15
The Netherlands ..................................................................................................................................................... 19
Sweden ....................................................................................................................................................................... 23
Cross-cutting conclusion from the case studies ......................................................................................... 27
Current balances and future challenges........................................................................................................ 29
4.
Conclusion and future perspectives ................................................................................................................................... 27
4.1.
4.2.
Appendix A: Main conclusions concerning DFiR’s six hypotheses ................................................................................. 32
References .............................................................................................................................................................................................. 36
The present report has been commissioned by the Danish Council for Research and Innovation-policy (DFiR) and carried out
by a consortium consisting of CFA, Technopolis Group and NIFU. The report reflects the views of the authors only.
2
 UFU, Alm.del - 2016-17 - Bilag 26: Henvendelse af 17/11-16 fra Danmark Forsknings- og Innovationspolitiske råd
Executive summary
Based on a study commissioned by The Danish Council for Research and Innovation Policy (DFiR) this
report examines the relationship between research policy and research performance in a long term
perspective. The study takes its departure from six hypotheses formulated by DFiR related to research
funding balances, excellence schemes, PhD education, governance structures and collaboration
(international as well as cross sectoral). A cross-country comparison between Denmark, The
Netherlands and Sweden serves to examine the links between such policy changes and aggregated
academic impact in different national settings. By providing an improved understanding of the
foundation for sustained high systemic research performance across countries the study thus aims to
strengthen the foundation for future research policy decisions.
Overall, the study finds support for several of DFiR’s hypotheses: For Denmark it is showed how
changes in the research funding system (including the establishment of the Danish National Research
Foundation), the prioritization of PhD education and a general increased emphasis on academic
quality and internationalization during the late 1980s and early 1990s contributed to reverse a
negative development and subsequently laid the foundation for the constant increase in the
aggregated Danish research performance during the following decades. Also with regard to Sweden
and the Netherlands several factors associated with the hypotheses are highlighted as important
elements in the explanations of the observed national developments. However, as soon as we move
from a single-country perspective towards a comparative cross-country perspective, more
generalizable patterns between specific types of research policy changes and developments in
research performance become less obvious. Rather than universal relationships between specific
policy measures and their effects, we find a number of highly context- specific explanations.
Nevertheless, two central conclusions stand out at a more general level.
First, the highly positive development in performance observed for both Denmark and the Netherlands
since the early 1990s can be partly attributed to the way in which strong national research cultures
have been institutionalized at the system level in the two countries. Although the actual policy
measures put to use differ, they have played the same overall role in creating long-term, stable points
of orientation for institutions and individuals alike. As such they have underpinned a coherent,
national research culture supporting excellence and internationalization. For Sweden, on the other
hand, such long-lasting institutions with sustained system-effects have not been established. Rather,
the Swedish science system has been the subject of more mixed political signals with strategic and
utility-based considerations outweighing a clear focus on research quality as the most central pillar of
the system.
Secondly, the developments in performance across the three countries are also related to the way in
which the central balances of the national science systems have been calibrated over time. While a
strong research culture (and several long-lasting institutions supporting such a culture) is an
important condition for high academic performance at an aggregated level, it is not a sufficient factor.
In order to provide conditions conducive to high research performance across a broad spectrum of
disciplines, national research systems need to uphold a number of central balances for prolonged
periods of time. This includes finding appropriate balances between opportunities for excellent groups
on the one hand and securing a broad and strong growth-layer on the other. It also includes balances
between steering and protected space, as well as balances between renewal and stability. Not least the
funding system, the governance mechanisms and the organization and volume of the PhD education
play important roles in maintaining such balances.
3
 UFU, Alm.del - 2016-17 - Bilag 26: Henvendelse af 17/11-16 fra Danmark Forsknings- og Innovationspolitiske råd
The cases illustrate how systemic imbalances rather quickly can lead to decreasing results, but also
how ongoing well-executed calibrations of the central balances can create stable, long term
foundations for high research performance. The case-studies also show, however, that multiple
balance points between different policy measures may lead to sustained high national academic
performance.
Finally, the study highlights a number of emerging imbalances in all three countries which may have
long- term effects on the research performance of the science systems if they are not corrected. The
study may thus spark a number of research policy discussions on whether current adjustments are
needed to secure sustainable, coherent and well-functioning national science systems in the decades to
come. This calls for informed policy considerations of the future calibration of the central systemic
balances and underlines the importance and the timeliness of the agenda that DFiR has set with this
study.
4
 UFU, Alm.del - 2016-17 - Bilag 26: Henvendelse af 17/11-16 fra Danmark Forsknings- og Innovationspolitiske råd
1. Introduction
This report presents the results of a study commissioned by The Danish Council for Research and
Innovation Policy (DFiR) under the heading ‘World Class Knowledge’ (Viden i verdensklasse). The
report consists of four chapters of which the first briefly outlines the objectives, methodology and data
of the study. Chapter 2 presents the overall analytical approach which guides the analysis in the
subsequent chapter 3 examining the relationships between research policy and research performance
in Denmark, Sweden and the Netherlands. Finally, chapter 4 presents the overall conclusions of the
study and discusses potential future lessons based on the analysis. The present report has been limited
to 30 pages + an appendix, but further documentation and a rich list of references can be found in the
background report of the study.
1.1.
Research policy and academic performance: The study in context
The relationship between research policy and national research performance has high policy relevance
and has been discussed extensively in academic, administrative and political circles in recent years. In
particular the Danish case has attracted attention lately as Denmark currently stands out among the
top performing countries of the world in terms of academic performance measured by bibliometric
indicators. Not least a report by Öquist and Benner (2012) has highlighted the so called “Danish
Miracle” and has raised the question: What factors may explain the long term increase in Danish
research performance measured by mean normalized citation impact indicators?
Based on these discussions DFiR initiated a project aiming to improve our understanding of the
development of Danish research performance. As a first step, a ‘Scientometric mapping of
developments in Danish research performance in the period 1980-2013 at macro- and meso-levels’
was carried out (Schneider and Aagaard 2015). By examining a variety of indicators and comparing
the current standing of Danish research to a selected group of comparable countries the report
investigated the robustness of the performance claims and the consistency of the long term trends.
Overall, the report showed a very robust and consistent picture with a strong Danish performance
documented by a variety of indicators. The performance of Danish research did, however, display a
remarkable drop during the 1980s. This trend was reversed by the end of the 1980s and since then
few other countries have shown the same rate of improvement as Denmark.
Just like Denmark, the two benchmark countries of this study, Sweden and the Netherlands, also had
high performing research systems by the early 1980s. The Dutch system has since then displayed a
consistent high level of performance for more than three decades, while Sweden’s strong position has
faded somewhat. Figure 1 below shows the three selected countries for this study alongside some of
the traditionally strongest research nations of the world (countries such as USA, UK and Switzerland).
A number of more modest performing countries (such as France, Germany and Norway) as well as
some upcoming nations (Spain and China) are also included to provide for the broader context.
However, there are important database-effects that need to be taken into account when the long-term
developments within and between countries are interpreted (see section 1.3 in the background report
and Schneider and Aagaard 2015 for details). In particular, the strong increases in impact for most
countries since 2006 should mainly be ascribed to a large intake of low impact regional journals in the
Web of Science (WoS) database as well as to increased international collaboration which inflates full
counts measures.
5
 UFU, Alm.del - 2016-17 - Bilag 26: Henvendelse af 17/11-16 fra Danmark Forsknings- og Innovationspolitiske råd
1689070_0006.png
Figure 1. Development in mean normalized citation scores (MNCS) based on full counts for 11 countries; based on
three year overlapping publication blocks.
1,6
1,4
1,2
1
0,8
0,6
0,4
DENMARK
NETHERLANDS
SWEDEN
FRANCE
NORWAY
SWITZERLAND
GERMANY
PEOPLES R CHINA
USA
GREAT BRITAIN
SPAIN
Source: Schneider & Aagaard 2015
1.2.
Objectives and main questions
The present study examines the Danish relationship between research policy and research
performance in a comparative perspective and aims to strengthen the foundation for future research
policy decisions. The analysis has an explicit focus on academic performance measured by bibliometric
indicators only. It does not accordingly address other types of impact directly. While it is generally
assumed that there is a positive relationship between high academic research performance and both
societal impact and high quality teaching, it is also acknowledged that this relationship may not exist
under all circumstances. When the results of this study are discussed they must accordingly be placed
in this wider context. It should also be noted that aggregated national academic performance
measured by bibliometric indicators first and foremost reflects the performance of the medical and
health sciences and the natural sciences due to both volume- and coverage issues in the WoS.
From this outset the report presents a quantitative and qualitative based historical analysis exploring
the foundation for the development in Danish research performance from 1980 onwards in
comparison with the corresponding developments in Sweden and the Netherlands. Based on six
hypotheses formulated by DFiR a cross-country comparison serves to examine a number of proposed
explanations in different national settings and at different points in time. The hypotheses suggest that
the development in Danish research performance may be related to six selected policy factors (DFiR
2015). These factors are illustrated in Figure 2 and outlined in more detail below.
6
 UFU, Alm.del - 2016-17 - Bilag 26: Henvendelse af 17/11-16 fra Danmark Forsknings- og Innovationspolitiske råd
1689070_0007.png
Figure 2: Factors examined in the study
Input
Funding
- Institutional
funding
- Project funding
- CoE-funding
Personnel
- PhDs
- International
recruitment
Research process
Management
Output
Impact
Research activities
- International
collaboration
- Cross sectoral
collaboration
Publications
Citations
Hypothesis one
concerns the funding system and in particular the balance between institutional
funding and project funding. It states that the Danish system has benefitted from appropriate balances
between these funding streams and highlights in particular how increased competition over time may
have led to improved performance. It also states that the mix between different types of funding may
have been more appropriate in Denmark than in other comparable countries.
Hypothesis two
concerns the use of Excellence schemes. In particular, it states that the Danish system
has benefitted from an early and well executed prioritization of a large scale excellence initiative
related to the establishment of the Danish National Research Foundation (DNRF). It is argued that such
excellence schemes in addition to their direct effects may have indirect positive spill-over effects on
the research system as a whole.
The third hypothesis
deals with the volume and the organization of the PhD education (and in
broader terms with the recruitment and training of new researchers) and states that explicit and well
timed prioritizations of the PhD system have secured a strong and talented growth layer in the Danish
research system. The mechanisms mentioned include: increased competition for positions, improved
and more formalized frame-conditions and more international recruitment.
The fourth hypothesis
targets the governance of the universities and in particular the internal
research management. It is argued that two reforms of the management system carried out in 1993
and 2003 respectively have strengthened the research management resulting in a strong quality
culture and appropriate recruitment practices. It is in particular highlighted that the Danish 1993 act
may have led to a stronger attention from the central management levels towards attracting the best
international researchers.
Hypothesis five
concerns the internationalization of Danish research and states that the Danish
system has benefitted from a high degree of international research collaboration.
The sixth and final hypothesis
deals with cross sectoral collaboration and in particular
public/private research collaboration. It is stated that a strong Danish tradition of this type of
collaboration may be a contributing factor in explaining the high general impact of Danish research.
Although this study investigates the above mentioned hypotheses, it does not seek to confirm or
disconfirm each individual hypothesis in a strict sense. While factors such as different types of funding,
management, internationalization, collaboration, PhD training and recruitment obviously influence
academic performance, they do so in indirect and interconnected ways and often with considerable
time-lags.
7
 UFU, Alm.del - 2016-17 - Bilag 26: Henvendelse af 17/11-16 fra Danmark Forsknings- og Innovationspolitiske råd
1689070_0008.png
Rather than a set of isolated tests of the individual hypotheses the study is thus an examination of the
interconnectedness of these factors in different national settings at different points in time. The
approach acknowledges that a policy that was successful in one context may not have the same effect
in another and allow for the possibility that the same outcome can follow from different combinations
of conditions. In examining these questions the study provides a discussion of how different balance-
points between specific policy measures influence national research performance. To examine the
relevant relationships the study applies a mixed methods approach using both quantitative and
qualitative data-sources. Figure 3 below shows the work flow of the study.
Figure 3: Methodology and work flow of the study
Systematic literature review on relationships between research policy factors
and research performance
Quantitative and qualitative data
collection at country level
Re-analysis of existing interviews and
collection of new interviews
Brief analysis at country level: Detecting and discussing potential
relationships between research policy changes and changes in performance
for each country with specific emphasis on the six hypotheses
Examination of each hypothesis in a cross country perspective:
interpretation of differences and similarities between the countries and
discussion of possible links to performance
Integrative analysis: concluding analysis connecting the sub-studies and
drawing main conclusions
As outlined above, the point of departure for the study was a comprehensive literature review carried
out in order to establish a solid foundation for the subsequent analyses. Alongside this process the
data- collection and interview-gathering was started within each of the three countries. As part of this
process previously collected interview-material was re-analyzed. In the next phase brief country
descriptions were developed for the three countries in order to outline important system
characteristics and to highlight major policy changes related to the hypotheses for the period under
examination. Policy timelines were constructed for each country and compared to the developments in
performance. In continuation of this, specific analyses were carried out at hypothesis level. The
majority of these were carried out in a cross- country perspective, but in a number of cases the
comparative analyses have been supplemented with more detailed Danish analyses. Finally, all the
analyses at hypothesis level were then examined in an integrative perspective in order to draw the
overall conclusions of the study. The main conclusions are presented in this report, while the literature
review, the country descriptions, the policy timelines, and the documentation of the detailed analyses
at hypothesis level can be found in the background report. Further details on data and methods can
also be found in chapter 1 in the background report.
8
 UFU, Alm.del - 2016-17 - Bilag 26: Henvendelse af 17/11-16 fra Danmark Forsknings- og Innovationspolitiske råd
2. Factors influencing national academic research performance
Based on a literature review (see chapter 2 in the background report) this chapter highlights a number
of overall findings on the relationships between research policy and research performance and relates
these to DFiR’s hypotheses. Based on this, an analytical frame is presented which structures the
country level case studies carried out in chapter 3.
Studies examining aggregated normalized citation impact indicators highlight a stable group of top
performing countries (e.g. King 2004). This selection of countries has hardly changed the last three
decades. Studies also show that competitiveness in research tends to be a package: some nations
perform well at both a specific and a more general level, while others perform less well across the
board. With very few exceptions, all leading scientific nations, both smaller ones such as Switzerland,
Denmark and the Netherlands, and larger ones such as the U.S. and UK are thus world-leading not only
overall, but also in many individual scientific disciplines. This indicates that excellence in individual
disciplines or fields of research is hard to attain without a system that supports excellence at a more
general level (Adams 1998). It is thus highly relevant to study systemic factors of importance for high
impact research in a long term perspective. It also shows, however, that the group of high performing
countries is very diverse in terms of how the national science systems are organized and funded. There
is not one optimal national science policy model according to this literature, but rather a number of
quite different models which all have shown high performance for several decades.
In correspondence with this observation the literature reveal a mixed and complex picture concerning
the importance of individual policy factors. On the one hand it broadly supports the selection of factors
included in DFiR’s hypotheses: Funding mechanisms (including excellence schemes), PhD education,
governance structures and internationalization are highlighted in the literature as important factors
with regard to academic performance. The literature is, however, far from conclusive when it comes to
characterizing these relationships in details. There are in other words no clear crosscutting
conclusions on the mix of factors which together create the conditions for high performance in
national research systems; except that we are dealing with highly complex questions and multiple
possible answers (Geuna and Martin 2003; Auranen and Nieminen 2010). For all the examined factors
positive aspects are highlighted, but it is at the same time underlined that there may be substantial
negative side-effects associated with the same selection of factors. Most of the potential explanatory
factors are likely only conducive to performance within a certain range and under certain contextual
circumstances, meaning that both too much and too little of a factor may have negative effects. In
addition, multiple balance points between these factors may lead to well-functioning systems.
Across all the examined factors a group of key concepts stand out as important preconditions for
continuous high academic performance at both group levels and system levels. These factors include
stability in funding and governance-conditions, flexibility to adjust and adapt research agendas along
the way, long-term horizons, and diversity along a number of dimensions; not least in relation to
funding mechanisms, funding sources and personnel composition (e.g. gender, age, nationality,
disciplinary background) (Laudel 2006; Heinze 2008; Hollingsworth & Gear 2012). Together these
factors create what is labelled ‘protected space’ (Whitley 2011) or ‘room to maneuver’ (Mintzberg
1983). Competition is also highlighted as an important factor, but it is underlined that competition for
funding not necessarily is the only or even the most efficient type of competition. National science
systems can be highly competitive in a variety of other ways; not least through competition for
positions and as the result of strong reputational competition between individuals, groups and
institutions. Furthermore, in high performing research systems these factors operate within strong
10
 UFU, Alm.del - 2016-17 - Bilag 26: Henvendelse af 17/11-16 fra Danmark Forsknings- og Innovationspolitiske råd
1689070_0010.png
national ‘research cultures’ defined as sets of shared values, assumptions, beliefs, rituals and other
forms of behavior which recognize, value and support academic research as a core activity at the
national research institutions. Central characteristics of such cultures are meritocratic mechanisms
based on notions of research excellence integrated in both funding schemes and hiring and promotion
processes. Strong national research cultures also have internationalization as a key element and
rewards and recognizes an explicit orientation towards the most prestigious, international journals as
well as presence in leading international networks within each individual field of science (Altbach and
Salmi 2011; Hollingsworth and Gear 2012).
Based on these observations, a general model outlining the interplay of several important factors for
high performing research systems has been developed. The starting point is the notion that science
systems can be viewed as complex eco-systems where a number of vital balances must be maintained
and continually calibrated in order to secure stable, long term high performance. In this perspective a
national science system can thus be seen as a configuration of a large number of highly interdependent
elements which reinforce each other positively or negatively depending on both internal and external
dynamics. When the system is well balanced the interactions between the various elements are
characterized by positive feedback mechanisms, while the opposite is the case if central imbalances
are allowed to persist.
High performing national research systems
succeed in balancing:
A diverse growth layer and room for
excellence
Stability and change
Diversity and concentration
Steering and protected space
Renewal and continuity
Flexibility and a strong research
culture
Figure 4: The science eco-system
Excellence
Continuous high performance reflects an
ability to recalibrate these balances when
changes in frame-conditions occur, or when
single factors are changed – and an ability to
operate with different balances within
different areas and different scientific fields of
the system.
Stability
Renewal
Flexibility
Diversity
Collaboration
Growth-layer
10
 UFU, Alm.del - 2016-17 - Bilag 26: Henvendelse af 17/11-16 fra Danmark Forsknings- og Innovationspolitiske råd
The factors selected by DFiR play important roles in securing these vital system-balances. The
following section briefly outlines how each factor may influence the overall balances of a national
research system and emphasizes why they must be carefully calibrated in order not to create
unintended systemic effects:
Institutional funding:
Is allocated to secure infra-structure, long-term stability, flexibility, diversity
and a strong growth layer. In it-self institutional funding often lacks incentives for individuals and
institutions to strive for excellence and this type of funding may thus lead to stagnation if the research
culture is weak and/or if other types of competitive forces are absent from the system.
Project funding:
Is allocated to draw out the best ideas and encourage research collaboration. The
composition of the project funding is however important. Few, large grants support the top of the
system and leads to concentration of resources, while many smaller grants draw the balance towards
the growth layer and increased diversity of the system. Similarly, the academic orientation of the
system may be affected by the balance between funding for curiosity-driven research vs. funding for
more strategic or applied research
1
. Also the time horizon of the grants plays a role: Long time-
horizons increase flexibility, protected space and explorative research, while shorter horizons may
lead to more ‘safe’ research. Finally, a high share of project funding creates large transaction costs and
potential unintended effects.
Excellence funding:
Builds on the assumption that stimulating a small number of excellent
performers will have positive effects on the vitality, attractiveness, and productivity of the whole
science system. These schemes have the potential to combine the benefits of both institutional funding
and project funding, but may also have unintended effects if they become too dominant. In such
instances they may lead to excessive concentration of funding, reduced diversity and a more
vulnerable growth layer.
PhD education:
Socializes students to research, reproduces faculty and is thus the primary source of
renewal of any scientific community. PhD education therefore lies at the core of any nation’s research
capacity. The volume must however be calibrated carefully as it can displace investments at the
expense of other parts of the science system and as it may skew the balance between continuity and
renewal in the staff composition.
University Governance:
The external university governance defines the autonomy of the research
institutions and can limit/increase the room to maneuver at lower levels of the institutions. The
internal university governance at the central level may secure healthy system balances within the
institutions, but can at the same time create risks of increased bureaucratization, standardization and
may influence the conditions allowing adaptive organizational forms to develop.
Internationalization:
Plays an important role in explaining research performance, but is mainly
driven by internal scientific dynamics. Internationalization lies at the core of strong research cultures
(within the hard sciences in particular) and may lead to increased competition for positions, access to
additional funding etc., but may also be a threat to the stability and continuity of science systems
which serve other, more nationally oriented, functions than pure academic research (e.g. education and
knowledge exchange).
Cross sectoral collaboration:
Public-private research collaborations may lead to high citation impact
for individual publications, but it is unlikely to be a main driver of aggregated national performance.
Our examinations of this issue are documented in the background report and the appendix of this
report, but the results are not explicitly integrated in this overall analysis.
As briefly outlined above, the factors selected by DFiR play central roles in securing appropriate
balances in high performing science systems. The factors are, however, highly interdependent and
11
 UFU, Alm.del - 2016-17 - Bilag 26: Henvendelse af 17/11-16 fra Danmark Forsknings- og Innovationspolitiske råd
1689070_0012.png
fairly similar factors may lead to different effects in different contexts. A meaningful assessment of the
importance of each factor can thus only be conducted in a system perspective. This is done for all three
countries in chapter 3.
1
The distinction between these categories is often blurred in reality and also here careful interpretations are
needed.
12
 UFU, Alm.del - 2016-17 - Bilag 26: Henvendelse af 17/11-16 fra Danmark Forsknings- og Innovationspolitiske råd
1689070_0013.png
3. Research policy and academic performance: Case studies
The present study deals with three very similar countries. They are all relatively small, fairly wealthy
and very open western countries with highly developed welfare systems including strong education
and health care systems. All three countries have also been characterized by longstanding and very
strong research traditions with internationally highly renowned research environments established
long before research became a theme on the political agenda. Furthermore, they are all both EU- and
OECD members and have accordingly been subjects of the same international forces shaping the
governance of the national research systems. Also with regard to the basic structures of the public
research systems there are clear similarities between the three countries. The university systems all
share a distinct Humboldtian legacy and thus a strong emphasis on the research-teaching nexus.
Similarly, these countries have research funding systems which have developed from traditional dual
funding systems with a relatively high share of institutional funding towards more mixed funding
configurations with large shares of project funding. In terms of funding volume all three countries
today spend close to the Barcelona goal of investing 1 percent of GDP in public research. In 2013
Denmark held the highest share, but it is noticeable that Denmark started out from a significantly
lower level than Sweden and the Netherlands during the 1980s as Figure 5 shows.
Figure 5. Higher Education Expenditure on R&D (HERD) + Expenditures to R&D performed in the Government
sector (GOVERD) as share of GDP, 1981-2013
1,2
1,1
1
0,9
0,8
0,7
0,6
0,5
0,4
1981
1982
1983
1984
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
Denmark
Sweden
Netherlands
Source: OECD MSTI
Another interesting difference between the three countries can be observed with regard to the
division between Higher Education Expenditure on R&D (HERD) and expenditures to R&D performed
in the government sector (GOVERD). As Figure 6 and Figure 7 show: up until around 2000 both
Denmark and the Netherlands had substantial government research institute sectors, while Sweden
throughout the period has had a public research sector almost fully dominated by Higher Education
(HE) institutions. However since 2000 the three countries have become more similar in this respect.
13
 UFU, Alm.del - 2016-17 - Bilag 26: Henvendelse af 17/11-16 fra Danmark Forsknings- og Innovationspolitiske råd
1689070_0014.png
Figure 6. HERD
HERD as a percentage of GDP
1,2
1
0,8
0,6
0,4
0,2
0
1981
1983
1985
1987
1989
1991
1993
1995
1997
1999
2001
2003
2005
2007
2009
2011
2013
Figure 7. GOVERD
GOVERD as a percentage of GDP
0,4
0,35
0,3
0,25
0,2
0,15
0,1
0,05
0
1981
1983
1985
1987
1989
1991
1993
1995
1997
1999
2001
2003
2005
2007
2009
2011
2013
Denmark
Netherlands
Sweden
Denmark
Netherlands
Sweden
Source: OECD MSTI
Source: OECD MSTI
Also in contrast to Denmark and the Netherlands, Sweden has had a unified HE sector for most of the
period under examination, while the two other countries have had binary systems with a clear division
between research universities on the side and non-research HE-institutions on the other. The result
has been a concentration of research resources in a smaller number of HE-institutions in these two
countries.
Finally, Figure 8 and Figure 9 show the developments in the three countries with regard
to their share of project funding and their PhD graduation volume. It is noticeable that the Swedish
system has a larger share of project funding than Denmark and the Netherlands throughout the
period, and also that the two latter countries have very similar developments up until 1990. Since then
the Dutch system has operated with a lower share of project funding than the two other countries, but
a less developed overhead system in the Netherlands means that the differences between the
countries may appear larger than they are in reality. With regard to the PhD graduation volume, it is in
particular noticeable that the Danish system was characterized by a very low degree of PhD education
throughout the 1980s.
Figure 8: Share of project funding, 1973-2010
0,6
0,5
0,4
0,3
0,2
0,1
0
1973
1975
1977
1979
1981
1983
1985
1987
1989
1991
1993
1995
1997
1999
2001
2003
2005
2007
2009
2011
Figure 9: Number of graduated PHDs per 10.000
people in population: 1985 is estimated for Denmark.
4
3,5
3
2,5
2
1,5
1
0,5
0
1980
1985
1990
1995
2000
2005
2010 2013
Denmark
Sweden
Netherlands*
Denmark
Sweden
The Netherlands
Source: National statistics
Source: National statistics
14
 UFU, Alm.del - 2016-17 - Bilag 26: Henvendelse af 17/11-16 fra Danmark Forsknings- og Innovationspolitiske råd
1689070_0015.png
As mentioned in the introduction, Denmark, Sweden and the Netherlands can be labelled as high
performing research nations in terms of mean normalized citation scores at an aggregated level. There
are, however, interesting differences in the trajectories of the three countries as can be seen from
Figure 10 below.
All three countries performed at a high
international level in the early 1980s where
the time-series start, but from there on the
developments differ: The Netherlands remains
at a high and relatively stable level throughout
the period. Denmark experiences a significant
decline during the 1980s, but shows a strong
catch-up during the 1990s bringing the
performance back on par with the Netherlands
for the remaining part of the period. Also
Sweden experiences a drop during the 1980s,
but unlike Denmark, Sweden is unable to fully
reverse this trend. A main question in the
following analysis is to what extent
relationships between policy changes (or non-
changes) and developments in academic
performance can be detected across the three
countries.
Figure 10. Developments in MNCS for Denmark and
the two benchmark countries. Based on full counts
and calculated for three year overlapping blocks.
1,6
1,5
1,4
1,3
1,2
1,1
1
1980-1982
1982-1984
1984-1986
1986-1988
1988-1990
1990-1992
1992-1994
1994-1996
1996-1998
1998-2000
2000-2002
2002-2004
2004-2006
2006-2008
2008-2010
2010-2012
DENMARK
NETHERLANDS
SWEDEN
Source: Schneider and Aagaard 2015
3.1.
Denmark
The Danish science system was performing remarkably well during the 1970s and up until the early
1980s in spite of a low level of public funding. The overall high performance was not least the result of
a number of very strong research groups within the technical and natural sciences (Schneider and
Aagaard 2015). However, during the 1980s the aggregated Danish mean citation impact dropped
dramatically and reached a low point by the end of the decade. It is however interesting to notice (as
shown in chapter 8 in the background report) that the decline in impact first and foremost was seen
for national publications, while internationally co-authored publications continued to show a
remarkably high citation impact. Several explanations for this decline are presented in the following –
most of them closely related to the hypotheses formulated by DFiR.
From the 1970s onwards the Danish science system began to show signs of a number of growing
imbalances. First of all the system was suffering from a very low degree of renewal. As a result of a
rapid expansion during the 1960s and early 1970 a large number of researchers of roughly the same
age were hired at Danish universities within a short period. As the growth of the system slowed down
during the 1970s and 1980s almost no new positions became available. At the same time and partly for
the same reasons the number of graduated PhDs was very low: between 140 and 167 per year from
1980-1987 (Forskningsstyrelsen 2003). The result was a system with a weak growth layer, very little
renewal and dynamic, and lack of competition for positions. According to critics a contributing factor
to the lack of renewal and dynamic within the universities was a weak and consensus-oriented
leadership model.
15
 UFU, Alm.del - 2016-17 - Bilag 26: Henvendelse af 17/11-16 fra Danmark Forsknings- og Innovationspolitiske råd
While the situation at the personnel side was characterized by stagnation and lack of renewal, the
funding system went through rapid changes. In the early 1970s, less than 10 percent of all public
research funding was allocated as project-funding. By the end of the 1980s this figure had increased to
close to 30 percent. The increase was first and foremost the result of a number of large-scale, strategic
programs implemented from the mid-1980s onwards. In addition to the lack of stability in the funding
system there were limited possibilities for excellent research groups to expand. The situation was thus
characterized by challenges at all important levels of the research system: at the bottom of the system
(a weak growth layer); in the middle (lack of competition and dynamic); and at the top (limited
possibilities for excellent groups). These imbalances were however addressed through a number of
steps in the years around 1990.
As a first step the so called Researcher Academy (Forskerakademiet) was established in 1987 with the
aim to support and stimulate the education of highly qualified young researchers. The establishment
did not only lead to a strong growth in the volume of graduated PhDs during the following 10 years (as
shown in Figure 9 above), but resulted also in a much more formalized and quality-oriented PhD
education structure. The aim to increase the internationalization of Danish research was an integrated
part of this development right from the beginning. This process, driven by the Researcher Academy,
was eventually formalized in 1993 when a new PhD reform was passed. As documented in chapter 6 in
the background report, the increased volume of graduated PhDs started to level out the highly skewed
age distribution within the universities from the middle of the 1990s and onwards as the public
research system entered a period of stable growth. During the following decade the age distribution
within the universities thus became much more balanced as the increased number PhDs progressed to
the next levels of the career ladder.
As the background report also documents it is however difficult to establish direct, quantifiable links
between most of the policy changes of the early 1990s and the subsequent changes in performance.
The most direct relationship can be found in relation to DNRF where the funding of the Centres of
Excellence (CoEs) can be linked directly to our performance indicators (as shown in Figure 11 and
Figure 12), but even in this case the results need to be interpreted cautiously. First of all, the
publications attributed to DNRF can also be attributed to a number of other funding sources. There is
thus a risk of overestimating the impact of DNRF. Secondly, a counterfactual analysis cannot be
established: What would for instance have happened if the same amount of money had been allocated
through the traditional research councils rather than through DNRF? Nevertheless, both our
quantitative and qualitative data support the view that DNRF plays an important role in the Danish
science system and that it has been a driver of a strengthened research culture not only within the
CoEs, but in the system as a whole. The timing of the establishment of DNRF was thus important as it
contributed to the general academic reorientation of the system taking place during these years.
16
 UFU, Alm.del - 2016-17 - Bilag 26: Henvendelse af 17/11-16 fra Danmark Forsknings- og Innovationspolitiske råd
1689070_0017.png
Figure 11: Developments in Danish impact
17%
16%
15%
14%
13%
12%
11%
10%
9%
8%
1980
1982
1984
1986
1988
1990
1992
1994
1996
1998
2000
2002
2004
2006
2008
2010
Figure 12: DNRF share of publications and share of
highly cited publications
16%
14%
12%
10%
8%
6%
4%
2%
0%
1993 1995 1997 1999 2001 2003 2005 2007 2009 2011
share of highly
cited
share of pubs
Danish share of highly cited publications
Danish share of highly cited publications excluding
publications from DNRF
Source: CFA
Source: CFA
For the other relevant factors the difficulties in establishing direct links are even greater. Here we have
to rely on a careful interpretation including both quantitative and qualitative data. With regard to the
balance between institutional funding and project funding the positive development in research
performance gained momentum as the balance stabilized after more than a decade of rapid change. In
addition, the overhead reform implemented in 1995 further contributed to improve the calibration of
the system. However, also the composition of the project funding changed during these years. Most
importantly, the responsibility for the allocation of the majority of the project funding was gradually
returned to the disciplinary and more academically oriented research councils after a decade
dominated by strategic funding allocated by independent committees outside of the traditional
research council system. The importance of the funding changes can thus not only be attributed to the
simple balance between institutional funding and project funding, but rather to the fact that a long
term stability was found, where the institutional funding, the overhead system, the composition of the
project funding and the contributions from DNRF together created a dynamic and diverse funding
landscape with a strong academic orientation and support for a diverse growth layer as well as
excellent groups.
The changes in the late 1980s/early 1990s thus restored a number of central systemic balances: The
growth layer and the recruitment was strengthened through the Researcher Academy and the
subsequent PhD reform; the system as a whole was improved by providing stability, flexibility and
competition through the changes in the funding mechanisms and the increased number of young
researchers; and the top of the system was supported by creating better opportunities to establish
world leading research environments – which in turn had positive spill-over effects for the system as a
whole. Integrated in the majority of these changes was an increased emphasis on internationalization
and a general strengthening of an academically oriented, national research culture. The following
period up until the mid-2000s was subsequently characterized by relative stability in both funding and
governance; appropriate balances between diversity and concentration; long term stability and
flexibility, a strong growth layer and support for excellent groups.
17
 UFU, Alm.del - 2016-17 - Bilag 26: Henvendelse af 17/11-16 fra Danmark Forsknings- og Innovationspolitiske råd
Uncertain effects of recent research policy reforms
However, the Danish system has undergone a number of major changes during the last decade. Once
again the majority of the factors associated with DFiR’s hypotheses have been affected. While some of
the reforms were presented already in 2002 and 2003, most of the reforms did not take real effect
before the latter half of the decade. To increase the competition in the system and to avoid spreading
the funding too thinly the Government aimed for a turn towards a 50/50 balance between institutional
funding and project funding. In addition, the research council structure has been reformed rather
fundamentally. The existing councils of the 1990s were all restructured and renamed and a few new
ones were established. The changes were most visible in relation to the strategic or innovation-
oriented research funding channels while only DNRF was allowed to continue unchanged. Although
the ambition to reach a 50/50 balance was never fully reached the result was a gradual shift from
institutional funding towards project funding. The composition of the project funding has however
also changed in this process as a shift from curiosity-driven research towards applied or strategic
research has been observed alongside a shift from the allocation of many small grants towards fewer
and larger projects (Aagaard 2011). In particular from 2006 and onwards these changes have been
accompanied by a significant increase in total public R&D investments as figure 5 shows.
Alongside the changes in the funding system a new University Act from 2003 has introduced boards
with a majority of external members and prescribed employed leaders instead of elected leaders at all
levels of the institutions. The objective was to sharpen up the profiles of individual institutions, to
professionalize and empower managerial structures, and to increase collaboration between the actors
of the research and innovation system. The new boards were in place in late 2005 and most
universities had the appointed leaders installed by 2006. Shortly after this the Government
implemented a far-reaching merger process which reduced the number of universities from twelve to
eight and transferred 12 out of 15 Government Research Institutes (GRIs) to one of the remaining
universities - in reality closing down the majority of the GRI sector.
2
The result was a large
concentration of resources within a few select institutions, and also a clear break with the former
division of labor between academic research and the more applied GRI research (Aagaard 2011).
Finally, PhD education has once again been highly prioritized with a 100 percent increase in uptake in
the period 2004-2010. This increase has been quite selective as almost 90 percent of the increase has
taken place within the natural, medical and technical sciences.
Most of the recent reforms have thus targeted the same areas as the reforms of the early 1990s, but
this time in a quite different context. Where the prioritization of PhD education during the late 1980s
and the early 1990s mainly was driven by an internal demand for renewal within the universities, the
expansion observed from 2004 onwards was driven by perceived external demands for more PhDs
outside of the public research sector. Similarly, the new funding instruments were not mainly
introduced to improve academic performance but rather to strengthen the societal and (in particular)
the economic impact of the public R&D investments. The impact of these policy changes on
performance is however difficult to assess at this point. First of all, they have been implemented
alongside a substantial increase in overall funding as shown in Figure 5. Potential negative effects of
the reforms in isolation may thus been outweighed by the benefits of the increased funding.
2
A more limited number of mergers involving GRIs and universities were already implemented in 2004 as a
prelude to the subsequent large scale merger process in 2007.
18
 UFU, Alm.del - 2016-17 - Bilag 26: Henvendelse af 17/11-16 fra Danmark Forsknings- og Innovationspolitiske råd
Secondly, the changes have coincided with a period characterized by large changes in the WoS
database as well, which for almost all developed western countries have resulted in a perceived
improvement of their performance.
The three countries in this study thus all have very steep, almost identical improvements in
performance in the period from 2006 and onwards. However, most recently the Danish public R&D
sector has experienced stagnation and even cuts in the funding which will test the current balances of
the system. While clear links between these policy changes and the effects on performance cannot be
established, the changes can be analyzed in a systemic perspective in order to discuss to what extent
the reforms have altered the balances of the public Danish research system. We return to these
discussions in the final part of chapter 4.
3.2.
The Netherlands
The Dutch science system has been performing rather well internationally for more than three
decades. According to many of the interviews conducted for this study, a longstanding, strong
academic culture has been a decisive factor for this success. In contrast to many other continental
European countries the Netherlands was an early adopter of its own version of a competitive (Anglo-
American) output-oriented research culture. Although the output-oriented model was met with some
concern and discussion it quickly gained broad support. Researchers, university boards and
policymakers thus internalized this new research culture which soon became undisputed.
In particular, the introduction of a research assessment system in 1986, which entailed the
establishment of a model of assessing research areas and research units, must be seen as a key reform
in Dutch research governance. The system fast became a mechanism of regulating academic behavior,
in particular publication practices. The initial ambition behind the system was to link the research
assessment to resource allocation but this proved to be a controversial issue. Instead, universities
committed themselves to adapting to the outcomes of the assessments and demote areas that were not
of international significance. This entailed gearing resources towards more promising and viable
activities and transferring staff and activities not active in these fields to other universities where
these fields were more prominent (which in turn entailed coordination between universities, which
had hitherto been unusual). Even 30 years on, the voluntary system has remained intact and has
become an institutionalized component of Dutch research governance, where universities self-select
areas to be assessed and use the assessment for their own purposes without government interference.
Various sources indicate that the assessment system has given a major boost to the scientific impact of
Dutch research, primarily by demoting research which resulted “only” in domestic publications and by
reducing research activities of limited international significance. Since its inception, the protocol for
these assessments has been renewed every five years. In the latest protocol the emphasis on outputs
(publications) decreased, while more attention is given to the societal relevance of the research.
The introduction of the assessment system was part of a broader set up of policy measures to support
excellence. One such was introduced in 1991 by the Ministry of Education, Culture and Science when
the national research schools were established as a new policy instrument. The aim of this instrument
was to stimulate focus and mass by concentrating the best researchers at a national level and combine
it with PhD training. The Research schools have to be accredited by the Royal Academy of Sciences. In
the late 1990s the Ministry also introduced the Bonus Incentive Scheme. Its purpose was to identify
and encourage national concentrations of outstanding scientific research in top research schools. The
effect of the (top) research schools was more national collaboration, a more interdisciplinary approach
19
 UFU, Alm.del - 2016-17 - Bilag 26: Henvendelse af 17/11-16 fra Danmark Forsknings- og Innovationspolitiske råd
and better PhD- training. Both instruments were replaced by the Gravitation program of the
Netherlands Organisation for Scientific Research (NOW) in 2009. Besides the research schools and the
Bonus Incentive Scheme many other excellence initiatives have been introduced. Some have targeted
specific scientific areas (in which the Netherlands could excel), while others have been concerned
promising economic areas or societal challenges. These initiatives did not exist in isolation, but often
strengthened each other. Although the Dutch policy for excellence initiatives has been somewhat
fragmented, the total amount of additional funding generated by these initiatives has been substantial.
All these policy measures secured sufficient support for excellence and in addition allowed room for a
tailor-made approach for different domains and different aims.
The voluntary assessment system and the different excellence initiatives were all part of a more
general realignment of the relations between the state and the universities. The state receded from its
relatively fine-grained model of steering into a model of ‘steering at a distance’. Increased university
autonomy was therefore coupled to performance expectations and evaluations (De Boer et al 2006).
The new governance arrangements also changed the management of universities and research
institutes. The institutional autonomy was used for more pro-active research management where the
boards of the universities put more emphasis on quality and excellence and promoted an output-
oriented research culture. In terms of policy the management of universities and institutes created
local focal points and changed the Human Resource policy: e.g. by use of performance indicators for
appraisals, increased demands for hiring new staff and the introduction of tenure track positions. For
individual staff members the prerogative of ‘total academic freedom’ disappeared and was replaced by
a much more competitive system with performance indicators and increased demands in general.
Finally, the academic research culture has also been supported by a strong and long standing
international orientation. Dutch researchers have always been open to international collaboration and
have historically been well embedded in the international research community. The open mindset (as
part of a long cultural tradition), the relatively central location in Europe and good infrastructure, a
research infrastructure and environment that is attractive to foreign researchers and good language
skills (often one or more other languages besides English) have all contributed to the international
orientation of Dutch research. These international contacts have been beneficial for the visibility and
impact of the research as it encourages informal peer review, mutual exchange of ideas and stimulates
a multidisciplinary approach.
A second important, overall factor for the strong research performance in the Netherlands is the long
term funding stability. The data collected for this study show an increase in both basic and project
funding over the years (from 1975 on). The Ministry of Education, Culture and Science primarily
provides the institutional funding to universities and other public research institutes. The data show
how both types of funding increased at the same pace until 1990 and how university institutional
funding has increased at a faster pace afterwards. Only in recent years has the institutional funding
shown a slight relative decrease.
The lump sum size of the institutional funding for universities is formula based, and made operational
in a funding model that includes several parameters. In addition to that, universities can receive extra
funding for meeting certain additional indicators that are referred to as ‘performance agreements’. The
budget for this can make the overall lump-size increase by 6 percent. The formula contains indicators
like the number of PhDs, the number of MA diplomas, a contribution for research schools and specific
tasks. A larger part (almost 60 percent) of the funding is based on ‘strategic considerations’. However,
20
 UFU, Alm.del - 2016-17 - Bilag 26: Henvendelse af 17/11-16 fra Danmark Forsknings- og Innovationspolitiske råd
the distribution of this ‘strategic part’ amongst the universities has remained stable for a long time and
is mostly based on historical grounds. Overall the distribution of the institutional funding shows a
rather stable pattern. The increased emphasis on output funding has not influenced the distribution of
institutional funding among universities. This is done within universities and by the increase of
competitive project funding.
In addition to the institutional funding there is also project based funding. This encompasses the
project funding from research councils, contract research, governmental programs, etc. The project
funding shows a clear upward trend from 1975 to 1990. The share of the project based funding almost
tripled (from 10 to 30 percent of the total amount). Afterwards the share of project funding remained
stable at about 25 percent. The growth in project financing is mainly due to the availability of
increased project financing by the Ministry of Economic Affairs. In 1975 the ministry introduced a
temporary large new innovation instrument (the INSTIR), which explains a peak of funding in 1990. As
this instrument seized to exist, other instruments continued to supply project funding (but they were
relatively smaller in size). Another reason for the increase of project funding has been the expansion of
the budget of the NWO. Most of the NWO budget is aimed at supporting excellent research.
An important characteristic of the Dutch funding system is an equal distribution of resources for basic
research. Unlike the REF system in the UK, research assessments in the Netherlands are not linked to
resource allocation. And as mentioned, in the allocation of research funding at the level of universities
and research institutes output indicators are not a decisive factor. Although there are clear differences
in the level of funding (mostly based on historical grounds), all universities and research institutes can
build on a sufficient level of institutional funding. This enables them to develop and maintain a state-
of-the-art local research infrastructure and attract good researchers. This is, according to the
interviewees, one of the explanations why all of the 12 universities are in the top-100 of most
university rankings. At the same time, it might also explain why none of the Dutch universities are in
the top-20 of rankings. The metaphor of an elevated plain field without peaks is often used to describe
this phenomenon. One of the benefits of the egalitarian system might be that it offers a good base for
national collaboration. The institutions can not only profit from complementary expertise and
infrastructures but may also benefit from high-level research quality of their counterparts. The whole
research system thus offers a high level platform for collaboration, multidisciplinary approaches and
mutual learning. The fact that the Netherlands is a small country in a geographical sense also
contributes to close interactions within the system.
A third and final important factor for the sustained Dutch research excellence is a well-balanced policy
mix at the national level. The expansion of research funding in the 1990s and 2000s has been
operationalized in a series of loosely coordinated, but mutually reinforcing steps to propel funding,
and create a more coherent, yet diverse, funding system – all under the auspices of a national model of
policy coordination and policy advice. The national policy mix contains a number of initiatives
supporting focus and mass in research. The different kind of excellent schemes stimulate national
collaboration and offers sufficient funding for excellent research. In addition the major players in the
research landscape (the Ministry, NWO, universities, the Academy) instigated a scheme for talented
researchers. In 2000, the so-called Innovation Renewal Research Incentives Scheme
(Vernieuwingsimpuls) was established. The scheme offers personal grants to talented, creative
researchers and enables applicants to conduct their own line of research. The scheme is expected to
boost innovative research and promotes mobility within scientific research institutes. The Scheme
comprises three grants geared to different stages in a researcher’s scientific career:
Veni
- for
21
 UFU, Alm.del - 2016-17 - Bilag 26: Henvendelse af 17/11-16 fra Danmark Forsknings- og Innovationspolitiske råd
researchers who have recently obtained their PhD;
Vidi
- for researchers who have gained several
years of research experience after their PhD; and
Vici
-for senior researchers who have demonstrated
an ability to develop their own line of research. Evaluations of the scheme have been positive and
functioned as a blueprint for the ERC grants and the Danish ‘Sapere Aude’ program. It thus not only
supports the established top researchers but also facilitate career paths for young (innovative) talents.
In addition, as a more recent feature, there is increased attention for the importance of (large)
research infrastructures. This kind of research infrastructures are today considered to be of
paramount importance for scientific excellence as they push the limits of possible data collection,
support pioneering experiments and offer access to comprehensive databases. NWO provides access to
such research facilities for many dedicated schemes. The Netherlands was an early adopter of an
explicit national policy for large research infrastructures. In 2007 – as a follow up to the European
Strategy Forum on Research Infrastructures – the Dutch Minister of Education, Culture and Science set
up the National Roadmap Committee for Large-Scale Research Facilities. In order to implement the
national roadmap the Ministry provides funding for research infrastructures: Initially, mostly as ad hoc
funding, but lately as a recurrent annual budget. NWO organized a biannual call for proposals for large
research infrastructures with a budget between € 75–100 million.
Concerns about the sustainability of the Dutch research performance
Although the Dutch research system has undergone extensive reforms positively affecting
international performance, the interviewees also expressed some concerns regarding the
sustainability of the research performance. The main concerns are related to the following issues:
The emphasis on output indicators in the university system has created unprecedented competition
throughout the whole system. In order to be successful researchers are aiming for the production of
articles in (top) journals, and attracting competitive funds for their research, while the institutions also
aim for high numbers of PhDs awarded. The increased significance of output indicators could lead to
perverse behavior: indicators can be misused or gamed (‘sharing’ or ‘slicing’ publications) and there
are many examples of fraudulent scientists. More important is however, that the personal interest of
the researcher does not automatically (any longer) correspond with the higher goal of science.
In addition the interviewees state that the system has become too much of an ‘aggregation machine’.
The system has very well developed processes in place to produce a high number of articles,
conference papers and make sure that they are well-cited. The outputs of the system reflect the
inherent demands of the same system. But there is a concern that the system as a whole lacks ‘change
agents’. The capacity for or focus on real ground-breaking research has been diminished. Likewise, the
system doesn’t seem to be very successful in adopting new disciplines (like nanotechnology, ICT, etc.).
In these domains there are excellent groups, but they are on average rather small and dependent upon
a limited number of top researchers. The main part of the most excellent research still comes from
traditional disciplines like astronomy, physics and medical research.
There also seems to be an emerging imbalance between institutional funding and project funding. For
their personal careers, and for attracting PhDs and postdocs, researchers are heavily dependent upon
competitive funding. This has led to an increased number of applications for funding from research
councils and as a result to decreasing success rates. In some (prestigious) schemes the success rate is
now below 10 percent. The whole system of competitive funding thus requires substantial efforts of
the research community. Designing a research proposal is time consuming and this also applies – as
22
 UFU, Alm.del - 2016-17 - Bilag 26: Henvendelse af 17/11-16 fra Danmark Forsknings- og Innovationspolitiske råd
the system is based on peer review – for the assessment of individual proposals by multiple peers and
the participation in panels and juries. The interviewees question whether there is still a right balance
between the cost and the benefits of this system. In addition, they indicated a worry that research
councils fund sound but not ground-breaking (and therefore often less feasible) research. Studies
show that it is very difficult to select the best proposals from the middle group (among the category
very good). The existing criteria are not useful to define a clear cut off in this middle group, making it a
kind of ‘lottery.
A final remark concerns the increasing top down steering of university research. There is an increasing
emphasis on the societal relevance of university research. In recent years a larger part (50%) of the
competitive funding of NWO has been aligned to the innovation agenda (the top sectors) of the
Ministry of Economic Affairs. Currently, a National Science Agenda is being developed and this agenda
is considered to be an important tool for research programming in the future. Although many
researchers are convinced that university research can contribute significantly to economic and
societal challenges, there is a dispute about the way this should be organized. In addition there are
some concerns about the funding possibilities for more fundamental ground-breaking research.
3.3.
Sweden
The impact of research carried out by the Swedish academic system was generally very high
throughout the 1970s and early 1980s. In part, this can be explained by a comparatively generous
state support; Sweden has for many years had a high level of investment in R&D in the Higher
Education sector, which was shown in Figure 6. However, during the latter half of the 1980s the
Swedish mean citation impact dropped, and for around two decades until around 2005, Sweden
continued to lose ground to most other comparable countries, not least to Denmark and the
Netherlands, which had a more positive development than many other high-performing countries.
Only during the last ten years has Swedish research managed to improve its performance on par with
that of Denmark and the Netherlands, but it has not been able to close the gap opened up in the two
preceding decades.
There are multiple reasons to why Sweden kept falling behind during this period. Some of these are
closely related to the hypotheses that DFiR has formulated, while others depend on other factors. Due
to the exceptionally complex nature of research systems, it is not possible to clearly separate single
explanations from each other or to provide precise estimates of the importance that single factors have
played. Underpinned by the quantitative and qualitative empirical material put together for this study,
our assessment is therefore provided in a chronological structured discussion. Our main argument is
that Sweden during this period has provided less favourable conditions for elite research than
Denmark and the Netherlands.
The overall level of research funding has certainly not been a problem; Sweden has during practically
the whole period allocated more government expenditures to research than the two other countries.
We also cannot see that Swedish research has been less internationalized than its Danish and Dutch
counterparts. The explanations are instead found in a number of imbalances and structural problems
in the Swedish academic system. Starting from the 1960s, Sweden focused its HE-system on
supporting society in other ways than through elite research. Throughout the 1960s, 1970s and 1980s,
more emphasis was put on the education activities than research, reflected not only in a strong growth
in the number of students, but also in an increased focus of university staff on education. In addition, a
large number of regional university colleges were established in 1977 and were integrated into the
23
 UFU, Alm.del - 2016-17 - Bilag 26: Henvendelse af 17/11-16 fra Danmark Forsknings- og Innovationspolitiske råd
university system, although until the late 1990s without institutional funding for research.
Already in the 1960s, efforts were made to increase the output of PhDs in the Swedish system,
including making the PhD education more structured and efficient. Still it took around 30 years before
strong measures were taken to make the reform fully effective. Throughout the 1980s, Sweden
nevertheless trained far more PhDs per capita than Denmark did, and from the early 1990s and
onwards Sweden has had a considerably higher production of PhDs than the Netherlands has.
Consequently, there has not been any lack of research trained individuals in Sweden.
Around 1980, Sweden once and for all decided to focus the vast majority of its government R&D
spending on the university system. Already in the 1960s, Sweden pioneered the customer-contractor
model of research governance in Europe under the banner of ‘sectoral research’ (Stevrin 1978). The
state expanded research funding dramatically in areas with hitherto very limited research funding or
activities – environment, housing, work environment, social affairs, etc. A notable share of these
mission-oriented activities took place in a university setting. Unlike the other countries, Sweden thus
abstained from a full- blown research institute sector and primarily channelled the sectoral funding via
the university system.
3
Sweden thereby established one of the very first non-binary academic systems
in the world. The traditional academic order with research councils and faculty structures existed
alongside this widening of both research and the HE system more generally, but it was not as
dominant as in Denmark and the Netherlands.
Sweden’s relative focus on sectoral research in the 1970s and 1980s has however only had a limited,
direct negative impact on Sweden’s research performance measured by mean citation impact
indicators, essentially because the sectoral research mainly was concentrated in areas that are
marginal in WoS. However, it may have had an impact by providing opportunities for application-
oriented researchers to make university careers.
In comparison, the large increase in student numbers probably had greater impact on the research
performance, mainly because it led to university staff spending more time on education and less on
research. Besides professor, university lecturer was the only academic employment that was
permanent – and professors were until the early 1990s holders of chairs, which could only be applied
for when the previous holder resigned. Unlike in many other countries, the Swedish university
lecturers had practically
no time for research in their employment contracts; they more or less entirely had to rely on attracting
external research funding. While temporary positions as full-time researchers existed (primarily
aimed for young researchers), highly qualified researchers could be stuck in positions as university
lecturers with prolonged periods offering very limited opportunities to conduct research.
In the 1980s the very strong expansion of funding via sectoral agencies came to a halt and resources
were partially redeployed to the research councils, which included the establishment of two new
councils. Major changes were also made in the constitution of funding in other areas, such as housing
and agriculture (Premfors 1986). In 1989, the funding of PhD education was reformed, with the
introduction of fully funded PhD positions instead of the earlier system of stipends. Hence, the 1980s
represented a modest retreat of research policy and of university governance into more ’orderly
forms’.
3
A significant share of the sectoral research was however performed by the sectoral agencies themselves,
primarily because they did not believe in the universities’ capacities to produce results that were relevant
24
 UFU, Alm.del - 2016-17 - Bilag 26: Henvendelse af 17/11-16 fra Danmark Forsknings- og Innovationspolitiske råd
enough for policy.
The 1990s, which most markedly is the period where Swedish research was losing ground, was a
hectic period for reforms of Swedish research funding and governance (Benner 2001). A wide-ranging
series of changes were incepted in 1992–1994 when a string of independent research foundations
were established, mostly oriented towards strategic research. The allocation of resources for research
to universities was altered at the same time, and the fine-grained allocation to individual faculties
among Sweden’s universities was replaced with a lump-sum allocation to universities. Universities
were also given the right to appoint professors on their own. In 1995 this transformation was to some
extent reversed, as a consequence of an economic crisis. Institutional funding to universities and
research council budgets were cut with up to ten percent, but the new government failed to dismantle
the strategic research foundations. In parallel important changes took place within the university
college sector. Institutional research funding was appropriated for university colleges; the university
colleges were given the right to establish professorships independently in 1997, and in 1999 three
university colleges were elevated to universities, including the right to confer PhD degrees and to
receive higher levels of institutional funding from the government.
It is also noticeable, that the Swedish system for all of the investigated period has had a higher share of
external funding than Denmark and the Netherlands, between 40 and 50 percent during most of the
period. Funding from the research councils and the new research foundations, originally intended to
provide better opportunities for gifted researchers, in effect became a replacement for institutional
funding. In many fields it became practically impossible to pursue a research career without a constant
inflow of external funding. Besides spending much time writing research proposals, many researchers
had to rely on relatively short-term funding, which increased the incentives to conduct less risky
projects and to publish continuously. As competition for research council funding grew, projects
tended to receive less generous support. Young researchers, recognised as a group with particular
potential for conducting high- impact research, have had a particularly challenging situation
throughout the investigated period. One reason is that they have often been dependent on a professor
and thereby less able to develop their own lines of research. Another consequence of the reliance of
external funding is that university managements and faculty boards have lost power. Strategic
decisions on, for instance, to which research areas additional resources should be channelled were in
practice often taken by the research council committees in their decision on whom to fund. Such
committees are often rather conservative; they tend to be defenders of their subfields, universities etc.
rather than organs that take an overall long-term strategic responsibility. In the 2000s many funders
also requested co-funding from the HEIs, which has tied parts of the institutional funding.
A couple of unfunded reforms in the late 1990s further challenged the universities. The most profound
of these was the reform in 1999 that reformed the chair-system for appointing professors, whereby
university lecturers were given the right to apply for promotion to professor. If they had enough
merits to qualify as a professor, they had to be appointed, regardless of the need or the availability of
resources. Most universities felt a pressure to support the newly promoted professors with more time
for research in their positions, besides a higher wage. This, since the reform was unfunded, increased
the internal competition for institutional funding. Also the PhD reform in 1998, which in essence
prevented non-funded recruitment of PhD students, put pressure on the institutional funding. The PhD
students had to be supported by external funding, scholarships, or plain salary from the host
institution. Step by step during the 2000s practically all universities have also been pressured to
guarantee their PhD students four years of regular employment, instead of scholarships with
significantly lower wage (and no social security) during the first one or two years of studies.
25
 UFU, Alm.del - 2016-17 - Bilag 26: Henvendelse af 17/11-16 fra Danmark Forsknings- og Innovationspolitiske råd
To remedy the problem of inadequate opportunities for elite research, many external funders started
to initiate their own excellence schemes. Such schemes have existed since the mid-1990s, but around
2003– 2004 there was a significant growth as several funders established more long-term excellence
schemes which targeted larger constellations of researchers. In 2008 the government established a
number of generously funded Strategic Research Areas to support elite research in certain fields. It is
however unclear whether the schemes have led to higher performance; studies of smaller samples of
centres and mid-term evaluations have indicated marginal improvements at best and often pointed at
insufficient leadership.
While the overall funding for research has been high, the system has been characterised by a high
degree of competition for resources. One may argue that Sweden has spread the resources too thinly,
employing too many researchers, and awarding too small research grants. On the other hand, the HEIs
have been under pressure to employ more staff for teaching, to maintain the link between research
and education, and not close the doors for new generations of researchers. Similarly, the research
funders have felt a need to grant a certain degree of applications and to be reasonably “fair” with
regard to specific disciplines and universities. The situation is also a consequence of the expanded PhD
training – while the political intention has been to provide PhDs to society outside of academia, PhDs
have often tried to remain in academia, even if the conditions have been harsh.
Unsolved problems in the Swedish system
As outlined, there have accordingly been many features and circumstances in Swedish research policy
from the 1980s onwards that did not play in favour of exercising a rigorous internal quality control
and maintaining a high research quality across the institutions. While it is not possible to assess the
exact impact of those changes, it seems reasonable to assume that a weakened quality culture and,
over time, insipid internal quality control, has been the key explanation to why Sweden has fallen
behind Denmark and the Netherlands. The high dependence on external funding has made it difficult
for universities to steer their research environments, including which recruitments to make. The
increasing reliance of external funding may also have affected researchers’ interests in maintaining a
collegial structure.
The high dependence on external funding has also resulted in research priorities that have been short-
term and safe, in order to secure funds the coming years, rather than long-term and novel. A rather
constant output of decent publications has typically paid off better than one top publication after five
years of uncertain research. In addition, collegial steering at faculty level was largely abolished already
in the late 1960s. Since then the main role of the faculty collegiate has been to elect their
representatives to faculty boards etc. The elimination of faculty meetings and the consequential
weakening of the cross-faculty networks that came with such a structure, has without a doubt resulted
in less quality control of single departments and research environments.
Also the quality control of PhD examinations has been partly eroded. Opponents and PhD committee
members are in practice appointed by the department in question and not the faculty, which they used
to be. As the opponent thus came to represent a field rather than a collegial structure, this change of
procedure lowered the internal quality control. In addition, Sweden abolished a grade system for PhD
dissertations already in the 1960s; since then the PhD committee have not even been allowed to
provide written or oral assessment or comment of the quality of the dissertation.
26
 UFU, Alm.del - 2016-17 - Bilag 26: Henvendelse af 17/11-16 fra Danmark Forsknings- og Innovationspolitiske råd
Furthermore, when research peers from abroad evaluate Swedish research environments or funding
programmes, they often point at an insufficient academic leadership. One may ask to what extent that
situation is an effect of the quite powerful external funders and the relative absence of collegial
systems for internal quality control.
Finally, the structure of the Swedish research system implicates that HEIs must take on roles that in
many other countries are played by research institutes. While that structure perhaps benefits Swedish
industry and other parts of society, it also provides career opportunities for less academically oriented
researchers and in general introduces other goals than performing research that gives high impact in
academic journals.
4. Conclusion and future perspectives
This final, concluding chapter follows up on the case studies of chapter 3 and highlights a few cross-
cutting conclusions. In continuation of this a number of current challenges facing the three national
systems are outlined and it is discussed to what extent the results of the analysis can inform future
research policy decisions in order to support continuous high academic performance. Finally,
Appendix A presents the main findings of the analyses at hypothesis-level with a brief discussion of
how changes in the selected factors influence research performance in the different national contexts.
4.1.
Cross-cutting conclusion from the case studies
The previous chapter presented three individual accounts of relationships between research policy
and research performance. Overall, empirical support was found for several of DFiR’s hypotheses for
Denmark and it was shown how changes in the funding system (including the establishment of DNRF),
the prioritization of PhD education and a general increased emphasis on internationalization during
the late 1980s and early 1990s contributed to reverse a negative development and subsequently laid
the foundation for the constant increase in the aggregated Danish research performance during the
following decades. Also in Sweden and the Netherlands several factors associated with the hypotheses
were highlighted as important elements in the explanations of the observed national developments.
However, as soon as we move from a single-country perspective towards a comparative cross-country
perspective, more generalizable patterns between specific types of research policy changes and
research performance become a lot less obvious. Rather than universal relationships between specific
policy measures and their effects, we find a number of highly context-specific explanations – often
involving a complex interplay between several factors. Nevertheless, two central conclusions stand out
at a more general level; the first is related to whether and how strong national research cultures have
been institutionalized at the system level within the three countries; the second concerns the
importance of continuous calibrations of the central balances of national science systems.
Institutionalization of a strong national research culture:
The 1970s and 1980s can be perceived
as an important transition period from a traditional ‘republic of science’ model towards systems based
on ‘New Public Management’-inspired funding and governance mechanisms and increased societal
demands towards public research. While this transition period for all three countries resulted in
stagnating or decreasing national academic performance at first, the subsequent national policy
responses differed in important respects. The period during the late 1980s and early 1990s, where
new balance-points between classical academic values on the one hand and demands for
27
 UFU, Alm.del - 2016-17 - Bilag 26: Henvendelse af 17/11-16 fra Danmark Forsknings- og Innovationspolitiske råd
accountability, utility and competition on the other were to be found, can in other words be seen as a
critical juncture which contributes to the understanding of the subsequent developments in
performance up until today.
The following section outlines the main point in this explanation:
During the post-war period and up until the 1980s Denmark, Sweden and the Netherlands all had high
performing, internationally oriented, research groups. The strong research culture of these groups
was, however, in most cases locally anchored at group- or institution-level rather than the result of
policy- induced initiatives at the systemic level. But beginning from the late 1980s both Denmark and
the Netherlands established independent institutions with a clear responsibility of upholding
academic quality and with a high degree of both internal and external legitimacy. In Denmark, the
Researcher Academy, DNRF and increasingly also the traditional research councils became such
institutions with system-level effects. In the Netherlands it was in particular the research assessment
system established in the late 1980s which played this role, but also the strong tradition of
coordination between the most important actors of the academic system must be highlighted as an
important element in this explanation. Although we are dealing with quite different policy measures
put to use in different national contexts, they have played the same role: In both Denmark and the
Netherlands the initiatives implemented in this period have created long-term and stable points of
orientation for institutions and individuals alike and thus contributed to a general strengthening of a
coherent, national research culture supporting excellence and internationalization. While there have
been an ongoing tension between excellence and utility in both countries throughout the period, the
measures and actors supporting the first have been long-lasting, deeply institutionalized elements of
the systems, while the latter have been of shorter duration and often characterized by shifting
orientations. For Sweden, on the other hand, such long-lasting institutions with strong system-effects
were not to the same extent established during the crucial years around 1990. Rather, the Swedish
system in this period became the subject of more mixed political signals with strategic and utility-
based considerations outweighing a clear focus on research quality as the most central pillar of the
system. This trend was further strengthened as the result of the economic crisis in the mid-1990s
which in particular led to cuts in the funding of the most academically oriented parts of the Swedish
system. For individuals as well as institutions the Swedish system has thus lacked the permanent,
strong, quality- oriented and highly institutionalized points of orientation that we have seen in
Denmark and the Netherlands throughout the period.
Continuous calibration of system-balances:
A strong research culture (and several long-lasting
institutions supporting such a culture) is a important condition for high academic performance at an
aggregated level, but it is by no means a sufficient factor. In order to provide conditions conducive to
high research performance, the research system as a whole needs to uphold a number of central
balances for prolonged periods of time. As outlined in chapter 2 this includes finding appropriate
balances between opportunities for excellent groups on the one hand and securing a strong growth-
layer on the other (balance between concentration of funding and diversity). It also includes balances
between steering and protected space, as well as balances between renewal and stability. Not least the
funding system, the governance mechanisms and the organization and volume of the PhD education
play important roles in maintaining such balances.
In particular the dramatic drop in the performance of the Danish science system during most of the
1980s illustrates how a number of systemic imbalances rather quickly can lead to negative effects. In
28
 UFU, Alm.del - 2016-17 - Bilag 26: Henvendelse af 17/11-16 fra Danmark Forsknings- og Innovationspolitiske råd
the Danish case it was not least lack of renewal in the science system and the absence of a strong and
diverse growth- layer that resulted in almost a decade of decreasing performance. A number of
funding imbalances most likely also contributed to this development. However, the Danish case shows
how the imbalances were corrected during the 1990s by focusing on a strengthened PhD-education,
the establishment of a new funding mechanism targeting the top of the system through DNRF, and a
recalibration of the remaining part of the funding system in order to support the base-layer of the
system. In fact, these policy changes can be seen as the culmination of a general academic
reorientation of the system which already was gaining momentum within the academic system
towards the end of the 1980s. In the Dutch case the overall development has been much more stable
and the need for large corrections has thus been more modest. But also in this case we observe how a
series of smaller calibrations over time have secured long-term stable balances in the system as a
whole. Finally, also the Swedish case shows how, in particular, funding imbalances can create long-
term effects by diminishing flexibility and protected space. It furthermore shows how lack of long-
term stability in the orientation of the system may have negative effects at both the individual and the
institutional level.
The case studies underline, that to get a proper understanding of the factors conducive to high system
performance we need to move from the single-factor perspective to a broad system-perspective. The
crucial question is not whether a given policy factor differs from a similar policy factor in a comparable
system, but rather how well it is calibrated to other important elements within the system in order to
secure the vital balances in the national science systems as a whole. Both the Danish and the Dutch
cases illustrate that multiple balance points between different policy measures may lead to sustained
high national academic performance.
4.2.
Current balances and future challenges
As highlighted in the country analyses in chapter 3, causes for concern are voiced today in all three
countries related to the current balances of the systems. The interviews as well as the rest of the
empirical material indicate that the science systems of the three countries are starting to suffer from a
number of growing imbalances, and that these imbalances may have long-term negative effects on the
aggregated research performance of the systems if they are not corrected. In the following we will
briefly outline the concerns related to the Swedish and the Dutch systems, before the current Danish
imbalances are addressed in a little more detail.
The current challenges of the three national systems are not identical, although there are clear
overlaps. Also here we observe very context-specific problems related to the overall national policy
mix. In the Swedish case it is highlighted that a weakly developed tradition of academic leadership in
turn leads to a weaker academic research culture and a less developed internal quality control system.
A problem of blurred university missions where the academic orientation is outweighed by
prioritizations of societal utility and mass-education is also identified as a problem. These challenges
appear to be amplified by a funding- and career system which favors short-term horizons and ‘safe’
research.
In the Netherlands the major concerns are related to a perceived over-emphasis on output indicators
which may lead to unintended behavior at both individual and institutional levels. In addition, there is
a concern that the capacity for creating real ground-breaking research in emerging areas has been
diminished in the Dutch system. The majority of the most excellent research still comes from
traditional well established disciplines. As in Sweden this is linked to an emerging imbalance in the
29
 UFU, Alm.del - 2016-17 - Bilag 26: Henvendelse af 17/11-16 fra Danmark Forsknings- og Innovationspolitiske råd
funding system which not only is related to the balance between institutional funding and project
funding, but also to the more general composition of the funding and to the conditions related to the
allocation of the research resources. It is questioned whether there is still a right balance between the
cost and the benefits of this system and it is discussed whether the balance between funding of
strategic or applied research on the one hand and the funding possibilities for more fundamental,
ground-breaking research on the other still is appropriate.
Also the Danish system appears to be facing a number of growing imbalances which to some extent
resemble the ones found in Sweden and the Netherlands. Some of the potential Danish problems are
however amplified as a decade of strong funding growth has been replaced by current cuts in the
funding of the universities. There is accordingly a need to consider whether the systemic balances
which have worked reasonably well in a period of sustained growth will be appropriate in a situation
characterized by contraction. The emerging Danish imbalances are in particular related to the
following points:
Growing imbalances in the funding system:
As the Danish country analysis showed, a
number of major reforms have changed several important systemic balances during the latest
decade. The changes have not least affected the funding system. The picture is however rather
complex as it not only concerns the balance between institutional funding and project funding,
but also the balance between smaller and larger grants as well as the balance between strategic
or applied funding on the one hand and funding targeting curiosity-driven research on the
other. In spite of the complexity, the changes may all contribute to a potential diminishing of
the diversity of the system and to a reduced room to maneuver at local levels. Reduced levels of
institutional funding available for actual curiosity-driven research at the ‘shop floor’ of the
institutions and a diminishing number of smaller grants may lead to a decreased breadth and
depth of the Danish growth-layer. This tendency may be amplified by two additional factors: 1)
A negative spiral of a steadily increasing pressure on the available project funding which in
turn reduces the success rates of the applications; 2) The combination of internal and external
mechanisms leading to concentration of funding in already established strongholds.
While it is uncertain how the optimal balance between diversity and concentration should be
set, it is important to underline that both the Danish and the Dutch systems have performed
extremely well across many different fields in periods with very diverse funding landscapes, a
rather strong egalitarian culture and a quite limited concentration of resources. This supports
the view that even relatively small science systems can perform well across the board – and
contradicts the often voiced notion that lack of concentration of funding to existing strongholds
leads to an undesirable thinning of the resources detrimental to quality. From an eco-system
perspective the argument would be that diverse eco-systems are more resilient, more
productive, and more innovative than simpler ecologies in the long run. While strong
concentration of research funding within the highest performing environments may lead to
good results in the short run, there is a risk that it may have negative long term effects for the
diversity, dynamic and renewal of the system.
Growing imbalances in the career system:
Another, closely related, potential imbalance
concerns the career system and thus the balance between the numbers of available positions at
different levels within the Danish science system. Unlike the situation during the 1980s the
problem today is not a lack of highly qualified PhD students, rather the opposite. There is still a
30
 UFU, Alm.del - 2016-17 - Bilag 26: Henvendelse af 17/11-16 fra Danmark Forsknings- og Innovationspolitiske råd
risk however, that the problem of the 1980s where the system suffered from lack of renewal
due to very few open positions at senior levels at the universities may be repeated. The strong
general growth during the latest decade and the high prioritization of PhD education has
created a situation with a substantial number of short-term employed young researchers
within the system. As the general growth has stopped, a likely scenario is that very few
positions will be available at the associate professor and professor level during the coming
years. This may in effect close the door for most of the current generation of talented young
researchers and once again lead to a situation with lack of renewal and an undesirable age-
distribution in the system as a whole. As part of a future discussion of balances it could be
considered whether the large amount of resources currently invested in PhD education should
be redistributed in order to create a more balanced system offering more opportunities at the
next steps of the career ladder within the research system. While the strong prioritization of
PhD education may have been an asset of the system in a period of strong overall growth at all
levels, it may not be appropriate in a situation characterized by cuts in the overall level of
funding.
Growing goal-complexity at the institutional level:
Finally, concerns have also been voiced
with regard to the internal dynamics at the Danish universities as the result of the large scale
mergers and the latest management reforms. It is argued that the Danish system may be in the
process of imitating the Swedish situation with (too) many, partly conflicting, demands to the
universities, and that this process may affect the research culture and the previous strong
academic orientation of the institutions. Furthermore, one may argue that while the societal
demands to the universities have become more diverse and complex, the internal and external
steering has in many cases become more centralized, hierarchical and standardized which may
lead to diminished flexibility and adaptability related to the research activities and
consequently may reduce the protected space at the research level.
Is there a need for recalibrations of the Danish system in a new funding situation?
As outlined
above the current shift in the Danish system, from a decade of strong resource growth to the recent
cuts in the overall funding, may challenge a number of central balances of the system. While the policy
mix of the past obviously would not fit the more internationalized and more competitive, global
research system of today, there is still a need to carefully consider whether the current balances are
optimal in the present situation. The study may thus spark a number of research policy discussions on
whether current adjustments are needed to secure sustainable, coherent and well-functioning national
science systems in the decades to come. This calls for informed policy considerations of the future
calibration of the central systemic balances and underlines the importance and the timeliness of the
agenda that DFiR has set with this study.
31
 UFU, Alm.del - 2016-17 - Bilag 26: Henvendelse af 17/11-16 fra Danmark Forsknings- og Innovationspolitiske råd
Appendix A: Main conclusions concerning DFiR’s six hypotheses
Although few cross-cutting conclusions could be drawn from the comparative analysis with regard to the
relationship between individual hypotheses and developments in performance, the analyses clearly showed that
the associated factors play central roles in the calibration of high performing research systems. In the following
we outline the analyses carried out in the background report and briefly present our main findings with regard
to each individual hypothesis. In addition to the analyses outlined here, a comprehensive interview material has
contributed to the interpretation of the results for each of the hypotheses. Further details on the analyses and the
results linked to each individual hypothesis can be found in the background report.
A.1. Balance between institutional funding and project funding
Two main questions have been analyzed in relation to the developments in funding of public research: First, to
what extent does the development in the overall level of funds available for the public research institutions play
a role for the development in citation impact? Secondly, to what can we detect any links between developments
in the balance between institutional funding and project funding on the one hand and developments in
publication volume and citation impact on the other? Following up on this a number of additional analyses were
carried out based on Danish data alone. These analyses address the same questions but include more contextual
material and longer time series and they were also carried out at the level of the main scientific fields.
As indicated in the present report, no generalizable relationships between levels of institutional funding and
performance were found in our analysis. This is evident within the selected countries where both Denmark and
the Netherlands have performed at a very high level in different periods under very different funding conditions.
Danish research was performing well in 1980 in a situation almost fully dominated by institutional funding, but
the Danish science system performed at least equally well around 2000 when the share of project funding had
stabilized around 40 percent. The same picture can be found for the Netherlands. The Swedish case on the other
hand indicates that lack of institutional funding in this particular context has been at least part of the explanation
for the development observed. Across all three countries we do however see indications of negative effects of
drastic shifts from one stable balance to another. All three countries thus show a decreasing or stagnating
performance trend in the 1980s where the increase in project funding was most rapid. Finally, also at a
disaggregated level for Denmark we find no clear patterns, when we look for relationships within individual
fields as shown in chapter 4 in the background report.
The lack of a clear pattern between the share of institutional funding and performance is also evident across a
wider set of countries. Switzerland and the US for instance, are at opposite sides of the continuum with regard to
the balance between institutional funding and project funding, but both systems have displayed academic
performances at an absolute world class level for more than four decades. These examples show that if the rest of
the system is calibrated well, both a very high and a very low level of institutional funding may contribute to high
performance. The simple distinction between institutional funding and project funding does not accordingly fully
capture the importance of the research funding when we look at this balance in isolation – and such a figure does
not tell us much about the performance of a given national science system. It does, however, in many cases work
well as a proxy for the balance between the stability, diversity and flexibility of the system on the one hand and
the competition and dynamic on the other. But, as outlined in the background report, a scientific system can be
very competitive even if the share of project funding is low. Similarly, a scientific system can offer a high degree
of protective space and long term stability to (some) individual researchers and research groups even in
situations with very high shares of project funding. We thus need to look at both the composition of the project
funding (small vs. large grants and applied or strategic vs. curiosity driven grants) and the time horizons of the
grants to get a clearer picture of the systemic conditions for conducting high impact research. This fits well with
our overall conclusions from the national case studies: the balance between different funding streams is of very
high importance, but we need to include more than just a simple distinction between institutional funding and
project funding.
32
 UFU, Alm.del - 2016-17 - Bilag 26: Henvendelse af 17/11-16 fra Danmark Forsknings- og Innovationspolitiske råd
A.2. Excellence initiatives
The analysis of the excellence hypothesis in the background report includes an overview of existing schemes in
the three countries and a brief discussion of relevant evaluations. In addition a comparison of the ability to
produce highly cited/breakthrough articles in the three countries has been carried out. For the Danish case a
more detailed examination of DNRF was conducted based on data which enabled us to link the CoE-funding
directly to a number of performance measures.
Overall, the analyses underline that DNRF’s CoE-scheme has played an important role in the positive Danish
development since the mid-1990s, but also that the Dutch system has performed equally well without a
comparable scheme. In the Netherlands, individual excellence incentives as well as a long running research
assessment system seem to be the most important explanations for the continued high performance of this
system as a whole. Also in this case we do accordingly find more than one way to high performance at the system
level. Finally, with regard to Sweden: Academically oriented CoE-schemes have been introduced in the Swedish
system since 2005 and this development correlates with a quite steep increase in impact in the same years.
However, the Swedish development in performance is not different from the developments in Denmark and the
Netherlands in the same years and it cannot (based on the available data) be concluded whether the CoE-
schemes have made a measurable difference. The available Swedish evaluations do however indicate that the
schemes have had limited effects so far.
But even in the Danish case DNRF’s CoE-scheme does not by any means stand alone in explaining the high
performance of the system. As shown in the background report the positive Danish development was well
established before the CoEs started to play a role. Even after the CoEs have become an established and integrated
part of the Danish system they are only one piece in the larger puzzle of explaining the strong Danish overall
performance. Both the Danish and the Dutch cases thus seem to suggest that the most important factor in
explaining strong system performance cannot be found in isolated excellence schemes. Rather, it can be found
when the overall system is well calibrated and offer good opportunities across a broad spectrum of disciplines
and sub-disciplines where both upcoming and established researchers find chances to develop their ideas – and
when these mechanisms are underpinned by strong national research cultures. CoE-schemes may be one
element in such systems, but they rely on a strong growth layer. A diversified funding landscape with a strong
academic orientation and long time-horizons thus appear to offer the best conditions for breakthrough research
by securing flexibility and protected space for more than a small part of the system. In the countries studied in
this report this has been secured with a fairly large share of institutional funding as well as many small and
medium-sized grants and fewer large grants.
A.3. PhD education
With regard to the PhD education hypothesis the background study examined to what extent the Danish case
differs from the developments in Sweden and the Netherlands and whether developments in the scope and
content of PhD education quantitatively or qualitatively can be linked to the aggregated development in research
performance. A broad quantitative overview of the development in the volume of PhD education in the three
countries is presented in combination with a more qualitative account of the development in the organization
and formal regulatory framework of PhD education in the three countries. In addition, an indicative bibliometric
analysis examining the relationship between the growth in the number of PhDs and the growth in the number of
‘visible’ researchers in WoS has been carried out.
In the Danish case the hypothesis concerning PhD education stands out as one of the most important
explanations for the reversal of the negative trend during the early 1990s. The power of the explanation has
however more to do with the character of the imbalances in the Danish system during the 1980s and thus with
the timing of the initiatives, than with the actual content of the changes. As shown in the analysis and in the
background report the Danish system suffered from a very low volume of graduated PhDs and limited renewal
in the composition of the staff at the universities throughout the 1980s. The Researcher Academy and the
33
 UFU, Alm.del - 2016-17 - Bilag 26: Henvendelse af 17/11-16 fra Danmark Forsknings- og Innovationspolitiske råd
subsequent PhD reform addressed this imbalance and in turn created a solid foundation for the growth that
characterized the Danish system throughout the following decades. Neither the Netherlands nor Sweden were
suffering from the same problem during the 1980s as they already had a fairly large PhD-production during the
1980s.
While the PhD education hypothesis thus stands out as an important factor in explaining both the Danish drop in
performance and the subsequent reversal of the trend, we find no indications that the Danish PhD education
system today should be seen as fundamentally different from the systems in Sweden and the Netherlands. Seen
in a comparative perspective the Danish PhD reform of the early 1990s and the following changes throughout the
period have shared a great deal of similarities with the PhD reforms carried out in other countries during the
same period. PhD education do however lie at the core of the research capacity of all nations, and as such form a
central part of the foundation for high academic performance. The volume must however be calibrated carefully
as it can displace investments at the expense of other parts of the science system and as it may skew the balance
between continuity and renewal in the staff composition.
A.4. University governance
To examine the university governance hypothesis a structured comparison of the developments in both external
and internal governance models in Denmark, Sweden and the Netherlands between 1980 and 2015 was carried
out as part of the background study. Based on this comparison it cannot be substantiated that differences in
university governance play an important role in explaining developments in performance. It is evident that all
three countries underwent rather similar changes towards more autonomy, notably in the 1990s, albeit at
slightly different moments in time. There certainly are some differences between the three models, whereby
Sweden has chosen a slightly different path than the Netherlands and Denmark, with a little less autonomy in
terms of strategy, finances and cooperation (until 2008) and notably more autonomy when it comes to human
resource management and quality assurance. However, these distinguishing factors alone do not appear to
contribute directly to the over- or under- performance of the three countries. As these countries are rather
similar in their approach to governance, it would be of interest to further test this hypothesis with models that
contain substantially less autonomy.
This interpretation of the hypothesis was also dominant in the interviews conducted in relation to the Danish
case. The management reforms were in general not seen as highly important factors in explaining the Danish
development. One, more indirect, effect should however be mentioned. The two major governance reforms in
Denmark in 1993 and 2003 may not have had a direct effect in improving the conditions for high impact research
within the institutions, but they both played an important role in restoring the confidence and trust in the
university sector from the political system. This in turn led to increased resources – and in particular in relation
to the 1993 reform to more protected space for the academic system as a whole.
A.5. Internationalization
In order to examine the internationalization hypothesis empirically the bibliometric findings presented in
Schneider and Aagaard (2015) have been utilized and extended in our background study. The developments in
volume and impact of different collaboration types, i.e. publications with respectively no institutional
collaboration, national institutional collaboration or international collaboration were examined. In addition, a
number of analyses also investigated the relationship between country size, impact and degree of international
collaboration, as well as the collaboration networks between high-performing countries and the potential impact
benefits coming out of this. Finally, mobility was examined through desk research with a main emphasis on
mobility rates and potential associations between higher-impact levels and mobility rates.
34
 UFU, Alm.del - 2016-17 - Bilag 26: Henvendelse af 17/11-16 fra Danmark Forsknings- og Innovationspolitiske råd
Overall, it can be concluded that internationalization plays an important role in explaining academic
performance. It is, however, only weakly and often indirectly linked to specific policy initiatives. If we look at a
wider set of comparable countries we find almost identical patterns in the developments of the share of
internationally co-authored publications over time. This underlines the fact that increased internationalization
mainly is driven by forces internal to the scientific system and most likely also to the increased importance of EU-
funding etc.
If we look at the three countries included in the present study we thus find more or less the same patterns.
Although a few minor differences can be observed they are unlikely to hold major explanatory power. Denmark’s
high performance cannot be explained as a result of a higher degree of internationalization than Sweden. Rather
as shown in chapter 8 in the background report, the Danish drop in performance during the 1980s was mainly
the result of a decreasing impact of the national publications. It is important to notice that in 1980s and 1990s
the proportion of internationally co-authored papers only constituted between 20 and 50 percent of the total
Danish output. The decline in Danish impact in 1980s was thus mainly attributable to substantial drops in the
impact levels of the nationally authored papers at a time when they constituted 70-80 percent of the total output.
The subsequent increase in the Danish impact during the 1990s was also mainly attributable to an improvement
of the nationally authored papers. Although this happened in a period when their relative shares were
decreasing. At the same time, the relative share of internationally co-authored papers was increasing, and this
obviously had a stabilizing effect on the total impact level for Denmark.
A.6. Cross sectoral collaboration
Finally, the sixth hypothesis deals with the impact of public-private research collaboration. In order to examine
this hypothesis empirically, we have carried out a number of bibliometric analyses utilizing the special coding of
publications linked to “industry” in the CWTS’ version of the WoS database. We have examined developments in
volume and impact for Danish, Swedish and Dutch publications coded as “university-industry collaboration”
(UIC); and we have mapped Danish UIC-clusters for three time periods. This hypothesis has however not been
integrated explicitly in the overall analysis of this main report as it has a distinctively different character than the
other hypotheses, but it has been examined and documented in the background report.
The main conclusions of these analyses are summarized in the following: The hypothesis states that a strong
Danish tradition for public-private research collaboration has been a contributing factor in explaining the
general high national impact. Based on our bibliometric analyses, we find limited support for this claim when we
look at the three selected countries only. It may however be of importance if we look at a wider set of countries.
The overall developments in volume for the three countries included here are very similar, and while the share of
public-private collaboration is highest for Denmark in periods, it is never more than 1 to 2 percentage points
above the corresponding figures for Sweden or the Netherlands. But more importantly, the overall developments
in citation impact for public-private collaboration papers between the three countries are even more similar,
disregarding annual fluctuations. There is thus no indication that Danish public-private collaboration papers
have played a major role in the continuous impact increase from the early 1990s onwards. While removing these
papers from the total Danish publication set does reduce the overall impact level somewhat on the second
decimal, it does not influence the general development. Similar observations are found for the Netherlands and
Sweden. Furthermore, interpretations of these relationships should be carried out with utmost caution as we
know very little about the assumed ‘collaborative’ activities implied by the co-authorships counted between
public research institutions and private profit organizations.
35
 UFU, Alm.del - 2016-17 - Bilag 26: Henvendelse af 17/11-16 fra Danmark Forsknings- og Innovationspolitiske råd
References
Aagaard, K. (2011). Kampen om basismidlerne. PhD-dissertation. The Danish Centre for studies in
research and research policy.
Aagaard, K., Mejlgaard, N. (Eds.). (2012). Dansk Forskningspolitik efter årtusindskiftet.
Aarhus Universitetsforlag.
Aagaard, K., & Schneider, J. W. (2015). Research funding and national academic performance:
Examination of a Danish success story. Science and Public Policy, scv058.
Adams, J. (1998). Benchmarking international research. Nature, 396 (6712), 615-618.
Altbach, P. G., & Salmi, J. (Eds.). (2011). The road to academic excellence: The making of world-class
research universities. World Bank Publications.
Auranen, O. and Nieminen, N. (2010). University research funding and publication
performance—An international comparison. Research Policy 39, 822–834.
Benner, M. (2001). Kontrovers och konsensus. Bokförlaget Nya Doxa.
De Boer, H., Leisyté, L., & Enders, J. (2006). The Netherlands-'Steering from a distance'. Reforming
University Governance. Changing Conditions for Research in Four European Countries. Bonn: Lemmens.
DFiR (2015). Viden i spil: Danmarks Forsknings- og Innovationspolitiske Råds Årsrapport.
Forsknings- og Innovationsstyrelsen. Copenhagen.
Forskningsstyrelsen 2003. Fra Forskerakademiet til FUR. Forskningsstyrelsen.
Geuna, A. and Martin, B.R. (2003). University research evaluation and funding: an international
comparison. Minerva 41(4), 277–304.
Godin, B. (2005). Measurement and Statistics on Science and Technology. Routledge, London
Goodall, A. (2009). Highly cited leaders and the performance of research universities. Research Policy, 38,
7.
Hollingsworth, J. R., & Gear, D. M. (2012). The Rise and Decline of Hegemonic Systems of Scientific
Creativity. Templeton Press.
King, D. A. (2004). The scientific impact of nations. Nature, 430(6997), 311-316.
Laudel, G 2006. The art of getting funded: how scientists adapt to their funding conditions. Science and
Public Policy, 33 (7), 489–504.
OECD (2014). Promoting Research Excellence: New Approaches to Funding. Paris: OECD.
Sandström U., Wold A., Jordansson B., Ohlsson B., Smeberg Å. (2010) Hans Excellens: om
miljardssatsingarna på starka forskningsmiljöer. Rapport 2010:4. Stockholm: Delegationen för
jämställdhed i högskolan.
Premfors, R (1986) Svensk forskningspolitik, Lund, Studentlitteratur.
Schneider, J.W. and Aagaard, K. (2015). Scientometric mapping of developments in Danish
research performance in the period 1980-2013 at macro- and meso-levels. CFA. Aarhus.
Shelton, R.D. (2006). Relations between national research investment and publication output:
Application to an American paradox. Paper presented in the keynote session of the Ninth International
Conference on Science and Technology Indicators, Leuven, Sept. 7, 2006.
Stevrin, P (1978) Den samhällsstyrda forskningen, Stockholm, LiberFörlag.
Ståhle, B. (2007). Fornyelse i forskerstaben. Forskerpersonale og forskerrekruttering på danske
universiteter 2004-2006. Uni-C. Copenhagen.
Swedish Research Council. (2012). Mid-Term Evaluation Report of the 2006 Linnaeus
Environments and Doctoral Programmes. Stockholm: Swedish Research Council.
Thune, T. Kyvik, S., Sörlin, S., Olsen, T.B., Vabø, A. and Tømte, C. (2012). PhD education in a knowledge
society
- An evaluation of PhD education in Norway. Report 25/2012. Nifu. Oslo.
Wendt, K., Aksnes, D. W., Sivertsen, G., & Karlsson, S. (2012). Challenges in cross-national
comparisons of R&D expenditure and publication output. Indicators, 2011(2).
Whitley, R., Gläser, J. and Engwall, L. (Eds.) (2010). Reconfiguring Knowledge Production: Changing
Authority Relations in the Sciences and Their Consequences for Intellectual Innovation. Oxford: Oxford
University Press.
Whitley, R. (2011). Changing governance and authority relations in the public sciences. Minerva,
49(4), 359- 385.
Öquist G. and Benner, M. (2012). Fostering breakthrough research: a comparative study.
Kungliga Vetenskapsakademien, Stockholm.
36