UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

Uddannelses- og Forskningsudvalget 2015-16
UFU Alm.del
Offentligt

Evaluation of the

strategic research

programme NABIIT

Final evaluation report

21 December 2012

Commissioned by the Danish Council

for Strategic Research

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

Published by:

The Danish Council for Strategic Research

Danish Agency for Science, Technology and Innovation

February 2013

The publication is available on the website of the

Danish Council for Strategic Research: www.fivu.dk/en/dsf

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

Contents

Forord (in Danish)

Foreword

Executive summary

2.1

2.2

2.3

2.4

Introduction

Aims and methods of the evaluation

Purpose of the evaluation

Evaluation questions and themes

imitations

Data and methods

Main conclusions and recommendations of

the panel

4.1

4.2

4.3

4.4

5.1

5.2

About the NABIIT programme

Background

Grants and instruments

Size of grants

Participants

General reflections and recommendations

from the panel

Both small and large grants are needed

Greater differentiation between centre grants

and project grants

6.1

6.2

6.3

7.1

7.2

Research quality

A high level of scientific productivity

Scientific impact is internationally competitive

Thought leadership

Broad dissemination of research results

Interdisciplinarity and technological

convergence

Projects are multidisciplinary, but the degree of

interdisciplinarity can be improved

Technology convergence

8.1

Breakthroughs, innovations and commercial

exploitation

Many good scientific results and some scientific

breakthroughs achieved, but low level of

commercial impact

8.2

8.3

8.4

Almost no contribution to public sector innovation

– but potential contributions to solutions to

societal problems

Limited contribution to private sector innovation

Projects involve company participants – but there is

great variety in the types and degree of involvement

of these firms

9.1

9.2

9.3

9.4

Training of young researchers

A significant contribution to educationand training

of researchers

Young researchers gain from multidisciplinary

collaboration

Young researchers were content – and expressed

limited ambition

A potential to increase identification with the

NABIIT programme

Young researchers must be trained for positions

both in academia and industry

9.5

Development of Danish research environments

competences

10.1 Impact on participants’ research activities and

10.2 Supplementary/subsequent funding of research

activities

10.3 Characteristics and outcomes of collaboration

10.4 Researcher mobility

10.5 Acquisition and efficient use of research infrastructure

Appendices

11.1 The evaluation panel

11.2 List of interview respondents

11.3 Overview of the evaluation process

11.4 Overview of the 36 projects

11.5 The Terms of Reference

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

Forord

Evalueringen af Det Strategiske Forskningsråds

(DSF) bevillinger til forskning i nano-, bio- og IT-tek-

nologi (NABIIT-programmet) er gennemført i 2012

af et uafhængigt ekspertpanel bistået af konsulent-

virksomheden Damvad. Panelet bestod af koncern-

direktør, ph.d. Søren Isaksen, NKT Holding A/S (for-

mand), professor og direktør Gabriel Aeppli, London

Centre for Nanotechnology, England samt professor

Fredrik Höök, Institut for Anvendt Fysik, Chalmers

Tekniske Universitet, Sverige.

Evalueringen omfatter 36 projekter, som har modta-

get i alt 320 mio. kr. fra Det Strategiske Forsknings-

råd.

Bevillingerne er givet i perioden 2005 – 2008,

og på evalueringstidspunktet var 19 af de 36 pro-

jekter afsluttede.

Samlet anbefaler panelet meget kraftigt, at

program-

Bevillingerne har allerede resulteret i adskillige

videnskabelige gennembrud.

Produktiviteten målt

som videnskabelige publikationer pr. støttekrone er

høj både sammenlignet med dansk og international

målestok. Rådet vil fortsat stille høje krav til den vi-

denskabelige kvalitet.

Panelet fandt en betydelig grad af

multidisciplinari-

tet

i forskningsaktiviteterne.

Endvidere anerkender panelet, at DSF allerede har

bidraget til styrket

interdisciplinaritet

i forskningsak-

tiviteterne, dvs. samarbejde hvor forskerne samar-

bejder om et projekt, hvor de i fællesskab har udvik-

let metoder og processer. DSF er enig med panelet

i, at multidisciplinaritet og interdisciplinaritet ikke er

et mål i sig selv, men et middel der kan bidrage til at

styrke forskningens anvendelsesmuligheder.

met fortsætter

. Rådet håber, at det vil afspejle sig i,

at Folketinget afsætter bevillinger til en fortsættelse

ved de kommende finanslovsforhandlinger.

Panelet konkluderer, at programmet er et

velfun-

DSF vil særligt i opfølgningen på bevillingerne være

opmærksom på, om projekterne i tilstrækkelig grad

udnytter potentialerne ved multi- og interdisciplinær

forskning. Problemstillingen vil også fortsat indgå i

vurderingen af ansøgningerne.

Det er vigtigt at være opmærksom på, at DSF’s

opgave er at støtte forskning, som kan bidrage til

løsning af væsentlige samfundsudfordringer, her-

under bidrage til forskeruddannelse og uddannelse

af kommende kandidater. Derfor er et væsentligt

formål med forskningen også at generere ny viden,

gerende og vigtigt program

, som har leveret forsk-

ningsresultater af meget høj kvalitet. Panelet mener,

det er for tidligt at udtale sig om de kommercielle

resultater, fordi de typisk først vil komme senere,

ofte efter 10 år.

Termen ”projekt” anvendes som samlet betegnelse for

forskningsaktiviteterne. Der er givet bevilling til to centre (store

bevillinger på ca. 30 mio. kr. ), to alliancer (mellemstore bevillin-

ger op til 15 mio. kr.) og 32 projekter (bevillinger på mellem ca.

3 mio. kr. og 12 mio. kr.). Her bemærkes, at beløbet inkluderer

overhead, som udgjorde 20 procent frem til 2007, hvor det ud-

gjorde 35 procent og derefter udgjorde 44 procent fra 2008.

som kan anvendes i det offentlige og det private.

Panelet konkluderer, at

forskeruddannelsen

i NA-

BIIT-programmet er

excellent

, og her fremhæves

især styrken ved, at forskeruddannelsen foregår i et

multi- og interdisciplinært forskningsmiljø. Dette er

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

meget væsentligt, da godt halvdelen af rådets be-

villinger anvendes til forskeruddannelse.

Det er panelets opfattelse, at det er for tidligt at vur-

dere de

teknologiske og kommercielle

gennembrug

allerede fem år efter de første bevillinger er givet,

da de kommercielle resultater først kan forventes

noget senere, ofte efter 10 år. Panelet vurderer, at

sandsynligheden for teknologiske og kommercielle

gennembrud generelt kan øges ved i højere grad at

involvere private virksomheder og ved at involvere

dem fra projektets start. Både i denne evaluering og

i tidligere analyser har de deltagende virksomheder

angivet, at formålet med de projekter, de deltog i,

var at generere viden.

DSF har allerede taget skridt til, at der i forbindelse

med midtvejsstatus for de enkelte projekter skal

være større fokus på den mulige anvendelse af

forskningsresultaterne, herunder særligt de tekno-

logiske og kommercielle anvendelsesmuligheder,

men også den mulige anvendelse i den offentlige

sektor.

Panelet konstaterer, at 43 procent af de publicere-

de artikler var publiceret sammen med

en eller flere

Panelet vurderer, at

samarbejdet internt i projekterne

og det

internationale samarbejde

er på niveau med,

hvad der findes i tilsvarende internationale pro-

grammer.

Jeg vil gerne benytte lejligheden til at takke panelet

for en interessant evalueringsrapport, som har givet

rådet nogle gode input til at fortsætte arbejdet med

fokus på, at den strategiske forskning skal skabe

størst mulig værdi for det danske samfund gennem

offentligt-privat samarbejde, tværdisciplinær forsk-

ning, brugerinddragelse, samarbejde på tværs af

universiteter og internationalt samarbejde.

Peter Olesen

Det Strategiske Forskningsråd

Formand for bestyrelsen

21. februar 2013

internationale samarbejdspartnere

. Det er udtryk for

rådets bevidste prioritering af det internationale

samarbejde.

Det er panelets vurdering, at bevillingerne fra DSF

har været af væsentlig betydning for

opbygning og

udvikling af nye danske forskergrupper

og panelet

er kritisk over for det politiske krav om, at DSF ikke

må kræve mere end

10 procent i medfinansiering

fra

universiteterne.

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

Foreword

The grants given by the Danish Council for Strategic

Research (DSF) for research in nano-, bio- and IT

technology (the NABIIT Programme Commission)

were evaluated in 2012 by an independent panel

of experts from the consultancy firm Damvad. The

panel comprised Søren Isaksen, Ph.D., Group Ex-

ecutive Director and CTO, NKT Holding A/S (chair),

Professor Gabriel Aeppli, Director of the London

Centre for Nanotechnology, UK, and Professor Fre-

drik Höök, the Department of Applied Physics, Chal-

mers University of Technology, Sweden.

The evaluation covers 36 projects which have re-

ceived a total of DKK 320 million from the Danish

Council for Strategic Research. The grants were

that has generated research findings of very high

quality. The panel feels that it is too early to draw

conclusions on the commercial outcome of the pro-

gramme, as these results typically do not appear

until later on, often after a period of 10 years.

The grants have already resulted in a number of

scientific breakthroughs.

The productivity measured

in scientific publications per DKK of funding is high

compared to both Danish and international stand-

ards. The Council will maintain its exacting require-

ments regarding scientific quality.

The panel found a significant degree of

multidisci-

plinarity

in the research activities, recognising that

the Council has already contributed to improving

the

interdisciplinarity

of the research activities, i.e.

collaborations where researchers join forces on a

project to develop methods and processes. Howev-

er, both the Council and the panel agree that multi-

disciplinarity and interdisciplinarity are not goals in

themselves, but a means to increasing the range of

applications of the research.

The Council intends to pay close attention to wheth-

awarded in the period 2005–2008, and at the time

of the evaluation, 19 of the 36 projects had been

concluded.

Overall the panel strongly recommends the

continua-

tion of the programme

. It is the Council’s hope that the

Parliament will comply with this wish by earmarking

funds for the continuation of the programme in the

Government’s upcoming budget negotiations.

It is the conclusion of the panel that the programme

is an

efficiently run and important programme

er the projects exploit the potentials of multidisci-

plinary and interdisciplinary research sufficiently,

especially when following up on the grants. This

issue will also continue to be considered in the as-

sessment of applications.

It is vital to be aware that the Council’s aim is to

support research that can contribute to solving

significant societal challenges and that this includes

contributing to research training and the education

of upcoming graduates. Another of the research’s

The term “project” is used as an umbrella term, describing

the research activities. Grants were awarded to two centers

(large grants of approx. DKK 30 million), two alliances (medium-

sized grants of up to DKK 15 million) and thirty-two projects

(grants of between DKK 3–12 million). It should be noted that

these figures include overheads, which made up 20 percent

until 2007, when they represented 35 percent, and since 2008

44 percent.

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

key objectives is to generate new knowledge that

can be employed in both the public and private

sectors.

This reflects of the Council’s conscious decision to

make international collaboration a high priority.

It is the panel’s assessment that the grants from the

It is the panel’s conclusion that the

research train-

Council have been of significant importance to

build-

ing

offered by the NABIIT programme is

excellent

deeming it a strength that the training takes place

in a multidisciplinary and interdisciplinary research

environment. This is very significant as more than

half of the Council’s grants are put towards research

training.

The panel estimates that it is much too early to

assess the

technological and commercial

break-

throughs just five years after the first grants were

made as the commercial results cannot be expected

to be visible until much later, often after 10 years.

They assess that the likelihood of achieving tech-

nological and commercial breakthroughs can gen-

erally be increased by involving of private-sector

companies to a greater extent and involving them

from the start. Both in this evaluation and in previous

analyses, the participating companies have indicat-

ed that the aim of the projects they participated in

was to generate knowledge.

The Council has already taken steps in connection

with the mid-term reports to increase focus on the

possible applications of the research findings, in

particular the technological and commercial ap-

plications, but also the possible applications in the

public sector.

The panel has ascertained that 43 percent of the

published articles were published together with

one

ing and developing new Danish research teams

and

the panel is critical of the political requirement for-

bidding the Council to require more than

10 percent

in co-financing

from the universities.

They estimate that the

internal collaboration on the

projects

and the

international cooperation

are on a

par with similar international programmes.

I would like to take this opportunity to thank the pan-

el for an interesting evaluation which has given the

Council some excellent input for our continued work

with focus on ensuring that strategic research gen-

erates the greatest possible value for Danish soci-

ety through public-private partnerships, interdisci-

plinary research, user involvement, collaborations

among universities and international cooperation.

Peter Olesen

Danish Council for Strategic Research

Chair of the Board

21 February 2013

or more international collaborative partner

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

Executive

summary

plinarity (and the combined use of technologies),

3) Breakthroughs, innovations and commercial

exploitation, 4) Training of young researchers and

5) Development of Danish research environments.

The main conclusions and recommendation of the

evaluation panel are stated in the following.

Overall, the panel considers the NABIIT pro-

gramme to be a well-functioning and important,

high-quality research programme. The combina-

tion of the three technologies – nanotechnology,

biotechnology and ICT – aimed at solving soci-

etal challenges is both relevant to the research

community and to society. It is also in line with

research trends seen in leading research environ-

ments elsewhere in the world. The panel strongly

recommends that the NABIIT programme is con-

This report presents the results of the evaluation of

the strategic research programme “Interdisciplinary

Use of Nanotechnology, Biotechnology and Informa-

tion and Communication Technology” (NABIIT).

The evaluation, which was commissioned by The

Danish Council for Strategic Research (hereafter

also referred to as the “Council”) under the Ministry

of Science, Innovation and Higher Education, has

been carried out by an independent, international

peer review panel, assisted by the Danish consul-

tancy firm, DAMVAD.

The purpose of the evaluation of NABIIT was to as-

sess the extent to which and how (1) the specific

objectives of the programme have been fulfilled and

(2) the programme has contributed to fulfilling the

general objectives for strategic research, as formu-

lated by the Danish Council for Strategic Research.

The NABIIT programme was established based on

a political ambition to benefit industry and society

through combined research on and application of

nanotechnology, biotechnology and information

and communication technology (hereafter ICT). The

programme ran from 2005 to 2008, both years

included. It awarded 36 grants for a total grant

sum of just below 318 million Danish kroner. These

grants were given to 32 strategic research projects,

two strategic research centres, and two strategic

research alliances.

At the time of evaluation, just 19 of the 36 activities

funded by NABIIT had been completed. All remain-

ing activities are expected to reach completion by

the year 2014. As a result, the evaluation can only

assess the preliminary outcomes of research activ-

ities supported by NABIIT, i.e. those outcomes that

had been achieved at the time of evaluation.

The evaluation panel has defined five main eval-

uation themes: 1) Research quality, 2) Interdisci-

tinued.

In all programmes and research communities,

there is scope for improvement. Therefore, this

evaluation report mainly focuses on areas where

there is a possibility for improvement and thus to

further increase the positive impact of the NABIIT

programme. The conclusions and recommenda-

tions of the panel should been seen in this light.

The panel found the

scientific quality

of the re-

search activities funded by NABIIT to be of high

quality and clearly above average. In addition,

the panel concluded that scientific

productivity

measured as the number of publications per krone

invested by the NABIIT programme, to be high by

national as well as international standards.

Available data moreover indicated the scientific

impact

of the research produced by NABIIT-funded

projects, as indicated by citations to publications,

is above average by national standards and

on par with comparable international research

groups.

Taking the general quality of Danish research into

account and the type of research area, the evalu-

ation panel however believes that there is room for

strengthening the impact of the research funded

by NABIIT even further. A move towards a higher

level of interdisciplinarity, thought leadership and

greater focus on application perspectives is likely

to support such an improvement.

Most projects have a multidisciplinary approach to

their research involving participants from different

fields working on a joint research topic from with-

in their own field while including knowledge and

insights from other fields. Nonetheless, the panel

saw a few cases of truly novel combined uses of

the technologies emphasised in NABIIT, that is,

nanotechnology, biotechnology and ICT.

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

Several projects have delivered

scientific break-

throughs.

In contrast, the number of

technical

breakthroughs

is low, and the

commercial ben-

efits

of the projects are – at least until now - very

limited. This is likely to be due to the limited time that

has passed since the programme was initiated, but

in the assessment of the panel a somewhat weak

involvement of private companies as well as a lack

of sufficient value-adding support from technology

transfer offices (TTO’s) at the participating univer-

sities also plays a role. Challenges relating to com-

mercialisation are, however, not unique to NABIIT

and are faced by other countries as well.

Moreover, the panel observed that in many cases,

company participation in NABIIT-funded projects

did not seem to be aiming at solving important is-

sues for the company, but rather to build fundamen-

tal insights on more generic issues.

In the experience and opinion of the panel, com-

mercial outcomes of the programme are likely to be

improved by having more equal and committed par-

ticipation by industrial partners and by placing more

emphasis on the pursuit of issues that are core to

companies’ R&D activities. The programme is be-

lieved to benefit from more attention to such factors

at the reviewing process.

Education and training of young researchers

the NABIIT programme appears to be excellent. In

particular, multi- and interdisciplinary interaction

in the projects funded by NABIIT seemed to provide

a valuable platform for education and training of

young scientists.

Furthermore, young researchers in NABIIT-funded

projects seem to be very content with their positions

and their base at Danish universities. The panel saw

little evidence of young researchers from the NABIIT

programme establishing and demonstrating their

intellectual independence. The panel, however,

highlights that this challenge appears to be generic

to Danish universities and hence span the bounda-

ries of the NABIIT programme.

The panel advises the Council to maintain its strong

emphasis on Ph.D. and post.doc training and to

take steps to encourage more independent re-

search efforts to better prepare young scientists

for their subsequent research roles in academia or

industry.

Concerning the

development of Danish research

environments,

the panel finds that grants from

NABIIT appear to have helped to open new research

fields and establish new research groups, which

underlines the relevance and value of this type of

research programme. Moreover, NABIIT-funded

projects appear to have given rise to a number of

follow-on projects and activities. The panel sug-

gests the Council to give priority to stimulating and

supporting spin-off projects from NABIIT projects,

as this is likely to increase the ultimate value creat-

ed by the initial grant, for example by giving projects

the possibility of applying for a follow-up grant to

pursue novel research projects that emerge from

the original project.

In conclusion, given the success of the existing pro-

gramme, the panel believes that there is a need in

the Danish research portfolio for a continuation of

a NABIIT-like programme, and such a programme

will benefit by having a stronger focus, enforced

through the reviewing process, on the stated aim

to deliver highly esteemed results with application

perspectives and a clearer emphasis on the com-

bined use of new technologies.

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

Introduction

Professor Kenneth A. Dawson, Director of the Centre

for BioNano Interactions at the University College

Dublin, was initially part of the evaluation panel but

had to step down for health reasons.

Box 1.2. Overview of the NABIIT

programme

The NABIIT programme was established based on a po-

litical ambition to realise and utilise new opportunities for

research and innovation through combined research on

and application of nanotechnology, biotechnology and

information and communication technology (hereafter

ICT). The programme ran from 2005 to 2008, both years

included. It awarded 36 grants for a total grant sum of

just below 318 million Danish kroner. These grants were

This report presents the main results of an evaluation

of the strategic research programme “Interdiscipli-

nary Use of Nanotechnology, Biotechnology and In-

formation and Communication Technology” (NABIIT).

The evaluation, which was commissioned by The

Danish Council for Strategic Research under the

Ministry of Science, Innovation and Higher Educa-

tion, has been carried out by an independent, inter-

national peer review panel, assisted by the Danish

consultancy firm DAMVAD.

The evaluation panel consists of three experts ap-

pointed by the Council for Strategic Research:

— Søren Isaksen (chairman), Group Executive Di-

rector, CTO, NKT

— Gabriel Aeppli, Professor and Director of the

London Centre for Nanotechnology

— Fredrik Höök, Professor in Biological Physics,

Chalmers University of Technology.

See appendix 11.1 for a short bibliography of the

panel members.

given to 32 strategic research projects, two strategic

research centres, and two strategic research alliances

(please see section 4.2 for a description of these three

instruments). These 36 research activities will be col-

lectively referred to as “projects” in the text, even though

they include two centres and two alliances. Please see

appendix 11.4 for an overview of the 36 projects.

At the time of evaluation, just 19 of the 36 projects had

been completed. All remaining 17 projects are expected

to reach completion by the year 2014.

Responsibility for the implementation and administration

of the NABIIT programme was carried by the Programme

commission on Strategic Growth Technologies under the

Council for Strategic Research.

Source: Adapted from Ministry of Science, Education and

Higher Education,

Terms of reference for evaluation of the

strategic research theme Interdisciplinary Use of Nanotech-

nology, Biotechnology and Information and Communication

Technology (NABIIT),

26 January 2012.

Box 1.1. Purpose of the evaluation

(1) To assess the extent to which and how research fund-

ed by NABIIT has yielded breakthroughs through combi-

nations of nanotechnology, biotechnology and ICT that

can create future opportunities for innovation in industry

and solutions to societal challenges within e.g. the areas

of health, environment and energy.

(2) The extent to which and how research activities support-

ed by NABIIT have helped to fulfil overall goals for strategic

research, notably to promote: (a) research of high interna-

tional standing, (b) increased interaction between public

and private research, (c) cross-cutting (interdisciplinary)

research, (d) the internationalisation of Danish research,

(e) postgraduate education and researcher training, and (f)

the strengthening of Danish research environments.

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

Aims

and methods of

the evaluation

— The extent to which and how research activities

supported by NABIIT have helped to fulfil overall

goals for strategic research, notably to pro-

mote:

— Research of high international standing

— Increased interaction between public and

private research

— Cross-cutting (interdisciplinary) research

— The internationalisation of Danish research

— Postgraduate education and researcher

training

— The strengthening of Danish research envi-

ronments.

It is important to note that the NABIIT programme

must be evaluated based on the degree to which

and means by which it has

fulfilled

the objectives

set for the programme. As the objectives are under

ongoing development and have changed since the

establishment of the NABIIT programme, the evalu-

ation will primarily assess the degree to which and

means by which NABIIT has

contributed

to meeting

2.1 Purpose of the evaluation

The purpose of the evaluation of NABIIT, as stated

in the terms of reference for the evaluation, was to

the overall goals for strategic research and, when

appropriate, also comment on the changes that

have been made.

2.2 Evaluation questions and themes

Based on the requirements specified in the terms of

reference and the evaluation panel’s interpretation

hereof, a list of evaluation themes were formulated.

These themes have been refined throughout the

evaluation process and have been instrumental in

focusing data collection and analysis.

Table 2.1 on the following page presents the five

evaluation themes, along with key indicators for

each theme. These evaluation themes also form the

backbone for the structure for this evaluation report,

which addresses one evaluation theme in every

chapter.

assess the extent to which (1) the objectives of NA-

BIIT have been fulfilled and (2) the programme has

contributed to fulfilling the general objectives for

strategic research, as formulated by the Council for

Strategic Research.

The aims of the evaluation were to assess the results

of the NABIIT programme as a whole, not the re-

sults achieved in the individual projects or research

groups supported by the programme.

Specifically, the evaluation panel were given the

task of assessing:

— The extent to which and how the research fund-

ed by NABIIT has yielded breakthroughs through

combinations of nanotechnology, biotechnology

and ICT that can create future opportunities for

innovation in industry and solutions to societal

challenges within e.g. the areas of health, envi-

ronment and energy.

Ministry of Science, Education and Higher Education,

Terms

of reference for evaluation of the strategic research theme Interdisci-

plinary Use of Nanotechnology, Biotechnology and Information and

Communication Technology (NABIIT),

26 January 2012.

The general objectives for strategic research have been

formulated over a period of several years and were under

continuous development during the NABIIT programme period.

These general objectives are described in their current form in a

2012 brief from the Danish Council for Strategic Research,

Stra-

tegic research – Principles and Instruments,

1 January 2012, which is

available from the Council’s webpage.

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

Table 2.1. Evaluation themes, key indicators, and their relationship to evaluation questions

Evaluation theme

Research quality

(Chapter 6)

Key indicators

— Scientific productivity and scientific impact

— Novelty and thought leadership

Main evaluation questions addressed

To which extent has NABIIT contributed to the overall goal for

strategic research to advance

research of high international

standing?

—

Broad dissemination of research results (to the academic

community, to the general public, and to the private and/

or public sector)

Interdisciplinarity (and the

combined use of technologies)

(Chapter 7)

Breakthroughs, innovations and

commercial exploitation

(Chapter 8)

— Degree of interdisciplinarity

— New, combined uses of the technologies

— Scientific and/or technical breakthroughs

— Patents, licenses and spin-off companies

To which extent has NABIIT contributed to

breakthroughs

through

combinations

of nanotechnology, biotechnology

and ICT that can create future

opportunities for innovation

industry and

solutions to societal challenges?

To which extent has NABIIT contributed to the overall

goal for strategic research to promote

cross-cutting (i.e.

— Innovations, i.e. new or significantly improved products,

interdisciplinary) research initiatives?

technologies, methods, processes or equipment

To which extent has NABIIT contributed to the overall goal for

— Actual or expected applications and other realised or

strategic research to promote

increased interaction between

expected outcomes for the private and/or public sector

public and private research?

— Collaboration with users

Training of young researchers

(Chapter 9)

— Training of young researchers through Ph.D. and post.

doc. fellowships

— Recruitment and mobility of young researchers

To which extent has NABIIT contributed to the overall goal for

strategic research to contribute to

postgraduate education and

researcher training?

To which extent has NABIIT contributed to the overall goal for

strategic research to contribute to a

strengthening of Danish

research environments?

To which extent has NABIIT contributed to the overall goal

for strategic research to support the

internationalisation

Danish research?

Development of Danish research — Impact on participants’ research activities and

environments

(Chapter 10)

competences

— Supplementary/subsequent funding of research

activities

— Characteristics and outcomes of collaboration within

NABIIT projects

— Internationalisation

— Researcher mobility

— Acquisition and efficient use of research infrastructure

DAMVAD 2012

2.3 imitations

By June 21, 2012, when data for the evaluation was

collected, just 19 (53 percent) of the 36 projects

funded by NABIIT had been completed;

the majority

of these projects were only recently completed (i.e.

in 2011 or 2012). Moreover, 17 projects (47 per-

cent) are still ongoing and expected to be complet-

ed between 2012 and 2014.

This has the following important consequences for

the evaluation. First, the evaluation can only provide

partial picture of the results

of research activities

supported by NABIIT, i.e. those results that had

been obtained at the time of evaluation. However,

ongoing projects and, to a lesser extent, also com-

pleted projects are likely to be generating results

(for example scientific publications, Ph.D. gradu-

ates, patents etc.) for some time to come.

Second, the evaluation cannot fully and hence re-

liably measure the broader

effects

of NABIIT that

the results of the programme will give rise to in the

long term. Such broader effects may take years to

materialise, and may for example take the form of

contributions to long-term developments in science

or technology, substantial changes in the organisa-

tion or activities of partners in the funded projects,

growth in industry or major developments in the

public sector’s use of science and technology. Such

long term effects cannot be estimated in this eval-

uation.

In consequence of this, the evaluation panel has

chosen to take a predominantly qualitative ap-

A project is deemed completed when the final report has been

submitted to and approved by the programme commission.

proach to the assessment of the preliminary re-

sults and outcomes of the NABIIT programme.

10/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

A central aim of this evaluation is moreover to

assess the degree of

additionality

created by the

NABIIT programme, that is, to assess the changes

brought about by the programme in compari-

son with the same amount of money distributed

through other channels and directly to the indi-

vidual research groups. Such changes could for

example be in the form of added research inputs

(e.g. if the NABIIT grants have attracted additional

funding to the research projects supported and/or

to the research area as a whole). Additionality can

also take the form of desirable changes in behav-

iour (e.g. more collaboration across disciplines

or between academia and users in the public or

private sector, or new ways of managing complex,

interdisciplinary (research activities) or in re-

search outputs (e.g. increased research capacity

through the education of young researchers, the

development of scientific breakthroughs, innova-

tions etc.).

based on documentation gathered during the

course of the evaluation process.

The independent consulting firm DAMVAD carried the

primary responsibility for the development of the eval-

uation design and methodology, and for conducting

data collection and analysis. DAMVAD also served

as secretariat to the panel throughout the evaluation

process, including the drafting of the evaluation re-

port. Nonetheless, the panel takes full responsibility

for the conclusions drawn in this evaluation report.

The assessments and recommendations of the

panel, which are presented in this report, are based

on the sum of data and insights collected through

five distinct studies that have been undertaken in

connection with the evaluation of the NABIIT pro-

gramme.

The results of these studies are described

and documented in separate reports.

For an overview of the five studies, please see box

On the one hand, the fact that the evaluation of

NABIIT took place when the programme was still

ongoing has the stated drawbacks with regards

to measurements of the effects of the programme.

On the other hand, it has had the beneficial impact

on the quality of insights gained through surveys

and interviews conducted in connection with the

evaluation, as participants had the projects rel-

atively fresh in mind. It was evident during inter-

views with participants of completed projects that

the memory of what happens in research projects

has a fast decay rate.

2.4 Data and methods

The evaluation panel was responsible for conduct-

ing the overall assessment of the NABIIT programme

2.1.

According to the terms of reference for this evaluation, case

studies of two NABIIT-funded projects were to be undertak-

en with the aim of communicating examples of projects that

promote public sector innovation, an important topic in Danish

innovation policy. Additionally, the terms of reference also re-

quested the development of five “profiles” delving into selected

themes in the NABIIT programme. Because the aim of both the

case studies and the profiles was to highlight and communicate

selected themes in the NABIIT programme, they do not form part

of the documentation for the evaluation. The evaluation panel

has not in any way been involved in these activities, which have

been undertaken by DAMVAD.

Box 2.1. Background studies and data

1. A desk study of aims, instruments and grants

under NABIIT,

undertaken by DAMVAD. The desk study

consists of three parts 1) Review of background docu-

ments, 2) Analysis of calls for applications, and 3) Analy-

sis of the project portfolio.

2. A self-assessment survey among project partic-

ipants.

Three survey questionnaires were administered:

one to academic participants, one to industry participants,

and one to international participants. 125 out of 154 or-

ganisations participating in the 36 NABIIT-funded projects

contributed resulting in an overall response rate of 81

percent. The survey was undertaken by DAMVAD, but the

evaluation panel played an instrumental role in the devel-

opment of the self-assessment survey questionnaire. An

analysis of additional funding applications from and grants

to participants in NABIIT projects was also performed.

3. A bibliometric analysis of scientific publications,

undertaken by DAMVAD. The bibliometric analysis

was based on exhaustive lists of peer reviewed journal

publications that appeared in peer reviewed scientific

journals as a direct result of activities supported by

NABIIT grants. The scope of the bibliometric analysis did

not enable a comparison of the level of e.g. publication

productivity and impact in comparable research pro-

grammes in other countries.

4. An analysis of researcher mobility,

undertaken by

the Ministry of Science, Innovation and Higher Education.

This report presents and outlines statistical impressions

of the volume, composition and vertical and horizontal

mobility of NABIIT project participants, based on a reg-

istry analysis. The analysis comprises just under 400

individuals, who at one time or another participated in

NABIIT projects.

11/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

5. Interviews with nine projects and centres funded

by NABIIT,

undertaken by the evaluation panel, assisted

by DAMVAD. The nine projects included the two strategic

research

centres

and seven strategic research

projects.

The seven projects were selected based on their distri-

bution across grant holders at three Danish universities:

University of Copenhagen, the Technical University of

Denmark, and University of Aarhus.

Additional selection criteria included the size of the grant

from NABIIT, whether the projects were completed (with

priority being given to completed projects), the degree of

public-private collaboration, and collaboration with both

small and medium sized firms as well as large interna-

tional companies.

For each project, separate interviews were conducted

on-site at the universities with multiple participants,

including the grant holder and other senior scientists,

young researchers (i.e. Ph.D.s, post.docs) and, whenev-

er possible, company participants.

The nine interviewed projects are listed below.

Arrays of Nanoscopic Biosensors on Surfaces

Grant size and period:

11.9 million kr.; 2005-2011

Grant holder:

Professor Thomas Bjørnholm, University of

Copenhagen (note: Thomas Bjørnholm was the grant

holder, while Dimitrios Stamou administered the grant on

a daily basis.)

Partners:

University of Copenhagen, IBM Zürich, Se-

masopht, Sophion BioScience, 7TM Pharma, Radiome-

ter, AQUAPorin, Novozymes

Centre for Antimicrobial Research CAR

Grant size and period:

27.5 miliion kr.; 2008-2014

Grant holder:

Professor Michael Givskov

Partners:

University of Copenhagen, Technical University

of Denmark, Universität Zürich, Teknologisk Institut, LEO

Pharma

Centre for Pharmaceutical Nanotechnology and

Nanotoxicology

Grant size and period:

28 million kr.; 2008-2014

Grant holder:

Professor Seyed Moein Moghimi

Partners:

University of Copenhagen, Technical University

of Denmark, H. Lundbeck, Nordic Vaccine Technology,

LiPlasome Pharma

A Nanotechnological Approach to Studying Interac-

tions of Biological Macromolecules

Grant size and period:

6.7 million kr.; 2005-2011

Grant holder:

Professor Jörg P. Kutter

Partners:

Technical University of Denmark, Universtity of

Copenhagen, Novo Nordisk

Nano-technology for ultra-high-speed optical com-

munications (Nano-Com)

Grant size and period:

5.9 million kr.; 2006-2011

Grant holder:

Professor Palle Jeppesen

Partners:

Technical University of Denmark, OFS

Metalloprotease sensitive drug delivery systems for

treating cancer and inflammatory diseases

Grant size and period:

7.8 million kr.; 2007-2012.

Grant holder:

Associate professor Thomas L. Andresen

Partners:

Technical University of Denmark, Bioneer

Computational models and tools for drug discovery

(COMODO)

Grant size and period:

7.8 million kr.; 2006-2011

Grant holder:

Associate professor Christian Nørgaard

Storm Pedersen

Partners:

Aarhus University, University of Copenhagen,

Molegro, Nuevolution

Development of new metal-oxide and -sulphide

catalysts

Grant size and period:

8 million kr.; 2006-2012

Grant holder:

Professor Flemming Besenbacher

Partners:

Aarhus University, Haldor Topsøe, Image Metro-

lology, SCF Technology

Nano- and Bio-functionalised Surfaces for Biofilm

Prevention

Grant size and period:

8 million kr.; 2007-2012

Grant holder:

Professor Niels Peter Revsbech

Partners:

Aarhus University, Teknologisk Institut, Alfa Laval

All five studies have made valuable contributions

to the evaluation report. The evaluation very much

benefitted from gaining dual insights into the projects

by combining self-assessment surveys with selective

interviews with a broad range of participants.

In particular, the evaluation panel underlines the

value of on-site visits and stresses that this ap-

proach is in line with international best practice

within evaluations of this type, where such visits

play a crucial role in gaining insight into research

projects and into the impacts of the programmes

that fund them.

The panel moreover emphasises the significance of

talking to both senior and junior scientists, as well

as company participants, to gain greater insight

into the activities and outcomes of the individual

projects. A thorough understanding of the true value

of a project in the form of research quality, potential

impact to industry or society, interdisciplinarity,

participants’ interdependency, and the role and

benefits of such projects for young researchers can

be much more reliably assessed when complement-

ing self-assessment surveys with face-to-face inter-

views of participants.

12/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

Main conclusions

and

recommendations

of the panel

due to benefits from the versatile education pro-

vided by interdisicplinary projects.

— Several projects have delivered scientific

breakthroughs. The number of major technical

breakthroughs, however, is low. Until now, the

commercial benefits of the projects are very

limited. As the evaluation has been undertaken

just five years after the launch of the NABIIT

programme, it is too early to conclude on the

commercial impact of the research undertaken.

In the experience and assessment of the panel,

the full commercial effects cannot be expected

to materialise in a programme of this nature

until approximately ten years after the research

activities took place. Nonetheless, results of the

self-assessment surveys and interviews under-

taken in connection with the evaluation point to

a need for promoting greater focus on possible

applications of ongoing research.

— The panel recommends a balanced use of small

and large grants. While large grants enable

ambitious, interdisciplinary projects with critical

research mass and extensive user involvement,

small grants play an important role in allowing re-

searchers to explore promising research avenues

before

establishing large scale research projects.

Although almost half of the projects funded by the

NABIIT programme had not been completed at the

time of evaluation, the panel finds that it is possible

to make some relevant and significant observations

on the overall performance and outcomes of the

programme.

Overall, the panel considers the NABIIT programme

to a well-functioning and important, high-quality

research programme.

In all programmes and research communities, there

is scope for improvement. Therefore, this evaluation

report focuses on areas where there is a possibility

for improvement and thus to further increase the

positive impact of the NABIIT programme. The con-

clusions and recommendations of the panel should

been seen in this light.

Evaluation of the NABIIT programme

The main conclusions regarding the NABIIT pro-

gramme can be summarised as follows:

— Scientific production and quality of the projects

funded by the NABIIT programme is considered

to be high and clearly above average.

— Scientific impact of publications from projects

funded by the programme is satisfactory. The

panel believes, however, that there is potential

to improve the impact even further by aiming for

a higher degree of interdisciplinarity and by tar-

geting top international scientific journals.

— Education and training of young researchers by

the programme appear to be excellent, primarily

Second, the current technology transfer system

in Danish universities does not appear to be ef-

fective in supporting the application of research.

Participants in NABIIT funded projects consistently

pointed to the technology transfer system as a ma-

jor obstacle to the commercialisation of research

results. This problem is common to most countries,

Additional, overall reflections from the panel

In connection with the evaluation of the NABIIT pro-

gramme, the panel spent a significant amount of

time examining this scientific area in Denmark. The

observations made by the panel have given rise to

some more general reflections, which lie beyond the

scope of the terms of references for this evaluation,

but which nonetheless hold great importance for

the research areas addressed by the NABIIT pro-

gramme.

First, there appears to be a general need to stimu-

late and create means by which young, talented re-

searchers in Denmark can start their own research

careers. The panel saw little evidence of young

researchers establishing and demonstrating their

intellectual independence, as many young scientists

consistently researched and published in collabo-

ration with senior scientists, notably their academic

advisors. The panel believes that this could have a

negative impact on the future supply of outstanding

scientists in Denmark. While ensuring this supply

is not the responsibility of NABIIT, the programme

could play a role in supporting the early careers of

young researchers.

13/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

but nonetheless an important issue which must be

remedied in order for the full potential of research

programmes like NABIIT to be realised.

Third, there seems to be a potential in estab-

lishing a world-class focal point for biomedical

engineering in Denmark, which could be realised

through a merger of existing strengths in engi-

neering at the Technical University of Denmark

and biomedical research at the University of Co-

penhagen.

The conclusions and recommendations of the

panel are presented in table 3.1 on the next page

and in more detail in the subsequent chapters of

this report.

Table 3.1. Main conclusions and recommendations of the evaluation

Evaluation theme

General reflections and

recommendations from the

panel

Main conclusions

Smaller project grants can play an important role in allow-

ing scientists to explore new research paths, methods,

means of interdisciplinary collaboration, or cooperative

relationships with industry

before

committing substantial

amounts of funding and time to large research projects

and centres. It is the panel’s observation from site visits

that small, well-focused projects can have a high impact.

Centres are particularly appropriate for new interdis-

ciplinary areas addressing important and challenging

problems to be solved, where the scientific partners have

demonstrated extraordinarily high potential in delivering

solutions to solving such problems and where there is also

a strong industrial and/or societal interest in exploiting

such solutions. More selective and ambitious use of cen-

tre grants would allow for e.g. higher degree of interdis-

ciplinarity, more focus on commercialisation (the whole

“value chain”), closer collaboration with users and better

training of young researchers

Research quality

To which extent has NABIIT con-

tributed to advancing research

of high international standing?

In the assessment of the panel, the scientific

quality

publications from NABIIT-funded projects is clearly above

average. Moreover, the

productivity,

measured as the

number of publications per krone invested by the NABIIT

programme, appears to be high by national as well as

international standards.

It is the assessment of the panel that the level of scien-

tific impact of the research produced by NABIIT-funded

projects seems to be above average when comparing to

similar research groups in Denmark and average when

compared to similar, groups funded by research grants in

comparable countries.

In two of the seven research projects interviewed by

the evaluation panel, there was clear evidence of true

thought leadership, which by definition means setting a

novel agenda on the stage of world science and technol-

ogy. Taking the overall quality level of Danish research

within NABIIT relevant fields into account, there should be

potential for increasing the overall degree of though lead-

ership in projects funded by NABIIT.

Based on the panel’s observations from the self-assess-

ments provided by the participants in the NABIIT-funded

projects, the majority of the projects have engaged in

some degree of dissemination targeted toward non-aca-

demic audiences. However, the degree and form of these

activities vary greatly from one project to the next.

A clear focus on promoting thought leadership in the call

for applications as well as in the subsequent peer review

and selection process is likely to improve this ratio and thus

result in a higher degree of thought leadership, by prioritis-

ing activities aspiring to deliver new and ground-breaking

results and with significant application perspectives. Such

factors should play a significant role in the reviewing and

the prioritisation process.

The Council should continue to prioritise and encourage

public dissemination. Dissemination activities, however,

should not be explicitly addressed in the calls or the peer

review process, but instead be included as an obligation as

part of the funding contract.

Reconsider the use of

centre

grants to make them more selec-

tive, ambitious and demanding – both with regard to quality

and relevance in the form of industry importance or solving so-

cietal needs. A way could be to use the project grants as the

basis for centre qualification, thus creating a highly compet-

itive environment among project grants and grant holders to

have the potential to qualify for a broader and more long-term

type of centre grant. A way to differentiate between centres

and projects could also be to give centre activities a chance of

an increase in funding on the level of 20 – 30 percent at a mid-

way evaluation, where certain achievements, like increased

industry participation, have been reached.

The evaluation panel is of the opinion that the overall

level of ambition of the programme as a whole and of the

individual projects supported by NABIIT both could and

should be higher as the panel believe the basis for higher

performance is there. A move towards a higher level of

genuine interdisciplinarity and greater focus on application

perspectives is likely to support such an improvement. This

should be backed by more room for risk taking in the proj-

ects and thus also in the prioritisation and review process.

These recommendations are expanded upon in subsequent

conclusions and recommendations.

Key recommendations from the panel

Small grants should be preserved; the Council should not

only award large grants and be careful to have the right

balance between small and large grants.

14/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

Interdisciplinarity (and

the combined use of

technologies)

To which extent has NABIIT stim-

ulated (a) more interdisciplinary

research and (b) novel com-

binations of nanotechnology,

biotechnology and ICT?

The panel found limited evidence of high degrees of in-

terdisciplinarity (defined as research activities where the

ability to find a solution to key tasks is dependent on inter-

linked contributions from different research disciplines) in

the research projects interviewed. This, however, is not a

challenge that is unique to NABIIT but also faced by other

similar research programmes in the rest of the world.

The panel only saw few cases of novel, combined uses of

two or more of the three specific technologies. Again, this

is a general challenge in the research community, and not

specific to the NABIIT programme.

The panel believes that a strong focus in the programme on

the stated aim to deliver highly esteemed results with appli-

cation perspectives should stimulate more interdisciplinary

proposals and activities within the projects.

The Council should place a stronger emphasis on the com-

bined use of technologies, and work at interfaces between

technologies in the review process.

In the experience and opinion of the evaluation panel,

the outcome of the projects is expected to be positively

influenced in nearly all aspects by having more equal, en-

gaged and committed participation by industrial partners.

Industry involvement in the projects is recommended from

the outset of the project (as this, in the panel’s experience,

increases the likelihood of having committed and significant

collaboration with industry partners).This should not be a

definite requirement as certain important research fields

having high application potentials could still be at such a

premature level that significant industrial participation is

difficult to obtain in a small country like Denmark.

Commercial benefits are likely to be improved by involving

companies to a higher degree in the projects and by focus-

ing on issues core to companies’ research and develop-

ment activities.

Breakthroughs, innovations

and commercial exploitation

To which extent has NABIIT (a)

contributed to breakthroughs

through novel technological

combinations that can create

future opportunities for inno-

vation in industry and solutions

to societal challenges, and (b)

to which extent has NABIIT

stimulated increased interaction

between public and private

research?

Both projects and centres are delivering substantial sci-

entific results, but the commercial impacts and number of

breakthroughs are low, until now. In the assessment of the

panel, this is primarily due to weak involvement of private

companies and a lack of value-adding support from tech-

nology transfer offices (TTO’s) at the participating universi-

ties. However, it is important to keep the timing in mind and

in the assessment of the panel; commercial results can only

be expected from NABIIT-type projects after approximately

10 years. Nonetheless, the evaluation indicates a lack of

attention (by both public and private participants) on facili-

tating the commercial application of their research.

In most cases, company participation did not seem to be

aiming at solving important company issues. Rather, the

majority of company partners appeared to participate in

NABIIT projects to build fundamental insights and engage

in generic, precompetitive research.

The type of company participation varies greatly across

the projects.

More attention should be allocated in the evaluation of

applications concerning the type and degree of company

participation. For example, if company participation is pri-

oritised, care should be taken to avoid pseudo-commercial

participation, e.g. where authorised technological service

institutes participate as de facto technical research partners

but are presented as industry partners. Similarly, reviewers

should pay particular attention to the degree of involvement

and commitment of company partners in projects.

There is very limited participation by public sector users in

the projects. Nevertheless, many projects have potential

societal benefits, e.g. through improvements in health

care, food technology, ICT etc.

Training of young research-

ers

To which extent has NABIIT con-

tributed to postgraduate educa-

tion and researcher training?

Multi- and interdisciplinary projects are excellent tools for

educating young scientists. Interviews indicate that multi-

and interdisciplinary interaction in the projects funded by

NABIIT provide an excellent platform for education and

training of young scientists .Young scientists operated

primarily within their own field, but appeared to gain sig-

nificant insight and networks into other, relevant research

disciplines. In the panel’s experience, these are valuable

qualities in preparing young scientists for future careers in

both academia and industry.

Young researchers in NABIIT-funded projects seem to be

very content with their positions and their base at Danish

universities. There was little evidence of young research-

ers establishing and demonstrating their intellectual

independence, as many of the respondents consistently

researched and published in collaboration with senior

scientists, notably their academic advisors.

It is however important to note that this conclusion may be

biased by the fact that the panel only met young scientists

who had remained in the research groups where they under-

took their NABIIT-funded PhDs or post.doc fellowships.

There seems to be no reasons for a change in the pro-

gramme in order to better meet societal needs, except to

improve commercial impact as per the previous recommen-

dation.

The Council should continue to have a strong focus on Ph.D.

and post.doc education. In the experience of the panel,

demonstration of intellectual independence - e.g. in the

form of single-authored publications - is a key parameter

when deciding for recruitment in international research

institutions. In view of this, a stronger focus on supporting

young scientists who aim at setting their own research

agenda is recommended.

In addition, a research programme such as NABIIT could en-

force special efforts in the form of smaller projects, aimed

at boosting scientific independence of young scientists

aspiring for assistant professorships. Such programmes

should be designed so they become highly prestigious with

e.g. award-like structures.

15/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

Several young scientists interviewed did not know that

their research position was funded through the NABIIT

programme.

The Council should take steps to increase identification with

the NABIIT programme through joint activities comprising

both team building activities and courses with participation

of young scientists, both within and across the different

projects funded by the programme.

Most young researchers will leave academia to take up

positions in the public or private sector. The projects inter-

viewed by the panel, however, appear to predominantly

train their young researchers for a career in academia.

Again, this conclusion may be biased by the fact that the

panel met exclusively with young scientists who are still at

the institution, where they undertook their NABIIT-funded

Ph.D. or post.doc fellowship.

Development of Danish

research environments

To which extent has NABIIT con-

tributed to a strengthening of

research environments, includ-

ing an increased internationali-

sation of Danish research?

Based on insights gained from the self-assessment survey

and interviews, grants from NABIIT have helped to open

up new research fields or establish new research groups.

NABIIT-funded projects have given rise to a number of

follow-on projects and activities.

The Council could initiate courses or other joint activities for

young scientists that are part of a NABIIT-funded project in

order to train their mind-set to be fit for both academia and

the industry. Moreover, a greater degree of involvement

of company partners in projects, as recommended above,

would also contribute to the training of young scientists

associated with NABIIT-funded projects in priorities and

approaches to work in industry.

Research programmes like NABIIT are highly relevant. The

efficiency could be further improved by bringing the collab-

orating parties closer together.

The Council should give priority to stimulating and support-

ing spin-off projects from NABIIT projects, as this is likely

to increase the ultimate value created by the initial grant.

In the experience of the panel, and based on the insights

gained from the self-assessment survey and the inter-

views, the Council could achieve this by giving projects the

possibility of applying for a follow-up grant to pursue novel

research projects that arise from the original project.

It is the impression of the panel that the overall level of in-

teraction in the projects, national as well as international,

is acceptable and similar to what is found in comparable

programmes in other countries. The panel also observed

a high degree of variation in the level of and approaches

to collaboration among project partners.

In-depth analysis of the projects interviewed indicated

that there is an unrealised potential to strengthen the de-

gree of collaboration among participants in projects. For

example, there were few projects which involved co-au-

thorship among several/all of the participating research

groups. In the experience of the panel, this indicates that

such collaboration was lacking; this conclusion was more-

over substantiated in several interviews.

There are obstacles in Denmark, as in several other coun-

tries, for mobility between academia and the public or

private sector.

There were examples of investments in expensive hard-

ware and research infrastructure that seemed to be

clearly influenced by funding of projects within the NABITT

programme.

DAMVAD 2012

The panel recommends that projects become more aware

of the need and benefits from close collaboration when

conducting true interdisciplinary research.

The Council should take steps to ensure that future pro-

grammes through highly application specific objectives

stimulate the integration of other disciplines at the centres

where large infrastructure investments in specific areas

have been made.

16/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

About

the NABIIT

programme

Figure 4.1. Completed and

ongoing projects

Ongoing

projects

Completed

projects

DAMVAD 2012

In total, NABIIT has supported 36 projects distribut-

ed across three types of instruments:

— 32 strategic research projects (granted in the

period from 2005 to 2008, both years included)

— Two strategic research alliances (2007)

4.1 Background

The NABIIT programme was established based on a

political ambition to realise and utilise new opportu-

nities for research and innovation through combined

research on and application of nanotechnology,

biotechnology and ICT.

A key motivation behind the programme was the ex-

pectation that novel, synergistic combinations of the

three technologies can help address key societal

challenges in e.g. the health sector and in regards

to the environment. In addition, new developments

in research on the three technologies were expected

to yield results and technological advances of rel-

evance to a number of key business sectors where

Denmark holds a strong international position, e.g.

Alliances

Projects

Centres

— Two strategic research centres (2008).

Where specific instruments are not specified in this

report, the term “project” is used to refer to all 36 re-

search activities that have received NABIIT funding.

Table 4.1 shows the distribution of the 36 grants by

the research institution that the grant holder was

affiliated with at the time of application.

Table 4.1. Grants, by type of

instrument and by affiliation

of grant holder (at the time of

application)

in biotechnology.

The programme was hence established with the

purpose to identify and develop future opportunities

for commercial innovation and solutions to societal

problems through combinations of nanotechnology,

biotechnology and ICT.

The NABIIT programme was managed and admin-

istered by the Programme Commission on Strategic

Growth Technologies.

4.2 Grants and instruments

As stated in chapter 1, a total of 36 projects have

received support from NABIIT during the programme

period 2005 to 2008. At the time of data collection,

19 of the 36 projects had been completed.

University of Copenhagen

Technical University (DTU)*

University of Aarhus**

Roskilde University

IT University of Copenhagen

Southern Danish University

Total

DAMVAD 2012

* Includes the national research laboratory RISØ, which merged with

the Technical University of Denmark in 2007

** Includes the national research laboratory

Dansk Jordbrugsforskning,

which merged with the University of Aarhus in 2007

For more detailed information on these instruments, please

A project is considered completed when the final report has

been approved by the programme commission.

see The Danish Council for Strategic Research,

Strategic research –

Principles and Instruments,

1 January 2012.

17/

Evaluation of The Strategic Research Programme NABIIT

Total

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

Box 4.1. presents the three different types of instru-

ments that is included in the NABIIT programme. The

description below is based on how the instruments

are described in 2012 and thus not how they were

during the years where the NABIIT programme

grants were granted. The instruments have evolved

over the years.

4.3 Size of grants

The variety in the size of grants awarded under the

NABIIT programme can be seen in table 4.2. The

average size of NABIIT grants was 8.8 million.

Box 4.1. Three instruments

Strategic research projects

are expected to find solu-

tions to a relatively restricted set of research issues and

promote participants’ development of interdisciplinary

expertise while contributing to innovation among stake-

holders in the public and private sectors. They have a

duration of 3-5 years and are eligible for upwards of DKK

10 million in funding from the Council.

Strategic research centres

are employed where there

is a need for research environments of a high scientific

standard focused on developing solutions to complex

problems. Research management should be given high

priority, and interdisciplinary research is expected to

build innovation and entrepreneurial potentials. Centres

are expected to develop into internationally leading re-

search environments and to continue their collaborative

activities with public, private and international partners

when funding ceases.

They have a duration of 5-7 years and are eligible for

upwards of DKK 30 million in funding from the Council.

Strategic research alliances

are used when there is a

need to create alliances between scattered, small-scale

research environments in Denmark and relevant public,

private and international actors to find solutions to soci-

etal challenges. Alliances must ensure that the parties to

the alliance pursue its principal objectives and that the

research activities are undertaken in a genuine collab-

oration. They have a duration of 5 years and are eligible

for 15-20 million in funding for research and networking

activities.

Source: Based on The Danish Council for Strategic Re-

search,

Strategic research – Principles and Instruments,

January 2012

Table 4.2. Grant size (minimum,

maximum, average and total), by

instrument (in million DKK)

Min.

Centres

Alliances

Projects

Total

DAMVAD 2012

Max.

28.0

15.0

11.9

28.0

Ave.

27.7

12.1

7.4

8.8

Total

55.5

24.2

238.1

317.8

27.5

9.2

3.4

While the total amount of funds granted on a yearly

basis by NABIIT has remained relatively stable during

the programme period (cf. figure 4.2.), there were

however substantial year-to-year differences in the

minimum, maximum and average size of grants (cf.

figure 4.3). These differences are in large part due

to the establishment of the two strategic research

networks in 2007 (awarded DKK 9.2 and 15 million

in Council funding) and the two strategic research

centres in 2008 (awarded DKK 27.5 and 28 million).

Figure 4.2. Total NABIIT grants, by

year (in million DKK)

120

100

2005

2006

2007

2008

54,9

99,7

76,1

87,1

DAMVAD 2012. N = 36 grants

Figure 4.3. Size range and

average size of NABIIT grants, by

year (in million DKK)

2005

2006

2007

2008

7,8

7,1

8,4

14,5

DAMVAD 2012. N = 36 grants

18/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

The programme commission behind NABIIT has

emphasised co-financing as part of the assessment

criteria for projects, alliances and centres. The

figure below thus shows the average size of grants

in each year compared to the average size of co-fi-

nancing per grant awarded in that year. The provi-

sion of co-financing has remained relatively stable

over the evaluation period, save for a decrease in

2007 and a substantial increase in 2008. The latter

is explained by the fact that funding was granted

in 2008 for two large strategic research centres,

which also involved considerable co-financing from

participants.

Figure 4.4. Average amount of

co-financing (in million DKK)

2005

2006

2007

2008

7,3

7,9

6,5

7,1

4,9

14,5

8,4

11,7

Average co-financing

Average granted amount

Data provided by Ministry of Science, Innovation and Higher

Education 2012, N = 36 grants

4.4 Participants

In total, 158 organisations have participated in

the 36 projects funded by NABIIT. Projects have

between two and nine participating organisations

On average, four organisations participate in each

project.

Approximately two-thirds of the participants were

university departments or research groups or, in

a few instances, hospitals. Private companies ac-

count for the rest of the participants; this category

also includes interest organisations representing

particular industry sectors and authorised tech-

nological services institutes (that is, not-for-profit

research organisations operating on commercial

terms).

These numbers are based on the original grant applications;

as such, there may have been subsequent changes to some of

the projects that are not reflected here.

19/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

General

reflections

and

recommendations

from the panel

laboration, or new cooperative relationships with

industry

before

committing substantial amounts

of funding and time to large research projects and

centres.

The panel recognises that the Council’s effort in pro-

moting interdisciplinary research projects that can

address societal challenges in collaboration with

industry often also calls for large-scale grants with

a longer time horizon. However, the panel believes

that a continuous stream of smaller grants should

be seen and promoted as an excellent opportunity

to create a very competitive environment for qualifi-

cation to larger centre-type grants.

Moreover, it can be easier for researchers to work

closely together on a specific challenge if they are

working in a small constellation of partners, where-

as large grants tend to involve larger groups of

collaborators. Smaller projects thus play an impor-

tant role in promoting diversity and enabling exper-

imentation, and could therefore play an important

role in creating a foundation for larger, subsequent

research ventures.

The panel has also seen examples of smaller projects

that have allowed young scientists to explore new

research paths, thereby laying an important founda-

tion for their own future research career. The panel

observed a general need to stimulate young talented

This chapter presents some general conclusions

and recommendations that the panel wishes to

highlight based on its evaluation of the NABIIT

programme. These conclusions are presented in a

separate chapter, as they cut across the selected

evaluation themes discussed in the subsequent

chapters of the report. The conclusions and recom-

mendations presented here should thus be seen as

overall recommendations to Council for Strategic

Research and the Programme commission on Stra-

tegic Growth Technologies that is responsible for

NABIIT and for other related research programmes.

5.1 Both small and large grants are needed

Conclusion:

Smaller, focused projects have clearly

researchers to step up and create their own research

agenda (see section 9.3); the Council could play

a role in providing this stimulation by encouraging

young scientists to apply for smaller grants.

The panel wishes to point out that a move from

smaller to larger grants has been seen elsewhere

in Europe, for instance in the United Kingdom. In the

observations of the panel, large grants concentrate

funds in the hands of a small number of individuals.

This can make it more difficult for young researchers

to establish their own research careers. Moreover,

it can have the adverse effect of reducing diversity

in the overall research portfolio, by decreasing the

availability of smaller grants for the exploration of

new research paths. The panel underlines that their

recommendation should be seen in light of the Coun-

cil’s decision to prioritise larger grants that can lift

more ambitious projects and involve more partners.

This represents a decision to move away from small

grants and towards larger grants. Today, the Council

only supports projects with a minimum size of 10

million DKK in public funding. This approach naturally

leads to conservative choices of projects and princi-

pal investigators, and reduces opportunities for larg-

er risk-taking within smaller budget envelopes.

Notwithstanding the above, there are also good

arguments to simultaneously provide larger grants

that enable the establishment of research centres

shown their merits

Recommendation:

Continue to provide smaller

grants, also below 10 million kroner, and use perfor-

mance on these to determine suitability for larger

grants in promising areas.

The panel has seen good examples of smaller pro-

jects being very efficient in delivering outstanding

results through a clear focus on specific challenges.

They have been able to demonstrate proof-of-prin-

ciple that can be attractive to certain industries.

Small project grants can play an important role in

allowing scientists to explore new research paths,

new methods, new means of interdisciplinary col-

20/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

with greater critical mass, broader research tar-

gets, and a longer lifetime, e.g. up to 10 years.

The panel thus suggests the Council to pursue a

balanced approach to grant size, which can enable

both small and large grants.

In connection with larger grants, the panel moreover

recommends putting provisions in place to prevent

rent seeking by senior scientists, e.g. co-author-

ships related only to financial intermediation.

5.2 Greater differentiation between centre

grants and project grants

Conclusion:

Centres are particularly appropriate

have been somewhat documented. Hence, there

must be a solid basis for thorough peer review of

the aspiring projects focusing on i) the importance

of the issues to be addressed by a centre, ii) the

robustness of interdisciplinarity in the activities con-

stituting the core of the centre and iii) the likelihood

that the partners will be able to meet the goals.

The stated objective of the NABIIT programme is to fill

a role in the middle of the innovation value chain, that

is, somewhere between basic science and applica-

tion

. To fulfil this role, research activities supported

by the programme must focus on a specific research

objective in a well-defined area and, at least to a cer-

tain degree, address issues in the entire value chain.

According to the panel, requirements and the level

of ambition as well as the scale and scope of ac-

tivities should be increased for a centre investment

to be justified and to support the attainment of the

objectives set for the programme.

for new interdisciplinary areas addressing important

and challenging problems to be solved, where the

scientific partners have demonstrated extraordinarily

high potential in delivering solutions to solving such

problems and where there is also a strong industrial

and/or societal interest in exploiting such solutions.

Recommendation:

Use the project grants as the ba-

sis for centre qualification. Create a highly competitive

environment among project grants and grant holders

to have the potential to qualify for a broader and

more long-term type of centre grant.

The panel has only met with two strategic research

centres, as only two such centres have been fund-

ed by the NABIIT programme. In the following, the

panel draws on their experience to present some

more general reflections on how centre-type grants

can be most effectively deployed as instruments for

strategic research. Overall, the panel recommends

a high degree of differentiation between the re-

quirements and expectations of project grants and

centre grants.

A key aim of the NABIIT programme was to yield

commercial breakthroughs through combinations

of nanotechnology, biotechnology and ICT that can

create future opportunities for innovation in the in-

dustry and solutions to societal needs.

The panel shares this ambition. It is the viewpoint of

the panel, based on their experience, that much of

today’s ground-breaking research takes place at the

interfaces between different disciplines, and that the

objective of such research is to look for answers to

major problems requiring a solution. This is illustrat-

ed by the fact that publishing in high impact journals

requires researchers to focus less on the technicali-

ties of their research and more on the broader con-

text and implications of their research for addressing

problems of importance to industry and society.

The outcome of research is to some degree unpre-

dictable. Nevertheless, it is the panel’s opinion that

centre grants should be used in situations where the

needs and chances for meeting such high ambitions

The panel suggests that requirements to obtain a

centre grant should be:

— The grant holder must have demonstrated

out-standing performance as manager of ap-

plication-relevant and interdisciplinary research

projects.

— Industry participants should be intimately in-

volved in the centre’s activities

and

, when possi-

ble, co-located on campus.

— The entire value chain – including business de-

velopment – should be represented in the centre

and addressed by the management of that cen-

tre. This would allow centres to pursue commer-

cialisation activities that cannot realistically be

pursued under project grants.

— The centre should have its own expertise (or

immediate access to such expertise) required

to help in commercialising research results. In

connection to this, the panel suggests that it

should be possible to set aside a small portion

of the funding to pay a fraction of the salary for

someone acting as a “business developer” for

the centre. The embedded developer (within the

project) should be responsible for commerciali-

sation of the research results, including patents

and licenses, formulating business plans, mar-

keting and private sector fund raising for next

step developments. The panel saw a positive

example at DTU of such a business driven advi-

sory function which seemed to stand in contrast

to other, more remote administrative or legal

approaches to technology transfer.

The objectives for the NABIIT programme are founded

within

the concept of research quality of the Council for Strategic

Research, which is based on three equivalent criteria: the rele-

vance, potential impact and academic quality of research.

21/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

The panel strongly emphasises that the centres

should also have a high level of co-financing from

the university. The current maximum commitment

of 10% co-financing is inadequate. Setting such a

important criteria in the evaluation process. Exam-

ples observed by the panel include:

— Several projects involve what the panel consid-

ers to be pseudo-company type of participa-

tion. This indicates that the criterion of industry

participation is often not sufficiently evaluated

and prioritised in the review process. It appears

that the evaluation of the relevance dimension,

quality, commitment and participation of the

industrial participants are not subject to the

same thorough evaluation as are the academic

participants.

— In one of the grants, a group (from the same

university) which was not convincingly relevant

to the overall activity was invited to join, where

the objective clearly seemed to be to raise the

scientific reputation of the application.

— The criteria of requiring involvement of at least

two distinct scientific areas (nanotechnology,

biotechnology or ICT) seems only rarely to have

led to true interdisciplinary work and instead

resulted in multidisciplinary projects with mod-

est levels of interactions (see section 7.1). The

panel agrees that the vision of working at the

interfaces between the three disciplines is a

good idea, given that this is where the interna-

tional research community is headed and also

where future technologies are likely to emerge.

If there is a lack of qualified true interdisciplinary

proposals, smaller monodisciplinary projects

should be preferred to “artificially constructed”

multidisciplinary projects.

As a way forward, the panel proposes that the

Council is very clear in its communication, evaluation

and prioritization of applications. One way of han-

dling this could be to clearly state what minimum

requirements for a centre grant are, for example:

— The application concerns a research activity

within one of the three defined areas (nanotech-

nology, ICT or biotech).

— The objective of the research activity is clearly

targeting a solution to answers on major prob-

lems that could lead to a very clear application

perspective, if successful.

— Has true and engaged participation from indus-

try partners or the like having a sincere interest

in exploiting a positive outcome of the research

activity (not only measured by their funding

contribution but also through their expectations

regarding further application possibilities).

limit in large strategic research ventures is against

the very basic idea behind strategic research

funding; namely the idea to set up mechanisms to

motivate and drive the public research community

in a direction considered of strategic value to the

nation. The stronger the motivation of the research

community to direct its activities in such a direction,

the better it is.

As part of a centre grant, the rectors of the univer-

sities involved should provide the physical space to

enable co-location where demanded, and agree

on the financing and other practical arrangements

to enable this. This could also help enforce and in-

crease interdisciplinary research collaboration.

The panel did not meet the two strategic research

alliances funded by NABIIT during the interview

week. However, having a look at the description of

alliances; “

alliances are used when there is a need

to create alliances between scattered, small-scale re-

search environments in Denmark and relevant public,

private and international actors to find solutions to so-

cietal challenges

”

the aim of the NABITT programme

of delivering breakthroughs based on interdiscipli-

nary activities could be a challenge for such type of

instrument. The panel further wish to convey a word

of caution relating the amount of instruments that

is offered in programmes like NABIIT. It may cause

confusion with too many different alternatives. Ad-

ditionally, energy should be focused on promoting

the real aim of the programme which is to enforce

excellent science with high application perspectives

and not different forms of instruments.

In the experience of the panel, assessment cri-

teria are important in differentiating a research

programme from other funding agencies and pro-

grammes. However, care must be taken not to set

out conflicting criteria or promote “box ticking”. By

“box ticking”, we refer to situations where appli-

cants design their proposals primarily with a view to

meeting as many explicit or implicit criteria as possi-

ble, rather than focusing on what would create opti-

mal conditions for the proposed research project.

In the course of the interviews, the panel observed

several examples of what could be considered

“window dressing” with the purpose of making the

application more likely to fulfil what is believed to be

According to “Aftale om fordeling af globaliseringsmidlerne

tilforskning og udvikling 5. november 2008”, the Council for

Strategic Research can at most require 10% in co-financing of

grants from public research institutions. Such institutions may

provide additional funding, but the Council is not allowed to take

this into account in the assessment of proposals.

22/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

Research

quality

gramme was collected from the grant holders in

June 2012. A total of 455 journal publications were

reported. 389 (85 percent) of these 455 publica-

tions reported could be retrieved from Thomson

Reuters Web of Science, a bibliometric database,

which indexes publications in leading international

peer reviewed scientific journals.

The 19 projects that had been completed at the

time of evaluation generated 288 (63 percent) ar-

ticles. By comparison, the remaining, ongoing 17

projects have published 167 (37 percent) articles.

Not surprisingly, completed projects have pro-

duced close to twice as many publications as

ongoing projects, as completed projects have, on

average, 15.2 publications, compared to 9.8 for

ongoing projects (cf. table 6.1).

It should however be noted that there is a large

degree of variation in the number of publications

6.1 A high level of scientific productivity

Conclusion:

It is the assessment of the panel that the

generated by the 36 projects. For example, as

can be seen from table 6.1, so far the smallest

number of journal articles to have come out of a

completed project is 4, while the highest number

of publications from a NABIIT-funded project is

43.

Moreover, as the final publications from com-

pleted projects are submitted to and accepted

by journals, and as ongoing projects approach

completion, the total number of publications from

NABIIT-funded projects is likely to increase sub-

stantially.

productivity of projects funded by NABIIT, as indicated

by the number of publications per krone invested by

the NABIIT programme, appears to be high by national

as well as international standards.

Scientific productivity refers to the amount of re-

search produced in NABIIT-funded projects, as

indicated by the volume of articles that have been

published in scientific journals.

It should however be noted that it was not possible

for the panel to determine the extent to which the

level of productivity is affected by cross-subsidies.

The research presented in the publications reported

is likely to have been directly or indirectly support-

ed by other sources of research funding, e.g. from

other programme commissions under the Council

for Strategic Research, from the Council for Inde-

pendent Research or from other public and private

research funding bodies, including the EU.

Data on scientific publications from the 36 projects

that have received funding from the NABIIT pro-

Table 6.1. Number of journal

publications reported

Total

Completed projects

Ongoing projects

Total

288

167

455

Min.

Max.

Ave.

15.2

9.8

12.6

Source: DAMVAD 2012 based on the bibliometric analysis

Table 6.2. Average cost per publication from NABIIT funded projects (in DKK)

Total

Number of publications in Web of Science

Total funding in DKK

Cost per publication in DKK

Number of publications per million DKK

DAMVAD 2012 based on the bibliometric analysis and funding data from NABIIT

Min.

138

174.403.345

1.263.792

0,79

Max.

389

317.837.341

817.062

1,22

251

143.433.996

571.450

1,75

23/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

Table 6.2 presents information on the number of

scientific publications (in Web of Science indexed

journals) produced by NABIIT funded projects in re-

lation to the amount of funding granted by the NABI-

IT programme. The table shows that completed pro-

jects have, on average, generated 1.75 publication

per million krone, while ongoing projects have (so

far) produced 0.79 publications per million krone. It

should however be noted that ongoing projects are

expected to generate more publications, ultimate-

ly leading to a higher number of publications per

krone of financing. In the assessment of the panel,

this amount of publications is reasonable, but the

panel emphasises that this kind of calculation must

be seen in relation to other results generated by the

projects, such as innovations and commercial appli-

cations and patents.

6.2 Scientific impact is internationally

competitive

Conclusion:

Based on the data available, it is the

tween “top level” and “other” journals. The second

approach is based on the Journal Impact Factor

(JIF) scores assigned to scientific journals indexed

in Thomson Reuters Web of Science. The third ap-

proach is based on the number of publications in

journals from the Nature Publishing Group (NPG).

Assessment based on the Danish authority lists.

Authority lists developed for use in the Danish bib-

liometric research performance indicator (which is

used to distribute base funding to university depart-

ments based on their scientific performance) can

be used as an indicator of the quality of the publica-

tions from NABIIT-funded projects.

These lists, which were developed by expert groups

appointed by the Ministry of Science, Innovation and

Higher Education, identify key scientific journals and

categorise them as “level 1” or “level 2” journals.

Level 2 publications are defined as the top 20 per-

cent of journals in their field.

Based on this classification, 184 (47 percent) of

the journal publications from NABIIT-funded pro-

jects appeared in top journals, the so-called “level

2”-journals in the Danish authority lists. This applies

to 52 and 38 percent of publications from complet-

ed and ongoing projects, respectively (cf. table

6.3).

assessment of the panel that the level of scientific

impact of the research produced by NABIIT-funded

projects is above average when comparing to similar

research groups in Denmark and average when com-

pared to similar, groups funded by research grants in

comparable countries.

Recommendation:

Given the level of talent in the

Danish research community and the quality of the

NABIIT programme, the panel believes that it is possi-

ble to raise the impact of NABIIT funded research even

further. The panel therefore urges funded academics

to aim for top international journals, for example by

increasing the level of genuine interdisciplinarity and

focus on application perspectives in projects. This

move could be further stimulated by an increased will-

ingness to take risks in the prioritisation and review

process by the Council

Scientific impact refers to the impact of a given re-

search project on the scientific community, typically

as indicated by the number of citations awarded to

publications presenting the results of this research

or to the journals in which these publications ap-

pear.

Three approaches to assess impact have been

chosen. The first uses the so-called Danish authority

lists of scientific journals and their distinction be-

Thus, the proportion of level 2 publications is con-

siderably higher than the top 20 percent that they

are intended to represent in the authority lists.

These results indicate that the quality of research

produced in NABIIT-funded projects is higher than

the average research quality in the Danish scientific

fields concerned, which should also be the case as

the projects have been selected based on a com-

petitive peer review process.

Table 6.3. Scientific articles, by

the “authority list” level of the

journals in which they appeared

Publications

from completed

projects

Publications

from ongoing

projects

No.

Level 2 (“top level”)

journals

Level 1 journals

131

107

251

Pct.

52%

43%

100%

No.

138

Pct.

38%

45%

17%

100%

Typically, the scientific impact of publications would be

assessed through an analysis of the citations. A robust citation

analysis however requires a three-year window from the time

of publication to allow for a reliable estimation of the impact

of a given publication. It was not possible to obtain such a

window for this evaluation, as the NABIIT programme ran from

2005 to 2008, and most of the scientific articles generated by

NABIIT-funded projects have therefore been published within

the past few years.

NA*

Total

DAMVAD 2012 based on the bibliometric analysis performed for the

evaluation. * NA (Not Available/Not Identified) indicates that the

journals were not included on the national authority list.

24/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

Assessment based on JIF scores.

Scientific im-

pact can also be indicated by the Journal Impact

Factor (JIF) scores assigned to all journals in the

Web of Science database.

109 (28 percent) of the publications from NABI-

IT-funded projects that could be identified in Web of

Science appeared in journals that have a JIF score

of 5 or higher. 13 percent (49 articles) appeared in

journals with a JIF score of 7.5 or higher, as indicat-

ed in table 6.4.

Figure 6.1 Number of scientific

articles in NPG journals (2007-2011)

normalised by population size

2007

Germany

2008

2009

2010

Switzerland

2011

Denmark

Netherlands

Table 6.4. Scientific articles, by

the JIF scores assigned to the

journals in which they appeared

Publications

from completed

projects

No.

0 > JIF > 4

4 > JIF > 5

5 > JIF > 7.5

7.5 > JIF > 10

JIF above 10

Total

150

251

Pct.

60%

12%

16%

100%

Publications

from ongoing

projects

No.

138

Pct.

51%

22%

14%

100%

Based on data collected from Web of Science by the Ministry of Sci-

ence, Innovation and Higher Education 2012. Based on the number of

articles with one or more authors from the indicated country

In total, six articles that have been published as a

result of NABIIT-funded projects are included in a

journal from the NPG journals.

It is the assessment of the panel that a research

programme such as NABIIT ought to have a higher

representation of NPG journals within the country’s

combined volume. Additionally, the panel encourag-

es the research communities to increase their ambi-

tion in the publication of research results and to aim

for journals within the NPG.

International co-publication

According to the bibliometric analysis, 168 articles

(43 percent) were co-authored by one or more in-

ternational collaborators.

DAMVAD 2012 based on the bibliometric analysis performed for the

evaluation.

Measured on impact, Danish research is significant-

ly above world average in fields like physics, chem-

istry, engineering and biochemistry in general.

Assessment of publications in NPG journals.

Figure 6.1 illustrates how Denmark as a whole ranks

relative to Germany, the Netherlands and Swit-

zerland, when comparing the volume of scientific

articles published in Nature Publishing Group (NPG)

journals. NPG journals comprise prestigious journals

within chemistry, clinical practice and research,

earth and environment, life sciences and physical

sciences. NPG comprises some of the top journals

within bio- and nanotechnology; thus, publications

in NPG journals can be seen as an indicator of the

performance of the projects.

The figure shows that when publications are normal-

ised relative to population size, Denmark appears

to be performing below Switzerland, on par with

Holland and above Germany.

For completed projects

USA

Netherlands

France

Germany

Sweden

China

England

Italy

Japan

Spain

For ongoing projects

Poland

England

Germany

Israel

Finland

France

Sweden

Hungary

Italy

Singapore

The top ten countries that NABIIT projects have

co-published papers with are listed in table 6.5.

Table 6.5. Top ten countries* that

NABIIT projects have

co-published scientific articles with

DAMVAD 2012, based on data from Web of Science.

* Ranked by number of articles including one or more authors from

that country

25/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

Potential for even higher scientific impact

It is the assessment of the panel that publications

from NABIIT-funded projects could and should aim

for an even higher level of scientific impact for two

reasons. First, these scientists have been selected

by the NABIIT programme based on the quality of

their research ideas and their academic track re-

cord. As such, their work is expected to have a high

scientific impact. Second, NABIIT has encouraged

interdisciplinary work, and high impact scientific

journals generally favour interdisciplinary work.

The panel wishes to stress, however, that the first

publications in new and emerging areas do not al-

ways reach high impact journals. Hence, if the pro-

gramme aims at renewing science and promoting

ground-breaking research, caution should be taken

towards placing too much emphasis on short term

impact.

The panel observed that many of the research

groups interviewed seemed to be heavily focused

on “technicalities” of their own research field and

paid less explicit attention to the ultimate applica-

tions, commercial value and benefits to society from

their research. However, in the experience of the

panel, publishing in high impact journals requires

researchers to focus less on the technicalities of

their research and more on the broader context and

implications of their research in addressing prob-

lems of importance to industry and society. It is the

assessment of the panel that the granted research

projects should place a stronger focus on address-

ing the (societal) problems to be solved and less on

technicalities within the individual research groups.

In the opinion of the panel, this will also strengthen

the impact of publications from projects funded by

the NABIIT programme.

It is moreover the assessment of the panel that if the

projects also had a higher degree of true interdisci-

plinarity in their research activities, they would have

a higher chance of being able to publish in highly

prestigious journals.

The Council could stimulate and support ambitions

to achieve even higher scientific impact by for exam-

ple calling for greater focus on solving real and im-

portant problems and on greater interdisciplinarity

in the review and selection of research proposals.

This also requires a greater willingness to take risks

– by applicants as well as reviewers.

6.2 Thought leadership

Conclusion:

In two of the seven research projects

fields into account, there is a potential for increasing

the overall degree of thought leadership in projects

funded by NABIIT even further.

Recommendation:

A clear focus on promoting

thought leadership in the call for applications as well

as in the subsequent peer review and selection pro-

cess is likely to improve this ratio and thus result in

a higher degree of thought leadership by prioritising

activities aspiring to deliver new and ground-breaking

results and with significant application perspectives.

In the assessment of the panel, two of the seven

research projects interviewed appeared to be defin-

ing entirely new research agendas and to be world

leaders within their respective fields. Both of these

projects had a very clear application perspective

and a clear and related research agenda. One of

these projects, which from a funding point of view

was a fairly small project, actually delivered out-

standing results in two quite different directions. To

the surprise of the panel, this activity was later not

prioritised as the foundation for a centre activity,

even though the research group had applied for a

subsequent centre grant.

Based on the interviews and self-assessment re-

ports, it is the assessment of the panel that many

of the projects focus on research areas that are the

subject of current research trends or “fads” in the

scientific community. The areas targeted in NABI-

IT-funded projects are, however, deemed by the

panel to be important research areas in terms of

both their scientific potential and their commercial

relevance.

The panel also observed that some projects were

more multidisciplinary than interdisciplinary and

“constructed” in a way so as to fulfil formal require-

ments by somewhat superficially involving different

fields and gratuitously including researchers for

their track records. This has lead the panel to make

a recommendation that lowering the formal con-

straints for projects might be a good way to further

improve the quality and outcome of the funded pro-

ject activities.

On an anecdotal note, the panel observed that a

surprisingly high number of both senior and junior

participants in the NABIIT funded projects have

been awarded grants from the European Research

Council (ERC). Though data on the total number of

ERC grants obtained by NABIIT funded scientists

was not available, this is a positive indication of

thought leadership in the projects.

6.3 Broad dissemination of research results

Conclusion:

Based on the panel’s observations from

interviewed by the evaluation panel, there was clear

evidence of true thought leadership, which by defi-

nition means setting a novel agenda on the stage

of world science and technology. Taking the overall

quality level of Danish research within NABIIT relevant

the self-assessments provided by the participants in

the NABIIT-funded projects, the majority of the pro-

26/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

jects have engaged in some degree of dissemination

targeted toward non-academic audiences. However,

the degree and form of these activities vary greatly

from one project to the next.

Recommendation:

The Council should continue to

supporting and assisting the projects in their dis-

semination efforts.

The panel suggests that dissemination via the web

could receive greater focus for the projects as it

offers a channel to reach a broad and internation-

al audience. Webpages are the most important

means for communicating with industry and also

with the general public. Moreover, data on traffic to

the website can provide valuable information on the

stakeholders of a research project.

It is the experience of the panel that individual web-

pages for small projects is not, however, always the

right solution. Instead, the panel emphasises the

importance of research groups having webpages

that are well-functioning and frequently visited by a

broad audience. Therefore, the panel suggests that

the projects should be more proactive in dissemi-

nating their research results through this channel.

Costs incurred in connection with maintenance of a

website can even be covered by NABIIT-type grants,

either as part of the overhead costs or as a sepa-

rate expense. For example, in some Swedish strate-

gic research grants, 3 percent of the total grant sum

is earmarked for activities related to project man-

agement, technology transfer and dissemination,

such as upkeep of a web presence.

prioritise and encourage public dissemination. Dis-

semination activities, however, should not be explicitly

addressed in the calls or the peer review process, but

instead be included as an obligation as part of the

funding contract.

Findings from the desk study and the review of the

calls for applications reveal that broad dissemina-

tion of research results is an area that was given

increased attention in later NABIIT calls.

In the self-assessment survey, some scientists indi-

cated that they have developed webpages to pres-

ent information on and disseminate scientific results

from their NABIIT funded projects. The self-assess-

ments also reveal that most dissemination activities

are targeted towards industry or the general public,

and employ communication channels such as talks,

participation in non-academic conferences, teach-

ing, publications in the popular press or trade jour-

nals, and various forms of popular scientific mag-

azines or shows on radio or television. Examples

provided in the self-assessment survey are:

— Blogs and popular science webpages (e.g. vi-

denskab.dk)

— TV shows with emphasis on science (such as

Danskernes Akademi)

— Press releases and articles in popular press

(e.g. in Ingeniøren)

— Courses and public lectures

— Exhibitions or lectures at “Experimentarium”

(a Copenhagen-based science centre).

Through the self-assessments, the panel has ob-

served how many different dissemination activities

the projects have initiated. In the opinion of the

panel, it can be difficult to assess the quality of

this type of popular scientific dissemination and to

determine whether or not the dissemination is ade-

quate. Instead, the panel wishes to emphasise that

dissemination to the general public is important and

should continue to be prioritised as part of the re-

search programmes. However, dissemination to the

public is not and should not be considered a core

activity of research projects. The panel therefore

suggests that dissemination requirements targeted

at the general public should not be part of the calls

for applications and thus should also be kept out of

the peer review process. Instead, and along with the

grant, projects should be obligated to disseminate

their research results, and this obligation should

be part of the funding contract. The programme

commission and the secretariat supporting the

programme commission should also play a role in

27/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

Inter-

disciplinarity

and technological

convergence

the form of

multidisciplinary

collaboration, i.e. in-

volving participants from different fields working

on a joint research topic from within their own field

while including knowledge and insights from other

fields.

Just over one fifth of academic respondents indi-

cated that the different participants in the project

engaged in

interdisciplinary

collaboration that is,

worked closely together on a joint research topic

through the use of processes and methods that

they have selected and developed together. Such

interdisciplinary collaboration was also found in a

few of the interviewed projects.

Finally, one fifth of the respondents stated that

participants from different research fields work on

a joint research topic but within their own field of

research, and thus engaged in limited interaction

between research fields.

The Council for Strategic Research and the pro-

gramme commission have strengthened their focus

on interdisciplinarity in the NABIIT programme.

The panel supports this priority. It is the percep-

tion of the panel that the focus in today’s strategic

research on a global scale is very much directed

towards trying to find answers to fundamental prob-

7.1 Projects are multidisciplinary, but the

degree of interdisciplinarity can be improved

Conclusion:

The panel found limited evidence of

lems and with a clear ambition to find new solutions.

Additionally, the complexity of the problems to be

solved entails that most answers are to be found at

the interfaces between different technologies or by

intimate use of combinations of different technolo-

gies.

7.2 Technology convergence

Conclusion:

The panel only saw few cases of novel,

high degrees of interdisciplinarity (defined as research

activities where the ability to find a solution to key

tasks is dependent on interlinked contributions from

different research disciplines) in the research projects

interviewed. This, however, is not a challenge that is

unique to NABIIT but also faced by other similar re-

search programmes in the rest of the world.

Recommendation:

The panel believes that a strong-

er focus in the programme on the stated aim to deliver

highly esteemed results with application perspectives

should stimulate more interdisciplinary proposals and

activities within the projects.

During the interviews, the panel observed that

there is a lack of co-publication and co-patenting

among project partners from different research

disciplines. This indicates a lack of true interdis-

ciplinary work. Indeed, the panel observed a sub-

stantial degree of

multidisciplinary

collaboration

within the projects interviewed. In other words,

most of the research groups participating in the

projects appeared to be working on common re-

search topics from within their own fields.

These conclusions are supported by findings from

the self-assessment survey among academic

participants. As illustrated in figure 7.1, more

than half of the academic participants in NABIIT

projects who contributed to the survey indicated

that research collaboration in their projects takes

combined uses of two or more of the three specific

technologies. Realising the potential from technology

convergence is not an issue which is specific to NABIIT,

but a more general challenge in this field of research,

which programmes such as NABIIT may help address

by pushing for more ambitious attempts to solve

important problems through novel combinations of

existing technologies.

Recommendation:

The Council should place a

stronger emphasis on the combined use of technolo-

gies and work at interfaces between technologies in

the review process.

A key objective of the NABIIT programme was to

identify and develop new combined use of na-

notechnology, biotechnology and/or ICT. In the

panel’s assessment, however, the programme has

had a negligible effect on the combined uses of the

three technologies. There is a global convergence

between these technologies, but few of the projects

were cutting edge in their approach to combining

the technologies.

28/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

The panel notes that if the interdisciplinary aims of

the programme had been rigorously enforced, many

of the interviewed projects that received funding

from NABIIT would not have been funded under the

programme.

The challenge of identifying and undertaking re-

search on novel combinations of nanotechnology,

biotechnology and ICT is not, however, specific to

the NABIIT programme but a general challenge

for the international research community in these

fields. Moreover, the panel recognises the effort that

has been invested by the Council to promote more

interdisciplinary research and increased techno-

logical convergence; realising such changes in the

scientific community is a lengthy process, which the

NABIIT programme appears to have contributed to.

Continued and increased efforts in this regard are

however recommended.

The aim to stimulate new technological combi-

nations is still a “live issue” and thus relevant for

efforts to increase the competitiveness of Danish

research and industry. This is particularly true as

both nanotechnology and ICT are technologies that

must be seen in an application perspective. In other

words, these technologies must be brought into use

in combination with other technologies for their full

potential to be realised.

For instance, combinations of nanotechnology

and ICT are creating new opportunities for health

care and planet care. Such innovations are being

pursued under similar funding calls by commercial

competitors of Denmark, including European na-

tions such as the United Kingdom.

To stimulate increased convergence of technologies

through the NABIIT programme, the proposal and

the reviewing process would have to be restructured

accordingly. According to the panel, ensuring con-

vergence of technologies is the responsibility of the

review process. This can for example be achieved

by ensuring that relevant users from e.g. industry

are involved in the projects, to inspire and guide

new uses of the technologies, and by ensuring that

all necessary areas of expertise are represented

among the academic participants in the project.

Figure 7.1. The nature of interdisciplinary collaboration in NABIIT, according to

academic participants

Project participants from different research fields work closely

together on a joint research topic through the use of processes,

terms and methods that they have selected or developed together.

25%

Project participants from different research fields work on a joint

research topic, from within their own field of research, but including

knowledge and insights from other research fields in their work.

57%

Project participants from different research fields work on a joint

research topic, but within their own field of research.

20%

Source: DAMVAD 2012 based on the self-assessment survey. N (academic participants) = 72

29/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

Breakthroughs,

innovations and

commercial

exploitation

The projects funded under the NABIIT programme

appeared to deliver a good level of scientific results

and also several scientific breakthroughs.

For example, in the assessment of the panel, four

of the seven projects and two centres that were se-

lected for interviews by the panel were considered

to have generated significant actual or potential

scientific breakthroughs.

Most of these scientific breakthroughs have not

been generated because of new combinations of

nanotechnology, biotechnology and/or ICT.

By comparison, the panel found evidence of very

few technical breakthroughs from the majority of the

projects funded by NABIIT, based on insights from

the interviews and on the assessments of project

participants as described in the self-assessment

survey. This may to some degree also be influenced

by the time frame as the majority of projects funded

by NABIIT are still ongoing or only recently complet-

ed. In the assessment of the panel, research funded

by programmes such as NABIIT can be expected to

result in commercial products approximately ten

years after the research has been undertaken. As

such, it is possible that some of the scientific results

achieved in the projects will contribute to technolog-

8.1 Many good scientific results and some

scientific breakthroughs achieved, but low level

of commercial impact

Conclusion:

Both projects and centres are deliver-

ical breakthroughs in the future.

The vast majority of academic participants in the

self-assessment survey, however, indicated that

their project was not expected to yield commercial

results. Many academic participants further under-

lined that commercialisation was never an explicit

objective of the project, which indicates that a basic

objective of the NABIIT programme was not well

communicated and prioritised in the execution of the

projects. In the self-assessment survey, only a few

academic participants stated that results of their

projects have been commercialised, typically by the

industrial partners in the projects, or that they are

currently making efforts to commercialise results.

In view of the lack of technical breakthroughs and

commercial attention, it is not surprising that few

innovations

patents, licenses or spin-off compa-

nies have emerged from NABIIT-funded projects.

In the experience of the panel, technical break-

throughs and commercial applications thereof

can take years to emerge from research projects.

However, it is important for research projects to pay

explicit attention to aspects of their research that

warrant patent protection from the outset of the

project. Patents signal and protect the potential

ing substantial scientific results, but the commercial

impacts and number of breakthroughs are low. In

the assessment of the panel, this is primarily due to a

weak involvement of private companies and a lack of

value-adding support from technology transfer offices

(TTOs) at the participating universities.

Recommendation:

In the experience and opinion

of the panel, outcome of the projects is expected to

be positively influenced in nearly all aspects by having

more equal, engaged and committed participation by

industrial partners. Industry involvement in the projects

is recommended from the outset of the project (as this,

in the panel’s experience, increases the likelihood of

having committed and significant collaboration with

industry partners).This should not be a definite require-

ment as certain important research fields having high

application potentials could still be at such a prema-

ture level that significant industrial participation is diffi-

cult to obtain in a small country like Denmark.

The panel underlines that breakthroughs should

include both scientific and technological break-

throughs.

Scientific breakthroughs

hold great

importance for research groups and thereby for

Denmark, as they increase the international visibility

of research groups and improve their funding op-

portunities and recruitment possibilities. In parallel

to this,

technological breakthroughs

hold importance

for industry and/or society.

Innovations are here defined as new or significantly

improved products, technologies, methods, processes or

equipment.

30/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

breakthroughs and applications can emerge rela-

tively early on. Indeed, the first patent applications

may even be filed in connection with the develop-

ment of the research proposal, particularly if indus-

try partners are closely involved in the project.

For the research funded by NABIIT to be able to

result in commercial inventions in approximately

ten years, the panel believes that it is necessary

for the projects to have a vision and a plan for how

their research will have an impact a decade into the

future. In the experience of the panel, such plans are

difficult if not impossible to formulate without close

collaboration with industry.

On a related note, it became apparent to the panel

during the course of the interviews that the majority

of projects are very much dominated by the uni-

versity partners. The projects and the objectives of

most projects seem to be defined by the university

researchers, after which relevant companies are

invited and added to the projects primarily to meet

the requirement of having company participation in

the projects.

The panel underlines that a much stronger partici-

pation of commercial partners in the formulation of

the aim, content and execution of the projects is like-

ly to improve the commercial outcome of these pro-

jects. The panel suggests that industry participation

should be encouraged and recommended from the

beginning of the project. However, there should be

a possibility of including industry participation at a

later stage in the projects where research results

are more likely to be closer to application. There-

fore, industry participation should not be a definite

criterion from the start of the project.

In the experience of the panel, establishing long-

term and close collaboration between academia

and industry requires the attention of high-level

people - that is, people who have the authority to

make decisions and commitments. For strategic

research programmes, like NABIIT the aim in the in-

dustry-related projects shall therefore be to ensure

industry participation at as high level as possible

and to address very important issues for the indus-

try partner(s).

The interviews and self-assessment also showed that

academic participants experience a large number of

barriers to commercial exploitation of their research

results, which are not connected specifically to the

NABIIT programme but which nonetheless appear to

affect researchers’ motivation and incentives to pur-

sue the commercial application and exploitation of

their findings from the NABIIT-funded projects.

In particular, the panel noted that the TTO system in

the participating universities was criticised by the

majority of the scientists for not adequately or suf-

ficiently supporting commercialisation of research

results. There seemed to be a need for a more busi-

ness driven approach rather than an administrative

and legally driven approach to TTO.

The criticism was especially related to the speed,

the level of competence and efficiency of the univer-

sities technology transfer offices.

The Department of Nanotechnology at DTU, how-

ever, seemed to have developed a practical solu-

tion to circumvent some of the technical transfer

difficulties by having an employee with a relevant

technical and commercial background who partici-

pated in a quick and qualitative evaluation of ideas

and functioned as a liaison to potential industrial

partners.

The panel believes that technology transfer cannot

occur effectively through non-experts. This argues

for “embedded technology transfer” rather than a

centralised TTO, that is, where individuals with both

scientific insight and commercial experience are

embedded in the research departments or groups

from which technology transfer is to take place.

Overall, it is the perception of the panel that the

universities’ technology transfer offices have not

been effective in supporting (potential) innova-

tions in NABIIT-funded projects. Universities are

an important contributor to innovation, but should

not see intellectual property or simple services to

industry as a source of revenue. Moreover, because

of a combination of the breadth of technologies

covered, HR policies incompatible with market value

and subservience to university politics technology

transfer offices at universities are rarely efficient

“middle men” in the transfer and commercial ex-

ploitation of scientific research results.

The self-assessment and interviews also pointed to

other barriers to increased commercial exploitation

of results from NABIIT-funded projects. These bar-

riers comprised a lack of capital and/or incentives

for academics to pursue to commercial exploitation.

Other respondents explained that the market for po-

tential applications of their research were either too

immature or non-existent, or pointed to difficulties

in converting their research results to commercial

products. Finally, numerous respondents argued

that the research in their projects was too funda-

mental to be expected to yield commercial results in

the short run.

The panel stresses that outstanding science and

commercialisation often go hand in hand, but

universities should mainly focus on producing out-

standing science. Outstanding science should in

this connection also be seen as state-of-the art

31/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

engineering that may bring about both scientific

breakthroughs of the highest possible impact,

including patentable inventions, and projects well

suited for prototype-generating activities that are at

the border to commercialisation and well suited for

industrial collaborations.

8.2 Almost no contribution to public sector

innovation – but potential contributions to

solutions to societal problems

Conclusion:

There is very limited participation by

the panel that much of the research undertaken has

some degree of relevance for the public sector in

that it may contribute to new solutions to important

challenges to society. For example, academic re-

spondents in the self-assessment survey referred to

potential positive implications of their research for

public health, the environment or food quality and

safety.

In certain specific areas, for example within the field

of interaction of new nanomaterials with living mat-

ters, there is a risk of very undesirable interactions

taking place. In the assessment of the panel, these

types of issues also seemed to be adequately ad-

dressed in the projects under NABIIT.

public sector users in the projects. Nevertheless, many

projects have potential societal benefits, e.g. through

improvements in health care, food technology, ICT etc.

Recommendation:

There seems to be no reasons

for a change in the programme in order to better meet

societal needs, except to improve commercial impact

as per the previous recommendation.

Just two of the 36 projects funded by NABIIT had

direct involvement of potential public sector users

in the scientific and technological research per-

formed. Two additional projects counted university

hospitals among their participants. Aside from

these projects, public sector users played no role

in NABIIT projects. Moreover, a cursory review of

the titles of the 36 projects, for example, reveals

that only very few projects are dealing with issues

relating to medical research. The panel thus noted

that medicine as a field of research seems to be un-

derrepresented in NABIIT projects, as compared to

similar programmes in other countries.

A meaningful involvement of and connection with

the medical profession is central to creating com-

mercial results from this type of projects. The panel

has not observed many partnerships with medical

schools or hospitals in the NABIIT programme, and

would therefore like to encourage such collabora-

tion.

The underrepresentation of medicine in the NABIIT

programme is likely explained by a general lack

of a strong presence of medical research in disci-

pline-crossing research. Programmes such as NABI-

IT can promote greater interaction among medical

research and other areas of research, but only to

a certain effect. For the full potential of the Danish

research environment to be realised, the panel

believes that it would be a strong move to combine

strengths from existing research environments in

bioengineering at the Technical University of Den-

mark and biomedicine at the University of Copen-

hagen into a joint physical centre that can serve as

a national focal point for world class biomedical

engineering.

In spite of the lacking involvement of the public sec-

tor in the NABIIT programme, it is the assessment of

8.3 Limited contribution to private sector

innovation

Conclusion

: In most cases, company participation did

not seem to be aiming at solving important company

issues. Rather, the majority of company partners ap-

peared to participate in NABIIT projects to build funda-

mental insights and engage in generic, precompetitive

research.

Recommendation:

Commercial benefits are likely to

be improved by involving companies to a higher de-

gree in the projects and by focusing on issues core to

companies’ research and development activities.

As stated in section 8.1, NABIIT-funded projects

have, at the time of evaluation, generated very few

technical breakthroughs. It follows from this that the

projects have (at least thus far) made a very limited

contribution to innovation in the private sector. This

was further supported by the statements provided

in the self-assessment survey among company

participants. However, as mentioned in section 8.1,

it is important to keep in mind that the NABIIT pro-

gramme has only been running for five years, and it

is therefore not possible to make a reliable assess-

ment of the impact of the programme on commer-

cial innovation.

Nonetheless, most company respondents stated

that the projects they participated in either have not

yet or are not expected to yield commercial results.

Moreover, most of the company participants stress

that the main purpose of the projects, they have

contributed to, was to yield new knowledge, and

not necessarily to generate innovations or other

commercial outputs. Just one fourth of company

respondents indicated that their project have gen-

erated or are expected to generate results that can

be exploited commercially.

According to company participants that contributed

to the self-assessment survey, the main benefits of

participating in NABIIT projects are as follows:

— Better and deeper understanding of the existing

research areas

32/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

— Further development of key research areas

— Adoption of new methods, applicable to the

company’s business areas

— Obtained new technical skills or competences

— The project made it possible to test ideas, which

would not otherwise have been possible to test.

It is the experience of the panel that companies

prefer educated talents generated by universities

more than the knowledge or inventions that origi-

nates from projects such as those financed through

NABIIT. A key motivation for companies is thus to

participate in projects where they support educa-

tion. Hence, the results of commercialisation could

be seen in this light.

panies in the projects included both R&D intensive

global industry leaders and two-person start-up

companies whose living and survival were virtually

based on the funding received from the project.

Of the 36 projects funded by NABIIT, 28 projects

have had one or more company participants, while

6 involved collaboration with a technological service

institute (the so-called “GTS institutes”).

The panel found the fact that the role of the techno-

logical service

institutes are not always clear. They

are formally registered as non-public participants,

but occupy different functions.

The panel observed in the interviews that, in some

Nonetheless, based on the interviews with the se-

lected projects, it is the assessment of the panel

that much private sector participation, unfortunate-

ly, was only weakly linked to important business

issues and core R&D themes for the larger compa-

nies. Moreover, responses in some cases indicated

that the company participants were rather aiming

to meet personal research goals than addressing

important and prioritised company challenges.

projects, the technological service institutes ap-

peared to participate as extensions of the academic

research environment rather than as companies

and having a strong focus on the funding aspect,

resulting in rather artificial constructions. This

should be seen in view of the fact that these partici-

pants in the evaluation were presented as industrial

partners.

The evaluation panel therefore advises the Council

The panel is of the opinion that projects in their over-

all scope as well as in their contribution to commer-

cial successes would benefit from having a stronger

involvement from industry at management level and

being focused on issues and areas that are key to

industries.

8.4 Projects involve company participants – but

there is great variety in the types and degree of

involvement of these firms

Conclusion:

The type of company participation var-

for Strategic Research, when evaluating company

involvement in grant applications, to be clearer on

the diversity among types of company participants,

and that different types of participants lead to very

different types of projects and output of projects.

It is also important to evaluate the degree and na-

ture of participation from industry. Unfortunately,

as stated in section 8.3, large companies generally

did not appear to collaborate on core areas of their

activities. This can to some degree be explained by

IPR and competitiveness concerns. It is however the

panel’s opinion that the Council should do its best

to set up a supportive framework and stimulate

companies to collaborate on core activities in pro-

grammes such as NABIIT.

Generally speaking, according to the self-assess-

ment surveys conducted among academic and

company participants, users contribute to NABIIT

projects by providing feedback and participating

in scientific discussions, particularly in the initial

phases of the project. Many users take part in de-

fining goals and testing results or prototypes, in the

development phase of e.g. materials, or in the joint

ies greatly across the projects.

Recommendation:

More attention should be allo-

cated in the evaluation of applications concerning the

type and degree of company participation. For exam-

ple, if company participation is prioritised, care should

be taken to avoid pseudo-commercial participation,

e.g. where authorised technological service institutes

participate as de facto technical research partners.

Similarly, reviewers should pay particular attention to

the degree of involvement and commitment of com-

pany partners in projects.

The NABIIT programme, like the Council for Strategic

Research in general, has placed a great deal of em-

phasis on securing co-funding from private sector

participants. This is based on the idea that partners

who provide significant co-funding are more likely

to be committed to the project and to participate

actively in both its design and completion.

The evaluation indicated great variety in the types

of firms that participated in NABIIT projects. Com-

Technological service institutes are not-for-profit organisa-

tions that engage in applied research and offer technology

based services to firms on commercial terms (particularly to

small and medium sized enterprises). Approximately 10 percent

of the technological service institutes’ income is direct public

funding, aimed at stimulating the diffusion of the results of sci-

entific and technological research to a broad population of

firms. In addition, the institutes apply for public funding from

Danish and international sources.

33/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

scientific publications of results from the project.

Company participants are more likely than their

academic counterparts to be responsible for the

application or commercial exploitation of research

results, when this is relevant.

However, the surveys also indicated that while 90

percent of academic participants contribute to

core research activities in NABIIT-funded projects,

the same only holds true of 43 percent of company

participants. Indeed, some academic respondents

also experienced that users from industry were too

disconnected or too loosely affiliated with the pro-

ject to make a significant contribution to their joint

efforts.

Based on the inputs from the interviews and the

self-assessment surveys, the panel suggests that

the Council enables company participation to take

place in different steps. It is the assessment of the

panel that it

can

be difficult to combine state-of-

the-art research with company participation. In

order to promote stronger industry participation

and to ensure the application of research results,

the panel suggests that projects and centres should

be asked to consider and given the opportunity to

more strongly engage companies in the second

phase of the project if results are promising and

company participation has not been included at an

earlier stage in the project. This decision could be

based on the mid-way evaluation of the project. At

the same time it is the panel’s perception that there

may only be a few projects where companies have

an incentive to participate in projects at such a late

stage where they are not able to participate in the

definition of the research direction. Thus, the panel

suggest that most effort is allocated to providing

a supportive framework for company participation

from the outset of the projects, given that commer-

cialisation of research results continues to be a firm

objective of research programmes such as NABIIT.

34/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

Training

of young

researchers

The interviews with selected projects indicated that

collaboration across disciplines occurs to a large

extent through interaction between young research-

ers.

While senior scientists in the NABIIT-funded re-

search projects shaped the overall design of the

projects, the projects are to a large extent undertak-

en by the Ph.D. students and post.docs recruited for

the projects. This is in line with the overall focus on

researcher education and training in the Council for

Strategic Research.

As stated above, interviews suggested that much

of the communication and collaboration across dis-

ciplinary boundaries in NABIIT projects takes place

through interaction among young researchers. In-

terviews indicated that while these young research-

ers generally worked within their own fields, they

were regularly and to a significant extent engaged

in exchanges of research materials, discussions of

results and joint seminars with both junior and sen-

ior scientists from other fields.

9.1 A significant contribution to education and

training of researchers

Conclusion:

Young researchers have gained valuable

The young researchers interviewed by the eval-

uation panel generally indicated their belief that

the quality of their training had been strengthened

by the fact that they had participated in a NABI-

IT-funded project, precisely because this provided

them with access to and regular interaction with

researchers from other fields. Students appeared to

benefit tremendously from the interaction between

disciplines.

9.3 Young researchers were content – and

expressed limited ambition

Conclusion:

Young researchers in NABIIT-funded

education and training due to NABIIT.

Like the rest of the Council for Strategic Research,

the NABIIT programme placed particular emphasis

on the education of young researchers. Approxi-

mately 65 percent of the research funding grant-

ed by the programme commission behind NABIIT

is dedicated to Ph.D. and post.doc positions.

According to responses to the self-assessment

survey, in the 36 projects that were awarded a

grant from the NABIIT programme:

— 73 academic participants hired a total of 146

Ph.D. students and 97 post.docs (or, on aver-

age, 2 Ph.D. students and 1.3 post.docs) in

connection with their NABIIT-funded projects

— 35 company participants hired a total of 8

Ph.D. students and 7 post.docs

9.2 Young researchers gain from

multidisciplinary collaboration

Conclusion:

Multi- and interdisciplinary projects

projects seem to be very content with their positions

and their base at Danish universities. There was lit-

tle evidence of young researchers establishing and

demonstrating their intellectual independence, as

many of the respondents consistently researched

and published in collaboration with senior scientists,

notably their academic advisors. This should not be

seen, however, as a particular challenge for the NABIIT

programme, but as a general shortcoming of young re-

searchers in Denmark in the research fields addressed

by NABIIT.

Recommendation:

A research programme such as

are excellent tools for educating young scientists.

Young scientists operated primarily within their own

field, but appeared to gain significant insight and

networks into other, relevant research disciplines. In

the panel’s experience, these are valuable qualities

in preparing young scientists for future careers in

both academia and industry.

Recommendation:

The Council should continue

to have a strong focus on Ph.D. and post.doc edu-

cation.

NABIIT could enforce special efforts in the form of

smaller projects, aimed at boosting scientific inde-

pendence of young scientists aspiring for assistant pro-

fessorships. Independence could further be strength-

ened through more single-authored publications by

young scientists, which the panel highlights as a key

parameter in recruitment decisions in internationally

competitive research institutions. The programme

should be designed so as to become highly prestigious

with e.g. award-like structures.

35/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

Most of the young researchers that the panel met,

both those of Danish and non-Danish nationality,

were very content with their positions at the Dan-

ish universities. This was especially prominent

concerning research themes and affiliations, but

they expressed little interest in advancing e.g. to

management positions in academia or industry.

Similarly, they did not express interest in moving to

new research themes or taking up positions at re-

search institutions outside of Denmark. Long-term

ambitions were generally speaking very modest

and unclear.

The panel is aware that the observations made

above could to a certain degree be biased by the

fact that the young researchers that participated in

interviews had either stayed on at the institution af-

ter the end of the NABIIT-funded project or were still

involved with the project. The panel did not meet the

young researchers who moved to other places after

the completion of their participation in the NABIIT

projects in the interviews with young scientists.

The panel is concerned that there was so little

evidence of young researchers establishing and

demonstrating their intellectual independence,

particularly due to the lack of mobility among

young researchers in the NABIIT-funded projects.

Additionally, the panel believe this situation may be

influenced by a potential gap in the funding struc-

tures where post.docs tend to get their permanent

positions at universities by working closely together

with a more senior professor that is successful in

bringing in large grants – and not by demonstrating

independence.

In several of the research groups that the panel met,

there was a tradition of senior scientists consist-

ently co-authoring publications by junior research-

ers. To promote the development of intellectual

independence and long-term career development

among young researchers, this practice should

be discouraged. The panel believe this is part of a

general shortcoming in the Danish research fields

addressed by NABIIT, where the funding system

in reality prevents younger researchers from es-

tablishing their own independent career. Although

this challenge is therefore not specific to NABIIT,

programmes such as NABIIT can play a role in ad-

dressing the need for greater independence among

young researchers.

For example, the panel suggests that positive re-

inforcement could be provided through fellowship

programmes, e.g. from the Council for Strategic Re-

search that allow young researchers to establish re-

search activities and build new groups of their own

within strategic research areas. This could also help

to encourage attracting researchers from abroad

or from industry. The panel suggests that such a

fellowship programme could also have a special

focus on providing funding for young researchers

returning from academic positions abroad or from

positions in industry

For inspiration on special programmes for young

scientists with the potential to become professor,

the panel suggests to look into the Swedish Foun-

dation for Strategic Research and their programme

“Individual Grants for Future Research Leaders” that

has been successful in promoting young research-

ers.

The panel suggests that the Council should track

the career development of the young researchers

that participate in NABIIT-funded projects, e.g. in

comparison with students funded by other means,

with the aim of comparing how effective NABIIT is

for the development of careers relative to other

programmes. The aim of this would be to see if the

NABIIT programme has fostered scientists that

choose careers that differ from the average career

potentially funded by other available sources.

Concerning the origin of the Ph.D.s and post.docs

associated with the NABIIT-funded projects, find-

ings from the self-assessment survey indicate that

more than half of the academic Ph.D. and post.doc

positions were filled with persons recruited from for-

eign institutions. All company-hired Ph.D. students

were recruited from Danish institutions, but half the

post.docs were recruited from abroad.

In order to strengthen international orientation and

mobility among researchers, the panel suggests the

construction of an open post.doc programme linked

to existing projects with a common call and the an-

nouncement made known around the world, thus

stimulating top scientists to choose to join leading

groups in Denmark. It is the assessment of the pan-

el that this could lead to increased international

mobility and a strengthening of Danish research

environments.

9.4 A potential to increase identification with

the NABIIT programme

Conclusion:

Several young scientists did not know

that their research position was funded through the

NABIIT programme.

Recommendation:

The Council should take steps

to increase identification with the NABIIT programme

through joint activities comprising both team building

activities and courses with participation of young sci-

This raises certain challenges, e.g. as someone with a

background in industry may have a difficult time returning to

academia because of the lack of publications and other dimen-

sions of academic track records.

36/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

entists, both within and across the different projects

funded by the programme.

Several of the young researchers interviewed were

not aware (at least at the outset of their employ-

ment) that they had been funded by a NABIIT pro-

ject. This indicates that the research theme in the

project proposal itself does not represent a high

priority for the research groups concerning dissem-

ination of the function of the project in the overall

work of the research group in general and in relation

of the education of the young scientists in particular.

Based on the panel’s experience, identification

with research programmes by young researchers

increases the likelihood that the aims of the pro-

gramme will be kept in mind and therefore met. The

panel thus suggest to increase identification with

the NABIIT programme by including teambuilding

activities for all project participants (especially

young researchers), particularly at the start of a

project. This could be supplemented by including

common activities such as courses on project man-

agement, workshops, international trips etc. Cours-

es on project management would not only strength-

en the association with the NABIIT programme but

also provide the young researchers with specific

tools that will assist them in future research ven-

tures. During the interviews, the panel was made

aware by young scientists that courses on project

management were requested.

Moreover, the Council for Strategic Research could

host networking events for young researchers

across projects funded by the Council. This could

also promote greater cross-fertilization of ideas

across research activities funded by the Council.

Additionally, networking activities could help stim-

ulate interdisciplinary interaction within the NABIIT

programme, and potentially across research pro-

grammes.

9.5 Young researchers must be trained for

positions both in academia and industry

Conclusion:

Most young researchers will leave ac-

Recommendation:

The Council could initiate cours-

es or other joint activities for young scientists that

are part of a NABIIT-funded project in order to train

their mind-set to be fit for both academia and the

industry. Moreover, a greater degree of involvement

of company partners in projects, as recommended

above, would also contribute to the training of young

scientists associated with NABIIT-funded projects in

priorities and approaches to work in industry.

A key priority and goal of the Council for Strategic

Research is to contribute to postgraduate education

and researcher training. During the interviews, the

panel observed that this task was taken very seri-

ously by all projects and, as indicated in the previ-

ous sections, the panel assesses that the training of

young researchers appears to be very well function-

ing. However, during the interviews, the panel also

observed that their training and education primarily

concern academic disciplines tailored for a career

in academia. The panel also saw examples of the

opposite where professors were assisting Ph.D.s

and postdocs in becoming aware of the possibilities

within the competences and know-how they ac-

quired during their education in the NABIIT-funded

project and the wider application options. Addition-

ally, the panel observed that some, but few, profes-

sors were very direct in providing career advice for

Ph.D. students when terminating their position as

a Ph.D. This was mainly related to whether or not

they were suited for a career in academia, but also

served the purpose of promoting a career in the

industry.

The panel suggests that the orientation within fu-

ture career opportunities should be strengthened

among young researchers. This should be done by

raising the awareness among young researchers on

the applicability of their research education.

Subsequently, there is a greater need for training a

broader skill set among young researchers that will

allow for a more smooth transition from academia

to industry after termination of their research posi-

tion. The panel thus suggests that special courses

for Ph.D.s and post.docs in their first years should

be included as part of the NABIIT programme.

The courses could cover themes such as project

management, R&D portfolio management, career

planning, innovation and application of research

results etc. The courses should be designed in such

a way that it has relevance for both academia and

industry. The panel suggest that the courses should

be run by a panel of principal investigators from the

funded projects in order to ensure the quality and

the relevance of the course.

ademia to take up positions in the public or private

sector. The projects interviewed by the panel, however,

appear to predominantly train their young researchers

for a career in academia. Again, this conclusion may

be biased by the fact that the panel met exclusively

with young scientists who are still at the institution,

where they undertook their NABIIT-funded Ph.D. or

post.doc fellowship.

This conclusion may be influenced by the fact that the

young researchers interviewed had either stayed on at the

institution after the end of the NABIIT-funded project or were still

involved with the project.

Another suggestion by the panel concerning young

researchers’ training and integration across aca-

37/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

demia and industry concerns a public-private post.

doc programme. This suggestion is inspired by both

the Novartis programme and The Danish National

Advanced Technology Foundation’s industrial post.

doc programme. The purpose of such a programme

is that it allows post.docs to work closely with both

academia and industry and thus keep an option

open in both sectors. The programme should be

based within the Council. It should be announced

as a common programme for the projects that are

funded as prestigious post.doc positions where

the likelihood of getting international applicants is

present. The panel assesses that this would provide

an important tool for programmes such as NABIIT in

order to ensure the bridge between academia and

industry for young researchers.

38/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

Development

of Danish research

environments

ing for multi- or interdisciplinary projects, particu-

larly when these are large-scale projects and when

company participants are involved.

Figure 10.1. Is the project within

a new research area or related to

established research areas?

Academic participants

49%

51%

10.1 Impact on participants’ research activities

and competences

Conclusion:

Based on insights gained from the

Company participants

self-assessment survey and interviews, grants from

NABIIT have opened up new research fields or estab-

lish new research groups.

47%

Recommendation:

Research programmes like NABI-

53%

IT are highly relevant. The efficiency could be further

improved by bringing the collaborating parties closer

together.

Insights from the self-assessment show that for both

academic and company participants; about half of the

projects are based on new research ventures, while

the remaining projects represent a continuation or ex-

tension of key existing research activities in the partici-

pating organisations. This is depicted in figure 10.1.

Both academic and company respondents in the

self-assessment survey were asked whether they

would have sought funding elsewhere in the ab-

sence of funding from NABIIT. 15 and 13 percent of

academic and company participants, respectively,

would have sought funding for the same project

elsewhere. Half of all participants would have

sought alternative funding for a similar or related

project. 40 percent of academic and 37 percent of

company participants would have abandoned their

project had it not been for the funding through NA-

BIIT. This indicates that NABIIT funding has played

a significant role, enabling projects that might not

otherwise have been undertaken.

In fact, academic respondents in the survey stated

there are no or few other adequate sources of fund-

The self-assessment survey among academic

respondents indicates that scientists assess that

there is an impact on the research activities and

competences when participating in a NABIIT-fund-

ed project. In particular, they highlight the follow-

ing themes as the main benefits to the research

activities:

— Enabled expansion into a new research field

— Strengthened research capacity and/or core

competences (e.g. application of new theo-

retical areas and methods; consolidation of

existing research activities etc.)

— Stronger international profile and greater visi-

bility in the academic community

— Improved ability to engage in interdisciplinary

research and/or combinations of technolo-

gies

New research area for the organisation

Closely related to key, established research areas and core

competences in the organization

Source: DAMVAD 2012 based on the self-assessment survey.

N (academic participants) = 76, N (company participants) = 38

39/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

In the interviews, the scientists participating in the

projects explained that NABIIT has played a central

role in the long-term development of critical mass

at the research centre. In this connection, the Na-

noScience Centre at the University of Copenhagen

and iNANO at the University of Aarhus was men-

tioned. DTU has succeeded in building up a strong

engineering environment at DTU Nanotech.

The panel recognises the impact that the scientists

express as a result of the participation in the NABI-

IT-funded project. However, the panel wonders if

the funding for the projects could have been spend

more efficiently if research groups to a larger ex-

tent were able to combine their core competences

within the individual projects. This could be in rela-

tion to application and innovation in the biomedical

sphere that requires combining competences in

fundamental scientific research, (e.g. from Co-

penhagen University) with engineering (e.g. from

DTU).

The panel further suggests that such competences

should be brought together in a physical unit which

particularly can pose a challenge when they span

university boundaries.

10.2 Supplementary/subsequent funding of

research activities

Conclusion:

NABIIT-funded projects have given rise

Figure 10.2. Spin-off projects

51%

49%

Yes - the project has resulted in spin-off projects requiring

additional external funding

No - the project has not resulted in spin-off projects requiring

additional external funding

Source: DAMVAD 2012 based on the self-assessment survey.

N (academic participants) = 77

According to the panel, follow-on funding for a few

very successful and relevant NABIIT projects once

they are completed, i.e. a renewal process, is de-

sirable. As proposed in another section, this could

provide a very competitive environment for qualify-

ing centres or more focused larger-scale projects

where the potential commercial or societal benefits

have also been documented to be high.

International grants

Individuals associated with the projects that the

panel met with seemed to be very successful in at-

tracting grants from the European Research Council;

this is a strong indication of the quality of the re-

search groups.

Since these projects involve several institutions and

companies, it is important to be careful in setting

up too many requirements for the legal agreements

with international partners. How international part-

ners should be brought in, should be up to the na-

ture of the project and the contributions to be made

by the international partner(s).

10.3 Characteristics and outcomes of

collaboration

Conclusion:

It is the impression of the panel that the

to a number of follow-on projects and activities.

Recommendation:

The Council should give priority

to stimulating and supporting spin-off projects from

NABIIT projects, as this is likely to increase the ulti-

mate value created by the initial grant. In the experi-

ence of the panel, and based on the insights gained

from particularly the self-assessment survey and the

interviews, the Council could achieve this for exam-

ple by giving projects the possibility of applying for

a follow-up grant to pursue novel research projects

that spring from the original project.

NABIIT-funded projects have given rise to a number

of follow-on projects and activities. For example,

as illustrated in figure 10.2, 49 percent of academ-

ic respondents have developed spin-off projects as

a result of their NABIIT-funded activities.

overall level of interaction in the projects, national as

well as international, is acceptable and similar to what

is found in comparable programmes in other coun-

tries. The panel also observed a high degree of varia-

tion in the level of and approaches to collaboration

among project partners.

In-depth analysis of the projects interviewed indicated

that there is an unrealised potential to strengthen

the degree of collaboration among participants in

40/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

projects. For example, there were few projects which

involved co-authorship among several/all of the

participating research groups. In the experience of

the panel, this indicates that such collaboration was

lacking; this conclusion was moreover substantiated in

several interviews.

Recommendation:

The panel recommends that

Both academic and company respondents bene-

fited from collaboration in the NABIIT-funded pro-

jects. Notably, participants improved their network

to research institutions in Denmark or abroad, and

strengthened their ability and motivation to partici-

pate in interdisciplinary research collaboration.

Projects are employing a combination of informal

and formal approaches to research and collabora-

tion management, e.g. frequent meetings, estab-

lishing of a steering group, or joint attendance of

conferences.

projects become more aware of the need and benefits

from close collaboration when conducting true inter-

disciplinary research

According to responses in the self-assessment

survey, key channels for interaction among pro-

ject participants include joint research activities,

informal exchanges of information and ideas, and

provision of access to research tools and infrastruc-

ture. Academic participants in NABIIT projects also

collaborate through e.g. co-authorship of scientific

publications and training of young researchers. In

comparison, company participants often collabo-

rate by providing access to insights or data in indus-

try (cf. figure 10.3).

During the interviews and based on the publications

from the projects, the panel noticed that with some

notable exceptions, there were few projects where

key publications involved all project partners.

Of the nine projects interviewed, most were not

heavily dependent on close and daily interaction

among project participants, although there were

also exceptions to this. Had the projects had a high-

er degree of interdisciplinarity, they would have re-

quired a higher level of close and daily interaction.

Responses to the self-assessment survey from

academic scientists provided by the scientists indi-

cate that there are numerous different approaches

to contact, collaboration and communication in

NABIIT-funded projects.

Most scientists indicated that they met on a regu-

lar basis, supplemented by larger meetings a few

times a year, and with occasional ad hoc meetings

where all participants were present. The second

most common model of interaction was bi-annual

meetings supplemented by frequent communi-

cation through e-mail or phone. A smaller group

of scientists engaged in frequent meetings (e.g.

every month) as their means of interacting with

partners.

A few scientists indicated that Ph.D. and post.docs

lead the day-to-day interaction and collaboration

in connection with the project, whereas other par-

ticipants met less frequently.

Figure 10.3. Types of interaction with partners in the project

Informal exchanges of information, ideas and the like

Joint research activities

Co-publication of scientific publications

Joint postgraduate education and researcher training

Access to research tools and infrastructure

Access to insights, data or problems in industry or in the public sector

Joint academic conferences

Joint research positions/exchange of employees

Joint dissemination activities; non-academic audiences

Joint patenting

20%

40%

60%

80%

100%

48%

27%

36%

16%

32%

27%

56%

78%

91%

86%

88%

84%

49%

74%

41%

74%

70%

Academic participants

Company participants

Source: DAMVAD 2012.N = (academic participants) = 77, N (company participants) = 38, Multiple answers were possible.

41/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

International collaboration

12 international participants (including both aca-

demic research groups and companies) were for-

mally involved in the 36 projects funded by NABIIT.

According to the survey responses provided by ten

of these participants, their role in the projects was

primarily related to participating in core research

activities (e.g. performing tests), development of

new methods or models, and optimisation, fabrica-

tion, and delivery of specific materials. A few inter-

national participants also acted as Ph.D. supervi-

sors, while others mentioned that they participated

in the preparation of joint articles and presentations

with other project partners.

All ten international participants described the ac-

ademic level in the project that they participated in,

as “very good”, “top level”, “of high international

standard” or “the leading edge of international re-

search in the field”.

10.5 Acquisition and efficient use of research

infrastructure

Conclusion:

There were examples of investments in

expensive hardware and research infrastructure that

seemed to be clearly influenced by funding of projects

within the NABIIT programme.

Recommendation:

The Council should take steps to

ensure that future programmes through highly appli-

cation specific objectives stimulate the integration of

other disciplines at the centres where large infrastruc-

ture investments in specific areas have been made.

The self-assessment surveys did not provide a clear

picture of NABIIT’s contribution to establishing new

infrastructures for work within the defined focus

areas. Such infrastructure can be relevant in order

to conduct competitive research and to attract the

best researchers to the same physical environment.

The panel saw clear indications that the NABITT

10.4 Researcher mobility

Conclusion:

There are obstacles in Denmark, as in

programme had contributed to the establishment

of new infrastructure at three universities, namely

University of Copenhagen, Technical University of

Denmark and University of Aarhus. It has not been

possible for the panel to determine the exact extent

to which NABIIT funding has contributed to the infra-

structure.

It is the opinion of the panel that care must be taken

in future programmes that other fields, like IT and/

or biotechnology are stimulated to be integrated in

such centres in order to promote true interdiscipli-

nary research.

several other countries, for mobility between academ-

ia and the public or private sector.

The analysis of researcher mobility undertaken in

connection with this evaluation showed that among

the total of 393 individual participants in NABIIT

projects that constituted the base population in

the analysis, just 58 individuals (15 percent) immi-

grated to Denmark and 31 individuals (8 percent)

emigrated from Denmark in the period from two

months prior to the start of the project period to two

months after the end of the project period (though

not beyond the end of 2011). Of the 58 individuals

who immigrated, 16 subsequently emigrated within

the survey period.

The highest level of international mobility, whether

it be immigration to or emigration from Denmark,

is found among project participants whose highest

appointment level during the project period was

assistant professor/post.doc: among these individ-

uals, 29 percent had immigrated while 17 percent

had emigrated. The second-highest proportion

is seen among Ph.D. students. In this group, 22

percent had immigrated while 16 percent had em-

igrated. At the professor and associate professor

level, international mobility (both immigration and

emigration) was insignificant.

Finally, it is interesting to note that mobility from

abroad to Denmark among NABIIT project partici-

pants in the university sector was lower overall than

at Danish universities in general. However, among

project participants at both Ph.D. and assistant

professor/post.doc level, the percentage that immi-

grated to Denmark from abroad matches the corre-

sponding percentage at all Danish universities.

42/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

Appendices

Honours include:

Fellowship of the Royal Society (FRS) (2010), the

IOP (Institute of Physics) Mott Prize (2008), the

APS Oliver Buckley Prize (2005), the IUPAP Mag-

netism Prize/Neel Medal (2003), Riso National

Laboratory Fellow (2002), Royal Society Wolfson

Research Merit Award (2002), Mildner Lecturer,

Department of Electronic & Electrical Engineering,

UCL (2002), Fellow of the American Physical So-

ciety (1997), Fellow of the Japan Society for the

Promotion of Science (1996).

Fredrik Höök

Professor in Biological Physics, Chalmers University

of Technology.

He holds a Ph.D. in Physics, and a M.Sc. in Engi-

neering Physics, 1992

11.1 The evaluation panel

The evaluation panel behind this evaluation is:

— Søren Isaksen (chairman), Group Executive Di-

rector, CTO, NKT

— Gabriel Aeppli, Professor and Director of the

London Centre for Nanotechnology

— Fredrik Höök, Professor in Biological Physics,

Chalmers University of Technology.

This section contains a short presentation of the

three panel members.

Søren Isaksen

Group Executive Director, CTO, NKT

He holds a Ph.D and MSc in Physics and Maths

Prior to his current position, he was a Professor of

Physics (Nanoscience for Biophysics) at Lund Uni-

versity, Sweden and Assistant Professor of Phys-

ics/Biological Physics at Chalmers and University

of Gothenburg, Sweden.

Distinctions, scholarships and awards:

— Winner of AkzoNobel’s Nordic Research award

2002

— Awarded the SSF INGVAR (Individual Grant for

the Avancement of Research Leaders) grant

2006

— Nominated (through Q-Sense AB) to Stora

teknikpriset, Sweden, 2007

Additionally, Fredrik is one of the co-founders (two

Other positions include:

— Chairman of the Board of Directors of the Pho-

tonic Group companies.

— Board member of NKT Cables.

— Member of the Confederation of Danish Indus-

try’s Committee on Energy and Climate Policy.

— Member of the Tuborg Foundation Board.

Additionally, Søren Isaksen has held a number of

research-oriented honorary positions, including

chairmanship of the Danish Research Council, the

Danish Technical Research Council, the Technology

Foresight on Danish Nanoscience and Nanotechnol-

ogy, as well as the Comittee Responsible for Assess-

ing the Danish Research Council System.

Gabriel Aeppli

Professor and Director of the London Centre for

Nanotechnology

He holds a B.Sc. in Mathematics and PhD, M.Sc. &

B.Sc in Electrical Engineering from MIT.

Prior to his position at London Centre for Nanotech-

nology, he was a Senior Research Scientist for NEC

(Princeton), a Distinguished Member of Technical

Staff at Bell Laboratories, a Research Assistant at

MIT, and an industrial co-op student at IBM.

patents) of Q-Sense AB (www.q-sense.com),

who has commercialized the QCM-D technology

(2 pending granted patents) and the key inventor

(one granted patent and three patent applica-

tions) behind LayerLab AB (www.layerlab.se)

which was initiated together with Chalmers School

of Entrepreneurship in spring 2003.

11.2 List of interview respondents

This section presents the nine projects that were

selected to be interviewed by the panel and the

different respondents from each project.

GRANT HOLDER AFFILIATED WITH THE

UNIVERSITY OF COPENHAGEN

Arrays of Nanoscopic Biosensors on Surfaces

Grant size and period:

11.9 million kr.; 2005-2011

Grant holder:

Professor Thomas Bjørnholm, University

of Copenhagen (note: Thomas Bjørnholm was the

grant holder, while Dimitrios Stamou administered the

grant on a daily basis.)

Partners

: University of Copen-

hagen, IBM Zürich, Semasopht, Sophion BioScience,

7TM Pharma, Radiometer, AQUAPorin , Novozymes

Interview respondents:

— Senior researchers: Prof. Dimitrios Stamou, As-

soc. Prof. Karen Martinez.

43/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

— Junior researchers: Nicky Ehrlich, Lars Iversen,

Nikos Hatzakis, Andreas Lauge Christensen,

Christina Lohr, Achebe Nzulumike, Vadym Tkach.

Centre for Antimicrobial Research CAR

Metalloprotease sensitive drug delivery systems

for treating cancer and inflammatory diseases

Grant size and period:

7.8 million kr.; 2007-2012.

Grant holder:

Senior Researchers Thomas L. An-

dresen, Technical University of Denmark

Grant size and period:

27.5 miliion kr.; 2008-2014

Grant holder:

Professor Michael Givskov, University

of Copenhagen

Partners

: Technical University of Denmark, Bioneer

Interview respondents:

— Senior researchers: Assoc. Prof. Thomas An-

dresen, Prof. Rolf Berg

— Junior researchers: Postdoc Jonas Henriksen,

Ph.D. Rasmus Jølck

— Company respondents: Simon Jensen (Bioneer)

GRANT HOLDER AFFILIATED WITH AARHUS

UNIVERSITY

Computational models and tools for drug

discovery (COMODO)

Partners

: University of Copenhagen, Technical Uni-

versity of Denmark, Universität Zürich, Teknologisk

Institut, LEO Pharma.

Interview respondents:

— Prof. and grant holder Michael Givskov.

— Senior researchers: Thomas E. Nielsen, Tim

Tolker Nielsen

— Junior researchers:

— Company respondents: Anne-Lise Høg Lejre and

Anna Svensson

Centre for Pharmaceutical Nanotechnology and

Nanotoxicology

Grant size and period:

7.8 million kr.; 2006-2011

Grant holder:

Grant holder: Associate Professor

Christian Nørgaard Storm Pedersen, Aarhus University

Grant size and period:

28 million kr.; 2008-2014

Grant holder:

Professor Seyed Moein Moghimi, Uni-

versity of Copenhagen

Partners

: Aarhus University, University of Copenha-

gen, Molegro, Nuevolution

Interview respondents:

— Senior researchers: Assoc. prof. and grant

holder Christian Nørgaard Storm Pedersen, Prof.

Birgit Schiøtt, Prof. Brian Vinter

— Junior researchers: Postdoc Mette Alstrup Lei,

Ph.D. Rune Friborg Møller

— Company respondents: Rene Thomsen and Mi-

kael Hvidtfeldt Christensen (Molegro)

Partners

: University of Copenhagen; Technical Uni-

versity of Denmark; H. Lundbeck; Nordic Vaccine

Technology; LiPlasome Pharma

Interview respondents:

— Prof. Steffen Loft; Prof. Dimitrios Stamou; Assoc.

Prof. Thomas L. Andresen

GRANT HOLDER AFFILIATED WITH THE TECHNI-

CAL UNIVERSITY OF DENMARK

A Nanotechnological Approach to Studying

Interactions of Biological Macromolecules - An

interdisciplinary project

Development of new metal-oxide and -sulphide

catalysts

Grant size and period:

8 million kr.; 2006-2012

Grant holder:

Professor Flemming Besenbacher,

Aarhus University

Grant size and period:

6.7 million kr.; 2005-2011

Grant holder:

Professor Jörg P. Kutter, Technical Uni-

versity of Denmark

Partners

: Aarhus University, Haldor Topsøe, Image

Metrolology, SCF Technology

Interview respondents:

— Senior researchers: Prof. and grant holder Flem-

ming Besenbacher, Assoc. Prof. Jeppe Laurit-

sen, Peter Thostrup

— Junior researchers: Postdoc Stefan Wendt

— Company respondents: Poul Hansen and Stig

Helveg (Haldor Topsøe), Jan Friis Jorgensen

(Image Metrology)

Nano- and Bio-functionalised Surfaces for

Biofilm Prevention

Partners

: Technical University of Denmark, Universti-

ty of Copenhagen, Novo Nordisk

Interview respondents:

— Senior researchers: Prof. Jörg P. Kutter

Nano-technology for ultra high-speed optical

communications (Nano-Com)

Grant size and period:

5.9 million kr.; 2006-2011

Grant holder:

Professor Palle Jeppesen, Technical

University of Denmark

Partners

: Technical University of Denmark, OFS Fitel

Denmark

Grant size and period:

8 million kr.; 2007-2012

Grant holder:

Professor Niels Peter Revsbech, Aar-

hus University

Interview respondents.

— Senior researchers: Prof. Leif Oxenløven

— Junior researchers: Postdoc Michael Gallili

— Company respondents: Lars Grüner-Nielsen

(OFS)

Partners

: Aarhus University, Teknologisk Institut

In-

terview respondents:

— Senior researchers: Prof. and grant holder Niels

Peter Revsbech, Assoc. Prof. Rikke Louise Meyer

— Company respondents: Helmer Søhoel and Allan

Poulsen (Danish Technological Institute), Jonas

Agenhammer (Alfa Laval)

44/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

11.3 Overview of the evaluation process

As stated in the terms of reference, the purpose

of the evaluation was to assess if

the objectives of

—

Inputs

refer to the resources that have been in-

vested in NABIIT-supported activities by NABIIT

but also from other sources. Thus, inputs include

NABIIT grants and additional funding (either

provided by the participants or by other external

sources of financing).

NABIIT

have been fulfilled and to assess the extent

to which the programme has contributed to fulfilling

the general objectives for strategic research, a

s speci-

fied in by the Danish Council for Strategic Research.

2012.

In order to fulfil this purpose, the evaluation of NABI-

IT has been based on an overall evaluation design.

Figure 11.1 provides an overview of key elements in

the evaluation design, i.e. the areas that the evalua-

tion method collected data on:

—

Activities

refer to the actual behaviour and activ-

ities that occur and create value in the projects

supported by NABIIT. This includes understand-

ing how project participants interact and collab-

orate and how they approach the combination

of technologies.

—

Outputs

refer to the measurable results of in-

vestments under the NABIIT. It includes concrete

results from projects funded by NABIIT (e.g. new

knowledge, scientific publications, doctoral

graduates, etc.).

The evaluation consisted of six different methodo-

logical elements each resulting in individual sub-re-

port. Elements from these sub-reports have been

included in this evaluation report.

Methodological elements in the evaluation

Desk study of aims

and activities

Self-assessment

survey among project

participants

—

Policy rationale for NABIIT and overall goals for

strategic research.

This includes the underlying

rationale and political motivation for the allo-

cation of funds to NABIIT and the establishment

of the programme; it also includes the overall

goals for strategic research that NABIIT should

contribute to, but is not evaluated upon.

— Thus, these are the overriding intentions and

goals that the programme is expected to live up

to in order to catalyse the intended outcomes

and effects.

—

Immediate outcomes and long-term effects of the

programme

. These are the short-term and long-

term effects of the activities funded by NABIIT,

as described previously in this chapter. These

effects can only to a very limited extent be as-

sessed at this point in time.

Interviews with

selected projects

Analysis of researcher

mobility

—

Objectives of NABIIT.

These are strategic and

operational objectives of the programme; its

results (i.e. outputs and immediate outcomes)

must be evaluated against these objectives.

Bibliometric analysis

of publications

Case studies of

selected projects

and profiles

—

Design and implementation

refer to how the pro-

gramme is organised and the intervention logic

that it is based on. This is for example reflected

in the selection and use of policy instruments

and in the criteria and focus areas described in

calls for applications.

According to the terms of reference for this evalu-

ation, case studies of two NABIIT-funded projects

were to be undertaken with the aim of communicat-

ing examples of projects that promote public sector

innovation, an important topic in Danish innovation

policy. Additionally, the terms of reference also re-

Figure 11.1. Key elements in the evaluation

POLICY RATIONALE BEHIND NABIIT

AND OVERALL GOALS FOR

STRATEGIC RESEARCH

IMMEDIATE OUTCOMES AND

LONG-TERM EFFECTS

OF THE PROGRAMME

OBJECTIVES

OF NABIIT

DESIGN AND

IMPLEMENTATION

Instruments, calls

INPUTS

Grants and

additional

financing

ACTIVITIES

Collaboration

and technology

combinations

OUTPUTS

Project results

Source: DAMVAD 2012. Inspired by DG Budget. 2004. Evaluating EU Activities: A practical guide for the Commission services

45/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

quested the development of five “profiles” delving

into selected themes in the NABIIT programme.

Because the aim of both the case studies and the

profiles was to highlight and communicate selected

themes in the NABIIT programme, they do not form

part of the documentation for the evaluation. The

evaluation panel has not in any way been involved

in these activities, which have been undertaken by

DAMVAD.

The following briefly describes the different ele-

ments in the evaluation.

research departments or centres at Danish uni-

versities. This group of respondents also includ-

ed two university hospitals in Denmark.

— A shorter self-assessment questionnaire was

sent to company participants in companies in

Denmark (including authorised technological

services institutes, also known as GTS insti-

tutes).

— A short list of questions was sent to international

participants including both academic and com-

pany participants.

Table 11.1 presents the number of invited partic-

Desk study of aims and activities

The desk study comprised three elements:

—

Review of background documents

that de-

scribe the policy rationale behind NABIIT as well

as the stated objectives of the programme.

—

Analysis of calls for applications

that shed

light on the requirements that applicants to

NABIIT had to fulfil. It also identified themes and

criteria that were especially emphasized in calls

(for example doctoral training or collaboration

with industry).

—

Analysis of the project

portfolio that provided

an overview of the contents of the programme’s

project portfolio.

The results of this study are described in the sepa-

rate report,

Evaluation of the strategic research pro-

ipants, how many completed the survey and the

response rate. The total response rate was 81 per-

cent.

The results of this study are described in the sep-

arate report,

Evaluation of the strategic research

programme NABIIT - Report no. 2: Self- assessment

by project participants.

This report also describes

the results of an analysis of additional funding ap-

plications from and grants to participants in NABIIT

funded projects.

Interviews with selected projects

Nine projects were selected for interviews by the

panel. These included the two strategic research

centres that have been financed by NABIIT and

seven selected interviews distributed across grant

holders at three Danish universities: University of

Copenhagen, the Technical University of Denmark,

and University of Aarhus. The projects were select-

ed based on their distribution across grant holders

at three Danish universities: University of Copenha-

gen, the Technical University of Denmark, and Uni-

versity of Aarhus. Additionally, the selection criteria

comprised the size of the grant, whether the pro-

jects were completed (with emphasis on terminated

projects), the degree of public-private collaboration

and representation of both SMEs and MNEs among

the company representatives.

For each project, separate interviews were conduct-

ed (on site at the universities) with multiple partic-

ipants, including the grant holder and other senior

scientists, young researchers (Ph.D.s, post.docs.)

gramme NABIIT - Report no. 1: A desk study of aims,

instruments and grants under NABIIT.

Self-assessment survey among project

participants

The overall purpose of the survey was to collect

data on activities and results in the funded projects.

The survey was based on the evaluation themes.

The survey was undertaken by DAMVAD, but the

evaluation panel played an instrumental role in the

development of the self-assessment survey ques-

tionnaire

The self-assessment survey was distributed to all

project partners through three separate surveys:

— An extended self-assessment was conducted

among academic participants, i.e. university

Table 11.1. Response rate on the self-assessment survey

Survey

Academic participants *

Company participants **

International participants

Total

Source: DAMVAD 2012. * Includes two university hospitals. ** Includes eight GTS institutes.

No. of

invitations

154

No. of

respondents

125

Response

rate

80%

83%

81%

46/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

and, whenever possible, company participants. See

the interview programme in the Appendix.

The results of this study are reported in this evalua-

tion report.

Analysis of researcher mobility

This study was undertaken by the Ministry of Sci-

ence, Innovation and Higher Education. The analysis

carried out an outline statistical impression of the

volume, composition and vertical and horizontal

mobility of NABIIT project participants, based on a

registry analysis. The analysis comprised just under

400 individuals, who at one time or another partici-

pated in the NABIIT projects.

The results of this study are described in the sepa-

rate report

Evaluation of the strategic research pro-

Bibliometric data like publication source, author

affiliation, citations and Journal Impact Factor (JIF)

scores were obtained through individual searches

for each publication in the leading international bib-

liometric database Web of Science from Thom-son

Reuters.

Assessments of the scientific quality of publications

were based on the Danish authority list developed.

The publications were analysed separately for com-

pleted and ongoing projects along the dimensions

described below.

(1) Scientific productivity

(2) Scientific quality

(3) Scientific impact

(4) Interdisciplinarity

(5) National co-publication.

(6) Internationalisation of research

The results of this study are described in the sepa-

rate report,

Evaluation of the strategic research pro-

gramme NABIIT - Report no. 4: Analysis of researcher

mobility.

Bibliometric analysis of publications

The bibliometric analysis was based on publication

lists provided by grant holders from each of the 36

grants.

gramme NABIIT - Report no. 3: A bibliometric analysis

of publications from NABIIT projects.

11.4 Overview of the 36 projects

Grant holder*

Professor Flemming Besenbacher, iNANO, Aarhus

University

Professor John Erland Østergaard, University of Southern

Denmark

Senior researcher Jørgen Schou; RISØ, Technical

University of Denmark

Professor Thomas Bjørnholm, Nano-science Center,

University of Copenhagen

Professor Anja Boisen, Department of micro- and

nanotechnology, Technical University of Denmark

Senior researcher Søren Højsgaard, Faculty of

Agricultural sciences, Aarhus University

Professor Jörg P. Kutter, MIC, Technical University of

Denmark

Professor Jens Ulstrup,

Faculty of chemistry, Technical University of Denmark

Professor Anders Stærmose Krogh,

Bioinformatics Center, University of Copenhagen

Lektor Marleen de Bruijne,

Department of computer sciences, University of

Copenhagen

Professor Herman Autrup,

Aarhus University

Professor Flemming Besenbacher, iNANO, Aarhus

University

Associate Professor , Troels Andreasen, Department of

Communication, Business and Information Technologies,

Roskilde University

Professor Henning Christiansen,

Department of Communication, Business and Information

Technologies, Roskilde University

Professor Ivan Bjerre Damgård,

Department of computer science, Aarhus University

Project title

Bioimaging in nanoparticles

Plasmonic optical chips for medical diagnostics and

chemical sensing

Wear control of tribological hard coatings with embedded

optical nanolayers

Arrays of Nanoscopic Biosensors on Surfaces

Nano Systems Engineering: NanoNose

Automated monitoring of health and welfare of dairy cows

in loose housing systems

A nanotechnological Approach to Studying Interactions of

Biological Macromolecules - An interdisciplinary project

Nanoscale Investigations of Biological Surfaces and Biofilm

by Scanning Probe Microscopy

Simulating proteins on a millisecond time-scale

Computer-Aided Assessment of COPD from CT Images

SUNANO - Risk assessment of free nanoparticlesl

Development of new metal-oxide and- sulphide catalysts

SIABO - Semanic Information Access through Biomedical

Ontologies

Logic-statistic modelling and analyses of biological

sequence data

Mobile Quantum Security

47/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

Professor Jakob E. Bardram, IT University of Copenhagen

Professor Robert Madsen,

Faculty of chemistry, Technical University of Denmark

Professor Palle Jeppesen,

TU Fotonik, Department of Photonics Engineering,

Technical University of Denmark

Professor Eva Bjørn Vedel Jensen,

Department of mathematical sciences, Aarhus University

Associate Professor, Christian Nørgaard Storm Pedersen,

Department of computer science, Aarhus University

Senior researcher, David Plackett,

Danish Polymer Center, Risø, Technical University of

Denmark

Professor Bjørk Hammer, Department of physics and

astronomy, Aarhus University

Professor Niels Peter Revsbech, Department of biology,

Aarhus University

Professor Knud Jørgen Jensen, Department of basic

sciences and environment, University of Copenhagen

Professor Bodil Jørgensen, Faculty of science, University

of Copenhagen

Associate Professor, Kristian Thygesen, Department of

physics, Technical University of Denmark

Professor Jan Halborg Jensen, Department of chemistry,

University of Copenhagen

Professor Anja Boisen, Department of micro- and

nanotechnology, Technical University of Denmark

Professor Anders Kristensen, Department of micro- and

nanotechnology, Technical University of Denmark

Associate Professor Thomas L. Andresen, Danish

Polymer Center, Risø, Technical University of Denmark

Professor André Chwalibog, Department of Basic Animal

and Veterinary sciences, University of Copenhagen

Professor Michael Givskov, Department of Internantional

Health, Immunology and Micro biology, University of

Copenhagen

Associate Professor, Jesper Nygård, NanoScience

Center, University of Copenhagen

Professor Kurt Vesterager Gothelf. Department of

chemistry, Aarhus University

Professor Seyed Moein Moghimi, Faculty of

Pharmaceutical Sciences, University of Copenhagen

Lektor Kim Lefmann, Niels Borh Institute, University of

Copenhagen

PC Mini-Grids for Prediction of Viral RNA Structure and

Evolution

Cancer Treatment by Drug Delivery with Lipid-Based Nano-

Particles

Nano-technology for ultra high-speed optical

communications (Nano-Com).

High-Speed Histomorphometry

Computational models and tools for drug discovery

(COMODO)

Biopolymer nanocomposite films for use in food packaging

applications (NanoPack)

Hybrid functionals for metal oxide surfaces and nano-

particles

Nano- and Bio-functionalised Surfaces for Biofilm

Prevention

Nano-scale properties of proteins for biopharmaceutical

applications

Pectic polysaccharides for coating of bone implants

Large-scale quantum simulations and informatics in

nanocatalysis

Computational Design of Stable Enzymes

Miniaturised sensors for explosives detection in air XSENSE

Liquid Core Waveguide Technology for Diagnositics - LiCorT

Metalloprotease sensitive drug delivery systems for

treating cancer and inflammatory diseases

Nanotechnology in poultry production. Can silver

nanoparticles promote health and growth of chickens?

Center for Antimicrobial Research CAR

Constructing Local Intracellular Probe Systems

Electrochemical sensors for specific monitoring of

neurotransmitters in the brain

Centre for Pharmaceutical Nanotechnology and

Nanotoxicology

MC-Xtrace, a simulation tool for X-ray investigations of

nanostructures

*The listed grant holders are from the time the grant was awarded. In some projects, however, changes have been made during the course

of the project. This has been taken into account based on information from the Danish Council for Strategic Research.

11.5 The Terms of Reference

Terms of reference for evaluation of the stra-

tegic research theme “Interdisciplinary Use of

Nanotechnology, Biotechnology and Informa-

tion and Communication Technology” (NABIIT)

Background

The strategic research theme “Interdisciplinary Use of

Nanotechnology, Biotechnology and Information and

Communication Technology” (NABIIT) was allocating

research grants from 2005 to 2008. There were a to-

tal of 36 grants amounting to almost DKK 320 million.

The political ambition behind the research theme

was to realise and utilise the expected research

potentials of each of the technologies and espe-

cially the research potentials of the technologies

by combining them. In launching the research

theme, there was an expectation to attain break-

throughs and discover innovative uses by com-

bining two or all three technologies which might

contribute to overcoming the challenges, for

example, within the areas of health, environment

and energy. Furthermore, it was expected that the

development within the three technologies would

affect the technological development within a

number of industries considered to be Danish posi-

tions of strength, e.g. the biotechnology industry.

Thus, the research theme was launched in order

to encourage a combined use of nanotechnology,

biotechnology and ICT.

48/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

The funded research projects have been evaluated

ex ante when it was decided which applications

should be funded. Parts of the funded research pro-

jects have also been exposed to interim and ex post

evaluations as part of the follow-up on the projects.

However, the overall results within the research

theme have not been settled and it has not been

assessed whether the aims of the research theme

have been attained. On this basis, the Danish Coun-

cil for Strategic Research together with the Ministry

of Science, Innovation and Higher Education have

decided to evaluate NABIIT.

Purpose, target groups and use of the

evaluation

The purpose of evaluating NABIIT is to assess if the

objectives of the research theme have been fulfilled

and if the research theme has contributed to fulfill-

ing the general objectives of strategic research. Be-

sides assessing the achievement of goals regarding

the research theme and strategic research in gener-

al, the purpose of the evaluation is to create aware-

ness of the results of the research investments.

Specifically, the evaluation will assess,

— to which extent and how the research activities

within the research theme have helped to

identify and develop future opportunities for

commercial innovation and solutions to societal

problems by combining nanotechnology,

biotechnology and ICT and thus create synergy

between the areas.

— to which extent and how the research activities

within the research theme have helped to fulfil

the objectives of strategic research, including

— advance research of high international

standing

— increased interaction between public and

private research

— cross-cutting (interdisciplinary) research

initiatives

— internationalisation of Danish research

— postgraduate education and research training

— strengthening Danish research environments

The evaluation will particularly emphasise the im-

pact of research results in the private sector and on

users in the public sector. Conclusions will be drawn

on basis of the overall results within the research

theme and not the individual research projects.

The evaluation has two target groups. One target

group is the political system and the general public

who will be informed about the results of invest-

ments made in the research theme. The other target

group of the evaluation is the board of the Danish

Council for Strategic Research and the Programme

Commission on Strategic Growth Technologies which

can use the results to develop future priorities and

for the future implementation of research funding.

Scope

The scope of evaluation is the research funds which

were allocated between 2005 and 2008 as part of

the strategic research theme NABIIT – a total of 36

grants amounting to DKK 317.8 million.

Table 1 contains a number of key funding figures

within NABIIT. The table shows that the funds have

grown over time. Thus, the average grant amount

has risen from DKK 7.9 million in 2005 to DKK 14.5

million in 2008. The average grant during the period

was DKK 8.8 million. The smallest grant was given

in 2005 and amounted to DKK 3.4 million while the

largest grant was given in 2008 and amounted to

DKK 28 million. On average, each grant has DKK 7.1

million of co-financing and the total budget of the

research projects is DKK 15.9 million on average.

18 out of 36 grants have been completed at the

end of 2011. Three were completed in 2010 and 14

were completed in 2011. Six out of 18 grants which

were completed at the end of 2011 ran for four

years, 11 grants have lasted for five years and one

grant lasted for six years. The remaining grants are

expected to end in 2012 (nine grants), 2013 (eight

grants) and 2014 (two grants).

Table 11.2. Key funding figures within NABIIT

2005

Number of grants

Total grant amount (current prices)

Average grant amount (current prices)

Minimum grant amount (current prices)

Maximum grant amount (current prices)

Average amount of co-financing (current prices)*

Average total budget (current prices)*

7.9

3.4

11.9

7.2

15.1

2006

99.7

7.1

4.6

6.5

13.6

2007

8.4

5.5

4.9

13.3

2008

14.5

5.9

11.7

26.2

Grand total

317.8

8.8

3.4

7.1

15.9

Source: The Danish Council for Strategic Research Note: The amounts of co-financing are subject to uncertainty.

49/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

According to the calls for applications, funding was

awarded to research activities which pursued an

interdisciplinary approach to established knowl-

edge areas and included collaboration between

public research institutions and private companies.

There are between two and eight project partners

in the research projects and the average number is

almost four. Universities as well as private compa-

nies generally participate in the projects. Hospitals

and approved technology institutes participate in

some projects. Universities are the dominant party,

with up to five departments participating in one pro-

ject. There are examples of partnerships between

departments at the same university and between

departments at different universities.

comparison of the overall results within the research

theme with the objectives of the research theme and

the objectives of strategic research, respectively.

The assessment of the achievement of goals will be

made by an independent international peer review

panel on the basis of all the different documentation

gathered as part of the evaluation. In assessing the

achievement of goals, the panel shall take into con-

sideration that not all of the research projects have

been completed while some research projects have

only just been completed. The panel shall also take

into consideration that some of the requirements in

the calls for applications have changed over time.

Table 2 presents a number of indicators and bench-

All of the research projects combine at least two of the

three areas nanotechnology, biotechnology and ICT.

The subjects of the projects are broad and encompass

e.g. cancer treatment by drug delivery with nano-par-

ticles, use of nanotechnology in food packaging ap-

plications, computational models and tools for drug

discovery and risk assessment of free nano-particles.

Basis of evaluation

The evaluation will assess to which extent and how

the objectives of the research theme have been

fulfilled and to which extent and how the research

activities within the research theme have helped to

fulfil the general objectives of strategic research, cf.

the purpose. The basis for the assessment will be a

marks which the peer review panel may use in its

assessment. The indicators and benchmarks apply

to the objectives of the research theme as well as

the objectives of strategic research. The indica-

tors and benchmarks are derived from the political

agreement, the calls for applications and the quality

concept of the Danish Council for Strategic Research

which the council uses when assessing the quality

of applications. The concept of quality is made up of

three equivalent criteria: the relevance, potential im-

pact and quality of research, e.g. “Strategic research

– Principles and instruments” from 1 January 2011.

In the evaluation report, the panel is requested to

account for the criteria used in its assessments and

how the criteria have been employed.

Table 11.3. Indicators and benchmarks applicable to the assessment of

the achievement of goals

Indicator

Purpose of research

Interdisciplinarity (cross-cutting

research)

Benchmark

Share basic research/applied research

Participants

Examples of interdisciplinarity

Organisation (distribution of roles and tasks, forms of collaboration, organisation

models, invested resources)

Publications (joint publication)

Combined use of

nanotechnology, biotechnology

and ICT

Participants and forms of collaboration

Examples of the combined use of technologies

New or improved products, technologies, methods, processes or equipment

Publications (joint publication)

Collaboration and synergies

Organisation (distribution of roles and tasks, forms of collaboration, invested

resources)

Collaboration with partners who are not formally part of the project

Research management

Publications (joint publication)

Scientific output (process and

product)

Seminar and conference activity

Keynote addresses given

Publications (publication channels)

Data set

50/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

Indicator

Quality of the research

Benchmark

CV’s

Seminar and conference activity

Keynote addresses given

Publications (publication channels)

Data set

Development of research

environments in Denmark

Continuation of the research activities in the projects, for example new funding

Examples of increasing R & D activities among the participants

Recruitment of researchers (nationally and internationally)

Research training for PhDs and postdocs

Training of research managers

Postgraduate education and

education generally

Development of new lesson plans, including courses and conference activities

Participants PhD scholars

Participants postdoc scholars

Publications (PhD thesis)

Internationalisation

International seminar and conference activity

International visiting research fellows

Collaboration with international research institutions (for example as part of the

project or an advisory board)

Recruitment of international researchers

Research stays abroad

Applications for EU framework participation and others

Publications (joint publication and national/international publication channels)

Commercialisation of research-

generated intellectual property

Patents, patent applications

Licensing agreements

Spin out companies

New or improved products, technologies, methods, processes or equipment

Public and private-sector collaboration on new or improved products,

technologies, methods, processes or equipment

Public outreach (dissemination

of results to the general public

and companies which are not

part of the project)

Use and potential use in the

public sector

Media appearances (newspapers, radio and television)

Workshops and conference activity

Publications (publication channels)

Innovation in the public sector (new or improved products, technologies,

methods, processes or equipment)

Changes in legislation/control

Organisation

The board of the Danish Council for Strategic Re-

search appoints an independent international peer

review panel to carry out the evaluation. The panel

is responsible for the overall assessment of the

documentation gathered as part of the evaluation

and for an evaluation report being drawn up. The

panel is also to conduct qualitative interviews with

representatives from the projects and to help im-

prove the other parts of the evaluation documen-

tation.

The panel shall be comprised of a chairman and up

to five members. All members shall be international-

ly acknowledged researchers and have experience

with broad research areas in relation to nanotech-

nology, biotechnology and ICT.

Furthermore, the panel shall have experience and

knowledge of:

— Management of/participation in large research

projects

— Collaboration between public research institu-

tions, private companies and public users of

research

— Utilisation and commercialisation of research

— International research collaboration

— Evaluation methods.

An independent secretary is responsible for drawing

up the evaluation report on behalf of the peer panel.

Besides the interviews with representatives from the

projects conducted by the panel, the other parts of

the evaluation documentation will be carried out by

one or several external consultants. The consultants

shall have confirmed competencies in the different

methods of data collection outlined in these terms

of reference. The consultants shall also have knowl-

edge about strategic research and research policy

in Denmark.

The Ministry of Science, Innovation and Higher Edu-

cation is responsible for the quality assurance of the

51/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

documentation. Moreover, the ministry will, if nec-

essary, gather one or several parts of the documen-

tation on behalf of the panel. Furthermore, the min-

istry will assist the panel and the consultants with

relevant information about the scope of the evalu-

ation, including the population of project partners,

and, if necessary, with practical matters such as the

planning of the interviews with representatives from

the projects by the panel and travel planning.

Methods of data collection

The documentation gathered as part of the evalua-

tion shall ensure that the evaluation sheds light on

the questions asked from different perspectives and

with different kinds of data. Data will be collected

using the following methods:

be compared with regard to networking and

quality of the research.

The purpose of the analysis of mobility is to

examine to which extent the research projects

have contributed to national and international

mobility, including mobility between different

research institutions and between the public and

the private sector.

The purpose of the analysis of funding is to

examine to which extent researchers who are

participating in the projects simultaneously or

subsequently plan to apply, have applied or have

obtained further national or international funding

for the research activities in the projects.

Reporting

The evaluation will result in a written report. The

report is to contain a description and analysis of the

different parts of documentation, the conclusions

and assessments made by the panel and an execu-

tive summary and at least five portraits of different

projects suitable for publication.

The evaluation report is expected to be published

in the first quarter of 2013. In continuation of the

publication of the report, the panel is expected to

participate in a meeting with the Danish Council for

Strategic Research in order to discuss the evalua-

tion report.

—

Self evaluation among the participants

The purpose of the self evaluation among uni-

versities, private companies, hospitals and

approved technology institutes is to expose

the qualitative judgment of the participants as

regards for example the collaboration in the pro-

jects and the achieved results.

Furthermore, the self evaluation shall produce

quantifiable factual knowledge about the

organisation and the results of the research

activities including the scientific output, com-

mercialisation, postgraduate education and

internationalisation.

—

Qualitative interviews with selected representa-

tives from the projects conducted by the panel

The purpose of the qualitative interview with

selected universities, companies, hospitals and

approved technology institutes is to let the panel

meet representatives from the projects in order

to validate and qualify the assessments made

by the panels on the basis of the self evaluation.

—

Case studies of the use and potential use of re-

search results in the public sector

The purpose of the case studies is to examine

concrete examples of the use and potential use

of research results in the public sector.

It shall be examined to which extent the use of

research results is expected to or already has

contributed to innovation, improved the basis

for decisions or led to changes in public control.

—

Three different register based analyses are to

be carried out:

— A bibliometric analysis

— An analysis of mobility

— An analysis of funding.

The purpose of the bibliometric analysis is to ex-

amine the types of publications, national and in-

ternational joint publication and public outreach

from the projects in order to identify different

publication profiles across the projects. In that

regard, the publication profiles shall, if possible,

52/

Evaluation of The Strategic Research Programme NABIIT

UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren

The

Danish Council

for Strategic Research

Secretariat of The Danish Council for Strategic Research

Danish Agency for Science, Technology and Innovation

Bredgade 40

DK-1260 Copenhagen K

Telephone +45 3544 6200

Fax +45 3444 6201

E-mail: [email protected]