Uddannelses- og Forskningsudvalget 2015-16
UFU Alm.del
Offentligt
1647746_0001.png
Analyses of Danish Innovation Programmes
– a compendium of excellent econometric impact analyses
Innovation: Analysis and evaluation 13/2013
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0002.png
Analyses of Danish Innovation Programmes
– a compendium of excellent econometric impact analyses
Published by:
The Danish Agency for Science, Technology and Innovation
Bredgade 40
1260 København K
Tel. +45 3544 6200
ISBN: 978-87-92776-70-9
2
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0003.png
Analyses of Danish Innovation Programmes
– a compendium of excellent econometric impact analyses
Copenhagen, November 2013
3
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0004.png
CONTENTS
FOREWORD
AN EVALUATION OF THE DANISH INNOVATION ASSISTANT PROGRAMME
ANALYSIS OF THE INDUSTRIAL PHD PROGRAMME
THE INNOVATION CONSORTIUM SCHEME – AN ANALYSIS OF FIRM
GROWTH EFFECTS
CENTRAL INNOVATION MANUAL ON EXCELLENT ECONOMETRIC
EVALUATION OF THE IMPACT OF INTERVENTIONS ON R&D AND INNOVATION
IN BUSINESS (CIM)
6
9
141
181
231
4
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0005.png
5
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0006.png
FOREWORD
Progress depends on knowing what works and what does not. The primary tool for
attaining knowledge is analysis based on proper scientific principles and methods.
It is my hope that this compendium contributes to the advancement of sound
knowledge about what works in innovation policy, which is a notoriously difficult
subject ripe for methodological improvement.
The compendium consists of impact analyses of three Danish national innovation
programmes funded by the Danish Council for Technology and Innovation:
−
−
−
The Innovation Assistant scheme,
which subsidises employment of highly
educated individuals in small and medium-sized enterprises,
The Industrial PhD Programme,
where the PhD fellow is simultaneously
employed by an enterprise and enrolled at a university, and
Innovation Consortia,
which are large scale innovation project collaborations
between enterprises and public sector knowledge institutions.
Within the field of quantitative analysis of innovation policy, these three analyses are
unrivalled in scope, detail and accuracy for two main reasons:
The first is the comprehensive data sets collected by Danish authorities that are
available for research. They go far beyond what is generally offered in other
countries.
The second reason is the methods used for these analyses. These are in accordance
with the current state-of-the-art of econometric research, using the highest care in
the selection of comparison groups and subsequent mathematical processing.
In combination with each other, these two factors provide for analyses that should be
considered international best practice, and which provide a template for quantitative
evaluations of other types of interventions in industry and society.
For this purpose, this compendium also includes a manual for carrying out such
high-quality analyses.
I hope and encourage other agencies, in Denmark and elsewhere, to use this manual
to measure the impacts of their own initiatives, and, through a dialogue with the
Danish Ministry of Science, Innovation and Higher Education, to improve its
methods of attaining sound knowledge. This is how progress is made.
Thomas Alslev Christensen
Head of Department
Danish Agency for Science, Technology and Innovation
6
4
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0007.png
Fremskridt afhænger af, at man ved, hvad der fungerer, og hvad der ikke gør. Det
primære redskab til at opnå viden er analyse funderet på sunde, videnskabelige
principper og metoder. Jeg håber, at dette kompendium bidrager til fremme af
velfunderet viden om hvilken innovationspolitik, der fungerer – et overordentligt
svært emne med potentiale for metodologiske forbedringer.
I kompendiet er effektmålinger af tre danske, landsdækkende innovationsordninger,
som finansieres af Rådet for Teknologi og Innovation:
–
–
–
Videnpilotordningen,
som giver tilskud til ansættelse af højtuddannede i små
og mellemstore virksomheder,
ErhvervsPhD-ordningen,
hvor den ph.d.-studerende er ansat i en virksomhed
og samtidig indskrevet på et universitet, og
Innovationskonsortier,
som er større samarbejdsprojekter om innovation
mellem virksomheder og offentlige videninstitutioner.
Indenfor kvantitativ analyse af innovationspolitik er disse tre analyser uovertrufne
med hensyn til omfang, detaljegrad og præcision af to grunde:
Den ene er de omfangsrige datasæt, som danske myndigheder indsamler og stiller til
rådighed for forskning. De overgår langt det, man sædvanligvis kan tilbyde i andre
lande.
Den anden grund er metoderne, der anvendes i analyserne. De er i overensstemmelse
med state-of-the-art indenfor økonometrisk forskning, hvor man anvender den
største omhu i udvælgelsen af sammenligningsgrundlaget og den efterfølgende
matematiske behandling.
Tilsammen muliggør disse to faktorer analyser, som bør anses for højeste
internationale klasse, og som udgør en skabelon for kvantitativ evaluering af andre
former for indgriben i erhverv og samfund.
Til dette formål indeholder kompendiet også en manual om, hvordan man udfører
sådanne højkvalitetsanalyser.
Jeg håber på og opfordrer til, at andre institutioner, i Danmark og i udlandet, bruger
manualen til at måle effekterne af deres egne indsatser, og i dialog med Styrelsen
for Forskning og Innovation medvirker til at forbedre dens metoder til at opnå
velfunderet viden. Sådan gør man fremskridt.
Thomas Alslev Christensen
Kontorchef
Styrelsen for Forskning og Innovation
7
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0008.png
8
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0009.png
An evaluation of the Danish Innovation
Assistant Programme
En effektmåling af Videnpilotordningen
Copenhagen, August 2013
9
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0010.png
CONTENTS
SAMMENFATNING PÅ DANSK
EXECUTIVE SUMMARY
DEUTSCHSPRACHIGE ZUSAMMENFASSUNG
1. INTRODUCTION
2. THE INNOVATION ASSISTANT PROGRAMME (VIDENPILOTORDNINGEN)
3. DATA
DASTI data
The Statistics Denmark data
The Experian data
A first look at the data
4. INDIVIDUAL-LEVEL ANALYSIS
General methodological issues
Selection of controls
Empirical specification
Individual-level analysis: the regression model
Individual-level analysis: descriptive statistics
Individual-level analysis: Results
Potential employment effects
Potential earnings effects
Individual-level potential effects for different subsamples
12
15
18
20
22
23
23
23
24
24
26
26
27
28
29
30
38
38
46
53
10
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0011.png
5. COMPANY-LEVEL ANALYSIS
Empirical specification
Company-level analysis: selection of controls
Company-level analysis: the empirical model
Company-level analysis: Descriptive Statistics
Company-level analysis: Results
Potential employment effects
Potential effects on value added, net income (profits) and return on assets
Potential effects on average wage costs and labour productivity
Results for subsamples
Potential effects on the number of highly educated employees
Potential effects on the number of employees
Potential effects on value added
Potential effects on net income (profits) and return on assets
Potential effects on wages and labour productivity
6. EXTENSIONS
The survival of VP-companies
A comparison of VP-companies and companies participating in Innovation
Networks
A comparison of VP-companies and an extended sample of control companies
7. CONCLUSIONS
APPENDIX 1: ADDITIONAL TABLES OF THE COMPANY-LEVEL ANALYSIS
58
58
58
59
60
69
72
74
81
85
86
88
90
92
96
100
100
101
104
110
114
11
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0012.png
SAMMENFATNING PÅ DANSK
Videnpilotordningen
under
Rådet for Teknologi og Innovation
blev lanceret som en
del af
’Viden flytter ud’-tiltaget
under regeringen i 2004. Ordningen har eksisteret
siden 2005 og har som formål at øge små og mellemstore virksomheders vækst ved
at øge incitamentet til og nedbryde barrierer for ansættelsen af akademikere i disse
virksomheder.
På baggrund af den danske vækstudfordring generelt og den økonomiske afmatning
i kølvandet på finanskrisen indtager Videnpilotordningen en central rolle blandt de
politikinstrumenter, der sigter at skabe vækst og øge virksomheders kompetencer
i forhold til innovation og nytænkning. Interessen for ordningen skyldes også,
at en række tidligere analyser (f.eks. Junge og Skaksen, 2010, CEBR, 2011
2
)
har vist positive sammenhænge mellem virksomheders andel af højtuddannede
medarbejdere og deres produktivitet, og at udbygningen af ordningen kan
argumenteres for at kunne reducere den for tiden høje arbejdsløshed blandt
akademikere i Danmark.
Som led i sin løbende evalueringsstrategi har
Styrelsen for Forskning og
Innovation,
der administrerer ordningen, bedt
Centre for Economics and Business
Research (CEBR)
om at belyse, hvorvidt det kan vises, at ordningen lever op til
sin målsætning. Til dette formål har CEBR fulgt både deltagende personer og
virksomheder i et omfattende datamateriale. Denne rapport beskriver tilhørende
analyse.
Med hensyn til metodologi, analysevariation samt hvilke indikatorer, der vurderes,
er denne effektmåling af Videnpilotordningen i international sammenhæng ’best
practice’. Den kan tjene som målestok for evaluering af effekten af en specifik
indgriben i erhvervslivet, der kan udføres, hvis behandlingsgruppens etablerede
datakvalitet er ganske høj, og der findes højt detaljerede landsdækkende registre med
dataserier over tid for virksomheder og individer.
Analysen sammenligner løn- og beskæftigelsesudvikling for en stikprøve af
individer, der deltager i ordningen (videnpiloter) med andre, sammenlignelige
personer, der ikke deltager. Analysen sammenligner også vækst og
produktivitetsudviklingen i en stikprøve af virksomheder, der deltager i ordningen,
med andre (meget) sammenlignelige virksomheder, der ikke deltager.
Junge og Skaksen, 2010, Produktivitet og videregående uddannelse, CEBR, 2011, Ansættelse af Ph.D.er og
produktivitet.
2
12
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0013.png
Analysens resultater kan sammenfattes som følger:
Personer, der deltager i ordningen, øger deres beskæftigelsesrate i forbindelse med
deltagelsen i ordningen. Dette er ikke overraskende, da ansættelse er en definerende
karakteristik af selve ordningen. Efter mere end et år efter begyndelsen af
deltagelsen kan det dog ikke længere vises, at beskæftigelsesraten blandt deltagerne
er højere end i en referencegruppe af højt sammenlignelige individer – men det kan
nævnes, at analysens observationsperiode delvist ligger i en højkonjunktur med lav
arbejdsløshed blandt højtuddannede.
Personer, der deltager i ordningen, øger deres lønindkomst i forbindelse med
deltagelsen i ordningen. Lønindkomsten forbliver højere end i referencegruppen i
årene efter begyndelsen af deltagelsen, men konvergerer herefter.
Virksomheder, der deltager i ordningen, øger deres årlige vækst i antallet af
højtuddannede medarbejdere i forbindelse med deltagelsen. Det kan dog ikke
vises, at virksomheder, der deltager i ordningen, bliver ved med at ansætte flere
højtuddannede i årene efter deltagelsen i ordningen.
Virksomheder, der deltager i ordningen, er også kendetegnet ved et midlertidigt
forhøjet antal medarbejdere i årene efter deltagelsen, men det viser sig at være
svært at finde robuste sammenhænge for finansielle succesparametre som
værditilvækst, profit eller arbejdsproduktivitet. Dette skyldes ret stor variation i
nogle virksomheders udvikling i disse variable, som ikke er relateret til, hvorvidt de
deltager i ordningen.
For delstikprøver af mindre virksomheder, som ikke er kendetegnet ved større
ændringer i deres succesvariable, findes, at deltagelsen i ordningen korrelerer positivt
med stigende værditilvækst og profit. Således forøger deltagende virksomheder deres
værditilvækst i gennemsnit med op til ca. 800.000 kr. og profitten med op til ca.
400.000 kr. i årene efter deltagelsen.
Disse resultater peger i retning af eventuelle positive effekter af ordningen og er
i tråd med en tidligere analyses
3
resultater, men er behæftede med en betydelig
statistisk usikkerhed. Så selvom datamaterialet er blevet betydelig udvidet i forhold
til den tidligere analyse, er det på baggrund af de nye resultater stadig ikke muligt at
træffe sikre udsagn om, i hvilket omfang deltagelsen i videnpilotordningen forøger
værdiskabelsen eller profitten i virksomheden.
Det er ikke muligt at påvise positive sammenhænge mellem deltagelsen
i programmet og arbejdsproduktivitet, lønniveau og afkastningsgraden
(return-on-assets).
DASTI, 2010, ”Effektmåling af videnpilotordningens betydning for små og mellemstore virksomheder
Innovation: Analyse og evaluering 4/2010”
3
13
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0014.png
Som sammenfatning kan det siges, at eventuelle positive effekter af ordningen
kommer til udtryk i, at videnpiloter kommer hurtigere i arbejde, hvilket er forbundet
med, at de kommer på et højere lønniveau i de første år efter deltagelsen end
andre, sammenlignelige personer, der ikke deltager. Disse potentielle effekter kan
forventes at være højere i de nuværende år, som i modsætning til en stor del af
analyseperioden er kendetegnet ved en lavkonjunktur.
Resultater for virksomhedsdelen peger i retningen af, at virksomheder, som deltager
i ordningen, oplever højere vækst i værditilvækst og profit, men en betydelig
statistisk usikkerhed medfører, at disse resultater skal fortolkes med forsigtighed.
14
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0015.png
EXECUTIVE SUMMARY
The Innovation Assistant Programme under the Danish Council for Technology
and Innovation was launched as part of the
“Knowledge is moving out”-initiative
by the Danish government in 2004. The programme has existed since 2005 and
has the purpose of increasing the growth of small and medium-sized enterprises by
increasing incentives and breaking down barriers to employment of highly educated
individuals in these enterprises.
Because of Denmark’s growth problems in general and the economic downturn in
the wake of the financial crisis, the Innovation Assistant Programme plays a central
part among the policy instruments aiming at creating growth and increasing the
competences of enterprises on innovation and creative thinking. The interest in the
programme is also due to a number of previous analyses (ie. Junge og Skaksen, 2010,
CEBR, 2011
4
) that have shown positive correlations between the share of highly
educated employees in enterprises and their productivity, and that the expansion
of the programme can be argued to reduce the presently high unemployment rate
among the highly educated in Denmark.
As part of its ongoing evaluation strategy, the Danish Agency for Science,
Technology and Innovation (DASTI), which administers the programme, has asked
the Centre for Economics and Business Research (CEBR) to cast light on whether it
can be shown that the programme fulfils its objectives. For this purpose, CEBR has
followed both participating individuals and enterprises in an extensive set of data.
This report describes the corresponding analysis.
With regard to methodology, variation of the analysis and the indicators taken
into consideration, this impact analysis of the Innovation Assistant Programme is
international best practice. It may serve as a standard for intervention evaluations
that can be carried out if the established data quality of the treatment group is quite
high, and highly detailed national registers with data time series for enterprises and
individuals are available.
The analysis compares salary and employment developments for a sample of
participating individuals (innovation assistants) with other comparable individuals
not participating. The analysis also compares growth and productivity developments
for a sample of participating companies with other (highly) comparable companies
not participating.
The results of the analysis can be summarised as follows:
Individuals who participate in the programme increase their employment rate
in association with participating in the programme. This is not surprising, since
employment is a defining characteristic of the programme itself. It cannot be shown
that the employment rate among participants is higher than for a reference group of
highly comparable individuals more than a year after starting to participate.
Junge og Skaksen, 2010, Produktivitet og videregående uddannelse, CEBR, 2011, Ansættelse af Ph.D.er og
produktivitet.
4
15
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0016.png
However, it should be noted that the observation period of the analysis falls partly
within an economic boom period with low unemployment among the highly
educated.
Individuals who participate in the programme increase their salary income in
association with participation. Salary income remains higher than for the reference
group in the years after starting to participate, but then converges.
Companies that participate in the programme increase their yearly growth of the
number of highly educated employees in association with participation. However,
it cannot be shown that companies that participate in the programme continue to
employ more highly educated individuals in the years after participation.
Companies that participate in the programme are also characterised by a temporary
increase in the number of employees in the years after participation, but it turns
out to be difficult to find robust associations for financial success parameters such
as value added, profits or labour productivity. This is due to a quite large variation
in certain companies’ developments for these variables, which is unrelated to their
participation in the programme.
For subsamples of smaller companies that are not characterised by large changes in
their success variables, it is found that participation in the programme is positively
correlated to increasing value added and profits. Thus, participating companies on
average increase their value added by up to approx. DKK 800,000 (EUR 106,000)
and their profits by up to approx. DKK 400,000 (EUR 53,000) in the years after
participation.
These results point to possible positive effects of the programme and correspond
with the results of a previous analysis,
5
but are subject to a significant statistical
uncertainty. So even though the data material has been expanded significantly
compared to the previous analysis, it is still not possible to make any certain
claims about the extent that companies’ value added and profits are increased by
participating in the programme on the background of the new results.
It is not possible to show positive correlations between programme participation and
labour productivity, salary levels and return on assets.
In conclusion, it can be said that any positive programme effects are expressed by
innovation assistants finding employment quicker, which is associated with a higher
salary level in the first years after participating than other comparable individuals
who do not participate. These potential effects can be expected to be higher in the
present years, which unlike a large part of the analysis period are characterised by
an economic downturn.
DASTI, 2010, ”Effektmåling af videnpilotordningens betydning for små og mellemstore virksomheder
Innovation: Analyse og evaluering 4/2010”
5
16
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0017.png
For the company part of the analysis, results indicate that participating companies
experience higher growth in value added and profits, but a significant statistical
uncertainty means that these results must be interpreted with care.
17
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0018.png
DEUTSCHSPRACHIGE ZUSAMMENFASSUNG
Die vorliegende Studie wurde vom
Centre for Economics and Business Research
(CEBR)
an der Handelshochschule Kopenhagen (CBS) für die
Styrelsen for
Forskning og Innovation (DASTI)
des Ministeriums für Forschung, Innovation und
weiterführende Bildung erstellt.
Sie betrachtet das Wissenspilotprogramm („Videnpilotordning“,
VP-Programm),
ein vom
DASTI
geführtes Innovationsprogramm. Dieses Programm existiert seit
2005 und subventioniert die Neuanstellung von Akademikern in kleinen und
mittelständischen Unternehmen mit geringem Anteil hochqualifizierter Fachkräfte
durch Gehaltszuschüsse. Ziel des Programms ist es, die Kompetenzen teilnehmender
Unternehmen zu erhöhen und deren Wettbewerbsfähigkeit zu steigern.
Die Studie folgt ca. 360 teilnehmenden Personen und ca. 320 teilnehmenden Firmen
in dänischen Registerdaten. Diese erlauben es, Aussagen über den Berufserfolg
der am Programm teilnehmenden Personen zu machen, sowie das Wachstum
teilnehmender Unternehmen zu analysieren.
Der Berufserfolg wird dabei anhand der Entwicklung des Beschäftigungsgrades
und Jahresgehaltes gemessen. Auf Unternehmensniveau betrachtet die Studie
Entwicklungen in der Anzahl hochausgebildeter Mitarbeiter, der Beschäftigung,
der Lohnkosten, sowie der finanziellen Variablen Wertschöpfung, Gewinn und
Arbeitsproduktivität.
Um den Berufserfolg der teilnehmenden Personen und das Wachstum der
Unternehmen beurteilen zu können, werden aus den umfangreichen vorliegenden
Registerdaten Kontrollgruppen von Personen oder Unternehmen ausgewählt, die
die gleichen oder sehr ähnliche äussere Merkmale aufweisen wie die Teilnehmer
im Jahr vor deren Teilnahme im VP-Programm. Die statistischen Methoden der
Studie bestehen aus Vergleichen der verschiedenen Erfolgsvariablen zwischen
den Teilnehmer- und den Kontrollgruppen. Zusätzlich dazu erlauben die Daten,
für teilnehmende Unternehmen die Entwicklungen von Erfolgsvariablen nach
Teilnahme im Programm mit den entsprechenden Entwicklungen vor der Teilnahme
zu vergleichen. Ein ähnlicher Vergleich für Unternehmen in der Kontrollgruppe
erlaubt es, auch unbeobachtbare Faktoren aus dem statistischen Modell
herauszufiltern.
Die Ergebnisse der Studie lassen sich wie folgt zusammenfassen:
Personen, die am VP-Programm teilnehmen, weisen im ersten Jahr nach
Beginn der Teilnahme am Programm eine höhere Beschäftigungsquote als
Personen der Vergleichsgruppe auf. Nach zwei und mehr Jahren haben sich die
Beschäftigungsquoten beider Gruppen jedoch weitgehend angeglichen, womit es
nicht möglich ist, einen langfristigen Beschäftigungseffekt des VP-Programms auf
individueller Ebene nachzuweisen. An dieser Stelle sei jedoch darauf hingewiesen,
dass ein grosser Teil der Beobachtungsperiode der Analyse in eine Zeit guter
Konjunktur mit allgemein geringer Akademikerarbeitslosigkeit fällt.
18
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0019.png
Personen, die am Programm teilnehmen, weisen eine bessere Gehaltsentwicklung
als Personen, die nicht teilnehmen, auf. Dieser Unterschied ist statistisch signifikant
für die ersten Jahre nach Beginn der Teilnahme.
Unternehmen, die am Programm teilnehmen, erhöhen die Beschäftigung
hochqualifizierter Mitarbeiter im Vergleich zu Unternehmen in der Kontrollgruppe,
sowie die Beschäftigung generell mit, im Durchschnitt, ca. einem zusätzlichen
Mitarbeiter in Verbindung mit der Teilnahme am Programm.
In Bezug auf die finanziellen Erfolgsvariablen lässt sich feststellen, dass es
grundsätzlich schwierig ist, potentielle Teilnahmeeffekte in den Daten zu
isolieren: erhebliche Heterogenität der Firmen in Bezug auf die Entwicklung
der Erfolgsvariablen relativ zu der Grösse der Stichprobe und der Grösse der
potentiellen Effekte führt dazu, dass die Ergebnisse der jeweiligen Analyse von der
Wahl des ökonometrischen Modells sowie der Stichprobenauswahl abhängen.
In Stichproben kleinerer teilnehmender Unternehmen mit geringer Heterogenität
in den Erfolgsvariablen und der Entwicklung dieser Variablen, sind teilnehmende
Unternehmen durch, im Durchschnitt, höheres Wachstum in der Wertschöpfung
sowie des Unternehmensgewinns gekennzeichnet. Hier liegen für teilnehmende
Unternehmen die potentiellen geschätzten Teilnehmereffekte bei bis zu ca. 800,000
Dänischer Kronen (ca. 106.000€) in Bezug auf die die jährliche Wertschöpfung
und 400,000 Kronen (53.000€) für Unternehmensgewinn in den Jahren nach
Programmteilnahme.
Diese Ergebnisse ähneln den Ergebnissen einer früheren Studie, die auf weniger
umfangreichem Datenmaterial beruht
6
, lassen sich jedoch aufgrund eines
Mangels an statistischer Signifikanz und fehlender Robustheit in Bezug auf die
Stichprobenauswahl nicht verallgemeinern.
Für die Erfolgsvariablen Rendite (return on assets), Lohnkosten (als Mass für
das Lohnniveau des Unternehmens) sowie Arbeitsproduktivität lassen sich keine
positiven potentiellen Teilnehmereffekte ermitteln. Auch in Bezug auf diese
Variablen lassen die Ergebnisse den Schluss zu, dass die Bedeutung der Anstellung
von Wissenspiloten in vielen Unternehmen von anderen Entwicklungen überlagert
wird, und dass auch das im Vergleich zu einer früheren Studie ausgeweitete
Datenmaterial noch nicht ausreicht, um gesicherte Aussagen über den Erfolg des
Programms treffen zu können.
DASTI, 2010, ”Effektmåling af videnpilotordningens betydning for små og mellemstore virksomheder
Innovation: Analyse og evaluering 4/2010”
6
19
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0020.png
1.
INTRODUCTION
This report presents the data, methodology, and results of an evaluation of the
Danish Innovation Assistant Programme (‘Videnpilotordningen’
- VP programme
in the following). The analysis was completed by CEBR for DASTI in 2012. It
contributes to DASTI’s strategy to continuously monitor and evaluate its innovation
support programmes, to develop and improve the designs of its initiatives, and to
improve programme evaluation techniques.
The VP programme was launched in 2005 and aims at increasing the growth and
productivity of small and medium-sized enterprises (SMEs) by increasing the
share of their employees with a higher education.
7
It is supposed to overcome any
mutual reservations between SME managers and university graduates and increase
academic knowledge in SMEs. To achieve this goal, the VP programme subsidizes
the employment of university graduates in small and medium-sized companies.
Although the programme is small-scale, especially when compared to e.g. U.S.
or European-level knowledge transfer programmes, schemes similar to the VP
programme are currently being discussed or implemented in other countries as well,
for example in a couple of local states in Germany and Austria. For this reason, the
present study might also have an interest outside Denmark. From an academic point
of view, the study furthermore contributes to our understanding of employment
subsidies for highly skilled employees and the effects of knowledge transfers to
SMEs.
The present analysis was supposed to address two questions: First, how do
individuals who participate in the programme perform with regard to their
employment and income developments? Second, how do participating companies
perform in terms of employment and productivity growth? For this purpose,
individuals and companies are followed in large-scale register data, and the success
of programme participants is compared to highly similar individuals and companies
that do not participate in the programme.
The two different questions imply that the present report is divided into two
parts. The first part addresses the question of the extent to which individuals
benefit from participating in the programme. This question has recently gained
increasing public attention in Denmark, as unemployment among especially
young university graduates is soaring in the aftermath of the recent financial crisis
and the current Danish economic slowdown. This part looks at employment and
salary developments of programme participants in association with programme
participation.
The education classifications of this study follow the International Standard Classification of Educations (ISCED).
In the following, employees with at least a post-secondary education (ISCED classifications 4,5, and 6) are re-
ferred to as ‘highly educated employees’.
7
20
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0021.png
The second part of the analysis looks at whether companies benefit from
participating in the VP programme. This company-level analysis is again based
on large-scale register data. It might be considered of primary interest, since the
purpose of the VP programme is to increase company performance, whereas any
individual employment effects are secondary.
The success parameters of interest in this part of the company-level analysis are
employment growth, the number of highly educated employees, and the growth in
value added, profits, return on assets, average wages, and labour productivity.
21
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0022.png
2.
THE INNOVATION ASSISTANT PROGRAMME
(VIDENPILOTORDNINGEN)
An Innovation Assistant (‘videnpilot’, VP) is an academic employee with a post-
secondary or tertiary-level education. In Danish educational terminology, this
corresponds to respectively a medium-length (bachelor level) and a long higher
education (postgraduate level). The employee has to be employed in an SME to solve
one or more specific development tasks.
A VP-project is subsidised by DASTI and is supposed to contribute to the company’s
innovation, growth and productivity. The subsidy pays up to half of the VP’s salary,
with a maximum of DKK 12,500 (€1,700) a month for 6-12 months.
Privately owned small or medium-sized companies with at least 2 and at most 100
employees can apply for funding if there are at most two highly educated employees
in the company, it has existed for at least a year, and its yearly revenues surpass
DKK 1 million (€130,000).
The programme was launched in the beginning of 2005. Until 2012, approximately
500 projects have been completed.
8
For the following analysis, it is relevant to have an idea of just how VP-projects
come into life to better understand what kind of individuals and companies
participate in the programme. However, it needs to be acknowledged that there is
little if any general knowledge about how VP-company collaborations are initiated.
Anecdotal evidence suggests that it is often the VP who contacts the company
and suggests an employment relationship under the VP programme. And yet, it
might also be presumed that companies hiring new employees might exploit the
opportunity of saving wage costs in the beginning of the employment relationship.
The analysis can only consider projects for which there is information in the data after they have been started, so
the most recent projects are not part of the analysis.
8
22
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0023.png
3.
DATA
The data for the analysis is from three sources:
1. DASTI supplied information on individual VP-projects. Information includes
individual identification numbers of participating individuals, company
identifiers, and the start date of the project. These data will henceforth be
called the
DASTI data.
2. Data from companies’ financial reports from
Experian A/S,
a credit rating
agency. These data will be referred to as the
Experian data
in the following
sections.
3. Register information from
Statistics Denmark.
This is matched employer-
employee data including information on individuals (demographic
information, information on education, wage and occupation) and companies
(e.g. size, turnover). These data will be referred to as the
Statistics Denmark
data.
DASTI data
Since the start of the programme in 2005, DASTI has continuously collected
information such as individual IDs of VPs, the start-up time of VP-projects, hosting
company IDs (VP-companies in the following) and whether or not projects were
completed or aborted before schedule. Individual IDs are social security numbers
(CPR numbers) while company IDs are the numbers by which companies are
registered by the public authorities (CVR numbers).
The Statistics Denmark data
Characteristics for individuals are drawn from Statistics Denmark’s register. Data
is available up to 2010, implying that there is no information on the most recent
projects. Statistics Denmark data is typically available on an annual basis, with
census date in mid-November. It allows associating individuals with their companies
using the unique company and individual IDs.
9
Over the last decades, the data
resources of Statistics Denmark have been continuously extended, as all Danish data
with an associated individual or company ID can be merged with the existing data.
For example, the present analysis benefits from Statistics Denmark’s individual-level
information on education (degrees, focus of electives, grades) and company-level
information on turnover.
Timmermans B. The Danish Integrated Database for Labor Market Research: Towards Demystification for the
English Speaking Audience. Aalborg. 2010
9
23
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0024.png
The Experian data
The Experian data consists of approximately 1.7 million financial records in the
period from 2000 to 2010. The timing of the records is based on the closing dates
of the financial report periods. In case of companies filing multiple reports in a
calendar year, only one of these is selected for the analysis. The closing date of the
financial reports sets the time structure of the company-level analysis (which is
relevant to before-after comparisons). When merging information from Statistics
Denmark with the Experian data (such as information on the number of highly
skilled employees), it is the latest available information in the Statistics Denmark
registers before a given financial report’s closing date which is used in association
with the financial report in question.
10
A first look at the data
As a point of departure, there are 416 VPs in the DASTI data. Six of these cannot be
found in the registers that form the basis of the analysis, and there is no information
on the highest educational degree of 16 individuals. Since education is a control
variable of key importance for the analysis, these individuals are not included,
leaving us with 394 individuals for the individual-level analysis. For 30 of these
individuals, it has proven impossible to find highly similar controls. This implies
that the individual-level analysis is based on 364 individuals who participated in the
VP programme.
370 companies which have hosted VP projects can be found in the Experian
database the year before the start of programme participation. The remaining
companies not in the Experian data must be presumed to be unincorporated and thus
not obliged to submit financial reports to the authorities. Companies can be followed
until 2009 in the Statistics Denmark data and until 2011 in the Experian data. In
the sample of companies employed for the subsequent analysis, the companies are
observed over an average time span of 6.7 years.
The results of this report are based on DASTI’s information on the company-VP
matches. This is important to note, because the identification of hosting companies
is not always straightforward: Single companies may have several CVR numbers,
and there might be an element of randomness or selection regarding which CVR
number hosting companies use to register their VP-projects. In approximately 30
percent of the projects, the Statistics Denmark data (described in greater detail
below) suggest that the VP is employed at a company with a different CVR number
than the one stated in the DASTI data.
11
Most companies have their closing date at the end of December, which implies a short time lag between the
Statistics Denmark information (of end-November) and the financial report information. However, there are also
companies that have chosen other dates, e.g. end of March, to close books. For these companies, the information
from the Statistics Denmark registers comes with a time lag of up to one year.
10
This will of course govern robustness checks of later findings. It might be noted that some of the companies
that the Statistics Denmark data suggests are the ‘real’ hosts of the VP-projects do not fulfill the conditions for
programme participation.
11
24
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0025.png
For one of the extensions of the analysis, DASTI provided data on companies
that have participated in the so-called
Innovation Networks.
These networks are
collaborations of typically small and medium-sized companies with the purpose
of increasing knowledge transfer and innovation. The data on Innovation Networks
consist of 1923 observations belonging to 1158 companies, the discrepancy owing to
the fact that a number of companies participate in these networks more than once.
We only consider the earliest participation in any of these networks for the following
analysis.
Of the 1158 firms that participated in any of the networks, 1121 are found in the
Experian data. The discrepancy must again be assumed to be a result of non-
incorporated firms.
25
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0026.png
4.
INDIVIDUAL-LEVEL ANALYSIS
General methodological issues
The empirical analysis addresses the basic evaluation problem: What is the causal
effect of participation in the programme on given outcome variables?
In accordance with the relevant econometrical literature, which again borrows
from the biometrics and epidemiological literature, programme participants will
subsequently also be referred to as treatments. Also, starting to participate in the
programme will also be referred to as receiving a treatment. Non-participants who
act as a control group for the statistical comparisons will be referred to as controls.
12
There are different ways of addressing the evaluation problem. One way is using a
linear regression model. This model is estimated on a sample of both participating
and non-participating individuals. The linear regression model includes a set of
conditioning variables which hold constant a set of observable characteristics and
identifies causal effects under a
conditional independence assumption,
by which
participants do – on average – not differ from non-participants in characteristics
that (a) have an impact on the outcome variables and (b) are not controlled for in the
regression model.
These characteristics, sometimes called ‘omitted variables’, prohibit interpreting
treatment-control differences in outcome variables as causal programme effects.
Instead, they offer alternative interpretations of latter results. And the above
‘identifying’ conditional independence assumption is equivalent to assuming that
there exists no other explanation for treatment-control differences in the outcome
variables than the fact that treatments have participated in the programme.
Obviously, any empirical model supposed to isolate programme effects needs to
maximise the validity of this assumption. A first step in this direction is to carefully
select a control group for the analysis by a
matching procedure.
These procedures
are explained in greater detail in the following sections. The procedures select one
(or more than one) ‘twin’ or ‘match’ for each treatment. They imply that controls are
highly similar to treatments in their observable characteristics, which also increases
the likelihood that treatments and controls are highly similar in their unobservable
characteristics.
Also, the way the dependent ‘outcome’ variable enters the model has implications
for the validity of the conditional independence assumption. For example, statistical
comparisons of individual-specific before-after developments over time or fixed
effects models will typically be preferred to cross-sectional comparisons.
And as noted earlier, a set of conditioning variables can control for any systematic
differences between the treatment and control group which might remain even if the
controls were selected in a way to make them as similar as possible to the treatments
in their observable characteristics.
The term ‘controls’ is also sometimes used for the conditioning variables in statistical models. In this report,
‘controls’ refers to subjects in a reference group and not conditioning variables.
12
26
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0027.png
Selection of controls
Obviously, the validity of any statistical comparison can be questioned if treatments
(individuals or companies) systematically differ from the controls in characteristics
that the subsequent regression model is unable to take fully into account. We
want to select individuals into the control groups that are as similar as possible
to the treatments in the most dimensions possible. The problem of finding ‘good’
matches is that there are no two absolutely identical individuals, so it should be
acknowledged that any analysis that identifies controls on the basis of a matching
procedure is nothing but a sophisticated comparison that requires additional all-else-
equal assumptions for causal interpretation.
The controls can be selected by a host of different matching procedures developed
over the last decades. Overviews of these procedures are found in
Caliendo and
Kopeinig, 2008,
and
Blundell and Costa Dias, 2009.
13
The basic idea is to find for
each participant one or more ‘twins’ that are as similar to the given participant as
possible, and to use these matches as the analysis’ control group.
The specific matching procedure depends on the nature of the data. The modeller
typically chooses between
matching on observables
and
propensity score matching,
or some combination of the two.
Matching on observables simply means that for each treatment, one or more ‘twins’
(referred to as matches in the following) are selected from the group of potential
controls that have the same observable characteristics in a number of dimensions.
For example, one could choose for each participating VP one control individual with
the same education, gender, and stays in the same geographic region. For companies,
one could select controls on the basis of industry, size, financial performance
measures, and other characteristics.
When treatments are not particularly unique and there are a lot of potential
candidates in the pool of potential controls, matching on observables might be the
preferred choice. But, matching on observables runs into a
multidimensionality
problem
when one uses too many observable characteristics as conditions in the
procedure: It becomes impossible to find controls for all participants when they are
required to be equal in too many dimensions.
Of course, one way of “solving” this problem would be to disregard a lot of
information in the data and only require equality in a few observable characteristics.
In this case one could, for each treatment, select one or more controls from the pool
of potential controls that are equal in a few dimensions (or use the entire population
of potential controls as controls and weigh them in the subsequent regressions).
Blundell, R., Costa Dias, M., 2009. Alternative Approaches to Evaluation in Empirical Microeconomics. Journal of
Human Resources 44(3., 565-640.
13
Caliendo, M., S. Kopeinig, 2008. Some Practical Guidance for the Implementation of Propensity Score Matching,
Journal of Economic Surveys (2008) Vol. 22, No. 1, pp. 31–72.
27
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0028.png
Yet another option is to combine the benefits of the matching-on-observables-
procedure with the benefits of the
propensity-score-matching-procedure.
The
latter method has the benefit of allowing the use of vastly more information than
the matching-on-observables method: It condensates all variables which might be
considered relevant for the choice of programme participation into one single metric.
This is simply the estimated predicted probability of programme participation,
called the
propensity score.
14
This way, it is possible to find matches that are most
similar in terms of the propensity score instead of a set of observable characteristics.
The number of matches selected for each participant is set by the modeller, who
faces a trade-off between bias and efficiency: By including many matches for each
participant into the control group, the sample size is increased and the variance of
the subsequent estimators is reduced. However, increasing the number of matches
for each participant might lead to selecting subjects into the control group that are
not very similar to the treatment. This decreases the validity of the conditional
independence assumption. So there is a trade-off between the precision of the
statistical estimates and minimizing the risk of matching participants with controls
that differ in observed and unobserved characteristics.
Empirical specification
The empirical implementation is done in the following steps: (i) select a group of
controls, (ii) specify the regression model.
For the individual-level analysis, the selection of controls is from the registers of
Statistics Denmark, which contain information on the entire Danish population, and
is carried out in three steps:
First, we adjust the sample of potential controls. This is achieved by deleting
individuals with characteristics not found for any VP. For example, we drop
individuals with educations that no single VP has taken, and younger than the
youngest VP in our sample. The resulting data is referred to as the
adjusted
individual-level sample.
Second, we calculate a probability model for the likelihood of VP programme
participation for any given individual. This model provides evidence of which
individual characteristics are associated with programme participation, which
might be interesting in its own right. It is also used to calculate the propensity score
for each individual in the data and for each year, which is simply the predicted
participation probability for the given individual in the given year.
Rosenbaum, P.R., and D. Rubin (1983). The central role of the propensity score in observational studies for
causal effects. Biometrika (1983) 70(1): 41-55.
14
28
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0029.png
The conditioning variables of the propensity score matching procedure are selected
in cooperation with DASTI and include all variables potentially important for
programme participation and available in the data. The list consists of factors
such as demographic information (age, gender, marital status), information on
education, including 15 education categories, whether the individual is currently
in any education programme, the average grade of the final secondary education
examination, and focus of secondary education electives (math, language). There
are also occupational codes (17 categories including unemployment and leave), wage
income (9 categories), labour market experience (5 categories), and geographical
location of residence (9 categories).
For programme participants, these characteristics are collected for the year before
treatment, called ‘year
0’
or
t=0
in the following. This ensures that no information
affected by treatment enters the propensity score model.
Finally, we apply a single nearest-matching procedure (by employing STATA Corp.’s
psmatch2-command) on the basis of the probability model’s predicted propensity
scores (participation probabilities). In this procedure we also impose the condition
that twins are exactly equal in terms of education (approximately 2,200 different
categories in total and approximately 175 different categories for VPs), gender,
occupation (11 categories) and highly similar in age. Again, all information entering
the matching procedure is from the year prior to programme participation. We strive
for minimum bias of the later estimators and choose only one control (instead of
several controls) for each treatment.
For the following treatment-control analysis, it is necessary to define a year 0
(t=0) for controls just as has been done for treatments. This allows modelling the
dynamics of potential treatment effects in association with programme participation.
For controls, year 0 or t=0 is simply the year in which a given control is selected into
the control group. This is the year in which the given individual is most similar to its
twin in terms of observable characteristics and propensity score.
The following comparisons over time will be relative to year 0 instead of calendar
time. E.g. for treatments, t=2 is two years after the year before treatment (i.e., one
year after the start of programme treatment). For controls, t=2 is two years after
being selected into the control group.
Individual-level analysis: the regression model
The individual-level analysis is carried out using separate multivariate regressions.
We consider the following success parameters:
(a) Whether or not the individual is employed in
t=1, t=2, ..., t=5,
implemented
by indicator (dummy) variables.
(b) The increase in wage income (salary) between year 0 and year
t=1,
t=2, ..., t=5.
29
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0030.png
The success parameters are regressed on a treatment dummy (taking the value 1 for
treatments and 0 for controls) and the following conditioning variables: age, gender,
experience, average grade of final secondary education (high school) examination,
occupation in year 0, the sum of the Statistics Denmarks unemployment index
(measuring the aggregated time an individual has been registered as being
unemployed).
Individual-level analysis: descriptive statistics
394 individuals who have participated in the VP programme can be found in the
Statistics Denmark registers. Of these, 364 can be associated with controls equal or
similar in the dimensions described in the previous section. These 364 individuals
form the basis for the subsequent analysis. TABLE 4.1 describes the adjusted
individual-level sample (the total pool of available controls), the sample of VPs, and
the samples of VPs and controls used for the subsequent analysis.
15
The variable on whether or not a person is in education at a given point in time is from Statistics Denmark’s edu-
cation registers, while the variable of having education as one’s occupation is from Statistics Denmark’s education
occupation classifications (pstill). Individuals who work while studying are classified as under education in the
education registers and as working in the occupation information.
15
30
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0031.png
TABLE 4.1: Individual-level characteristics
Adjusted
sample
excluding VPs
(N = 1.018.245)
Variable
General information
Age (years)
Female
Experience (years, since 1980)
Average grade, secondary education
(high school)
Average wage (DKK)
37.176
0.401
10.483
84.304
300574
11.358
0.490
8.169
9.173
199968
Mean
Std. dev
Treatment
group (N=394)
Analysis
sample,
Treatments
(N=364)
Mean
Std. Dev.
Analysis
sample,
Controls
(N=364)
Mean
Std. Dev.
Mean
Std. Dev.
34.226
0.419
6.104
84.265
171721
9.380
0.494
6.669
8.304
207544
34.162
0.409
6.203
84.354
178437
9.426
0.492
6.811
8.279
212787
34.110
0.409
6.332
84.511
183930
9.627
0.492
6.685
8.833
197212
Years of registered unemployment
Married
In education
Post-secondary or tertiary education
Education: arts and humanities
1.149
0.487
0.137
0.588
0.142
2.005
0.500
0.344
0.492
0.349
1.241
0.411
0.398
0.807
0.183
1.979
0.493
0.490
0.395
0.387
1.170
0.429
0.401
0.805
0.181
1.898
0.496
0.491
0.397
0.386
1.163
0.412
0.393
0.805
0.181
2.076
0.493
0.489
0.397
0.386
Education: social sciences
Education: technical sciences
Secondary education, elective
direction: no information
Secondary education, elective
direction: general
Secondary education, elective
direction: math
Secondary education, elective
direction: languages
0.273
0.253
0.606
0.193
0.125
0.041
0.445
0.434
0.489
0.395
0.331
0.198
0.274
0.355
0.330
0.231
0.226
0.157
0.447
0.479
0.471
0.422
0.419
0.365
0.288
0.346
0.332
0.231
0.223
0.157
0.454
0.476
0.472
0.422
0.417
0.364
0.288
0.346
0.363
0.187
0.245
0.151
0.454
0.476
0.481
0.390
0.430
0.359
31
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0032.png
Region of residence
Copenhagen
Zealand N
Zealand S
Funen, Bornholm
Jutland S
Jutland W
Jutland E
Jutland N
Region not specified
0.312
0.112
0.076
0.079
0.065
0.095
0.177
0.080
0.005
0.463
0.315
0.264
0.269
0.247
0.294
0.381
0.271
0.071
0.223
0.056
0.079
0.157
0.046
0.063
0.231
0.142
0.003
0.417
0.230
0.270
0.365
0.209
0.244
0.422
0.350
0.050
0.217
0.049
0.082
0.162
0.049
0.063
0.234
0.140
0.003
0.413
0.217
0.275
0.369
0.217
0.244
0.424
0.348
0.052
0.247
0.055
0.077
0.135
0.052
0.058
0.277
0.099
0.432
0.228
0.267
0.342
0.223
0.233
0.448
0.299
Occupation (from Statistics Denmark's variable 'pstill')
Self-employed
Manager
Employee, high level
Employee, medium level
Employee, basis level
Employee, other
Employee, no further information
0.000
0.031
0.324
0.123
0.227
0.055
0.103
0.015
0.173
0.468
0.328
0.419
0.228
0.305
0.003
0.025
0.259
0.074
0.099
0.030
0.063
0.050
0.157
0.439
0.261
0.299
0.172
0.244
0.025
0.277
0.077
0.102
0.022
0.069
0.156
0.448
0.267
0.303
0.147
0.253
0.025
0.277
0.077
0.102
0.022
0.069
0.156
0.448
0.267
0.303
0.147
0.253
32
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0033.png
Unemployed
On parental leave
On sickness pay
Non-salaried worker
Education measure
In job market training
On social benefits ("revalidering")
Unknown
Outside labour force, other
In education
0.036
0.003
0.001
0.003
0.007
0.003
0.001
0.000
0.015
0.031
0.187
0.051
0.036
0.056
0.082
0.055
0.035
0.020
0.122
0.173
0.241
0.013
0.005
0.018
0.030
0.005
0.003
0.008
0.033
0.058
0.428
0.112
0.071
0.132
0.172
0.071
0.050
0.087
0.179
0.235
0.228
0.008
0.005
0.019
0.027
0.003
0.003
0.008
0.036
0.060
0.420
0.091
0.074
0.138
0.164
0.052
0.052
0.091
0.186
0.239
0.228
0.014
0.003
0.005
0.038
0.003
0.005
0.003
0.038
0.060
0.420
0.117
0.052
0.074
0.193
0.052
0.074
0.052
0.193
0.239
Year
2005
2006
2007
2008
2009
0.436
0.080
0.082
0.077
0.122
0.496
0.272
0.274
0.266
0.327
0.234
0.142
0.152
0.140
0.152
0.424
0.350
0.360
0.347
0.360
0.225
0.137
0.148
0.143
0.162
0.418
0.345
0.356
0.350
0.369
0.225
0.137
0.148
0.143
0.162
0.418
0.345
0.356
0.350
0.369
33
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0034.png
We find that individuals who participate in the scheme are represented among all
occupations, age groups and income levels. There is no gender bias in programme
participation. However, many VPs are relatively young, are unemployed or recent
higher education graduates, have sparse labour market experience and low income.
A more systematic way of describing the propensity of programme participation
is to estimate a binary choice model. The results of this model (specified as a logit
model) are shown in Table 4.2 which displays a selection of coefficient estimates.
Note that this model is also the backbone of the matching procedure used to identify
one matched control for each programme participant.
A look at the estimates of the individual-level logit model reveals that they by and
large corroborate the findings of the mean comparisons of Table 4.2: Individuals
participating in the programme are often relatively young, there are regional
differences, they are not characterised by high or low secondary education grades,
and they have high unemployment rates and low salary incomes. When controlling
for these characteristics, labour market experience (as long as it is positive) does
not come out as an important explanatory factor with regard to programme
participation.
The matching procedure finds controls for 364 of the total 394 participants in the
adjusted individual sample. The remaining 30 participants remain unmatched
because no other individual in the adjusted individual sample (the total pool
of available controls) could be found who was equal to these individuals in the
dimensions of education, gender, occupation and age.
The matched sample of treatments and controls can be compared by referring to
the right hand side columns of TABLE 4.1 and 4.2. We conclude that the matching
procedure succeeded in finding a group of controls highly similar to the group of
participants. This allows us to analyse treatment-control differences in the success
factors associated with programme participation in the following section.
34
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0035.png
Tabel 4.2: Individual-level analysis. Logit estimation results. Dependent
variable: Individual participates in the VP-programme in the following year.
Selected coefficients.
Adjusted sample
N=1,018,245, LR chi2(78)
=1129.19, Pseudo R2 =
0.1618
Coeff.
General information
Female
Married
In education
-0.122
-0.004
-0.050
0.114
0.111
0.181
Sample of treatments
and controls
N=728, LR chi2(76)=
25,57, R2=0.026
Coeff.
Ste.
Ste.
-0.065
0.078
0.009
0.192
0.173
0.310
Age (in years, omitted: <25 years)
(25-29)
(30-34)
(35-39)
(40-44)
(45-49)
(50+)
0.750***
0.717***
0.611**
0.441
0.758**
0.024
0.216
0.256
0.301
0.339
0.352
0.359
0.187
0.304
0.316
0.456
0.548
0.688
0.351
0.427
0.503
0.609
0.617
0.610
Region of residence (omitted: Copenhagen)
Zealand N
Zealand S
Funen, Bornholm
Jutland S
Jutland W
Jutland E
Jutland N
Region not specified
0.325
1.446***
1.436***
1.018***
0.784***
0.775***
1.269***
-0.079
0.243
0.216
0.171
0.265
0.233
0.151
0.176
1.007
0.142
0.242
0.372
0.176
0.298
-0.063
0.516
0.406
0.344
0.273
0.402
0.386
0.237
0.302
35
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0036.png
Secondary education final grade average (omitted group: unknown)
(0-75)
(76-85)
(86-90)
(90+)
0.064
0.242
-0.227
0.011
0.205
0.163
0.210
0.190
0.281
0.308
0.502
0.332
0.324
0.257
0.350
0.301
Occupation (from Statistics Denmark's 'pstill' variable, omitted: pstill-category 12 ('VAT-
payer'))
Self-employed (pstill=14)
Manager
Employee, high level
Employee, medium level
Employee, basis level
Employee, other
Employee, no further
information
Unemployed
On parental leave
On sickness pay
Non-salaried worker
Undergoing education
measure
In job market training
On social benefits
("revalidering")
Unknown (pstill=57)
Outside labour force, other
In education
1.976*
0.774
0.270
0.159
0.192
0.702
0.372
1.998***
1.219**
1.681**
2.551***
1.909***
2.808***
1.292
2.645***
0.876**
0.415
1.051
0.475
0.348
0.384
0.366
0.438
0.379
0.321
0.543
0.773
0.487
0.418
0.785
1.063
0.663
0.398
0.378
-0.047
-0.090
-0.063
-0.040
-0.058
-0.016
-0.160
-0.604
0.474
0.804
-0.540
-0.265
-0.949
0.842
-0.311
-0.095
0.794
0.557
0.602
0.597
0.771
0.621
0.541
0.923
1.367
0.981
0.678
1.582
1.410
1.298
0.633
0.606
36
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0037.png
Salary (omitted: no information)
0-0.15% of sample mean
15-25% of sample mean
25-50% of sample mean
50-75% of sample mean
75-100% of sample mean
100-125% of sample mean
125-150% of sample mean
150-200% of sample mean
200%+ of sample mean
0.249
0.472**
0.275
-0.828***
-0.784***
-1.193***
-1.379***
-1.527***
-1.741***
0.183
0.219
0.201
0.287
0.275
0.287
0.302
0.313
0.408
0.087
0.265
-0.260
-0.033
0.105
-0.244
-0.115
0.027
-0.398
0.298
0.352
0.317
0.450
0.446
0.430
0.444
0.480
0.604
Notes: *, **, *** denote statistical significance at the 10, 5, and 1% level. Additional variables included in the re-
gressions, but not presented in this table are: education (15 categories), experience (five categories), high school
average grade (five categories), unemployment experience index (variable ‘sumgrad’, six categories).
37
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0038.png
Individual-level analysis: Results
This section presents treatment-control differences in the outcome variables wage
income (Statistics Denmark variable
slon)
and employment (Statistics Denmark
variable
pstill
with a value of less than
40).
Results are based on descriptive graphs
and estimations with conditioning variables taking any remaining treatment-control
differences into account. All conditioning variables are from t=0, i.e. they are
collected in the year before treatment or, in the case of controls, the year of selection
into the control group.
When interpreting results, it might be kept in mind that the available data suggest
that long-term employment relationships for VPs in their hosting companies are not
very common. For example, 69 VPs were hired in 2005 with the VP-company match
confirmed by the Statistics Denmark data. Of these employment relationships, 53
(77 percent) were terminated within three years. For the employment relationships
started in 2005 and 2006, 71 percent were terminated within two years.
16
Potential employment effects
In the following, employment rates of VPs are compared with the employment
rates of individuals in the control group. Employment is measured by the Statistics
Denmark variable ‘pstill’ assuming a value of less than 40.
17
Note that this
variable is conditioned on when controls were selected into the control group. As a
consequence, employment rates are exactly equal for the two groups of individuals
in year 0 (t=0).
FIGURE 4.1: Share of treatments and controls in an employment relationship
(’pstill’
<
40). By year after year 0 (on horizontal axis)
100
80
60
40
20
0
-5 -4 -3 -2 -1
0
1
2
3
4
5
Treatments
Controls
In this project’s vintage of the Statistics Denmark data, the individual-company-match can only be followed until
the year 2008, preventing us from following individual-company relationships over longer time periods or in more
recent VP-projects.
16
17
Individuals on leave are not counted as employed.
38
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0039.png
FIGURE 4.2: Share of treatments and controls, by occupation. By year after year 0
(on horizontal axis)
60
50
40
30
20
10
0
-5 -4 -3 -2 -1
0
1
2
3
4
5
Top level (TREATMENT)
Top level (CONTROL)
Other employment (TREATMENT)
Other employment (CONTROL)
Unemployed (TREATMENT)
Unemployed (CONTROL)
Other non-employment (TREATMENT)
Other non-employment (CONTROL)
A first look at the data, see FIGURE 4.1, suggests that VPs are characterised by
decreasing employment rates in the years before treatment. But in association
with treatment, the employment rate increases to almost 100 percent. This is not
surprising, since employment is a defining characteristic of the programme. This
increase is not matched by the control group’s development in year t=1. However,
employment rates of the two groups converge over time and are at the same level
two years after treatment.
FIGURE 4.2 splits up developments in occupation status by
top level employment
(pstill<33), other employment (33<pstill<39), unemployment (pstill=40),
and
other
non-employment (pstill>40).
Here, it is found that treatments and controls
are characterised by highly similar developments in these variables in the years
before year 0, suggesting that the matching procedure has been successful. The
graph further suggests that in year 0, a number of individuals in the two groups are
finishing education or have left employment in the year prior to treatment or being
selected into the control group. After treatment, a large share of treatments are
categorised as top level employees, while controls pick up and have the same shares
of individuals in this category after approximately two to three years.
Employment probabilities are more formally analysed by means of simple binary
choice logit models, with ‘the individual is employed’ at t=x, x=1,2...5 being the
dependent variable, where t=0 is the year before treatment, t=1 is the year in which
treatment takes place, etc. Estimation is by separate binary choice models for each
t=x, x=1,2...5.
Table 4.3 displays the results. The coefficient of interest is the one associated with
the treatment dummy ‘Treatment=1”.
39
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0040.png
We find a substantial potential programme effect on employment, as a coefficient
of 2.361 implies an increase in the odds ratio of being employed by factor
(exp(2.18)=) 10.5. This large increase comes as no surprise, given that employment
is a defining characteristic of the programme, and given that we already had seen
that employment is close to 100 percent for treatments in the year after the start of
programme participation.
18
Potential programme effects for the years following programme participation are
a (exp(0.271)=) factor 1.3 increase in employment probability in year t=2, which is
not significantly different from zero, and a factor 1.9 increase in year t=3, which just
fails to be significant at the ten percent level. After more than three years after the
start of participation, the signs of the coefficients switch around zero and become
insignificant.
For the most part, the remaining variables come out as insignificant. The exception
is low-wage individuals and individuals unemployed in year 0, who have the lowest
probability of being employed in subsequent years.
We conclude that overall results indicate a presence of potential short-run
employment effects and an absence of potential long-term effects of the programme.
However, it should be noted that most of the observation period is from a boom
period with high labour demand in the Danish economy. This implies that non-
participants cannot be assumed to catch up to the same extent in current years
compared to the analysis period.
The numbers of observations of the estimations are reduced by the fact that some of the explanatory variables
completely determine the outcome variables. As a robustness check, the models for employment and salary
developments were estimated without explanatory variables. This did not change the overall results in any signifi-
cant way.
18
40
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0041.png
41
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0042.png
TABLE 4.3: Logit binary choice model results. Dependent variable: The
individual is employed in t=x.
Dependent variable: The
individual is employed in
t=1
Coeff.
Treatment=1
Age (years)
Female
Annual wage (DKK
1000)
(Years of unemployment
up to t=0)*1000
Year of experience since
1980
Married
Secondary education,
no information
Secondary education,
elective direction: math
Secondary education,
elective direction:
languages
Secondary education: hf
("higher preparation")
Secondary education:
average grade
2.365***
-0.0669**
-0.360
0.004***
50.480
0.000
0.300
1.785
-0.580
-0.461
Dependent variable: The
individual is employed in
t=2
Coeff.
0.271
-0.011
-0.399
0.006***
-174.5**
-0.012
0.257
0.145
-0.390
-1.380**
Ste.
0.350
0.030
0.300
0.001
74.080
0.037
0.318
2.156
0.508
0.590
Ste.
0.293
0.029
0.308
0.002
78.900
0.040
0.349
2.069
0.605
0.586
0.309
0.028
0.706
0.026
-1.492**
0.017
0.682
0.025
42
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0043.png
Dependent variable: The
individual is employed in
t=3
Coeff.
0.577
-0.031
-0.179
0.002
-147.100
Dependent variable: The
individual is employed in
t=4
Coeff.
-0.423
-0.033
0.744
-0.001
-114.500
Dependent variable: The
individual is employed in
t=5
Coeff.
0.281
-0.104
0.961
0.007
-343.500
Ste.
0.354
0.036
0.392
0.002
101.000
Ste.
0.408
0.035
0.538
0.002
96.880
Ste.
0.716
0.120
1.078
0.004
317.200
0.017
-0.304
3.091
0.837
-0.977
0.045
0.418
2.413
0.753
0.667
0.035
0.384
5.429*
0.010
-1.491*
0.047
0.438
2.910
0.782
0.837
0.229*
0.696
7.224
2.769*
1.392
0.135
0.881
6.034
1.507
1.025
0.725
1.207
-0.451
1.012
-0.051
1.343
0.042
0.029
0.0759**
0.038
0.043
0.071
43
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0044.png
Occupation
Top level (pstill=31, 32, omitted category)
Employee, medium level
(pstill=34)
Employee, basic level
(pstill 35)
Employee, other
(pstill=36)
Salaried employee, no
further information
(pstill=37)
Unemployed
In education
Self-employed
On leave, and other non-
employed
-1.137*
0.601
-0.415
0.655
-0.581
0.683
-0.429
0.647
0.854
-0.676
1.000
0.627
-1.268*
-0.788
0.753
0.578
-1.480***
-1.295*
0.531
0.696
-0.382
-1.040
0.559
0.802
Immigrant status: not an immigrant (omitted category)
Immigrant status: first
generation
Year: 2005
Year: 2006
Year: 2007
Year: 2008
Region: North Jutland
Constant
Number of observations
-0.702
-0.071
-0.375
-0.655
-0.722
0.302
1.967
568
596
R-squared
0.28
0.639
0.397
0.489
0.477
0.491
0.476
2.369
-0.149
1.701
486
492
0.37
0.414
2.341
-0.099
0.396
0.766
-0.447
0.616
0.398
0.513
0.395
44
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0045.png
0.732
-0.466
1.268
0.662
-0.348
-1.219
0.888
0.835
-2.046*
1.102
-0.110
0.765
0.094
1.078
0.476
1.728
-0.380
-0.392
0.638
1.233
-0.805
-1.738
0.743
1.166
-1.621
1.104
-1.106
0.745
-0.892
0.915
0.618
1.723
-0.771
-0.065
-0.295
0.614
0.461
0.456
-0.101
-0.552
0.904
0.458
-0.393
1.149
-0.202
0.109
383
386
0.38
0.585
2.576
-1.178**
-1.334
286
293
0.48
0.515
2.900
0.936
-1.156
119
129
0.57
1.017
7.844
Notes: *, **, *** denote statistical significance at the 10%, 5%, 1% significance level. All monetary values are CPI-
adjusted to base year 2009.
45
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0046.png
Potential earnings effects
FIGURE 4.3 looks at the average salary developments (measured by the Statistics
Denmark variable ‘slon’) of treatments and controls. We find that earnings profiles
are highly similar for treatments and controls before year 0, and that VPs on average
experience increasing salaries in association with programme participation. These
increases are higher for treatments than controls. However, after two to three years
after year 0, developments converge and individuals in the control group are doing
as well as participants.
19
A look at the dynamics of the salary distributions (instead of the means) in FIGURE
4.4 suggests that this increase is driven by VPs with low salaries in year 0. VPs in
the bottom 25th percentile of the salary distribution in year 0 experience the largest
salary increases in association with programme participation, which might be
presumed to be a result of these individuals entering an employment relationship in
association with the programme. On the other hand, there are fewer VPs with very
high salaries after year 0 than is the case for controls.
FIGURE 4.3: Salary developments of treatments and controls, in DKK. Means. By
years after year 0 (on horizontal axis)
400000
350000
300000
250000
200000
150000
100000
50000
0
-5 -4 -3 -2 -1
0
1
2
3
4
5
Mean (TREATMENT)
Mean (CONTROL)
The estimations behind TABLE 4.3 are based on the total sample of treatments and controls except for individu-
als who experience extreme changes in their annual salaries (e.g. increases of more than DKK400,000 between
year 0 and year 1 or more than DKK1,000,000 between year 0 and t=5). See TABLE 4.6 for results on a sample
including these individuals.
19
46
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0047.png
FIGURE 4.4: Salary developments of treatments and controls, in DKK. Distribution
parameters. By year after year 0 (on horizontal axis)
700000
600000
500000
400000
300000
200000
100000
0
-5 -4 -3 -2 -1
0
1
2
3
4
5
Median (TREATMENT)
Median (CONTROL)
25thpercent (TREATMENT)
75thpercent (TREATMENT)
25thpercent (CONTROL)
75thpercent (CONTROL)
10thpercent (TREATMENT)
90thpercent (TREATMENT)
10thpercent (CONTROL)
90thpercent (CONTROL)
The graphs suggest positive potential programme effects on salary in the years after
treatment and an absence of long-run effects. TABLE 4.4 considers these potential
effects in a more stringent way by means of a
conditional diff-in-diff
model. The
parameters of interest are again those associated with the variable ‘Treatment=1’ that
measures the potential programme effect on income for participating individuals.
47
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0048.png
TABLE 4.4: Linear regression results. Dependent variable: Salary (’slon’)
increase between t=0 and t=x, in DKK.
Dependent variable:
salary increase between
t= 0 and t=1
Coeff.
Treatment=1
Age
Female
Annual wage (DKK 1000)
(Years of unemployment
before t=0)*1000
Years of experience since
1980
Married
Secondary education, no
information
Secondary education,
elective direction: math
Secondary education,
elective direction:
languages
Secondary education: hf
("higher preparation")
Secondary education:
average grade
Occupation
Top level management (pstill=31, omitted category)
Employee, high level
(pstill=32)
Employee, medium level
(pstill=34)
-28418
-20162
35165
36887
56456***
-2319**
-18141*
-0.258***
0.55
303
2766
35814
-9863
-17159
Dependent variable:
salary increase between
t=0 and t=2
Coeff.
21773*
-3131***
-16614
-0.516***
-10.52**
4966***
11503
-38180
-1468
-20279
Ste.
8534
942
10183
0.04
2.63
1308
9953
53968
14459
16908
Ste.
12193
1156
12261
0.05
4.19
1510
13036
70676
20176
21190
-33721*
500
19976
638
-43647
-329
28839
855
-88562**
-120011***
37002
41621
48
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0049.png
Dependent variable:
salary increase between
t=0 and t=3
Coeff.
25310*
-5543***
-33214**
-0.541***
-9.339*
5848***
489
-106950
6464
-14886
Dependent variable:
salary increase between
t=0 and t=4
Coeff.
4721
-6457***
-41379**
-0.734***
-18.16**
7111**
-34760
36331
14275
-31307
Dependent variable:
salary increase between
t=0 and t=5
Coeff.
42096
-8382**
-43810
-0.748***
-23.13*
6018
-32646
-224476
-10233
-73838
Ste.
14264
1516
15574
0.06
5.61
2092
15552
90991
22935
29109
Ste.
18928
2207
19401
0.10
7.54
3105
21773
129246
28555
33005
Ste.
34214
3882
35452
0.19
12.00
6024
32643
224830
59345
67547
-16760
-1126
27138
1080
-8739
365
24743
1447
-6107
-3405
95472
2768
-58624
-104792*
57540
59370
-17487
-88445
58814
67119
-46003
-334935***
51369
57211
49
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0050.png
Employee, basic level
(pstill=35)
Employee, other
(pstill=36)
Salaried employee, no
further information
(pstill=37)
Unemployed
In education
Self-employed
On leave, and other non-
employed
-34889
-53589
-59833
38840
40962
38815
-111962***
-179786***
-140168***
41296
49799
41893
-35792
-52257
-98106**
-25577
38724
38788
43751
40412
-91996**
-157389***
-216754***
-64130
43113
47033
60192
46789
Immigrant status: not an immigrant (omitted category)
Immigrant status: first
generation
Immigrant status: second
generation
Year: 2005
Year: 2006
Year: 2007
Year: 2008
Region: North Jutland
Constant
Number of observations
R-squared
-24764
18391
28159**
24741*
41843***
-15582
-4876
160822**
596
0.28
20393
24869
12322
14035
14572
16977
14430
66487
5762
432927***
492
0.37
15799
86265
13383
106801***
1196
16645
-8490
25690
20603
13384
16282
16329
50
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0051.png
-96960
-141922*
-98519
61138
73440
63464
-93042
-62744
-58705
65682
73003
70012
-135173*
-116249*
-82539
75067
61842
95957
-62057
-77928
-247578***
-15111
62101
66937
78454
66071
-76378
-82241
-264535***
28299
68055
73711
93911
71073
-94949
-92145
-75841
-61067
73276
78694
78306
87232
34296
39432
12203
15943
28970
72979
16679
19477
21333
-34498
-23334
42702
99428
19795
-16168
-141338*
57123
73137
-14781
576595***
386
0.38
22487
115808
-2496
552475***
293
0.48
28343
148468
-2139
963247***
129
0.57
43334
266452
Notes: *, **, *** denote statistical significance at the 10%, 5%, 1% significance level. All monetary values are CPI-
adjusted to base year 2009.
51
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0052.png
52
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0053.png
Although potential employment effects are restricted to earlier employment for VPs,
we find that potential salary effects are slightly more persistent, as coefficients come
out statistically significant (albeit only at the ten percent level) for time leads of up to
three years. TABLE 4.4 also allows calculating the total potential programme effect
as the sum of the coefficient estimates. This is approximately DKK150,000 for the
total sample of all treatments and controls, a number which might be related to the
average cost of the programme.
Individual-level potential effects for different subsamples
As an extension of the previous
analysis, the sample of VPs and associated
controls is split up by a number of project-specific and VP-specific background
characteristics. In particular, the following distinguishes between whether or not
the VP-project was completed or terminated before schedule. The sample is also
split up by the industrial sector of the companies that hire the VPs or the associated
controls, and the education and gender of the VP and the associated controls.
Findings of the estimations on the subsamples are found in TABLE 4.5 for
employment and 4.6 for salary increases. These tables are based on the same models
that were estimated earlier, but only report the relevant coefficients associated with
the treatment dummy variables.
It is found that that there is little heterogeneity in the estimated potential effects of
the programme.
20
Only completed projects are associated with larger increases in
employment. This indicates that uncompleted projects are not just aborted because
of the VP moving to another employment relationship, but becoming unemployed.
This is also reflected in the absence of any measurable potential salary effect for this
group of individuals.
It is only possible to detect statistically significant potential employment effects
in the year after treatment (t=2) for VPs with a technical sciences education. It is
possible to detect positive potential salary effects in the years after treatment only
for female VPs, VP-projects in ‘other industries’, and completed projects.
Although single coefficient estimates are in most cases not statistically significantly
different from zero, the sum of the estimates of TABLE 4.6 are still the best
guesses of any potential salary effects over the first five years after treatment. These
potential effects are largest for female VPs and VPs who are employed in service
industries, and lowest for VPs with degrees in arts and humanities or technical
sciences, and VPs with a tertiary education.
For a couple of estimations, not all coefficients could be estimated because of low variation in the data relative
to the number of observations and the number of conditioning variables.
20
53
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0054.png
TABLE 4.5: Linear regression results. Dependent variable: The individual is employed in t=x. By
subsamples. Results for treatment dummy variables
Dependent variable: the individual
is employed in t=1
Coeff.
All projects
N
Only completed projects
N
Only not completed projects
N
Manufacturing and construction
N
Services
N
Other industries
N
Males
N
Females
N
Tertiary-level education
N
Education in arts & humanities
N
Education in social sciences
N
Education in technical sciences
N
2.371***
328
2.635***
212
2.264***
405
2.008**
116
3.960**
70
2.948***
183
0.717
1.935
0.870
0.395
0.551
0.498
2.365***
568
3.138***
449
1.435
99
0.942
0.471
Dependent variable: the individual
is employed in t=2
Coeff.
0.271
486
0.507
377
-0.358
87
1.08
128
0.874
88
3.079*
122
0.438
182
0.072
213
0.252
387
-0.086
98
-0.438
79
1.182*
160
0.627
0.809
0.786
0.342
0.439
0.468
1.864
1.044
0.81
0.810
0.346
Ste.
0.350
Ste.
0.293
54
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0055.png
Dependent variable: the individual
is employed in t=3
Coeff.
0.577
383
1.019**
309
0.416
Dependent variable: the individual
is employed in t=4
Coeff.
-0.423
286
-0.423
246
0.460
Dependent variable: the individual
is employed in t=5
Coeff.
0.281
119
-1.603
94
1.686
Ste.
0.354
Ste.
0.408
Ste.
0.716
1.45
115
6.551
77
0.796
115
0.160
223
1.406**
131
0.280
308
0.98
-0.34
94
0.65
4.068
6.029***
36
2.109
1.150
-2.387*
67
1.390
0.502
-0.619
177
0.525
0.393
68
0.926
0.717
0.343
89
0.732
0.424
-0.451
206
0.485
-0.528
95
0.927
1.108
44
-0.641
108
1.292
0.783
-1.234*
99
0.728
Notes: *, **, *** denote statistical significance at the 10%, 5%, 1% significance level.
55
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0056.png
TABLE 4.6: Linear regression results. Dependent variable: Salary (’slon’) increase between t=0 and
t=x, in DKK. By subsamples. Results for treatment dummy variables
Dependent variable: salary
increase between t=0 and
t=1
Coeff.
All projects
N
All projects, including outliers
N
Only completed projects
N
Only not completed projects
N
Manufacturing and construction
N
Services
N
Other industries
N
Males
N
Females
N
Tertiary-level education
N
Education in arts & humanities
N
Education in social sciences
N
Education in technical sciences
N
56456***
596
48137***
605
69542***
467
8866
129
23180
136
67198***
170
79337***
156
49065***
349
63790***
247
55809***
468
43389*
116
44390**
161
59315***
205
15017
18527
22664
9855
13465
11794
16158
17814
18829
19561
9659
9685
Dependent variable: salary
increase between t=0 and
=2
Coeff.
21773*
492
11877
501
33476**
381
-8478
111
14164
125
47811
90
69038***
149
11342
277
34185*
215
16325
391
13979
98
10731
137
5266
168
21677
30264
30402
14205
18891
17030
22550
30202
30675
29591
13573
13440
Ste.
8534
Ste.
12193
56
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0057.png
Dependent variable: salary
increase between t=0 and
t=3
Coeff.
25310*
386
24982
393
29125*
311
22334
75
44672*
126
-10601
89
26789
149
19414
232
36790
154
14992
314
-15664
72
3897
108
12788
144
26287
30019
43284
16568
22989
19759
25844
38463
26317
33461
16547
15445
Dependent variable: salary
increase between t=0 and
t=4
Coeff.
4721
293
4721
293
9146
248
24120
45
11914
97
48873
64
-7511
118
-6218
181
29105
112
-7062
233
-53186
41
4058
87
-18634
115
34162
42777
43560
22172
28294
26843
32362
39917
41021
56060
-21466
18928
Dependent variable: salary
increase between t=0 and
t=5
Coeff.
42096
129
42096
129
25853
106
123197
23
19104
40
123624
26
13542
56
60274
79
66325
50
20373
107
-30421
27
61546
40
51668
45
87173
67346
169878
39581
44998
53426
47681
135240
90804
103104
39825
34214
Aggregated dif-
ferences from
t=1 to t=5
Ste.
14264
Ste.
18928
Ste.
34214
150356
131813
167142
170039
113034
276905
181195
133877
230195
100437
-41903
124622
110403
Notes: *, **, *** denote statistical significance at the 10%, 5%, 1% significance level. All monetary values are CPI-adjusted to base year 2009.
57
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0058.png
5.
COMPANY-LEVEL ANALYSIS
In the following, the setup and results of the company-level analysis are described.
We briefly describe the model which aims at removing as much unobserved
heterogeneity as possible from the statistical comparisons. We then take a look at
the company-level data and inspect the sample for the subsequent analysis. Finally,
we compare companies that participate (receive a
treatment)
in the programme
(‘treatments’
or ‘participants’ in the following) with highly similar companies that
act as a control group. In particular, we compare developments in:
1.
2.
3.
4.
5.
6.
the number of highly educated employees
the number of employees
value added
net income (profit) and return on assets
average wage cost
labour productivity, measured as turnover per employee
The analysis addresses the question of how VP-companies perform in terms of these
variables. This is answered by looking at the developments in these variables over
time and comparing them to developments in a control group comprised of other,
similar companies that do not participate in the VP programme.
It should be noted that the analysis of the number of (highly educated) employees,
value added and net income gives highest weight to companies experiencing the
largest changes in these variables. These are typically larger companies. For average
wage cost, return on assets and labour productivity, companies are treated equally
and, thus, higher weight is given to smaller companies.
Empirical specification
Company-level analysis: selection of controls
For the company-level analysis, the selection of controls is carried out in two steps.
First, select a pool of potential controls in the Experian data. Second, apply a
matching procedure.
Before applying the matching procedure, we go through the Experian data and
exclude observations of companies in industries without participant companies,
with ownership classifications where there are no participant companies, companies
larger than 150 employees, and companies for which a set of additional conditions is
not fulfilled.
21
The remaining sample is denoted the ‘adjusted Experian sample’.
These conditions are: equity being between DKK-20mio and 150mio., net income between DKK-20mio and
20mio., total assets between zero and DKK250 mio., short term debt between DKK15,000 and 70mio., an equity
share between -2.5 and 0.9, return on assets between -1.2 and 1, the number of employees with at least a post-
secondary education less than or equal to 25, the number of employees with a tertiary education less than or
equal to 5, and firm age less than 150 years. Imposing these conditions does not affect the number of participants
in the sample.
21
58
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0059.png
As a last step before the matching procedure, we exclude all observations
of participants that do not belong to the last financial report before starting
participation in a VP-project. We then estimate a binary choice model on the
adjusted Experian sample which is used to predict a participation probability (a
propensity score) for any given company for any given year in the reduced Experian
population.
The population is then grouped by year and industry. Within each group, a matched
twin is found for each participant company on the basis of the propensity score. This
procedure ensures equality between the participants and controls in terms of the
highly detailed industrial sector classification ‘Dansk Branchekode’ and timing.
22
This procedure implies that we identify 316 control firms for 316 participants. These
define the
analysis sample
of the study. The year in which a control company is
selected is this company’s year 0 (base year, t=0), which is the cut-off year for later
before-after comparisons. For VP-companies, year 0 is simply the last year before
participating in the programme.
23
Company-level analysis: the empirical model
We chose a model with fully specified dynamics, which is highly similar to Kaiser
and Kuhn, 2012.
24
This model is formulated as follows:
y_( i ,t)-y_(
=
i
x
+
(α
n
D (t
i
= n) + β
n
(D (treat
i
=1)
× D (t
i
= n)) ) + u
i
+ ε
i
y
i,t
–
y
i,t–1
,t-1)=x_t+∑_(n=1)^5▒▒(▒α_n D(t_( i )=n)+β_n (D(▒treat▒_(
i,t
t
)=1)×D(t_( i )=n))▒_ )+u_i+▒ε_(i,t)▒_ ▒
n=1
where
y
i,t
is the dependent variable,
i
is firm index,
t
is a time index, where
t=0
is
year 0, and
x
t
are year dummies to account for business cycle effects. The
D
are
dummy variables assuming the value of 1 if the logical conditions in their brackets
are fulfilled. This model is estimated subject to company-level fixed effects
u
i
and
has statistical errors
ε
i,t
.
The α and β are estimation coefficients, where the β measures the potential
treatment effects. Note that this model extends Kaiser and Kuhn’s analysis by
estimating post-year zero effects not just for participants but controls as well. These
are measured by the coefficient vector α, while the vector β collects the conditional
difference-in-difference estimators.
25
∑
5
The observation period is characterised by considerable business cycle movements, which implies the need to
match controls as exactly as possible with regard to the time when they are selected.
22
To be specific, the base year of participants is defined by the closing date of the last financial report before the
start of participation. This means the base year of participants is not necessarily the calender year before starting
to participate in the programme.
23
Kaiser, U., Kuhn, J.M., Long-run effects of public–private research joint ventures: The case of the Danish In-
novation Consortia support scheme. Res. Policy (2012).
24
25
Another minor extension is the clustering of statistical errors
ε
i,t
within treatment-control twin pairs.
59
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0060.png
The fixed effects setup implies that all time-invariant factors drop out of the model,
thus making the model robust to any omitted time-constant factors which might
be correlated with the decision to participate in the programme. The set of dummy
variables generates a difference-in-difference model setup, and the coefficients
of the dummy variables in the vector β estimate the potential programme effect.
Separate dummy variables for each year after the base year allow estimating the
dynamics of the potential programme effect.
26
Company-level analysis: descriptive statistics
Out of the 434 companies that have hosted VP-projects in the DASTI data, 370
can be found in the Experian data. The remaining 64 firms that cannot be found
in these data are probably non-incorporated firms that are not obliged to publish
their financial reports by submitting them to the Danish Business Authority. Of the
firms found in the Experian data, 338 filed a report in the year prior to programme
participation. Only these firms will be considered in the subsequent analysis
comparing performance both before and after the start of participation.
When setting the sampling criteria for this analysis, we need to decide how to
treat outliers (extreme observations). This decision trades off robustness of later
results with their representativeness. In the following, we choose to describe results
for ‘typical’ VP-companies and to not consider companies in the financial sector
(reducing the sample by eleven companies) nor companies with ownership codes
that only occur very rarely in the sample of VP-companies (reducing the sample by
five companies).
27
After deleting financial sector companies and companies with atypical ownership
codes, we are left with 319 observations. Of these, 318 have started their project
before 2011 and can be followed for at least one year in the Experian data.
The controls for the latter analysis are found in the adjusted Experian sample.
In these data, there are 296,000 company-level observations in the period from
2004 onwards that are roughly similar to the participants in a few dimensions, e.g.
industrial sector and number of employees. For 316 of the 318 VP-companies, the
matching procedure succeeds in finding controls for the analysis.
Means and standard deviations of a set of characteristics of these companies
are described in the first columns of TABLE 5.1. This table also shows the
characteristics of the adjusted Experian sample – which was selected in order
to roughly resemble the group of participants, and used for the estimation of
propensity scores for the matching procedure. TABLE 5.1 allows comparing the 316
programme participants with the two Experian samples and the control group of
companies selected by the matching procedure.
Also note that taking first-differences in the outcome variables addresses any potential problems of serially
correlated unobserved characteristics.
26
For example, we drop co-operations (two occurrences), funds (one occurrence), companies with limited liability
(one occurrence), and one company with an unidentified ownership code.
27
60
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0061.png
A look at the raw figures shows that VP-companies are distributed over most
industries, with relatively large shares in trade (21 percent), consultancies (12
percent) and IT services (9 percent). These shares follow the industry distribution of
the total sample of companies in the Experian database. However, VP-companies
are underrepresented in construction and overrepresented in manufacturing, metal,
construction, advertising and cleaning.
At first sight, the VP-companies look healthy: On average, they are slightly larger
(mean 15 employees) than the average company in the Experian database (mean 11
employees) and have survived longer (15 years vs. 10 years). Many (42 percent) are
registered as exporters in the Experian database, and almost 50 to 100 percent are
owned by other companies, e.g. holding companies (compared to 34 percent for all
companies in the Experian data). Also, 11 percent own other companies (compared
to 5-6 percent of all companies).
When it comes to employee characteristics, it is found that VP companies have a
relatively large share of employees with at least a secondary education and also an
above-average share of employees with a post-secondary or tertiary-level education.
They have a relatively low share of technically trained employees.
The fact that VP-companies are not fully representative companies implies that, if
one aims at comparing these companies with other companies, one must carefully
construct a control group of similar companies for the comparison.
A first step in this process is the estimation of a binary choice model to estimate
propensity scores. This model is based on the 239,000 company observations in the
adjusted Experian sample and the 318 participants in the year before treatment.
The results of the binary choice model (formulated as a logit model) are displayed
in the left hand side columns of TABLE 5.2. Findings largely agree with what was
seen in the mean comparisons: Companies are most likely to participate if they
are not in the construction industry, are incorporated as joint stock companies,
are relatively large, have high returns on assets and a relatively low equity share, a
low average employee age, a high share of highly educated employees, and a low
share of employees with primary school as their highest level of education. The VP
programme is relatively popular in rural districts, with high propensity on the island
of Funen and both Southern and Northern Jutland.
The results of the logit model allow us to calculate predicted participation
probabilities (propensity scores). These are used to select a control group of
companies for the subsequent treatment-control analysis.
61
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0062.png
TABLE 5.1: Means and standard deviations of key characteristics of company-level samples
Summary
of all firms,
N =296,087
Mean
Industry
Construction
Trade
IT, services
Manufacturing
Metal industries
Furniture and related
industries
Travel agencies, cleaning
services
Advertisement
Consulting, business
services
Paper&publishing
Other
Key figures
Number of employees
No number of employees
information
Number of employees=0
Number of highly educated
employees
1
Value added (DKK1,000)
No value added
information
Net income (profit,
DKK1,000)
Return on assets
11.21
0.28
0.13
0.19
4713
0.12
676
-0.41
64.13
0.45
0.34
0.31
39920
0.32
25560
42.66
0.13
0.18
0.07
0.01
0.02
0.02
0.02
0.03
0.13
0.01
0.38
0.34
0.39
0.26
0.10
0.14
0.12
0.13
0.16
0.34
0.11
0.48
Summary of
adjusted sample,
N = 238.375
Mean
Std. Dev
Summary of
treatments in
analysis sample,
N = 316
Mean
Std. Dev
Summary of
controls in
analysis sample,
N = 316
Mean
Std. Dev
Std. Dev
0.15
0.19
0.07
0.01
0.02
0.02
0.02
0.03
0.13
0.01
0.35
0.36
0.39
0.25
0.11
0.15
0.13
0.13
0.16
0.34
0.11
0.48
0.06
0.21
0.09
0.06
0.05
0.06
0.03
0.06
0.12
0.03
0.22
0.23
0.41
0.29
0.24
0.22
0.23
0.18
0.24
0.32
0.18
0.42
0.05
0.21
0.09
0.06
0.03
0.08
0.04
0.07
0.12
0.02
0.23
0.22
0.41
0.28
0.24
0.18
0.27
0.20
0.25
0.33
0.15
0.42
7.02
0.23
0.11
0.17
2903
0.08
302
0.02
12.80
0.42
0.31
0.30
5941
0.27
1654
0.23
14.75
0.03
0.01
0.22
6483
0.01
457
0.03
18.39
0.18
0.11
0.27
8425
0.11
2165
0.21
13.96
0.02
0.02
0.22
6279
0.03
567
0.04
17.46
0.15
0.14
0.29
8304
0.16
2070
0.22
Notes: 1: “highly educated” refers to post-secondary education and tertiary-level education.
62
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0063.png
Wage cost per employee
(DKK1,000)
No wage cost per employee
info.
Labour productivity
(DKK1,000)
No labour prod. Info.
Total assets (DKK1 mio.)
Equity share
Short term debt (DKK1,000)
410
0.43
3096
0.44
17.07
-1.23
7008
1540
0.49
97103
0.50
219.76
99.71
86627
400
0.37
2623
0.37
7.79
0.28
3428
660
0.48
65175
0.48
16.29
0.38
6928
395
0.09
2056
0.08
13.06
0.22
6532
217
0.29
5479
0.27
20.31
0.35
9240
377
0.09
1867
0.09
13.05
0.23
6579
163
0.28
2627
0.29
21.51
0.34
9931
Development in selected key figures (average annual increase in t=-3 to t=0)
Number of employees
Number of highly educated
employees
Value added (DKK1,000)
Net income (DKK1,000)
Wage cost per employee
(DKK1,000)
Labour productivity
(DKK1,000)
Year
2005
2006
2007
2008
2009
0.11
0.16
0.18
0.21
0.23
0.31
0.36
0.39
0.41
0.42
0.11
0.16
0.18
0.20
0.22
0.32
0.37
0.39
0.40
0.41
0.24
0.15
0.15
0.15
0.16
0.43
0.35
0.36
0.36
0.37
0.24
0.15
0.15
0.15
0.16
0.43
0.35
0.36
0.36
0.37
0.34
0.12
269
33.9
-4.2
94.0
9.25
2.64
7602
9435.8
1567.3
40919.0
0.24
0.04
154
2.1
-4.2
74.4
2.09
0.54
1233
860.6
1529.4
22814.4
0.88
0.19
448
-1.4
2.6
-114.0
3.01
0.91
1876
1412.6
161.3
2694.0
0.85
0.11
506
89.0
-17.3
-721.3
3.12
0.82
1870
995.0
239.7
11055.9
Company age and ownership information
Ownership code: joint stock
Company age
Company has mother
company
Company is mother company
Company is exporter
0.27
10.45
0.34
0.06
0.12
0.44
21.80
0.47
0.24
0.32
0.44
21.80
0.47
0.24
0.32
0.44
13.45
0.48
0.23
0.32
0.52
15.10
0.48
0.11
0.42
0.50
19.87
0.50
0.31
0.49
0.53
13.94
0.49
0.09
0.39
0.50
16.32
0.50
0.29
0.49
63
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0064.png
Region
Zealand N, Copenhagen
Zealand S
Funen, Bornholm
Jutland S
Jutland W
Jutland E
Jutland N
Region not specified,
overseas departments
Employee characteristics
Company: mean employee
age (years)
Company: share of
employees that is female
Company: share with a
secondary education
Company: share with a
post-secondary education
Company: share with a
tertiary education
Company: share social
sciences
Company: share arts &
humanities
Company: share technical
sciences
40.1
0.26
0.26
0.19
0.08
0.26
0.03
0.35
9.6
0.29
0.34
0.31
0.21
0.32
0.12
0.35
40.0
0.25
0.24
0.17
0.07
0.26
0.03
0.35
9.5
0.29
0.33
0.30
0.20
0.32
0.12
0.35
0.29
0.05
0.30
0.25
0.14
0.28
0.29
0.05
0.32
0.29
0.14
0.33
37.5
0.30
0.31
0.22
6.6
0.26
0.28
0.27
37.6
0.27
0.30
0.22
7.1
0.26
0.32
0.29
0.24
0.09
0.11
0.07
0.09
0.09
0.16
0.08
0.43
0.28
0.31
0.26
0.29
0.28
0.37
0.27
0.23
0.09
0.12
0.07
0.10
0.09
0.16
0.08
0.42
0.28
0.32
0.26
0.30
0.29
0.37
0.27
0.14
0.04
0.15
0.11
0.11
0.10
0.18
0.11
0.35
0.19
0.35
0.32
0.32
0.30
0.38
0.31
0.19
0.04
0.16
0.07
0.11
0.07
0.18
0.10
0.39
0.19
0.37
0.25
0.32
0.26
0.38
0.31
64
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0065.png
TABLE 5.2: Company-level analysis. Logit estimation results. Dependent
variable: The company participates in the VP-programme in the following
year
Adjusted sample
N = 238,693
Mean
Industry
Construction
Trade
IT, services
Manufacturing
Metal industries
Furniture and related industries
Travel agencies, cleaning
services
Advertisement
Consulting, business services
Paper&publishing
Other (omitted category)
-0.85***
-0.31*
-0.16
0.90***
0.16
0.55*
0.48
0.28
-0.19
0.19
0.28
0.19
0.25
0.28
0.30
0.28
0.34
0.28
0.24
0.36
Treatments and con-
trols sample
N = 632
Mean
Std. Dev
Std. Dev
0.46
0.02
0.28
0.23
0.66
-0.27
-0.31
-0.08
0.13
0.34
0.46
0.28
0.40
0.41
0.47
0.40
0.51
0.42
0.36
0.58
65
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0066.png
Key figures
Number of employees
Number of employees^2
No employees information
Number of employees=0
Value added (DKK 1 mio)
No value added information
Net income (DKK 1 mio)
Return on assets
Wage cost per employee
(DKK1,000)
No wage cost per employee info.
Labour productivity (DKK1,000)
No labour prod. info.
Total assets (DKK 1 mio)
Total assets (DKK1,000)^2
Equity share
Short term debt (DKK1,000)
0.04***
0.00***
-0.79
-1.18*
-0.01
-0.70
-0.04
0.64**
0.00
0.74
0.00
-0.41
0.01
0.00
-0.57***
0.00
0.01
0.00
0.74
0.69
0.02
0.63
0.04
0.32
0.00
0.48
0.00
0.50
0.01
0.00
0.17
0.00
0.00
0.00
1.37
0.40
0.02
-0.81
-0.04
0.32
0.00
0.02
0.00
-1.16
0.01
0.00
-0.35
0.00
0.02
0.00
1.22
1.04
0.03
0.95
0.07
0.53
0.00
0.93
0.00
0.78
0.02
0.00
0.33
0.00
Development in selected key figures (average annual increase in t=-3 to t=0)
Number of employees
Number of employees, missing
obs.
Number of highly educated
employees
Number of highly educated
employees, missing obs.
Value added (DKK 1 mio)
Value added, missing obs.
Net income (DKK 1 mio)
Wage cost per employee
(DKK1,000)
Wage cost per employee,
missing obs.
0.03
0.15
0.09
0.03
0.61
0.08
0.03
0.14
0.12
0.05
1.04
0.12
0.04
0.03
0.17
-0.03
0.00
0.74
0.85
0.05
0.42
0.08
0.00
0.48
0.47
-0.06
-0.02
0.00
0.00
0.02
1.26
0.09
0.76
0.11
0.00
0.93
66
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0067.png
Labour productivity (DKK 1 mio)
Labour productivity, missing
obs.
0.00
1.49***
0.01
0.45
0.01
0.56
0.02
0.71
Year
2005
2006
2007
2008
2009
0.53***
-0.27
-0.33
-0.34*
-0.23
0.19
0.21
0.21
0.21
0.20
-0.06
-0.19
-0.10
-0.22
-0.25
0.29
0.33
0.32
0.34
0.32
Company age and ownership information
Ownership code: joint stock
Company age
Company age^2
Company has mother company
Company is mother company
Company is exporter
0.30**
0.00
0.00
0.06
0.18
0.99***
0.14
0.01
0.00
0.12
0.19
0.14
-0.13
-0.01
0.00
-0.07
0.23
0.12
0.21
0.01
0.00
0.19
0.31
0.20
Region (omitted category: Copenhagen)
Zealand N
Zealand S
Funen, Bornholm
Jutland S
Jutland W
Jutland E
Jutland N
Region not specified, overseas
departments
-0.24
-0.44
0.79***
0.72**
0.41
0.23
0.34
0.67**
0.27
0.38
0.28
0.29
0.29
0.30
0.27
0.29
-0.42
-0.05
-0.06
0.64
0.14
0.44
0.06
0.26
0.40
0.56
0.43
0.46
0.44
0.46
0.40
0.45
67
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0068.png
Employee characteristics
Company: mean employee age
(years)
Company: share of employees
that is female
Company: share with a
secondary education
Company: share with a
post-secondary education
Company: share with a tertiary
education
Company: share social sciences
Company: share technical
sciences
-0.04***
0.00
0.16
-0.53
-0.67
-0.08
-0.73**
0.01
0.24
0.38
0.41
0.41
0.31
0.35
0.00
0.25
0.03
-0.21
0.49
0.04
-0.45
0.01
0.42
0.64
0.69
0.70
0.55
0.60
Before turning to the analysis, we need to establish an idea of just ‘how similar’ the
groups of matched treatments and controls really are. Accordingly, we will compare
the two groups of companies as follows:
First, we run a very simple test of the similarity of observable characteristics of the
two groups of companies and estimate the same logit model as earlier, but this time
on the matched treatment-control sample. The results of this exercise are displayed
in the right hand side columns of TABLE 5.2. We find that all coefficients have
decreased in absolute size and come out as insignificant, indicating an absence of
considerable differences in these variables across the two groups of companies.
Second, we look at the similarity of the two groups of companies in the matched
treatments-controls sample by simply comparing the means of observable
characteristics of the two groups, displayed in the two right hand side columns of
TABLE 5.1.
Inspection of TABLE 5.1 suggests that the matching procedure succeeded in
finding matched twin companies that highly resemble the group of treatments in
the year before treatment. Differences between the groups are typically one order of
magnitude smaller than the corresponding standard deviations, implying that none
of the differences are statistically different from zero.
So: If the VP programme significantly increases the performance variables of the
analysis, we should be able to see this by higher growth in the performance variables
after treatment than before treatment, and a greater growth increase around year 0
for treatments than for controls. This will be tested in the next section.
68
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0069.png
Company-level analysis: Results
In the following, developments in a number of performance variables for companies
that have participated in the VP programme are compared with the group of controls
selected by the matching procedure. These variables are: the number of highly
educated employees (i.e. employees with an education at a post-secondary or tertiary
level), the number of employees, value added, profits, return on assets, wage costs
per employee, and labour productivity.
TABLE 5.4 displays the results of the
conditional diff-in-diff
model with company
fixed effects.
The coefficients
‘TREAT=1 & t=1’, ‘TREAT=1 & t=2’,..., ‘TREAT=1 &
t=5’
correspond to the potential treatment effect estimates
β
n
while the coefficients
of ‘t+1’, ‘t+2’, etc. correspond to the
α
n
of the conditional diff-in-diff model
described in the previous section. The results are based on the approximately 300
programme participants and the same number of associated control companies. But
only companies that participated early in the programme can be observed after the
very first years after treatment, so results for more than a few years after year 0 are
based on a substantially reduced number of observations.
Before we look at the specific findings, it is necessary to consider how to treat
outliers. We have to do with company level data which by its very nature is highly
heterogenous, and the treatment of outliers is important to later results.
28
TABLE 5.4 is based on VP-companies and companies in the control group with
at most 50 employees that do not experience large year-to-year changes in their
numbers of employees, as well as regression-specific conditions imposed to further
reduce unobserved heterogeneity. Obviously, the results of the analysis depend on
these sampling conditions, and when interpreting later results one must be aware
that the results are only valid for companies that fulfil the conditions. In subsequent
robustness checks, these conditions are relaxed.
The results of TABLE 5.4 are summarized in the following sections.
Although there is a lot of background information in the data, we are unable to offer explanations (and, thus,
cannot control for) for a large amount of heterogeneity in the data. Clearly, we do not want to base overall results
of the analysis on single observations with extreme values - especially when it cannot be ruled out that these
values are statistical noise (e.g. due to company mergers or organisational restructuring).
28
69
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0070.png
TABLE 5.4: Comparison between VP-companies and companies in the reference group. Companies
with up to 50 employees in year zero. Diff-in-diff fixed effects regression results
Dependent variables
(in first
differences):
Treat=1 & t=1
Treat=1 & t=2
Treat=1 & t=3
Treat=1 & t=4
Treat=1 & t=5
t=1
t=2
t=3
t=4
t=5
Year dummies
2003
2004
2005
2006
2007
2008
2009
Number of highly
educated employees
1
Coeff.
0.458***
0.318**
0.01
-0.14
-0.22
-0.03
-0.05
-0.14
-0.09
0.02
Number of employees
Value added
(DKK1,000)
2
Coeff.
219.3
374.1
165.2
124.0
-563.1
-20.5
-268.9
-10.5
243.0
468.9
Ste.
0.12
0.14
0.17
0.21
0.26
0.10
0.13
0.17
0.20
0.25
Coeff.
0.596**
0.00
0.33
-0.45
-0.69
0.00
-0.15
-0.33
-0.11
0.86
Ste.
0.30
0.34
0.40
0.60
0.65
0.24
0.32
0.41
0.56
0.59
Ste.
217.2
239.6
324.4
448.2
580.3
194.5
233.5
324.3
393.0
528.6
0.01
0.06
0.01
0.01
0.01
-0.13
-0.14
0.12
0.11
0.10
0.12
0.15
0.17
0.20
0.00
0.36
0.36
0.43
0.37
0.24
-1.477***
0.24
0.28
0.29
0.34
0.36
0.43
0.50
-298.1
308.3
156.1
348.7
277.8
-285.7
-810.8**
217.1
221.5
210.2
243.1
280.0
309.8
370.5
Constant
Number of observations:
Number of companies:
R-squared
0.12
2609
535
0.03
0.09
0.34
2727
546
0.08
0.25
240.8
2611
533
0.04
187.6
Notes: Only observations with annual changes in the number of employees of less than 12. *, **, *** denote statistical significance at the 10%, 5%,
and 1% significance level.
1. Employees with post-secondary or tertiary education. Only observations with annual changes in the number of employees with post-secondary
and tertiary education
<
5.
70
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0071.png
Net income (DKK1,000)
3
Return on assets
4
Wage per employee
(DKK1,000)
5
Coeff.
8.28
5.25
-21.34
16.06
-19.32
-13.08
1.67
10.89
-0.61
2.41
Labour productivity
(DKK1,000)
6
Coeff.
-27.92
57.28
-137.30
-39.23
-159.70
-36.00
88.66
108.40
24.21
97.65
Coeff.
-48.5
136.5
133.3
205.5
-103.5
24.2
-125.6
38.2
14.1
190.1
Ste.
95.2
111.4
122.1
218.1
189.2
87.2
104.2
129.0
199.6
220.3
Coeff.
-0.03
-0.04
0.00
-0.04
-0.04
-0.01
-0.03
-0.03
-0.02
-0.07
Ste.
0.02
0.03
0.03
0.04
0.06
0.02
0.03
0.03
0.04
0.05
Ste.
11.26
10.27
13.79
17.18
29.92
11.55
11.32
15.10
20.35
27.05
Ste.
93.59
107.90
91.84
134.00
215.30
87.17
91.22
97.51
117.90
203.20
-96.4
44.5
65.2
2.2
24.5
-191.8
-362.4**
90.3
86.6
84.0
95.6
113.5
130.1
159.9
-0.0426*
0.01
0.01
0.01
0.02
-0.02
-0.01
0.02
0.02
0.02
0.03
0.03
0.03
0.04
16.90***
8.99
8.00
10.54
13.41
4.35
8.81
6.23
7.21
7.97
8.97
13.39
15.66
18.74
-141.0*
-34.30
-28.89
48.53
-65.07
-180.4*
-90.63
72.66
58.79
74.78
90.79
96.66
108.80
122.50
78.7
2553
542
0.03
70.7
0.01
2669
544
0.02
0.02
-1.77
1494
346
0.01
5.88
60.65
1693
323
0.02
59.57
2. Only observations with annual change in the value added by less than DKK 10 mio.
3. Only observations with annual change in net income by less than DKK 3 mio.
4. Only observations with annual change in return on assets by less than 1, and total assets
>
DKK100,000.
5. Only observations with number of employees
>
5. Only observations with change in average wage
<
DKK 500,000.
6. Only observations with number of employees
>
5 and change in labour productivity
<
DKK 3 mio.
71
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0072.png
Potential employment effects
A first question addressed in the empirical analysis is whether companies
participating in the programme do indeed increase the number of highly educated
employees (employees with an education level categorised as at least ‘post-
secondary-non-tertiary and tertiary’, ISCED 4-8) relative to companies in the
control group.
TABLE 5.5.a: Potential effects on the number of highly educated employees. Further results
Ordinary least squares
regression
Coeff.
Treat=1 & t=1
Treat=1 & t=2
Treat=1 & t=3
Treat=1 & t=4
Treat=1 & t=5
0.445***
0.286**
-0.118
-0.241
-0.269
Firm fixed-effects model
Coeff.
0.436***
0.413**
0.097
0.092
0.214
Conditional diff-in-diff
model
Coeff.
0.458***
0.318**
0.005
-0.143
-0.221
Ste.
0.109
0.108
0.113
0.124
0.201
Ste.
0.146
0.176
0.233
0.300
0.394
Ste.
0.115
0.143
0.169
0.205
0.257
Includes firm-fixed effects
Includes year dummy
variables
Includes information from
before year zero
Includes observations of
the control group
no
no
no
yes
yes
yes
yes
yes
yes
no
no
yes
Number of observations:
Number of companies:
R2:
631
274
0.05
1354
274
0.02
2609
535
0.03
Notes: Highly educated employees are employees with a post-secondary or tertiary-level education. Only observations with annual changes in the
number of employees with a post-secondary and tertiary education
<
5. Only observations with annual changes in the number of employees of less
than 12.
*, **, *** denote statistical significance at the 10%, 5%, and 1% level.
The coefficients of a simple ordinary least squares regression, which are equivalent
to the population means and found in the leftmost columns of TABLE 5.5.a, imply
that participating companies increase their number of highly educated employees by
(0.445+0.286=) 0.7 employees in the first two years after start of participation.
72
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0073.png
The results of a company fixed effects model, which implements a before-after
comparison for programme participants, are presented in the middle columns of
TABLE 5.5.a. The similarity of this model’s results and the results of the simple
ordinary least squares regression implies that the earlier finding of an increase in the
number of highly educated employees in association with programme participation
(the results of the full-fledged model of TABLE 5.4 are replicated on the right of
TABLE 5.5.a) is not to be interpreted as a continuation of any before-participation
growth trend.
This allows the conclusion that the finding of positive potential programme effects
with regard to highly educated employees is not just the result of the developments
in (or the choice of) the group of control companies in the fully specified model
behind TABLE 5.4. This observation, and non-positive coefficient estimates of
the α
n
-coefficient associated with
‘t+1’, ‘t+2’
indicate an absence of behavioural
additivity: Companies in the control group do not experience increases in the
number of highly educated employees in the years after the selection into the control
group.
Aggregated coefficients of the fully specified model are shown graphically in
Figure 5.1.
29
Findings suggest that a participating company increases the number
of highly educated employees by 0.46 additional individuals in the year of the
treatment. The reason this number is not equal to 1.0 is that some of the projects
(and associated employment relationships) last less than one year and have already
been terminated before the census date of year 1. Also, as noted earlier, in some
cases the information on highly educated employees is registered with time lags, if
the data is from different sources (for instance, VP projects starting between the end
of November and the closing date of the company’s financial report). In these cases,
potential effects occur between
t=0
and
t=2
instead of between
t=0
and
t=1.
30
Figure 5.1 (just like the figures to follow in the next subsections) presents aggregated estimated treatment
coefficients
β
n
. These measure the average deviation of the developments of treatment companies after treatment
from the developments of the control group and the (company-specific) developments before treatment.
29
The variable ‘number of highly educated employees’ is constructed from information from Statistics Denmark.
This information can be a couple of months older than the closing date of the given company’s financial report,
which sets the time structure of the analysis. For example, VPs hired between Statistics Denmark’s closing date
at the end of November and the end of March will, in companies closing their books at the end of March, first occur
in the data in the following year.
30
73
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0074.png
FIGURE 5.1: Number of employees. Aggregated estimated model coefficients.
Years after treatment on horisontal axis.
1
0.8
0.6
0.4
0.2
0
-0.2
-0.4
0
1
2
3
4
5
Number of highly
educated employees
Number of employees
As with the individual-level analysis, the coefficient estimates
‘TREAT=1 &
t=1’, ‘TREAT=1 & t=2’,..., ‘TREAT=1 & t=5’
can be summed up to calculate the
total potential effect up to five years after treatment. This potential effect is an
additional
(0.46+0.32=) 0.78
individuals in the first two years and an additional
(0.46+0.32+0.00-0.14-0.22 =) 0.42
individuals in the first five years after
treatment.
31
Accordingly, a first conclusion is that VP-companies on average increase the
number of employees with a post-secondary education and above by an additional
0.8
employees in association with programme participation. However, there are
no indications that participating companies continue to increase their number of
employees in the years after programme participation: They have, on average,
lower increases (greater declines) in the number of highly educated employees
than companies in the reference group in year four and five after year zero, but this
finding is not statistically significant.
Results for employment (independent of educational level) indicate that there is an
immediate potential effect of
0.6
additional employees in the year of treatment,
which is slightly larger than the potential effect found for highly educated
employees. This indicates that VPs are often hired in association with company
growth, or that some of the VPs are categorised as having an education below
ISCED 5 or 6 in the Statistics Denmark education registers.
These numbers are high in comparison with the previous finding that long-term relationships between VPs and
their hosting companies are relatively uncommon, suggesting that VPs are replaced by other highly educated
individuals after the end of their projects.
31
74
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0075.png
As is the case for highly educated employees, there is no sign that participating
companies continue to increase the number of employees in the years after
programme participation, with negative coefficients for year 4 and 5 after treatment
resulting in an aggregate potential treatment effect over the first five years of -0.2
additional employees. Even though this number is not statistically different from
zero, it is still the best guess of any long-run treatment effect of the programme.
TABLE 5.5.b: Potential effects on the number of employees. Further results
Ordinary
least squares
regression
Firm fixed-effects
model
Conditional diff-
in-diff model
Conditional diff-
in-diff model,
dependent vari-
able: annual
employment
growth in percent
1
Coeff.
8.834***
1.517
0.812
-1.618
-5.425
Coeff.
Treat=1 & t=1
Treat=1 & t=2
Treat=1 & t=3
Treat=1 & t=4
Treat=1 & t=5
1.164***
0.237
0.215
-0.896**
-1.168***
Ste.
0.175
0.241
0.280
0.383
0.421
Coeff.
0.642**
-0.118
0.004
-0.448
0.436
Ste.
0.278
0.411
0.520
0.651
0.789
Coeff.
0.596*
0.001
0.331
-0.446
-0.694
Ste.
0.296
0.335
0.400
0.596
0.646
Ste.
2.929
2.934
3.152
4.058
6.459
Includes firm-fixed
effects
Includes year dummy
variables
Includes information
from before year zero
Includes
observations of the
control group
no
no
no
no
yes
yes
yes
no
yes
yes
yes
yes
yes
yes
yes
yes
Number of
observations:
Number of
companies:
R2:
650
274
0.07
1399
278
0.08
2727
546
0.08
2520
525
0.07
Notes: Only observations with annual changes in the number of employees of less than 12. *, **, *** denote statistical significance at the 10%, 5%,
and 1% signficance level.
1: Only observations with annual growth between -50 and 100 percent.
75
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0076.png
We are again interested in whether or not the results regarding the potential
employment effects are because of higher growth in treatment companies after
participation relative to before participation, or if the results are due to control
companies having lower growth after year zero relative to before when compared
to the treatment companies. For employment developments, we find again that the
overall results do not depend on the choice of the control group, as the before-after
comparison of the fixed effects model (on the subpopulation of treatment companies)
gives estimators that are highly similar to the fully specified model.
Also, we are interested in learning how much the previous results depend on
measuring employment growth as either absolute increases or percentage-point
growth. Investigating absolute annual increases is the first choice for simple-to-
implement cost-benefit calculations, but this also implies that smaller companies
with small absolute changes in the performance parameters are given low weight in
the statistical estimations.
When considering percentage-point employment growth, we find again a statistically
highly significant positive potential employment effect in the years around treatment,
suggesting that treatment companies grow by an additional 10 percent in the first
two years after treatment. But also in this alternative model, there is no indication
that treatment companies continue to increase their number of employees in year 4
and 5 after treatment.
32
Potential effects on value added, net income (profits) and return
on assets
We now turn to the financial performance variables. The results for these variables
need to be interpreted with care, since they depend critically on the treatment of
data - first and foremost the definition and treatment of outliers, i.e. companies
experiencing large changes in the performance variables.
For the specific treatments of outliers and the given modelling choices, we find
mostly positive, albeit statistically insignificant potential treatment effects for both
value added
33
and net income (profits). Findings of TABLE 5.4 are depicted in
FIGURE 5.2 and show that participating companies gained up to an additional
DKK800,000 (EUR106,000) in annual value added and DKK400,000 (EUR53,000)
in net income. But given the lack of statistical significance, these results should be
interpreted as highly tentative.
We will also present results for percentage-point growth rates for some of the other success parameters: gross
profit, average wages, and labour productivity. There will be no such regressions for the performance measures
number of highly educated employees, net income and return on assets, because these measures often assume
the value zero or negative values – which implies that growth rates cannot be calculated.
32
This variable is from the financial statements that companies file with the public authority, where it is called
dækningsbidrag/bruttofortjeneste.
33
76
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0077.png
FIGURE 5.2: Gross profit and net income (DKK1,000) developments in small steady-
going companies. Aggregated estimated model coefficients. Years after treatment
on horisontal axis.
1000
800
600
400
200
0
-200
0
1
2
3
4
5
Value added
Net income
FIGURE 5.3: Return on assets developments (in percent) in small steady-going
companies. Aggregated estimated model coefficients. Years after treatment on
horisontal axis.
105
100
95
90
85
80
75
0
1
2
3
4
5
Return on assets
We also take a look at developments in return on assets, calculated as net income
over total assets. The reasoning is that we have already looked at company growth
variables, such as the number of employees and increases in value added, and that
return on assets is largely independent of company size (which is obviously not the
case for net income).
Cf. FIGURE 5.3, we find that companies that hire VPs on average do worse in
terms of return on assets relative to companies in the control group of highly similar
companies, but that coefficients are statistically insignificant.
77
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0078.png
TABLES 5.5.c-d further expand on the findings for value added, net income, and
return on assets.
A look at the left hand side coefficients of TABLE 5.5.c suggests that value added
developments are on average positive for treatment companies in the first three
years after treatment, and negative more than three years after treatment. Part of the
increases in the first years after treatment can be interpreted as a continuation of
pre-treatment growth developments, as coefficients drop from DKK 421,702 to
DKK 260,056 when controlling for company fixed effects. Controlling for
developments in highly similar control companies, on the other hand, does not
change the general picture, so the selection of the control group does not appear to
be important to the overall result.
Also, for given sampling criteria, the previous (statistically insignificant) finding that
treatment companies on average have higher value added growth is confirmed by the
regression of percentage point value added growth. This regression even suggests
the presence of positive and statistically significant potential effects for year two and
four after treatment. The findings of a lack of significance for the model of absolute
value added increases and the presence of significance for the growth rate model
lends itself to the interpretation that companies with initially low value added gain
the most in association with programme participation.
Turning to net income increases, we find that there is large heterogeneity in this
variable, and as a consequence no statistically significant potential treatment effects
can be detected for any of the different models. On average, absolute net income
growth is negative for treatment companies after treatment. This can be explained
by generally adverse business developments and company-specific time trends, as
controlling with year dummies and for company-fixed effects in the regressions
reverses the sign of the point estimates, making them positive. Again, taking into
account the developments in the control group does not have any major impact on
the overall results.
With regard to return on assets, it can be noted that the estimated coefficients are
typically significantly negative in the pure before-after comparison of the company-
fixed effects model: Treatment companies experience lower increases in return-on-
assets after treatment relative to before treatment. This finding is not replicated in
the fully specified conditional diff-in-diff model, where coefficients get closer to
zero and are no longer statistically significant. This indicates that companies in the
control group also experience adverse return-on-assets developments in the years
after being chosen into the control group.
78
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0079.png
TABLE 5.5.c: Potential effects on value added (DKK1,000). Further results
Ordinary
least squares
regression
1
Firm fixed-effects
model
1
Conditional diff-in-
diff model
1
Conditional diff-in-
diff model,
dependent vari-
able: annual value
added growth in
percent
2
Coeff.
4.07
10.10**
5.59
12.12**
-2.81
Coeff.
Treat=1 & t=1
Treat=1 & t=2
Treat=1 & t=3
Treat=1 & t=4
Treat=1 & t=5
421.702***
239.746*
57.844
-90.041
-884.891**
Ste.
113.743
134.653
186.946
305.445
353.632
Coeff.
260.056
181.428
237.428
396.622
-72.243
Ste.
214.888
281.134
384.165
514.582
630.232
Coeff.
219.298
374.106
165.199
124.020
-563.130
Ste.
217.152
239.569
324.385
448.158
580.313
Ste.
3.58
4.30
4.27
5.18
7.20
Includes firm-
fixed effects
Includes year
dummy variables
Includes
information from
before year zero
Includes
observations of
the control group
no
no
no
yes
yes
yes
yes
yes
yes
yes
yes
yes
no
no
yes
yes
Number of
observations:
Number of
companies:
R2:
620
272
0.03
1346
272
0.02
2611
533
0.03
2223
451
0.03
Notes: Only observations with annual changes in the number of employees of less than 12. *, **, *** denote statistical significance at the 10%, 5%,
and 1% level.
1: Only observations with annual change in value added of less than DKK 10 mio.
2: Only observations with annual growth between -50 and 100 percent.
79
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0080.png
TABLE 5.5.d: Potential effects on net income (DKK1,000). Further results
Ordinary least squares
regression
Coeff.
Treat=1 & t=1
Treat=1 & t=2
Treat=1 & t=3
Treat=1 & t=4
Treat=1 & t=5
-48.481
-41.199
51.065
-36.775
-277.485
Firm fixed-effects model
Coeff.
6.965
69.773
229.337
262.090
171.135
Conditional diff-in-diff
model
Coeff.
-48.457
136.506
133.285
205.536
-103.538
Ste.
48.225
67.556
81.233
126.978
152.530
Ste.
98.949
130.681
166.055
227.975
267.157
Ste.
95.159
111.445
122.101
218.128
189.155
Includes firm-fixed effects
Includes year dummy
variables
Includes information from
before year zero
Includes observations of the
control group
no
no
no
no
yes
yes
yes
no
yes
yes
yes
yes
Number of observations:
Number of companies:
R2:
600
276
0.03
1322
276
0.02
2553
542
0.02
Notes: Only observations with annual changes in the number of employees of less than 12. Only observations with annual change in net income of
less than DKK 3 mio.
80
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0081.png
TABLE 5.5.e: Potential effects on return on assets (profits over total assets). Further results
Ordinary least squares
regression
Coeff.
Treat=1 & t=1
Treat=1 & t=2
Treat=1 & t=3
Treat=1 & t=4
Treat=1 & t=5
-0.027*
-0.039***
-0.017
-0.036*
-0.099**
Firm fixed-effects
model
Coeff.
-0.038*
-0.066**
-0.047
-0.082*
-0.141**
Conditional diff-in-diff
model
Coeff.
-0.029
-0.036
-0.001
-0.036
-0.042
Ste.
0.013
0.015
0.021
0.021
0.046
Ste.
0.023
0.030
0.036
0.045
0.069
Ste.
0.023
0.026
0.028
0.039
0.060
Includes firm-fixed effects
Includes year dummy
variables
Includes information from
before year zero
Includes observations of
the control group
no
no
no
no
yes
yes
yes
no
yes
yes
yes
yes
Number of observations:
Number of companies:
R2:
630
277
0.04
1361
277
0.01
2669
544
0.01
Notes: Only observations with annual changes in the number of employees of less than 12. *, **, *** denote statistical significance at the 10%, 5%,
and 1% level.
Only observations with annual change in return on assets of less than 1, and total assets
>
DKK 100,000.
Potential effects on average wage costs and labour productivity
Results for average wage costs and labour productivity (measured as turnover per
employee) are in the rightmost columns of TABLE 5.4, and illustrated in FIGURE
5.4.
34
With regard to the average wage costs per employee, it appears that any
potential treatment effects are too small relative to the variation in the data and
the number of observations. TABLE 5.5.f suggests that on average there are no
substantial changes in wage cost per employee after treatment, a finding which is
unaltered by the before-after comparisons for the subsample of treatment companies,
or when considering growth rates rather than absolute changes.
The variable ‘ wage cost per employee’ is from the balance sheet information of the KOB/Experian database,
and is characterised by a share of missing observations.
34
81
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0082.png
Labour productivity is measured as turnover per employee.
35
For absolute changes
in labour productivity, it is not possible to demonstrate that VP-companies have
higher productivity increases than the highly similar companies in the control
group: Negative signs for t>2 indicate that VP-companies have lower increases than
their counterparts in the reference group. However, this finding is not statistically
significant and thus highly tentative. The picture also changes when we consider
annual percentage-point growth in labour productivity rather than absolute annual
increases: In this model specification, treatment companies generally outperform
control companies in terms of labour productivity growth.
This finding – that treatment companies on average perform better than controls in
terms of percentage-point growth and not significantly better in terms of absolute
increases – implies that results are not robust with regard to model reformulation.
This should advise us against drawing too strong conclusions on the basis of the
statistical results. However, the fact that treatment companies seem to perform best
when the performance is measured in percentage-point growth rather than absolute
increases is an indication that it is in particular small companies that gain the most
from programme participation.
FIGURE 5.4: Wage and labour productivity developments (DKK1,000). Aggregated
estimated model coefficients. Years after treatment on horisontal axis.
100
0
-100
-200
-300
-400
0
1
2
3
4
5
Wage cost per employee
Turnover per employee
Turnover is from the Statistics Denmark registers instead of the Experian data. This is because (a) only compa-
nies above certain size thresholds are obliged to report this variable to the public authorities (which is why it is of-
ten missing in the Experian database) and (b) turnover is found for almost all companies in the Statistics Denmark
registers (because VAT is registered for almost all companies).
35
82
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0083.png
TABLE 5.5.f: Potential effects on wage cost (DKK1,000) per employee. Further results
Ordinary least
squares regression
1
Firm fixed-effects
model
1
Conditional diff-in-
diff model
1
Conditional diff-in-
diff model,
dependent vari-
able: growth of
wage cost per em-
ployee in percent
2
Coeff.
-3.90
-0.43
-4.95
1.36
-5.15
Coeff.
Treat=1 & t=1
Treat=1 & t=2
Treat=1 & t=3
Treat=1 & t=4
Treat=1 & t=5
3.20
14.98*
-6.34
16.04
5.94
Ste.
5.94
7.57
9.74
12.27
13.15
Coeff.
-8.96
0.41
-17.40
7.09
-26.35
Ste.
10.86
15.59
21.95
24.88
35.12
Coeff.
8.28
5.24
-21.34
16.06
-19.32
Ste.
11.26
10.27
13.79
17.18
29.92
Ste.
2.94
2.41
3.52
4.50
6.70
Includes firm-fixed
effects
Includes year dummy
variables
Includes information
from before year
zero
Includes
observations of the
control group
no
no
no
yes
yes
yes
yes
yes
yes
yes
yes
yes
no
no
yes
yes
Number of
observations:
Number of
companies:
R2:
355
190
0.02
794
190
0.02
1494
346
0.01
1474
343
0.01
Notes: Only observations with annual changes in the number of employees of less than 12. *, **, *** denote statistical significance at the 10%, 5%,
and 1% significance level.
1: Only observations with number of employees
>
5. Only observations with change in average wage
<
DKK 500,000.
2: Only observations with annual growth between -50 and 100 percent.
83
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0084.png
TABLE 5.5.g: Potential effects on labour productivity (DKK 1,000). Further results
Ordinary
least squares
regression
1
Firm fixed-
effects model
1
Conditional diff-in-
diff model
1
Conditional diff-in-
diff model,
dependent vari-
able: annual la-
bour productivity
growth in percent
2
Coeff.
2.50
6.64*
2.90
4.26
-7.57
Coeff.
Treat=1 & t=1
Treat=1 & t=2
Treat=1 & t=3
Treat=1 & t=4
Treat=1 & t=5
-67.67
146.02*
-70.54
-43.03
-133.16
Ste.
53.38
78.44
69.29
103.33
83.80
Coeff.
-78.40
127.63
-45.92
-46.98
-115.44
Ste.
85.83
128.59
124.43
176.48
202.26
Coeff.
-27.92
57.28
-137.30
-39.23
-159.68
Ste.
93.59
107.95
91.84
134.02
215.28
Ste.
3.71
3.72
4.07
5.41
11.12
Includes firm-fixed
effects
Includes year dummy
variables
Includes information
from before year zero
Includes observations of
the control group
no
no
no
no
yes
yes
yes
no
yes
yes
yes
yes
yes
yes
yes
yes
Number of observations:
Number of companies:
R2:
369
171
0.02
898
171
0.02
1693
323
0.02
2186
483
0.02
Notes: Only observations with annual changes in the number of employees of less than 12. *, **, *** denote statistical significance at the 10%, 5%,
and 1% significance level.
1. Only observations with number of employees
>
5 and change in labour productivity
<
DKK 3 mio.
2: Only observations with annual growth between -50 and 100 percent.
84
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0085.png
Results for subsamples
In the following, we look at whether previous findings are different for different
industries, VP- and project-specific characteristics.
This functions as a robustness check of the previous results, but it also offers an
opportunity to see under what circumstances the programme might be considered to
be most successful. In particular, the previous regression models will be applied on
the following samples:
1. All companies, with no outliers removed.
2. Only companies where the DASTI and Statistics Denmark data are in
accordance with regard to the company-VP match.
3 Only VP-projects that were not aborted before schedule.
4. Only companies without any tertiary-level educated employees in the year
prior to programme participation.
5. Only VP-projects in, respectively, manufacturing, services, and other
industries.
6. Only male VPs, only female VPs.
7. Only VPs with a tertiary education.
8. Only VPs with education degrees in, respectively, arts and humanities,
social sciences, and technical sciences subjects.
For ease of reading, the results can be found in the appendix of this report.
Aggregated regression coefficients, which measure potential treatment effects, are
for most of the subsamples illustrated graphically and discussed below.
Let us first turn our attention to the results for the sample of all companies, with
no outliers removed. For this sample, estimated standard errors are often much
larger than the absolute sizes of the coefficient estimates (TABLE A.1). Thus, for
all participant companies (including the larger ones), it is not possible to make
statements on the potential treatment effects with any degree of accuracy, with the
exception of the employment of highly educated employees.
85
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0086.png
Potential effects on the number of highly educated employees
We find the largest potential treatment effects for companies without tertiary-level
educated employees in the year prior to treatment, and for companies hiring male
VPs, and for those hiring VPs with a technical sciences education. The lowest
potential effects are found for those hiring VPs with an education in arts and
humanities, and, especially over a time horizon beyond the very first years after
treatment, female VPs. There is only a small immediate potential effect for service
industries. However, companies in these industries increase the number of highly
educated employees in the years after treatment.
FIGURE 5.5.a: Number of highly educated employees. Estimated potential treatment
effects. Years after year zero on horisontal axis.
1.2
1.0
0.8
0.6
0.4
0.2
0.0
0
1
2
3
4
5
Companies w/o
tertiary-level educat-
ed prior to treatment
Only completed projects
Agreement on VP-
company match in
DASTI and DST data
FIGURE 5.5.b: Number of highly educated employees. Estimated potential treatment
effects. Years after year zero on horisontal axis.
1.2
1.0
0.8
0.6
0.4
0.2
0.0
0
1
2
3
4
5
Manufacturing and
construction indus-
tries
Services
Other industries
86
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0087.png
FIGURE 5.5.c: Number of highly educated employees. Estimated potential treatment
effects. Years after year zero on horisontal axis.
1.5
1.0
0.5
0.0
0
-0.5
-1.0
1
2
3
4
5
VP is male
VP is female
FIGURE 5.5.d: Number of highly educated employees. Estimated potential treatment
effects. Years after year zero on horisontal axis.
1.5
1.0
0.5
0.0
-0.5
-1.0
-1.5
-2.0
0
1
2
3
4
5
VP has education in
technical sciences
VP has education in
social sciences
VP has education in
arts&humanities
87
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0088.png
Potential effects on the number of employees
For total employment, it proves to be important that the VP-project is completed and
not aborted before schedule. Again, it is companies that hire female VPs and VPs
with an education in arts and humanities that have the poorest growth performance.
For VPs with a technical sciences education, a positive potential programme effect
for highly educated employees and the absence of any detectable potential effect
for employees of all educations indicate that companies that hire these VPs would
have employed other individuals with lower educations in the counterfactual case of
non-participation.
FIGURE 5.6.a: Number of employees. Estimated potential treatment effects. Years
after year zero on horisontal axis.
2.0
1.5
1.0
0.5
0.0
-0.5
-1.0
-1.5
0
1
2
3
4
5
Companies without
highly educated
employees prior to
treatment
Agreement in VP-com-
pany match in DASTI
and DST data
Completed VP projects
FIGURE 5.6.b: Number of employees. Estimated potential treatment effects. Years
after year zero on horisontal axis.
1.5
1.0
0.5
0.0
-0.5
-1.0
-1.5
0
1
2
3
4
5
Manufacturing and
construction industries
Services
Other industries
88
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0089.png
FIGURE 5.6.c: Number of employees. Estimated potential treatment effects. Years
after year zero on horisontal axis.
3.0
2.0
1.0
0.0
-1.0
-2.0
-3.0
-4.0
0
1
2
3
4
5
VP is male
VP is female
FIGURE 5.6.d: Number of employees. Estimated potential treatment effects. Years
after year zero on horisontal axis.
2.0
1.5
1.0
0.5
0.0
-0.5
-1.0
-1.5
-2.0
0
1
2
3
4
5
VP has education in
technical sciences
VP has education in
social sciences
VP has education in
arts&humanities
89
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0090.png
Potential effects on value added
The comparison of potential value added effects agrees to large extent with the
findings for employment: Subgroups of companies that are characterised by low
average increases in the number of (highly educated) employees in association with
programme participation are also characterised by low increases in value added.
This is notably the case for companies hiring female VPs and VPs with an education
categorised as within arts and humanities. The highest average increases are found
in the manufacturing industries and for VPs with a social sciences-related education.
With regard to value added, it is again important that the project was completed,
while there is no indication that companies without tertiary educated employees
prior to treatment gain the most in terms of value added.
FIGURE 5.7.a: Value added (DKK1,000). Estimated potential treatment effects.
Years after year zero on horisontal axis.
1500
1000
500
0
0
-500
-1000
1
2
3
4
5
Companies without
highly educated
employees prior to
treatment
Agreement on VP-com-
pany match in DASTI
and DST data
Completed VP projects
FIGURE 5.7.b: Value added (DKK1,000). Estimated potential treatment effects.
Years after year zero on horisontal axis.
2500
2000
1500
1000
500
0
-500
-1000
0
1
2
3
4
5
Manufacturing and
construction industries
Services
Other industries
90
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0091.png
FIGURE 5.7.c: Value added (DKK1,000). Estimated potential treatment effects.
Years after year zero on horisontal axis.
2000
1500
1000
500
0
-500
-1000
0
1
2
3
4
5
VP is male
VP is female
FIGURE 5.7.d: Value added (DKK1,000). Estimated potential treatment effects.
Years after year zero on horisontal axis (year zero=100).
3000
2000
1000
0
-1000
-2000
0
1
2
3
4
5
VP has education in
technical sciences
VP has education in
social sciences
VP has education in
arts&humanities
91
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0092.png
Potential effects on net income (profits) and return on assets
What was true for the developments of value added does not necessarily hold
true for net income. For example, companies hiring female VPs are on average
not characterised by less favourable net income developments. It can be noted
that companies without tertiary-level educated employees prior to treatment and
companies hiring VPs with a technical sciences education do best in terms of
return-on-assets developments.
FIGURE 5.8.a: Net income (DKK1,000). Estimated potential treatment effects. Years
after year zero on horisontal axis.
500
400
300
200
100
0
-100
0
1
2
3
4
5
Companies without
highly educated
employees prior to
treatment
Agreement on VP-
company match in
DASTI and DST data
Completed VP projects
FIGURE 5.8.b: Net income (DKK1,000). Estimated potential treatment effects. Years
after year zero on horisontal axis.
700
600
500
400
300
200
100
0
-100
-200
Manufacturing and
construction industries
Services
Other industries
0
1
2
3
4
5
92
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0093.png
FIGURE 5.8.c: Net income (DKK1,000). Estimated potential treatment effects. Years
after year zero on horisontal axis.
1000
800
600
400
200
0
-200
0
1
2
3
4
5
VP is male
VP is female
FIGURE 5.8.d: Net income (DKK1,000). Estimated potential treatment effects. Years
after year zero on horisontal axis.
1000
500
0
-500
-1000
-1500
-2000
0
1
2
3
4
5
VP has education in
technical sciences
VP has education in
social sciences
VP has education in
arts&humanities
93
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0094.png
FIGURE 5.9.a: Return on assets. Estimated potential treatment effects. Years after
year zero on horisontal axis.
0.00
0
-0.05
-0.10
-0.15
-0.20
-0.25
1
2
3
4
5
Companies without
highly educated
employees prior to
treatment
Agreement on VP-
company match in
DASTI and DST data
Completed VP projects
FIGURE 5.9.b: Return on assets. Estimated potential treatment effects. Years after
year zero on horisontal axis.
0.00
-0.05
-0.10
-0.15
-0.20
-0.25
-0.30
-0.35
0
1
2
3
4
5
Manufacturing and
construction industries
Services
Other industries
94
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0095.png
FIGURE 5.9.c: Return on assets. Estimated potential treatment effects. Years after
year zero on horisontal axis.
0.00
-0.05
-0.10
-0.15
-0.20
-0.25
0
1
2
3
4
5
VP is male
VP is female
FIGURE 5.9.d: Return on assets. Estimated potential treatment effects. Years after
year zero on horisontal axis.
0.00
-0.10
-0.20
-0.30
-0.40
-0.50
-0.60
0
1
2
3
4
5
VP has education in
technical sciences
VP has education in
social sciences
VP has education in
arts&humanities
95
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0096.png
Potential effects on wages and labour productivity
The comparison of wage costs per employee leaves us with no clear results. Instead,
erratic movements in the estimates over time suggest that these are mostly due to
statistical noise rather than underlying trends.
For labour productivity, we find that companies in other industries than
manufacturing and services, and companies that hire VPs with technical educational
degrees, do well relative to other companies. Those that hire VPs with an
educational background in arts and humanities, and those in the service industry, are
characterised by the most negative estimates.
FIGURE 5.10.a: Average wage cost per employee (DKK1,000). Estimated potential
treatment effects. Years after year zero on horisontal axis.
40
30
20
10
0
-10
-20
-30
0
1
2
3
4
5
Companies without
highly educated
employees prior to
treatment
Agreement on VP-
company match in
DASTI and DST data
Completed VP projects
FIGURE 5.10.b: Average wage cost per employee (DKK1,000). Estimated potential
treatment effects. Years after year zero on horisontal axis.
50
40
30
20
10
0
-10
-20
-30
-40
Manufacturing and
construction industries
Services
0
1
2
3
4
Other industries
96
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0097.png
FIGURE 5.10.c: Average wage cost per employee (DKK1,000). Estimated potential
treatment effects. Years after year zero on horisontal axis.
80
60
40
20
0
-20
-40
-60
0
1
2
3
4
5
VP is male
VP is female
FIGURE 5.10.d: Average wage cost per employee (DKK1,000). Estimated potential
treatment effects. Years after year zero on horisontal axis.
60
40
20
0
0
-20
-40
1
2
3
4
5
VP has education in
technical sciences
VP has education in
social sciences
VP has education in
arts&humanities
97
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0098.png
FIGURE 5.11.a: Labour productivity (DKK1,000). Estimated potential treatment
effects. Years after year zero on horisontal axis.
200
0
-200
-400
-600
-800
-1000
-1200
0
1
2
3
4
5
Companies w/o
tertiary-level educated
prior to treatment
Agreement on VP-
company match in
DASTI and DST data
Completed VP projects
FIGURE 5.11.b: Labour productivity (DKK1,000). Estimated potential treatment
effects. Years after year zero on horisontal axis.
400
200
0
-200
-400
-600
-800.
0
1
2
3
4
5
Manufacturing and
construction industries
Services
Other industries
98
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0099.png
FIGURE 5.11.c: Labour productivity (DKK1,000). Estimated potential treatment
effects. Years after year zero on horisontal axis.
200
0
-200
-400
-600
-800
-1000
VP is male
0
1
2
3
4
5
VP is female
FIGURE 5.11.d: Labour productivity (DKK1,000). Estimated potential treatment
effects. Years after year zero on horisontal axis.
500
0
0
-500
-1000
-1500
1
2
3
4
5
VP has education in
technical sciences
VP has education in
social sciences
VP has education in
arts&humanities
99
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0100.png
6.
EXTENSIONS
The survival of VP-companies
As a first extension of the analysis, we look at the survival/closure rate of VP-
companies in comparison with the reference group of control companies. This
is achieved by simply comparing closure rates as depicted in Figure 6.1 and an
estimation of a binary choice model which has company closure in a given year
as its dependent variable.
36
The results of this regression are displayed in TABLE
6.1 and corroborate the finding that there are no significant differences between
companies that hire VPs and other similar companies that do not participate in the
programme.
FIGURE 6.1: Company closure rates, by year after year 0 (horizontal axis).
0.05
0.04
0.03
0.02
0.01
0.00
TREATMENT
CONTROL
-5 -4 -3 -2
1
0
1
2
3
4
5
Closure is measured between year t and year t+1, where year t is the last year in which the company is found in
the Experian database. The Experian database has information on the status of companies that allow distinguish-
ing company closures from, for example, company sales or mergers.
36
100
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0101.png
TABLE 6.1: Comparison of company closure probabilities of VP-companies and companies in the
reference group. Logit binary choice regression results. Dependent variable: bankruptcy after t=x.
Dependent variables (in
first differences):
All companies in treatment and
control group
Coeff.
Ste.
0.49
0.47
0.83
0.61
0.78
Companies with less than 50
employees
Coeff.
0.67
-0.20
1.09
0.41
0.29
Ste.
0.51
0.50
0.83
0.66
0.78
Treat=1 & t=1
Treat=1 & t=2
Treat=1 & t=3
Treat=1 & t=4
Treat=1 & t=5
t=1 (omitted category)
t=2
t=3
t=4
t=5
Year dummies
2005 (omitted category)
2006
2007
2008
2009
2010
Constant
Number of observations:
Pseudo-r-squared
0.54
-0.15
1.11
0.30
0.25
0.43
-1.33
0.12
0.93
0.51
0.82
0.61
0.76
0.48
-1.17
0.01
1.05
0.55
0.83
0.67
0.78
0.13
1.32
2.24
0.98
-0.04
-4.82
1987
0.08
1.17
1.07
1.05
1.07
1.13
1.05
0.13
1.24
2.16
0.99
-0.26
-4.816
1876
0.08
1.17
1.08
1.05
1.08
1.15
1.054
For one of the extensions of the analysis, DASTI provided data on companies that
have participated in the so-called
Innovation Networks.
These networks or clusters
are financially supported by DASTI and have the purpose of increasing knowledge
diffusion by providing a platform for collaborations between companies, knowledge
institutions and other cluster participants.
A comparison of
VP-companies
and companies participating in
Innovation Networks
101
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0102.png
These data consist of 1923 observations belonging to 1158 companies, the
discrepancy owing itself to the fact that some firms participate in these networks
more than once. In the following, these companies’
(IN-companies)
performance is
compared with the performance of the VP-companies.
First, we compare developments in some of the performance variables between
IN- and VP-companies. This comparison is highly informal since the two groups
of companies differ in observable characteristics and must be assumed to differ in
unobservable characteristics as well.
The left hand side columns of TABLE 6.2 compare VP-companies with all IN-
companies present in the Experian data that participated in the clusters after 2004.
We initially find that IN companies are on average significantly larger and have
more highly educated employees than VP-companies. Also, a larger share of the IN-
companies are in the IT industry.
To increase the comparability of the two groups of companies for the subsequent
comparisons, only companies with a net income between DKK -7 million and DKK
7 million and a maximum size of 50 employees in the year before treatment (which
is roughly the 99% percentile of the VP-companies’ distribution of this variable) are
considered.
Summary statistics of the adjusted sample used for the statistical comparison are in
the right hand side columns of TABLE 6.2. The adjustments in terms of company
size and profit have made the two groups of companies surprisingly similar in their
observable characteristics in the year before treatment, with the exception that IN-
companies are characterised by a higher share of highly educated employees.
The results of the new comparison are shown in TABLE 6.3 and are in concordance
with earlier findings based on the comparison of VP-companies with a reference
group of highly similar companies: VP-companies increase their numbers of highly
educated employees in the year of treatment and sometimes in the first years after
treatment.
However, it cannot be shown that VP-companies grow faster than IN-companies in
the number of employees. On the contrary, they appear to have lower growth, i.e.
shrink faster, than IN-companies more than three years after treatment. Additional
regressions (not shown) further indicate that this finding becomes even more
accentuated when considering percentage point employment growth rather than
absolute increases in the number of employees.
Findings also suggest that VP-companies have a lower growth in value added
and net income, but these findings are generally not statistically significant. VP-
companies have wage developments and labour productivity (turnover/employees)
developments approximately equal to the group of IN-companies in most years after
treatment and higher in single years.
102
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0103.png
In sum, earlier findings that VP-companies do not have statistically significant
higher increases in the set of financial success variables relative to the reference
group of highly similar companies are replicated in the comparison with a sample of
small companies that have participated in an Innovation Network.
TABLE 6.2: Summary statistics of companies participating in Innovation Networks (IN) vs. VP-
companies, in year t=0.
Raw data
VP-companies
N=314
Mean
Number of highly educated
employees
Number of employees
Turnover (DKK1,000)
Value added (DKK1,000)
Net income (DKK,1,000)
Return on assets
Labour productivity
(DKK1,000)
Wage cost per employee
(DKK1,000)
Industry: Construction
Industry: Trade
Industry: IT
Industry: Manufacturing
Industry: Metal
Industry: Furniture
Industry: Service
Industry: Business service
Industry: Consulting
Industry: Wood/paper
Industry: Other
2.42
17.45
43682.70
7896.05
2129.82
0.28
4898.19
218.67
0.23
0.41
0.29
0.25
0.22
0.21
0.19
0.23
0.32
0.18
0.42
Comparison sample
1
VP-companies
N=297
Std.
239.96
1043.84
3897574.00
793678.70
591004.40
0.52
2676.78
387.54
0.14
0.38
0.33
0.24
0.15
0.23
0.15
0.16
0.31
0.18
0.48
IN-companies
N=828
Mean
58.98
246.88
598268.60
130868.60
26234.95
-0.06
2063.51
477.04
0.02
0.18
0.13
0.06
0.02
0.06
0.02
0.03
0.11
0.03
0.35
IN-companies
N=479
Mean
4.34
14.12
24178.86
7304.33
213.33
-0.07
1821.75
447.00
0.01
0.20
0.15
0.05
0.02
0.05
0.02
0.04
0.13
0.04
0.29
Std.
2.42
17.45
43682.70
7896.05
2129.82
0.28
4898.19
218.67
0.23
0.41
0.29
0.25
0.22
0.21
0.19
0.23
0.32
0.18
0.42
Mean
1.72
11.22
19600.08
5234.49
357.50
0.03
2175.00
396.27
0.06
0.23
0.10
0.07
0.05
0.04
0.04
0.06
0.12
0.03
0.22
Std.
2.19
11.07
40120.67
5547.89
1311.80
0.24
5010.00
220.95
0.23
0.42
0.30
0.26
0.21
0.20
0.19
0.23
0.33
0.16
0.41
Std.
6.54
13.71
29720.16
8648.45
1871.56
0.55
1793.93
205.25
0.11
0.40
0.35
0.21
0.15
0.21
0.14
0.20
0.34
0.19
0.45
Notes: The comparison sample consists of companies with maximum 50 employees and net income between DKK 7 million and DKK 7 million in year
zero.
103
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0104.png
TABLE 6.3: Diff-in-diff fixed effects regression results for VP- and IN-companies. Companies with up
to 50 employees in year zero.
Dependent variables
(in first
differences):
Treat=1 & t=1
Treat=1 & t=2
Treat=1 & t=3
Treat=1 & t=4
Treat=1 & t=5
t=1
t=2
t=3
t=4
t=5
Year dummies
2005
2006
2007
2008
2009
2010
2011
Number of highly
educated employees
1
Coeff.
0.508***
0.33
0.502*
-0.34
0.96
-0.05
0.07
-0.506**
0.41
-0.68
Number of employees
Value added
(DKK1,000)
2
Coeff.
174.4
-455.1*
-55.7
-156.5
-15.4
255.2
522.9**
328.4
855.2**
786.4
Ste.
0.17
0.22
0.28
0.35
0.87
0.13
0.18
0.25
0.32
0.84
Coeff.
0.24
-0.19
-0.53
-1.285**
-0.91
0.02
0.29
0.31
0.873*
0.57
Ste.
0.29
0.35
0.44
0.55
0.80
0.22
0.29
0.40
0.51
0.76
Ste.
214.2
255.5
315.7
386.7
491.2
165.7
218.3
290.7
373.0
487.8
0.15
0.08
0.173*
-0.07
-0.391***
0.10
0.10
0.10
0.11
0.13
0.07
0.347*
0.372**
0.01
-1.342***
-1.115***
-0.72
0.18
0.18
0.19
0.20
0.24
0.33
0.99
-211.1
50.5
-191.7
-662.9***
-1652***
-1252***
498.1
143.1
148.6
151.5
162.3
189.8
239.2
663.0
Constant
Number of observations:
R-squared
Number of companies:
0.052
3208
0.03
698
0.07
0.572***
3706
0.06
743
0.13
731.9***
4127
0.05
754
104.6
Notes: Only observations with annual changes in the number of employees by less than 12. *, **, *** denote statistical significance at the 10%, 5%,
and 1% significance level.
1. Employees with post-secondary or tertiary education. Only observations with annual changes in the number of employees with post-secondary
and tertiary education
<
5.
104
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0105.png
Net income (DKK1,000)
3
Return on assets
4
Wage per employee
(DKK1,000)
5
Coeff.
-17.41
-8.09
14.72
11.02
10.40
-9.66
5.65
-21.34
19.84
-11.45
Labour productivity
(DKK1,000)
6
Coeff.
92.24
127.60
987.1***
-108.80
425.60
13.37
21.82
-1073***
281.20
-661.30
Coeff.
-433.8
-110.1
-279.2
-137.9
-580.6
687.5
413.5
486.4
352.7
844.3
Ste.
760.0
904.9
1116.0
1365.0
1730.0
581.8
767.8
1025.0
1313.0
1717.0
Coeff.
-0.0575**
0.00
0.01
-0.123**
-0.09
0.02
-0.01
0.02
0.04
0.05
Ste.
0.03
0.03
0.04
0.05
0.06
0.02
0.03
0.04
0.05
0.06
Ste.
16.00
18.91
23.58
28.84
41.93
11.81
15.32
21.11
26.66
39.18
Ste.
234.40
298.60
375.40
465.40
1855.00
171.60
239.80
326.80
416.30
1819.00
-41.2
-883.1*
-413.5
-944.5*
-980.4
-663.4
574.5
506.2
523.9
533.2
571.1
665.2
840.9
2357.0
-0.01
-0.03
-0.0569***
-0.0567***
-0.0993***
-0.04
0.04
0.02
0.02
0.02
0.02
0.02
0.03
0.08
1.93
-6.76
8.95
4.42
-1.30
12.71
28.23
9.26
9.64
9.91
10.62
12.38
16.92
49.73
-138.40
-172.50
-331.2**
-236.2*
-202.40
124.40
129.00
132.00
141.90
166.60
46.950
4224
0.0
769
368.7
0.0216*
4222
0.02
764
0.01
3.75
1917
0.02
515
6.81
116.20
2301
0.01
459
89.61
2. Only observations with annual change in the value added of less than DKK 10 mio.
3. Only observations with annual change in net income of less than DKK 3 mio.
4. Only observations with annual change in roa of less than 1, and total assets
>
DKK 100,000.
5. Only observations with number of employees
>
5. Only observations with change in average wage
<
DKK 500,000.
6. Only observations with number of employees
>
5 and change in labour productivity
<
DKK 3 mio.
105
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0106.png
A comparison of VP-companies and an extended sample of control
companies
Earlier findings were characterised by a large heterogeneity in the companies’
financial success variables. This argues for a test of the robustness of earlier findings
by running the same regressions on a larger control group of companies. This
reduces the variance of the estimators but comes at the cost of lower similarity
between the group of treatments and the group of controls.
In the following, we depart from the propensity scores calculated earlier and select
five controls (instead of one) for each treatment into the control group. This time,
matching is based purely on the propensity score, without additional conditions on
industry etc.
This procedure selects 1,596 companies into the control group for the 318
participant companies. The similarity of the two groups can be assessed by
inspecting TABLE 6.4. As expected, the two groups are not as similar as the sample
of the earlier analysis, with e.g. slightly larger companies in the extended control
group. However, the conditional diff-in-diff model still allows for a meaningful
comparison between the two groups of companies, and its estimates should be less
affected by statistical noise thanks to an increase in the sample size.
106
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0107.png
TABLE 6.4: Summary statistics of VP-companies and companies in the
extended control group (5 controls per treatment). Companies with up to 50
employees.
VP-companies
N=300
Mean
Number of highly educated
employees
Number of employees
Turnover (DKK1,000)
Value added (DKK1,000)
Net income (DKK,1,000)
Return on assets
Labour productivity (DKK1,000)
Wage cost per employee
(DKK1,000)
Industry: Construction
Industry: Trade
Industry: IT
Industry: Manufacturing
Industry: Metal
Industry: Furniture
Industry: Service
Industry: Business service
Industry: Consulting
Industry: Wood/paper
Industry: Other
1.80
11.65
18437.84
5540.32
296.99
0.04
2165.72
395.80
0.06
0.22
0.10
0.07
0.05
0.06
0.03
0.06
0.12
0.03
0.21
Companies in the
extended control group
N=1,488
Std.
2.26
11.36
Mean
1.78
11.16
18103.18
5249.33
236.41
0.03
2037.00
397.12
0.04
0.22
0.09
0.05
0.04
0.05
0.04
0.07
0.13
0.03
0.23
Std.
2.62
11.86
25721.50
6234.30
1139.21
0.23
3856.43
390.80
0.20
0.41
0.29
0.21
0.20
0.22
0.19
0.26
0.34
0.18
0.42
27033.64
5739.63
981.43
0.21
4973.50
220.14
0.23
0.41
0.30
0.25
0.21
0.23
0.18
0.24
0.33
0.16
0.41
The results of the new comparisons are in TABLE 6.5. We find that increasing the
sample size only marginally reduced the standard errors of the estimates, and that
the results of this model do not alter the previous findings that VP-companies do in
general not experience statistically significant positive developments in the financial
success variables. However, for return on assets, positive (though insignificant) signs
of the relevant coefficient estimates imply that the previous finding of treatment
companies that experience lower growth in this variable in association with
treatment is not robust to changes in how the control group is selected. Also, there
are weak signs that participants experience higher growth in wage costs and value
added, and lower growth in labour productivity.
107
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0108.png
TABLE 6.5: Diff-in-diff fixed effects regression results for VP-companies and companies in the
extended control group (5 controls per treatment). Companies with up to 50 employees in year zero.
Dependent variables
(in first
differences):
Treat=1 & t=1
Treat=1 & t=2
Treat=1 & t=3
Treat=1 & t=4
Treat=1 & t=5
t=1
t=2
t=3
t=4
t=5
Year dummies
2005
2006
2007
2008
2009
2010
2011
Number of highly
educated employees
1
Coeff.
0.510***
0.451***
0.06
-0.01
0.19
-0.03
-0.10
-0.179**
-0.09
-0.27
Number of employees
Value added
(DKK1,000)
2
Coeff.
235.8
33.7
-47.5
497.2**
-325.1
63.3
51.2
159.6
-11.5
164.0
Ste.
0.10
0.13
0.15
0.16
0.26
0.05
0.07
0.09
0.12
0.18
Coeff.
0.621***
0.25
0.11
-0.16
-0.975*
-0.17
-0.23
-0.29
-0.31
0.20
Ste.
0.21
0.27
0.28
0.41
0.57
0.12
0.16
0.21
0.25
0.31
Ste.
145.6
172.7
190.2
251.8
331.9
82.1
108.8
142.4
174.6
221.6
0.182***
0.173***
0.180***
0.02
0.00
0.05
0.05
0.07
0.08
0.10
0.455***
0.781***
0.645***
0.614***
-0.632**
-0.51
-2.452***
0.17
0.17
0.20
0.22
0.26
0.34
0.93
-11.5
302.9***
87.2
-343.7***
-925.4***
-633.2***
661.4
87.3
97.1
112.2
131.7
168.1
189.9
577.3
Constant
Number of observations:
R-squared
Number of companies:
-0.045
7130
0.03
1633
0.04
0.036
8088
0.07
1706
0.15
304.5***
8945
0.05
1709
77.4
Notes: Only observations with annual changes in the number of employees of less than 12. *, **, *** denote statistical significance at the 10%, 5%,
and 1% significance level.
1. Employees with post-secondary or tertiary education. Only observations with annual changes in the number of employees with post-secondary
and tertiary education
<
5.
108
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0109.png
Net income (DKK1,000)
3
Return on assets
4
Wage per employee
(DKK1,000)
5
Coeff.
3.64
16.17
24.43*
29.97**
28.57
6.08
0.49
2.75
5.91
11.66
Labour productivity
(DKK1,000)
6
Coeff.
-301.50
-251.70
-555.7**
-126.40
-282.50
334.80
414.90
491.6**
272.50
66.82
Coeff.
-85.9
-2.1
-100.0
29.7
-199.6
13.0
13.9
28.8
-72.2
152.1
Ste.
85.2
103.3
123.6
152.0
204.0
51.4
67.6
83.4
93.5
108.8
Coeff.
-0.02
0.06
0.06
0.00
0.01
-0.0248*
-0.08
-0.0555**
-0.0770***
-0.07
Ste.
0.03
0.07
0.04
0.05
0.06
0.02
0.06
0.02
0.03
0.04
Ste.
10.79
12.21
13.01
14.87
21.65
7.36
8.82
11.57
14.76
18.71
Ste.
248.00
304.70
272.90
205.40
493.90
248.30
349.10
222.50
188.70
430.30
114.7
129.1*
7.5
-174.4**
-221.3**
23.5
870.7***
69.8
74.9
74.8
86.0
94.5
109.4
289.8
0.0444**
0.0513**
0.03
0.01
-0.0436*
0.04
0.11
0.02
0.02
0.02
0.02
0.03
0.03
0.08
-7.87
-4.08
1.10
-16.05
-5.68
-17.47
82.72**
6.26
6.95
8.42
11.75
13.48
18.33
33.91
-189.0*
-183.60
-314.50
-365.6***
-272.3*
105.80
152.80
259.40
140.70
149.80
-26.130
7609
0.0
1494.0
63.0
-0.0326***
9094
0.01
1748.00
0.01
3.15
3106
0.02
1051.00
4.84
67.26
4500
0.00
960.00
77.79
2. Only observations with annual change in the value added of less than DKK 10 mio.
3. Only observations with annual change in net income of less than DKK 3 mio.
4. Only observations with annual change in roa of less than 1, and total assets
>
DKK100,000.
5. Only observations with number of employees
>
5. Only observations with change in average wage
<
DKK 500,000.
6. Only observations with number of employees
>
5 and change in labour productivity
<
DKK 3 mio.
109
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0110.png
7.
CONCLUSIONS
This study has taken a look at the potential effects of the Danish Innovation
Assistant Programme (‘Videnpilotordningen’, VP programme) on the individual and
company level. For this purpose, the analysis considers the employment probabilities
and salary developments of individuals participating in the programme (VPs) and
follows a number of performance variables for participating companies.
To form an understanding of the absolute potential effects of the programme, we
compare participating individuals and companies with highly similar individuals
and companies that do not participate. These comparisons indicate that:
(a) Individuals who participate in the programme have higher employment
probability than similar control individuals in the year after starting to
participate. This is no surprise, since employment is a defining element of
the programme.
(b) Individuals who participate in the programme do not have higher
employment probability than controls more than one year after starting to
participate in the programme, but earn higher wages in the first years.
Here it should be noted that the observation period falls within an
economic boom period with low unemployment. It might be assumed
that the wage and employment developments of programme participants
and non-participants do not converge at the same speed in the current
economic slow-down.
(c) Participating companies increase their numbers of highly educated
employees in association with programme participation. The analysis finds
no signs of behavioural additivity of the programme, i.e. non-
participants increasing their number of highly educated employees.
There are no indications that companies continue to increase the number of
highly educated employees in the years after programme participation.
(d) Participating companies increase the number of employees in association
with programme participation. However, in this case there are also no
indications that the companies continue to increase their employment in
the years after programme participation.
(e) It is difficult to detect statistically significant positive potential effects of
the programme on participating companies’ financial performance
variables. For subsamples of small companies that do not experience large
year-to-year changes in employment or financial measures, participant
companies on average increase their gross profit and net income
in association with programme participation. Findings are again not
statistically significant, and need to be interpreted with care.
(f) Participating companies do not experience increases in return on assets,
wage costs per employee, or labour productivity in association with
programme participation.
110
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0111.png
There are no strong findings about which particular projects are more successful
than others, but it appears that VPs with a tertiary-level education gain less from
participation than VPs with a post-secondary education, while females and VPs
finding employment in service industries gain the most.
For companies, it is important to note that results related to specific characteristics
of the VP or the hosting company are tentative, due to the presence of substantial
statistical uncertainties. This said, one can note that the largest potential effect on
the number of highly educated employees is estimated for small companies that hire
VPs with a technical sciences education as well as male VPs, and for companies that
had no tertiary-level educated employees before treatment.
Also, small companies in manufacturing do well in terms of value added and
net income (profits) developments in association with programme participation,
while participant companies that hire female VPs do relatively poorly in terms of
value added and employment, but not net income. Companies that hire VPs with
an educational background within arts and humanities are characterised by low
growth in association with programme participation, while those hiring VPs with a
technical sciences education do the best, not just in terms of increasing the number
of highly educated employees, but also with regard to net income, return on assets
and labour productivity developments.
The general finding that it is difficult to measure statistically significant potential
effects of the programme proved to be robust to comparing participant companies
with other companies that participated in a similar programme administered by
DASTI (the
Innovation Network
programme) as well as an alternative control group
consisting of several highly comparable control companies for each participant
company.
The VP programme has been analysed earlier on the basis of less extensive data.
This earlier study found potential effects of similar size to the present study.
However, it also found large unexplained year-to-year variation in the performance
variables, leading to statistically insignificant coefficient estimates.
The current analysis supports the earlier analysis’ findings. But the fact that it is still
difficult or impossible to establish statistical significance for most of the relevant
financial variables implies that we still cannot be certain that increased company
performance is a general feature of the programme.
111
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0112.png
So while there are indications of positive potential programme effects for restricted
subsamples in our data, the general lack of statistical significance implies that any
positive effects of hiring a VP on company performance are small in the face of
the high data demands of our econometric model, a still very limited number of
observations in our data, and the large variation in the companies’ performance
measures. The latter observation also suggests that other company developments, for
example initiated by product developments, must be assumed often to be of major
importance relative to the presence of a VP in the company.
37
Fox, J.T., V. Smeets, 2011, Does Input Quality Drive Measured Differences In Firm Productivity?, International
Economic Review, vol. 52(4), 961-989.
37
112
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0113.png
113
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0114.png
APPENDIX 1: ADDITIONAL TABLES OF THE COMPANY-LEVEL ANALYSIS
TABLE A.1: Comparison between VP-companies and companies in the reference group. All companies
irrespective of outliers. Diff-in-diff fixed effects regression results
Dependent
variables
(in first
differences):
Treat=1 & t=1
Treat=1 & t=2
Treat=1 & t=3
Treat=1 & t=4
Treat=1 & t=5
t=1
t=2
t=3
t=4
t=5
Year dummies
2003
2004
2005
2006
2007
2008
2009
Number of highly
educated employees
1
Coeff.
0.289*
0.16
-0.17
-0.380*
0.29
-0.09
-0.07
-0.23
-0.05
-0.59
Number of employees
Value added (DKK1,000)
2
Ste.
0.17
0.15
0.20
0.23
0.70
0.12
0.15
0.20
0.25
0.64
Coeff.
0.14
0.25
0.63
-0.30
-0.88
-0.41
-0.51
-0.60
-0.99
0.68
Ste.
0.74
0.52
0.55
0.86
1.41
0.45
0.63
0.80
1.09
1.35
Coeff.
48.9
73.5
66.4
-43.8
-558.7
78.8
-394.6
-103.4
-13.9
109.0
Ste.
302.4
244.8
313.4
473.8
452.6
246.0
288.3
418.1
451.8
521.9
0.08
0.02
0.02
0.17
0.14
0.00
-0.10
0.15
0.11
0.12
0.16
0.18
0.23
0.22
-0.33
0.49
0.47
1.185*
0.56
0.22
-2.647**
0.53
0.52
0.55
0.61
0.74
0.89
1.07
-382.7
219.7
361.8
446.4
251.4
-207.5
-1024.0
494.3
492.3
506.6
522.5
571.0
592.5
654.6
Constant
Number of
observations:
Number of companies:
R-squared
0.12
3046
596
0.02
0.10
0.53
2989
580
0.08
0.50
338.2
3664
611
0.03
481.1
Notes: Only observations with annual changes in the number of employees of less than 12. *, **, *** denote statistical significance at the 10%, 5%,
and 1% significance level.
1. Employees with post-secondary or tertiary education. Only observations with annual changes in the number of employees with post-secondary
and tertiary education
<
5.
114
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0115.png
Net income (DKK1,000)
3
Return on assets
4
Wage per employee
(DKK1,000)
5
Coeff.
44.45
-20.82
16.87
30.80
44.40*
3.72
38.96
25.52
-16.65
-25.41
Labour productivity
(DKK1,000)
6
Coeff.
-664.00
436.50
125.50
-62.61
-71.96
821.50
172.30
273.40
209.60
178.50
Coeff.
-52.7
12.3
-53.7
-476.7
185.5
35.8
57.8
69.6
299.2
10.5
Ste.
124.2
162.3
221.0
336.1
267.5
132.2
169.7
228.6
276.1
269.5
Coeff.
-0.0580*
0.05
0.00
-0.04
-2.39
0.02
-0.01
0.02
0.04
0.09
Ste.
0.03
0.04
0.04
0.06
2.38
0.04
0.06
0.08
0.06
0.17
Ste.
53.58
26.08
23.68
25.34
26.05
16.63
28.42
28.06
24.65
31.03
Ste.
680.20
318.80
315.90
310.20
593.80
814.70
481.80
588.60
614.10
833.40
-333.5
-29.1
54.4
0.3
-152.3
-468.2
-620.8*
259.2
253.6
257.4
278.0
291.4
308.2
326.0
0.19
0.16
0.14
0.12
0.147**
0.05
0.22
0.13
0.11
0.09
0.08
0.07
0.07
0.19
10.05
-1.59
-17.10
-28.85
-32.51
-39.87
9.14
12.95
13.24
19.86
28.64
39.48
42.35
25.26
-159.60
-53.96
-240.10
117.90
-142.80
-331.50
227.80
660.90
524.50
508.60
601.20
663.30
813.00
673.60
177.9
3799
626
0.02
246.3
-0.120***
3867
627
0.01
0.04
8.19
3107
588
0.01
12.37
-267.80
2856
567
0.01
477.70
2. Only observations with annual change in the value added of less than DKK 10 mio.
3. Only observations with annual change in net income of less than DKK 3 mio.
4. Only observations with annual change in roa of less than 1, and total assets
>
DKK 100,000.
5. Only observations with number of employees
>
5. Only observations with change in average wage
<
DKK 500,000.
6. Only observations with number of employees
>
5 and change in labour productivity
<
DKK 3 mio.
115
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0116.png
TABLE A.2: Comparison between VP-companies and companies in the reference group. Companies
with up to 50 employees in year zero. Diff-in-diff fixed effects regression results. Only companies with
agreement on the VP-company-match in the DASTI and DST data.
Dependent
variables
(in first
differences):
Treat=1 & t=1
Treat=1 & t=2
Treat=1 & t=3
Treat=1 & t=4
Treat=1 & t=5
t=1
t=2
t=3
t=4
t=5
Year dummies
2003
2004
2005
2006
2007
2008
2009
Number of highly
educated employees
1
Coeff.
0.516***
0.283*
-0.06
-0.34
-0.26
0.00
0.02
-0.16
-0.07
-0.02
Number of employees
Value added (DKK1,000)
2
Ste.
0.15
0.16
0.20
0.24
0.29
0.11
0.15
0.21
0.24
0.32
Coeff.
0.57
-0.10
0.42
-0.981*
-0.96
0.14
0.16
0.08
0.64
1.549*
Ste.
0.37
0.40
0.42
0.54
0.77
0.31
0.42
0.53
0.71
0.90
Coeff.
-23.2
413.1
127.0
-0.2
-768.1
102.3
-248.9
171.8
443.6
475.8
Ste.
234.1
257.1
366.9
521.9
682.9
216.2
275.6
412.0
453.0
663.0
-0.01
-0.01
0.02
0.05
-0.02
0.01
-0.15
0.15
0.15
0.13
0.12
0.15
0.18
0.22
-0.18
-0.18
0.15
0.08
0.15
0.15
-0.14
0.26
0.26
0.31
0.33
0.40
0.47
0.61
-371.0
-371.0
401.6
98.9
343.1
272.3
-270.2
257.6
257.6
257.2
239.8
283.5
354.3
413.8
Constant
Number of
observations:
Number of companies:
R-squared
0.10
1632
289
0.05
0.10
0.34
1697
294
0.09
0.25
95.9
1627
290
0.04
203.6
Notes: Only observations with annual changes in the number of employees of less than 12. *, **, *** denote statistical significance at the 10%, 5%,
and 1% significance level.
1. Employees with post-secondary or tertiary education. Only observations with annual changes in the number of employees with post-secondary
and tertiary education
<
5.
116
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0117.png
Net income (DKK1,000)
3
Return on assets
4
Wage per employee
(DKK1,000)
5
Coeff.
6.42
8.99
-17.89
34.01*
-28.68
-12.53
-1.81
3.28
-26.61
-12.68
Labour productivity
(DKK1,000)
6
Coeff.
-80.61
125.30
-178.3*
-144.90
-54.19
-95.96
114.80
141.00
48.66
122.60
Coeff.
-67.7
208.0
191.4
258.7
-104.8
-29.0
-277.0**
-81.9
-177.5
-32.8
Ste.
114.2
131.1
140.0
256.9
222.2
113.3
137.1
177.0
252.0
292.2
Coeff.
-0.04
-0.0656**
0.01
-0.02
-0.08
-0.02
-0.04
-0.0709*
-0.07
-0.113*
Ste.
0.03
0.03
0.03
0.05
0.07
0.03
0.03
0.04
0.05
0.06
Ste.
13.33
12.07
15.11
19.80
34.35
13.70
12.21
16.46
22.88
31.07
Ste.
124.30
130.80
105.30
161.80
241.70
112.50
120.10
128.00
152.70
262.00
-129.6
-129.6
99.0
5.3
50.3
104.4
-71.4
107.5
107.5
101.3
94.2
117.5
147.7
185.9
-0.0597**
-0.0597**
0.02
-0.01
0.01
0.03
0.01
0.03
0.03
0.02
0.02
0.03
0.03
0.04
13.46*
13.46*
13.24
13.22
10.87
20.12
4.24
7.06
7.06
8.06
9.19
10.59
14.66
17.45
-175.6**
-175.6**
-35.98
-8.17
21.71
-77.02
-154.30
87.28
87.28
71.15
90.36
111.20
117.50
145.40
66.0
1579
291
0.02
74.9
0.02
1658
292
0.03
0.02
-3.65
978
202
0.03
5.68
63.03
1052
178
0.04
65.53
2. Only observations with annual change in the value added of less than DKK 10 mio.
3. Only observations with annual change in net income of less than DKK 3 mio.
4. Only observations with annual change in roa of less than 1, and total assets
>
DKK 100,000.
5. Only observations with number of employees
>
5. Only observations with change in average wage
<
DKK 500,000.
6. Only observations with number of employees
>
5 and change in labour productivity
<
DKK 3 mio.
117
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0118.png
TABLE A.3: Comparison between VP-companies and companies in the reference group. Companies
with up to 50 employees in year zero. Diff-in-diff fixed effects regression results. Only companies with
completed VP-projects.
Dependent
variables
(in first
differences):
Treat=1 & t=1
Treat=1 & t=2
Treat=1 & t=3
Treat=1 & t=4
Treat=1 & t=5
t=1
t=2
t=3
t=4
t=5
Year dummies
2003
2004
2005
2006
2007
2008
2009
Number of highly
educated employees
1
Coeff.
0.555***
0.429***
0.00
-0.24
-0.25
-0.13
-0.15
-0.25
-0.24
-0.16
Number of employees
Value added (DKK1,000)
2
Ste.
0.13
0.16
0.18
0.22
0.28
0.11
0.15
0.19
0.23
0.29
Coeff.
0.708**
0.26
0.63
-0.45
-0.68
0.06
-0.16
-0.51
-0.31
0.84
Ste.
0.33
0.34
0.42
0.65
0.61
0.26
0.35
0.43
0.58
0.58
Coeff.
273.2
374.5
278.5
-85.2
-678.6
92.7
-210.3
11.4
441.7
571.3
Ste.
245.3
255.4
364.8
489.9
626.9
207.0
242.4
372.8
419.4
581.9
-0.04
0.02
0.05
0.06
0.12
-0.03
-0.01
0.14
0.12
0.11
0.14
0.18
0.20
0.24
-0.04
0.22
0.26
0.21
0.44
0.22
-1.304**
0.23
0.27
0.29
0.34
0.36
0.45
0.52
-255.3
390.3
103.4
279.2
269.8
-226.3
-749.3*
245.7
242.5
234.2
260.6
302.5
347.1
429.8
Constant
Number of
observations:
Number of companies:
R-squared
0.12
2122
431
0.04
0.10
0.38
2217
440
0.08
0.23
204.9
2120
359
0.04
207.3
Notes: Only observations with annual changes in the number of employees of less than 12. *, **, *** denote statistical significance at the 10%, 5%,
and 1% significance level.
1. Employees with post-secondary or tertiary education. Only observations with annual changes in the number of employees with post-secondary
and tertiary education
<
5.
118
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0119.png
Net income (DKK1,000)
3
Return on assets
4
Wage per employee
(DKK1,000)
5
Coeff.
0.71
0.47
-7.51
8.34
-26.68
3.99
21.19*
34.80**
27.78
25.04
Labour productivity
(DKK1,000)
6
Coeff.
-72.72
52.31
-163.7*
-134.50
-121.00
-68.50
73.53
88.85
-7.39
13.19
Coeff.
-58.8
228.1*
148.0
112.0
-134.4
25.5
-142.4
45.2
123.6
271.5
Ste.
107.2
122.4
124.9
236.1
195.4
97.8
116.1
146.5
216.8
240.7
Coeff.
-0.03
-0.03
0.01
-0.04
-0.04
-0.02
-0.04
-0.05
-0.03
-0.08
Ste.
0.03
0.03
0.03
0.04
0.07
0.02
0.03
0.03
0.04
0.06
Ste.
11.04
11.92
14.10
19.07
31.65
11.48
12.35
16.25
21.93
30.60
Ste.
114.10
140.10
97.86
145.80
258.60
107.60
116.40
115.30
144.10
242.40
-129.2
36.0
49.0
-15.1
-8.6
-239.4
-392.2**
101.3
95.8
92.5
106.1
126.9
149.1
184.5
-0.0530**
0.00
0.01
0.00
0.02
-0.02
-0.01
0.03
0.02
0.02
0.03
0.03
0.04
0.04
15.23**
7.18
3.37
-2.78
-7.74
-23.69
-16.50
6.89
7.80
7.01
9.59
13.91
16.81
20.86
-60.82
8.70
80.78
166.4*
3.10
-82.71
39.04
55.73
57.82
62.20
95.99
104.70
116.70
133.70
105.5
2084
438
0.03
78.6
0.01
2174
439
0.02
0.02
3.29
1175
274
0.02
6.02
-19.71
1307
386
0.03
49.68
2. Only observations with annual change in the value added of less than DKK 10 mio.
3. Only observations with annual change in net income of less than DKK 3 mio.
4. Only observations with annual change in roa of less than 1, and total assets
>
DKK 100,000.
5. Only observations with number of employees
>
5. Only observations with change in average wage
<
DKK 500,000.
6. Only observations with number of employees
>
5 and change in labour productivity
<
DKK 3 mio.
119
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0120.png
TABLE A.4: Comparison between VP-companies and companies in the reference group. Companies with
up to 50 employees in year zero. Diff-in-diff fixed effects regression results. VPs in companies without
employees with a tertiary education prior to programme participation.
Dependent
variables
(in first
differences):
Treat=1 & t=1
Treat=1 & t=2
Treat=1 & t=3
Treat=1 & t=4
Treat=1 & t=5
t=1
t=2
t=3
t=4
t=5
Year dummies
2003
2004
2005
2006
2007
2008
2009
Number of highly
educated employees
1
Coeff.
0.690***
0.426***
0.00
-0.19
-0.27
0.02
0.03
0.08
0.07
0.10
Number of employees
Value added (DKK1,000)
2
Ste.
0.12
0.16
0.19
0.24
0.34
0.10
0.13
0.15
0.20
0.28
Coeff.
0.35
0.02
0.41
-0.13
-1.05
0.27
0.03
0.17
0.50
1.512*
Ste.
0.37
0.41
0.47
0.73
0.91
0.31
0.43
0.54
0.74
0.85
Coeff.
52.5
449.0
79.6
181.5
-478.5
132.5
-251.3
296.5
485.9
112.8
Ste.
255.3
316.4
332.1
594.6
729.5
218.4
293.7
359.3
498.9
625.3
0.07
0.09
0.07
0.05
0.00
-0.11
-0.26
0.10
0.10
0.08
0.11
0.13
0.15
0.17
-0.16
0.33
0.42
0.46
0.33
-0.08
-2.371***
0.38
0.41
0.43
0.49
0.51
0.61
0.68
-475.2*
117.2
34.4
307.5
181.1
-509.3
-1222***
286.9
270.9
263.7
321.8
351.2
379.6
453.6
Constant
Number of
observations:
Number of companies:
R-squared
0.03
1711
347
0.07
0.07
0.33
1716
348
0.12
0.36
327.3
1671
342
0.07
240.6
Notes: Only observations with annual changes in the number of employees of less than 12. *, **, *** denote statistical significance at the 10%, 5%,
and 1% significance level.
1. Employees with post-secondary or tertiary education. Only observations with annual changes in the number of employees with post-secondary
and tertiary education
<
5.
120
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0121.png
Net income (DKK1,000)
3
Return on assets
4
Wage per employee
(DKK1,000)
5
Coeff.
15.72
3.22
-24.15
14.08
-29.47
-15.93
0.61
19.23
-3.87
0.94
Labour productivity
(DKK1,000)
6
Coeff.
9.63
170.70
-58.23
-37.22
-74.40
-80.51
76.28
101.50
-108.10
51.62
Coeff.
58.0
155.5
95.1
114.6
-17.5
16.3
-124.6
141.5
161.9
126.8
Ste.
126.6
145.1
148.7
284.5
310.1
113.6
121.6
155.4
253.2
271.7
Coeff.
0.00
-0.02
-0.01
-0.03
0.00
0.00
-0.03
-0.02
0.00
-0.135**
Ste.
0.03
0.03
0.03
0.05
0.07
0.03
0.03
0.03
0.04
0.06
Ste.
13.69
12.19
16.50
21.63
36.69
15.31
14.47
18.95
25.53
34.50
Ste.
91.46
115.10
95.90
140.10
226.50
91.12
103.30
108.50
129.00
236.80
-124.8
-34.4
-9.4
-92.3
-86.0
-372.3**
-551.5***
97.9
106.1
88.2
113.0
127.4
149.0
191.3
0.01
0.01
0.01
0.02
-0.03
-0.01
0.00
0.02
0.02
0.02
0.03
0.03
0.03
0.04
14.56*
2.25
0.58
6.07
4.48
-4.13
7.49
7.53
8.26
10.18
12.99
16.76
20.49
23.38
-167.1*
-60.30
-73.07
13.41
-107.60
-168.70
-81.89
89.11
66.77
92.82
99.39
116.80
132.30
134.90
161.7**
1620
345
0.04
76.6
0.00
1686
347
0.03
0.02
3.07
1010
230
0.02
7.30
83.71
1168
215
0.04
71.79
2. Only observations with annual change in the value added of less than DKK 10 mio.
3. Only observations with annual change in net income of less than DKK 3 mio.
4. Only observations with annual change in roa of less than 1, and total assets
>
DKK 100,000.
5. Only observations with number of employees
>
5. Only observations with change in average wage
<
DKK 500,000.
6. Only observations with number of employees
>
5 and change in labour productivity
<
DKK 3 mio.
121
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0122.png
TABLE A.5: Comparison between VP-companies and companies in the reference group. Companies
with up to 50 employees in year zero. Diff-in-diff fixed effects regression results. Companies in
manufacturing industries and contruction.
Dependent
variables
(in first
differences):
Treat=1 & t=1
Treat=1 & t=2
Treat=1 & t=3
Treat=1 & t=4
Treat=1 & t=5
t=1
t=2
t=3
t=4
t=5
Year dummies
2003
2004
2005
2006
2007
2008
2009
Number of highly
educated employees
1
Coeff.
0.675***
0.37
-0.25
-0.18
-0.14
-0.07
-0.22
-0.03
-0.28
0.02
Number of employees
Value added (DKK1,000)
2
Ste.
0.20
0.29
0.27
0.43
0.49
0.15
0.22
0.21
0.37
0.23
Coeff.
0.61
-0.47
0.40
-0.10
-1.61
0.39
0.29
1.09
0.95
3.974***
Ste.
0.85
0.79
0.90
1.29
1.36
0.64
0.86
1.05
1.37
1.23
Coeff.
383.2
623.6
1010.0
220.3
-2228.0
161.8
-117.0
385.7
958.8
2767.0
Ste.
506.1
610.9
760.4
1454.0
3169.0
433.3
530.1
709.5
1008.0
2760.0
-0.11
0.09
-0.05
-0.07
-0.04
-0.07
-0.22
0.27
0.25
0.21
0.22
0.27
0.27
0.29
0.45
0.91
1.43
1.40
1.06
0.14
-3.382**
0.72
0.82
0.88
0.99
0.97
1.24
1.40
-438.8
480.5
212.1
987.6
491.2
-530.1
-1705**
614.6
598.5
617.4
660.3
686.3
688.6
845.5
Constant
Number of
observations:
Number of companies:
R-squared
0.15
643
125
0.07
0.20
-0.32
667
128
0.17
0.72
100.1
640
126
0.11
527.1
Notes: Only observations with annual changes in the number of employees of less than 12. *, **, *** denote statistical significance at the 10%, 5%,
and 1% significance level.
1. Employees with post-secondary or tertiary education. Only observations with annual changes in the number of employees with post-secondary
and tertiary education
<
5.
122
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0123.png
Net income (DKK1,000)
3
Return on assets
4
Wage per employee
(DKK1,000)
5
Coeff.
15.16
-21.46
-25.78
38.30
-160.2*
-19.52
-13.16
7.90
-42.36
76.77
Labour productivity
(DKK1,000)
6
Coeff.
-48.80
165.80
-76.95
-5.01
-417.10
-22.15
75.43
26.12
39.84
405.70
Coeff.
0.2
67.4
378.3
159.7
-289.4
176.8
65.6
178.8
274.7
0.0
Ste.
200.4
226.3
257.3
695.7
393.7
172.5
197.9
266.5
560.0
0.0
Coeff.
-0.04
-0.04
-0.05
-0.09
-0.08
-0.03
-0.04
-0.05
-0.07
-0.14
Ste.
0.04
0.04
0.05
0.09
0.08
0.03
0.04
0.05
0.07
0.14
Ste.
15.91
14.91
22.17
29.98
82.57
15.82
16.90
23.95
27.50
67.54
Ste.
102.20
101.60
135.60
240.00
409.70
107.00
130.50
184.40
244.60
481.60
110.2
100.8
139.5
121.7
0.7
-294.3
-516.3
215.9
231.5
173.5
217.0
228.7
272.4
368.1
-0.03
-0.02
-0.02
-0.03
-0.03
-0.04
-0.05
0.03
0.02
0.02
0.03
0.03
0.04
0.05
14.69*
19.67**
4.48
29.13**
34.80*
32.23
31.22
8.25
9.03
5.95
11.75
20.13
20.59
24.77
-45.30
49.33
6.06
120.40
20.63
-74.83
-66.23
68.40
69.99
67.65
121.30
145.40
194.10
246.80
3.5
612
128
0.04
159.2
0.01
660
128
0.05
0.01
-7.64
463
98
0.07
5.32
24.76
532
98
0.05
64.36
2. Only observations with annual change in the value added of less than DKK 10 mio.
3. Only observations with annual change in net income of less than DKK 3 mio.
4. Only observations with annual change in roa of less than 1, and total assets
>
DKK 100,000.
5. Only observations with number of employees
>
5. Only observations with change in average wage
<
DKK 500,000.
6. Only observations with number of employees
>
5 and change in labour productivity
<
DKK 3 mio.
123
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0124.png
TABLE A.6: Comparison between VP-companies and companies in the reference group. Companies
with up to 50 employees in year zero. Diff-in-diff fixed effects regression results. Companies in service
industries
Dependent
variables
(in first
differences):
Treat=1 & t=1
Treat=1 & t=2
Treat=1 & t=3
Treat=1 & t=4
Treat=1 & t=5
t=1
t=2
t=3
t=4
t=5
Year dummies
2003
2004
2005
2006
2007
2008
2009
Number of highly
educated employees
1
Coeff.
0.27
0.24
0.30
-0.16
-0.30
0.03
0.00
-0.34
-0.01
0.00
Number of employees
Value added (DKK1,000)
2
Ste.
0.17
0.20
0.25
0.29
0.30
0.15
0.18
0.25
0.26
0.29
Coeff.
0.619**
0.11
0.55
-0.69
-0.45
-0.18
-0.35
-0.775*
-0.46
-0.07
Ste.
0.28
0.41
0.45
0.65
0.69
0.25
0.34
0.43
0.62
0.64
Coeff.
178.0
157.0
-75.1
269.1
-758.7
-95.7
-134.4
115.0
511.8
878.6*
Ste.
271.4
267.0
413.6
407.7
565.8
250.1
254.9
416.6
469.1
448.3
0.15
0.23
0.22
0.18
0.23
0.13
0.03
0.15
0.14
0.14
0.16
0.21
0.23
0.26
0.07
0.23
0.02
0.23
0.22
0.43
-0.49
0.18
0.24
0.24
0.30
0.36
0.38
0.45
-288.5
307.4
65.0
163.7
83.9
-373.1
-796.4**
243.6
225.5
206.1
243.8
291.3
365.6
399.1
Constant
Number of
observations:
Number of companies:
R-squared
-0.04
1434
300
0.02
0.12
0.433**
1492
304
0.06
0.21
340.2*
1430
293
0.04
199.4
Notes: Only observations with annual changes in the number of employees of less than 12. *, **, *** denote statistical significance at the 10%, 5%,
and 1% significance level.
1. Employees with post-secondary or tertiary education. Only observations with annual changes in the number of employees with post-secondary
and tertiary education
<
5.
124
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0125.png
Net income (DKK1,000)
3
Return on assets
4
Wage per employee
(DKK1,000)
5
Coeff.
1.60
23.14
-13.65
27.89
-5.02
-18.10
-2.07
-14.01
-6.00
-15.28
Labour productivity
(DKK1,000)
6
Coeff.
-99.96
-148.10
-255.60
-53.12
-15.26
-75.44
162.60
181.50
2.36
-113.40
Coeff.
-107.4
133.6
40.3
161.7
65.6
-76.4
-178.6
-22.0
56.4
271.1
Ste.
118.2
130.3
145.6
224.9
236.3
111.0
131.9
165.4
236.5
270.5
Coeff.
-0.04
-0.0702*
-0.0692*
-0.02
-0.03
-0.03
-0.02
-0.03
-0.05
-0.10
Ste.
0.03
0.04
0.04
0.05
0.10
0.03
0.04
0.04
0.05
0.08
Ste.
22.15
19.69
25.40
30.68
41.22
22.37
21.77
27.33
41.05
39.36
Ste.
184.50
221.60
172.20
255.70
466.20
169.40
177.30
179.50
220.80
356.90
-197.0*
179.0*
65.3
100.1
32.1
-105.6
-335.3*
118.3
92.0
103.4
118.0
134.5
156.8
191.1
-0.0709*
0.03
0.01
0.03
0.02
0.00
0.01
0.04
0.03
0.04
0.04
0.04
0.05
0.06
15.75
2.12
14.59
4.45
18.34
2.49
13.96
10.89
15.19
16.26
16.37
23.56
29.14
33.24
-176.80
-32.71
49.76
126.20
-16.60
-185.10
-5.86
116.90
121.00
124.00
157.40
164.20
179.40
187.10
59.4
1420
300
0.05
86.5
0.01
1444
302
0.03
0.03
1.98
634
159
0.03
12.72
13.71
722
142
0.03
107.70
2. Only observations with annual change in the value added of less than DKK 10 mio.
3. Only observations with annual change in net income of less than DKK 3 mio.
4. Only observations with annual change in roa of less than 1, and total assets
>
DKK 100,000.
5. Only observations with number of employees
>
5. Only observations with change in average wage
<
DKK 500,000.
6. Only observations with number of employees
>
5 and change in labour productivity
<
DKK 3 mio.
125
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0126.png
TABLE A.7: Comparison between VP-companies and companies in the reference group. Companies
with up to 50 employees in year zero. Diff-in-diff fixed effects regression results. Companies in ’other’
industries
Dependent
variables
(in first
differences):
Treat=1 & t=1
Treat=1 & t=2
Treat=1 & t=3
Treat=1 & t=4
Treat=1 & t=5
t=1
t=2
t=3
t=4
t=5
Year dummies
2003
2004
2005
2006
2007
2008
2009
Number of highly
educated employees
1
Coeff.
0.670***
0.44
-0.42
-0.11
-0.03
-0.04
0.07
0.28
0.06
0.19
Number of employees
Value added (DKK1,000)
2
Ste.
0.23
0.33
0.38
0.37
0.65
0.23
0.27
0.39
0.53
0.72
Coeff.
0.93
0.30
-0.57
-0.18
-0.92
-0.02
-0.06
-0.18
0.43
0.91
Ste.
0.74
0.64
0.94
1.57
1.58
0.60
0.71
0.92
1.21
1.25
Coeff.
253.8
541.6
-548.7
-656.4
150.0
-70.8
-467.1
-210.0
-651.9
-1383.0
Ste.
451.0
546.5
581.5
839.1
733.3
416.4
616.7
640.7
799.8
1119.0
-0.08
-0.25
-0.35
-0.28
-0.45
-0.806*
-0.45
0.26
0.20
0.23
0.30
0.37
0.46
0.55
-0.61
-0.01
-0.13
-0.44
-0.24
-0.61
-2.684***
0.47
0.53
0.51
0.63
0.82
0.96
0.97
-182.5
57.0
282.4
-85.5
324.9
-105.5
-257.5
335.3
434.0
393.3
485.6
612.0
660.1
906.3
Constant
Number of
observations:
Number of companies:
R-squared
0.374*
532
110
0.07
0.19
0.914**
568
114
0.12
0.44
233.3
541
114
0.06
298.5
Notes: Only observations with annual changes in the number of employees of less than 12. *, **, *** denote statistical significance at the 10%, 5%,
and 1% significance level.
1. Employees with post-secondary or tertiary education. Only observations with annual changes in the number of employees with post-secondary
and tertiary education
<
5.
126
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0127.png
Net income (DKK1,000)
3
Return on assets
4
Wage per employee
(DKK1,000)
5
Coeff.
10.97
12.11
-18.98
-31.56
40.93
2.13
18.42
41.87
49.64
8.49
Labour productivity
(DKK1,000)
6
Coeff.
91.45
238.20
-61.10
-63.79
-188.00
59.35
21.04
186.10
89.03
265.60
Coeff.
33.9
101.2
26.8
248.8
24.6
154.2
-44.4
140.6
-266.0
-259.3
Ste.
251.8
298.5
336.2
413.3
241.0
237.0
282.5
318.4
432.8
459.0
Coeff.
-0.02
-0.03
0.00
-0.09
-0.03
0.01
-0.06
-0.03
-0.02
-0.08
Ste.
0.05
0.05
0.05
0.09
0.03
0.05
0.07
0.06
0.08
0.07
Ste.
14.59
14.07
19.72
32.56
34.04
19.13
17.63
26.48
32.94
39.20
Ste.
142.50
175.10
177.20
152.20
264.70
144.00
162.50
161.50
170.50
287.80
-108.7
-354.1**
6.3
-401.5*
26.0
-307.0
-273.9
174.2
171.2
227.6
240.2
344.1
357.7
409.5
-0.03
-0.04
0.02
-0.01
0.03
-0.02
0.02
0.04
0.05
0.04
0.05
0.06
0.06
0.07
21.23*
7.11
-2.79
-6.87
-23.19
-27.97
-23.52
12.66
11.21
14.73
16.59
24.68
27.68
36.09
-201.00
-124.40
-218.50
-158.40
-279.20
-311.00
-315.10
189.30
107.80
201.00
215.50
204.20
226.30
258.40
189.3
521
114
0.06
166.5
0.00
565
114
0.03
0.03
2.66
405
89
0.04
10.41
173.80
439
83
0.03
141.40
2. Only observations with annual change in the value added of less than DKK 10 mio.
3. Only observations with annual change in net income of less than DKK 3 mio.
4. Only observations with annual change in roa of less than 1, and total assets
>
DKK 100,000.
5. Only observations with number of employees
>
5. Only observations with change in average wage
<
DKK 500,000.
6. Only observations with number of employees
>
5 and change in labour productivity
<
DKK 3 mio.
127
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0128.png
TABLE A.8: Comparison between VP-companies and companies in the reference group. Companies with
up to 50 employees in year zero. Diff-in-diff fixed effects regression results. Male VPs.
Dependent
variables
(in first
differences):
Treat=1 & t=1
Treat=1 & t=2
Treat=1 & t=3
Treat=1 & t=4
Treat=1 & t=5
t=1
t=2
t=3
t=4
t=5
Year dummies
2003
2004
2005
2006
2007
2008
2009
Number of highly
educated employees
1
Coeff.
0.390**
0.523***
0.18
-0.05
0.05
-0.09
-0.21
-0.31
-0.17
-0.16
Number of employees
Value added (DKK1,000)
2
Ste.
0.15
0.18
0.24
0.26
0.28
0.13
0.18
0.23
0.25
0.32
Coeff.
0.665*
0.39
0.933*
0.13
-0.19
-0.07
-0.41
-0.68
-0.32
0.32
Ste.
0.40
0.44
0.52
0.68
0.78
0.32
0.44
0.54
0.71
0.75
Coeff.
354.9
429.9
308.0
333.1
-532.8
-215.0
-391.1
-281.8
329.6
254.0
Ste.
310.2
301.0
436.8
605.4
635.7
280.9
316.6
456.2
516.9
628.9
0.02
0.16
0.06
0.07
0.24
-0.04
0.02
0.18
0.15
0.14
0.17
0.20
0.23
0.26
0.11
0.55
0.50
0.66
0.71
0.61
-0.89
0.32
0.38
0.41
0.48
0.48
0.57
0.65
-284.2
545.6*
247.3
723.6**
513.9
-48.9
-631.3
315.5
311.3
309.1
351.6
401.7
429.1
529.3
Constant
Number of
observations:
Number of companies:
R-squared
0.05
1605
331
0.03
0.13
0.08
1666
336
0.06
0.34
81.5
1585
327
0.05
269.0
Notes: Only observations with annual changes in the number of employees of less than 12. *, **, *** denote statistical significance at the 10%, 5%,
and 1% significance level.
1. Employees with post-secondary or tertiary education. Only observations with annual changes in the number of employees with post-secondary
and tertiary education
<
5.
128
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0129.png
Net income (DKK1,000)
3
Return on assets
4
Wage per employee
(DKK1,000)
5
Coeff.
5.67
-2.21
-27.79
33.80
-59.65
-22.36
3.95
13.10
-14.17
13.29
Labour productivity
(DKK1,000)
6
Coeff.
3.25
104.10
-211.8*
-28.19
-8.74
3.28
140.50
165.40
51.63
84.58
Coeff.
-90.9
192.3
39.8
198.4
-208.6
96.0
-108.7
188.9
263.3
552.6*
Ste.
130.5
144.9
153.4
283.4
233.9
119.4
146.8
169.9
253.6
290.8
Coeff.
-0.03
-0.04
-0.01
0.00
-0.05
-0.02
-0.06
-0.04
-0.03
-0.08
Ste.
0.03
0.04
0.04
0.05
0.08
0.03
0.04
0.04
0.05
0.06
Ste.
17.47
14.71
18.29
21.22
36.94
18.49
17.31
22.52
28.77
36.89
Ste.
122.90
139.50
117.10
181.00
276.50
114.80
134.70
136.30
162.10
251.20
-115.0
78.4
-12.9
19.6
-79.9
-295.5*
-608.3***
126.4
121.8
112.2
133.4
152.2
171.3
214.6
-0.03
0.02
0.01
0.05
0.04
-0.01
0.00
0.03
0.02
0.03
0.03
0.04
0.04
0.05
16.51**
6.10
11.56
15.95
13.88
10.00
2.84
7.78
10.08
12.43
13.89
19.70
23.55
28.28
-97.60
-44.83
-18.77
120.10
-135.60
-218.40
-135.40
66.09
71.81
78.32
105.90
125.40
139.40
154.00
113.9
1547
334
0.04
95.1
-0.01
1628
336
0.03
0.02
-1.53
898
214
0.03
8.62
46.79
1032
199
0.05
63.64
2. Only observations with annual change in the value added of less than DKK 10 mio.
3. Only observations with annual change in net income of less than DKK 3 mio.
4. Only observations with annual change in roa of less than 1, and total assets
>
DKK 100,000.
5. Only observations with number of employees
>
5. Only observations with change in average wage
<
DKK 500,000.
6. Only observations with number of employees
>
5 and change in labour productivity
<
DKK 3 mio.
129
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0130.png
TABLE A.9: Comparison between VP-companies and companies in the reference group. Companies with
up to 50 employees in year zero. Diff-in-diff fixed effects regression results. Female VPs.
Dependent
variables
(in first
differences):
Treat=1 & t=1
Treat=1 & t=2
Treat=1 & t=3
Treat=1 & t=4
Treat=1 & t=5
t=1
t=2
t=3
t=4
t=5
Year dummies
2003
2004
2005
2006
2007
2008
2009
Number of highly
educated employees
1
Coeff.
0.580***
0.03
-0.29
-0.30
-0.80
0.07
0.16
0.11
0.01
0.31
Number of employees
Value added (DKK1,000)
2
Ste.
0.17
0.22
0.21
0.35
0.49
0.14
0.19
0.23
0.33
0.32
Coeff.
0.51
-0.54
-0.58
-1.27
-1.53
0.10
0.27
0.22
0.13
1.674*
Ste.
0.45
0.52
0.61
1.12
1.02
0.36
0.45
0.64
0.94
0.86
Coeff.
24.4
277.6
-31.2
-192.4
-642.9
248.3
-140.2
330.3
-39.9
827.2
Ste.
279.2
393.4
464.4
658.6
1242.0
240.5
344.3
440.7
601.3
1072.0
-0.03
-0.09
-0.06
-0.06
-0.361*
-0.25
-0.36
0.14
0.14
0.14
0.18
0.21
0.25
0.32
-0.25
0.04
0.12
0.06
-0.21
-0.38
-2.392***
0.36
0.37
0.38
0.43
0.57
0.65
0.77
-381.7
-72.9
7.5
-216.8
-60.3
-645.1
-1034**
233.4
249.8
210.1
270.9
331.7
396.8
434.4
Constant
Number of
observations:
Number of companies:
R-squared
0.229*
1004
204
0.06
0.12
0.747**
1061
210
0.14
0.35
498.4**
1026
206
0.05
192.7
Notes: Only observations with annual changes in the number of employees of less than 12. *, **, *** denote statistical significance at the 10%, 5%,
and 1% significance level.
1. Employees with post-secondary or tertiary education. Only observations with annual changes in the number of employees with post-secondary
and tertiary education
<
5.
130
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0131.png
Net income (DKK1,000)
3
Return on assets
4
Wage per employee
(DKK1,000)
5
Coeff.
13.67
18.14
-12.73
-20.79
58.27
0.98
-4.02
7.51
24.47
-35.51
Labour productivity
(DKK1,000)
6
Coeff.
-117.00
-23.48
-23.92
-63.13
-571.6*
-108.10
-14.89
-3.76
-33.94
249.60
Coeff.
15.0
51.2
279.7
241.2
176.7
-74.9
-173.5
-217.4
-418.2
-497.3*
Ste.
134.2
175.9
200.8
347.2
297.4
119.5
143.0
191.9
324.6
270.9
Coeff.
-0.03
-0.03
0.01
-0.108*
-0.03
0.01
0.01
-0.03
0.01
-0.05
Ste.
0.03
0.04
0.04
0.06
0.11
0.03
0.03
0.04
0.06
0.10
Ste.
12.56
13.81
19.98
29.18
36.94
11.76
13.02
17.31
26.46
32.44
Ste.
147.30
165.90
148.50
194.40
313.80
134.00
110.10
133.50
164.80
323.70
-63.7
11.3
183.9
-28.7
192.6
-28.4
21.2
125.3
115.4
125.5
131.5
168.6
188.8
218.8
-0.06
0.00
0.00
-0.04
-0.01
-0.04
-0.01
0.04
0.04
0.03
0.04
0.04
0.05
0.05
19.05*
14.57
3.99
3.92
11.08
-5.40
19.74
9.62
8.94
8.03
10.40
16.59
17.29
20.54
-209.30
-17.98
-48.35
-63.25
61.72
-93.17
-8.21
163.20
107.20
155.80
169.10
161.40
180.10
206.20
23.1
1006
208
0.03
102.8
0.02
1039
208
0.03
0.03
-2.26
597
132
0.04
6.70
84.41
661
124
0.03
124.00
2. Only observations with annual change in the value added of less than DKK 10 mio.
3. Only observations with annual change in net income of less than DKK 3 mio.
4. Only observations with annual change in roa of less than 1, and total assets
>
DKK 100,000.
5. Only observations with number of employees
>
5. Only observations with change in average wage
<
DKK 500,000.
6. Only observations with number of employees
>
5 and change in labour productivity
<
DKK 3 mio.
131
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0132.png
TABLE A.10: Comparison between VP-companies and companies in the reference group. Companies
with up to 50 employees in year zero. Diff-in-diff fixed effects regression results. VPs with a tertiary
education.
Dependent
variables
(in first
differences):
Treat=1 & t=1
Treat=1 & t=2
Treat=1 & t=3
Treat=1 & t=4
Treat=1 & t=5
t=1
t=2
t=3
t=4
t=5
Year dummies
2003
2004
2005
2006
2007
2008
2009
Number of highly
educated employees
1
Coeff.
0.533***
-0.06
-0.28
-0.15
-0.570**
-0.07
-0.05
-0.21
-0.16
0.04
Number of employees
Value added (DKK1,000)
2
Ste.
0.18
0.21
0.29
0.31
0.29
0.15
0.18
0.25
0.25
0.28
Coeff.
0.48
-0.63
0.27
-1.577**
-1.20
-0.16
-0.37
-0.62
0.50
0.55
Ste.
0.43
0.52
0.51
0.64
1.02
0.31
0.37
0.49
0.60
0.67
Coeff.
206.2
281.6
-432.1
313.6
-1277*
-211.0
-440.3
383.1
294.4
1023**
Ste.
287.9
376.1
399.5
397.5
711.0
259.5
332.4
436.5
489.5
498.8
0.26
0.10
0.26
0.23
0.30
0.02
0.14
0.16
0.17
0.18
0.20
0.23
0.27
0.30
-0.42
0.02
-0.05
-0.01
0.19
0.24
-1.182*
0.37
0.40
0.35
0.41
0.51
0.52
0.63
-87.3
328.3
179.6
198.6
313.0
-320.2
-696.4
272.5
264.6
240.2
279.3
365.7
404.2
473.5
Constant
Number of
observations:
Number of companies:
R-squared
-0.06
1177
251
0.05
0.16
0.577*
1239
257
0.08
0.34
219.8
1186
250
0.05
214.3
Notes: Only observations with annual changes in the number of employees of less than 12. *, **, *** denote statistical significance at the 10%, 5%,
and 1% significance level.
1. Employees with post-secondary or tertiary education. Only observations with annual changes in the number of employees with post-secondary
and tertiary education
<
5.
132
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0133.png
Net income (DKK1,000)
3
Return on assets
4
Wage per employee
(DKK1,000)
5
Coeff.
20.51
6.89
-35.96
13.84
-22.74
-20.35
-6.48
-3.69
-9.57
4.62
Labour productivity
(DKK1,000)
6
Coeff.
-79.42
7.22
-413.1**
140.30
-714.2**
-0.79
239.30
301.7*
-6.43
679.1**
Coeff.
146.6
164.1
197.4
296.2
236.7
140.3
148.6
199.4
274.7
343.0
Ste.
146.6
164.1
197.4
296.2
236.7
140.3
148.6
199.4
274.7
343.0
Coeff.
-0.04
-0.06
-0.01
-0.03
-0.167*
-0.03
-0.06
-0.05
-0.04
-0.10
Ste.
0.04
0.05
0.05
0.06
0.09
0.03
0.04
0.04
0.06
0.06
Ste.
21.29
20.56
30.15
31.59
60.93
21.33
19.92
23.96
36.18
29.02
Ste.
162.80
184.90
175.80
237.30
309.20
155.20
154.70
164.50
194.90
299.60
164.9
130.6
126.1
142.1
167.1
189.6
234.3
164.9
130.6
126.1
142.1
167.1
189.6
234.3
-0.06
0.00
0.00
0.02
0.02
0.01
0.03
0.04
0.04
0.04
0.04
0.05
0.05
0.06
10.16
-1.77
6.54
0.26
26.97
-3.31
15.84
12.16
15.68
16.66
16.46
22.13
25.36
26.69
-220.4*
-203.5*
-116.30
-90.73
-204.00
-343.4*
-216.70
127.40
116.70
132.30
151.20
147.50
178.30
182.00
111.3
1179
256
0.04
111.3
0.01
1208
256
0.03
0.03
-0.83
624
157
0.04
12.81
172.20
732
146
0.05
117.50
2. Only observations with annual change in the value added of less than DKK 10 mio.
3. Only observations with annual change in net income of less than DKK 3 mio.
4. Only observations with annual change in roa of less than 1, and total assets
>
DKK 100,000.
5. Only observations with number of employees
>
5. Only observations with change in average wage
<
DKK 500,000.
6. Only observations with number of employees
>
5 and change in labour productivity
<
DKK 3 mio.
133
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0134.png
TABLE A.11: Comparison between VP-companies and companies in the reference group. Companies
with up to 50 employees in year zero. Diff-in-diff fixed effects regression results. VPs with degrees in
art&humanities
Dependent
variables
(in first
differences):
Treat=1 & t=1
Treat=1 & t=2
Treat=1 & t=3
Treat=1 & t=4
Treat=1 & t=5
t=1
t=2
t=3
t=4
t=5
Year dummies
2003
2004
2005
2006
2007
2008
2009
Number of highly
educated employees
1
Coeff.
0.40
-0.473*
-0.53
-0.32
-0.53
-0.13
-0.11
-0.04
-0.22
-0.11
Number of employees
Value added (DKK1,000)
2
Ste.
0.33
0.26
0.32
0.69
0.71
0.21
0.23
0.29
0.49
0.36
Coeff.
-0.24
-1.486*
0.33
-0.36
0.96
0.15
-0.24
-0.69
-1.830*
-0.13
Ste.
0.74
0.76
0.68
1.15
1.17
0.42
0.52
0.64
0.98
0.98
Coeff.
-585.4
90.0
-118.4
-588.1
541.7
220.7
-519.9
-241.8
619.9
316.3
Ste.
479.1
532.2
734.4
1190.0
952.4
396.2
485.4
629.0
795.6
759.0
0.16
0.22
0.580**
0.27
0.38
0.18
0.43
0.26
0.23
0.22
0.32
0.32
0.36
0.39
0.90
0.87
0.81
0.84
1.20
1.38
0.00
0.61
0.63
0.68
0.68
0.83
0.89
0.96
78.5
175.4
-59.8
-98.9
128.9
-171.3
-285.9
370.6
307.9
302.1
369.8
439.7
557.0
464.9
Constant
Number of
observations:
Number of companies:
R-squared
-0.16
366
79
0.06
0.21
-0.18
377
80
0.14
0.62
349.3
374
78
0.04
270.5
Notes: Only observations with annual changes in the number of employees of less than 12. *, **, *** denote statistical significance at the 10%, 5%,
and 1% significance level.
1. Employees with post-secondary or tertiary education. Only observations with annual changes in the number of employees with post-secondary
and tertiary education
<
5.
134
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0135.png
Net income (DKK1,000)
3
Return on assets
4
Wage per employee
(DKK1,000)
5
Coeff.
26.45
6.66
-5.97
-2.70
0.00
-12.28
-24.61
-15.68
4.38
-16.95
Labour productivity
(DKK1,000)
6
Coeff.
-332.90
-339.00
-293.00
-435.9*
0.00
31.47
392.70
156.80
63.65
495.30
Coeff.
-310.9
225.6
56.4
-1233.0
-250.1
-130.5
-423.7*
-304.1
909.4
341.1
Ste.
241.5
252.0
356.6
849.7
847.8
243.6
215.5
338.8
674.0
703.5
Coeff.
-0.06
-0.07
-0.08
-0.17
-0.19
-0.05
-0.07
-0.07
-0.18
-0.10
Ste.
0.06
0.07
0.08
0.17
0.19
0.05
0.07
0.07
0.18
0.10
Ste.
25.89
27.80
54.10
67.91
0.00
24.17
27.57
43.30
39.65
45.84
Ste.
304.30
432.00
323.30
255.10
0.00
376.00
409.80
361.60
232.50
327.90
18.3
47.2
99.2
143.3
102.2
145.3
76.8
176.9
134.7
147.3
166.6
221.8
260.5
294.6
-0.08
-0.08
-0.08
-0.09
-0.09
-0.10
-0.11
0.08
0.08
0.08
0.09
0.09
0.10
0.11
-3.70
-12.41
-19.93
-17.82
6.56
-16.19
29.41
26.77
32.46
21.77
27.03
33.61
42.94
49.11
-2.09
-11.39
95.89
194.20
195.40
14.77
56.61
351.50
312.40
316.00
393.80
316.70
379.90
392.80
18.2
370
79
0.06
113.1
0.08
365
79
0.08
0.08
6.67
183
44
0.07
21.30
-87.24
224
44
0.06
320.30
2. Only observations with annual change in the value added of less than DKK 10 mio.
3. Only observations with annual change in net income of less than DKK 3 mio.
4. Only observations with annual change in roa of less than 1, and total assets
>
DKK 100,000.
5. Only observations with number of employees
>
5. Only observations with change in average wage
<
DKK 500,000.
6. Only observations with number of employees
>
5 and change in labour productivity
<
DKK 3 mio.
135
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0136.png
TABLE A.12: Comparison between VP-companies and companies in the reference group. Companies
with up to 50 employees in year zero. Diff-in-diff fixed effects regression results. VPs with degrees in
social sciences.
Dependent
variables
(in first
differences):
Treat=1 & t=1
Treat=1 & t=2
Treat=1 & t=3
Treat=1 & t=4
Treat=1 & t=5
t=1
t=2
t=3
t=4
t=5
Year dummies
2003
2004
2005
2006
2007
2008
2009
Number of highly
educated employees
1
Coeff.
0.32
0.05
-0.12
0.07
-0.16
-0.11
-0.04
-0.67
-0.46
-0.18
Number of employees
Value added (DKK1,000)
2
Ste.
0.26
0.30
0.39
0.43
0.33
0.22
0.28
0.41
0.39
0.45
Coeff.
1.512***
-0.55
0.79
-0.38
0.31
-0.997**
-0.828*
-1.286**
-0.59
-1.65
Ste.
0.52
0.78
0.78
1.01
1.55
0.41
0.45
0.53
0.85
1.17
Coeff.
896.0**
89.0
-333.9
1194*
449.3
-805.9**
-582.5
353.1
-487.7
-22.4
Ste.
389.2
490.5
435.6
661.3
899.2
350.7
447.8
461.1
753.3
921.3
0.01
0.07
0.13
0.18
0.12
0.22
0.12
0.23
0.22
0.21
0.24
0.32
0.36
0.43
-0.45
-0.02
0.07
0.26
0.37
0.67
-0.61
0.39
0.49
0.42
0.50
0.56
0.57
0.67
-506.4
32.3
137.4
228.8
249.2
-470.2
-610.7
362.3
283.6
259.4
342.6
461.9
523.0
668.2
Constant
Number of
observations:
Number of companies:
R-squared
0.10
630
127
0.04
0.18
0.65
658
130
0.09
0.40
444.6*
621
124
0.07
239.0
Notes: Only observations with annual changes in the number of employees of less than 12. *, **, *** denote statistical significance at the 10%, 5%,
and 1% significance level.
1. Employees with post-secondary or tertiary education. Only observations with annual changes in the number of employees with post-secondary
and tertiary education
<
5.
136
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0137.png
Net income (DKK1,000)
3
Return on assets
4
Wage per employee
(DKK1,000)
5
Coeff.
-6.09
20.51
-48.35
35.91
33.74
-21.91
9.09
-5.14
-13.95
13.19
Labour productivity
(DKK1,000)
6
Coeff.
-2.87
-155.00
-387.40
297.70
-636.4*
-25.13
321.1**
395.6**
-126.50
581.0*
Coeff.
-31.8
-32.2
-213.2
541.7
24.8
-176.0
-169.5
369.5
-25.2
273.3
Ste.
206.1
204.4
259.8
364.3
390.6
204.1
223.3
261.0
373.0
452.1
Coeff.
-0.06
-0.07
-0.05
-0.09
-0.07
-0.02
-0.04
-0.01
-0.03
-0.09
Ste.
0.04
0.06
0.04
0.08
0.05
0.04
0.05
0.05
0.07
0.10
Ste.
27.41
30.54
32.39
29.57
21.08
26.86
25.73
31.11
43.11
38.30
Ste.
213.40
242.20
239.20
301.30
323.10
148.10
141.70
196.90
183.50
339.10
32.6
172.8
257.8*
152.4
117.9
-212.0
-364.7
160.9
148.7
145.2
168.3
226.0
237.9
317.6
0.0578*
0.0863**
0.0983**
0.07
0.05
0.11
-0.0649***
0.02
0.03
0.04
0.04
0.05
0.05
0.07
-1.98
-5.47
11.68
-3.27
22.46
-4.65
5.39
15.25
18.16
22.17
20.85
28.50
33.52
36.33
-191.30
-73.31
-90.01
-154.90
-257.90
-421.0*
-215.60
131.00
145.50
162.80
157.80
177.00
224.20
218.40
12.3
626
129
0.07
117.2
-0.0649***
638
130
0.03
0.02
0.76
368
85
0.09
15.95
172.10
416
79
0.06
125.10
2. Only observations with annual change in the value added of less than DKK 10 mio.
3. Only observations with annual change in net income of less than DKK 3 mio.
4. Only observations with annual change in roa of less than 1, and total assets
>
DKK 100,000.
5. Only observations with number of employees
>
5. Only observations with change in average wage
<
DKK 500,000.
6. Only observations with number of employees
>
5 and change in labour productivity
<
DKK 3 mio.
137
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0138.png
TABLE A.13: Comparison between VP-companies and companies in the reference group. Companies
with up to 50 employees in year zero. Diff-in-diff fixed effects regression results. VPs with degrees in
technical sciences.
Dependent
variables
(in first
differences):
Treat=1 & t=1
Treat=1 & t=2
Treat=1 & t=3
Treat=1 & t=4
Treat=1 & t=5
t=1
t=2
t=3
t=4
t=5
Year dummies
2003
2004
2005
2006
2007
2008
2009
Number of highly
educated employees
1
Coeff.
0.516**
0.417*
-0.08
-0.17
-0.17
0.09
0.19
0.09
0.11
0.08
Number of employees
Value added (DKK1,000)
2
Ste.
0.21
0.24
0.30
0.34
0.39
0.17
0.23
0.25
0.31
0.40
Coeff.
-0.11
0.44
0.19
0.05
-2.006***
0.921*
0.35
0.89
0.96
3.272***
Ste.
0.58
0.53
0.72
0.93
0.70
0.51
0.72
0.88
1.06
1.00
Coeff.
-282.6
915.7*
283.7
-103.5
-1962*
745.2**
-154.7
517.3
1157*
2082**
Ste.
383.9
466.0
645.6
904.3
991.6
355.2
441.8
673.9
692.2
858.6
-0.10
0.06
-0.14
-0.20
-0.16
-0.41
-0.31
0.23
0.19
0.18
0.22
0.26
0.29
0.30
-0.01
0.31
0.47
0.32
-0.06
-0.58
-2.893***
0.45
0.52
0.57
0.73
0.79
0.90
1.05
-441.7
579.8
104.6
552.6
207.8
-625.7
-1549**
393.0
443.9
419.3
489.5
538.8
566.5
666.5
Constant
Number of
observations:
Number of companies:
R-squared
0.14
932
187
0.05
0.17
0.30
985
193
0.10
0.45
129.1
926
190
0.09
352.2
Notes: Only observations with annual changes in the number of employees of less than 12. *, **, *** denote statistical significance at the 10%, 5%,
and 1% significance level.
1. Employees with post-secondary or tertiary education. Only observations with annual changes in the number of employees with post-secondary
and tertiary education
<
5.
138
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0139.png
Net income (DKK1,000)
3
Return on assets
4
Wage per employee
(DKK1,000)
5
Coeff.
27.09
-3.78
-19.81
47.24*
-58.75
-29.64
-23.62
-3.06
-57.71*
-20.32
Labour productivity
(DKK1,000)
6
Coeff.
-55.06
230.80
-12.40
-104.30
-63.98
50.05
-8.47
-10.69
63.38
82.76
Coeff.
-168.3
337.1
260.3
134.9
-333.1
413.9***
-6.6
339.9
403.0
701.6**
Ste.
172.7
217.5
206.9
414.8
230.7
150.8
186.6
208.4
339.2
276.1
Coeff.
-0.01
-0.04
0.01
0.01
-0.10
0.00
-0.01
0.00
0.04
0.03
Ste.
0.04
0.05
0.05
0.06
0.10
0.04
0.05
0.05
0.06
0.07
Ste.
19.75
14.09
23.65
27.10
49.85
20.67
19.63
26.29
32.57
52.61
Ste.
130.90
148.80
97.78
205.20
370.80
137.80
160.20
176.70
235.40
352.10
-218.3
35.5
-169.1
-171.8
-260.2
-497.4**
-874.0***
167.4
166.6
144.1
174.6
176.9
220.2
264.0
-0.0717*
0.00
-0.05
-0.03
0.00
-0.09
-0.09
0.04
0.03
0.04
0.05
0.05
0.05
0.07
21.43***
23.56***
15.60*
40.25***
36.60
39.29
39.78
7.97
8.29
8.73
15.14
22.51
28.01
31.08
-203.60
-31.67
-155.00
111.30
-105.00
-143.90
-130.50
123.00
84.93
125.70
154.60
192.80
204.90
246.20
188.7
894
192
0.06
118.9
0.04
977
193
0.03
0.03
-11.02*
557
125
0.04
6.52
76.85
647
119
0.05
95.41
2. Only observations with annual change in the value added of less than DKK 10 mio.
3. Only observations with annual change in net income of less than DKK 3 mio.
4. Only observations with annual change in roa of less than 1, and total assets
>
DKK 100,000.
5. Only observations with number of employees
>
5. Only observations with change in average wage
<
DKK 500,000.
6. Only observations with number of employees
>
5 and change in labour productivity
<
DKK 3 mio.
139
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0140.png
140
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0141.png
Analysis of the Industrial PhD Programme
Copenhagen, February 2011
141
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0142.png
CONTENTS
EXECUTIVE SUMMARY
SAMMENFATNING (DANISH SUMMARY)
1. INTRODUCTION
2. DESCRIPTION OF THE INDUSTRIAL PHD PROGRAMME
3. INDIVIDUAL LEVEL ANALYSIS
3.1 Results of the individual level analysis
4 . COMPANY LEVEL ANALYSIS
4.1 Data and methodology of the company level analysis
4.2 Results of the company level analysis
5. SUMMARY AND CONCLUSIONS
6. APPENDIX 1: SELECTION OF CONTROLS
143
145
147
149
150
151
159
159
164
177
178
142
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0143.png
EXECUTIVE SUMMARY
This report has been prepared by the Centre for Economic and Business Research
(CEBR). It presents an analysis of the economic impact of the Danish Industrial
PhD Programme on participating companies and on wage and career characteristics
of Industrial PhD graduates.
The Industrial PhD Programme is funded by the Danish Council for Technology
and Innovation and is administered by the Danish Agency for Science, Technology
and Innovation (DASTI). The programme subsidises PhD studies where the stu-
dent is employed in a private sector company and simultaneously enrolled as a PhD
student at a university.
The analysis follows approx. 430 individuals and approx. 270 companies that have
participated in the programme and for whom relevant data is available in the se-
lected registers.
On the individual level, we compare wage income and occupation of Industrial
PhDs with regular PhDs and other university level graduates.
On the company level, we analyse company level developments within four success
parameters:
•
the number of patents applications,
•
gross profit growth,
•
total factor productivity, and
•
employment growth.
For a sample of companies which have hosted a maximum of three Industrial PhD
projects, we identify a control group of highly similar companies which have not
hosted any Industrial PhD projects. We then compare developments in the success
parameters in these two groups. Under identifying assumptions, the difference
between the sample group and the control group isolate the causal impact of the
programme on companies hosting Industrial PhD projects.
The results of the analysis can be summarised as follows: Industrial PhDs earn ap-
prox. 7-10 percent higher wages than both regular PhDs and comparable university
graduates. This comparison is illustrated in FIGURE 1.
FIGURE 1: Hourly wage (DKK) in 2006, by Individual age
Industrial PHD Graduates
Regular PHD Graduates
31 32 33 34 35 36 37 38 39 40 41 42 43 44
Age (in years)
143
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0144.png
They are also more likely to be found at the top levels of their organisations’ hierar-
chies compared to normal PhDs and more likely to be found in positions requiring
high-level specialist knowledge than regular university graduates.
Companies that host Industrial PhDs see on average increasing patenting activity,
illustrated by FIGURE 2. They are characterised by high growth in gross profit,
and more positive developments in gross profit and employment growth than com-
panies in the control group. We are not able to identify robust relationships between
hosting Industrial PhD projects and total factor productivity developments.
FIGURE 2: Number of patent applications, high-quality
matches.
Average number of patent applications per company, change relative to year before
first initiating an Industrial PhD project
0,20
0,12
0,10
0,05
0,00
-0,05
-0,10
Age (in years)
-5 - 4 -3 -2 -1 0
1 2 3
4
5 6 7 8 9 10
Companies with Industrial PhD projects
Companies without Industrial PhD projects
144
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0145.png
SAMMENFATNING (DANISH SUMMARY)
Denne rapport er skrevet af Centre for Economic and Business Research (CEBR).
Den beskriver en analyse af ErhvervsPhD-ordningens potentielle effekter på ud-
viklingen i de deltagende virksomheder og løn- og karrieremønstre for personer,
som har erhvervet deres ph.d.-grad gennem ordningen.
Et ErhvervsPhD-projekt er et treårigt erhvervsrettet ph.d.-projekt, hvor den studer-
ende ansættes i en privat virksomhed og samtidig indskrives på et universitet.
Ved hjælp af registerdata følger analysen ca. 430 individer og 270 virksom-
heder, som har deltaget i ordningen. På individniveau studeres væksten i
ErhvervsPhD’ernes lønindkomst i forhold til almindelige ph.d.’ere og sammen-
lignelige kandidater.
For virksomheder studeres udviklingen i patentering, bruttofortjeneste, totalfak-
torproduktivitet og beskæftigelse. Hertil identificerer vi en gruppe af kontrolvirk-
somheder, som ikke ansætter en ErhvervsPhD, men som ellers ligner de ansættende
virksomheder i størrelse, branche, alder og region.
Dermed kan vi besvare spørgsmålet om, hvorvidt de virksomheder, som ansatte
en ErhvervsPhD, har haft en mere positiv udvikling i succesparametrene, end man
ville have forventet på basis af udviklingen for kontrolvirksomhederne.
Analysens resultater kan sammenfattes som følger:
Efter uddannelsens afslutning har ErhvervsPhD’er i gennemsnit mellem 7 og 10
procent højere lønindkomst end normale ph.d.’er og personer med en afsluttet uni-
versitetsuddannelse. Dette er illustreret i FIGUR 1.
ErhvervsPhD’ere har endvidere en væsentligt højere sandsynlighed for at blive
ansat i lederstillinger end almindelige ph.d.’ere og er stærkere repræsenteret i
FIGURE 1: Timeløn (i kr.) in 2006, efter alder
ErhvervsPhD
Normal PhD
31 32 33 34 35 36 37 38 39 40 41 42 43 44
Alder
145
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0146.png
gruppen af medarbejdere med jobfunktioner, som kræver specialviden på højeste
niveau. Virksomheder, som ansætter ErhvervsPhD’ere, har i gennemsnit højere
patenteringsaktivitet efter ansættelsen end før. Dette er illustreret i FIGUR 2.
FIGURE 2: Antal patentansøgniner, højkvalitetssammenligning
Gennemsnitlig antal patentansøgninger pr. virksomhed
(i afvigelser ift. året før første ErhvervsPhD-projekt)
0,20
0,12
0,10
0,05
0,00
-0,05
-0,10
År før/efter første
Erhvervs PhD
-5 - 4 -3 -2 -1 0
1 2 3
4
5 6 7 8 9 10
Virksomheder med ErhvervsPhD-
projekter
Virksomheder uden ErhvervsPhD-
projekter
De er også kendetegnet ved højere vækst i bruttofortjenesten/værdiskabelsen og har
en mere positiv udvikling i væksten i bruttofortjenesten og medarbejderantallet end
virksomhederne i kontrolgruppen.
Det er på nuværende tidspunkt ikke muligt at påvise, at ErhvervsPhD-ordningen
bidrager til højere vækst i virksomhedernes totalfaktorproduktivitet.
146
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0147.png
1
INTRODUCTION
This report has been prepared by the Centre for Economic and Business Research
(CEBR). It presents an analysis of the economic impact of companies participating
in the Danish Industrial PhD Programme in terms of growth and value creation,
and on wage income and career patterns of Industrial PhD graduates.
Even though this analysis is an evaluation of a specific Industrial PhD subsidy
programme, its results might be of general interest, as programmes similar to the
Danish Industrial PhD Programme have been implemented or are considered for
implementation in a number of countries. However, general knowledge of their ef-
fects which can be integrated into cost-benefit analyses of these programmes is still
rare.
The Industrial PhD Programme aims at increasing knowledge sharing between
universities and private sector companies, promoting research with commercial
perspectives, and taking advantage of competences and research facilities in private
business to increase the number of PhDs.
For this purpose, the Industrial PhD students typically spend 50 percent of their
time in a company and 50 percent of their time at a university while taking the de-
gree. The Danish Agency for Science, Technology and Innovation (DASTI) subsi-
dises the Industrial PhD’s salary with a fixed monthly amount, roughly correspond-
ing to 30-50 percent of the Industrial PhD’s total salary.
The Industrial PhD programme was initiated in 1971 under the name “The
Industrial Researcher Programme”. In 1988 it was made possible to qualify for
a PhD degree when graduating. The programme was subsequently reformed to
comply with Danish PhD regulations, making every graduate a formal PhD gradu-
ate. Until 2009, approx. 1,200 projects have been started. As part of its evaluation
policy, DASTI has asked CEBR to analyse the company and individual level effects
of the Industrial PhD Programme. The main questions of the evaluation are whether
and how participating in the Industrial PhD Programme is associated with com-
pany performance and, with regard to individuals, to what extent an Industrial PhD
degree is associated with future career developments, measured by wage income
and occupation.
To answer the questions outlined above, this analysis considers 430 individuals and
approx. 270 companies that have participated in the programme using a matched
employer-employer register dataset.
On the individual level, we compare wage income developments of Industrial PhD
graduates with regular PhD graduates and individuals with a university level de-
gree (and who have graduated at approximately the same time as the Industrial PhD
graduates).
147
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0148.png
On the company level, we analyse company level developments within four success
parameters:
•
number of patent applications,
•
gross profit growth,
•
total factor productivity (TFP), and
•
employment growth
Gross profit is defined as annual net sales subtracted annual costs of variable inputs
(raw materials, energy, intermediate goods purchases, etc.), except labour costs.
Thus, gross profit is a measure of the company’s value creation.
1
Total factor productivity is gross profit corrected for the company’s use of capital
and the number of employees. It is measured as the percentage-wise deviation of a
company’s gross profit from the gross profit that would have been expected on basis
of the company’s number of employees and its capital stock.
2
To identify innovation, growth and productivity effects of hosting an Industrial
PhD, we analyse increases in the number of patent applications, gross profit growth,
total factor productivity and employment growth for a sample of companies which
have participated in the Industrial PhD Programme. By using a control group of
highly similar companies which have not participated in the programme, we can
compare the developments of the success parameters of the two groups of compa-
nies to each other.
1 Gross profit is the most precise measure of the company’s value creation, but one should, of course, keep in
mind that a part of the company’s total value creation may be passed on to consumers, may be retained in the
company and increase its value (for which there is no data available for this analysis), or may take the form
of positive externalities, such as knowledge and/or innovations which benefit other companies or society in
general.
2 For this analysis, we measure TFP as the residuals of a Cobb-Douglas-production function estimation with
total assets and the number of employees as right hand side variables.
148
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0149.png
2
DESCRIPTION OF THE INDUSTRIAL PHD PROGRAMME
An Industrial PhD project is a three-year industrially focused PhD project where
the student is hired by a company and enrolled in a university at the same time.
3
The company receives a monthly wage subsidy of (currently) DKK 14,500 (approx.
€2,000) while the university has its expenses for supervising etc. covered. The PhD
student works full time on the project and divides his or her time equally between
the company and the university. There are additional subsidies available for project-
relevant stays abroad.
Currently, there are allocated annually approx. DKK 100-150 million (€15-20
million) for new projects. Approval rates for applications are currently above 60
percent.
Different aspects of the Danish Industrial PhD programme were addressed in
earlier evaluations. DASTI (2007a)
4
concludes that Industrial PhDs are character-
ised by earning higher wages and are more likely to be a part of their organisation’s
management compared to regular PhDs. Companies hosting Industrial PhD projects
expect increased patenting activity and growth.
DASTI (2007b)
5
lists several positive benefits for the participating companies.
Among other things, companies may gain new knowledge, patents and licenses,
growth and new market opportunities, and an increased network inside the aca-
demic world.
A similar conclusion is reached in a report by Right, Kjaer and Kjerulf from 2003.
Based on interviews with participating candidates and companies in 2002, they
find that a majority of companies expect the Industrial PhD to contribute to patents,
while close to half of all companies expect increased earnings.
International evaluations include a report from the European University
Association,
6
which concludes that participating candidates enjoy better employ-
ment opportunities due to improved skills. Two studies for the Swedish agency
KK-stiftelsen
7
have also been carried out. These conclude (a) that certain conditions
need to be met for projects to be successful, and (b) that the different stakeholders
of Industrial PhD projects report that the programme is achieving its goals.
3 This section draws extensively on the information published by DASTI.
4DASTI, 2007a: ”ErhvervsPhD - Et effektivt redskab for innovation og vidensspredning”.
5DASTI, 2007b: ”ErhvervsPhD - Ny viden til erhvervslivet og universiteterne”.
6European University Association, 2009: “Collaborative Doctoral Education - University-Industry partnerships for
enhancing knowledge exchange”.
7(a) KK-stiftelsen, 2003: ”KK-stiftelsens företagsforskarskolor - utvärdering av ett koncept för ökat samarbete
mellan akademi och näringsliv”.
(b) KK-stiftelsen, 2006: ”Småföretags- och institutsdoktorander för kunskaps- och kompetensutveckling”.
149
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0150.png
3
INDIVIDUAL LEVEL ANALYSIS
For the individual level analysis, information gathered by DASTI on participating
individuals was merged with public register information typically referred to as the
“Integrated Database of Labour Market Research (IDA)”. These data cover the pe-
riod from 1980 onward and contain information on a multitude of individual demo-
graphic background characteristics, like education, gender and age.
The IDA data have – for the period 1997 to 2006 – been merged with information
from the ”Wage Statistics Database”, which includes detailed information on wages
and occupation, including hierarchical levels.
Also, information from education-related registers has been added to the data, to
make it possible to control for inherent human capital endowments – pproximat-
ed by the grades of secondary education certificates – in the regressions.
The following analysis compares wages and careers of Industrial PhD graduates with:
(a) individuals with a university degree, but no PhD degree, and
(b) regular PhD graduates.
The validity of these comparisons depends on how similar the two groups are with
the Industrial PhD graduates and the potential to control for observable factors pre-
sumably related to educational choices and, later, income and career developments.
Both objectives raise some issues regarding the optimal sampling strategy, which is
presented in the following:
When selecting the sample for the analysis, we obviously include all individuals
who have completed an Industrial PhD education. Individuals who have completed
a regular PhD education form the first control group.
With regard to university graduates, who form the second control group of individ-
uals for comparison, there is an issue which needs to be resolved: There is a large
number of secondary educations where it is not entirely clear whether they should
be defined as university level educations or not.
We choose to address this issue by identifying the highest educational degrees of
the Industrial PhD graduates before obtaining their Industrial PhD degrees. As a
first step in the sampling procedure, we only select individuals with the same set of
educations for the control group.
But without further conditions on sampling, the educational fields of the Industrial
PhDs and the university graduates would be very different. For example, there
would be a large share of individuals with university degrees in arts and Humanities
in the control group, while these degrees are relatively uncommon in the group of
Industrial PhDs. This would bias any comparison between the two groups.
For this reason, we also align the composition of the educational fields of Industrial
PhDs prior to obtaining the Industrial PhD degree and the educational fields of the
control group by selecting a fixed number of individuals into the control group for
each Industrial PhD graduate.
150
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0151.png
Specifically, we select ten individuals into the control group for each Industrial
PhD. The number ten is a compromise between being able to find individuals with
the same educational degrees and a sample size large enough to isolate relationships
in the data.
These individuals, referred to as the group of ‘university graduates’ are randomly
selected, but must correspond to the educational field of the given Industrial PhD
graduate (before he/she obtains her PhD degree). In the selection process, we also
prefer persons of the same gender and origin (Danish vs. non-Danish), and persons
who are of similar age. This way, we base the comparisons on groups of individuals
similar not only in terms of their educational field, but also age, gender and origin.
The individual level analysis is based primarily on information for the year 2006,
which is the last year where the data provides detailed information on wages and
occupation.
3.1
Results of the individual level analysis
At present there are approx. 1,200 individuals who have participated or are partici-
pating in the Industrial PhD Programme. In year 2006, which is the last year for
which all relevant data is available, 999 Industrial PhDs can be identified in the
register data.
Of these, the register data shows 442 completed their projects, i.e. obtained their
Industrial PhD degree, before 2006.
Additionally, there is wage information for 430 of these 442 individuals.
The wage concept used in the following analysis is Statistics Denmark’s ‘nw’-var-
iable of the Wage Statistics Database. This variable is a description of the person’s
hourly wage income excluding pension contributions and cleaned for peculiarities
such as overtime, dirty work premiums, etc.
Career developments are measured by Statistics Denmark’s ‘disco’-variable, also
from the Wage Statistics Database. This variable categorises occupation by differ-
ent hierarchical levels and work functions. The question to be considered is whether
Industrial PhDs are over- or underrepresented in leadership positions (disco code
1000-1999) or positions which require high-skilled specialist knowledge – these
will be denoted as specialist positions in the following (disco code 2000-2999).
Descriptive statistics
In this subsection, we describe the gross sample of all individuals associated with
the Industrial PhD Programme – with or without completed Industrial PhD de-
grees - and of all individuals with a PhD degree in the last year in which they are
observed, and all individuals selected for the group of university graduates. This
ensures the most comprehensive description of these groups. However, when we
151
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0152.png
turn to the comparison of wages and career patterns, we will concentrate on those
individuals with completed educations (either university or PhD) in 2006.
When taking a first look at the data (TABLE 3.2.1), we find that in 2006, the last
year for which data is available for this analysis, Industrial PhDs earn approx. 10
percent lower wages than regular PhDs, but are, in the current sample, almost eight
years younger on average.
Industrial PhDs have slightly lower grades than normal PhDs in their secondary
education examinations, but this difference is negligible relative to this variable’s
variation.
About four percent of the Industrial PhDs are represented at the top of their or-
ganisation’s hierarchy, which is very similar to the two control groups. Approx. 60
percent of Industrial PhDs work in specialist positions, a share which locates them
between regular PhDs and university graduates, where this is the case for 74 and
44 percent, respectively. Obviously, we can expect differences in both wages and
positions to increase when focusing only on individuals with completed educations
in the next subsection.
TABLE 3.2.1: Descriptive statistics of the individual level data (2006), mean values
Industrial PhD
students and
graduates
Hourly wage (DKK)
Female
Age (year)
Grade of university-
entrance diploma
(standard deviation: 8.9)
Non-Danish origin
Leadership position
Specialist position
Number of
observations
228,61
0,35
34,55
91,83
0,06
0,04
0,58
999
Regular PhD
students and
graduates
223,44
0,34
42,66
92,63
0,08
0,04
0,74
12369
University
graduates
223,44
0,35
34,72
87,33
0,05
0,03
0,44
9625
All
243,06
0,34
38,98
90,07
0,07
0,04
0,61
22993
Cf. TABLE 3.2.2, we find that approx. 38 percent of all Industrial PhDs had a degree in engineering, and
another approx. 23 percent had degrees in chemistry or electronics engineering before receiving their
Industrial PhD degree.
8
Obviously, the group of university graduates is supposed to only consist of individuals who actually have
graduated. Thus, when we formally compare the different groups of individuals in the results subsection, please
note that we will not consider individuals registered as having a university-entrance diploma as their highest
educational degree in 2006.
8
152
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0153.png
TABLE 3.2.2: Highest educational degree in 2006 (for Industrial PhD and regular PhD
graduates: highest degree before receiving the PhD degree), in percent
Industrial PhDs
Regular PhDs
University
Graduates
32,95
0,08
4,01
11,16
11,42
All
Master's in engineering
Unknown
Master's in medical science
Master's in biology
Master's in chemical
engineering
Master's in electronics
engineering
University-entrance diploma
Master's in pharmaceutics
Master's in biochemistry
37,99
3,89
2,52
9,84
10,53
12,38
30,24
25,42
14,26
5,3
20,99
18,08
16,61
12,94
7,76
12,81
3,46
12,02
6,99
9,38
6,41
6,64
0,6
4,04
4,3
15,79
6,73
5,84
6,54
5,13
4,96
Before turning to wage and career comparisons, we take a brief look at the kind of
PhD degrees of Industrial and regular PhDs - see TABLE 3.2.3 for the most popular
subjects. We find these two groups to be quite different in their compositions of the
specific degrees. Consequentially, later comparisons of wages and careers between
Industrial and regular PhDs will have to take these differences into account.
Interestingly, there are a number of Industrial PhDs in medical sciences, yet only
relatively few of these individuals had medical science university degrees before
taking their PhD.
153
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0154.png
TABLE 3.2.3: Type of PhD degrees, in percent
Industrial PhDs
Technical sciences
Natural sciences
Other disciplines
Medical sciences
Veterinarian/agricultural
Pharmaceutical sciences
Social sciences
59,5
10,6
3,4
14,3
6,3
3,2
2,7
Regular PhDs
24,5
22,5
20,8
19,3
8,9
2,0
1,9
All
25,9
22,0
20,1
19,1
8,8
2,1
2,0
Results
As a first step, we compare average hourly wages of the different groups of indi-
viduals under consideration in 2006 and graph the averages as a function of age
in FIGURE 3.2.1.
FIGURE 3.2.1: Hourly wage (DKK) in 2006, by Individual age
Industrial PhD Graduates
Regular PhD Graduates
31 32 33 34 35 36 37 38 39 40 41 42 43 44
Age (in years)
154
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0155.png
We find that wages of Industrial PhD graduates are higher than those of both
regular PhD graduates and university graduates. Differences are largest in the
early and mid-forties. Regular PhDs have lower wages relative to both Industrial
PhDs and university graduates.
An obvious explanation of these differences may be sought in different employ-
ment patterns of the different groups of employees, with regular PhDs being
overrepresented in public sector research institutions, which are generally char-
acterised by lower wages than private sector employers.
Comparisons between Industrial PhDs and regular PhDs
In this section, we compare wages between Industrial and regular PhD gradu-
ates by using a linear regression, holding constant a set of (pre-determined)
background characteristics (age, gender, etc.).
These comparisons are based on the 430 Industrial and approx. 5,850 regular
PhD graduates. We choose a logarithmic specification of the wage variable,
implying that regression coefficients are the expected (approximately) percent-
age-wise changes in the wage when the condition of the associated explanatory
variable is fulfilled.
TABLE 3.2.4: Hourly wage of Industrial and regular PhD graduates, linear regression
results, dependent variable: log (hourly wage), sample: Industrial and regular PhD
graduates (2006)
Variables
The person is an Industrial PhD graduate
The person is female
Coefficient
0.090
-0.060
***
***
Standard
error
0.014
0.007
Coefficient
0.063
-0.065
***
***
Standard
error
0.014
0.007
The person is an immigrant (or descendant)
Grade of secondary education diploma (normalised)
Age (in years)
0.028
0.026
0.016
***
***
0.027
0.004
0.001
0.005
0.019
0.021
0.027
0.003
0.001
***
***
Additional controls
Secondary education: elective
courses (7 categories)
Secondary education: elective
courses (7 categories); specific
PhD degree (10 categories); age
when receiving the PhD degree
6.283
Number of observations
6.283
Notes: ***: significant at the 1% level. Heteroscedasticity-consistent standard errors.
155
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0156.png
The regression results confirm the findings of FIGURE 3.2.1: Industrial PhDs earn
approx. (exp(0.090)=1.094) 9 percent higher hourly wages compared to their counter-
parts who have taken a regular PhD degree. When including additional variables in
the regressions (which control for the different compositions of the subjects of the PhD
projects and for age differences when obtaining the PhD degree), the difference drops
to approx. 6 percent, but remains statistically highly significant.
Here, it might be noted that the result of positive wage income differences is robust
when considering gross hourly wages (i.e. total wage income including pensions
divided by the number of working hours) or annual income instead of the current
wage concept. In the first case, the relevant coefficient dropped to approx. 5 per-
cent (instead of approx. 9 percent). In the second case, when considering annual
income without correcting for working hours, the coefficient increased to between
10 (in the specification with additional controls) and 15 percent (in the more simple
specification). This indicates that Industrial PhDs register more working hours than
regular PhDs.
Comparing the career developments between the two types of PhDs, we first note
that 6.3 percent of Industrial PhD graduates are employed in leadership positions,
as opposed to 3.9 percent of regular PhD graduates. The formal comparison is by
estimating a so-called binary choice model (assuming a logistic distribution). The
coefficients of this model are displayed in TABLE 3.2.5.
TABLE 3.2.5: Occupation of Industrial and regular PhD graduates, binary choice (logit)
model, sample: Industrial and regular PhD graduates (2006)
Dependent variable:
The person has a
leadership position
Variables
The person is an Indu-
strial PhD graduate
The person is female
The person is an immi-
grant (or descendant)
Grade of secondary
education diploma
(normalised)
Age (in years)
Additional controls
Number of observations
Coefficient
1.087
-0.577
0.985
***
***
**
Standard
error
0.217
0.178
0.403
Dependent variable:
The person has a
specialist position
Coefficient
-0.738
0.268
-0.344
***
***
Standard
error
0.112
0.064
0.227
0.108
0.103
***
0.078
0.020
0.077
0.010
**
0.033
0.009
Secondary education: elective courses (7
categories)
7,214
Secondary education: elective courses
(7 categories)
7,214
Notes: ***: significant at the 1% level, ** significant at the 5% level.
156
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0157.png
The exponents of the model’s coefficients equal the increases in the probability that
the individual is a leader or a specialist when the logical conditions of the corre-
sponding variables are true. We find that Industrial PhDs are almost three times
more likely (exp(1.087)=2.95) to hold a leadership position than regular PhDs when
holding constant the set of background characteristics included in the regression.
Industrial PhDs have an approx. (exp(-0.738)=0.48) 50 percent lower probability of
being employed in a specialist position than regular PhDs. We conclude that, while
regular PhDs are almost entirely employed in specialist positions, Industrial PhDs
are more evenly distributed across the different occupational levels.
Comparisons between Industrial PhDs and university graduates
In the following, we compare wages between Industrial PhD graduates and univer-
sity graduates.
TABLE 3.2.6 summarises the results of the comparison of hourly wages. They
suggest that Industrial PhDs earn a wage premium of approx. 7 percent relative to
university graduates in similar fields of study while controlling for demographic
factors, secondary education grades and course specialisation.
TABLE 3.2.6: Hourly wage of Industrial PhD and university graduates, linear regression
results, de-pendent variable: log (hourly wage), sample: Industrial PhD graduates and
university graduates (2006)
Variables
The person is an Industrial PhD graduate
The person is female
Coefficient
0.066
-0.129
***
***
Standard error
0.014
0.009
The person is an immigrant (or descendant)
Grade of secondary education diploma (normalised)
Age (in years)
-0.053
0.043
0.028
***
***
0.034
0.005
0.001
Additional controls
Secondary education: elective courses (7 categories)
Number of observations
Notes: ***: significant at the 1% level. Heteroscedasticity-consistent standard errors.
5,246
157
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0158.png
Again, it may be noted that the result of positive wage income differences is unaf-
fected by considering gross hourly wages (i.e. total wage income including pensions
divided by the number of working hours) or annual income instead of the current
wage concept. In the first case, the relevant coefficient dropped to 0.056 (instead of
0.066). In the second case, when considering annual income without correcting for
working hours, the coefficient increased to 0.11 - again indicating that Industrial
PhDs register more working hours than other graduates.
The results of the career development comparisons are found in TABLE 3.2.7. In
comparison with university graduates, Industrial PhDs are overrepresented in lead-
ership positions and specialist positions. However, the difference regarding leader-
ship positions is not statistically significant and must be regarded as tentative.
For specialist positions, the coefficient 0.397 corresponds to an approx. 50 percent
higher probability that Industrial PhDs are employed in specialist positions than
university graduates.
TABLE 3.2.7: Occupation of Industrial PhD graduates and university graduates, binary
choice (logit) model, sample: Industrial PhD graduates and university graduates (2006)
Dependent variable: The person
has a leadership position
Variables
The person is an Industrial PhD graduate
The person is female
Coefficient
0.182
-0.741
***
Standard
error
0.217
0.156
Dependent variable: The
person has a specialist position
Coefficient
0.397
0.105
***
**
Standard
error
0.113
0.051
The person is immigrant (or descendant)
Grade of secondary education diploma (normalised)
Age (in years)
-0.572
0.139
0.095
**
***
0.710
0.064
0.015
-0.120
0.151
0.072
***
***
0.217
0.025
0.006
Additional controls
Secondary education: elective
courses (7 categories)
7,465
Secondary education: elective
courses (7 categories)
7,465
Number of observations
Notes: ***: significant at the 1% level, ** significant at the 5% level.
158
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0159.png
4
COMPANY LEVEL ANALYSIS
4.1
Data
Data and methodology of the company level
analysis
The data used for the company level analysis is from three sources:
•
First, data from DASTI on the participation of companies and individuals
in the Industrial PhD Programme.
•
Second, information on financial reports that companies above certain
size thresholds must file to a public authority.
•
Third, information on patenting activity from the European patent office.
The data from DASTI on the participation of companies and individuals in the
Industrial PhD Programme contain information on the year an individual was em-
ployed as an Industrial PhD student, and in many cases also the employing compa-
ny’s registration number (‘cvr-number’), which is filed at the public authorities and
which is also available in the other datasets used in this study.
Data on financial reports is from the private information provider company
Købmandsstandens Oplysningsbureau, now Experian A/S. This dataset, henceforth
denoted as the KOB data, contains information from the financial reports that com-
panies with a certain size and ownership structure must file to the public authorities.
Data on patenting is from the CEBR patent database, which has information on all
patent applications at the European Patent Office by at least one applicant residing
in Denmark.
The sample
In the original data from DASTI, there are 1,224 Industrial PhD projects in 536 dif-
ferent companies; 47 projects are registered as abandoned. Excluding these projects
from the sample (including one project which lacks information on when the project
was started) leaves us with 1,177 projects and 514 different companies.
However, it should be noted that in the original sample, the 514 different companies
are defined by their names. This number is partly due to registering the same com-
pany under slightly different names in the DASTI data.
For the following performance analysis, we have to merge the sample of 1,177 pro-
jects in 514 companies with the information from the KOB database.
To accomplish this, we first had to find company registration numbers (‘cvr’-num-
bers) of companies with missing or erroneous registration numbers in the original
DASTI data. We managed to find these registration numbers for 509 different
companies as defined by their names (hosting 1,161 projects). These 509 differ-
ent company names in the DASTI data correspond to 445 different companies as
defined by their company registration numbers. This is the definition of companies
we will use henceforth.
The first Industrial PhD projects were initiated in 1988. Up to 2003, the number of
projects initiated each year was relatively stable at approx. 30 to 50. However, in re-
159
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0160.png
cent years the number of projects initiated per year has increased steadily and is now
in the range of 80 to 120 projects. It should be noted that approx. 30 percent of the
companies in the sample have hosted more than one project, and that some companies
have hosted a considerable number of projects (e.g. more than 20).
1,053 out of the 1,161 projects and 387 out of the 445 companies can be identified in
the KOB database. A large share of the attrition is related to companies which have
either been established too recently to be covered by the KOB database or closed
down before the KOB database assumed full coverage.
For 383 companies, there is financial report information in the KOB data. These
companies are observed on average for 15.6 years, which implies that there are a
total of 5,018 annual financial reports for companies that have hosted at least one
Industrial PhD project. However, it should be noted that any potential bottom-line
effects of Industrial PhD projects may take a couple of years to materialise, and
that a considerable share of Industrial PhD projects was initiated at the end of the
observation period.
Of the 383 companies in the KOB data, 72 companies are not observed after first
initiating an Industrial PhD project. These obviously cannot be used for the following
analysis, leaving us with 311 different companies have hosted a total of 851 Industrial
PhD projects. Out of these, 195 companies have hosted only one Industrial PhD project,
48 percent have hosted two projects, 27 companies three projects, 9 companies four
projects, and 32 (approx. 10 percent) companies more than five projects. There are also
a few companies which have hosted more than 20 projects.
The companies with many projects are typically large companies for which it is
difficult, if not impossible, to find similar companies for the comparisons in the
statistical analyses to follow. Also, the statistical model which is preferred by the
precision of its estimates requires fixing a year before a company first participates
in the Industrial PhD Programme. For companies with many projects, this year is
not well-defined, and the year before hosting the first Industrial PhD is often before
the KOB database assumes full coverage.
Accordingly, we will only consider companies that have hosted a maximum of
three projects for the following analysis. These represent approx. 85 percent of all
companies participating in the Industrial PhD Programme, which leaves us with
270 companies for the company level analysis.
Of the 270 companies, approx. 120 are observed five years before first initiating an
Industrial PhD project, approx. 160 are observed five years after, and 86 are ob-
served ten years after first initiating a project.
9
However, it should be noted that missing information for a number of observations means that the number
of records which can be used for the analysis is reduced. For example, total factor productivity figures are
available for 91 companies five years after first initiating an Industrial PhD project, and for 46 companies ten
years after first initiating a project.
9
160
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0161.png
The characteristics of the companies in the sample used for analysis are de-
scribed in greater detail in the leftmost column of TABLE 4.2.1. In this table, we
also summarise the characteristics of two control groups of companies, which
are identified by a matching procedure briefly presented in the next section and
explained in greater detail in Appendix 1.
161
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0162.png
TABLE 4.2.1: Descriptive statistics of the matched treatment-control samples
All companies with a maximum
of three projects
Companies that have
hosted at least one
Industrial PhD project
Number of companies
Total factor
productivity
Gross profit per
employee (DKK1,000)
270
-0.056
Control
companies
All
companies
High-quality matches
Companies that have
hosted at least one
Industrial PhD project
129
0.090
Control
companies
All
companies
539
-0.006
809
-0.023
283
0.016
412
0.039
1529.4
689.5
971.9
445.5
466.6
460.0
Patent applications
Number of employees
Gross profit (DKK1,000)
Total assets (DKK1,000)
Establishment year
Industries
Business services
Research and
development
IT
Medical equipment,
instruments
manufacturing
Finance
Wholesale trade
Chemicals,
pharmaceuticals
Food production
Manufacturing
Other
Zip-codes
1000-1999
2000-2999
3000-3999
4000-4999
5000-5999
6000-6999
7000-7999
8000-8999
9000-9999
2.4
520.0
651460.8
18800000
1978.8
0.7
212.0
154181.9
951008
1977.8
1.3
314.8
320482.4
6894683
1978.2
0.8
28.8
14828.6
22515.98
1988.6
0.3
31.8
15841.2
22661.74
1988.2
0.5
30.9
15522.5
22616.1
1988.4
18.52
9.26
8.89
18.55
9.09
8.91
18.54
9.15
8.90
26.87
14.93
11.94
24.09
12.54
11.22
24.94
13.27
11.44
8.52
8.53
8.53
7.46
7.92
7.78
8.52
7.78
8.53
7.79
8.53
7.79
5.97
5.97
6.60
4.95
6.41
5.26
4.81
4.82
4.82
3.73
3.96
3.89
4.44
3.33
25.93
4.45
3.34
25.97
4.45
3.34
25.96
0.00
2.99
0.00
2.97
2.31
0.00
2.06
2.52
0.00
11.85
41.85
10.00
4.07
7.41
3.33
4.81
9.63
7.04
11.13
39.89
9.83
4.27
7.98
5.38
3.53
11.13
6.86
11.37
40.54
9.89
4.20
7.79
4.70
3.96
10.63
6.92
12.69
44.03
11.19
3.73
8.21
1.49
0.75
8.96
5.22
11.55
38.61
10.89
4.62
6.93
4.29
1.98
10.23
4.29
11.90
40.27
10.98
4.35
7.32
3.43
1.60
9.84
4.58
162
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0163.png
Unsurprisingly, we find that Industrial PhDs are typically hosted by companies in
knowledge-intensive industries. Also, hosting companies are geographically con-
centrated in the Copenhagen area (zip-codes below 3000).
Companies hosting Industrial PhD projects are, on average, relatively large compa-
nies with sometimes very high capital intensities (which is mostly due to the pres-
ence of large financial sector companies).
Methodology of the company level analysis
Our statistical model compares two groups of companies:
(a) companies that have hosted at least one Industrial PhD project, and
(b) companies that have not hosted any Industrial PhD projects.
In accordance with the academic project evaluation literature, the group of compa-
nies which have hosted Industrial PhD projects will henceforth be called the ‘treat-
ment group’, while the comparison group of companies which have not hosted any
Industrial PhD projects will be denoted as the ‘control group’.
When interpreting the results of the statistical comparisons, one must take into
account the fact that it is not possible to include all relevant factors in the models
because they are unobservable in the data. Examples include different kinds of
company competences and other immeasurable company characteristics.
This implies that interpreting any systematic treatment-control differences in com-
pany performance developments as genuine causal effects of hosting an Industrial
PhD project will have to rest on an ‘all-else-equal’ assumption, i.e. the assumption
that factors omitted from the model are either irrelevant or, on average, equal for
treatments and controls.
To maximise the validity of this ‘all-else-equal’ assumption, we identify the control
group using a matching procedure which ensures that we compare the treatment
group companies with a control group of highly similar companies.
The identification procedure is described in greater detail in Appendix 1. Here, it
may be sufficient to note that in the analysis to follow, we will compare develop-
ments in the success parameters over time of two groups of companies highly
similar in a number of observable characteristics.
Of interest in the following analysis is whether treatment group companies expe-
rience more positive developments in the success parameters in association with
hosting Industrial PhD projects compared to control group companies.
The modelling setup was chosen to generate the most precise estimates possible.
However, it should be noted that the associated before/after comparisons imply that
this procedure is only applicable to analysing companies that have hosted one or
very few projects, as otherwise the timing issue cannot be resolved.
As a compromise between the precision of the before/after time period definition
and having a sufficient number of observations for the analysis, we consider com-
163
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0164.png
panies with a maximum of three Industrial PhD projects. As noted earlier, these
companies represent approx. 85 percent of the participating companies.
The year that separates a company’s pre-participation period from its post-partici-
pation period will be denoted as “year 0” or the “base year”. For companies hosting
an Industrial PhD project, year 0 is defined as the year before initiating the first
Industrial PhD project. For a company in the control group, year 0 is the year in
which it most resembled one of the project hosting companies in its base year.
Using this method, we can measure participating companies’ developments in the
success parameters before and after their base year - the year before initiating the
first Industrial PhD project - and compare these developments to the developments
of the control group companies.
4.2
Results of the company level analysis
In the following sections, the results of the company level analysis will be de-
scribed. Here, two introductory remarks should be made:
Firstly, it must be assumed that it is practically impossible to isolate any per-
formance effects of hosting Industrial PhD projects on large companies, as any
contribution of an Industrial PhD project on aggregate company performance
would be small relative to the companies’ considerable heterogeneity in the success
measures. For this reason, we will also present results for an alternative sample
where companies with more than 300 employees or total assets of at least DKK 100
million in year 0 are not considered. This sample will be denoted the ‘sample of
small companies’.
Secondly, it proved to be difficult to find highly similar control companies for a
number of treatment companies. For this reason, we also consider a separate sample
of companies with less than 300 employees and total assets of less than DKK 100
million where these low-quality matches are excluded. This results in a sample of
highly similar treatment and control group companies, denoted as the sample of
‘high-quality matches’.
Before turning to the comparisons of the company performance parameters, we
will address the question of how successful the matching procedure is in finding
highly similar groups of treatment and control companies. Turning back to TABLE
4.2.1, which is a snapshot of the companies in year 0, we can compare the observ-
able characteristics of the treatment and control group companies – both for the
sample of all companies with a maximum of three Industrial PhD projects and their
corresponding control companies, and for the sample of high quality matches.
10
Note that the sampling procedure implies that the base years of the two groups of companies is distributed
highly similarly over time.
10
164
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0165.png
While industry and geographical distributions are almost identical for treatment and
control group companies in the two samples (implied by the matching procedure),
some of the very large Industrial PhD companies in the sample of all companies
lack counterparts in the control group. In this sample of all companies, treatment
group companies have a lower total factor productivity and a higher gross profit
(which is consistent with a higher capital intensity) than the control group compa-
nies. However, the large heterogeneity in these variables implies that these differ-
ences are not statistically significant.
Companies in the high quality match sample are on average considerably smaller,
younger and, of course, generally more similar in their observable characteristics.
We conclude that it was possible to find highly similar matches in terms of geo-
graphic location and company age. For the sample of high-quality matches, controls
are also highly similar in company size.
Patenting activity
Patenting activity is measured by the company’s number of patent applications per
year.
11
To isolate any Industrial PhD programme participation effects, we calculate for every
company and year the difference between the number of patent applications in the
given year and the number of patent applications filed in year 0.
FIGURES 4.2.1-3 display developments of these differences, i.e. current patenting
activity relative to activity in year 0 for treatment and control group companies,
respectively.
We find large movements over time for companies that host Industrial PhD projects
relative to companies in the control group. This is likely to be a result of generally
higher absolute patenting activity in treatment companies.
FIGURE 4.2.1: Number of patent applications, all companies
1,00
0,08
0,06
0,04
0,02
0
-0,02
-0,04
-0,06
-5
-3
-1
1
3
5
7
9
Years before/after
year o
Companies with
Industrial PhD projects
Companies without
Industrial PhD projects
An alternative measure would have been to consider granted patents. However, the long patent approval
process renders it difficult to associate this variable to current innovation output.
11
165
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0166.png
FIGURE 4.2.2: Number of patent applications, small companies
Average number of patent applications per company relative to year 0
-5
-4
-3
-2
-1
0
1
2
3
4
5
6
7
8
9
10
Years before/after
year o
Companies with
Industrial PhD projects
Companies without
Industrial PhD projects
FIGURE 2: Number of patent applications, high-quality matches
Average number of patent applications per company, change relative to year before first initiating an
Industrial PhD project
-5
-4
-3
-2
-1
0
1
2
3
4
5
6
7
8
9
10
Years before/after
year o
Companies with
Industrial PhD projects
Companies without
Industrial PhD projects
All graphs indicate that after year 0, the developments over time for treatment com-
panies are equal to or larger than developments for control companies, indicating
greater increases in patenting activity for the group of treatment companies com-
pared to the group of control companies.
One could note that there are also differences between pre-base year trends in
patenting activity depending on the sample under consideration, indicating the dif-
ficulties of finding control companies with patenting activities similar to the compa-
nies hiring Industrial PhDs.
166
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0167.png
Whether or not one is willing to interpret the graphs as evidence of positive effects
of hosting Industrial PhD projects depends on one’s underlying assumptions. E.g.
in FIGURE 4.2.1, there is a positive trend of increasing patenting activity before
hosting the first Industrial PhD project, but not in the years after. But over a longer
time horizon, activity is higher after year 0 than before. So the interpretation of the
results depends on whether one assumes that:
(a) trends would continue in the absence of programme participation, or,
(b) activity would stay at the same level in the long run in the absence of the
programme.
The estimates of the statistical model presented below will be based on a pre-partic-
ipation period specified as the five years up to year zero, and the post-participation
period as the ten years after year zero. Obviously, the lengths of these time periods
are computed are arbitrary, and the robustness of later results when choosing differ-
ent before/after time intervals needs to be checked in the numerical analysis.
A look at the raw data reveals that participant firms in the sample of high-quality
matches apply for on average 0.07 patents per year before year zero, and 0.18 after
year zero (i.e., an increase of 0.11). Control firms have almost the same patenting
activity both before and after year zero. Under the assumption that both groups of
firms would have experienced the same developments in their patenting activity
in the absence of the programme, the programme increases patenting activity with
0.11 patent applications per year.
To address the robustness of the graphs’ suggestions and to quantify the strength of
these associations in the data, we apply a model that estimates the expected per-
centage-point changes in the number of patent applications in a given year depend-
ing on whether the company is a treatment or a control company, and on whether
the year under consideration is before or after the base year.
The results of this model are presented in TABLE 4.2.2. Of particular interest are
the coefficients for the variable “The observation is after the base year and belongs
to an Industrial PhD company”. Under the assumption that patenting of treatments
and controls would develop in similar ways in the absence of the programme, this
variable identifies the genuine causal effect of hosting an Industrial PhD project on
patenting activity.
167
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0168.png
TABLE 4.2.2: Count data regression results, dependent variable: number of patent
applications in a given year. The table presents exponentiated coefficients, i.e. multiples of
the number of patent applications when the logical conditions of the associated variable are
fulfilled.
Sample: all compa-nies with a
maximum of three Industrial
PhD projects
Sample: small
companies
Sample: high-
quality matches
Variable
The observation is
after year 0
The observation
belongs to an
Industrial PhD
company
4,13
***
4,17
***
4,36
***
0,88
0,76
0,84
The observation is after
year 0 and belongs to an
Industrial PhD company
1,70
**
2,19
**
1,94
*
Constant term
0,06
***
0,04
***
0,04
***
Notes: ***: significant at the 1% level; **: significant at the 5% level; *: significant at the 10% level. All regressions based on STATA Corp.’s ’xtpoisson’
routine.
Findings of the statistical analysis of patenting activity can be summarised as fol-
lows: We find positive potential effects of hosting an Industrial PhD project for
the sample of all companies with a maximum of three Industrial PhD projects.
According to the estimates, hosting an Industrial PhD almost doubles (1.70) the
number of patents per year in the years after year 0.
For the other samples, associations between hosting Industrial PhD projects and
changes in patenting activity are also positive, and stay significant the ten-percent
significance level also for the considerable reduced sample of high-quality matches.
In sum, one can conclude that there is evidence of positive associations between
hosting Industrial PhD projects and changes in patenting activity in the data.
12
These relationships were robust when changing the lengths of the before- and after-base year periods con-
sidered for the estimations. Also, computing average numbers of patents of both participants and controls
both before and after year zero, and estimating a linear model of the pre-post base-year differences revealed
very similar (and also statistically significant) results.
12
168
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0169.png
Gross profit growth
The analyses of gross profit and TFP in the next subsection follow the same blue-
print as the previous look at patenting activity. Recall that gross profit is the surplus
of annual revenues over costs (excluding wages), and accordingly measures the
value creation of a company in a given year.
First, for every year we calculate the difference between the year’s gross profit
and the gross profit in the base year (the year before the company first initiated an
Industrial PhD project). Next, we calculate the average of these differences for both
the group of treatment companies (which have hosted Industrial PhD projects) and
the group of control companies (which have not hosted any Industrial PhD project).
FIGURES 4.2.4-6 show these averages for the treatment and control companies
for the three different samples. They suggest that companies which host Industrial
PhD projects are characterised by high growth in gross profit. While FIGURE
4.2.4, which compares all sampled companies both with and without Industrial PhD
projects, show a decrease in the growth trend in association with hosting the first
Industrial PhD project, FIGURE 4.2.5 and FIGURE 4.2.6, respectively comparing
small companies and high quality matches, show a consistent gross profit growth
which has no equivalent in the corresponding control group’s gross profit growth
pattern.
FIGURE 4.2.4: Gross profit developments (in DKK1,000), all companies
Average values relative to year 0
150.000
100.000
50.000
0
-50.000
-100.000
-150.000
Years before/after
year o
-5
-4
-3
-2
-1
0
1
2
3
4
5
6
7
8
9
10
Companies with
Industrial PhD projects
Companies without
Industrial PhD projects
169
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0170.png
FIGURE 4.2.5: Gross profit developments (in DKK1,000), small companies
Average values relative to year 0
200.000
150.000
100.000
50.000
0
-50.000
-100.000
-150.000
-5
-4
-3
-2
-1
0
1
2
3
4
5
6
7
8
9
10
Years before/after
year o
Companies with
Industrial PhD projects
Companies without
Industrial PhD projects
FIGURE 4.2.6: Gross profit developments (in DKK1,000), high-quality
matches
Average values relative to year 0
200.000
150.000
100.000
50.000
0
-50.000
-100.000
-150.000
-5
-4
-3
-2
-1
0
1
2
3
4
5
6
7
8
9
10
Years before/after
year o
Companies with
Industrial PhD projects
Companies without
Industrial PhD projects
Thus, assuming that treatment group companies in the absence of the programme
would experience similar gross profit growth (or in this case: a similar decline in
growth) as control group companies, the vertical distance between the graphs shows
a considerable genuine (causal) effect on the gross profit growth of companies host-
ing Industrial PhD projects.
We turn now to formally estimating before/after year 0 differences in gross
profit growth for treatment and control groups respectively.
13
We consider before/after differences in growth rather than levels, since gross profit levels show clear time
trends which need to be taken into consideration in the estimations to avoid generating biased estimates.
13
170
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0171.png
Accordingly, we divide each company’s observation period into two periods: one
period before year 0, and one period after year 0. For every company and for both
two periods, gross profit growth is measured by the average of the annual (absolute)
increases in gross profit.
We can now compare these averages both over time and between treatment and control
group companies. In the statistical model, we use the same variables as in the count
data regression used of the patenting analysis as right-hand-side variables.
Hence, the difference between the developments in gross profit growth before/after
year 0 for treatment group companies and the gross profit growth developments
before/after year 0 for control group companies is measured by the coefficient as-
sociated with the variable:
“The observation is after the base year and belongs to
an Industrial PhD company”
14.
The results of this comparison, which is again carried out by using a simple linear
regression model, are summarised in TABLE 4.2.3. The table shows the results for
high-quality matches, i.e. the treatment and control group companies most similar
to each other with regard to their observable characteristics, and for which the com-
parison accordingly has the highest validity.
TABLE 4.2.3: Linear regression results, dependent variable: annual increase in gross profit
(in DKK1,000, in prices of 2007), sample: high-quality matches.
Observation period: three years
before to five years after year 0
Observation period: three years
before to ten years after year 0
Coefficient
Standard
error
*
797,83
Variable
The
observation is
after year 0
The
observation
belongs to an
Industrial PhD
company
The observation is after year 0
and belongs to an Industrial PhD
company
Constant term
Number of
observations
Coefficient
Standard
error
**
764,36
-1792,35
-1422,68
-458,33
905,19
-458,33
906,09
2267,23
*
1259,42
1458,82
1457,88
1488,27
381
**
607,25
1488,27
321
**
607,86
Notes: ***: significant at the 1% level; **: significant at the 5% level; *: significant at the 10% level. Estimated with heteroscedasticity-
consistent standard errors.
E.g., if the increase in annual gross profit of treatment companies is on average DKK 5m before the base year
and DKK 7m after year 0, and if gross profit for control firms increases on average DKK 3m before and DKK 4m
after year 0, the coefficient associated with “The observation is after the base year and belongs to an Industrial
PhD company”, measured in DKK, is equal to (7m-5m)-(4m-3m)= 1m.
14
171
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0172.png
In the first model, which compares growth trends in the three-year period before
and the five-year period after year 0, the coefficient of
“The observation is after
year 0”
(-1,792.35) suggests that gross profit growth has slowed down by almost
DKK 2m per year.
But the estimate of the coefficient for the variable “The
observation is after the
base year and belongs to an Industrial PhD company”
of 2,267.23 implies that
gross profit growth of Industrial PhD companies maintains its positive trend. So,
for Industrial PhD companies, growth after first initiating an Industrial PhD project
is approx. DKK 2m higher per year than would otherwise be expected if they had
experienced a similar decline in gross profit growth as the control group companies.
So the approx. DKK 2m growth difference per year, implying an additional gross
profit of (2+4+6+8+10) DKK 30m in the first five years of programme participation,
is the genuine causal effect of programme participation, assuming that Industrial
PhD companies’ growth in gross profit would otherwise have followed the exact
same pattern of the control companies if they had not participated.
It becomes clear that the programme might be considered successful even if only
a part of this difference is because of a genuine causal effect of the Industrial PhD
Programme.
When we compare the growth patterns of the two groups of companies between
both the three-year time period before and the ten-year time period after year 0,
the difference still suggests higher growth for participating companies, but be-
comes statistically insignificant (i.e. it becomes more likely that the finding is
coincidental).
Total factor productivity
For this analysis, total factor productivity (TFP) was calculated on an annual basis
for all companies in the entire KOB database in the given year.
Total factor productivity is gross profit ‘corrected for’ the number of employees and
total assets. It is calculated as the residuals of a Cobb-Douglas-production function
regression. In other words, TFP is the share of the company’s value creation which
cannot be explained by its number of employees or its capital stock.
Thus defined, TFP approximates the percentage-wise deviation in gross profit from
the gross profit that we would have expected to observe, given the company’s num-
ber of employees and its stock of assets.
For the analysis, we first take a look at the developments using a graphical depiction
of the data. FIGURES 4.2.7-9 summarise.
172
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0173.png
FIGURE 4.2.7: Total factor productivity developments, all companies.
Average values relative to year 0
0,15
0,10
0,05
0
-0,05
-0,10
-0,15
-0,20
-5
-4
-3
-2
-1
0
1
2
3
4
5
6
7
8
9
10
Years before/after
year o
Companies with
Industrial PhD projects
Companies without
Industrial PhD projects
FIGURE 4.2.8: Total factor productivity developments, small companies
Average values relative to year 0
0,15
0,10
0,05
0
-0,05
-0,10
-0,15
-5
-4
-3
-2
-1
0
1
2
3
4
5
6
7
8
9
10
Years before/after
year o
Companies with
Industrial PhD projects
Companies without
Industrial PhD projects
FIGURE 4.2.9: Total factor productivity developments, high-quality matches
Average values relative to year 0
-5
-4
-3
-2
-1
0
1
2
3
4
5
6
7
8
9
10
Years before/after
year o
Companies with
Industrial PhD projects
Companies without
Industrial PhD projects
173
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0174.png
The figures illustrate that developments are very different depending on whether or
not large companies are excluded from the sample observed.
While there is a negative trend in TFP for the sample of all companies, there are
no such trends for the subsamples. The erratic movements in the graphs (in spite of
smoothing) suggest large heterogeneity in TFP over time and between companies.
For the subsamples, which are unaffected by the presence of very large companies,
TFP is between 5 to 10 percentage points higher approx. two to six years after year
0 in the subsamples.
Again, we qualify the suggestions of the graphs by use of linear regression, the
results of which are depicted in TABLE 4.2.4.
TABLE 4.2.4: Linear regression results, dependent variable:
(TFP in a given year) - (TFP in year 0)
Sample: all companies with a
maximum of three Industrial
PhD projects
Variable
Coefficient
Standard
error
The
observation
is after year
0
-0,084
***
0,023
-0,050
0,036
-0,066
*
0,038
Sample: small
companies
Coefficient
Standard
error
Sample: high-quality
matches
Coefficient
Standard
error
The
observation
belongs to
an Industrial
PhD
company
The
observation
is after
year 0 and
belongs to
an Industrial
PhD
company
Constant
term
0,003
0,014
0,013
0,019
0,019
0,020
-0,002
0,040
0,042
0,064
0,068
0,067
0,057
0,028
0,008
0,043
-0,027
0,043
Notes: ***: significant at the 1% level; **: significant at the 5% level; *: significant at the 10% level. Estimated with
heteroscedasticity-consistent standard errors
174
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0175.png
We see the negative TFP trends of companies hosting Industrial PhD projects and
their counterparts in the high-quality match control group corroborated by the nega-
tive coefficients associated with the variable
“the observation is after year 0”.
Also, TFP has increased more (or decreased less) in the treatment group compa-
nies compared to the control group the samples of small companies and that of the
high-quality matches.
This is indicated by the positive coefficients of the variable
“the observation is after
year 0 and belongs to an Industrial PhD company”,
which, for high-quality matches,
show that companies which have hosted Industrial PhD projects have on average ap-
prox. 7 percentage points higher TFP than would otherwise be expected if they had
experienced a TFP development similar to the control group companies.
Under the assumption that treatment group companies would experience TFP de-
velopments similar to those for control group companies in the absence of initiating
Industrial PhD projects, this 7 percentage point difference is the most qualified as-
sumption of the Industrial PhD Programme’s causal total factor productivity effect.
However, although positive, the TFP differences between treatment and control
groups are too small compared to the large variations in TFP to interpret them as
statistically significant, and must accordingly be interpreted tentatively. In con-
clusion, one cannot claim any strong association between hosting Industrial PhD
projects and TFP development.
15
Employment growth
We conclude the company level analysis by taking a look at employment growth.
The finding of high growth in gross profit but not in total factor productivity might
be an indication that companies hosting Industrial Phd projects are high-growth
companies. This is strongly supported by a closer look at the data, illustrated by
FIGURE 4.2.10, with companies hosting Industrial PhD projects being character-
ised by high growth in their number of employees both before and after first initiat-
ing a project.
To establish the statistical significance of this result, we formally test the growth
difference by means of linear regression, the results of which (for high-quality
matches) are presented in TABLE 4.2.5. The results of these regressions suggest
that companies participating in the programme sustain an annual employment
growth of approximately (-3.48-1.33+3.44+2.95=) 1.58 employees per year in the
first five years after first initiating an Industrial PhD project, while companies in
the control group decrease their number of employees by approximately -(2.95-
3.48=) 0.5 employees per year. Qualitatively, this finding is independent of whether
one follows the firms for five or ten years after the base year, and is statistically
highly significant.
This finding was robust to changes of the lengths of the time periods before and after the base year which
were considered in the regressions. The findings was also robust to changing the regression model, e.g.
using each firm’s average total factor productivity in the time periods before and after the base year as the
dependent variable, or using different specifications of the production function which was employed for the
calculation of TFP.
15
175
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0176.png
FIGURE 4.2.10: Number of employees developments, high-quality matches
Average values relative to year 0
-5
-4
-3
-2
-1
0
1
2
3
4
5
6
7
8
9
10
Years before/after
year o
Companies with
Industrial PhD projects
Companies without
Industrial PhD projects
TABLE 4.2.5: Linear regression results, dependent variable: annual increase in number of
employees. Sample: high-quality matches
Observation period: three years before to
five years after year 0
Variable
Coefficient
Standard error
Observation period: three years
before to ten years after year 0
Coefficient
Standard
error
***
0.73
The observation is after year 0
The observation belongs to an Industrial
PhD company
The observation is after year 0 and
belongs to an Industrial PhD company
Constant term
Number of observations
Notes: ***: significant at the 1% level; **:
significant at the 5% level; *: significant
at the 10% level. Estimated with
heteroscedasticity-consistent standard
errors.
-3.48
***
0.77
-3.13
-1.33
1.00
-1.33
1.00
3.44
2.95
349
***
***
1.18
0.65
2.73
2.95
267
**
***
1.19
0.65
Notes: ***: significant at the 1% level; **: significant at the 5% level; *: significant at the 10% level. Estimated with heteroscedasticity-
consistent standard errors
176
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0177.png
5
SUMMARY AND CONCLUSIONS
>
This analysis considers approx. 430 individuals and 270 companies which have
participated in the Industrial PhD Programme and can be found in register data. On
the individual level, we compare wage income and the occupations of Industrial PhD
graduates with regular PhDs and individuals who have a university degree (and who
are similar in terms of their fields of study, gender, etc.).
In the analysis, we take into account a set of demographic background characteris-
tics, like age and gender, but also the average grade of the school-leaving examina-
tion, which to some extent controls for individual abilities.
On the company level, we analyse developments across four success parameters: the
number of patents, gross profit and employment growth and total factor productiv-
ity. For a sample of companies which have hosted a maximum of three Industrial
PhD projects before 2009, we identify a control group of highly similar companies
which have not hosted any Industrial PhD projects, and compare developments in
the success parameters between these two groups. Under identifying assumptions,
these models isolate the causal impact of the programme on companies hosting
Industrial PhD projects.
The results of the analysis can be summarised as follows: Industrial PhD earn ap-
prox. 7-10 percent higher wages than both regular PhDs and university graduates.
They are more likely to be found at the top levels of their organisations’ hierarchies
compared to regular PhDs and more likely to be found in positions requiring high-
level specialist knowledge than regular university graduates. Companies which host
Industrial PhD projects see on average increasing patenting activity in association
with hosting the projects. They are characterised by high growth in gross profit
(value creation) and employment.
The comparison with a control group of highly similar control companies sug-
gests that companies hosting Industrial PhD projects would have considerably less
positive gross profit and employment developments if they did not participate in the
programme.
We cannot find robust differences in total factor productivity developments between
companies which have hosted Industrial PhD projects and companies which have
not. This finding might be due to firm growth being negatively associated with pro-
ductivity developments.
16
The relative high wages of Industrial PhD graduates, on
the other hand, indicate that they have high individual productivity.
Summing up, earlier studies which found that Industrial PhDs are characterised by
positive labour market outcomes have been corroborated. Findings on the company
level indicate that the Danish Industrial PhD Programme also has positive effects
for participating companies in terms of firm growth and patenting activity.
This would be the case if there are decreasing returns to labour, which is one of economic theory’s most stan-
dard arguments. Empirical support for this argument can be found in: Bingley, P., Westergaard-Nielsen N., 2004,
“Personnel policy and profit.” Journal of Business Research 2004; 57: 557-563.
16
177
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0178.png
6
APPENDIX 1: SELECTION OF CONTROLS
The KOB dataset is a panel dataset which has repeated observations for most of the
companies - one for each annual account filed to the authorities. So for every com-
pany, there are typically multiple company-year observations (where a company-
year observation refers to a record, i.e. a data-point of a given company in a given
year). In the following, we will use the expression ‘control observation’ to describe
a single company-year observation (record) of a control company.
Control companies are selected in the year in which they most closely resemble one
of the companies participating in the programme, based on the participating com-
pany’s characteristics in the year before hosting its first Industrial PhD project.
Note that the similarity between participating companies and potential control com-
panies is determined by (a) the companies’ region, size, age and industry, and (b)
the expected probability of participation, which is derived as follows:
We run an auxiliary regression on the universe of approx. 370,000 company-year
observations in KOB in the period from 1994 to 2008 which roughly resemble the
group of participants (for example, we do not consider industries in which there is
no single participating company).
The auxiliary regression is formulated as a simple probit model where the depend-
ent variable is initiating an Industrial PhD project the following year, and company
size, industry, region, productivity, total assets and time period as the model’s right-
hand-side variables. The regression’s pseudo R squared, which is a measure of the
model’s goodness-of-fit, is 0.29, which we consider to be high.
The probit regression predicts how likely programme participation is for a given
company. This allows us to find pairs or groups of companies for which this pre-
dicted probability is very similar. For two companies, A and B, with similar partici-
pation probability, the fact of company A participating and company B not partici-
pating can accordingly be interpreted as coincidental.
Under this interpretation, the identification setup resembles an experiment where
programme participation is random, which would allow systematic differences in
outcome variables between participants and controls to be interpreted as the pro-
gramme’s causal effect on participating companies.
Yet, even companies with similar predicted participation probabilities can be quite
different, and to avoid systematic differences in industry affiliation, size, etc.
between participants and controls, we also require that a number of observable
characteristics are equal for a given participant and its matched control company(s).
To do this, we divide the total number of company-year observations into groups
with the same industry affiliation, same geographic location, of similar size and
observed in the same year.
178
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0179.png
For each participating company, we select the company-year observation of a
non-participating company within the same group and with a participation prob-
ability closest to the participating company’s participation probability. This selected
company-year observation defines the participating company’s control company,
and the control company’s ‘base year’ (or ‘year 0’) – which is the year in which it
is most similar to one of the participating companies in its base year, and in which
it is selected as a control company. For each of the control companies found by this
procedure, the base year forms the basis for comparisons of given success param-
eters over time.
By repeating the matching procedure, we can find an arbitrary number of control
observations for each participant. Here, a greater number of control observations
increases the robustness of later results. However, increasing this number also
makes it increasingly difficult to find highly similar control observations for some
of the participants.
As a compromise between these two considerations, we choose to find two control
observations (company-year observations of non-participants) for each participating
company. The selection of the two control observations per participating company
is made in two rounds. In each of the rounds we select one control observation for
each participating company.
In the first round, we find 270 control observations of non-participating companies.
In the second, we find another 269 control observations of non-participating com-
panies (the reason for only 269 instead of 270 is that in a single case, one company-
year observation is chosen as a control observation for two participants).
In each of the two rounds, we first require that many factors are highly similar
when selecting control observations. This leaves a number of participating com-
panies for which no control observations could be found. In subsequent steps, we
reduce the number of factors and start choosing control observations which are
increasingly less similar, until each round has identified one control observation for
every participating company.
When control observations are equal in terms of industry (when distinguishing
between at least 36 different categories), number of employees (at least 11 differ-
ent categories), gross profit (at least 7 categories), time period (at least 7 different
categories) and company age (at least 3 different categories), they are regarded as
‘high-quality matches’ in the analysis.
Note that in each of the rounds, we select only one control observation per par-
ticipating company. This does not rule out selecting different control observations
(belonging to different years) of the same control company. This implies that there
are a number of control observations that occur more than once in the data forming
the basis.
179
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0180.png
180
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0181.png
The Innovation Consortium Scheme
– an analysis of firm growth effects
Copenhagen, April 2010
181
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0182.png
TABLE OF CONTENTS
EXECUTIVE SUMMARY
SAMMENFATNING (DANISH SUMMARY)
1. INTRODUCTION
2. DESCRIPTION OF THE IC SCHEME
3. DATA
4. SAMPLING
5. IDENTIFICATION OF THE CONTROL GROUP
6. ESTIMATION SET-UP
7. DESCRIPTIVE STATISTICS
8. RESULTS
8.1 Gross Profit Developments
8.2 Employment Developments
9. ALTERNATIVE SAMPLES AND ROBUSTNESS
10. CONCLUSIONS
11. REFERENCES
APPENDIX 1: SELECTION OF CONTROLS
APPENDIX 2: ILLUSTRATION OF THE DIFF-IN-DIFF ESTIMATION SETUP UP
APPENDIX 3: EXIT AND SURVIVAL OF PARTICIPANTS AND CONTROLS
183
185
187
189
190
191
193
194
196
199
199
205
210
216
218
219
225
228
182
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0183.png
EXECUTIVE SUMMARY
This report has been prepared by the Centre for Economic and Business
Research (CEBR). It presents an analysis of the economic impact of
’Innovationskonsortieordningen’ (Innovation Consortium scheme, IC scheme) on
participating firms.
The IC scheme is a Danish subsidy scheme granted by Rådet for Teknologi og
Innovation (The Danish Council for Technology and Innovation, RTI) in coopera-
tion with Forsknings- og Innovationsstyrelsen (The Danish Agency for Science,
Technology and Innovation, FI).
This analysis follows 220 firms which have participated in at least one Innovation
Consortium using a firm-register dataset. We primarily study firm level develop-
ments in two success parameters: gross profit and employment.
While employment is simply the number of employees in a given firm at a given
point in time, gross profit is a measure of the firm’s value creation.
In this study, we consider (absolute and percentage wise) growth in gross profit
and the number of employees both before and after programme participation and
analyse the changes in the growth patterns in association with participating in the
programme. Moreover, we identify a control group of firms that do not participate
(non-participants), but which are similar to the participants in terms of size, industry,
and region.
Again, we can use firm-level data to calculate the changes in gross profit and
employment for the non-participants, allowing us to address the question of whether
participants have higher increases in growth than what would be expected on basis
of the growth patterns of non-participants.
Under the assumption that gross profit and employment developments of participants
and non-participants would be symmetric in the absence of programme participa-
tion, differences between the two groups of firms can be interpreted as the causal
impact of the programme on participating firms.
The results of the analysis can be summarized as follows: It is possible to show that
firms that participated in the IC scheme have experienced significant increases in the
growth of gross profit and employment in association with programme participation.
These results are robust to controlling for pre-participation growth and developments
in the growth of firms in the control group.
183
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0184.png
Findings depend on the participant firms under consideration.
1
We, for example, find
positive potential gross profit effects that are significant at the five percent signifi-
cance level for firms with a gross profit below 150 million DKK (approx. €20) in the
year before the programme. We also find potential employment effects for firms with
less than 150 employees in the year before the programme.
For firms with gross profit less than DKK150 million in the year before partici-
pation, estimates show that, on average, annual gross profit in a participating firm
has grown by an additional approx. DKK2 million per year relative to firms in the
control group. This implied an on average approx. DKK20 million difference in
annual gross profit after 10 years. It should be noted that one should be careful when
interpreting this result, both because of statistical uncertainty and the possibility of
participant and controls firms being different in unobserved factors potentially being
important with regards to the observed differences. But when one relates the approx.
DKK20 million difference to the programme’s research subsidies – corresponding
to approx. DKK3 million (approx. €370,000) per participant firm – it becomes clear
that the programme is a success even in case of only a share of the observed gross
profit differences owing itself to a genuine causal effect of the programme.
This result is robust to changing sampling conditions and using firms that applied for
funding and got their application rejected as an alternative control group. Results for
employment growth are not robust to using the alternative control group, and should
thus be interpreted as being more tentative.
For the largest participant firms, any effects of the programme are small relative to these firms’ large variations
in the success parameters, and inclusion of large firms in the sample renders impossible finding any potential
positive programme effects.
1
184
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0185.png
SAMMENFATNING (DANISH SUMMARY)
Denne rapport er skrevet af Centre for Economic and Business Research (CEBR).
Den beskriver en analyse af Innovationskonsortie-ordningens potentielle effekter på
udviklingen i de deltagende virksomheder.
Innovationskonsortie-ordningen er et virkemiddel under Rådet for Teknologi og
Innovation (RTI). Rådet administrerer virkemidlet i samarbejde med Forsknings- og
Innovationsstyrelsen (The Danish Agency for Science, Technology and Innovation,
FI). Gennem Innovationskonsortier støtter RTI samarbejde mellem virksomheder og
vidensinstitutionerne (f.eks. universiteter, GTS-institutter m.fl.).
Ved hjælp af registerdata følger analysen 220 virksomheder som har deltaget i ord-
ningen. Vi studerer væksten i to succesmål: bruttofortjeneste og beskæftigelse.
Mens beskæftigelse er antallet af medarbejdere på et givet tidspunkt, er bruttofortje-
neste et mål for virksomhedens værdiskabelse.
I dette studie betragter vi væksten i bruttofortjeneste og beskæftigelse både før og
efter starten af programdeltagelsen. Yderligere identificerer vi en gruppe af kontrol-
virksomheder som ikke deltager, men som ellers ligner de deltagende virksomheder
i størrelse, branche, alder og region. Også for dem udregner vi vækst i bruttofortje-
neste og beskæftigelse. Det betyder, at vi kan besvare spørgsmålet hvorvidt de del-
tagende virksomheder har haft højere vækst end man ville have forventet - ikke kun
på basis af deres vækst før programdeltagelsen, men også på basis af udviklingen for
kontrolvirksomhederne.
Ud fra antagelsen om at udviklingen i bruttofortjeneste og beskæftigelse ville være
symmetrisk i fraværet af ordningen, kan differencen mellem de to gruppers udvik-
ling fortolkes som ordningens direkte effekt for de deltagende virksomheder.
Analysens resultater kan sammenfattes som følger: Mindre virksomhederne, som
har deltaget i Innovationskonsortie-ordningen, har oplevet større vækst i bruttofortje-
nesten og i antallet af medarbejdere end virksomhederne i kontrolgruppen, der ikke
har deltaget. Disse resultater er robuste overfor at der korrigeres for væksten inden
programdeltagelsen og korrigeres for udviklingen i væksten i kontrolgruppen.
Der skal dog lægges mærke til, at resultaterne afhænger af størrelsen af de virksom-
heder, som betragtes.
For eksempel er den potentielle effekt på bruttofortjenesten signifikant på et 5 %
niveau for deltagervirksomheder, der havde under 150 millioner Kr. i bruttofortjene-
ste i året før programdeltagelsen. Differencen på bruttofortjenesten kan her estimeres
til ca. 2 millioner kr. ekstra vækst i deltagervirksomhedernes årlige bruttofortjeneste
om året. Dette betyder, at deltagervirksomhedernes årlige bruttofortjeneste er blevet
forøget med ca. 20 millioner kr. over en ti års tidshorisont. Sådan en sammenlig-
ning skal fortolkes med en vis forsigtighed grundet statistisk usikkerhed, og det at
185
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0186.png
forskellen sandsynligvis delvis skyldes faktorer, som analysen ikke kan tage højde
for. Med programomkostninger svarende til ca. 3 millioner kr. pr virksomhed kan
det dog konkluderes, at ordningen er en succes selv i tilfældet at kun en mindre del
af differencen skyldes en kausal effekt.
Vi finder yderligere signifikant positive potentielle beskæftigelseseffekter for virk-
somheder, der havde mindre end 150 medarbejdere året før programdeltagelsen.
Disse potentielle effekter svarer til ca. 50 ekstra ansatte over en fem til ti-års periode
efter starten af programdeltagelsen.
Resultatet vedr. bruttofortjeneste er robust overfor ændringer i dataopsætning og
overfor at man bruger virksomheder, hvis ansøgning om støtte til finansiering af del-
tagelsen i et Innovationskonsortium ikke blev imødekommet, som alternativ kontrol-
gruppe. Resultatet vedr. beskæftigelsesvæksten viser sig derimod ikke at være robust
overfor at bruge denne alternative kontrolgruppe, og må dermed fortolkes med større
forsigtighed.
186
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0187.png
1.
INTRODUCTION
This report has been prepared by the Centre for Economic and Business
Research (CEBR). It presents an analysis of the economic impact of
’Innovationskonsortieordningen’ (Innovation Consortium scheme, IC scheme) on
participating firms in terms of growth and value creation.
The report is a follow-up to an earlier CEBR analysis (FI, 2007 and FI, 2008) and
exploits the availability of more recent data, which allow following the participating
firms for another 3 years.
Although this analysis is an evaluation of a specific subsidy scheme, its results might
be of general interest, as schemes similar to the IC scheme have been implemented
in a number of countries. However, general knowledge of their effects which can be
integrated into cost-benefit analyses of these schemes is still rare.
2
The IC scheme is a Danish subsidy scheme granted by Rådet for Teknologi og
Innovation (The Danish Council for Technology and Innovation, RTI) in cooperation
with Forsknings- og Innovationsstyrelsen (Danish Agency for Science, Technology
and Innovation, FI).
ICs subsidise and facilitate cooperation between private firms and research and
knowledge institutions (see next section 2 of the report for a more detailed descripti-
on of the scheme). Cooperating institutions can apply for financial grants at the RTI/
FI, and the grants subsequently finance the expenses incurred by the research and
knowledge institutions whilst undertaking the cooperative project. Typically grants
amount to DKK7-15 mio (approx. €1-2 million).
The IC programme has existed since 1995 (until 2003 under the heading “Centre
Contracts”). Until 2003, 80 ICs covering 274 different firms (denoted participants in
the following) had been completed, representing total grant costs of DKK766 milli-
on (approx. €100million), which corresponds to DKK2.8million (approx. €370,000)
per firm.
This analysis follows 220 of these firms in a firm-register dataset that covers the
period up to (and including) the year 2008. We study firm level developments in two
success parameters: gross profit and employment.
3
See Schibany et al. (2004) for a study based on a similar Austrian subsidy scheme. Branstetter og Sakakibara
(2002) consider a similar Japanese scheme and Adams et al. (2003) analyse the effects of the cooperation between
private and public R&D for firms in the U.S.
2
We also take a look at firm closure as an additional success parameter. However, given that this is not central to
the analysis, the results of this exercise are reported in Appendix 3.
3
187
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0188.png
While employment is simply defined as the number of employees in a given firm at a
given point in time, gross profit is defined as annual net sales subtracted annual costs
of variable inputs (raw materials, energy, intermediate goods purchases, etc.) except
labour costs. Gross profit is the most precise measure of the firm’s value creation,
but one should, of course, keep in mind that part of the firm’s total value creation
may be passed on to consumers, may be retained in the firm and increase its value
(of which there is no data available for this analysis), or may take the form of posi-
tive externalities, such as knowledge and/or innovations, that benefits other firms or
society as such.
4
In this study, we consider (absolute and percentage wise) growth in gross profit and
the number of employees both before and after programme participation. In addition,
we also analyse the changes in the growth patterns in association with participating
in the programme.
Moreover, we identify a control group of firms that do not participate (non-par-
ticipants), but which are similar to the participants in terms of size, industry, and
region. Again, we can use firm-level data to calculate the changes in gross profit and
employment for the non-participants, allowing us to address the question of whether
participants have higher increases in growth than what would be expected on basis
of the growth patterns of non-participants.
Under the assumption that growth in gross profit and employment of participants
and non-participants would be equal in the absence of programme participation, dif-
ferences between the two groups of firms can be interpreted as the causal impact of
the programme on participating firms.
The results of this exercise can be summarized as follows: Of the firms that parti-
cipated in the IC scheme it appears that relatively small firms have experienced a
significant increase in (the growth of) gross profit and employment.
It is important to note, that the size and statistical significance of these potential
effects depend on the size of the firms under consideration. We, for example, find po-
sitive potential gross profit effects that are significant at the five percent significance
level for firms with a gross profit below 150 million DKK (approx. €20) the year
before the programme. We also find potential employment effects for firms with less
than 150 employees in the year before the programme.
Finally, we also look at the survival rates of participant firms and compare these
with the survival rates of firms in the control group. Here, we find high survival rates
(most likely due to IC participants and their control counterparts being relatively
large) and no difference in the survival rates of participants and non-participants.
As a measure of knowledge creation, we could in principle also have considered firm-level patenting activity. No
actual data on patenting activities were, however, available for this analysis.
4
188
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0189.png
2.
DESCRIPTION OF THE IC SCHEME
An innovation consortium is a flexible framework for collaboration between compa-
nies, research institutions and non-profit advisory/knowledge dissemination parties.
An innovation consortium must consist of at least two companies that participate
throughout the entire project, one research institution and one advisory and know-
ledge dissemination party. Additionally, an innovation consortium may involve or
attach other types of partners that are considered relevant to the project.
The consortiums’ collaboration should be based on a joint project aimed at develo-
ping and bringing research based knowledge to maturity, so that it can form the
foundation for Danish companies’ innovation in the years to come.
The joint project should result in the completion of high-quality research relevant to
Danish companies. Furthermore, the project should ensure that the new knowledge
is converted into competences and services specifically aimed at companies, and
that the acquired knowledge is subsequently spread widely to the Danish business
community – including in particular small and medium-sized companies.
Any project initiated by the consortiums must comply with the following:
• The project should have generic content and the results must be of relevance
to a wide group of companies.
• The project should be at a high level of innovation and research.
• The project should not have the character of product development for
individual companies.
• The project should require close collaboration between the consortium
parties.
• The project should have a duration of two to four years.
The role of companies in the consortiums is to ensure that the joint research and
development project is based on relevant development needs within Danish compa-
nies. Consequently, the project theme should be of significance to the participating
companies’ business development. However, it should not take the form of actual
product development.
The company participation is also to ensure that the business community’s knowled-
ge and competences are utilised in the project. Therefore, the participating compa-
nies should contribute knowledge and competences at a high level within the project
field.
The companies may be Danish or foreign (or both).
Over the period 1995-2003, 274 different firms have participated in an IC, but a
number of firms have participated more than once. On average there were approx. 40
firms starting to participate in an IC per year, but there are large differences across
years, with the years 1998-2000 being characterised by the highest activity with on
average almost 70 firms starting to participate.
Approx. 50 percent of all participating firms are in manufacturing, 25 percent are in
financial or business services and 15 percent are in trade in services.
189
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0190.png
3.
DATA
The data for this analysis comes from three sources:
1.
Data on program participants, which were assembled by CEBR based on FI’s
(paper) file records of the IC-programme for an earlier analysis (Forsknings-
og Innovationsstyrelsen, 2007 and 2008). These data will in the following be
called ’IC data’.
2. Data from the private information provider company Købmandsstandens
Oplysningsbureau, now Experian A/S. This dataset, henceforth denoted as the
KOB data, has information from the financial reports that firms of a certain
size and ownership structure are obliged to file at the public authorities. Thus,
there are typically a number of observations for a given firm (one for each an-
nual account), denoted firm-year observations in the following.
3.
Information on firm transitions (e.g., mergers, liquidations or bankruptcies)
are included from the ‘cvr-register’ of the Danish Commerce and Companies
Agency (Erhvervs- og Selskabsstyrelsen). These data will be put to use when
we analyse survival probabilities of participating firms.
Note the KOB data provide information on a host of accounting-related variables,
including employment and gross profit. Note that only large firms are obliged to file
information about sales. This would make sales growth a potentially skewed indica-
tor of the IC impact upon firms.
The KOB data include firm-level information about industry and geographical loca-
tion, which will be exploited later when we identify a control group for the empirical
analysis.
190
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0191.png
4.
SAMPLING
There are a total of 405 firm observations in the IC data over the period 1995-2003.
These belong to firms that participated in one of the programmes which go under the
umbrella ’Innovationskonsortier’. For 19 observations, it was not possible to identify
the point in time when the project was started, for another 35 observations it was not
possible to find firm-identification numbers which were necessary to match the IC
data with the KOB data (leaving us with 351 observations).
A number of firms are registered more than once in the data, because they have
participated more than once in the programme. We treat participation as a zero/one
variable, independently of how many times a firm has participated, and consider the
earliest time a firm is registered as participating as the starting point of programme
participation. This leaves us with 274 firm observations.
For 20 of these firms, there is no information in the KOB database, which leaves us
with 254 observations, and for 34, there is no accounting information in the KOB
data before the start of the program. This information, however, is necessary for the
before-after estimation set-up employed in the following. So we are left with 220
participant firms for the analysis.
As in any firm accounting database, considerable variation can be observed in the
KOB-data, which owes itself to some firms being part of corporate groups, organi-
zational changes and/or because firms change accounting policies and practices. We
treat this issue differently depending on the stage of the analysis, which is, basically,
divided in two steps:
As a first step, we identify a control group of comparison firms. In this step, we
will exploit the total universe of firms available in the data, independent of missing
observations or zero reporting.
As a second step, we compare the performance of participant firms with the perfor-
mance of firms in the control group. In this step, there is a need to make decisions of
how to treat the data in case of missing values in the data or when firms report zero
activity. This will also direct our robustness checks of the results of the analyses. In
this context, we will commit to one sampling strategy, and subsequently check the
robustness of the results when changing the strategy.
In essence, we want to analyse samples that are as ‘clean’ as possible, i.e., concen-
trate on firms which report regularly, and which do not raise suspicions of significant
organizational or accounting issues. By implication:
(a) When analysing gross profit we consider the 61 percent of firms that do not
report zero gross profit in the KOB database. The argument being that, if
there is any economic activity, zero gross profit is an event having (almost)
zero probability, indicating non-reporting rather than gross profit being zero.
191
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0192.png
(b) When analyzing employment growth, we consider the 62 percent of all firms
that report a strictly positive number of employees in the KOB data.
Our sampling scheme implies that we start the analysis with the cleanest data pos-
sible. Obviously, robustness checks will address whether these decisions are cri-
tical for results. By implication, the sensitivity of results with regard to the rather
restrictive sampling scheme will be addressed subsequent to the presentation of the
performance comparisons.
192
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0193.png
5.
IDENTIFICATION OF THE CONTROL GROUP
To identify the control group of this analysis, we use a ’matching-on-observables’
technique, which can be seen as the workhorse of programme evaluation (see, for
example, Woolridge, 2002).
According to this method a control is identified for each firm participating in the
scheme. Except for not having participated in the programme, the controls are
selected to be as similar as possible to the given participant firm before programme
participation. In the following, these comparison firms will be called ’control firms’
or just ’controls’.
The details of the identification process are described in Appendix 1 of this report.
At this place it may be sufficient to note that, in the latter analysis, we will compare
developments in gross profit and employment over time of two highly similar groups
of firms, one which consists of the programme participants, the other of the controls
(non-participants).
Note also that the selection of highly similar controls increases the realism of the
assumption that participants and controls would have had similar developments in
gross profit and employment in the absence of the programme. By this, differences
in the developments can be interpreted as the programme’s genuine causal effect.
The selection of highly similar controls is an improvement of CEBR’s earlier analy-
sis (Forsknings- og Innovationsstyrelsen, 2007 and 2008), which simply uses private
sector firms for comparison purposes.
For participants, we will also compare the growth in employment and gross profit
in the time period before participating in the IC scheme with the growth in employ-
ment and gross profit in the years after having participated. The cut-off year which
separates the pre-participation period from the after-participation period is the year
just prior to participation. This year will in the following be denoted the ‘base year’.
For controls, we also define a base year, which now refers to the year the given
control was selected. This is the year in which it most closely resembled one of the
participants in its base year. So we can also compare controls’ growth in gross profit
and employment between before and after the base year.
In the analysis, we will consider the growth of any of the two success parameters
193
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0194.png
6.
ESTIMATION SET-UP
(gross profit and employment) before and after the base year, where growth will be
measured both as absolute and percentage wise annual increases. We will analyse
changes in growth between before and after the base year, and will compare these
changes between participants and controls.
E.g., when growth accelerates after the base year for participants, but not for con-
trols, this indicates positive programme effects. The acceleration is interpreted as the
programme’s causal effect for participating firms under the (‘identifying’) assump-
tion that participants’ growth would accelerate by just as much as the controls in the
absence of the programme.
Note this set-up further improves upon the method employed in CEBR’s earlier
evaluation (Forsknings- og Innovationsstyrelsen, 2007 and 2008). Here, the evalua-
tion was based on a comparison of the levels of participant’s and control’s success
parameters and, thus, addressed the question whether participants had grown faster
than non-participants.
This methodology could not take into account the possibility that participant firms
might generally have higher growth independent of whether they decide to partici-
pate in the programme or not. Any inherent growth difference between participants
and controls, however, should show in the years before the base year and, thus, can
be controlled for in the present analysis.
This is achieved by no longer comparing pre-participation levels of success parame-
ters with post-participation levels. Instead, we compare pre-participation growth (or
increases) with post-participation growth (or increases). So the evaluation is based on
participant-control differences in the acceleration of growth, rather than just growth
differences. This allows taking account of innate growth differences that can be
measured before programme participation (for participants) or before the base year.
In analyzing growth developments, we will in the following consider both abso-
lute and (approximately) percentage wise changes in the growth in gross profit and
employment – the latter being measured by increases in the logarithms of these two
success parameters.
There are good reasons for analyzing both absolute and relative changes in firm level
growth. Considering absolute increases allows us to make statements in absolute
terms, (e.g., ‘ICs increase participants’ gross profit by on average XYZ DKK’) which
can be integrated into cost-benefit analyses, whilst inclusion of relative (percentage-
wise) growth gives greater weight to smaller firms in case of absolute programme
effects being larger for larger firms. If, for example, the programme is assumed to
have a proportional effect on growth rather than increasing gross profit by the same
amount for all participants independent of their size, then the analysis of relative
growth will allow us to estimate these proportional effects.
However, it should be noted at this point that percentage wise growth can only be
measured for those firms that have positive gross profit (or nonzero employees) in
194
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0195.png
the base year and the years to follow. By implication, analyzing growth instead of
increases necessarily restricts the sample to these firms.
The estimation set-up of potential programme effects is explained in greater detail
in Appendix 2 of this report. At this point, it should, however, be noted that we only
look at firms, which we can follow for at least three years before the base year and
five (in a second iteration: ten) years after the base year.
When considering absolute increases instead of percentage wise growth, we calcu-
late the average of each firm’s annual increases in the success parameter in the three
year period before the base year. This defines a firm’s average annual increase in the
pre-base-year period. Also, we calculate for each firm the average annual increase in
the success parameter in the five (ten) year period after the base year, which defines
the firm’s average increase in the after-base year period.
This implies that we have two observations for each firm: one describing average
increases before the base year, and one describing average increases after the base
year. As a result, we can calculate for each firm, whether average annual increases
have become larger or smaller in association with passing the base year. In other
words, we can evaluate the development of average annual increases.
So this study’s performance analysis takes a look at participants’ average increases
in the average annual increases in association with participating in an IC, and com-
pares them with the average increases of the annual increases for controls following
their assigned base year. If the increase in the average annual increases is larger for
participants than controls, this implies that there is a more positive change in growth
developments of participants than controls. Thus, the comparison estimates the po-
tential effect of the IC programme on the participating firms.
In this case, any differences in the increase of annual growth can not be interpreted
as the result of different pre-base year developments, nor can it be explained by
reference to differences in the two group’s characteristics given the similarity of
participants and controls (and given that we additionally include some control va-
riables in the models to take account of potentially remaining differences). In short,
this approach makes it more likely that positive differences between participants and
controls must be attributed their participation in IC schemes.
195
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0196.png
7.
DESCRIPTIVE STATISTICS
The above-mentioned identification procedure yields 439 control observations
belonging to 334 different firms, implying that repeated observations occur for a
number of control firms.
To interpret the results of the following analysis as measuring the impact of the
programme, one needs to assume that participants and controls would experience
the same changes in growth if not it was for the programme. It can be argued that
this assumption becomes increasingly realistic the more similar the participants and
controls are in terms of their observable characteristics. Hence, we have sought to
identify a highly comparable group of controls.
TABLE 1 illustrates how successful we were at identifying a group of controls that
is highly similar to the group of participants by describing both groups of firms in
the base year.
We find that the distribution across industries is highly similar for the two groups,
but that there are differences w.r.t. the mean size of the participants vs. controls.
Also, participants are slightly more concentrated in the Copenhagen area (zip codes
below 2999). The size difference between participants vs. controls owes itself to the
fact that some participating firms belong to the biggest firms in Denmark, for which
it is not possible to find controls of similar size.
Although we of course will test differences in the success parameters between all
participants and controls, any effects of the programme might in this case be unde-
tectable as they may be washed out by the large variations in the success parameters
in large firms for reasons outside the statistical models.
As a consequence, we will analyse different samples distinguished by the maximum
size of the firms under consideration. As a starting point, we consider small and
medium size firms separately. More specifically, employment growth will be analy-
sed separately for firms below 300 employees in the base year. Growth profit will be
analysed separately for firms with gross profit less than 150 million DKK (approx.
€20 million) in the base year. Although it may appear restrictive, these thresholds
imply that the resulting samples still represent approx. 75 per cent of all observati-
ons, reflecting the large share of SMEs in Denmark.
For these subgroups of firms, expected unobserved heterogeneity is smaller and,
thus, the power of the analysis’ statistical tests (i.e., the probability of finding effects
in case there are any) is larger compared to the sample where large firms are inclu-
ded. Also, participants and controls are more similar in their observable characteri-
stics, which increases the realism of the ‘identifying’ assumption that, in the absence
of the programme, growth developments would be similar for participants and
controls.
Please note that the chosen size thresholds are completely arbitrary, and constitute a
compromise between being representative for the entire population on the one hand
and the desired robustness of findings and the realism of the identifying assumption
on the other. Note, moreover, that the thresholds can be moved easily, and we will do
so to test how this affects analyses and results.
196
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0197.png
>
TABLE 1. Mean values of key variables for participants and controls in the base
year
All firms
Participants
Number of employees
Gross profit (1000DKK)
Industries (shares of total)
Agriculture
Construction
Electricity
Finance, business service
Manufacturing
Trade, hotels, restaurants
Transport, telecom
Services
Not stated
Region (zip codes)
1000-2999
3000-3999
4000-4999
5000-5999
6000-6999
7000-7999
8000-8999
9000-9999
0,455
0,068
0,064
0,041
0,077
0,077
0,159
0,059
0,380
0,068
0,064
0,046
0,109
0,093
0,169
0,071
0,005
0,036
0,009
0,255
0,500
0,155
0,014
0,014
0,014
0,005
0,039
0,007
0,260
0,499
0,155
0,009
0,014
0,014
612
364.271
Controls
279
145.825
Firms with less than
300 employees in the
base year
Participants
83
51.439
Controls
87
42.554
0,000
0,030
0,007
0,289
0,452
0,178
0,015
0,015
0,015
0,003
0,027
0,009
0,247
0,509
0,165
0,012
0,015
0,012
0,444
0,089
0,059
0,037
0,074
0,096
0,141
0,059
0,363
0,061
0,064
0,046
0,091
0,110
0,189
0,076
197
197
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0198.png
For the sub-sample of firms with less than 300 employees in the base year, we find
the difference in the number of employees in the base year between participants and
controls to be within the ‘natural’ statistical variation, i.e., not significantly different
from each other at any commonly used significance level. However, participants
remain overrepresented in the Copenhagen area (a finding which is significant at the
10% significance level), and have higher gross profit in the base year (significant at
the 10% level).
For the sub-sample of firms with gross profit less than 150 million DKK in the
base year, we find gross profit (and the number of employees) in the base year to
be not significantly different between participants and controls at any commonly
used significance level. However, participants remain again overrepresented in the
Copenhagen area (significant at the 10% significance level).
In total, there are 10,167 firm-year observations belonging to 554 different firms.
Note here that the same control firm may occur repeatedly in the data, if more
than one of its firm-year observations were selected by the procedure outlined in
Appendix 1.
It should be noted here that there are only relatively few observations that enable us
to follow firms long before and long after the base year: only firms that participated
early in the programme or the controls associated with these firms can be followed
over a long time period after having participated or selected as controls.
This is, for example, reflected in the fact that there are only 15 observations with
employee information available ten years before base year. There are, however, 106
observations where data is available eight years before the base year, and 275 obser-
vations where data is available five years prior to the base year. Five years after base
year we have information on 340 firms, whilst 178 firms remain in the database ten
years after base year.
Only part of this attrition is due to firms leaving the data before the end of the obser-
vation period: Of the 554 firms in the final sample, approx. 25 per cent leave the data
before 2008.
198
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0199.png
8.
RESULTS
The following considers developments of employment and gross profit over time, and
compares these developments between participants and controls. This performance
analysis is split up into two parts:
1.
The first part of the performance analysis is based on two subsamples
excluding large firms: one, in which gross profit is below DKK 150
million in the base year, and another one, in which the number of employees
is below 300 employees in the base year.
2. The second part of the performance analysis is based on a set of alternative
samples and extends (and checks the robustness of) the previous results.
Results are reported in section 9 of this report.
Choosing sub-samples of relatively small firms as the point of departure for the per-
formance analysis, instead of the total sample, is motivated by the following reasons:
First, we have difficulties finding highly similar controls for large participants such
as, for example, multinationals in specific industries of which there are only a few in
Denmark. As result, the assumption necessary to identify causal effects of the pro-
gramme, which is that firms in both groups would change their growth patterns in
the same way in the base year if not it was for the presence of the programme - can
be argued to be more realistic for a sub-sample of small and medium size firms than
in a sample that include the few, very large companies.
Second, we find that results for this subgroup are well-suited to illustrate the esti-
mation technique employed to answer the question of whether findings should be
interpreted as being the result of underlying processes (in which case they are ‘stati-
stically significant’) or just ‘coincidental’.
Still, as mentioned already, the chosen cut-offs are, of course, arbitrary. Hence, the
robustness of findings with respect to changing the thresholds will be discussed in
section 9.
A last point to mention here is that we will depart from only analyzing firms that
always report nonzero and non-missing information. Again, we will subsequently
check whether these strict sampling conditions are critical for the results.
8.1
Gross profit developments
After these introductory remarks, we are now ready to take a look at the numbers.
A graphical depiction of the absolute differences in gross profit is displayed in
FIGURE 1:
199
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0200.png
FIGURE 1: Gross profit (in DKK1,000). Mean differences compared to base year.
Firms with gross profit less than DKK150 million in the base year. 3-year moving
averages
80000
60000
40000
20000
0
-20000
-40000
-10 -8 -6 -4 -2 0 2
4
6 8 10 12 14
Controls
Participants
Years after the base year
We find similar increases in gross profit for participants and controls in the years
before the base year. This suggests absence of any inherent differences in gross
profit growth between the two groups of firms, which also indicates that the match-
ing procedure succeeded in finding a group of controls of similar inherent growth
compared to the group of participants.
After the base year, the gaps between the graphs widen, with participants having
larger increases in gross profit compared to the controls. Under the assumption that
participants and controls would have continued their pre-participation (pre-base-
year) growth patterns in the absence of the programme or would have changed their
growth patterns in the same fashion, the higher increase in the group of participants
measure positive effects of the programme on participants’ employment and gross
profit.
We will have a closer look at the size of the differences between participants’ and
controls’ growth patterns in a more formal treatment below. For now, we may note
that, if pre-base-year trends are indeed equal, the graphs suggest participation in an
ICs to have a gross profit effect of approx. DKK13,4 million five years and approx.
DKK15,4 million ten years after the base year.
Obviously, a next step is to establish evidence on whether or not the finding of di-
verging growth trends is statistically significant, i.e., the result of underlying mecha-
nisms, or just incidental and within the statistical variation which must be expected
for firm data typically being characterized by large variations.
200
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0201.png
However, before addressing this issue, some minor remarks regarding FIGURE 1
(and those figures to follow) might be in place: Note that firms, to be observable long
after the base year, need to have participated or have to be selected as controls at the
time of the start of the programme in the mid-nineties, and must not have left the
data before the end of the observation period. Also, to be observable long before the
base year, firms need to have started to participate or been selected late in the obser-
vation period, and need to have existed long before the base year.
As a result, there exist only a limited number of observations long before and after
the base year, implying that findings based on these observations get increasingly
tentative at the left and the right sides of the figures.
5
Also, when determining (li-
near) growth trends before and after the base year, observations long before and after
the base year are given a higher weight, so firms with high or low growth have a
higher leverage on trend estimates when being observable for extended time periods.
Note also that observations long after the base year belong to the same cohort or
nearby cohorts, and findings for these observations may be due to business cycle
effects - which does not matter for the results of the analysis unless business cycles
affect participants and controls in different ways.
To establish evidence on whether or not the above differences in the two groups’
growth patterns are statistically significant, i.e., too large compared to the general
variation in the data to be considered coincidental, we employ the regression model
as described in section 6 and Appendix 2. Results for the changes in gross profit
developments in association with programme participation relative to the changes in
the group of controls are summarized in TABLE 2A and TABLE 2B:
Of course, one could right-censor the graphs at, say, ten years after the base year to avoid that large variation at
the end of the observation period steals the picture. This would, however, be highly arbitrary and even manipula-
ting, leading us to present results for the entire observation period independently of the number of observations
long before and long after the base year.
5
201
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0202.png
TABLE 2A: Growth in gross profit up to five years after the base year: Diff-in-
diff regression results
Model 1: Dependent
variable: Average annual
increase in gross profit in
either the three-period up
to the base year or in the
five-year period after the
base year
Coefficient
Constant term k
Observation is after base
year, d1
-1914,5
-2425,2***
Standard
error
2.496,4
681,9
Model 2: Dependent
variable: Average annual
growth in gross profit in
either the three-period up
to the base year or in the
five-year period after the
base year
Coefficient
0,100
-0,086**
Standard
error
0,206
0,035
Observation belongs to a
participant, d2
Observation belongs to a
participant and is after the
base year, d1d2
-846,8
1.090,9
-0,066
0,055
3668,9**
1.738,3
0,145**
0,067
R2=0.13
517 observations
R2=0.30
510 observations
Notes: *** significant at 1%. ** significant at 5%, * significant at 10%; only firm observations with positive
gross profit are used in Model 2; gross profit is measured in DKK1000. The following set of controls
was included in the regressions: Seven industry dummy variables, eight dummy variables for the firms’
geographical regions, three calendar time dummy variables for when the firm has its base-year, and six
dummy variables describing the firm’s gross profit in the base year. Base category: firms in manufactur-
ing industries, with gross profit 0-500 million DKK in the base year and zip-code
<3000.
202
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0203.png
TABLE 2B: Growth in gross profit up to ten years after the base year:
Diff-in-diff regression results
Model 1: Dependent
variable: Average annual
increase in gross profit in
either the three-period up
to the base year or in the
ten-year period after the
base year
Coefficient
Constant term k
Observation is after base
year, d1
138,5
-696,5
Standard
error
1.889,7
827,5
Model 2: Dependent
variable: Average annual
growth in gross profit in
either the three-period up
to the base year or in the
ten-year period after the
base year
Coefficient
0,399***
-0,086***
Standard
error
0,093
0,018
Observation belongs to a
participant, d2
Observation belongs to a
participant and is after the
base year, d1d2
-940,9
1.030,4
-0,030
0,038
1.981,7
1.916,7
0,121 **
0,058
R2=0.17
399 observations
R2=0.38
390 observations
Notes: *** significant at 1%. ** significant at 5%, * significant at 10%; only firm observations with positive
gross profit are used in Model 2; gross profit is measured in DKK1000. The following set of controls
was included in the regressions: Seven industry dummy variables, eight dummy variables for the firms’
geographical regions, three calendar time dummy variables for when the firm has its base-year, and six
dummy variables describing the firm’s gross profit in the base year. Base category: firms in manufactur-
ing industries, with gross profit 0-500 million DKK in the base year and zip-code
<3000.
The coefficient estimates presented in the tables have the following interpretations:
-
the constant term k estimates the average annual increases for a specific
subgroup of controls (in this case controls in manufacturing, with gross profit
between zero and 500 million DKK and with zip code less than 3000) before
the base year,
the coefficient associated with d1 estimates the difference in the average
annual increases (or the increase in annual growth) for all controls between
before and after the base year,
-
203
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0204.png
-
the coefficient associated with d2 estimates the difference between the ave-
rage annual increases (growth) between participants and controls before the
base year.
The coefficient associated with d1d2 estimates the difference in the increases
of the average annual increases (growth) between participants and controls.
-
To illustrate, consider the case where we follow firms over ten years (TABLE 2B).
After the base year, the average annual increase in the gross profit of controls is
DKK 696,500 (approx. €95,000) lower than before the base year. Participants’
average annual increase in the years before the base year is DKK 940,100 (approx.
€130,000) lower than the controls’. Finally, the difference in the increases of the
annual average increases between participants and controls is found to be 1,981,700
DKK (approx. €260,000). As a result, the average annual increase of the gross profit
of participants in association with participating in the IC programme exceeds the
controls’ increases by almost two million DKK in the ten-year period after the base
year.
Turning to TABLE 2A, we find that the average annual gross profit increase for
participants over the first five years after the base year is approx. 3.6 million DKK
higher (and statistically significant at the 5% level) compared to what would be ex-
pected in absence of participation in the IC scheme.
Looking at relative change (average annual logarithmic differences translating inter-
preted as average annual percentage wise growth), we find that the average annual
growth in gross profit for participants over the first five years after the base year is
approx. 15 per cent higher compared to what would be expected in absence of parti-
cipation in the IC scheme.
Note the percentage-wise growth difference gets smaller when one only considers
firms above a certain size in the base year. E.g. when only considering firms with
gross profit above 50 million DKK in the base year, the estimated average annual
growth difference goes down to approx. eight percent but remains to be statistically
significant at the 10% level. Hence, we can conclude that the positive differences in
the growth of gross profit is not (exclusively) driven by very small firms.
Over a ten-year period, the average annual increase in excess of what would be
expected in absence of the programme for participation is (as noted earlier) approx.
two million DKK, and growth is approx. 12 per cent higher than in the absence of
the programme.
In summary, our findings agree with the presence of considerable effects of the IC
programme on participants’ increases on gross profit. Findings for both absolute and
logarithmic differences are statistically significant at the 5% significance level for
firms followed over the first five years after the base year and significant at the five
per cent level for percentage-wise increases for those firms which are able to follow
for at least ten years after the base year.
204
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0205.png
The finding of estimated differences in average increases being 3.6 million DKK
when one follows firms over five, and approx. two million DKK when one fol-
lows them over ten years might indicate that differences in absolute increases are
largest in the years directly following the start participation (the base year).
8.2
Employment developments
The analysis of employment developments follows the blueprint of the pre-
vious subsection. We, however, depart from focusing on firms with less than
300 employees in the base year, which always report nonzero and non-missing
employment information, and leave the consideration of different samples to the
next section of this report.
Firms under 300 employees in the base year represent approx. 75 percent of the
total sample of firms, and approx. 71 percent of the present sample of firms which
never report missing or zero employment information.
For each firm, we consider average annual increases and annual average growth
rates for the three-year-period before the base year and the time period between
the base year and five years later. As a second step, we follow the firms for ten
years after the base year. Again, we only consider firms that always report non-
zero employees, which considerably reduce the number of observations, and leave
relaxing this strict sampling condition for later.
When taking a look at the average employment differences between a given year
after the base year and the base year in FIGURE 2, we do find IC participants to
have slightly higher growth in the first years after the base year. When following
firms for more than eight years, the picture changes: participants have conside-
rable lower growth eight to twelve years after the base year. But when following
(a greatly reduced number of) firms for more than 12 years, we find that those
controls which can be followed for so long have experienced considerably lower
growth than the corresponding participants. Again, the end of the curves should
be interpreted with caution.
205
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0206.png
FIGURE 2: Number of employees. Mean difference compared to base year. Firms
with less than 300 employees in base year. 3-year moving averages
80
60
40
20
0
-20
-40
-10 -8 -6 -4 -2
0
2
4
6
8 10 12 14
Participants
Controls
Years after the base year
FIGURE 2 does not suggest robust positive IC programme effects, although one
might notice that growth accelerates for participants but not controls in the years
close to the base year.
We estimate the same statistical model to substantiate the findings suggested by
FIGURE 2, and present the results of this exercise in TABLE 3A and 3B:
206
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0207.png
TABLE 3A: Employment growth up to five years after the base year: Diff-in-diff
regression results
Model 1: Dependent
variable: Average annual
employment increase in
either the three-period up
to the base year or in the
five-year period after the
base year
Coefficient
Constant term k
Observation is after base
year, d1
-1,33
-1,96
Standard
error
2,37
1,48
Model 2: Dependent
variable: Average annual
employment growth in
either the three-period up
to the base year or in the
five-year period after the
base year
Coefficient
0,100*
-0,080***
Standard
error
0,051
0,019
Observation belongs to a
participant, d2
Observation belongs to a
participant and is after the
base year, d1d2
-2,00
2,32
-0,019
0,032
5,12
3,70
0,061
0,039
R2=0.067
495 observations
R2=0.088
495 observations
Notes: *** significant at 1%. ** significant at 5%, * significant at 10%. The following set of controls was
included in the regressions: Seven industry dummy variables, eight dummy variables for the firms’
geographical regions, three calendar time dummy variables for when the firm has its base-year, and four
dummy variables describing employment in the base year. Base category: firms in manufacturing indus-
tries, with 5-10 employees in the base year and zip-code
<3000.
207
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0208.png
TABLE 3B: Employment growth up to ten years after the base year:
Diff-in-diff regression results
Model 1: Dependent
variable: Average annual
employment increase in
either the three-period up
to the base year or in the
ten-year period after the
base year
Coefficient
Constant term k
Observation is after base
year, d1
3,7
1,3
Standard
error
2,6
3,3
Model 2: Dependent
variable: Average annual
employment growth in
either the three-period up
to the base year or in the
ten-year period after the
base year
Coefficient
0,071**
-0,084***
Standard
error
0,036
0,022
Observation belongs to a
participant, d2
Observation belongs to a
participant and is after the
base year, d1d2
-2,4
2,3
-0,021
0,032
-1,6
4,1
0,023
0,039
R2=0.08
389 observations
R2=0.10
389 observations
Notes: *** significant at 1%. ** significant at 5%, * significant at 10%. The following set of controls was
included in the regressions: Seven industry dummy variables, eight dummy variables for the firms’
geographical regions, three calendar time dummy variables for when the firm has its base-year, and four
dummy variables describing employment in the base year. Base category: firms in manufacturing indus-
tries, with 5-10 employees in the base year and zip-code
<3000.
Results of the statistical analysis confirm the findings of FIGURE 2: in the group of
firms which can be followed over five years, participants increased employment by
five additional employees per year, and had 6 percent (not to be confused with per-
centage points) higher growth. These results have t-probabilities of 17% (for absolute
increases) and 13% (for percentual growth). This means that the probability of being
wrong when stating that participation in an IC’s generally increases employment
growth is 17% and 13%, respectively.
208
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0209.png
We suggest interpreting this result as follows: there are positive relationships bet-
ween growth and programme participation. However, the probability of these rela-
tionships being coincidental is too high to claim that there exist underlying mecha-
nisms implying that these positive relationships are a general feature of participation
in an IC.
Also, results are not robust to following firms for time periods of different lengths.
For those firms which can be followed over at least ten years, participants have on
average had lower increases in the number of employees than controls, which further
advise us to be careful with regards to statements regarding general employment ef-
fects of ICs.
These results for the ten-year period are again not significant. We conclude that – at
least for this sample of firms with up to 300 employees in the base year - it is not
possible to find relationships which are strong enough to claim that ICs generally
have positive employment growth effects.
209
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0210.png
9
ALTERNATIVE SAMPLES AND ROBUSTNESS
In this section, we report the results of the above-described model for alternative
samples distinguished by the size of the firms under consideration and their samp-
ling criteria. Also, we will run separate regressions for firms in service industries.
Finally, we use another group of firms as a control group than before, which consists
of firms that have applied for funding of an IC, but found their applications being
rejected by FI.
To take a look at firms in the service sector is motivated by FI’s special interest in
service industries as a potential growth industry. There are of course large inherent
differences between firms in the service sector, and it is tempting to differentiate
between knowledge intensive and less knowledge intensive industries. We, however,
came to the conclusion that we will not distinguish firms along these lines. First,
because we have too of few observations in the service sector, and second, because
it is difficult to argue that knowledge is not relevant for the service firms that partici-
pate in a collaboration which aims to enhance innovation activity.
Thus, when considering firms in services industries, we analyse on all firms coded
65-97 in the Danish standard industry classification (db93), which covers firms
which according to db93 are firms in “Financial and business services” and just
“Services”.
In this section, we also address the robustness of the results with regards to including
firms that report irregularly. Especially when considering employment developments
in the previous section, focusing on clean data implied that we lost a relatively large
share of firms which report zero employees in single years.
Firms may grow by hiring new employees, or by integrating organizational units
from other firms in the same corporate group, e.g., merging a holding company
(with no employees) with its operating company (with employees), by acquisitions
or organisational reshuffle within corporate groups. Focusing on clean data might
be assumed to reduce the impact of the latter explanations, but it is still relevant to
check whether this is critical with regards to the results of the analysis.
Finally, we exploit the data that CEBR has collected for the earlier study (FI, 2008)
on 133 firms that applied for funding before 2003, but did not receive it (and did
not receive funding later on). These firms, denoted ‘rejected firms’ in the following,
are equal to the participants with respect to the fact that they have applied, but the
fact that their project was declined funding indicates lower quality projects or lower
quality applications (which again may be correlated to firm characteristics that also
are related to the firm’s growth potential).
These problems notwithstanding, using this alternative control group for an additio-
nal robustness check makes sense, as the potential finding of rejected firms doing
just as well as participants would advise us not to interpret earlier finding as the
programme’s causal effect.
210
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0211.png
To implement the comparison of participants and rejected firms, we define the year
of the application as the base year for comparisons. For participants, this is typically
the year preceding the start of the project. The relatively low number of rejected
firms implies that for this robustness check, we only follow firms over a five-year
period.
In the exposition of the results of the various robustness checks, we only report the
relevant coefficient estimate which is associated with the indicator (dummy) variable
“Observation belongs to a participant and is after the base year, d1d2”. Recall that
this parameter estimates the deviation between actual post-participation average
annual increases for participants and the increases which would be expected in the
absence of participation. Under the identifying assumptions, this coefficient estima-
tes the effect of the programme on participating firms.
Results of the different regressions are summarised in TABLE 4. In this table, we
report t-statistics, which are the probabilities of being wrong when stating that there
are non-zero underlying relationships in the data. E.g., the probability of being
wrong with the claim “Firms that report gross profit less than 75 million DKK in
the base year and always report nonzero gross profit experience a different average
annual increase in gross profit (compared to controls) in the first five years after the
base year” has a 6% estimated probability of being wrong.
The following sums up the result of the different regressions:
(a) We find that no potential programme effects can be identified when conside-
ring the total sample of all firms. This comes as no surprise, as there are large
players among participants with gross profit (and large variations in gross
profit) being orders of magnitude too large to potentially allow us to find
any impact of the programme. The large variation in gross profit in the total
sample superimposes any potential (in this case relatively small scale) effects
of the programme.
(b) However, looking at smaller firms with gross profit below 75 million in the
base year corroborates the picture of significant higher gross profit growth
for participating firms after the programme and relative to pre-programme
growth, relative to the developments of firms in the matched control group,
and taking account of potential differences in observable factors between the
participant and the control firms.
(c) We cannot find relationships for service sector firms, which might be because
we have too few observations in this sector to allow identifying relationships
of any degree of reliability.
(d) We find the strictness of the sampling conditions with regards to whether
or not to sample firms that sometimes report zero activity or have missing
values not having any effect on the general results.
211
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0212.png
TABLE 4: Regression estimates of the parameter of “Observation belongs to a
participant and is after the base year, d1d2” for alternative samples. Dependent
variable: Average annual increase in gross profit (in DKK1000).
Firms that report regularly (i.e., always report nonzero and non-missing gross profit):
Sample
Observation
period (in
years)
5
10
Parameter
estimate
t-probability
Number of
observations
All firms
-2967
-3178
0,380
0,256
651
505
Firms that report gross
profit less than 75
million DKK in the base
year
Firms that report gross
profit less than 150
million DKK in the base
year in the service
sector
5
10
6035***
3491*
0,001
0,060
394
305
5
10
-576
-3302
0,820
0,383
86
57
Firms that occasionally report zero gross profit or have occasionally missing gross profit
information:
Firms that report gross
profit less than 150
million DKK in base
year
5
10
4888*
8331
0,050
0,173
696
552
Alternative control group: Rejected firms:
Firms that report
gross profit less than
150 million DKK in
base year, and always
nonzero and non-
missing gross profit
5
4710**
0,036
233
*** significant at 1%. ** significant at 5%, * significant at 10%; the estimations include the same controls
as specified in TABLE 2.
212
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0213.png
(e) Also, participants grow faster after the start of participation compared to the
alternative control group consisting of firms the applications for funding were
rejected. So the necessary condition for giving previous findings a causal
interpretation is fulfilled. The size of the potential effect of this comparison
is similar to the previous findings (an approx. DKK4.5 vs. DKK3.7 million
difference in average annual increases in gross profit).
The general conclusion is that there are stable differences in gross profit develop-
ments between participants and controls after the base year for up to medium size
firms – differences that cannot be easily explained by other factors than IC program-
me participation.
We turn now to employment developments, and summarise results in TABLE 5:
213
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0214.png
TABLE 5: Regression estimates of the parameter of “Observation belongs to a
participant and is after the base year, d1d2” for alternative samples. Dependent
variable: Average annual increase in the number of employees.
Firms that report regularly (i.e., always report nonzero or nonmissing number of employees):
Sample
Observation
period (in
years)
5
10
5
10
5
10
5
10
5
10
Parameter
estimate
t-proba-
bility
Number of
obser-
vations
703
560
325
274
206
145
95
66
87
58
All firms
-2,8
-11,0
11,2**
4,6
11,9**
9,2*
22,0
5,2
24,0*
11,2
0,794
0,288
0,016
0,179
0,028
0,084
0,104
0,635
0,392
0,392
Firms that have less than 150
employees in the base year
Firms that have less than 75
employees in the base year
Firms that that have less than 300
employees in the base year in the
service sector
Firms that that have less than 150
employees in the base year in the
service sector
Firms that occasionally report having zero employees or have occasionally missing
employment information:
Firms that have less than 300
employees in the base year
Firms that have less than 150
employees in the base year
Alternative control group: Rejected firms:
Firms that have less than 150
employees in the base year
and always report nonzero or
nonmissing number of employees
5
10
5
10
6,7**
2,0
9,2***
4,1*
0,023
0,540
0,007
0,076
693
529
554
356
5
1,4
0,873
165
*** significant at 1%. ** significant at 5%, * significant at 10%; the estimations include the same controls
as listed in TABLE 3.
214
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0215.png
In the case of employment growth, changing the sampling conditions reveals new
results: for participating firms of size below 150 employees in the base year, we find
large and statistically significant potential employment effects of ICs of about eleven
additional employees per year. One may note here that approx. 24 percent of all 220
participants have less than 50 employees and approx. 50 percent have less than 150
employees, so we find potential effects for the participants in the lower half of their
size distribution. We also find statistically significant effects for firms of size below
300 employees in the base year, when we include firms that report zero employees in
some years.
Again, we cannot find relationships when considering samples of all firms which
have participated in an IC – some of which have several thousand employees.
We find weakly significant positive potential employment effects for firms in the ser-
vice sector, which is remarkable given the relatively small size of this sample (less
than 100). We advise not to take the large potential effect of annually 24 additional
employees at face value. The combination of large heterogeneity and relative few
observations implies that this result is associated with a high level of uncertainty.
Finally, we do not find potential employment effects when comparing participants
with the group of firms the project applications were rejected. The absence of any
significant result might be due to large variation in employment growth in the group
or rejected firms - in association with a relatively small number of observations.
However, it also implies that high employment growth in small participant firms
after the base year might not be so much an effect of IC participation, but might
instead be the result of strategic decisions correlated to applying to the programme,
and shared by participants and controls.
TABLE 4 and 5 only present a small but representative share of the robustness tests
undertaken for this analysis, but none of our alternative sampling or modelling
6
strategies have changed the general conclusion of there being positive potential ef-
fects for the firms at the lower half of the total sample’s size distribution, which in
some cases even can be shown to be significant when considering long-run averages
over ten years after the base year.
This includes, for example, estimating the models with the inclusion of firm random effects – which is possible
because there are two observations per firm. This does, however, not change any of the previous results. Also,
random effects estimations of annual increases (instead of average annual increases over a couple of years) on
the panel of firm year observations give very similar results.
6
215
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0216.png
10
CONCLUSIONS
This report summarises the results of an evaluation study of the IC programme. For
this purpose, we follow firms which have participated in the programme before and
after the start of participation, and analyse their developments with regards to gross
profit and employment.
This is possible, because firm information from FI regarding programme participa-
tion has been merged with register data on key accounting variables that firms are
obliged to file at public authorities. We can find 220 firms that participated out of
285 in total in the register data, and can follow 203 of these firms for at least five
years after they have started to participate in an IC.
For our analysis, it is natural to distinguish firms by their size. Some of the firms that
participate in the IC programme are very large, having gross profits of several billion
DKK and several thousand of employees. It would be unrealistically optimistic to
search for potential effects of the IC scheme in a group of firms in which these large
firms are included.
Hence, for the analysis, we consider firms that represent roughly the smallest 75
percent of all firms in the sample, and find positive potential gross profit effects of
programme participation – a finding which is based on a joint comparison of growth
patterns of participant firms and a highly similar group of comparison firms, in
which we correct for potential differences in inherent (pre-participation) growth
trends before the start of programme participation.
We find that participants have annual increases in gross profit in the first five years
after the start of participation, which are on average 3.7 million DKK above what
would be expected in the absence of programme participation. Under the assump-
tion that participants would have experienced the same developments in gross profit
growth as the controls in the absence of the programme, the additional 3.7 million
per year in the first five years after participation is the genuine effect of participating
in an IC.
Over a ten year-period, the average potential effect gross profit effect is smaller and
is approx. two million DKK per year, and is no longer statistical significant. An
obvious explanation might be that potential effects of the programme are realised
in the first years after starting to participate in the programme, so the average of the
annual increases over a period of time becomes smaller the longer the time period
under consideration.
If participants’ counterfactual growth in the absence of participating in the pro-
gramme is indeed appropriately measured by the growth of the controls, then the
most qualified guess of the programme’s effect is that it increases annual gross profit
per year of smaller firms by approx. DKK20 million over a five to ten year time
period after participation. It should, however, be noted that this number is associated
with statistical uncertainty, which advises us to be careful when making predictions
regarding future programme effects.
216
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0217.png
It is difficult (and has not been part of the present analysis) to estimate what the
counterfactual behaviour of participants in the absence of the IC scheme might have
been. Maybe ICs are a means of helping to implement firms’ strategic decisions and
innovations, which are the true reasons of the positive developments, maybe partici-
pants have higher growth than controls for reasons we could not observe in the data
and did not control for in this analysis.
Still, the back-on-the-envelope calculation resulting in a DKK20 million difference
in annual gross profit after five to ten years suggests that the programme is a success
even in case of only a share of this difference owing itself to a genuine causal effect
of the programme. Here, it could be noted that differences in annual gross profit
accumulate over time, implying substantial differences between participants’ and
control firms’ value creation when measured over several years.
We also consider employment developments and can again not find significant
results for the sample of firms where we include large firms. It is, however, possible
to demonstrate that smaller (in this case firms having less than 150 employees in the
year before participating in the programme) participants have an additional annual
employment growth of approx. eleven employees. This difference is statistically
significant at the 5% significance level, but the sum of the evidence advises us to be
careful to interpret it as a causal effect of the IC scheme.
We conclude that it comes as no surprise that we do not find potential programme
effects for those samples which include large firms. Instead, we find positive poten-
tial effects of the programme on gross profit and employment for relatively small
firms, where we expected to have a chance of finding them in case of their existence.
The difficulty of finding potential effects for large firms is likely to be due to a mea-
surement issue, and should not been taken as evidence of ICs having no effect for
large firms.
Even though the present data at hand must be seen as favourable for this kind of
analysis, regularly updating them might in the future allow analyzing which firms
benefit more from participating in an IC than others, and which ICs work better than
others.
In the current case, it was for example not possible to make statements of any reliabi-
lity regarding the experiences of participant firms in the service sector.
Furthermore, we did not have data on the patenting activities of the participating
firms, but we expect this type of data to become available for subsequent analyses.
This additional information may be exploited for the identification of controls and
may also be used to directly estimate the effects of IC programme participation on
innovation output.
217
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0218.png
11
REFERENCES
Adams, James D., Eric P. Chiang and Jeffrey L. Jensen (2003): The Influence
of Federal Laboratory R&D on Industrial Research.
Review of Economics and
Statistics
85(4), pp. 1003-1020.
Bercovitz, J., M. Feldman, I. Feller and R. Burton (2001): Organizational Structure
as Determinants of Academic Patent and Licensing Behavior: An Exploratory Study
of Duke, John Hopkins and Penn State Universities.
Journal of Technology Transfer
26, pp. 21-35.
Blanes, J. Vicente and Isabel Busom (2004): Who Participates in R&D Subsidy
Programs? The Case of Spanish Manufacturing Firms.
Research Policy
33, pp.
1459-1476.
Branstetter, Lee G. og Mariko Sakakibara (2002): When Do Research Consortia
Work Well and Why: Evidence from Japanese Panel Data.
American Economic
Review
92(1), pp. 143-159.
Erhvervsfremmestyrelsen (1998):
Centerkontrakter. Evaluering udarbejdet af Oxford
Research A/S for Erhvervsfremmestyrelsen.
København.
Erhvervsfremmestyrelsen (2001):
Evaluering af Centerkontraktordningen.
København.
Forsknings- og Innovationsstyrelsen (2007):
Effektmåling af Innovationskonsortier -
Data og metode.
København.
Forsknings- og Innovationsstyrelsen (2008):
Effektmåling af forsknings- og innova-
tionssamarbejder – fokus på innovationskonsortier.
Inside Consulting og Oxford Research (2005):
Evaluering af centerkontrakt-/innova-
tionskonsortiumordningen.
Udarbejdet af Inside Consulting og Oxford Research for
Videnskabsministeriet.
Schibany, Andreas, Gerhard Streicher, Nikolaus Gretzmacher, Martin Falk, Rahel
Falk, Norbert Knoll, Gerhart Schwarz and Martin Wörter (2004):
Evaluation FFF.
Impact Analysis.
Background report 3.2. Joanneum Research.
Wooldridge, J. (2002), Econometric Analysis of Cross Section and Panel Data, MIT
Press.
218
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0219.png
APPENDIX 1: SELECTION OF CONTROLS
The KOB dataset is a panel dataset which has repeated observations for most of the
firms - one for each annual account filed at the authorities. So for each firm, there
are typically multiple firm-year observations (where a firm-year observation refers to
a data-point of a given firm in a given year). In the following, we will use the expres-
sion ‘control observation’ to describe a single firm-year observation of a control.
Control firms are chosen in the year in which they are most similar to one (in a
single case: two) of the participants in the year before participation. This defines
each control firm’s ‘base year’ as the year in which it is selected as a control firm.
For each control, the base year forms the basis for comparisons of given success
parameters over time.
Note similarity between participants and potential controls is in terms of (a) the
firms’ industry, region, size and age and (b) the expected probability of participation,
derived as follows:
We run an auxiliary regression on the universe of approx. 370,000 firm-year ob-
servations in KOB in the period 1994 to 2001 that roughly resemble the group of
participants (we do for example not consider industries in which there is no single
participant).
The auxiliary regression is formulated as a simple probit model, with starting to par-
ticipate in the programme next year being the dependent variable, and 32 controls in
total, covering firm size , industry, region and time period. The regressions’ pseudo
R2, which is a measure of the model’s goodness-of-fit, is 0.22, which we consider as
being high.
The probit regression allows making statements of how likely program participa-
tion is for a given firm. This allows finding pairs or groups of firms, in which this
probability is very similar. For two firms A and B with similar participation proba-
bility, the fact of firm A participating and not firm B can now be interpreted as being
coincidental.
Under this interpretation, the identification set-up resembles an experiment, in
which programme participation was at random, and which would allow interpreting
systematic differences in outcome variables between participants and controls as the
programme’s causal effect on participating firms.
Yet, even firms with similar predicted participation probabilities can be quite dif-
ferent, and to avoid systematic differences in industry affiliation, size, etc., between
participants and controls, we also condition on a number of observable characteri-
stics being equal for a given participant and its matched control firm(s).
For this purpose, we divide the total number of firm-year observations into groups
having the same industry affiliation and being in the same region, of similar size and
observed in the same year.
219
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0220.png
For each participant, we select the firm-year observation of a non-participant firm
being within the same group and having a participation probability which comes clo-
sest to the participant’s. This selected firm-year observation defines the participant
firm’s control firm, and the control firm’s base year.
By repeating this matching procedure, we can find an arbitrary number of control
observations for each participant. Here, a greater number of control observations
increases the robustness of later results, however, increasing this number also makes
it increasingly difficult to find highly similar control observations for some of the
participants.
As a compromise within this trade-off, we chose to find for each participant two
control observations (firm-year observations of non-participants). The selection of
the two control observations per participant is in two rounds. In each of the rounds
we select one control observation for each participant.
In the first round we find 220 control observations of non-participants, in the second
we find another 219 control observations of non-participants (the reason for only 219
instead of 220 being that in a single case one firm-year observation is chosen as a
control observation for two participants).
In each of the two rounds, we first condition on many factors being highly similar
when selecting control observations. This leaves a number of participants, for which
no control observations could be found. In subsequent steps, we reduce the num-
ber of factors and start choosing control observations which are increasingly less
similar, until each round has identified one control observation for each participant.
This selection of control observations is described in greater detail in TABLES A1.1
and A1.2.
In each of the rounds we only select one control observation per participant. This
does not rule out selection of different control observations (belonging to different
years) of the same control firm, which implies that there are a number of control
observations which occur repeatedly in the data which form the basis of the perfor-
mance analysis.
220
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0221.png
TABLE A1.1: Identification of first neighbours by balanced score
matching procedure
Step 1:
Participants and controls are restricted to be equal in terms of ….
Industry (143) categories
Number of employees (11 categories)
Gross profit (7 categories)
Year in which they are observed (9 years)
Region (8 regions)
Firm age (3 categories)
This identifies control firms for 61 participant firms (27.7%).
Participants and controls are restricted to be equal in terms of ….
… industry (143) categories
… number of employees (11 categories)
… gross profit (7 categories)
… time period in which in which they are observed (5 periods covering 2 years each)
… region (8 regions)
… firm age (3 categories)
This identifies control firms for 67 participant firms (30.5%).
Participants and controls are restricted to be equal in terms of ….
… industry (143) categories
… number of employees (11 categories)
… gross profit (7 categories)
… time period in which in which they are observed (5 periods covering 2 years each)
… firm age (3 categories)
This identifies control firms for 103 participant firms (46.8%).
Participants and controls are restricted to be equal in terms of ….
… industry (33) categories
… number of employees (11 categories)
… gross profit (7 categories)
… time period in which in which they are observed (5 periods covering 2 years each)
… firm age (3 categories)
This identifies control firms for 165 participant firms (75.0%).
Participants and controls are restricted to be equal in terms of ….
… industry (33) categories
… number of employees (9 categories)
… gross profit (6 categories)
… time period in which in which they are observed (5 periods covering 2 years each)
… firm age (3 categories)
This identifies control firms for 169 participant firms (76.8%).
Participants and controls are restricted to be equal in terms of ….
… industry (33) categories
… number of employees (6 categories)
… gross profit (5 categories)
… time period in which in which they are observed (5 periods covering 2 years each)
… firm age (3 categories)
This identifies control firms for 184 participant firms (83.4%).
Step 2:
Step 3:
Step 4:
Step 5:
Step 6:
221
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0222.png
Step 7:
Participants and controls are restricted to be equal in terms of ….
… industry (9 categories)
… number of employees (6 categories)
… gross profit (5 categories)
… time period in which in which they are observed (5 periods covering 2 years each)
… firm age (3 categories)
This identifies control firms for 199 participant firms (90.5%).
Participants and controls are restricted to be equal in terms of ….
… industry (9 categories)
… number of employees (4 categories)
… gross profit (4 categories)
… time period in which in which they are observed (4 periods covering 3 years each)
… firm age (3 categories)
This identifies control firms for 202 participant firms (91.8%).
Participants and controls are restricted to be equal in terms of ….
… industry (9 categories)
… time period in which in which they are observed (5 periods covering 2 years each)
This identifies control firms for 220 participant firms (100.0%).
Step 8:
Step 9:
222
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0223.png
TABLE A1.2: Identification of second neighbours by balanced score matching
procedure
Step 1:
Participants and controls are restricted to be equal in terms of ….
Industry (143) categories
Number of employees (11 categories)
Gross profit (7 categories)
Year in which they are observed (9 years)
Region (8 regions)
Firm age (3 categories)
This identifies control firms for 33 participant firms (15.0%).
Step 2:
Participants and controls are restricted to be equal in terms of ….
… industry (143) categories
… number of employees (11 categories)
… gross profit (7 categories)
… time period in which in which they are observed (5 periods covering 2 years each)
… region (8 regions)
… firm age (3 categories)
This identifies control firms for 54 participant firms (24.6%).
Step 3:
Participants and controls are restricted to be equal in terms of ….
… industry (143) categories
… number of employees (11 categories)
… gross profit (7 categories)
… time period in which in which they are observed (5 periods covering 2 years each)
… firm age (3 categories)
This identifies control firms for 87 participant firms (40.0%).
Step 4:
Participants and controls are restricted to be equal in terms of ….
… industry (33) categories
… number of employees (11 categories)
… gross profit (7 categories)
… time period in which in which they are observed (5 periods covering 2 years each)
… firm age (3 categories)
This identifies control firms for 144 participant firms (65.5%).
Step 5:
Participants and controls are restricted to be equal in terms of ….
… industry (33) categories
… number of employees (9 categories)
… gross profit (6 categories)
… time period in which in which they are observed (5 periods covering 2 years each)
… firm age (3 categories)
This identifies control firms for 151 participant firms (68.6%).
223
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0224.png
Step 6:
Participants and controls are restricted to be equal in terms of ….
… industry (33) categories
… number of employees (6 categories)
… gross profit (5 categories)
… time period in which in which they are observed (5 periods covering 2 years each)
… firm age (3 categories)
This identifies control firms for 173 participant firms (78.6%).
Step 7:
Participants and controls are restricted to be equal in terms of ….
… industry (9 categories)
… number of employees (6 categories)
… gross profit (5 categories)
… time period in which in which they are observed (5 periods covering 2 years each)
… firm age (3 categories)
This identifies control firms for 194 participant firms (88.2%).
Step 8:
Participants and controls are restricted to be equal in terms of ….
… industry (9 categories)
… number of employees (4 categories)
… gross profit (4 categories)
… time period in which in which they are observed (3 periods covering 3 years each)
… firm age (3 categories)
This identifies control firms for 200 participant firms (90.0%).
Step 9:
Participants and controls are restricted to be equal in terms of ….
… industry (9 categories)
… time period in which in which they are observed (5 periods covering 2 years each)
This identifies control firms for 220 participant firms (100.0%).
224
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0225.png
APPENDIX 2:
ILLUSTRATION OF THE DIFF-IN-DIFF ESTIMATION SET-UP
Estimation of the programme’s effects is by a difference-in-difference model: For
both participants and controls, we calculate the average annual increases of the
success parameters in the years before the base year. We also calculate the average
annual increases of the success parameters in the years after the base year for both
participants and controls.
Thus, we can compare the (a) average increases of participants before they start par-
ticipating in an IC, (b) the average increases of participants after they have started to
participate in an IC, (c) the average increases of controls before they were selected as
controls (i.e. were most similar to one of the participants before it started to partici-
pate) and (d) the average increases of controls after they were selected.
So let a be a participant’s pre-base-year average annual increase in either success
parameter, b a participant’s after-base-year average annual increase in either suc-
cess parameter, c a control’s pre-base-year average annual increase in either success
parameter and d a control’s after-base-year average annual increase in either success
parameter.
Note b-a measures by how much a participant’s average annual increase in the suc-
cess parameter changes when the participants starts participating in the programme.
For controls, the difference d-c measures the difference in the average annual increa-
ses between before and after the base year.
Under the assumption that participants would continue having average annual
increases a in the absence of the programme, the average of the participant-specific
differences b-a estimates the IC’s causal average effect on participant firms.
However, this assumption is relatively strong, as b may be different from a for other
reasons than programme participation (e.g., business cycle or firm age effects).
But given the similarity of participants and controls in the base year, one may
argue that these ‘other reasons’ should have the same effect for both participants
and controls, and assume that b-a would on average be equal to d-c in the absence
of programme participation. Under this ‘identifying’ assumption, (b-a)-(d-c) is the
change in participants’ average annual increases between before and after the start
of participating in the IC which can only be explained by the programme, in other
words: the programme’s causal effect on participating firms.
To the extent that there remain dissimilarities between participants and controls in
observable factors such as industry, size or geographical region which potentially
could generate differences in the growth patterns of participants and controls, these
will be taken account of by including control variables in the regressions to follow.
When taking this model to the data, (b-a)-(d-c) is estimated by a simple linear re-
gression (with heteroscedasticity-consistent standard errors).
225
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0226.png
Here, we need to make decisions regarding the length of the time periods over which
pre-base-year and after-base-year average annual increases are computed. We made
the following choices: average pre-base-year increases are computed over a three-ye-
ar period before the base year. Average post-base-year increases are computed over a
five year and, as a second step, over a ten year period.
7
For the estimation of the model, we generate (typically) two observations per firm:
First, the average increases of the success parameters in the three-year period before
the base year. Second, the average increases of the success parameters in the five
(ten) year period after the base year.
This implies that we will only consider firms that were observed three years before
the base year, and at least five or ten years after the base year for the estimations.
The regression equation is taking the following form:
A = k + β1
*
d1 + β2
*
d2 + β3
*
d1d2 + β4
*
x + ε,
where A is the average increase in the success parameter in either the time period
before or after the base year and k is the constant term. d1 is an indicator variable
taking the value one (and zero otherwise) if the observation is after the base year,
d2 is an indicator variable taking the value one (and zero otherwise) if the obser-
vation belongs to a participant. cd2 takes the value one if the observation belongs
to a participant and is after the base year (and zero otherwise). x is a set of control
variables with an associated set of coefficients β4 to be estimated. β1, β2, and β3 are
also coefficients to be estimated, and ε is an error term assumed to satisfy standard
specifications.
Note that inclusion of the vector x is redundant in the sense that the matching
procedure implies high similarity in observable characteristics across participants
and controls. Still, inclusion of x increases the explanatory power of the model, and
might safeguard against potential differences between participants and controls.
These choices reflect compromises between the wish not to lose too many firms for the analysis which only are
observed for shorter time periods and the wish to being able to follow firms long enough to being able to detect
any effects in case they exist. Also, the precision of the growth trend measures increases with the length over
which the averages are calculated, which is relevant here because of considerable year-to-year volatility in the
success parameters. Basing estimates of pre-base-year time trends on a three-year period is a compromise
between not to lose too many firms for the analysis and the wish to generate reasonably stable estimates of pre-
base-year growth patterns.
7
226
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0227.png
Note further that
- the constant term k estimates average c, i.e., the average annual increases for
controls before the base year
8
,
- k+β1 estimates average d, the average annual increases for controls after the
base year,
- k+β2 estimates average a, i.e., the average annual increases for participants
before the base year,
- k+β1+β2+β3 estimates average b, i.e., the average annual increases for parti-
cipants after the base year.
Thus, β3 estimates average (b-a)-(d-c), which is, under the indentifying assumption,
the programme’s average causal effect for firms that participate in the programme.
In the language of the evaluation literature, β3 estimates the ‘average treatment ef-
fect on the treated (ATT)’.
Strictly speaking does k estimate the average annual increases for controls with all variables in the vector x
taking the value zero before the base year.
8
227
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0228.png
APPENDIX 3:
EXIT AND SURVIVAL OF PARTICIPANTS AND CONTROLS
The last additional step of the analysis is comparing participants’ and controls’ exit
and closure behaviour. In the following, exit will refer to a firm leaving the data be-
fore 2008 (which is the end of the observation period) – without making any distinc-
tions between the potential reasons for doing so.
Closure on the other hand is defined as one of the following transitions: bankruptcy,
liquidation, or forced exit. Information of these transitions is from the cvr-register of
the ministerial body ’
Erhvervs- og Selskabsstyrelsen
’.
There are 162 exit and 60 closure events in the data.
This appendix addresses two issues: first, whether fast growing participants have a
higher probability of staying in the data compared to controls. This would imply that
growth increase estimates for participants in association with the programme are
biased upwards.
The second question is whether participants have lower closure probability compa-
red to controls, which might be – given the similarity of participants and controls –
interpreted as a positive effect of the programme on participants’ survival.
TABLE A3.1 presents participants’ and controls’ exit status when leaving the data.
We find that approx. 76 per cent all firms stay in the data until 2008, which is the
end of the observation period. There is a higher share of participants that can be fol-
lowed until 2008. Participants have a lower propensity to exit in general, and espe-
cially to exit by a merger/acquisition event.
There is a higher share of participants that can be followed until 2008. Participants
have a lower propensity to exit in general, and especially to exit by a merger/acquisi-
tion event.
228
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0229.png
TABLE A3.1: Firm transitions (in per cent of total)
Participants
Continued until at least 2008
Merger/acquisition
Bankruptcy
Liquidation
Dissolution
Split up
Restructured
Forced exit
Erased from register
Total
88,18
3,18
4,55
2,73
0
0,45
0,91
0
0
100
Controls
69,93
14,81
5,01
3,87
1,82
1,37
1,14
1,14
0,91
100
Both
76,03
10,93
4,86
3,49
1,21
1,06
1,06
0,76
0,61
100
To test the statistical significance of this result, we employ a simply binary choice
logit model, which allows making statements on the differences in the expected exit
or closure probabilities of participants and controls. Results are presented in TABLE
A3.2, and can be summarized as follows:
The results of Model 1 provide evidence of participants having a significantly lower
probability of leaving the data as exits. The coefficient -0.856 implies that their pro-
bability of exiting in a given year is less than half of controls’ exit probability. Given
that ‘exit’ is by no means to be associated with ‘failure’, this finding is no indication
of participants being more successful than controls.
The result of Model 2 implies that there is no significant difference in the probability
to exit by a closure event (which might be interpreted as a success measure) between
participants and controls.
229
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0230.png
230
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0231.png
Central Innovation Manual on Excellent Econometric
Evaluation of the Impact of Interventions on R&D
and Innovation in Business
(CIM)
February 2013
231
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0232.png
CONTENTS
1 OBJECTIVE, VISION AND DELIMITATION
1.1
Objective
235
235
235
236
237
238
240
1.2 Focus and delimitation of the manual
1.3 Vision
1.4 Establishing minimum requirements and standards
1.5
Overview of the most important standards and minimum requirements
1.6 Overview of the most important impact assessments and results
2 PERFORMANCE OBJECTIVES STANDARD:
KEY PERFORMANCE INDICATORS
2.1. Independence and excellence
2.2. Ex ante evaluation
2.3 Baseline measurement at ex post evaluation
2.4 Key performance indicators/objectives: Results of impact evaluations
3 STANDARDS FOR COMPARISON GROUPS (CONTROL GROUPS)
3.1 Minimum requirements for the selection of comparable enterprises
3.2 Minimum requirements for the selection of comparable individuals
3.3 Standard method for selection of comparable control groups
3.3.1 Control groups may be selected using a so-called ‘propensity
score matching’ and ‘nearest neighbour’ method
3.3.2 The control group may be selected through comparison with
other innovation programmes
243
243
243
244
250
252
252
253
253
253
254
232
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0233.png
4 STANDARDS FOR STATISTICAL ANALYSIS METHODS
4.1 The difference-in-difference method
4.2 Balanced panel data
5 STANDARDS FOR CALCULATING ECONOMIC EFFECTS
6 STANDARDS FOR DATA TREATMENT
6.1 Causality and use of control groups
6.2 Standards for analysis of R&D-active or innovative enterprises
6.3 Treatment of outliers
6.4 Structure of output variable and valuation
6.5 Modelling of connection between instrument and effect
6.6 Spillover effects
7 STATISTICS FOR PERFORMANCE MEASUREMENTST
255
255
256
257
258
258
258
259
259
260
260
262
233
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0234.png
An excellent econometric impact evaluation of innovation policy is defined as
a performance measurement of an innovation policy instrument that has been
implemented in accordance with state-of-the-art econometric research methods,
and is of a quality on par with state-of-the-art research, facilitating publication of
methods and results in the most respected international research journals in the
relevant fields.
1
The main target group of this
Central Innovation Manual on Excellent Econometric
Evaluation of the Impact of Interventions on R&D and Innovation in Business
(CIM)
is programme owners in the Danish Ministry of Science, Innovation and
Higher Education and other government agencies who work with R&D, innovation
and business instruments, and who require better information and guidance on
the best methods for evaluating the impact of these instruments as well as wider
innovation and business policy.
Other target groups are external expert stakeholders, evaluation experts and
researchers who are interested in following and discussing how to conduct impact
evaluation studies with the Danish Ministry of Science, Innovation and Higher
Education. The manual also has the purpose of disseminating knowledge about the
best methods for performance measurements of research, innovation and business
policy in Denmark and elsewhere.
This manual (CIM) is not identical to similar work done in other countries
2
since
the key objective is to establish a clear set of minimum requirements for so-called
excellent econometric impact evaluations of innovation policy. CIM focuses on how
to set up a framework for a “standard” impact assessment procedure that makes it
possible to compare the impact of different instruments. CIM is not an attempt to
establish a practical guide on a broader number of methods on how to evaluate the
wider impact of R&D and innovation interventions. In this way, CIM complements
existing documents and reports.
3
See e.g. Kaiser and Kuhn (2012), Long-run Effects of Public-private Research Joint Ventures: the Case of the
Danish Innovation Consortia Support Scheme, Journal of Research Policy (forthcoming 2012).
1
See Guidance on evaluating the impact of interventions on business, Department for Business, Innovation and
Skills (BIS), august 2011
2
E.g. The role of evaluation in evidence-based decision-making, Department for Business, Innovation and Skills
(BIS), august 2010, and The Green Book – Appraisal and Evaluation in Central Government, Treasury Guidance,
London, United Kingdom, and The Magenta Book: guidance notes for policy evaluation and analysis, Government
Social Research Unit, HM Treasury, London, United Kingdom (October 2007)
3
234
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0235.png
1
OBJECTIVE, VISION AND DELIMITATION
1.1
Objective
The objective of this manual is to establish a number of minimum requirements and
standards for the implementation of excellent econometric impact evaluation of the
innovation policy instruments of the Danish Ministry of Science, Innovation and
Higher Education. However, anyone interested in econometric impact evaluation
in ministries and agencies might find it useful and informative. Accordingly, the
manual has been prepared in collaboration with Danish and non-Danish researchers.
It has been discussed at seminars with researchers
4
and policy makers and has
been presented for comments in the Danish Ministries of Finance, of Business and
Growth, of Climate and Energy, of Food, Agriculture and Fisheries, of Foreign
Affairs, and of the Environment. The manual is the result of the evaluation strategy
of the Danish Agency of Science, Technology and Innovation (DASTI) and has
been implemented as a 5-year research and innovation project about performance
measurements in the innovation field.
5
CIM summarises some key methodical
results, but the main elements of the 5-year project are 50+ evaluations that have
been conducted from 2007 to 2011.
1.2
Focus and delimitation of the manual
As the manual focuses on minimum requirements on excellent econometric impact
evaluations, it does not contain guidelines on other types of evaluation and perfor-
mance measurements of research and innovation programmes, such as research,
learning, organisational, internationalisation, equality or environment-related effects.
In particular, I would like to thank PhD Johan Moritz Kuhn and Professor PhD Anders Sørensen at Center for
Economic and Business Research (CEBR) at CBS (Copenhagen), and Michael Mark (DAMVAD Consulting) for their
comments on CIM.
4
Since 2007, the development of methods for performance measurements has been an ongoing work. See e.g.
the report DASTI (2007), Data og metoder ved effektmåling af innovationskonsortier (Data and methods for
performance measurements of innovation consortia) and DASTI (2009), Data og metoder ved effektmåling af
videnpiloter (Data and methods for performance measurements of knowledge pilots). Also see DASTI (01/2011),
which describes methods and data selection in relation to analyses of Industrial PhDs and Innovation Consortia,
respectively. Further developments are to be found in, Kaiser and Kuhn (2012), Long-run Effects of Public-private
Research Joint Ventures: the Case of the Danish Innovation Consortia Support Scheme, Journal of Research
Policy (2012). Also see DASTI (01/2010) and DASTI (02/2011).
5
235
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0236.png
Although CIM lists standards for impact evaluations, the intention has been
to do this in a way that makes room for flexibility. This is partly because the
recommended
‘propensity score matching method by nearest neighbour’
will not
be the most relevant method for all policy instruments. For instance, this can be
the case if the analysis has a different or wider focus than enterprise performance.
For other instruments, impact evaluations of economic performance targets are less
relevant if the main instrument purposes are non-economic activities. To provide
an example, this is the case for impact evaluations of clusters where the main
objectives are not just economic performance targets, but also include non-economic
behaviour-regulating performance objectives.
It is thus important that specific impact evaluations take into consideration the
objective of a given instrument. Accordingly, the manual also includes an overview
of the non-economic performance objectives that are listed for the most important
innovation instruments in the Danish Ministry of Science, Innovation and Higher
Education.
Finally, for many instruments it is challenging to establish sufficiently consistent
data series in terms of timeframe and number of observations. It is also challenging
to identify a (high quality) qualified control group in accordance with the same
conditions. So for new instruments, or instruments where only a relatively small
number of businesses have participated, it may be necessary to show a certain
amount of flexibility due to the data quality. Alternatively, it will be necessary
when such limitations occur to insist that impact evaluations be implemented using
methods that test the robustness of the results.
1.3
Vision
The vision of the Danish Agency of Science, Technology and Innovation (under the
Danish Ministry of Science, Innovation and Higher Education) is that the excellent
impact evaluations and analyses of R&D and innovation instruments carried out will
be examples of international best practice over the coming decade.
6
Internationally, there is an increasing interest in carrying out quantitative analyses
of the effects of enterprises’ activities in research, development and innovation.
The increased focus has been encouraged, among others, by the OECD,
7
which has
paid great attention to the subject through a coordinated effort among a majority of
the 27 EU countries as well as Korea, Norway, Switzerland, Russia, Turkey, South
Africa, and most of the countries in South America.
In the reports ‘Clusters Are Individuals – Benchmarking Insights from Cluster Management Organizations
and Cluster Programs’ by Kompetenznetze Deutschland (VDI/VDE Innovation + Teknik) and ‘Service innovation:
Impact analysis and assessment indicators’ by the European Commission’s Pro-Inno Net EPISIS, the Danish Min-
istry of Higher Education’s econometric performance measurements are singled out as being international best
practice.
6
7
OECD (2008), Science, Technology and Industry Outlook.
236
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0237.png
Most of these countries do not have the same possibilities as Denmark (or e.g.
Norway, Sweden and the Netherlands) due to limited access to quantitative
micro data and very long time series. In most of these countries, it is difficult to
establish the micro data basis needed to carry out solid and validated quantitative
econometric analyses that can document and calculate the effects of R&D and
innovation instruments historically.
In Denmark, the policy concerning evaluations and performance measurements is
that:
• The effect must be
documented
consistently for all innovation instruments.
•
Unambiguous key performance objectives
must be listed for all instruments
by the responsible authorities.
• Impact evaluations and performance measurements must be applied when
making decisions on possible
continuation, mergers or adjustments of
innovation instruments.
1.4
Establishing minimum requirements and standards
When establishing minimum requirements and standards, a number of issues must
be taken into consideration:
• The purpose of impact evaluations is firstly to document, in the best way
possible, economic effects and other key performance effects of existing
innovation instruments.
• Secondly, in more general terms it is important to be able to document
innovation policy effects in order to strengthen innovation policy as a policy
and political discipline.
• Thirdly, it is important to be able to establish a better understanding of the
different instruments in the innovation policy toolbox. This can be achieved,
for instance, by ensuring comparability of results
across
analyses and
across
innovation instruments in a far better way than has been the case until now.
• Fourthly, there is a need for evidence-based development and renewal of the
prioritisation tools for innovation policy.
The problem is that there are many degrees of freedom for impact evaluations, e.g.
in the choice of key performance indicators, success variables, choice of data basis,
treatment of outliers, choice of statistical analysis methods, interpretation of results
achieved etc. This means that a whole string of choices have to be made in the
course of carrying out excellent performance measurements.
237
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0238.png
The objective of this manual (CIM) is to create a framework for establishing
performance objectives (key performance indicators) in order to ensure a common
framework for the analysis methods and databases used for impact evaluations
and performance objectives, and to make possible better comparisons of key
performance indicators across instruments in Denmark and abroad.
1.5 Overview of the most important standards and
minimum requirements
CIM establishes a number of standards and minimum requirements for impact
analyses in order to illuminate the effects of innovation policy on key performance
indicators.
The manual is aimed at R&D and innovation instruments that may involve both
public and private participants. It is not aimed at instruments whose primary
purpose is to further basic research at public research institutions, universities etc.
The CIM requirements for an excellent econometric impact evaluation are high
data quality, use of the most recent research-based statistical methods, and a high
control group quality. With this in mind, CIM sets out 12 principles as minimum
requirements for an excellent impact evaluation.
238
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0239.png
12 principles: Minimum requirements for excellent econometric impact evaluations
List of key performance indicators for object to be analysed
1.
Unambiguous key performance indicators (based on ex ante evaluations of the instrument) formulated as
indicators of effects (input variables), throughput variables and results (output variables) must be listed in
instrument descriptions which are to be approved by the ministry’s management.
Identification and harmonisation of data collection
2.
Establish standards for data collection, including standards for input variables and registration in databases.
Standards for data collection are to be harmonised across all R&D and innovation instruments in the Danish
Ministry of Science, Innovation and Higher Education through a common electronic application system.
Data quality and long time series
3.
Ensure high data quality with long time series of at least 6-15 years with minimal data gaps in the time series. Use
of national registers for enterprise data and personal data as well as the ministry’s databases for applications,
appropriations, rejections and projects. Databases are to be established with time series of up to
20-25 years, depending on the instrument applied.
Treatment of data and quality requirements in identifying control groups
4.
5.
6.
Use of the difference-in-difference method and balanced panel data.
Use of the propensity score and nearest neighbour matching method for selecting the most comparable control
group / comparison group.
Use of alternative control groups / comparison groups with a clear and unambiguous interpretation option: e.g.
propensity score matching group, group of participants in other innovation policy instruments, rejection group
(group of enterprises and individuals whose applications have been rejected), group of enterprises within the same
industrial sector, etc. This facilitates analyses of an instrument's additionality (additional effect) and comparison
across instruments.
Selection of comparable (control) enterprises must be based on matching as many relevant parameters as
possible. The very highest requirements on quality and interpretation of data for comparison groups must be
stipulated.
Selection of comparable individuals (persons, researchers) must be based on matching as many relevant
parameters as possible. The very highest requirements on quality and interpretation of data for comparison groups
must be stipulated.
Outliers must be handled in accordance with the most established international methods in the fields of economic
research and econometric methods.
The key impact indicators must be relative in order to avoid comparison of uneven entities, e.g. through differences
in growth rates.
7.
8.
9.
10.
Robustness test
11.
Robustness tests are recommended in analyses with long time series and many observations. In case of data
limitations in the form of limited time series and observations, it is a requirement that impact evaluations be carried
out using methods that thoroughly test the robustness of the results.
Interpretation and peer review of results
12.
The quality and utility value of impact evaluations must be discussed with independent research organisations
that are not affiliated with the analyses, e.g. through peer reviews, research seminars, policy maker workshops
etc. Preferably, the results of the impact evaluations should be suitable for acceptance into the most reputable
international research journals.
239
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0240.png
This manual does not contain standards for criteria and administration. For
these, please refer to the evaluation and impact assessment strategy of the Danish
Ministry of Science, Innovation and Higher Education, the Danish Council for
Technology and Innovation, and to the action plans InnovationDenmark 2007-2010,
InnovationDanmark 2009-2012 and InnovationDenmark 2010-2013, which describe
the overall guidelines for administration, evaluation criteria and key performance
indicators of the various innovation instruments.
The establishment of standards for administration and evaluation criteria is
described in further detail for each innovation instrument in separate performance
description. These also describe the correlation between the innovation instrument
objectives and the key performance indicators for activities, effects and results alike.
1.6 Overview of the most important impact assessments
and results
More than 13 impact evaluations of various R&D, innovation and education
activities have been carried out since 2007. The impact evaluations have been
carried out by independent researchers or organisations and were commissioned
by the ministry or by independent institutions. 10 major impact assessments of
innovation instruments were conducted between 2010 and 2013 alone.
240
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0241.png
Examples of impact evaluations
Focus area
Study no. 1
Cluster and network policies
The independent impact evaluation of Innovation Networks Denmark Programme (DASTI
18/2011):
The programme supports the establishment and running of cluster and network
organisations. Among 1,200 non-innovative enterprises participating in the programme, the likelihood
of being innovative increased 300 percent compared to 1,200 statistically identical enterprises not
participating in the Innovation Networks Denmark infrastructure.
9
Among R&D-active or innovative
enterprises participating in the programme, the likelihood of initiating their first R&D collaboration
project with a research institution increased 300 percent compared to statistically identical enterprises
not participating in the programme.
Focus area
Study no. 2-4
R&D collaboration projects between business and research
Three independent impact evaluations (DASTI 06/2008, DASTI 03/2010, DASTI 01/2011 and
Kaiser & Kuhn (2012)) of the Danish Innovation Consortium Programme
(public grants to large
research collaboration projects between several enterprises, research institutions and technology
institutes) show that there are statistically significant impacts for enterprises as well as for individual
researchers depending on the key impact indicators analysed. Key performance indicators are for
gross profits, individual employment, employment in enterprises, patenting activity, salary and total
factor productivity. Some of the analyses show positive and statistically significant impact for small
and medium-sized enterprises with respect to labour productivity, patenting activity and employment.
None show similar impact on total factor productivity or for large enterprises. One study shows positive,
statistically significant impact on the salary level of researchers at the research institutions. Gross
profits increased on average EUR 2.7 million in an enterprise participating in an innovation consortium
over a period of nine years after the innovation consortium started. Enterprises did not receive public
grants.
An independent impact evaluation (DASTI 17/2011) of international research and development
collaboration projects (EUREKA projects)
was conducted in 2010. The impact of EUREKA
participation with respect to labour productivity, employment, turnover and exports was analysed.
The analysis shows a positive, statistically significant impact on growth rates in labour productivity,
employment, turnover and exports compared to statistically similar enterprises not participating in
EUREKA projects. EUREKA participation also results in significantly higher growth rate in exports and
employment compared to enterprises that only participate in the Innovation Consortium Programme
(and not in international projects).
An independent impact evaluation (DASTI 02/2011) on national research and innovation
collaboration projects between enterprises and universities or GTS-institutes
was conducted
in 2010 and 2011. Both projects with or without grants from public research funding bodies were
included. More than 1,500 R&D-active enterprises engaging in one or more R&D collaboration projects
with research and technology institutions in the period 1999-2006 were compared to more than
1,500 statistically identical enterprises that did not collaborate, selected from 20,000 Danish R&D-
active enterprises. The labour productivity is 9 per cent higher for the average enterprise with R&D
collaboration compared to statistically identical R&D-active but non-collaborating enterprises in the
analysed period. The analysis also looks at differences across different sectors, types of enterprises and
research institutions. Impacts are higher in large enterprises than in small enterprises. Impacts are also
higher in exporting enterprises than non-exporting enterprises. Finally, impacts increase with the skill
level in the enterprises.
Study no. 5
Study no. 6
http://fivu.dk/publikationer/2011/innovationsnetvaerk-skaber-vaekst
http://fivu.dk/publikationer/2011/innovationsnetvaerkenes-performanceregnskab-2011
9
241
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0242.png
Focus area
Study no. 7-8
Education and academics (candidates and PhDs) in the business sector
Two independent impact studies of the Danish Industrial PhD Programme (DASTI 2007 and
DASTI 01/2011)
show positive, statistically significant impacts. 200-300 participating enterprises and
400 participating Industrial PhD graduates, depending on the key impact indicators, are analysed.
The programme provides a subsidy to enterprises hiring PhD students to work on a PhD project. Key
performance indicators are labour productivity, individual employment, total employment in enterprises,
patenting activity, individual salary and total factor productivity. The 01/2011-analysis shows positive
and statistically significant impacts for small and medium-sized enterprises with respect to labour
productivity, patenting activity and employment compared to statistically similar enterprises without
Industrial PhD projects. Patenting activity nearly doubles and employment is nearly 2 persons higher
per PhD project per year. Both analyses show positive impacts for individual employment and salaries in
enterprises. Neither shows any impact on total factor productivity or on large enterprises.
An independent impact evaluation of the Danish Innovation Assistant Scheme (Knowledge
Pilots) (DASTI 04/2010)
shows positive but statistically insignificant impacts for enterprises. The
scheme provides a subsidy of up to EUR 20,000 to SMEs hiring university graduates. Key performance
indicators analysed are gross profits, total employment and survival rates of enterprises.
An evaluation of the Danish Innovation Assistant Programme (Videnpilotordningen) (DASTI ?/
2013)
shows that there are positive short-term employment effects for the innovation assistants, but no
statistically significant impacts for enterprises. The scheme provides a subsidy of up to EUR 20,000 for
SMEs hiring university graduates. Key performance indicators analysed are gross profits, value added,
return on assets, labour productivity, total employment and survival rates of enterprises.
Study no. 9-10
Study no. 11
An independent study of the impact of PhD candidates on productivity in enterprises (DASTI
2012, prepared by CEBR – Centre for Economic and Business Research at CBS, Copenhagen,
23 September 2011)
shows that the average labour productivity in enterprises with at least one PhD
candidate is approximately 34 percent higher compared to enterprises with the same mix of educations
and skills but without a PhD candidate. The impact of PhD candidates seems to be smaller in small
enterprises than in larger enterprises. The average labour productivity difference for small enterprises
with and without PhD candidates is 11 percent. The salary of PhD candidates is approximately 10 percent
higher than the salary of non-PhD individuals with same educational background, age and sex and
working in the same type of enterprise and business sector.
The Report on ‘Productivity and higher education’ conducted by the Centre for Economic and
Business Research (CEBR) for the Danish Business Research Academy (DEA) in 2010.
The effect
of different types of highly-educated working capacities on productivity (added value) in 138,372 Danish
enterprises over a nine year period (from 1999 to 2007) is analysed. The analysis shows that productivity
for each individual increases with the length of the individual’s educational background, regardless of
the field of education. An education within social sciences results in the highest individual productivity.
Technical and health sciences and scientific educations result in a slightly lower productivity than social
sciences. An increase of one percentage point in the share of employees with an education at a master’s
degree level will cause an increase in GNP by approximately 1 per cent.
Study no. 12
Focus area
Study no. 13
Commercial exploitation of public inventions
An independent impact evaluation of the Incubator Programme (DASTI 01/2010)
shows that there
are no statistically significant impacts for more than 300 enterprises and more than 300 entrepreneurs.
The programme provides public risk capital for the establishment of new knowledge intensive
enterprises. Key performance indicators analysed are individual salaries, total factor productivity, total
employment and survival rates of enterprises. Because of the lack of sufficient data and observations,
a new independent impact evaluation is to be conducted in 2014. The focus of the upcoming study is
impacts at enterprise level and for individual entrepreneurs.
242
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0243.png
2
PERFORMANCE OBJECTIVES STANDARD:
KEY PERFORMANCE INDICATORS
The Danish Ministry of Science, Innovation and Higher Education has headed an
international effort on performance indicators in the EU Pro-INNO project called
EPISIS. This collaboration had participants representing government agencies,
ministries and researchers from countries including Denmark, Sweden, Germany,
United Kingdom and Finland as well as the European Commission. Good practice
on evaluations and performance measurements was exchanged, and a manual was
elaborated with recommendations for indicators that can be used for setting out
performance objectives and key performance indicators.
2.1.
Independence and excellence
Decisive emphasis is placed on carrying out independent evaluations and
performance measurements. The intention is to carry out external performance
measurements based on the best and most widely accepted international research-
based statistical methods. Evaluations are carried out by independent researchers
and knowledge consultants. Efforts are undertaken to ensure the quality and utility
value of all impact evaluations by having the external and independent parties
discuss the evaluations with other independent research organisations that are not
behind the analyses. This can be achieved for instance by establishing steering
committees or conducting peer reviews, seminars etc. on a par with the procedures
and processes that also apply to publishing in international research journals.
Emphasis is placed on publishing the results of the completed impact evaluations in
for example the most accepted international journals or at high-level international
conferences.
2.2. Ex ante evaluation
The objectives and expected effects of each innovation instrument is stated in
separate instrument descriptions approved by the ministry’s management. This
means the instrument description also includes an ex ante evaluation of the
instrument. On this basis, the Danish Ministry of Science, Innovation and Higher
Education sets out key performance indicators for each innovation instrument,
which can be key performance objectives in the form of so-called output and input
objectives. The assessment of indicators to be selected follows the EPISIS project
work as well as national legislation.
In each instrument description, the ministry aims to document the choice of the
listed performance objectives, the work to follow up on the performance objectives,
and the plans for verifying the effects of the innovation instrument in question.
The overview below shows known key performance indicators for output (results),
input (effects) and assessment criteria for each innovation instrument in the ministry.
243
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0244.png
2.3
Baseline measurement at ex post evaluation
Emphasis is placed on ensuring baseline measurements of the efforts in order to be
able to document the situation before the launch of the innovation instrument and
the situation if the instrument had not been implemented. This enables estimating
the effects of the innovation instrument relative to a situation where the instrument
did not exist.
To this end, the most recent research-based methods are applied by choosing
advanced control groups that represent the situation if the instrument had not been
implemented. If the analysis includes a sufficiently large number of observations, the
propensity score matching method can be used for making baseline measurements,
cf. below. On this basis, ex post evaluations can be carried out with estimations of
instrument effects.
244
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0245.png
245
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0246.png
Key performance indicators
(impact) for each instrument
Individual employment
Effect on employment in enterprise
Added value growth in enterprise
(gross product)
Productivity per employee in enterprise
Individual salary effect
Survival rate for enterprises
Innovation
voucher
Innovation
consortia
Innovation
Assistant
X
X
X
X
X
X
X
X
X
X
X
X
X
X
Key performance indicators
(input and output) for each
instrument
Innovation ability
Investments in private research
Investments in innovation
PhD production, patenting etc.
Mobility of labour between public and
private sector
Regional distribution of activities
Collaboration projects between
enterprises and knowledge institutions
Gender distribution
Participation of small enterprises
Number of enterprises
Number of newly established
enterprises
Innovation
voucher
X
X
X
Innovation
consortia
X
X
X
X
Innovation
Assistant
X
X
X
X
X
X
X
X
X
X
X
X
X
X
246
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0247.png
Euro-stars
Industrial PhD
X
Networks and
clusters
GTS
institutes
Innovation
incubators
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
Euro-stars
Industrial PhD
Networks and
clusters
X
X
X
GTS
institutes
X
X
X
X
Innovation
incubators
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
247
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0248.png
Assessment criteria
Research height
Innovation height
Commercial utility
Social utility
Education
Employment
Project control and project management
Knowledge dissemination
Requirement on participation of small
enterprises
Partner composition and enterprise
participation
Economy and private co-funding
Professional focus area
Innovation
voucher
Innovation
consortia
X
Innovation
Assistant
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
248
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0249.png
Euro-stars
X
X
X
X
Industrial PhD
X
Networks and
clusters
GTS
institutes
X
X
X
X
X
X
(X)
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
249
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0250.png
2.4 Key performance indicators/objectives: Results of
impact evaluations
Econometric impact evaluations have been carried out for most instruments. Tables
1-3 show whether there are significant effects of the innovation instrument relative
to the control groups. The control groups may consist of either similar enterprises or
individuals that did not participate in the instrument.
Table 1 looks at instruments that involve direct enterprise grants.
Table 2 looks at instruments that focus on R&D, but where there are no direct
enterprise grants. In general, national business-research collaboration projects only
receive indirect enterprise grants through R&D funding at knowledge institutions.
Table 3 looks at initiatives where patenting activities have been analysed.
TABLE 1. Direct enterprise grants: Status of performance measurements
(effect relative to control group)
Performance
objective and
documented
effect in
evaluations
Innovation
assistant
Industrial PhD
Productivity
per
employee
Added
value in
enterprises
Employment
in
enterprises
Individual
salary
effect
Survival rate
Individual
employment
Insignificant
Insignificant
Insignificant
Insignificant
Insignificant
Significant on
a short-term
basis
Significant
Significant
Significant
Significant
Significant
Not a
performance
objective
Insignificant
Innovation
incubators
Eureka
Insignificant
Insignificant
Not studied
Insignificant
Not a
performance
objective
Not a
performance
objective
Significant
Significant
Significant
Insignificant
Not a
performance
objective
250
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0251.png
TABLE 2. No direct enterprise grants: Status of performance measurements
(effect relative to control group)
Performance
objective and
documented
effect in
evaluations
Innovation
consortia
Productivity
per
employee
Added
value in
enterprises
Employment
in
enterprises
Individual
salary effect
Export
growth
Share of
innovative
enterprises
in Denmark
Not a
performance
objective
Significant
Significant
Significant
Significant
Significant
Not a
performance
objective
Not a
performance
objective
Not a
performance
objective
Not a
performance
objective
Not a
performance
objective
Not a
performance
objective
Innovation
networks and
clusters
GTS
collaboration
University
collaboration
Purchase
of R&D at
knowledge
institution
Business
sector's
investments in
R&D
Significant*
Significant*
Not a
performance
objective*
Not a
performance
objective
Not a
performance
objective
Not a
performance
objective
Significant
Not a
performance
objective
Not a
performance
objective
Not a
performance
objective
Not a
performance
objective
Not a
performance
objective
Significant
Significant
Not a
performance
objective
Not a
performance
objective
Not a
performance
objective
Not a
performance
objective
Significant
Significant
Insignificant
Insignificant
Significant
Significant
* The innovation networks generate, among other things, collaboration projects with universities, GTS institutes and innovation consortia, and the
results follow the performance measurements for innovation consortia, GTS institutes and universities.
TABLE 3. Patenting activity
(effect relative to control group)
Documented
effect in impact
evaluations
Innovation
consortia
Industrial PhD
Patenting
activity
Significant*
Significant*
* CEBR research projects and DASTI 01/2011
251
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0252.png
3
STANDARDS FOR COMPARISON GROUPS (CONTROL GROUPS)
In order to assess the isolated results and effects of an innovation instrument or
the difference in results and effects between two instruments, the development of
key performance indicators for the enterprises or individuals participating in an
innovation policy instrument must always include a comparison group (control
group) of enterprises or individuals. The purpose is to study the difference in
the results between two instruments, or whether there is an added effect from
participating in an instrument as opposed to not participating.
3.1 Minimum requirements for the selection of
comparable enterprises
When selecting enterprise control groups, it is important to consider that the
enterprises that participate in an instrument are compared with other enterprises
that are not participating, but are similar in as many relevant parameters as possible
that may be of significance to the effect of the analysed instrument. The minimum
requirements are that as many different factors as possible are taken into account,
but this also depends on the instrument analysed. When selecting control groups,
enterprises must be chosen that are more or less equally likely to participate, but
have not. The probability model can be based on the following variables:
•
•
•
•
•
•
•
•
Educational level of the enterprise’s employees
R&D intensity
R&D department
Export intensity
R&D investments
Profit, surplus or contribution margin
Enterprise size
Industrial sector
It is recommended that a propensity score matching be used. The point of the
comparison group is to figure out what would have happened at participating
enterprises if they had not participated. If the alternative to participation would
be that the enterprises participated in a similar initiative, it makes good sense to
compare with other enterprises that participated in a similar initiative - otherwise, it
does not.
However, it is important to avoid including too many explanatory variables, which
may give overlapping results, either individually or in combination. By including
too many identical variables, there is a risk that multicollinearity will occur along
with too great a correlation between the explanatory variables. This means that
the parameters become insignificant and the result becomes biased. An example is
if R&D intensity is included along with R&D investments, R&D department and
enterprise size, as there is interdependency between these variables.
252
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0253.png
3.2 Minimum requirements for the selection of
comparable individuals
When selecting control groups for individuals, individuals must be chosen who were
as likely to participate as the participating individuals, yet did not. The probability
model can be based on the following variables:
•
•
•
•
•
•
Education
Educational institution
Enterprise size
Industrial sector
Gender and age
Any other socioeconomic variables, such as salary, background etc.
It is recommended that a propensity score matching be used. When comparing with
what would have happened if the individual had participated in another initiative, a
control group can also be a group of individuals who participate in the other similar
initiative.
3.3 Standard method for selection of comparable control
groups
3.3.1 Control groups may be selected using a so-called
‘propensity score matching’ and ‘nearest neighbour’ method
The recommended standard method is the ‘propensity score nearest neighbour
matching method’, which is used to establish and delimit, on a one-to-one scale, the
group of enterprises that participate in an instrument, and a statistically comparable
control group of enterprises that do not participate, but are as likely to do so.
It is impossible to find a control group that is completely identical.
11
The probability
models for enterprises’ participation in an instrument, which are used to identify
factors that affect the likelihood of participation, are set out as logistic regressions
and used in connection with the propensity score matching method.
In most cases, it will be an advantage to put together a control group that has
as many control enterprises as possible – based on the law of large numbers.
Accordingly, one-to-one is a minimum requirement, but the standard should be
one-to-many. Furthermore, this should be supplemented by balance tests in order to
analyse the difference between the treatment group and the control group.
11
Examples of application of this method are found in DASTI (01/2010) and DASTI (02/2011).
253
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0254.png
The idea of the propensity score matching method is that for an enterprise T which
participates in the instrument, an enterprise C is found among the other enterprises
in the statistical material. For a number of statistical parameters, enterprise
C should resemble enterprise T by having the same probability (‘propensity
score’) of participating in the instrument, except that in actuality, enterprise C
has not participated. In this way, enterprise T (designated as the ‘treatment’ or
‘participating’ enterprise) can be compared to a similar enterprise C (designated as
the ‘comparison’ or ‘control’ enterprise) found in the statistical material. Statistically,
enterprise C must resemble enterprise T with regard to industrial sector, enterprise
size, export pattern, staff education, profit, contribution margin and composition as
well as R&D or innovation activities.
It naturally follows that it is not possible to find a control group that is completely
identical in all partially unobservable factors using this or other methods. Another
selected control group may give different results. It is thus important to be able to
interpret the characteristics found in the control group.
3.3.2 The control group may be selected through comparison
with other innovation programmes
When comparing effects across instruments, the standard is that the comparison
group is found among participating individuals or enterprises in the instruments to
be compared. Here, it is important that observation sorting and data cleaning as a
minimum is done the same way for all instruments.
12
Examples of a programme comparison is the comparison between ordinary PhDs and Industrial PhDs in DASTI
(01/2011), and the comparison between enterprises participating in EUREKA projects and innovation consortia in
DASTI (15/2011).
12
254
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0255.png
4
STANDARDS FOR STATISTICAL ANALYSIS METHODS
The possibilities depend on the design of the instruments. For some innovation
instruments, considerably more precise estimation methods than the matching
method described above and the difference-in-difference method described below
are possible. This depends on, for instance, whether a regression-discontinuity
design is possible.
4.1
The difference-in-difference method
One of the recommended central statistical methods that has been used until now
is the difference-in-difference method. This method is used to calculate differences
between developments of the treatment group and the control group.
13
The difference-in-difference method is based on comparing output changes (the
performance objective). The model looks as follows:
�½
Y
1
T
Y
0
T
Y
1
C
Y
0
C
- in which
is
Y
1
T
effect
the instrument. This is calculated on basis of the
�½
the
Y
0
T
of
Y
1
C
Y
0
C
difference between the performance indicator development, called Y, of the
treatment group (T) - defined as the performance indicator at time 1 minus the
performance indicator at time 0 - and the performance indicator development of
the control group (C) - defined as the performance indicator in time 1 minus the
performance indicator in time 0. Whether there is a significant difference between
the two can be tested subsequently by e.g. standard t-tests or linear regression.
BOX 1. Central analysis method: Difference-in-difference
Difference-in-difference:
(a) before-after comparison for enterprises that participate in the instrument
(participant)
(b) before-after comparison for enterprises that do not participate in the
instrument (control)
See whether (a) is more positive than (b).
T1 – success parameter of participant before.
T2 – success parameter of participant after.
C1 – success parameter of non-participant before.
C2 – success parameter of non-participant after.
The difference (T2-T1)-(C2-C1) measures the difference between the increases.
13
Examples of application of this method may be found in DASTI (1/2010) and DASTI (2/2011).
255
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0256.png
4.2
Balanced panel data
The effect of enterprises’ R&D investments on added value and productivity per
employee is a dynamic process which may vary over time. Cross-sectional analyses
based on a single year are not adequate for analysis of the variation over time. In
addition, there may be unobservable effects on the individual enterprise which
the models are not able to take into consideration. The before-after comparison
that results from applying the difference-in-difference method means that panel
data (cross-sectional data over time) and methods are needed to check for these
unobservable effects.
Large enterprises are included in R&D statistics every year, while samples of
small and medium-sized enterprises are selected randomly. The result is a very
‘unbalanced’ panel. For some enterprises, observations are available for all years,
while others only have data for one or a few years.
Because of this, it is recommended that the panel data set is put together as follows:
• Panel data analyses are only to be made for enterprises with at least two
observations. In order to ensure that the analyses are as representative as
possible, all enterprises with two or more observations are to be included.
If the data basis allows, the requirements may be made more stringent, so
only enterprises with three or more observations are included. Naturally, this
will reduce the number of enterprises in the analysis.
• The following approach is recommended for missing observations in
time series: If a single observation is missing in a time series, the single
missing observation should be estimated. If two or more years are missing in
the time series, the most recent continuous part of the time series should be
kept.
• Extensive changes in the variables may indicate a merger or division of the
enterprise. Such changes may have a disproportionately large effect on the
results. It is recommended that enterprises with annual growth rates in added
value, fixed assets, number of employees or R&D capital of less than - 50 %
or more than 300 % be removed. This is in accordance with the standard set
out in international literature.
• It is recommended that sensitivity analyses be carried out when basic data are
changed.
256
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0257.png
5
STANDARDS FOR CALCULATING ECONOMIC EFFECTS
The Cobb-Douglas productivity function is used as a standard for indicating
the effects of a given instrument in pounds and pence in the form of increased
productivity per employee, profits, etc. This is typically modelled as an OLS
regression.
14
Depending on the chosen key performance indicator (the analysed success variable),
changes of levels over time may also be seen as might annual growth rate changes
over time. An example of changes in levels would be changes in the number of
employees and in the level of employment.
An example of relative changes would be the survival rate of enterprises or
employment quotas. Examples of changes in growth rates are growth in productivity
per employee, growth in turnover or growth in added value in enterprises.
In general, the standard for calculating economic effects depends on the key
performance objectives that are assessed and estimated.
When selecting background factors, it is important to consider how the individual
background factors affect both outcome and treatment. For instance, there may be
a time-related challenge with background variables which might be affected by
treatment in a model that includes lagged variables.
14
Examples of application of this method may be found in DASTI (02/2011).
257
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0258.png
6
STANDARDS FOR DATA TREATMENT
6.1
Causality and use of control groups
The effect of R&D investments and a particular instrument are often indirect and
therefore difficult to measure and identify. It is difficult to isolate the actual effect
that may be the result of many and varying external factors. It is also difficult to
identify causality.
The selection of control groups is important for the question of causality.
Accordingly, a standard is recommended for the establishment of control
groups based on information about the enterprises’ industry, export, size,
internationalisation, R&D characteristics, employee composition and employee
education. This way, a basis is established for making it probable whether there is a
causal connection between the factor analysed and the performance objective, along
with the basis for measuring the isolated effect.
6.2 Standards for analysis of R&D-active or innovative
enterprises
In general, analyses of R&D instruments are based on employee productivity,
employment, profits, survival rates, patent activity etc. at R&D-active enterprises
only. If enterprises that do not conduct research and development, e.g. innovative or
non-innovative enterprises, were to be included in the econometric analysis, it would
be necessary to apply suitable methods to allow for differences between R&D-active
and non-R&D-active enterprises. The methods are relatively complex and require an
extensive analysis of the factors that make enterprises choose to invest in R&D.
15
It should be assessed whether a control group should be established from R&D-
active enterprises only or whether innovative enterprises and non-innovative
enterprises should also be included.
If the control group consists of R&D-active enterprises only this must be justified,
e.g. by how the analysed instrument is not an instrument that all enterprises can
participate in overnight, but is restricted to R&D-active enterprises only.
This is a strict assumption which will undoubtedly exclude enterprises that were
predisposed for the analysed activity. Conversely, it may also be a conservative
assumption that helps ensure robustness of results, as it avoids a control group of
enterprises where the probability of participation is very low.
The methods first estimate the tendency to invest in R&D and then estimate what the enterprises’ R&D activities
would have been if the enterprises had chosen to invest in R&D. These estimated values can be used in productiv-
ity analyses or other performance measurements. The so-called CDM model (Crépon et al, 1998) applies a similar
approach to analysing the relationship between innovation and e.g. productivity, albeit only in part. Crépon et al
estimate the tendency to be innovative in order to check for selection bias, but only include R&D-active enter-
prises in the analysis.
15
258
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0259.png
6.3
Treatment of outliers
For results to be as representative as possible, econometric models should be able
to measure effects in a wide range of enterprises. However, extreme values may
distort the effects and reduce precision. In some cases, there may be good reasons
for removing extreme values. An example is young enterprises where large and risky
investments are made, which affect the enterprises’ added value for a short period of
time. Such enterprises will potentially experience extreme increases from one year
to the next.
However, whether or not extreme values should be removed depends on the purpose
of the analysis and the innovation instrument. Thus, a careful assessment of outliers
should be carried out for each analysis and instrument before they are excluded.
Furthermore, data have been found to include extreme values measured against e.g.
enterprises’ average productivity per employee, added value, employment, etc. These
are assumed to be incorrect registrations connected either to the enterprise’s added
value or to the number of full-year employee equivalents. Regardless of where the
incorrect registration is found, it is recommended that such values are removed from
the data.
There may however be other methods, for example to include or exclude extreme
data to see whether this has any effect on the results, or to consider medians, etc.
6.4
Structure of output variable and valuation
It is not always easy to identify and delimit effects. Also, differences occur in
valuation depending on players and stakeholders. An example is an enterprise’s
market value. One way is to use the market’s valuation of the individual enterprise
as a measure for the price or value of the total ‘tangible’ and ‘intangible’ assets.
However, this would require the enterprises in the analysis to be quoted on the stock
exchange. This means this method is not used, as most enterprises are not quoted on
the stock exchange.
When effects in enterprises are analysed, it is recommended that a key performance
indicator relative to labour input is used. This way, it is ensured that the effects
cannot be attributed to an endless supply of labour.
259
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0260.png
6.5 Modelling of connection between instrument and
effect
Effect measuring is complex, since a linear connection between the analysed
instrument and a subsequent effect is hardly ever found. Accordingly, there are a
number of conditions that may make it difficult to measure effects, such as potential
time layers before the effects set in, different starting points for the enterprises,
differences between the enterprises’ characteristics and the enterprises’ experience
and competences with regard to the instrument.
As a standard, the econometric models must therefore be able to make allowance
for:
• Time lag between the analysed instrument and its effects. The effects may set
in with varying delays.
• Correction for enterprise differences. The enterprises in the analyses will
vary in size, industry, market conditions, globalisation and other objective
factors. It is important to check for these factors when isolating the effects. To
avoid ‘losing’ some of the effects in the analyses because the data set includes
many different enterprises where there will be different effects, the analyses
should both treat data as a whole and include information for each enterprise/
individual about their industrial sector and number of employees.
• It is also recommended to carry out enterprise analyses for different industrial
sectors and enterprise sizes if the data basis allows.
6.6
Spillover effects
The transaction mechanisms between activities and their yield are complex, as there
is no linear connection between activities and yield. Besides, there may be multiple
gains that may be difficult to delimit and valuate.
16
In OECD contexts, the concept of behaviour additionality is used increasingly to measure and define the multiple
gains from innovation instruments, among other things. However, it is still very difficult to attach a value on the
additionalities.
16
260
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0261.png
One of the challenges in measuring the effects of innovation instruments is that
knowledge is a ‘non-competing’ advantage. This means that enterprises, individuals
or public institutions may benefit from knowledge produced by others. And if such
knowledge is transferred, it can be further developed through other innovation
instruments. This becomes even more evident for innovation policy instruments that
are meant to be combined with other instruments, e.g. if an Industrial PhD student
takes part in an innovation consortium, or if innovation consortia or innovation
voucher collaboration projects are facilitated through activities in the Danish
Ministry of Higher Education’s innovation networks. In literature, some researchers
argue that knowledge increases in value when it is shared and used by several
different players and enterprises. The increase and dissemination of knowledge
between the different players and enterprises is achieved by collecting knowledge
and through the mobility of labour, as employees carry with them knowledge they
have gained through other enterprises and research institutions’ investments in
research, development and innovation.
Other enterprises than the one that has participated in the analysed activity will have
higher marginal earnings on a product, either because manufacturing the product
has become more efficient and thus cheaper, or because the production value has
increased and the product can be sold at a higher price. However, the effect does not
only benefit the manufacturer, but all links in the value chain, right through to the
wholesaler or retailer.
The spillover effect from knowledge can also create so-called creative destruction.
Here, innovation and development of new products and services will remove
value from existing products and services. As a result, it has a negative impact
on the effects for other enterprises. Hence, performance measurements should be
supplemented by other types of economic models which may pick up transmission
mechanisms and spillover effects better than microeconomic models, if the full
effect of the analysed activity at a socio-economic level is to be exposed.
261
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0262.png
7
STATISTICS FOR PERFORMANCE MEASUREMENTST
The following national statistics are used in connection with the impact evaluations:
•
•
•
•
•
•
•
•
•
R&D statistics (Denmark’s National Statistical Bureau)
Accounts statistics (Denmark’s National Statistical Bureau)
Community Innovation Survey (CIS) (Denmark’s National Statistical Bureau)
Education statistics (Denmark’s National Statistical Bureau)
Project databases in ministries and funding agencies
Patent statistics (Denmark’s National Statistical Bureau)
Labour market statistics (Denmark’s National Statistical Bureau)
Salary statistics (Denmark’s National Statistical Bureau)
The Danish Commerce and Companies Agency’s Central Business Register /
Købmandsstandens Oplysningsbureau/Experian A/S
(Danish Business
World’s Information Agency)
262
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0263.png
263
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0264.png
PUBLICATIONS
Publications from the Danish Agency for Science, Technology and
Innovation in the series
Innovation: Analysis and evaluation
2013
13/2013
12/2013
11/2013
10/2013
09/2013
08/2013
07/2013
06/2013
05/2013
04/2013
03/2013
02/2013
01/2013
2012
14/2012
13/2012
12/2012
10/2012
09/2012
08/2012
07/2012
06/2012
05/2012
04/2012
03/2012
02/2012
01/2012
2011
20/2011
19/2011
18/2011
17/2011
16/2011
Analyses of Danish Innovation Programmes – a compendium of excellent
econometric impact analyses
An evaluation of the Danish Innovation Assistant Programme
The Effect of the Industrial PhD Programme on Employment and Income
Strategi for samarbejde om Danmarks klynge- og netværkindsats
De skjulte helte – eksportsucceser i dansk industris mellemklasse
An Analysis of the Level of Consistency in the Danish Innovation
Ecosystem
Key Success Factors for Support Services for Cluster Organisations
Performanceregnskab for GTS-net 2013
Kommercialisering af forskningsresultater – Statistik 2012 (Public
Research Commercialisation Survey – Denmark 2012)
Performanceregnskab for Innovationsnetværk Danmark 2013
Productivity Impacts of Business Investments in R&D in the Nordic
Countries - A microeconomic analysis
Erhvervslivets forskning, udvikling og innovation i 2013
Performanceregnskab for innovationsmiljøerne 2013
Evaluering af GTS-instituttet DFM
Evaluering af GTS-instituttet Alexandra
Evaluering af GTS-instituttet Agrotech
Let’s make a perfect cluster policy and cluster programme: Smart
recommendations for policy makers
The Perfect Cluster Programme - Nordic-German-Polish-Baltic project
The impacts of Danish and Bavarian Cluster Services – results from the
Nordic-German-Polish Cluster Excellence Benchmarking
Kommercialisering af forskningsresultater – Statistik 2011 (Public
Research Commercialisation Survey – Denmark 2011)
Performanceregnskab for GTS-net 2012
Performanceregnskab for Innovationsmiljøer 2012
Innovation Network Denmark – Performance Accounts 2012
Clusters are Individuals II: New Findings from the European Cluster
Management and Cluster Program Benchmarking
Erhvervslivets forskning, udvikling og innovation i 2012
Evaluering af innovationsmiljøerne
Access to Research and Technical Information in Denmark
Universiteternes Iværksætterbarometer 2011
Impact Study: The Innovation Network Programme
Clusters are Individuals: Nordic-German-Polish Cluster Excellence
Benchmarking
24 ways to cluster excellence – successful case stories from clusters in
Germany, Poland and the Nordic countries
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
264
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0265.png
15/2011
14/2011
13/2011
12/2011
11/2011
10/2011
09/2011
08/2011
07/2011
06/2011
05/2011
04/2011
03/2011
02/2011
01/2011
Impact Study of Eureka Projects
Evaluering af GTS-instituttet Teknologisk Institut
Evaluering af GTS-instituttet DBI
Evaluering af GTS-instituttet DELTA
Kommercialisering af forskningsresultater – Statistik 2010 (Public
Research Commercialisation Survey – Denmark 2010)
Performanceregnskab for Videnskabsministeriets GTS-net 2011
Performanceregnskab for Videnskabsministeriets Innovationsmiljøer 2011
Innovation Network Denmark – Performance Accounts 2011
Erhvervslivets Outsourcing af FoU
Evaluering af GTS-instituttet FORCE Technology
Evaluering af GTS-instituttet Bioneer
Evaluering af GTS-instituttet DHI
Erhvervslivets forskning, udvikling og innovation i 2011
Økonomiske effekter af erhvervslivets forskningssamarbejde med
offentlige videninstitutioner
Analysis of Danish innovation policy - The Industrial PhD Programme
and the Innovation Consortium Scheme
2010
12/2010 Brugerundersøgelse af GTS-institutterne 2010
10/2010 Universiteternes Iværksætterbaromenter 2010
09/2010 Performanceregnskab for Videnskabsministeriets Innovationsmiljøer
2010
08/2010 Innovationsnetværk Danmark - Performanceregnskab 2010
07/2010 Performanceregnskab for Videnskabsministeriets GTS-net 2010
06/2010 Kommercialisering af forskningsresultater - Statistik 2009
05/2010 InnovationDanmark 2009 - resultater og evalueringsstrategi
04/2010 Effektmåling af videnpilotordningens betyd¬ning for små og mellemstore
virksomheder
03/2010 An Analysis of Firm Growth Effects of the Danish Innovation
Consortium Scheme
02/2010 Erhvervslivets forskning, udvikling og innova¬tion i Danmark 2010
01/2010 Produktivitetseffekter af erhvervslivets forsk¬ning, udvikling og
innovation
265
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647746_0266.png
266
Analyses of Danish Innovation Programmes – a compendium of excellent econometric impact analyses