Uddannelses- og Forskningsudvalget 2015-16
UFU Alm.del
Offentligt
1647742_0001.png
Samarbejde
An Analysis of Firm Growth Effects of the
Danish Innovation Consortium Scheme
Innovation: Analyse og evaluering 3/2010
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0002.png
Kolofon
>
An Analysis of Firm Growth Effects of the Danish
Innovation Consortium Scheme
Udgivet i april 2010
ISBN (web): 978 87 923 7267 3
Udgivet af:
Publikationen kan downloades fra Forsknings- og
Innovationsstyrelsens hjemmeside:
Forsknings- og Innovationsstyrelsen
Bredgade 40
1260 København K
Telefon: 3544 6200
Fax: 3544 6201
E-mail: [email protected]
http://www.fi.dk
Tekst: CEBR – Centre for Economic and Business
Research - Johan M. Kuhn, Ph.D.
Grafisk design: Formidabel
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0003.png
>
An Analysis of Firm Growth Effects of the
Danish Innovation Consortium Scheme
Innovation: Analyse og evaluering 3/2010
by
CEBR - Centre for Economic and Business Reasearch
Johan M. Kuhn, Ph.D.
Danish Agency for Science, Technology and Innovation, April 2010
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
3
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0004.png
Table of Contents
>
Executive summary
Sammenfatning (Danish summary)
1. Introduction
2. Description of the IC scheme
3. Data
4. Sampling
5. Identification of the control group
6. Estimation set-up
7. Descriptive Statistics
8. Results
8.1 Gross profit developments
8.2 Employment developments
9. Alternative samples and robustness
10. Conclusions
11. References
Appendix 1: Selection of controls
Appendix 2: Illustration of the diff-in-diff
estimation set-up
Appendix 3: Exit and survival of
participants and controls
5
7
10
13
14
15
17
18
20
23
23
29
34
40
42
44
50
53
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
4
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0005.png
Executive summary
>
This report has been prepared by the Centre for Economic and Business
Research (CEBR). It presents an analysis of the economic impact of
’Innovationskonsortieordningen’ (Innovation Consortium scheme, IC scheme) on
participating firms.
The IC scheme is a Danish subsidy scheme granted by Rådet for Teknologi og
Innovation (The Danish Council for Technology and Innovation, RTI) in cooperation
with Forsknings- og Innovationsstyrelsen (The Danish Agency for Science,
Technology and Innovation, FI).
This analysis follows 220 firms which have participated in at least one Innovation
Consortium using a firm-register dataset. We primarily study firm level
developments in two success parameters: gross profit and employment.
While employment is simply the number of employees in a given firm at a given
point in time, gross profit is a measure of the firm’s value creation.
In this study, we consider (absolute and percentage wise) growth in gross profit
and the number of employees both before and after programme participation and
analyse the changes in the growth patterns in association with participating in the
programme. Moreover, we identify a control group of firms that do not participate
(non-participants), but which are similar to the participants in terms of size, industry,
and region.
Again, we can use firm-level data to calculate the changes in gross profit and
employment for the non-participants, allowing us to address the question of whether
participants have higher increases in growth than what would be expected on basis
of the growth patterns of non-participants.
Under the assumption that gross profit and employment developments of
participants and non-participants would be symmetric in the absence of programme
participation, differences between the two groups of firms can be interpreted as the
causal impact of the programme on participating firms.
The results of the analysis can be summarized as follows: It is possible to show that
firms that participated in the IC scheme have experienced significant increases in the
growth of gross profit and employment in association with programme participation.
These results are robust to controlling for pre-participation growth and developments
in the growth of firms in the control group.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
5
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0006.png
>
Findings depend on the participant firms under consideration.
1
We, for example,
find positive potential gross profit effects that are significant at the five percent
significance level for firms with a gross profit below 150 million DKK (approx. €20)
in the year before the programme. We also find potential employment effects for
firms with less than 150 employees in the year before the programme.
For firms with gross profit less than DKK150 million in the year before
participation, estimates show that, on average, annual gross profit in a participating
firm has grown by an additional approx. DKK2 million per year relative to firms in
the control group. This implied an on average approx. DKK20 million difference in
annual gross profit after 10 years. It should be noted that one should be careful when
interpreting this result, both because of statistical uncertainty and the possibility of
participant and controls firms being different in unobserved factors potentially being
important with regards to the observed differences. But when one relates the approx.
DKK20 million difference to the programme’s research subsidies – corresponding
to approx. DKK3 million (approx. €370,000) per participant firm – it becomes clear
that the programme is a success even in case of only a share of the observed gross
profit differences owing itself to a genuine causal effect of the programme.
This result is robust to changing sampling conditions and using firms that applied for
funding and got their application rejected as an alternative control group. Results for
employment growth are not robust to using the alternative control group, and should
thus be interpreted as being more tentative.
1
For the largest participant firms, any effects of the programme are small relative to these firms’ large
variations in the success parameters, and inclusion of large firms in the sample renders impossible finding
any potential positive programme effects.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
6
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0007.png
Sammenfatning (Danish summary)
>
Denne rapport er skrevet af Centre for Economic and Business Research (CEBR).
Den beskriver en analyse af Innovationskonsortie-ordningens potentielle effekter på
udviklingen i de deltagende virksomheder.
Innovationskonsortie-ordningen er et virkemiddel under Rådet for Teknologi og
Innovation (RTI). Rådet administrerer virkemidlet i samarbejde med Forsknings- og
Innovationsstyrelsen (The Danish Agency for Science, Technology and Innovation,
FI). Gennem Innovationskonsortier støtter RTI samarbejde mellem virksomheder og
vidensinstitutionerne (f.eks. universiteter, GTS-institutter m.fl.).
Ved hjælp af registerdata følger analysen 220 virksomheder som har deltaget i
ordningen. Vi studerer væksten i to succesmål: bruttofortjeneste og beskæftigelse.
Mens beskæftigelse er antallet af medarbejdere på et givet tidspunkt, er
bruttofortjeneste et mål for virksomhedens værdiskabelse.
I dette studie betragter vi væksten i bruttofortjeneste og beskæftigelse både før
og efter starten af programdeltagelsen. Yderligere identificerer vi en gruppe af
kontrolvirksomheder som ikke deltager, men som ellers ligner de deltagende
virksomheder i størrelse, branche, alder og region. Også for dem udregner vi vækst
i bruttofortjeneste og beskæftigelse. Det betyder, at vi kan besvare spørgsmålet
hvorvidt de deltagende virksomheder har haft højere vækst end man ville have
forventet - ikke kun på basis af deres vækst før programdeltagelsen, men også på
basis af udviklingen for kontrolvirksomhederne.
Ud fra antagelsen om at udviklingen i bruttofortjeneste og beskæftigelse ville
være symmetrisk i fraværet af ordningen, kan differencen mellem de to gruppers
udvikling fortolkes som ordningens direkte effekt for de deltagende virksomheder.
Analysens resultater kan sammenfattes som følger: Mindre virksomhederne,
som har deltaget i Innovationskonsortie-ordningen, har oplevet større vækst
i bruttofortjenesten og i antallet af medarbejdere end virksomhederne i
kontrolgruppen, der ikke har deltaget. Disse resultater er robuste overfor at der
korrigeres for væksten inden programdeltagelsen og korrigeres for udviklingen i
væksten i kontrolgruppen.
Der skal dog lægges mærke til, at resultaterne afhænger af størrelsen af de
virksomheder, som betragtes.
For eksempel er den potentielle effekt på bruttofortjenesten signifikant på et
5 % niveau for deltagervirksomheder, der havde under 150 millioner Kr. i
bruttofortjeneste i året før programdeltagelsen. Differencen på bruttofortjenesten
kan her estimeres til ca. 2 millioner kr. ekstra vækst i deltagervirksomhedernes
årlige bruttofortjeneste om året. Dette betyder, at deltagervirksomhedernes årlige
bruttofortjeneste er blevet forøget med ca. 20 millioner kr. over en ti års tidshorisont.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
7
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0008.png
>
Sådan en sammenligning skal fortolkes med en vis forsigtighed grundet statistisk
usikkerhed, og det at forskellen sandsynligvis delvis skyldes faktorer, som analysen
ikke kan tage højde for. Med programomkostninger svarende til ca. 3 millioner kr.
pr virksomhed kan det dog konkluderes, at ordningen er en succes selv i tilfældet at
kun en mindre del af differencen skyldes en kausal effekt.
Vi finder yderligere signifikant positive potentielle beskæftigelseseffekter for
virksomheder, der havde mindre end 150 medarbejdere året før programdeltagelsen.
Disse potentielle effekter svarer til ca. 50 ekstra ansatte over en fem til ti-års periode
efter starten af programdeltagelsen.
Resultatet vedr. bruttofortjeneste er robust overfor ændringer i dataopsætning og
overfor at man bruger virksomheder, hvis ansøgning om støtte til finansiering af
deltagelsen i et Innovationskonsortium ikke blev imødekommet, som alternativ
kontrolgruppe. Resultatet vedr. beskæftigelsesvæksten viser sig derimod ikke at være
robust overfor at bruge denne alternative kontrolgruppe, og må dermed fortolkes
med større forsigtighed.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
8
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0009.png
>
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0010.png
1.
Introduction
>
This report has been prepared by the Centre for Economic and Business
Research (CEBR). It presents an analysis of the economic impact of
’Innovationskonsortieordningen’ (Innovation Consortium scheme, IC scheme) on
participating firms in terms of growth and value creation.
The report is a follow-up to an earlier CEBR analysis (FI, 2007 and FI, 2008) and
exploits the availability of more recent data, which allow following the participating
firms for another 3 years.
Although this analysis is an evaluation of a specific subsidy scheme, its results might
be of general interest, as schemes similar to the IC scheme have been implemented
in a number of countries. However, general knowledge of their effects which can be
integrated into cost-benefit analyses of these schemes is still rare.
2
The IC scheme is a Danish subsidy scheme granted by Rådet for Teknologi og
Innovation (The Danish Council for Technology and Innovation, RTI) in cooperation
with Forsknings- og Innovationsstyrelsen (Danish Agency for Science, Technology
and Innovation, FI).
ICs subsidise and facilitate cooperation between private firms and research
and knowledge institutions (see next section 2 of the report for a more detailed
description of the scheme). Cooperating institutions can apply for financial grants
at the RTI/FI, and the grants subsequently finance the expenses incurred by the
research and knowledge institutions whilst undertaking the cooperative project.
Typically grants amount to DKK7-15 mio (approx. €1-2 million).
The IC programme has existed since 1995 (until 2003 under the heading “Centre
Contracts”). Until 2003, 80 ICs covering 274 different firms (denoted participants
in the following) had been completed, representing total grant costs of DKK766
million (approx. €100million), which corresponds to DKK2.8million (approx.
€370,000) per firm.
This analysis follows 220 of these firms in a firm-register dataset that covers the
period up to (and including) the year 2008. We study firm level developments in two
success parameters: gross profit and employment.
3
2
See Schibany et al. (2004) for a study based on a similar Austrian subsidy scheme. Branstetter og Sakaki-
bara (2002) consider a similar Japanese scheme and Adams et al. (2003) analyse the effects of the coopera-
tion between private and public R&D for firms in the U.S.
3
We also take a look at firm closure as an additional success parameter. However, given that this is not
central to the analysis, the results of this exercise are reported in Appendix 3.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
10
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0011.png
>
While employment is simply defined as the number of employees in a given firm at
a given point in time, gross profit is defined as annual net sales subtracted annual
costs of variable inputs (raw materials, energy, intermediate goods purchases, etc.)
except labour costs. Gross profit is the most precise measure of the firm’s value
creation, but one should, of course, keep in mind that part of the firm’s total value
creation may be passed on to consumers, may be retained in the firm and increase its
value (of which there is no data available for this analysis), or may take the form of
positive externalities, such as knowledge and/or innovations, that benefits other firms
or society as such.
4
In this study, we consider (absolute and percentage wise) growth in gross profit and
the number of employees both before and after programme participation. In addition,
we also analyse the changes in the growth patterns in association with participating
in the programme.
Moreover, we identify a control group of firms that do not participate (non-
participants), but which are similar to the participants in terms of size, industry, and
region. Again, we can use firm-level data to calculate the changes in gross profit and
employment for the non-participants, allowing us to address the question of whether
participants have higher increases in growth than what would be expected on basis
of the growth patterns of non-participants.
Under the assumption that growth in gross profit and employment of participants
and non-participants would be equal in the absence of programme participation,
differences between the two groups of firms can be interpreted as the causal impact
of the programme on participating firms.
The results of this exercise can be summarized as follows: Of the firms that
participated in the IC scheme it appears that relatively small firms have experienced
a significant increase in (the growth of) gross profit and employment.
It is important to note, that the size and statistical significance of these potential
effects depend on the size of the firms under consideration. We, for example,
find positive potential gross profit effects that are significant at the five percent
significance level for firms with a gross profit below 150 million DKK (approx. €20)
the year before the programme. We also find potential employment effects for firms
with less than 150 employees in the year before the programme.
4
As a measure of knowledge creation, we could in principle also have considered firm-level patenting acti-
vity. No actual data on patenting activities were, however, available for this analysis.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
11
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0012.png
>
Finally, we also look at the survival rates of participant firms and compare these
with the survival rates of firms in the control group. Here, we find high survival rates
(most likely due to IC participants and their control counterparts being relatively
large) and no difference in the survival rates of participants and non-participants.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
12
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0013.png
2.
Description of the IC scheme
>
An innovation consortium is a flexible framework for collaboration between
companies, research institutions and non-profit advisory/knowledge dissemination
parties. An innovation consortium must consist of at least two companies that
participate throughout the entire project, one research institution and one advisory
and knowledge dissemination party. Additionally, an innovation consortium may
involve or attach other types of partners that are considered relevant to the project.
The consortiums’ collaboration should be based on a joint project aimed at
developing and bringing research based knowledge to maturity, so that it can form
the foundation for Danish companies’ innovation in the years to come.
The joint project should result in the completion of high-quality research relevant to
Danish companies. Furthermore, the project should ensure that the new knowledge
is converted into competences and services specifically aimed at companies, and
that the acquired knowledge is subsequently spread widely to the Danish business
community – including in particular small and medium-sized companies.
Any project initiated by the consortiums must comply with the following:
•
The project should have generic content and the results must be of relevance
to a wide group of companies.
•
The project should be at a high level of innovation and research.
•
The project should not have the character of product development for
individual companies.
•
The project should require close collaboration between the consortium
parties.
•
The project should have a duration of two to four years.
The role of companies in the consortiums is to ensure that the joint research
and development project is based on relevant development needs within Danish
companies. Consequently, the project theme should be of significance to the
participating companies’ business development. However, it should not take the form
of actual product development.
The company participation is also to ensure that the business community’s
knowledge and competences are utilised in the project. Therefore, the participating
companies should contribute knowledge and competences at a high level within the
project field.
The companies may be Danish or foreign (or both).
Over the period 1995-2003, 274 different firms have participated in an IC, but a
number of firms have participated more than once. On average there were approx. 40
firms starting to participate in an IC per year, but there are large differences across
years, with the years 1998-2000 being characterised by the highest activity with on
average almost 70 firms starting to participate.
Approx. 50 percent of all participating firms are in manufacturing, 25 percent are in
financial or business services and 15 percent are in trade in services.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
13
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0014.png
3.
Data
The data for this analysis comes from three sources:
>
1. Data on program participants, which were assembled by CEBR based on FI’s
(paper) file records of the IC-programme for an earlier analysis (Forsknings-
og Innovationsstyrelsen, 2007 and 2008). These data will in the following be
called ’IC data’.
2.
Data from the private information provider company Købmandsstandens
Oplysningsbureau, now Experian A/S. This dataset, henceforth denoted
as the KOB data, has information from the financial reports that firms
of a certain size and ownership structure are obliged to file at the public
authorities. Thus, there are typically a number of observations for a given
firm (one for each annual account), denoted firm-year observations in the
following.
3. Information on firm transitions (e.g., mergers, liquidations or bankruptcies)
are included from the ‘cvr-register’ of the Danish Commerce and Companies
Agency (Erhvervs- og Selskabsstyrelsen). These data will be put to use when
we analyse survival probabilities of participating firms.
Note the KOB data provide information on a host of accounting-related variables,
including employment and gross profit. Note that only large firms are obliged to
file information about sales. This would make sales growth a potentially skewed
indicator of the IC impact upon firms.
The KOB data include firm-level information about industry and geographical
location, which will be exploited later when we identify a control group for the
empirical analysis.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
14
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0015.png
4.
Sampling
>
There are a total of 405 firm observations in the IC data over the period 1995-2003.
These belong to firms that participated in one of the programmes which go under the
umbrella ’Innovationskonsortier’. For 19 observations, it was not possible to identify
the point in time when the project was started, for another 35 observations it was not
possible to find firm-identification numbers which were necessary to match the IC
data with the KOB data (leaving us with 351 observations).
A number of firms are registered more than once in the data, because they have
participated more than once in the programme. We treat participation as a zero/one
variable, independently of how many times a firm has participated, and consider the
earliest time a firm is registered as participating as the starting point of programme
participation. This leaves us with 274 firm observations.
For 20 of these firms, there is no information in the KOB database, which leaves us
with 254 observations, and for 34, there is no accounting information in the KOB
data before the start of the program. This information, however, is necessary for the
before-after estimation set-up employed in the following. So we are left with 220
participant firms for the analysis.
As in any firm accounting database, considerable variation can be observed in
the KOB-data, which owes itself to some firms being part of corporate groups,
organizational changes and/or because firms change accounting policies and
practices. We treat this issue differently depending on the stage of the analysis,
which is, basically, divided in two steps:
As a first step, we identify a control group of comparison firms. In this step, we
will exploit the total universe of firms available in the data, independent of missing
observations or zero reporting.
As a second step, we compare the performance of participant firms with the
performance of firms in the control group. In this step, there is a need to make
decisions of how to treat the data in case of missing values in the data or when firms
report zero activity. This will also direct our robustness checks of the results of the
analyses. In this context, we will commit to one sampling strategy, and subsequently
check the robustness of the results when changing the strategy.
In essence, we want to analyse samples that are as ‘clean’ as possible, i.e.,
concentrate on firms which report regularly, and which do not raise suspicions of
significant organizational or accounting issues. By implication:
(a) When analysing gross profit we consider the 61 percent of firms that do not
report zero gross profit in the KOB database. The argument being that, if
there is any economic activity, zero gross profit is an event having (almost)
zero probability, indicating non-reporting rather than gross profit being zero.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
15
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0016.png
>
(b) When analyzing employment growth, we consider the 62 percent of all firms
that report a strictly positive number of employees in the KOB data.
Our sampling scheme implies that we start the analysis with the cleanest data
possible. Obviously, robustness checks will address whether these decisions are
critical for results. By implication, the sensitivity of results with regard to the rather
restrictive sampling scheme will be addressed subsequent to the presentation of the
performance comparisons.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
16
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0017.png
5.
Identification of the control group
>
To identify the control group of this analysis, we use a ’matching-on-observables’
technique, which can be seen as the workhorse of programme evaluation (see, for
example, Woolridge, 2002).
According to this method a control is identified for each firm participating in the
scheme. Except for not having participated in the programme, the controls are
selected to be as similar as possible to the given participant firm before programme
participation. In the following, these comparison firms will be called ’control firms’
or just ’controls’.
The details of the identification process are described in Appendix 1 of this report.
At this place it may be sufficient to note that, in the latter analysis, we will compare
developments in gross profit and employment over time of two highly similar groups
of firms, one which consists of the programme participants, the other of the controls
(non-participants).
Note also that the selection of highly similar controls increases the realism of the
assumption that participants and controls would have had similar developments in
gross profit and employment in the absence of the programme. By this, differences
in the developments can be interpreted as the programme’s genuine causal effect.
The selection of highly similar controls is an improvement of CEBR’s earlier
analysis (Forsknings- og Innovationsstyrelsen, 2007 and 2008), which simply uses
private sector firms for comparison purposes.
For participants, we will also compare the growth in employment and gross
profit in the time period before participating in the IC scheme with the growth in
employment and gross profit in the years after having participated. The cut-off year
which separates the pre-participation period from the after-participation period is
the year just prior to participation. This year will in the following be denoted the
‘base year’.
For controls, we also define a base year, which now refers to the year the given
control was selected. This is the year in which it most closely resembled one of the
participants in its base year. So we can also compare controls’ growth in gross profit
and employment between before and after the base year.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
17
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0018.png
6.
Estimation set-up
>
In the analysis, we will consider the growth of any of the two success parameters
(gross profit and employment) before and after the base year, where growth will be
measured both as absolute and percentage wise annual increases. We will analyse
changes in growth between before and after the base year, and will compare these
changes between participants and controls.
E.g., when growth accelerates after the base year for participants, but not for
controls, this indicates positive programme effects. The acceleration is interpreted
as the programme’s causal effect for participating firms under the (‘identifying’)
assumption that participants’ growth would accelerate by just as much as the
controls in the absence of the programme.
Note this set-up further improves upon the method employed in CEBR’s earlier
evaluation (Forsknings- og Innovationsstyrelsen, 2007 and 2008). Here, the
evaluation was based on a comparison of the levels of participant’s and control’s
success parameters and, thus, addressed the question whether participants had
grown faster than non-participants.
This methodology could not take into account the possibility that participant
firms might generally have higher growth independent of whether they decide to
participate in the programme or not. Any inherent growth difference between
participants and controls, however, should show in the years before the base year
and, thus, can be controlled for in the present analysis.
This is achieved by no longer comparing pre-participation levels of success
parameters with post-participation levels. Instead, we compare pre-participation
growth (or increases) with post-participation growth (or increases). So the evaluation
is based on participant-control differences in the acceleration of growth, rather than
just growth differences. This allows taking account of innate growth differences that
can be measured before programme participation (for participants) or before the base
year.
In analyzing growth developments, we will in the following consider both absolute
and (approximately) percentage wise changes in the growth in gross profit and
employment – the latter being measured by increases in the logarithms of these two
success parameters.
There are good reasons for analyzing both absolute and relative changes in firm level
growth. Considering absolute increases allows us to make statements in absolute
terms, (e.g., ‘ICs increase participants’ gross profit by on average XYZ DKK’) which
can be integrated into cost-benefit analyses, whilst inclusion of relative (percentage-
wise) growth gives greater weight to smaller firms in case of absolute programme
effects being larger for larger firms. If, for example, the programme is assumed to
have a proportional effect on growth rather than increasing gross profit by the same
amount for all participants independent of their size, then the analysis of relative
growth will allow us to estimate these proportional effects.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
18
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0019.png
>
However, it should be noted at this point that percentage wise growth can only be
measured for those firms that have positive gross profit (or nonzero employees) in
the base year and the years to follow. By implication, analyzing growth instead of
increases necessarily restricts the sample to these firms.
The estimation set-up of potential programme effects is explained in greater detail
in Appendix 2 of this report. At this point, it should, however, be noted that we only
look at firms, which we can follow for at least three years before the base year and
five (in a second iteration: ten) years after the base year.
When considering absolute increases instead of percentage wise growth, we
calculate the average of each firm’s annual increases in the success parameter in the
three year period before the base year. This defines a firm’s average annual increase
in the pre-base-year period. Also, we calculate for each firm the average annual
increase in the success parameter in the five (ten) year period after the base year,
which defines the firm’s average increase in the after-base year period.
This implies that we have two observations for each firm: one describing average
increases before the base year, and one describing average increases after the base
year. As a result, we can calculate for each firm, whether average annual increases
have become larger or smaller in association with passing the base year. In other
words, we can evaluate the development of average annual increases.
So this study’s performance analysis takes a look at participants’ average increases
in the average annual increases in association with participating in an IC, and
compares them with the average increases of the annual increases for controls
following their assigned base year. If the increase in the average annual increases
is larger for participants than controls, this implies that there is a more positive
change in growth developments of participants than controls. Thus, the comparison
estimates the potential effect of the IC programme on the participating firms.
In this case, any differences in the increase of annual growth can not be interpreted
as the result of different pre-base year developments, nor can it be explained by
reference to differences in the two group’s characteristics given the similarity of
participants and controls (and given that we additionally include some control
variables in the models to take account of potentially remaining differences).
In short, this approach makes it more likely that positive differences between
participants and controls must be attributed their participation in IC schemes.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
19
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0020.png
7.
Descriptive Statistics
>
The above-mentioned identification procedure yields 439 control observations
belonging to 334 different firms, implying that repeated observations occur for a
number of control firms.
To interpret the results of the following analysis as measuring the impact of the
programme, one needs to assume that participants and controls would experience
the same changes in growth if not it was for the programme. It can be argued that
this assumption becomes increasingly realistic the more similar the participants and
controls are in terms of their observable characteristics. Hence, we have sought to
identify a highly comparable group of controls.
TABLE 1 illustrates how successful we were at identifying a group of controls that
is highly similar to the group of participants by describing both groups of firms in
the base year.
We find that the distribution across industries is highly similar for the two groups,
but that there are differences w.r.t. the mean size of the participants vs. controls.
Also, participants are slightly more concentrated in the Copenhagen area (zip codes
below 2999). The size difference between participants vs. controls owes itself to the
fact that some participating firms belong to the biggest firms in Denmark, for which
it is not possible to find controls of similar size.
Although we of course will test differences in the success parameters between
all participants and controls, any effects of the programme might in this case be
undetectable as they may be washed out by the large variations in the success
parameters in large firms for reasons outside the statistical models.
As a consequence, we will analyse different samples distinguished by the maximum
size of the firms under consideration. As a starting point, we consider small and
medium size firms separately. More specifically, employment growth will be
analysed separately for firms below 300 employees in the base year. Growth profit
will be analysed separately for firms with gross profit less than 150 million DKK
(approx. €20 million) in the base year. Although it may appear restrictive, these
thresholds imply that the resulting samples still represent approx. 75 per cent of all
observations, reflecting the large share of SMEs in Denmark.
For these subgroups of firms, expected unobserved heterogeneity is smaller and,
thus, the power of the analysis’ statistical tests (i.e., the probability of finding
effects in case there are any) is larger compared to the sample where large firms
are included. Also, participants and controls are more similar in their observable
characteristics, which increases the realism of the ‘identifying’ assumption that,
in the absence of the programme, growth developments would be similar for
participants and controls.
Please note that the chosen size thresholds are completely arbitrary, and constitute a
compromise between being representative for the entire population on the one hand
and the desired robustness of findings and the realism of the identifying assumption
on the other. Note, moreover, that the thresholds can be moved easily, and we will do
so to test how this affects analyses and results.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
20
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0021.png
>
TABLE 1. Mean values of key variables for participants and controls in the base
year
All firms
Participants
Number of employees
Gross profit (1000DKK)
Industries (shares of total)
Agriculture
Construction
Electricity
Finance, business service
Manufacturing
Trade, hotels, restaurants
Transport, telecom
Services
Not stated
Region (zip codes)
1000-2999
3000-3999
4000-4999
5000-5999
6000-6999
7000-7999
8000-8999
9000-9999
0,455
0,068
0,064
0,041
0,077
0,077
0,159
0,059
0,380
0,068
0,064
0,046
0,109
0,093
0,169
0,071
0,005
0,036
0,009
0,255
0,500
0,155
0,014
0,014
0,014
0,005
0,039
0,007
0,260
0,499
0,155
0,009
0,014
0,014
612
364.271
Controls
279
145.825
Firms with less than
300 employees in the
base year
Participants
83
51.439
Controls
87
42.554
0,000
0,030
0,007
0,289
0,452
0,178
0,015
0,015
0,015
0,003
0,027
0,009
0,247
0,509
0,165
0,012
0,015
0,012
0,444
0,089
0,059
0,037
0,074
0,096
0,141
0,059
0,363
0,061
0,064
0,046
0,091
0,110
0,189
0,076
21
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0022.png
>
For the sub-sample of firms with less than 300 employees in the base year, we find
the difference in the number of employees in the base year between participants and
controls to be within the ‘natural’ statistical variation, i.e., not significantly different
from each other at any commonly used significance level. However, participants
remain overrepresented in the Copenhagen area (a finding which is significant at the
10% significance level), and have higher gross profit in the base year (significant at
the 10% level).
For the sub-sample of firms with gross profit less than 150 million DKK in the
base year, we find gross profit (and the number of employees) in the base year to
be not significantly different between participants and controls at any commonly
used significance level. However, participants remain again overrepresented in the
Copenhagen area (significant at the 10% significance level).
In total, there are 10,167 firm-year observations belonging to 554 different firms.
Note here that the same control firm may occur repeatedly in the data, if more
than one of its firm-year observations were selected by the procedure outlined in
Appendix 1.
It should be noted here that there are only relatively few observations that enable us
to follow firms long before and long after the base year: only firms that participated
early in the programme or the controls associated with these firms can be followed
over a long time period after having participated or selected as controls.
This is, for example, reflected in the fact that there are only 15 observations with
employee information available ten years before base year. There are, however,
106 observations where data is available eight years before the base year, and 275
observations where data is available five years prior to the base year. Five years after
base year we have information on 340 firms, whilst 178 firms remain in the database
ten years after base year.
Only part of this attrition is due to firms leaving the data before the end of the
observation period: Of the 554 firms in the final sample, approx. 25 per cent leave
the data before 2008.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
22
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0023.png
8.
Results
>
The following considers developments of employment and gross profit over time, and
compares these developments between participants and controls. This performance
analysis is split up into two parts:
1.
The first part of the performance analysis is based on two subsamples
excluding large firms: one, in which gross profit is below DKK 150
million in the base year, and another one, in which the number of employees
is below 300 employees in the base year.
2. The second part of the performance analysis is based on a set of alternative
samples and extends (and checks the robustness of) the previous results.
Results are reported in section 9 of this report.
Choosing sub-samples of relatively small firms as the point of departure for the
performance analysis, instead of the total sample, is motivated by the following
reasons:
First, we have difficulties finding highly similar controls for large participants such
as, for example, multinationals in specific industries of which there are only a few
in Denmark. As result, the assumption necessary to identify causal effects of the
programme, which is that firms in both groups would change their growth patterns
in the same way in the base year if not it was for the presence of the programme -
can be argued to be more realistic for a sub-sample of small and medium size firms
than in a sample that include the few, very large companies.
Second, we find that results for this subgroup are well-suited to illustrate the
estimation technique employed to answer the question of whether findings should
be interpreted as being the result of underlying processes (in which case they are
‘statistically significant’) or just ‘coincidental’.
Still, as mentioned already, the chosen cut-offs are, of course, arbitrary. Hence, the
robustness of findings with respect to changing the thresholds will be discussed in
section 9.
A last point to mention here is that we will depart from only analyzing firms that
always report nonzero and non-missing information. Again, we will subsequently
check whether these strict sampling conditions are critical for the results.
8.1
Gross profit developments
After these introductory remarks, we are now ready to take a look at the numbers.
A graphical depiction of the absolute differences in gross profit is displayed in
FIGURE 1:
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
23
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0024.png
>
FIGURE 1: Gross profit (in DKK1,000). Mean differences compared to base year.
Firms with gross profit less than DKK150 million in the base year. 3-year moving
averages
80000
60000
40000
20000
0
-20000
-40000
-10 -8 -6 -4 -2 0 2
4
6 8 10 12 14
Controls
Participants
Years after the base year
We find similar increases in gross profit for participants and controls in the years
before the base year. This suggests absence of any inherent differences in gross
profit growth between the two groups of firms, which also indicates that the
matching procedure succeeded in finding a group of controls of similar inherent
growth compared to the group of participants.
After the base year, the gaps between the graphs widen, with participants having
larger increases in gross profit compared to the controls. Under the assumption that
participants and controls would have continued their pre-participation (pre-base-
year) growth patterns in the absence of the programme or would have changed their
growth patterns in the same fashion, the higher increase in the group of participants
measure positive effects of the programme on participants’ employment and gross
profit.
We will have a closer look at the size of the differences between participants’ and
controls’ growth patterns in a more formal treatment below. For now, we may note
that, if pre-base-year trends are indeed equal, the graphs suggest participation in an
ICs to have a gross profit effect of approx. DKK13,4 million five years and approx.
DKK15,4 million ten years after the base year.
Obviously, a next step is to establish evidence on whether or not the finding of
diverging growth trends is statistically significant, i.e., the result of underlying
mechanisms, or just incidental and within the statistical variation which must be
expected for firm data typically being characterized by large variations.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
24
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0025.png
>
However, before addressing this issue, some minor remarks regarding FIGURE 1
(and those figures to follow) might be in place: Note that firms, to be observable
long after the base year, need to have participated or have to be selected as controls
at the time of the start of the programme in the mid-nineties, and must not have left
the data before the end of the observation period. Also, to be observable long before
the base year, firms need to have started to participate or been selected late in the
observation period, and need to have existed long before the base year.
As a result, there exist only a limited number of observations long before and after
the base year, implying that findings based on these observations get increasingly
tentative at the left and the right sides of the figures.
5
Also, when determining
(linear) growth trends before and after the base year, observations long before and
after the base year are given a higher weight, so firms with high or low growth
have a higher leverage on trend estimates when being observable for extended time
periods.
Note also that observations long after the base year belong to the same cohort or
nearby cohorts, and findings for these observations may be due to business cycle
effects - which does not matter for the results of the analysis unless business cycles
affect participants and controls in different ways.
To establish evidence on whether or not the above differences in the two groups’
growth patterns are statistically significant, i.e., too large compared to the general
variation in the data to be considered coincidental, we employ the regression model
as described in section 6 and Appendix 2. Results for the changes in gross profit
developments in association with programme participation relative to the changes in
the group of controls are summarized in TABLE 2A and TABLE 2B:
5
Of course, one could right-censor the graphs at, say, ten years after the base year to avoid that large varia-
tion at the end of the observation period steals the picture. This would, however, be highly arbitrary and even
manipulating, leading us to present results for the entire observation period independently of the number of
observations long before and long after the base year.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
25
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0026.png
>
TABLE 2A: Growth in gross profit up to five years after the base year: Diff-in-
diff regression results
Model 1: Dependent
variable: Average annual
increase in gross profit in
either the three-period up
to the base year or in the
five-year period after the
base year
Coefficient
Constant term k
Observation is after base
year, d1
-1914,5
-2425,2***
Standard
error
2.496,4
681,9
Model 2: Dependent
variable: Average annual
growth in gross profit in
either the three-period up
to the base year or in the
five-year period after the
base year
Coefficient
0,100
-0,086**
Standard
error
0,206
0,035
Observation belongs to a
participant, d2
Observation belongs to a
participant and is after the
base year, d1d2
-846,8
1.090,9
-0,066
0,055
3668,9**
1.738,3
0,145**
0,067
R2=0.13
517 observations
R2=0.30
510 observations
Notes: *** significant at 1%. ** significant at 5%, * significant at 10%; only firm observations with posi-
tive gross profit are used in Model 2; gross profit is measured in DKK1000. The following set of controls
was included in the regressions: Seven industry dummy variables, eight dummy variables for the firms’
geographical regions, three calendar time dummy variables for when the firm has its base-year, and six
dummy variables describing the firm’s gross profit in the base year. Base category: firms in manufacturing
industries, with gross profit 0-500 million DKK in the base year and zip-code
<3000.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
26
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0027.png
>
TABLE 2B: Growth in gross profit up to ten years after the base year:
Diff-in-diff regression results
Model 1: Dependent
variable: Average annual
increase in gross profit in
either the three-period up
to the base year or in the
ten-year period after the
base year
Coefficient
Constant term k
Observation is after base
year, d1
138,5
-696,5
Standard
error
1.889,7
827,5
Model 2: Dependent
variable: Average annual
growth in gross profit in
either the three-period up
to the base year or in the
ten-year period after the
base year
Coefficient
0,399***
-0,086***
Standard
error
0,093
0,018
Observation belongs to a
participant, d2
Observation belongs to a
participant and is after the
base year, d1d2
-940,9
1.030,4
-0,030
0,038
1.981,7
1.916,7
0,121 **
0,058
R2=0.17
399 observations
R2=0.38
390 observations
Notes: *** significant at 1%. ** significant at 5%, * significant at 10%; only firm observations with posi-
tive gross profit are used in Model 2; gross profit is measured in DKK1000. The following set of controls
was included in the regressions: Seven industry dummy variables, eight dummy variables for the firms’
geographical regions, three calendar time dummy variables for when the firm has its base-year, and six
dummy variables describing the firm’s gross profit in the base year. Base category: firms in manufacturing
industries, with gross profit 0-500 million DKK in the base year and zip-code
<3000.
The coefficient estimates presented in the tables have the following interpretations:
-
the constant term k estimates the average annual increases for a specific
subgroup of controls (in this case controls in manufacturing, with gross profit
between zero and 500 million DKK and with zip code less than 3000) before
the base year,
the coefficient associated with d1 estimates the difference in the average
annual increases (or the increase in annual growth) for all controls between
before and after the base year,
-
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
27
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0028.png
>
-
the coefficient associated with d2 estimates the difference between the
average annual increases (growth) between participants and controls before
the base year.
The coefficient associated with d1d2 estimates the difference in the increases
of the average annual increases (growth) between participants and controls.
-
To illustrate, consider the case where we follow firms over ten years (TABLE 2B).
After the base year, the average annual increase in the gross profit of controls is
DKK 696,500 (approx. €95,000) lower than before the base year. Participants’
average annual increase in the years before the base year is DKK 940,100 (approx.
€130,000) lower than the controls’. Finally, the difference in the increases of the
annual average increases between participants and controls is found to be 1,981,700
DKK (approx. €260,000). As a result, the average annual increase of the gross profit
of participants in association with participating in the IC programme exceeds the
controls’ increases by almost two million DKK in the ten-year period after the base
year.
Turning to TABLE 2A, we find that the average annual gross profit increase for
participants over the first five years after the base year is approx. 3.6 million DKK
higher (and statistically significant at the 5% level) compared to what would be
expected in absence of participation in the IC scheme.
Looking at relative change (average annual logarithmic differences translating
interpreted as average annual percentage wise growth), we find that the average
annual growth in gross profit for participants over the first five years after the base
year is approx. 15 per cent higher compared to what would be expected in absence
of participation in the IC scheme.
Note the percentage-wise growth difference gets smaller when one only considers
firms above a certain size in the base year. E.g. when only considering firms with
gross profit above 50 million DKK in the base year, the estimated average annual
growth difference goes down to approx. eight percent but remains to be statistically
significant at the 10% level. Hence, we can conclude that the positive differences in
the growth of gross profit is not (exclusively) driven by very small firms.
Over a ten-year period, the average annual increase in excess of what would be
expected in absence of the programme for participation is (as noted earlier) approx.
two million DKK, and growth is approx. 12 per cent higher than in the absence of
the programme.
In summary, our findings agree with the presence of considerable effects of the IC
programme on participants’ increases on gross profit. Findings for both absolute and
logarithmic differences are statistically significant at the 5% significance level for
firms followed over the first five years after the base year and significant at the five
per cent level for percentage-wise increases for those firms which are able to follow
for at least ten years after the base year.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
28
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0029.png
>
The finding of estimated differences in average increases being 3.6 million
DKK when one follows firms over five, and approx. two million DKK when one
follows them over ten years might indicate that differences in absolute increases
are largest in the years directly following the start participation (the base year).
8.2
Employment developments
The analysis of employment developments follows the blueprint of the previous
subsection. We, however, depart from focusing on firms with less than 300
employees in the base year, which always report nonzero and non-missing
employment information, and leave the consideration of different samples to the
next section of this report.
Firms under 300 employees in the base year represent approx. 75 percent of
the total sample of firms, and approx. 71 percent of the present sample of firms
which never report missing or zero employment information.
For each firm, we consider average annual increases and annual average growth
rates for the three-year-period before the base year and the time period between
the base year and five years later. As a second step, we follow the firms for
ten years after the base year. Again, we only consider firms that always report
nonzero employees, which considerably reduce the number of observations, and
leave relaxing this strict sampling condition for later.
When taking a look at the average employment differences between a given year
after the base year and the base year in FIGURE 2, we do find IC participants
to have slightly higher growth in the first years after the base year. When
following firms for more than eight years, the picture changes: participants
have considerable lower growth eight to twelve years after the base year. But
when following (a greatly reduced number of) firms for more than 12 years,
we find that those controls which can be followed for so long have experienced
considerably lower growth than the corresponding participants. Again, the end of
the curves should be interpreted with caution.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
29
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0030.png
>
FIGURE 2: Number of employees. Mean difference compared to base year. Firms
with less than 300 employees in base year. 3-year moving averages
80
60
40
20
0
-20
-40
-10 -8 -6 -4 -2
0
2
4
6
8 10 12 14
Participants
Controls
Years after the base year
FIGURE 2 does not suggest robust positive IC programme effects, although one
might notice that growth accelerates for participants but not controls in the years
close to the base year.
We estimate the same statistical model to substantiate the findings suggested by
FIGURE 2, and present the results of this exercise in TABLE 3A and 3B:
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
30
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0031.png
>
TABLE 3A: Employment growth up to five years after the base year: Diff-in-diff
regression results
Model 1: Dependent
variable: Average annual
employment increase in
either the three-period up
to the base year or in the
five-year period after the
base year
Coefficient
Constant term k
Observation is after base
year, d1
-1,33
-1,96
Standard
error
2,37
1,48
Model 2: Dependent
variable: Average annual
employment growth in
either the three-period up
to the base year or in the
five-year period after the
base year
Coefficient
0,100*
-0,080***
Standard
error
0,051
0,019
Observation belongs to a
participant, d2
Observation belongs to a
participant and is after the
base year, d1d2
-2,00
2,32
-0,019
0,032
5,12
3,70
0,061
0,039
R2=0.067
495 observations
R2=0.088
495 observations
Notes: *** significant at 1%. ** significant at 5%, * significant at 10%. The following set of controls was
included in the regressions: Seven industry dummy variables, eight dummy variables for the firms’
geographical regions, three calendar time dummy variables for when the firm has its base-year, and four
dummy variables describing employment in the base year. Base category: firms in manufacturing indus-
tries, with 5-10 employees in the base year and zip-code
<3000.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
31
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0032.png
>
TABLE 3B: Employment growth up to ten years after the base year:
Diff-in-diff regression results
Model 1: Dependent
variable: Average annual
employment increase in
either the three-period up
to the base year or in the
ten-year period after the
base year
Coefficient
Constant term k
Observation is after base
year, d1
3,7
1,3
Standard
error
2,6
3,3
Model 2: Dependent
variable: Average annual
employment growth in
either the three-period up
to the base year or in the
ten-year period after the
base year
Coefficient
0,071**
-0,084***
Standard
error
0,036
0,022
Observation belongs to a
participant, d2
Observation belongs to a
participant and is after the
base year, d1d2
-2,4
2,3
-0,021
0,032
-1,6
4,1
0,023
0,039
R2=0.08
389 observations
R2=0.10
389 observations
Notes: *** significant at 1%. ** significant at 5%, * significant at 10%. The following set of controls was
included in the regressions: Seven industry dummy variables, eight dummy variables for the firms’
geographical regions, three calendar time dummy variables for when the firm has its base-year, and four
dummy variables describing employment in the base year. Base category: firms in manufacturing indus-
tries, with 5-10 employees in the base year and zip-code
<3000.
Results of the statistical analysis confirm the findings of FIGURE 2: in the group
of firms which can be followed over five years, participants increased employment
by five additional employees per year, and had 6 percent (not to be confused with
percentage points) higher growth. These results have t-probabilities of 17% (for
absolute increases) and 13% (for percentual growth). This means that the probability
of being wrong when stating that participation in an IC’s generally increases
employment growth is 17% and 13%, respectively.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
32
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0033.png
>
We suggest interpreting this result as follows: there are positive relationships
between growth and programme participation. However, the probability of these
relationships being coincidental is too high to claim that there exist underlying
mechanisms implying that these positive relationships are a general feature of
participation in an IC.
Also, results are not robust to following firms for time periods of different lengths.
For those firms which can be followed over at least ten years, participants have on
average had lower increases in the number of employees than controls, which further
advise us to be careful with regards to statements regarding general employment
effects of ICs.
These results for the ten-year period are again not significant. We conclude that – at
least for this sample of firms with up to 300 employees in the base year - it is not
possible to find relationships which are strong enough to claim that ICs generally
have positive employment growth effects.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
33
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0034.png
9
Alternative samples and robustness
>
In this section, we report the results of the above-described model for alternative
samples distinguished by the size of the firms under consideration and their
sampling criteria. Also, we will run separate regressions for firms in service
industries. Finally, we use another group of firms as a control group than before,
which consists of firms that have applied for funding of an IC, but found their
applications being rejected by FI.
To take a look at firms in the service sector is motivated by FI’s special interest in
service industries as a potential growth industry. There are of course large inherent
differences between firms in the service sector, and it is tempting to differentiate
between knowledge intensive and less knowledge intensive industries. We, however,
came to the conclusion that we will not distinguish firms along these lines. First,
because we have too of few observations in the service sector, and second, because
it is difficult to argue that knowledge is not relevant for the service firms that
participate in a collaboration which aims to enhance innovation activity.
Thus, when considering firms in services industries, we analyse on all firms coded
65-97 in the Danish standard industry classification (db93), which covers firms
which according to db93 are firms in “Financial and business services” and just
“Services”.
In this section, we also address the robustness of the results with regards to including
firms that report irregularly. Especially when considering employment developments
in the previous section, focusing on clean data implied that we lost a relatively large
share of firms which report zero employees in single years.
Firms may grow by hiring new employees, or by integrating organizational units
from other firms in the same corporate group, e.g., merging a holding company
(with no employees) with its operating company (with employees), by acquisitions
or organisational reshuffle within corporate groups. Focusing on clean data might
be assumed to reduce the impact of the latter explanations, but it is still relevant to
check whether this is critical with regards to the results of the analysis.
Finally, we exploit the data that CEBR has collected for the earlier study (FI, 2008)
on 133 firms that applied for funding before 2003, but did not receive it (and did
not receive funding later on). These firms, denoted ‘rejected firms’ in the following,
are equal to the participants with respect to the fact that they have applied, but the
fact that their project was declined funding indicates lower quality projects or lower
quality applications (which again may be correlated to firm characteristics that also
are related to the firm’s growth potential).
These problems notwithstanding, using this alternative control group for an
additional robustness check makes sense, as the potential finding of rejected firms
doing just as well as participants would advise us not to interpret earlier finding as
the programme’s causal effect.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
34
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0035.png
>
To implement the comparison of participants and rejected firms, we define the year
of the application as the base year for comparisons. For participants, this is typically
the year preceding the start of the project. The relatively low number of rejected
firms implies that for this robustness check, we only follow firms over a five-year
period.
In the exposition of the results of the various robustness checks, we only report the
relevant coefficient estimate which is associated with the indicator (dummy) variable
“Observation belongs to a participant and is after the base year, d1d2”. Recall that
this parameter estimates the deviation between actual post-participation average
annual increases for participants and the increases which would be expected in
the absence of participation. Under the identifying assumptions, this coefficient
estimates the effect of the programme on participating firms.
Results of the different regressions are summarised in TABLE 4. In this table, we
report t-statistics, which are the probabilities of being wrong when stating that there
are non-zero underlying relationships in the data. E.g., the probability of being
wrong with the claim “Firms that report gross profit less than 75 million DKK in
the base year and always report nonzero gross profit experience a different average
annual increase in gross profit (compared to controls) in the first five years after the
base year” has a 6% estimated probability of being wrong.
The following sums up the result of the different regressions:
(a) We find that no potential programme effects can be identified when
considering the total sample of all firms. This comes as no surprise, as there
are large players among participants with gross profit (and large variations
in gross profit) being orders of magnitude too large to potentially allow us to
find any impact of the programme. The large variation in gross profit in the
total sample superimposes any potential (in this case relatively small scale)
effects of the programme.
(b) However, looking at smaller firms with gross profit below 75 million in the
base year corroborates the picture of significant higher gross profit growth
for participating firms after the programme and relative to pre-programme
growth, relative to the developments of firms in the matched control group,
and taking account of potential differences in observable factors between the
participant and the control firms.
(c) We cannot find relationships for service sector firms, which might be because
we have too few observations in this sector to allow identifying relationships
of any degree of reliability.
(d) We find the strictness of the sampling conditions with regards to whether
or not to sample firms that sometimes report zero activity or have missing
values not having any effect on the general results.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
35
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0036.png
>
TABLE 4: Regression estimates of the parameter of “Observation belongs to a
participant and is after the base year, d1d2” for alternative samples. Dependent
variable: Average annual increase in gross profit (in DKK1000).
Firms that report regularly (i.e., always report nonzero and non-missing gross profit):
Sample
Observation
period (in
years)
5
10
Parameter
estimate
t-probability
Number of
observations
All firms
-2967
-3178
0,380
0,256
651
505
Firms that report gross
profit less than 75
million DKK in the base
year
Firms that report gross
profit less than 150
million DKK in the base
year in the service
sector
5
10
6035***
3491*
0,001
0,060
394
305
5
10
-576
-3302
0,820
0,383
86
57
Firms that occasionally report zero gross profit or have occasionally missing gross profit
information:
Firms that report gross
profit less than 150
million DKK in base
year
5
10
4888*
8331
0,050
0,173
696
552
Alternative control group: Rejected firms:
Firms that report
gross profit less than
150 million DKK in
base year, and always
nonzero and non-
missing gross profit
5
4710**
0,036
233
*** significant at 1%. ** significant at 5%, * significant at 10%; the estimations include the same controls
as specified in TABLE 2.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
36
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0037.png
>
(e) Also, participants grow faster after the start of participation compared to the
alternative control group consisting of firms the applications for funding were
rejected. So the necessary condition for giving previous findings a causal
interpretation is fulfilled. The size of the potential effect of this comparison
is similar to the previous findings (an approx. DKK4.5 vs. DKK3.7 million
difference in average annual increases in gross profit).
The general conclusion is that there are stable differences in gross profit
developments between participants and controls after the base year for up to medium
size firms – differences that cannot be easily explained by other factors than IC
programme participation.
We turn now to employment developments, and summarise results in TABLE 5:
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
37
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0038.png
>
TABLE 5: Regression estimates of the parameter of “Observation belongs to a
participant and is after the base year, d1d2” for alternative samples. Dependent
variable: Average annual increase in the number of employees.
Firms that report regularly (i.e., always report nonzero or nonmissing number of employees):
Sample
Observation
period (in
years)
5
10
5
10
5
10
5
10
5
10
Parameter
estimate
t-proba-
bility
Number of
obser-
vations
703
560
325
274
206
145
95
66
87
58
All firms
-2,8
-11,0
11,2**
4,6
11,9**
9,2*
22,0
5,2
24,0*
11,2
0,794
0,288
0,016
0,179
0,028
0,084
0,104
0,635
0,392
0,392
Firms that have less than 150
employees in the base year
Firms that have less than 75
employees in the base year
Firms that that have less than 300
employees in the base year in the
service sector
Firms that that have less than 150
employees in the base year in the
service sector
Firms that occasionally report having zero employees or have occasionally missing
employment information:
Firms that have less than 300
employees in the base year
Firms that have less than 150
employees in the base year
Alternative control group: Rejected firms:
Firms that have less than 150
employees in the base year
and always report nonzero or
nonmissing number of employees
5
10
5
10
6,7**
2,0
9,2***
4,1*
0,023
0,540
0,007
0,076
693
529
554
356
5
1,4
0,873
165
*** significant at 1%. ** significant at 5%, * significant at 10%; the estimations include the same controls
as listed in TABLE 3.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
38
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0039.png
>
In the case of employment growth, changing the sampling conditions reveals new
results: for participating firms of size below 150 employees in the base year, we find
large and statistically significant potential employment effects of ICs of about eleven
additional employees per year. One may note here that approx. 24 percent of all 220
participants have less than 50 employees and approx. 50 percent have less than 150
employees, so we find potential effects for the participants in the lower half of their
size distribution. We also find statistically significant effects for firms of size below
300 employees in the base year, when we include firms that report zero employees in
some years.
Again, we cannot find relationships when considering samples of all firms which
have participated in an IC – some of which have several thousand employees.
We find weakly significant positive potential employment effects for firms in the
service sector, which is remarkable given the relatively small size of this sample
(less than 100). We advise not to take the large potential effect of annually 24
additional employees at face value. The combination of large heterogeneity and
relative few observations implies that this result is associated with a high level of
uncertainty.
Finally, we do not find potential employment effects when comparing participants
with the group of firms the project applications were rejected. The absence of any
significant result might be due to large variation in employment growth in the group
or rejected firms - in association with a relatively small number of observations.
However, it also implies that high employment growth in small participant firms
after the base year might not be so much an effect of IC participation, but might
instead be the result of strategic decisions correlated to applying to the programme,
and shared by participants and controls.
TABLE 4 and 5 only present a small but representative share of the robustness tests
undertaken for this analysis, but none of our alternative sampling or modelling
6
strategies have changed the general conclusion of there being positive potential
effects for the firms at the lower half of the total sample’s size distribution, which in
some cases even can be shown to be significant when considering long-run averages
over ten years after the base year.
6
This includes, for example, estimating the models with the inclusion of firm random effects – which is
possible because there are two observations per firm. This does, however, not change any of the previous
results. Also, random effects estimations of annual increases (instead of average annual increases over a
couple of years) on the panel of firm year observations give very similar results.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
39
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0040.png
10
Conclusions
>
This report summarises the results of an evaluation study of the IC programme. For
this purpose, we follow firms which have participated in the programme before and
after the start of participation, and analyse their developments with regards to gross
profit and employment.
This is possible, because firm information from FI regarding programme
participation has been merged with register data on key accounting variables that
firms are obliged to file at public authorities. We can find 220 firms that participated
out of 285 in total in the register data, and can follow 203 of these firms for at least
five years after they have started to participate in an IC.
For our analysis, it is natural to distinguish firms by their size. Some of the firms
that participate in the IC programme are very large, having gross profits of several
billion DKK and several thousand of employees. It would be unrealistically
optimistic to search for potential effects of the IC scheme in a group of firms in
which these large firms are included.
Hence, for the analysis, we consider firms that represent roughly the smallest 75
percent of all firms in the sample, and find positive potential gross profit effects of
programme participation – a finding which is based on a joint comparison of growth
patterns of participant firms and a highly similar group of comparison firms, in
which we correct for potential differences in inherent (pre-participation) growth
trends before the start of programme participation.
We find that participants have annual increases in gross profit in the first five years
after the start of participation, which are on average 3.7 million DKK above what
would be expected in the absence of programme participation. Under the assumption
that participants would have experienced the same developments in gross profit
growth as the controls in the absence of the programme, the additional 3.7 million
per year in the first five years after participation is the genuine effect of participating
in an IC.
Over a ten year-period, the average potential effect gross profit effect is smaller and
is approx. two million DKK per year, and is no longer statistical significant. An
obvious explanation might be that potential effects of the programme are realised
in the first years after starting to participate in the programme, so the average of the
annual increases over a period of time becomes smaller the longer the time period
under consideration.
If participants’ counterfactual growth in the absence of participating in the
programme is indeed appropriately measured by the growth of the controls, then the
most qualified guess of the programme’s effect is that it increases annual gross profit
per year of smaller firms by approx. DKK20 million over a five to ten year time
period after participation. It should, however, be noted that this number is associated
with statistical uncertainty, which advises us to be careful when making predictions
regarding future programme effects.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
40
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0041.png
>
It is difficult (and has not been part of the present analysis) to estimate what the
counterfactual behaviour of participants in the absence of the IC scheme might have
been. Maybe ICs are a means of helping to implement firms’ strategic decisions
and innovations, which are the true reasons of the positive developments, maybe
participants have higher growth than controls for reasons we could not observe in
the data and did not control for in this analysis.
Still, the back-on-the-envelope calculation resulting in a DKK20 million difference
in annual gross profit after five to ten years suggests that the programme is a success
even in case of only a share of this difference owing itself to a genuine causal effect
of the programme. Here, it could be noted that differences in annual gross profit
accumulate over time, implying substantial differences between participants’ and
control firms’ value creation when measured over several years.
We also consider employment developments and can again not find significant
results for the sample of firms where we include large firms. It is, however, possible
to demonstrate that smaller (in this case firms having less than 150 employees in the
year before participating in the programme) participants have an additional annual
employment growth of approx. eleven employees. This difference is statistically
significant at the 5% significance level, but the sum of the evidence advises us to be
careful to interpret it as a causal effect of the IC scheme.
We conclude that it comes as no surprise that we do not find potential programme
effects for those samples which include large firms. Instead, we find positive
potential effects of the programme on gross profit and employment for relatively
small firms, where we expected to have a chance of finding them in case of their
existence. The difficulty of finding potential effects for large firms is likely to be
due to a measurement issue, and should not been taken as evidence of ICs having no
effect for large firms.
Even though the present data at hand must be seen as favourable for this kind of
analysis, regularly updating them might in the future allow analyzing which firms
benefit more from participating in an IC than others, and which ICs work better than
others.
In the current case, it was for example not possible to make statements of any
reliability regarding the experiences of participant firms in the service sector.
Furthermore, we did not have data on the patenting activities of the participating
firms, but we expect this type of data to become available for subsequent analyses.
This additional information may be exploited for the identification of controls and
may also be used to directly estimate the effects of IC programme participation on
innovation output.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
41
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0042.png
11
References
>
Adams, James D., Eric P. Chiang and Jeffrey L. Jensen (2003): The Influence
of Federal Laboratory R&D on Industrial Research.
Review of Economics and
Statistics
85(4), pp. 1003-1020.
Bercovitz, J., M. Feldman, I. Feller and R. Burton (2001): Organizational Structure
as Determinants of Academic Patent and Licensing Behavior: An Exploratory Study
of Duke, John Hopkins and Penn State Universities.
Journal of Technology Transfer
26, pp. 21-35.
Blanes, J. Vicente and Isabel Busom (2004): Who Participates in R&D Subsidy
Programs? The Case of Spanish Manufacturing Firms.
Research Policy
33, pp. 1459-
1476.
Branstetter, Lee G. og Mariko Sakakibara (2002): When Do Research Consortia
Work Well and Why: Evidence from Japanese Panel Data.
American Economic
Review
92(1), pp. 143-159.
Erhvervsfremmestyrelsen (1998):
Centerkontrakter. Evaluering udarbejdet af Oxford
Research A/S for Erhvervsfremmestyrelsen.
København.
Erhvervsfremmestyrelsen (2001):
Evaluering af Centerkontraktordningen.
København.
Forsknings- og Innovationsstyrelsen (2007):
Effektmåling af Innovationskonsortier -
Data og metode.
København.
Forsknings- og Innovationsstyrelsen (2008):
Effektmåling af forsknings- og
innovationssamarbejder – fokus på innovationskonsortier.
Inside Consulting og Oxford Research (2005):
Evaluering af centerkontrakt-/
innovationskonsortiumordningen.
Udarbejdet af Inside Consulting og Oxford
Research for Videnskabsministeriet.
Schibany, Andreas, Gerhard Streicher, Nikolaus Gretzmacher, Martin Falk, Rahel
Falk, Norbert Knoll, Gerhart Schwarz and Martin Wörter (2004):
Evaluation FFF.
Impact Analysis.
Background report 3.2. Joanneum Research.
Wooldridge, J. (2002), Econometric Analysis of Cross Section and Panel Data, MIT
Press.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
42
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0043.png
>
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0044.png
Appendix 1: Selection of controls
>
The KOB dataset is a panel dataset which has repeated observations for most of
the firms - one for each annual account filed at the authorities. So for each firm,
there are typically multiple firm-year observations (where a firm-year observation
refers to a data-point of a given firm in a given year). In the following, we will use
the expression ‘control observation’ to describe a single firm-year observation of a
control.
Control firms are chosen in the year in which they are most similar to one (in a
single case: two) of the participants in the year before participation. This defines
each control firm’s ‘base year’ as the year in which it is selected as a control firm.
For each control, the base year forms the basis for comparisons of given success
parameters over time.
Note similarity between participants and potential controls is in terms of (a) the
firms’ industry, region, size and age and (b) the expected probability of participation,
derived as follows:
We run an auxiliary regression on the universe of approx. 370,000 firm-year
observations in KOB in the period 1994 to 2001 that roughly resemble the group of
participants (we do for example not consider industries in which there is no single
participant).
The auxiliary regression is formulated as a simple probit model, with starting to
participate in the programme next year being the dependent variable, and 32 controls
in total, covering firm size , industry, region and time period. The regressions’
pseudo R2, which is a measure of the model’s goodness-of-fit, is 0.22, which we
consider as being high.
The probit regression allows making statements of how likely program participation
is for a given firm. This allows finding pairs or groups of firms, in which this
probability is very similar. For two firms A and B with similar participation
probability, the fact of firm A participating and not firm B can now be interpreted as
being coincidental.
Under this interpretation, the identification set-up resembles an experiment, in
which programme participation was at random, and which would allow interpreting
systematic differences in outcome variables between participants and controls as the
programme’s causal effect on participating firms.
Yet, even firms with similar predicted participation probabilities can be quite
different, and to avoid systematic differences in industry affiliation, size, etc.,
between participants and controls, we also condition on a number of observable
characteristics being equal for a given participant and its matched control firm(s).
For this purpose, we divide the total number of firm-year observations into groups
having the same industry affiliation and being in the same region, of similar size and
observed in the same year.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
44
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0045.png
>
For each participant, we select the firm-year observation of a non-participant
firm being within the same group and having a participation probability which
comes closest to the participant’s. This selected firm-year observation defines the
participant firm’s control firm, and the control firm’s base year.
By repeating this matching procedure, we can find an arbitrary number of control
observations for each participant. Here, a greater number of control observations
increases the robustness of later results, however, increasing this number also makes
it increasingly difficult to find highly similar control observations for some of the
participants.
As a compromise within this trade-off, we chose to find for each participant two
control observations (firm-year observations of non-participants). The selection of
the two control observations per participant is in two rounds. In each of the rounds
we select one control observation for each participant.
In the first round we find 220 control observations of non-participants, in the second
we find another 219 control observations of non-participants (the reason for only 219
instead of 220 being that in a single case one firm-year observation is chosen as a
control observation for two participants).
In each of the two rounds, we first condition on many factors being highly similar
when selecting control observations. This leaves a number of participants, for which
no control observations could be found. In subsequent steps, we reduce the number
of factors and start choosing control observations which are increasingly less
similar, until each round has identified one control observation for each participant.
This selection of control observations is described in greater detail in TABLES A1.1
and A1.2.
In each of the rounds we only select one control observation per participant. This
does not rule out selection of different control observations (belonging to different
years) of the same control firm, which implies that there are a number of control
observations which occur repeatedly in the data which form the basis of the
performance analysis.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
45
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0046.png
>
TABLE A1.1: Identification of first neighbours by balanced score
matching procedure
Step 1:
Participants and controls are restricted to be equal in terms of ….
Industry (143) categories
Number of employees (11 categories)
Gross profit (7 categories)
Year in which they are observed (9 years)
Region (8 regions)
Firm age (3 categories)
This identifies control firms for 61 participant firms (27.7%).
Participants and controls are restricted to be equal in terms of ….
… industry (143) categories
… number of employees (11 categories)
… gross profit (7 categories)
… time period in which in which they are observed (5 periods covering 2 years each)
… region (8 regions)
… firm age (3 categories)
This identifies control firms for 67 participant firms (30.5%).
Participants and controls are restricted to be equal in terms of ….
… industry (143) categories
… number of employees (11 categories)
… gross profit (7 categories)
… time period in which in which they are observed (5 periods covering 2 years each)
… firm age (3 categories)
This identifies control firms for 103 participant firms (46.8%).
Participants and controls are restricted to be equal in terms of ….
… industry (33) categories
… number of employees (11 categories)
… gross profit (7 categories)
… time period in which in which they are observed (5 periods covering 2 years each)
… firm age (3 categories)
This identifies control firms for 165 participant firms (75.0%).
Participants and controls are restricted to be equal in terms of ….
… industry (33) categories
… number of employees (9 categories)
… gross profit (6 categories)
… time period in which in which they are observed (5 periods covering 2 years each)
… firm age (3 categories)
This identifies control firms for 169 participant firms (76.8%).
Participants and controls are restricted to be equal in terms of ….
… industry (33) categories
… number of employees (6 categories)
… gross profit (5 categories)
… time period in which in which they are observed (5 periods covering 2 years each)
… firm age (3 categories)
This identifies control firms for 184 participant firms (83.4%).
Step 2:
Step 3:
Step 4:
Step 5:
Step 6:
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
46
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0047.png
>
Step 7:
Participants and controls are restricted to be equal in terms of ….
… industry (9 categories)
… number of employees (6 categories)
… gross profit (5 categories)
… time period in which in which they are observed (5 periods covering 2 years each)
… firm age (3 categories)
This identifies control firms for 199 participant firms (90.5%).
Participants and controls are restricted to be equal in terms of ….
… industry (9 categories)
… number of employees (4 categories)
… gross profit (4 categories)
… time period in which in which they are observed (4 periods covering 3 years each)
… firm age (3 categories)
This identifies control firms for 202 participant firms (91.8%).
Participants and controls are restricted to be equal in terms of ….
… industry (9 categories)
… time period in which in which they are observed (5 periods covering 2 years each)
This identifies control firms for 220 participant firms (100.0%).
Step 8:
Step 9:
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
47
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0048.png
>
TABLE A1.2: Identification of second neighbours by balanced score matching
procedure
Step 1:
Participants and controls are restricted to be equal in terms of ….
Industry (143) categories
Number of employees (11 categories)
Gross profit (7 categories)
Year in which they are observed (9 years)
Region (8 regions)
Firm age (3 categories)
This identifies control firms for 33 participant firms (15.0%).
Step 2:
Participants and controls are restricted to be equal in terms of ….
… industry (143) categories
… number of employees (11 categories)
… gross profit (7 categories)
… time period in which in which they are observed (5 periods covering 2 years each)
… region (8 regions)
… firm age (3 categories)
This identifies control firms for 54 participant firms (24.6%).
Step 3:
Participants and controls are restricted to be equal in terms of ….
… industry (143) categories
… number of employees (11 categories)
… gross profit (7 categories)
… time period in which in which they are observed (5 periods covering 2 years each)
… firm age (3 categories)
This identifies control firms for 87 participant firms (40.0%).
Step 4:
Participants and controls are restricted to be equal in terms of ….
… industry (33) categories
… number of employees (11 categories)
… gross profit (7 categories)
… time period in which in which they are observed (5 periods covering 2 years each)
… firm age (3 categories)
This identifies control firms for 144 participant firms (65.5%).
Step 5:
Participants and controls are restricted to be equal in terms of ….
… industry (33) categories
… number of employees (9 categories)
… gross profit (6 categories)
… time period in which in which they are observed (5 periods covering 2 years each)
… firm age (3 categories)
This identifies control firms for 151 participant firms (68.6%).
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
48
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0049.png
>
Step 6:
Participants and controls are restricted to be equal in terms of ….
… industry (33) categories
… number of employees (6 categories)
… gross profit (5 categories)
… time period in which in which they are observed (5 periods covering 2 years each)
… firm age (3 categories)
This identifies control firms for 173 participant firms (78.6%).
Step 7:
Participants and controls are restricted to be equal in terms of ….
… industry (9 categories)
… number of employees (6 categories)
… gross profit (5 categories)
… time period in which in which they are observed (5 periods covering 2 years each)
… firm age (3 categories)
This identifies control firms for 194 participant firms (88.2%).
Step 8:
Participants and controls are restricted to be equal in terms of ….
… industry (9 categories)
… number of employees (4 categories)
… gross profit (4 categories)
… time period in which in which they are observed (3 periods covering 3 years each)
… firm age (3 categories)
This identifies control firms for 200 participant firms (90.0%).
Step 9:
Participants and controls are restricted to be equal in terms of ….
… industry (9 categories)
… time period in which in which they are observed (5 periods covering 2 years each)
This identifies control firms for 220 participant firms (100.0%).
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
49
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0050.png
Appendix 2:
Illustration of the diff-in-diff estimation set-up
>
Estimation of the programme’s effects is by a difference-in-difference model: For
both participants and controls, we calculate the average annual increases of the
success parameters in the years before the base year. We also calculate the average
annual increases of the success parameters in the years after the base year for both
participants and controls.
Thus, we can compare the (a) average increases of participants before they start
participating in an IC, (b) the average increases of participants after they have
started to participate in an IC, (c) the average increases of controls before they were
selected as controls (i.e. were most similar to one of the participants before it started
to participate) and (d) the average increases of controls after they were selected.
So let a be a participant’s pre-base-year average annual increase in either success
parameter, b a participant’s after-base-year average annual increase in either success
parameter, c a control’s pre-base-year average annual increase in either success
parameter and d a control’s after-base-year average annual increase in either success
parameter.
Note b-a measures by how much a participant’s average annual increase in the
success parameter changes when the participants starts participating in the
programme. For controls, the difference d-c measures the difference in the average
annual increases between before and after the base year.
Under the assumption that participants would continue having average annual
increases a in the absence of the programme, the average of the participant-specific
differences b-a estimates the IC’s causal average effect on participant firms.
However, this assumption is relatively strong, as b may be different from a for other
reasons than programme participation (e.g., business cycle or firm age effects).
But given the similarity of participants and controls in the base year, one may
argue that these ‘other reasons’ should have the same effect for both participants
and controls, and assume that b-a would on average be equal to d-c in the absence
of programme participation. Under this ‘identifying’ assumption, (b-a)-(d-c) is the
change in participants’ average annual increases between before and after the start
of participating in the IC which can only be explained by the programme, in other
words: the programme’s causal effect on participating firms.
To the extent that there remain dissimilarities between participants and controls in
observable factors such as industry, size or geographical region which potentially
could generate differences in the growth patterns of participants and controls, these
will be taken account of by including control variables in the regressions to follow.
When taking this model to the data, (b-a)-(d-c) is estimated by a simple linear
regression (with heteroscedasticity-consistent standard errors).
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
50
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0051.png
>
Here, we need to make decisions regarding the length of the time periods over which
pre-base-year and after-base-year average annual increases are computed. We made
the following choices: average pre-base-year increases are computed over a three-
year period before the base year. Average post-base-year increases are computed
over a five year and, as a second step, over a ten year period.
7
For the estimation of the model, we generate (typically) two observations per firm:
First, the average increases of the success parameters in the three-year period before
the base year. Second, the average increases of the success parameters in the five
(ten) year period after the base year.
This implies that we will only consider firms that were observed three years before
the base year, and at least five or ten years after the base year for the estimations.
The regression equation is taking the following form:
A = k + β1
*
d1 + β2
*
d2 + β3
*
d1d2 + β4
*
x + ε,
where A is the average increase in the success parameter in either the time period
before or after the base year and k is the constant term. d1 is an indicator variable
taking the value one (and zero otherwise) if the observation is after the base year, d2
is an indicator variable taking the value one (and zero otherwise) if the observation
belongs to a participant. cd2 takes the value one if the observation belongs to a
participant and is after the base year (and zero otherwise). x is a set of control
variables with an associated set of coefficients β4 to be estimated. β1, β2, and β3 are
also coefficients to be estimated, and ε is an error term assumed to satisfy standard
specifications.
Note that inclusion of the vector x is redundant in the sense that the matching
procedure implies high similarity in observable characteristics across participants
and controls. Still, inclusion of x increases the explanatory power of the model, and
might safeguard against potential differences between participants and controls.
7
These choices reflect compromises between the wish not to lose too many firms for the analysis which only
are observed for shorter time periods and the wish to being able to follow firms long enough to being able
to detect any effects in case they exist. Also, the precision of the growth trend measures increases with the
length over which the averages are calculated, which is relevant here because of considerable year-to-year
volatility in the success parameters. Basing estimates of pre-base-year time trends on a three-year period
is a compromise between not to lose too many firms for the analysis and the wish to generate reasonably
stable estimates of pre-base-year growth patterns.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
51
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0052.png
>
Note further that
- the constant term k estimates average c, i.e., the average annual increases for
controls before the base year
8
,
- k+β1 estimates average d, the average annual increases for controls after the
base year,
- k+β2 estimates average a, i.e., the average annual increases for participants
before the base year,
- k+β1+β2+β3 estimates average b, i.e., the average annual increases for
participants after the base year.
Thus, β3 estimates average (b-a)-(d-c), which is, under the indentifying assumption,
the programme’s average causal effect for firms that participate in the programme.
In the language of the evaluation literature, β3 estimates the ‘average treatment
effect on the treated (ATT)’.
8
Strictly speaking does k estimate the average annual increases for controls with all variables in the vector
x taking the value zero before the base year.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
52
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0053.png
Appendix 3:
Exit and survival of participants and controls
>
The last additional step of the analysis is comparing participants’ and controls’ exit
and closure behaviour. In the following, exit will refer to a firm leaving the data
before 2008 (which is the end of the observation period) – without making any
distinctions between the potential reasons for doing so.
Closure on the other hand is defined as one of the following transitions: bankruptcy,
liquidation, or forced exit. Information of these transitions is from the cvr-register of
the ministerial body ’Erhvervs-
og Selskabsstyrelsen’.
There are 162 exit and 60 closure events in the data.
This appendix addresses two issues: first, whether fast growing participants have a
higher probability of staying in the data compared to controls. This would imply that
growth increase estimates for participants in association with the programme are
biased upwards.
The second question is whether participants have lower closure probability
compared to controls, which might be – given the similarity of participants and
controls – interpreted as a positive effect of the programme on participants’ survival.
TABLE A3.1 presents participants’ and controls’ exit status when leaving the data.
We find that approx. 76 per cent all firms stay in the data until 2008, which is the
end of the observation period. There is a higher share of participants that can be
followed until 2008. Participants have a lower propensity to exit in general, and
especially to exit by a merger/acquisition event.
There is a higher share of participants that can be followed until 2008. Participants
have a lower propensity to exit in general, and especially to exit by a merger/
acquisition event.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
53
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0054.png
>
TABLE A3.1: Firm transitions (in per cent of total)
Participants
Continued until at least 2008
Merger/acquisition
Bankruptcy
Liquidation
Dissolution
Split up
Restructured
Forced exit
Erased from register
Total
88,18
3,18
4,55
2,73
0
0,45
0,91
0
0
100
Controls
69,93
14,81
5,01
3,87
1,82
1,37
1,14
1,14
0,91
100
Both
76,03
10,93
4,86
3,49
1,21
1,06
1,06
0,76
0,61
100
To test the statistical significance of this result, we employ a simply binary choice
logit model, which allows making statements on the differences in the expected exit
or closure probabilities of participants and controls. Results are presented in TABLE
A3.2, and can be summarized as follows:
The results of Model 1 provide evidence of participants having a significantly lower
probability of leaving the data as exits. The coefficient -0.856 implies that their
probability of exiting in a given year is less than half of controls’ exit probability.
Given that ‘exit’ is by no means to be associated with ‘failure’, this finding is no
indication of participants being more successful than controls.
The result of Model 2 implies that there is no significant difference in the probability
to exit by a closure event (which might be interpreted as a success measure) between
participants and controls.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
54
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0055.png
>
TABLE A3.2: Results of logit model regressions
Model 1: Dependent variable: exit
before 2008
Coefficient
Firm is participant
firm
Constant term
-0,856***
-3,303***
Standard error
0,209
0,093
Model 2: Dependent variable: firm
closure (before 2008)
Coefficient
-0,291
-4,469
Standard error
0,299
0,163
Pseudo R2=0.014
5,238 observations
Pseudo R2=0.0016
5,238 observations
*** significant at 1%. ** significant at 5%, * significant at 10%; both models are estimated on all firm-year
observation after the base year and before 2008.
Finally, we investigate whether the relationships between the growth in the success
parameters and exit probability are different for participants and controls.
If for example fast-growing controls have a disproportionally low propensity of
leaving the data, any effects of the programme would be underestimated – because
there would be disproportionally many fast-growing controls which are observed
five or ten years after the base year compared to relatively fewer fast-growing
participants.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
55
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0056.png
>
TABLE A3.3: Results of logit model regressions. Dependent variable:
the firm exits the data. Gross profit measured in million DKK
Coefficient
Firm is participant firm
#employees(t)-#employees(base year)
gross profit (t)-gross profit (base year)
(1)
(2)
(3)
-0,9124***
-0,0004
-0,0025**
0,0006
0,0025*
-0,0172
-3,4500***
Standard error
0,2654
0,0006
0,0013
0,0009
0,0013
0,0356
0,2087
{#employees(t)-#employees(base year)} *(Firm is
participant firm)
(4)
{gross profit (t)-gross profit (base year)}*(Firm is
participant firm)
(5)
Years after base year
Constant term
(6)
(7)
Pseudo R2=0.014
5,238 observations
*** significant at 1%. ** significant at 5%, * significant at 10%; the model is estimated on all firm-year
observation after the base year and before 2008.
The results of this model are presented in TABLE A3.3. Coefficient (5) suggests
that participants with high growth in gross profit have higher risk of leaving the data
compared to controls. A participant that increases gross profit by 10 million DKK
increases the probability of leaving the data by approx. 3 percent. This implies an
absolute percentage point increase of approx. 0.09 percent. Put different, differences
are negligible, and, if anything, the measured increases in growth in association with
programme participation would have been larger if not a number of high growth
participants would have left the data.
An Analysis of Firm Growth Effects of the Danish Innovation Consortium Scheme
56
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0057.png
Publikationer i serien
Innovation: Analyse og evaluering 2009 og 2010
01/2009 Effektmåling af innovationsmiljøernes
støtte til danske iværksættere
02/2009 Rammer for innovativ IKT-anvendelse
– erfaringer fra Den Regionale IKT-satsning
03/2009 Analyse af forsknings- og udviklings-
samarbejde mellem virksomheder og
videninstitutioner
04/2009 International Evaluation of the Danish
GTS-system – A step beyond
05/2009 Proof of concept-finansiering til offentlige
forskningsinstitutioner – Midtvejsevaluering
06/2009 Mapping of the Danish knowledge system
with focus on the role and function of the GTS-net
07/2009 International Comparison of Five
Institute Systems
08/2009 Review of science and technology foresight
studies and comparison with GTS2015
09/2009 Analyse af små og mellemstore
virksomheders internationale FoU-samarbejde
10/2009 Ikt-anvendelse og innovationsresultater i
små og mellemstore virksomheder
11/2009 Virksomhedernes alternative strategier til
fremme af privat forskning, udvikling og innovation
12/2009 Rådet for Teknologi og Innovation måler sin
indsats inden for metrologi i perioden 2007-2009
13/2009 Kommercialisering af forskningsresultater -
Statistik 2008
14/2009 Erhvervslivets forskning, udvikling og
innovation i Danmark 2009 – Den økonomiske
krises betydning
15/2009 Finanskrisens påvirkning på IT-startups
16/2009 Universiteternes Iværksætterbarometer 2009
17/2009 Kortlægning af iværksætter- og
entreprenørskabsfag ved de 8 danske universiteter
– 2009
18/2009 The Gazelle Growth Programme –
Mid Term Evaluation
19/2009 Nye former for samarbejde om
privat forskning, udvikling og innovation -
midtvejsevaluering af åbne midler
>
20/2009 Innovationsagenter - Nye veje til innovation
i små og mellemstore virksomheder. Erfaringer fra
midtvejsevaluering af pilotprojektet Regionale
Innovationsagenter
21/2009 Forskning, udvikling og innovation i små
og mellemstore virksomheder - erfaringer fra
midtvejsevaluering af videnkuponer
22/2009 Dansk innovationspolitik 2009 – Den
økonomiske krises betydning for fremme af
erhvervslivetsforskning, udvikling og innovation
23/2009 Serviceinnovation og innovations-
fremmesystemet
24/2009 Performanceregnskab for Forsknings- og
Innovationsstyrelsens innovationsnetværk 2009
25/2009 Performanceregnskab for
innovationsmiljøerne 2009
01/2010 Produktivitetseffekter af erhvervslivets
forskning, udvikling og innovation
02/2010 Erhvervslivets forskning, udvikling og
innovation i Danmark 2010
03/2010 An Analysis of Firm Growth Effects of
the Danish Innovation Consortium Scheme
04/2010 Effektmåling af videnpilotordningens
betydning for små og mellemstore virksomheder
05/2010 InnovationDanmark 2009 - resultater
og evalueringsstrategi
06/2010 Kommercialisering af forskningsresultater -
Statistik 2009
07/2010 Performanceregnskab for
Videnskabsministeriets GTS-net 2010
08/2010 Performanceregnskab for
Videnskabsministeriets Innovationsnetværk 2010
09/2010 Performanceregnskab for
Videnskabsministeriets Innovationsmiljøer 2010
10/2010 Universiteternes Iværksætterbaromenter
2010
 UFU, Alm.del - 2015-16 - Endeligt svar på spørgsmål 168: Spm. om ministeren vil fremsende relevante evalueringer af Globaliseringspuljen 2007-12, til Uddannelses- og forskningsministeren
1647742_0058.png
Virksomheder oplever store gevinster ved at samar-
bejde med videninstitutioner om forskning og udvikling
>
En virksomhed, der har deltaget i et forsknings- og udviklingssamarbejde med
universiteter og GTS-institutter, oplever i løbet af de næste 10 år en merværditilvækst,
der er ca. 20 mio. kroner højere end for lignende virksomheder, som ikke har været
med i et samarbejde. Der er endvidere signifikant positive beskæftigelseseffekter
for virksomheder, der havde mindre end 150 medarbejdere året før de indgik i
samarbejde.
Det dokumenterer denne analyse som Centre for Economic and Business Research
(CEBR) på Copenhagen Business School (CBS) har lavet for Forsknings- og
Innovationsstyrelsen. Analysen er baseret på registerdata fra 220 virksomheder,
der i perioden 1995-2003 har deltaget i et innovationskonsortium med statslig
medfinansiering.