SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Sundheds- og Ældreudvalget 2019-20
SUU Alm.del Bilag 109
Offentligt

Evaluering af Nordic Cochrane

Centre og Copenhagen Trial Unit

Kurt Houlberg, Amalie Martinus Hauge, Morten Sall Jensen, Arendse Tange

Larsen, Astrid Broni Heinemeier og Jakob Kjellberg

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Evaluering af Nordic Cochrane Centre og Copenhagen Trial Unit

e-ISBN: 978-87-7119-725-9

Projekt: 301306

VIVE – Viden til Velfærd

Det Nationale Forsknings- og Analysecenter for Velfærd

Herluf Trolles Gade 11, 1052 København K

www.vive.dk

VIVEs publikationer kan frit citeres med tydelig kildeangivelse.

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Forord

Sundheds- og Ældreministeriet har bedt VIVE om at gennemføre en evaluering af Nordic

Cochrane Centre (NCC) og Copenhagen Trial Unit (CTU).

VIVE har valgt at designe evalueringen som en klassisk forskningsevaluering og har lagt det

ud til et eksternt og uafhængigt evalueringspanel af internationale eksperter at udarbejde en

evalueringsrapport for henholdsvis NCC og CTU.

VIVE har designet og organiseret evalueringen og har i alle evalueringens faser faciliteret eva-

lueringspanelets arbejde. Afrapporteringsmæssigt adskiller evalueringen sig dog fra de fleste

andre VIVE projekter ved, at det ikke er VIVE, der har draget konklusionerne og udarbejdet

evalueringsrapporterne, men et evalueringspanel bestående af internationale eksperter.

I dette ”forklæde” til det internationale evalueringspanels evalueringsrapporter præsenterer

VIVE evalueringens formål og det valgte evalueringsdesign samt sammenfatter de konklusio-

ner, som evalueringspanelet har draget. Evalueringspanelets fulde evalueringsrapporter (på

engelsk) er optrykt i Bilag 3 (NCC) og Bilag 4 (CTU).

VIVE ønsker at sende en stor tak til de to centre og de tre tilknyttede Cochrane Review-grupper

for det store arbejde, de har lagt i evalueringen og for et godt og konstruktivt samarbejde gen-

nem hele evalueringen. Derudover skal der lyde en stor tak til de samarbejdspartnere og inte-

ressenter, som har stillet sig til rådighed for interview mv. i forbindelse med evalueringspanelets

site-visit

og i forbindelse med VIVEs supplerende kvalitative dataindsamling. Endelig skal lyde

en særlig tak til evalueringspanelets medlemmer og ikke mindst panelets formand og næstfor-

mand for et stort, kompetent og engageret evalueringsarbejde.

Evalueringen er bestilt og finansieret af Sundheds- og Ældreministeriet.

Pia Kürstein Kjellberg

Forsknings- og analysechef for VIVE Sundhed

2019

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Indhold

Sammenfatning og konklusion ........................................................................... 5

Baggrund og formål .................................................................................. 7

Evalueringsdesign..................................................................................... 8

2.1

2.2

2.3

2.4

2.5

Evalueringspanel .......................................................................................... 9

Selvevalueringsrapporter ............................................................................ 11

VIVEs indsamling af supplerende data og materiale .................................. 12

Site-visit og møde i København .................................................................. 13

Høring af evalueringspanelets udkast til evalueringsrapport ...................... 14

Bilag 1

Bilag 2

Bilag 3

Bilag 4

Evalueringsprotokol ....................................................................... 15

Skabelon for selvevalueringsrapport .............................................. 18

Evaluation Report Nordic Cochrane Centre ................................... 35

Evaluation Report Copenhagen Trial Unit ...................................... 65

Litteratur........................................................................................................... 90

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Sammenfatning og konklusion

Sundheds- og Ældreministeriet har bedt VIVE om at gennemføre en evaluering af Nordic

Cochrane Centre (NCC) og Copenhagen Trial Unit (CTU). Denne rapport præsenterer evalu-

eringens formål og design samt sammenfatter evalueringens resultater.

Hovedformålet med evalueringen har været at afklare, hvorvidt formålet bag finanslovsbevillin-

gen til NCC og CTU er opfyldt, samt at undersøge omfanget af forskningsproduktionen og den

forskningsmæssige kvalitet samt centrenes impact og eksterne samarbejdsrelationer.

VIVE har valgt at designe evalueringen som en klassisk forskningsevaluering og har i samar-

bejde med Danmarks Frie Forskningsfond nedsat et eksternt og uafhængigt evalueringspanel

bestående af seks internationale eksperter, som har haft ansvaret for at udarbejde en evalue-

ringsrapport for henholdsvis NCC og CTU.

Det internationale evalueringspanel har udarbejdet evalueringsrapporterne på grundlag af tre

hovedkilder af data:



Selvevalueringsrapporter fra de to centre

Supplerede data og analyser fra VIVE

Et tredages

site-visit

i København.

Evalueringsrapporterne indeholder et ”executive summary” samt evalueringspanelets vurde-

ring af de to centre inden for hvert af følgende fire evalueringstemaer:

1. Ledelse og organisering

2. Forskningsproduktion og kvalitet

3. Impact og relevans

4. Samarbejdsrelationer.

Evalueringspanelets fulde evalueringsrapporter er indsat som Bilag 3 og Bilag 4 i denne rap-

port.

Hovedkonklusion

For begge centre er evalueringspanelets overordnede konklusion, at formålet med finanslovs-

bevillingen er opfyldt.

I evalueringen af NCC

fremhæver evalueringspanelet, at NCC har bidraget til at informere og

kvalificere beslutninger inden for sundhedsområdet gennem tilvejebringelse og syntetisering

af højkvalitets evidensbaseret viden, samt at dette er opnået gennem et betydeligt bidrag til at

forbedre kvaliteten af den evidens, der er til rådighed for beslutningstagere, inklusive patienter

og offentligheden. Dette er sket gennem udviklingen af robuste metoder og advokering for brug

af disse gennem publicering i

high-impact

forskningstidsskrifter og gennem træning. Feedback

fra interessenter peger endvidere på, at NCC har et godt omdømme baseret på kvaliteten,

uafhængigheden og vigtigheden af centerets arbejde og ophænget til Cochranes overordnede

mission. Evalueringspanelet ser dog også et behov for et tættere samarbejde mellem NCC og

de i Danmark lokaliserede Cochrane Review-grupper samt en øget involvering af interessenter

og for reetablering af et advisoryboard af medlemmer med tilknytning til den sundhedspolitiske

dagsorden og kliniske dagsorden i Danmark.

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

I evalueringen af CTU

fremhæver evalueringspanelet, at CTU, ifølge panelets vurdering, har

indfriet centerets primære mål om at støtte, koordinere og gennemføre randomiserede kliniske

forsøg, deltagelse i udviklingen af metoder til randomiserede kliniske forsøg og meta-analyser,

uddanne studenter og forskere i evidensbaseret medicin, randomiserede kliniske forsøg, meta-

analyser og sekventielle analyser; og støtte, koordinere og gennemføre systematiske reviews

af litteraturen. Evalueringspanelet vurderer endvidere, at CTU har publiceret sin forskning i en

vifte af

high-impact

forskningstidsskrifter, og at centerets studier har haft betydelig impact gen-

nem indvirkning på klinisk praksis og stimulering af offentlig debat og diskussion.

Behov for at overveje muligheden for et ændret organisatorisk setup

Trods de to centres anseelige præstationer og den overordnede konklusion om, at formålet

med finanslovsbevillingen er opfyldt, stiller evalueringspanelet for både NCC og CTU spørgs-

målstegn ved bæredygtigheden af den nuværende organisationsmodel i lyset af centrenes be-

grænsede størrelse, afhængighed af ministeriel finansiering, placering på Rigshospitalet og

komplekse styringsstruktur. Evalueringspanelet konkluderer således, at der for både NCC og

CTU er et presserende behov for at overveje mulighederne for et ændret organisatorisk setup.

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Baggrund og formål

Sundheds- og Ældreministeriet har bedt VIVE om at gennemføre en evaluering af Nordic

Cochrane Centre og Copenhagen Trial Unit.

Baggrunden er, at Nordic Cochrane Centrr (NCC) og Copenhagen Trial Unit (CTU) hvert år

modtager en finanslovsbevilling. I Finansloven for 2018 var den samlede bevilling til de to cen-

tre på 17,8 millioner kroner.

I finansloven står, at

”Aktiviteterne indenfor Cochrane området omfatter bl.a. udarbejdelse af

systematiske oversigter over sundhedsvæsenets interventioner, forebyggelse af sygdomme,

diagnostik, behandling og pleje. CTU understøtter og udfører videnskabeligt relevante kliniske

forsøg, bl.a. som led i Cochrane-samarbejdet”

(Finansloven 2018, § 16.51.03.45).

Hovedformålet med evalueringen er at afklare, hvorvidt formålet bag finanslovsbevillingen er

opfyldt, samt at undersøge omfanget af forskningsproduktionen og den forskningsmæssige

kvalitet samt centrenes impact og eksterne samarbejdsrelationer.

Konkret sætter evalueringen fokus på følgende fire evalueringstemaer:

1. Ledelse og organisering

2. Forskningsproduktion og kvalitet

3. Impact og relevans

4. Samarbejdsrelationer.

Ud over NCC og CTU omfatter evalueringen også de tre Cochrane Review-grupper, som er

lokaliseret i Danmark:

1. The Cochrane Colorectal Group (CCG),

2. The Cochrane Anaesthesia Group and the Emergency and Critical Care Group (ACE)

3. The Cochrane Hepato-Biliary Group (CHBG).

Evalueringen er designet som en klassisk forskningsevaluering. Heri ligger blandt andet, at

evalueringen af de to centre og udarbejdelsen af evalueringsrapporterne er foretaget af et eva-

lueringspanel bestående af internationale eksperter. Evalueringspanelet har udarbejdet en en-

gelsksproget evalueringsrapport for henholdsvis NCC og CTU. Disse to evalueringsrapporter

er indsat som Bilag 3 og Bilag 4 i denne rapport og indeholder begge et ”executive summary”

samt en vurdering af de fire evalueringstemaer.

Evalueringspanelets sammensætning og det samlede evalueringsdesign er nærmere beskre-

vet i kapitel 2,

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Evalueringsdesign

Evalueringen af NCC og CTU anlægger en klassisk tilgang til forskningsevalueringer. Baseret

på en kombination af kvantitative og kvalitativ data er der indsamlet viden om både forsknings-

centrenes videnskabelige produktion og dens kvalitet samt centrenes forskningsmæssige im-

pact i samfundet.

Inden for forskningsevaluering er der en række forskellige tilgange og metoder, som hver især

trækker visse elementer ved institutionen i forgrunden (Hansen, 2009; Garret-Jones, 2000).

Forskningsevalueringer fokuserer typisk på enten forskningens indflydelse i det akademiske

miljø eller i samfundet generelt. Indflydelse i det akademiske miljø undersøges via data om

forskningsproduktionen, dennes impact og institutionens relationer i det akademiske miljø. Ind-

flydelse i samfundet undersøges via data om deres nytte og betydning for samfundet, samt om

hvordan institutionen indgår i samspil med omgivelserne.

Det primære formål med et evalueringsdesign er, at det indhenter de data, der er relevante for

evalueringens formål i egnet omfang og format. Data som ofte bruges til forskningsevaluering

er kvantitative opgørelser over forskningsproduktionen gennem fx optælling af forskningspro-

duktion, brug af bibliometriske forskningsindikatorer og citationsanalyser (se fx Hidalgo og

Espinar, 2017). Derudover anvendes også kvalitative data som fx interview og peer-review.

Ofte kombineres disse metoder (Pedersen, 2017; Hansen 2009), hvilket også er denne evalu-

erings fremgangsmåde.

Evalueringen af NCC og CTU følger dermed den gængse fremgangsmåde for forskningseva-

luering, og i evalueringsdesignet har VIVE lagt vægt på:



at selve evalueringen foretages af internationale, uvildige eksperter på området, som

sammensættes i et evalueringspanel

at de to centre foretager en selvevaluering på baggrund af en skræddersyet evalue-

ringsskabelon, udviklet og kvalitetssikret af VIVE

at VIVE både selvstændigt og på forespørgsel af evalueringspanelet indhenter rele-

vante data til at supplere selvevalueringerne

at evalueringspanelet mødes på centrene (site-visit) og vurderer det materiale, som er

indhentet om centrene

at evalueringspanelet udarbejder evalueringerne på baggrund af selvevalueringerne,

supplerede data og analyser udarbejdet af VIVE samt fra evalueringspanelets site-visit.

at der udarbejdes selvstændige evalueringsrapporter for hvert af de to centre.



Den samlede evalueringsproces er illustreret i Figur 1.

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Figur 1:

Evalueringsprocessen

Nedsættelse af evaluerings-

panel

Udarbejdelse af skabelon for

selvevaluering

Evalueringspanel fastsætter

temaer for VIVEs data-

indsamling

- NCC/CTU udarbejder

selvevaluering

- VIVE indsamler supplerende

materiale

Materiale tilgår evalueringspanel

Site-visit af evalueringspanel:

- drøftelse af selvevaluering

- specifikke drøftelser

- efterspørgsel af suppl. data

NCC/CTU + VIVE genererer

eventuelle supplerende data

Evalueringspanel udarbejder

evalueringsrapport

Hovedelementerne i evalueringsdesignet er beskrevet nærmere nedenfor

2.1 Evalueringspanel

For at sikre en uvildig og kvalificeret evaluering er nedsat et evalueringspanel bestående af

internationale eksperter. Det er evalueringspanelet, der har udarbejdet den endelige evalue-

ring, og det er panelet, som står som afsender af evalueringen. Evalueringspanelets rapport

dækker følgende områder:

1. Ledelse og organisering

2. Forskningsproduktion og kvalitet

3. Impact og relevans

4. Samarbejdsrelationer.

Formanden for evalueringspanelet har i samarbejde med panelets næstformand haft det over-

ordnede ansvar for evalueringspanelets arbejde og udarbejdelse af evalueringen.

Formanden for evalueringspanelet er blevet udpeget af Danmarks Frie Forskningsfond (DFF)

efter indstilling fra VIVE. Ved indstillingen af formanden for evalueringspanelet er der blevet

lagt vægt på, at denne skulle være en international anerkendt forsker inden for det kliniske

område, have kendskab til det ledelsesmæssige og strategiske arbejde med forskningsinstitu-

tioner på internationalt niveau samt have erfaring med at lede store, internationale forsknings-

projekter og/eller forskningscentre. I valget af formand er endvidere lagt stor vægt på uvildighed

i forhold til de to centre, og der er foretaget individuelle vurderinger af, om mulige kandidater til

formandsposten har en tilpas distance til både de danske organisationer og disses internatio-

nale bagland.

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

VIVE har udpeget de øvrige medlemmer af evalueringspanelet efter indstilling fra DFF. VIVE

bad DFF om at indstille tre internationalt anerkendte forskere fra det sundhedsvidenskabelige

råd under DFF

og to internationalt anerkendte forskere fra det samfundsvidenskabelige råd

under DFF

. Kandidaterne fra det sundhedsvidenskabelige råd skulle have indsigt i processer

og metoder knyttet til brugen af randomiserede studier, systematiske reviews og erfaring med

at vurdere kvaliteten og relevansen af sundhedsfaglig forskning. Kandidaterne fra det sam-

fundsvidenskabelige råd skulle have indsigt i organisering og ledelse af forskningscentre på

sundhedsområdet samt have indsigt i det danske sundhedsvæsens organisering og ledelse.

For at sikre evalueringens uvildighed var det endvidere et vigtigt kriterium, at panelmedlem-

merne ikke måtte have haft tæt samarbejde eller andre tætte relationer med de to centre eller

deres internationale baglandsorganisationer de sidste fem år, som kunne give anledning til

habilitetsproblemer i forhold til evalueringen af de to centre. I overensstemmelse med ligestil-

lingsloven indstiller DFF både en mand og en kvinde til hver plads i evalueringspanelet.

Efter at DFF havde gennemført en proces med indstilling og udsendelse af invitationer til de

indstillede, modtog VIVE en liste fra DFF med fem forskere fra det sundhedsvidenskabelige

råd og to forskere fra det samfundsvidenskabelige råd. På grundlag af denne liste udpegede

VIVE efter samråd med evalueringspanelets formand de øvrige medlemmer af evalueringspa-

nelet. Baseret på en samlet vurdering af kandidaternes individuelle og komplementære kvalifi-

kationer udpegede VIVE tre af de fem forskere, som havde accepteret invitationen fra det sund-

hedsvidenskabelige råd, og de to forskere, som havde accepteret invitationen fra det sam-

fundsvidenskabelige råd. Alle panelmedlemmer er udpeget i kraft af deres personlige kompe-

tencer. Sammensætningen af udvalget har været i høring hos NCC og CTU.

Evalueringspanelet består samlet set af seks medlemmer med komplementære kompetencer

inden for sundhedsvidenskabelige reviews, klinisk forskning,

trials

samt ledelse og organise-

ring af sundhedsforskning:



Amanda Sowden (formand). Professor and Deputy Director and leader of the University

of York’s Centre for Reviews and Dissemination’s research programme in public health,

United Kingdom

Terje P. Hagen (næstformand). Professor and Head at the Institute of Health and Soci-

ety, University of Oslo, Norway

Diederick Grobbee. Professor of Clinical Epidemiology and founder of the Julius Center,

University Medical Center Utrecht, The Netherlands

Eva Swahn. Professor at the Department of Medical and Health Sciences, Division of

Cardiology, Linköping University & the Department of Cardiology, University Hospital,

Linköping, Sweden

Merete Osler. Clinical Professor at the Department of Public Health, University of Co-

penhagen & Consultant at the Center for Clinical Research and Prevention, Bispebjerg

and Frederiksberg Hospital, Denmark

Mickael Bech. Professor in healthcare leadership at Crown Prince Frederik Center for

Public Leadership, Aarhus University, Denmark.



Dette internationale evalueringspanel har udarbejdet evalueringsrapporterne for henholdsvis

NCC og CTU. Panelets arbejde med at evaluere de to centre har været baseret på en

Danmarks Frie Forskningsfond | Sundhed og Sygdom

Danmarks Frie Forskningsfond | Samfund og Erhverv (FSE)

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

evalueringsprotokol med konkrete evalueringsspørgsmål (se Bilag 1) og tre hovedkilder af

data:

1. Selvevalueringsrapporter fra de to centre

2. Supplerede data og analyser fra VIVE

3. Et tredages

site-visit

i København.

Disse datakilder beskrives i de efterfølgende tre afsnit.

Evalueringspanelets arbejde har bestået dels af møder/site-visits, dels af vurdering af udsendt

skriftligt materiale. Panelet har mødtes i København i forbindelse med site-visits, hvor der også

har været afholdt arbejdsmøde. Panelet har fået tilsendt de to centres selvevaluering forud for

mødet i København samt supplerede data og analyser fra VIVE.

2.2 Selvevalueringsrapporter

Selvevalueringsrapporterne har skullet give evalueringspanelet viden om centrene og en for-

ståelse for deres nuværende situation og de udfordringer, som centrene står overfor. NCC og

CTU har udarbejdet hver deres selvevalueringsrapport, baseret på en evalueringsskabelon

udarbejdet af VIVE, og kommenteret og kvalitetssikret af formanden for evalueringspanelet. I

udviklingen af evalueringsskabelonen er der blevet lagt vægt på, at den skulle indhente data,

som er dækkende, systematiske og velunderbyggede. Hvert center har været ansvarlig for

udarbejdelsen af deres selvevalueringsrapport, og de har selv afgjort, hvordan arbejdet med

selvevalueringen skulle organiseres.

Den samlede skabelon for selvevalueringsrapporten er gengivet i Bilag 2 (på engelsk).

Selvevalueringsskabelonen indeholder i oversigtsform følgende hovedelementer:

Beskrivelse af centerets strategi, organisering, ledelse og økonomi

NCC og CTU skal her redegøre for deres vision og strategi, hvilke tiltag der er igangsat for

realiseringen af vision og strategi, samt hvordan opfølgningen på de strategiske målsætninger

sikres. Centrene skal beskrive, hvordan de er organiseret og ledet, hvorledes dette understøt-

ter deres arbejde og realisering af strategi og vision. De skal foretage en beskrivelse af deres

grundlæggende organisering, ledelsens opgaver og placering samt de forskellige støttefunkti-

oner på centrene. De skal også fremlægge en redegørelse for deres økonomi, økonomistyring

og deres evne til at tiltrække eksterne ressourcer fra fx forskningsråd, fonde og EU.

Forskningsproduktion og forskningskvalitet

Centrene skal her opgøre og kvalitetsvurdere deres forskningsproduktivitet. Opgørelsen om-

fatter en optælling af publikationer de seneste fem år fordelt på seks hovedtyper: peer-review

artikler, Cochrane Reviews, Cochrane Protocols, bidrag til bøger, bidrag til konferencer og an-

den forskningsformidling (forskellige medier) for centrene. Som led i dette opgøres årlige gen-

nemsnitstal af publikation pr. forsker og pr. bevilligede million kroner. Endvidere skal centrene

redegøre for deres undervisningsaktivitet samt antallet af ph.d.-afhandlinger de seneste fem

år, påbegyndte ph.d.-forløb, igangværende ph.d.-forløb, samt hvor ph.d.-kandidaterne har fået

ansættelse.

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

For at vurdere kvaliteten af forskningen skal artikler publiceret de seneste fem år inddeles efter

de forskningstidsskrifter, hvor de er publiceret, ligesom centrene skal redegøre for, hvilke forsk-

ningstidsskrifter de anser for at være top forskningstidsskrifter for publicering af centerets forsk-

ning.

Impact og relevans

Centrene skal her redegøre for, hvordan deres forskning og aktiviteter bruges og skaber nytte

forskellige steder i det danske samfund. Det skal bl.a. bidrage til at vurdere relevansen af den

forskning, som centrene bedriver. Redegørelsen skal dække følgende punkter:



Eksempler på mulig indvirkning på behandlingspraksis og kliniske retningslinjer de se-

neste fem år

Samarbejde med offentlige myndigheder de seneste tre år

Samarbejde med sygehuse og sundhedspersonale de seneste tre år

Deltagelse i rådgivning, kommissioner, nævn m.m. de seneste tre år

Arrangør af konferencer, symposier og andre vidensdelende aktiviteter de seneste tre

år

Andre måder, som centerets forskning og aktiviteter er blevet brugt på, som har bidraget

til værdi for det danske samfund inden for de seneste tre år.



Opgørelsen skal underbygges af dokumentation for impact og relevans.

Samarbejdspartnere – nationalt og internationalt

NCC og CTU skal her redegøre for, hvilke eksterne parter de har formelle samarbejdsaftaler

med – både nationalt og internationalt. Redegørelsen skal angive typen af samarbejde, hvilken

rolle centrene spiller i samarbejdsrelationerne, hvilket output samarbejdet har resulteret i, og

hvordan samarbejdet har bidraget til realiseringen af centrenes målsætninger.

2.3 VIVEs indsamling af supplerende data og materiale

Efter dialog med formanden for evalueringspanelet har VIVE indsamlet yderligere data og fo-

retaget supplerende analyser til evalueringspanelet inden for de fire overordnede temaer:

1. Ledelse og organisering

2. Forskningsproduktion og kvalitet

3. Impact og relevans

4. Samarbejdsrelationer.

Selvevalueringen giver et vigtigt input til evalueringen, men de supplerende data og analyser

gør det muligt for evalueringspanelet at få et bedre og mere dybdegående grundlag for evalu-

eringen.

Ledelse og organisering

For at sikre et relevant sammenligningsgrundlag i forhold til centrenes ledelse og organisering

har VIVE for evalueringspanelet afdækket, hvordan udvalgte centre med sammenlignelige

forskningsaktiviteter er organiseret og ledet. Der er blevet udvalgt to centre til sammenligning

med NCC og to centre til sammenligning med CTU.

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Forskningsproduktion og kvalitet

Som supplement til opgørelserne i selvevalueringsrapporten har VIVE foretaget en opgørelse

af de to centres involvering i forskningsaktiviteten ved en citationsanalyse. Da begge centre

primært publicerer i kliniske tidskrifter, opgøres de årlige antal af første-, med- og sidste forfat-

terskaber for centrene

. Dernæst opgøres antallet af citationer for publicerede artikler for de

sidste fem år, hvilket vurderes at give et mål for relevansen af de publicerede artikler, da et

højt citationsantal betyder, at andre forskere har brugt resultaterne i deres artikler og derfor

finder forskningen relevant. I forlængelse heraf udregnes to forskellige indikatorer for, om ar-

tiklen er citeret mere eller mindre end gennemsnittet for de valgte forskningstidsskrifters artik-

ler. Som sidste del opgøres de bibliometriske forskningsindikator point (BFI-point) for de pub-

licerede artikler.

Impact og relevans

For at afdække, hvordan eksterne parter, som offentlige myndigheder og sygehuse, vurderer

betydningen af centrenes arbejde, har VIVE foretaget interview med relevante samarbejds-

partnere. Disse er blevet udvalgt, så der er sikret en bred repræsentation af forskellige aktører

og samarbejdsparter.

Samarbejdsrelationer

Med afsæt i de samarbejdspartenere og samarbejdsrelationer, som er fremgået af selvevalu-

eringen, har VIVE foretaget en nærmere analyse af, hvem der samarbejdes med, typen af og

indholdet i samarbejdet. Dette er blevet gjort ved at foretage kvalitative interview med to til fire

af de samarbejdspartnere, centrene har peget på i deres selvevaluering med henblik på at få

deres perspektiv på samarbejdet.

2.4 Site-visit og møde i København

Efter at have fået tilsendt selvevalueringsrapporterne og det supplerende materiale fra VIVE,

har evalueringspanelet den 20.-22. maj 2019 været på et tredages

site-visit

i København. I

programmet var der afsat en dag pr. center til besøg. Besøget skulle give evalueringspanelet

mulighed for at få et generelt indtryk af centrene og samtidig have mulighed for at drøfte selv-

evalueringen med centrene.

Programmet for evalueringspanelets besøg på centrene omfattede følgende punkter:



Introduktion og rundvisning

Møde med centrenes ledelse/evalueringsgruppe om selvevalueringen

Interview med udvalgte medarbejdere

Interview med eksterne samarbejdspartnere

Interview med repræsentanter for Cochrane Review-grupper.

Herudover omfattede programmet for evalueringspanelets tre dage i København et interview

med Rigshospitalets ledelse, et Skype-møde med direktøren for den internationale Cochrane

organisation, Mark Wilson, samt en række arbejds- og planlægningsmøder internt i evalue-

ringspanelet eller mellem VIVE og evalueringspanelet.

Dette er indikatorer for involveringen i forskningsproduktiviteten, da førsteforfattere typisk er primær skriver på artiklen,

medforfattere har bidraget i varierende grad og sidste forfatterskab typisk er seniorforsker (lektor eller professor). Første-

og sidste forfatter vurderes at være de mest prestigefyldte pladser i forfatterrækkefølgen i medicinsk litteratur (Caminiti

2015).

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

2.5

Høring af evalueringspanelets udkast til evalueringsrapport

I perioden efter besøgsdagene i København har evalueringspanelet foretaget en samlet vur-

dering af evalueringsdataene fra selvevalueringsrapporter, supplerende materiale fra VIVE og

site-visit og har på grundlag af dette udarbejdet et udkast til evalueringsrapport for henholdsvis

NCC og CTU. Disse udkast er herefter blevet sendt til høring hos de to centre, de tre tilknyttede

Cochrane Review-grupper samt Sundheds- og Ældreministeriet. Efter at have modtaget hø-

ringssvarene og forholdt sig til disse har evalueringspanelet udarbejdet de endelige evalue-

ringsrapporter.

Evalueringspanelets endelige evalueringsrapporter er gengivet i Bilag 3 (NCC) og Bilag 4

(CTU).

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Bilag 1

Evalueringsprotokol

Dette bilag indeholder projektets evalueringsprotokol, dvs. de evalueringsspørgsmål, som har

været styrende for evalueringspanelets arbejde med at evaluere de to centre. Evalueringspro-

tokollen er udarbejdet af VIVE i samarbejde med evalueringspanelets formand.

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Evaluation protocol

Evaluation of The Nordic Cochrane Centre and Copenhagen Trial Unit

The Nordic Cochrane Centre (NCC) and Copenhagen Trial Unit (CTU) receive each year a

grant under the National Finance Act. According to the Finance Act, the activities at the

Cochrane Centre include production of systematic reviews of health care interventions, pre-

vention of diseases, diagnostics, treatment and care, and that CTU supports and conducts

scientifically relevant clinical trials. The main purpose of the evaluation is to assess whether

the purpose of the National Finance Act grant is fulfilled, to evaluate the scale of the research

production and its research quality as well as the societal impact of the centres and their ex-

ternal collaborations and partnerships. Accordingly the evaluation both include formative ele-

ments focusing on broadening the understanding of the institute's characteristics for the pur-

pose of continuous improvement and summative elements focusing on accountability to the

entity that funds the running of the centre.

Specifically, the evaluation must address the following topics:

•

Management and organization of the centres

Research production and quality

Impact and relevance of the centres’ activities

Collaborations and partnerships

An external evaluation panel carries out the evaluation. The members of the panel are

acknowledged national and international experts within medicine and evaluation of re-

search. The basis for the evaluation include the following:

•

Self-evaluation reports by NCC and CTU

•

Site-visit by the evaluation panel

•

Documentation and additional data collected by VIVE – The National Centre for Social

Science Research.

The result of the evaluation is presented in separate reports for each of the two institutions.

The purpose of the evaluation is not to compare the two institutions but to give individ-

ual assessments of each centre.

In the following, the topics of the evaluation are described further.

Management and organization

The evaluation panel is asked to give a general description and assessment of the governance

structure and the management and organizational structure of the centre. The description and

assessment should address the following:

•

An overall description and assessment of the governance structure of the centre, includ-

ing reflections on the governance structures ability to hold the management of the cen-

tre accountable

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

•

A reflection on the strategy of the centre in relation to the purpose and mission of the

centre

A description and assessment of the management and organizational structure of the

centre, including reflections on the organizational and management structures ade-

quacy for accomplishing the centre´s purpose and strategy, and the expediency of the

decision-making structure and the division of labor between the different sections of

the centre

An assessment of the organizational support for researchers

An assessment of the centre’s ability to attract external resources from research coun-

cils, research funds, EU etc.

A reflection on the expediency of governance structure, the management and organiza-

tional structure compared to similar type of institutions.

Research production and quality

The panel is asked to give an assessment of the quantity and quality of research from the

centre. The assessment should include the following elements:

•

A count of publications over the past five years divided into main types of publications

The average number of annual publications per researcher for the centre at large

An assessment of the quality of research based on bibliometric indicators

Reflections on the level and quality of output compared to economic resources

An assessment of the centres’ production of PhD candidates, including reflections on

the centres’ involvement in PhD programs

An overall assessment of the productivity of the centre and the importance and quality

of the research.

Impact and relevance

The evaluation panel is asked to give its assessment of the impact and relevance of the centre’s

activities for the Danish society. The assessment should include the following elements:

•

An assessment of the impact of the centre on clinical practice and clinical guidelines.

Reflections on how the centres research and other activities create value in various

parts of the Danish society

Reflections on the centres’ impact on the public debate in Denmark

Collaborations and partnerships

The evaluation panel is asked to give its assessment of the centres national and international

collaborations and partnerships. The assessment should include the following elements:

•

An assessment of the level of national and international collaboration of the centre, in-

cluding reflections on the level of national vs. international collaborations, and the role

of the centre in the collaborations

An assessment of the output of the collaborations

Reflections on importance of the collaboration for accomplishing the centre´s purpose

and strategy.

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Bilag 2

Skabelon for selvevalueringsrapport

Dette bilag indeholder den skabelon for selvevalueringsrapport, som de to centre har brugt i

udarbejdelsen af deres respektive selvevalueringsrapporter. Selvevalueringsskabelonen er ud-

arbejdet af VIVE i samarbejde med evalueringspanelets formand.

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Self-evaluation report template

Guidelines for the self-evaluation report

Evaluation of The Nordic Cochrane Centre and Copenhagen Trial Unit

It is the centre’s responsibility to draw up the self-evaluation document based on the guidelines given in

this guide. The centre is responsible for establishing the mechanisms for collecting information on the

centre needed to cover the themes mentioned in the guide, and to provide the requested documenta-

tion. The centre is free to increase the amount of evidence with the aim of broadening the scope of in-

formation provided and thereby improve the evaluation.

The self-evaluation report is a key part of the evaluation and together with the site-visit, the main evi-

dence for the external review process. The drawing up of the self-evaluation document by the centre

must enable the external review panel to know about and understand the centre's standing. The report

should be complete and rigorous, include evidence and documentation, systematic and detailed, and

balanced.

The self-evaluation report should cover the following themes:



Management and organization

Research production and quality

Impact and relevance

Collaborations and partnerships.

Use the centre’s own word-template for the report and structure the report according to this self-evalua-

tion template. Use Times New Roman 12 and single-spaced text. Include a front page and an index at

the start of the report.

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

1. Strategy, organization, management and economy

1.1

1.1.1

History and governance structure

Overview of the history of the centre

Please give a brief overview of the history of the centre, including a description of when,

how and why the centre was established and the most important changes in the organi-

sational setup and basic funding since the establishment.

1.1.2

The overall governance setup of the centre

Please describe which authority holds the contract for the centre and the centre’s con-

tractual relationship with this authority

1.1.3

1.1.4

How does the centre seek advice and support about its vision/mission and strategy?

Does the centre have and use a board

If the centre has a board or similar structure, please provide information about:

a) The role of the board

b) The election of members, their obligations, their compensation

c) How and how often the board is involved in the centre’s activities (i.e.

through regular meetings, ad-hoc consultations etc.)

d) One or more example(s) illustrating the role of the board for the centre.

If the centre does not have a board, please provide reflections upon this choice.

1.1.5

Does the centre have and use an advisory board

If the centre has an advisory board or similar structure, please provide information about:

a) The role of the advisory board

b) The election of members, their obligations, compensation

c) How and how often the advisory board is involved in the centre’s activities (i.e.

through regular meetings, ad-hoc consultations etc.)

d) One or more example(s) illustrating the role of the advisory board for the centre.

If the centre does not have an advisory board, please provide reflections upon this choice

1.1.6

The governance structure’s ability to hold the management of the centre accountable

Board or ’board of directors’ is a group of individuals formally elected by shareholders to oversee or

manage an organization

An advisory board is a group of individuals that provides non-binding strategic advice to the manage-

ment of an organization.

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Please elaborate on the ways in which the described governance structure enables ac-

countability.

Please reflect: Which opportunities for improvement do you see?

1.1.7

Relevant documents and additional information

Please provide documents and information that you consider relevant for the evaluation

of the centre’s governance structure.

1.2

1.2.1

Management and organization

Describe the centre’s organizational and economic relationship to each of the following organi-

zations:

The hosting organization, The Copenhagen University Hospital (‘Rigshospitalet’)

The other of the two centres

The three Danish Cochrane Review Groups

The Cochrane Organization

1.2.2

Which managerial positions exist at the centre and what are their tasks and responsibilities?

Please provide a description of all managerial positions of the centre, including the follow-

ing topics:

a) Title and managerial position

b) Primary task(s)

c) Staff responsibility (yes/no), If yes: number of staff members

1.2.3

How is the centre organized?

Please provide organizational diagram and/or description of the hierarchical structure of

the centre, illustrating the organization’s division of labour.

1.2.4

How does the centre provide organizational support for its researchers?

Please describe the support functions for researchers available within the organization.

Please describe the support functions for researchers made available by the centre out-

side of the organization.

1.3

1.3.1

Vision and strategy

What is the centre’s vision, mission and/or goals?

Please describe the centre’s vision, mission and/or goals of the centre

1.3.2

What is the centre’s strategy?

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Please describe the centre’s strategy, as well as the development process resulting in the

formulation of the centre’s strategy and the time period of the strategic plans and how fre-

quently are they assessed

1.3.3

How are the vision and strategy anchored in the centre’s activities?

Please describe the organizational activities specifically conducted to reflect upon, de-

velop or adjust the centre’s achievements of goals and strategy

1.3.4

Relevant documents and additional information

Please provide documents and documentation relevant to review the centre’s vision/mis-

sion and strategy, including but not limited to annual reports for the last five years

1.4

1.4.1

Economy and budget

How is the centre funded?

Please provide information about funding sources for the last five years, including both

government funding and other sources.

Please report in million DKK in 2018-prices with two decimals. Use figures from final ac-

counts if available. For years, where final accounts are not available, use budgets and

mark the year with a ‘B’.

In the first table, please exclude any funds for Danish Cochrane Review Groups affiliated

with the centre

Table 1.1. The Centre’s revenues 2014-2018 (excluding Cochrane Review Groups affiliated with

the centre)

Million DKK 2018 prices

\ year

Government grant (National Fi-

nance Act)

Research councils and research

funds

The Copenhagen University Hos-

pital (‘Rigshospitalet’)

Other revenues (including reim-

bursement of wage expenses by

external collaboration partners)

Total revenues

2014

2015

2016

2017

2018

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Please describe the source of the most important “other revenue” sources

Please provide the same information for the Danish Cochrane Review Groups affiliated

with the centre

Table 1.2. Revenues for Danish Cochrane Review Groups affiliated with the centre

Million DKK 2018 prices

\ year

Government grant (National Fi-

nance Act)

Research councils and research

funds

The Copenhagen University Hos-

pital (‘Rigshospitalet’)

Other revenues (including reim-

bursement of wage expenses by

external collaboration partners)

Total revenues

2014

2015

2016

2017

2018

Please provide information about the distribution of expenditures for the last five years,

including both wage costs and other operating costs.

Please report in million DKK in 2018-prices with two decimals. Use figures from final ac-

counts if available. For years where final accounts are not available use budgets and

mark the year with a ‘B’.

In the first table, please exclude expenditures for Danish Cochrane Review Groups affili-

ated with the centre

Table 1.3. The Centre’s expenditures 2014-2018 (excluding Cochrane Review Groups affiliated

with the centre)

Million DKK 2018 prices

\ year

Wage expenses:

Researchers, including manage-

ment (excluding PhD’s)

PhD’s

Administrative functions

Other employees

2014

2015

2016

2017

2018

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Reimbursement for employees on

sickness leave or maternity/pa-

rental leave (revenue)

Total wage expenses

Operating expenses:

Premises (rent, utility ex-

penses for water, heating etc.)

Office expenses (IT equip-

ment, printers, paper etc.)

Conferences and other aca-

demic activities

Other operating expenses

Total operating expenses

Total expenditures (wages

and operating expenses)

Table 1.4. Expenditures 2014-2018 for Cochrane Review Groups affiliated with the centre

Million DKK 2018 prices

\ year

Wage expenses:

Researchers, including manage-

ment (excluding PhD’s)

PhD’s

Administrative functions

Other employees

Reimbursement for employees on

sickness leave or maternity/pa-

rental leave (revenue)

Total wage expenses

Operating expenses:

2014

2015

2016

2017

2018

Premises (rent, utility ex-

penses for water, heating etc.)

Office expenses (IT equip-

ment, printers, paper etc.)

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Conferences and other aca-

demic activities

Other operating expenses

Total operating expenses

Total expenditures (wages

and operating expenses)

Please report the number of employees

(Full Time Equivalents/FTE)

at the centre for the

last five years.

In the first table, please exclude employees at Danish Cochrane Review Groups affiliated

with the centre

Table 1.5. Employees (FTE) at the centre (excluding employees at Danish Cochrane Review

Groups affiliated with the centre)

Full Time Equivalent/FTE

\ year

Researchers, including manage-

ment (excluding PhD’s)

PhD’s

Administrative functions

Other employees

Total number of employees

(FTE) excluding Danish

Cochrane Review Groups affili-

ated with the centre

2014

2015

2016

2017

2018

Table 1.6. Employees (FTE) at Danish Cochrane Review Groups affiliated with the centre

Full Time Equivalent/FTE

\ year

Researchers, including manage-

ment (excluding PhD’s)

PhD’s

Administrative functions

Other employees

2014

2015

2016

2017

2018

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Total number of employees

(FTE) at Danish Cochrane Re-

view Groups affiliated with the

centre

1.4.2

How successful is the centre in attracting funds?

Please provide a list of fund applications from the last five years, including the name of

the fund/organization, the applied for amount, status (rejected/granted), and if relevant,

the amount granted.

1.4.3

How is the use of resources linked to the centre’s vision/strategy?

Please describe how the centre allocates resources. Include a description of how it is pri-

oritized which clinical fields are to be the subject of reviews to be conducted in the com-

ing year(s). If relevant, describe how the resource allocation is related to the centre’s

goals and to the dimensioning of workload (Full Time Equivalent/FTE).

1.4.4

How is the financial management of the centre organized and linked to the centre’s vision/strat-

egy?

Please describe how the financial management of the centre is organized. If relevant, de-

scribe how the resource allocation and financial management is related to the centre’s

goals and to the dimensioning of workload (Full Time Equivalent/FTE).

1.4.5

Relevant documents and additional information

Please provide documents and information that you consider relevant for the evaluation

of the centre’s budgetary and financial accountability.

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

2 Research production and quality

2.1

2.1.1

Research production

The centre’s publication list for the last seven years.

In addition, to the accounts of publications below, please provide a full list of all publications for the past

seven years (01.01.2012-31.12.2018) for VIVEs analysis of citations. The list should be based on

PURE, and exported from PURE in CSV file. It is expected that all publications are available in PURE

for each researcher. If the publications are not found in PURE, they are not counted in the final evalua-

tion.

2.1.2

The centre’s research production for the last five years?

Please count the yearly scientific output of PURE publications for the last five years in the main catego-

ries of publications presented in the table below. Please provide the count for each year from 2014-

2018.

In the first table, please exclude publications by Danish Cochrane Review Groups affiliated with the cen-

tre

Table 2.1 Number of publications (excluding publications by Danish Cochrane Review Groups affiliated with the

centre)

Publication type \ year

Peer-reviewed journal articles

New Cochrane Reviews

Updated Cochrane Reviews

Cochrane Protocols

PhD/doctoral dissertations

Contributions to books (books

and book chapters)

Conference contributions

Other publications and none-

peer-reviewed dissemination of

research (reports, letters, com-

ments etc.)

Total (excluding publications

by Danish Cochrane Review

Groups affiliated with the cen-

tre)

2014

2015

2016

2017

2018

Total

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Table 2.2 Number of publications by Danish Cochrane Review Groups affiliated with the centre

Publication type \ year

Peer-reviewed journal articles

New Cochrane Reviews

Updated Cochrane Reviews

Cochrane Protocols

PhD/doctoral dissertations

Contributions to books (books

and book chapters)

Conference contributions

Other publications and none-

peer-reviewed dissemination of

research (reports, letters, com-

ments etc.)

Total (by Danish Cochrane

Review Groups affiliated with

the centre)

2014

2015

2016

2017

2018

Total

2.1.3

Number and type of studies in Cochrane Reviews

If relevant, please reflect upon trends over the last five years in the number and the type of studies in-

cluded in each Cochrane Review as well as the overall complexity of the reviews.

2.1.4

Prioritizing of Cochrane Reviews

Please provide information about how Cochrane Reviews are prioritised and who is involved in setting

the review questions

2.1.5

Teaching activities

If relevant, please account for teaching activities of researchers at the centre over the last five years

2.1.6

Training activities

If relevant, please account for training activities at the centre over the last five years

2.1.7

Publication productivity

For the centre at large, please provide the average number of publications per researcher and the aver-

age number of publications per million of the budget for the last five years. Please use the information

on total publications in

Table 2.1

divided with the number of researchers from

Table 1.5

and the funding

from government grant and total funding in

Table 1.1.

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

In the first table, please exclude publications, employees and budgets of Danish Cochrane Review

Groups affiliated with the centre

Table 2.3 Average number of publications per researcher and per million DKK (excluding publications, personnel

and budgets of Danish Cochrane Review Groups affiliated with the centre)

2014

Average number of publications

per employed researcher (FTE),

including management

Average number of publications

per 1 million DKK of government

grant

Average number of publications

per 1 million DKK of total funding

2015

2016

2017

2018

Please provide the same information for Danish Cochrane Review Groups affiliated with the centre,

based on the information on total publications in

Table 2.2,

the number of researchers from

Table 1.6

and the funding from government grant and total funding in

Table 1.2.

Table 2.4 Average number of publications per researcher and per million DKK for Danish Cochrane Review Groups

affiliated with the centre

2014

Average number of publications

per employed researcher (FTE),

including management

Average number of publications

per 1 million DKK of government

grant

Average number of publications

per 1 million DKK of total funding

2015

2016

2017

2018

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

2.2

2.2.1

Research Quality

Journals of publication

Please provide a list of the peer-reviewed journals in which the centre’s researchers have published arti-

cles in the last five years and count the number of articles published in each of these journals for each

of the last five years in a table like the one below.

In the first table, please exclude publications of Danish Cochrane Review Groups affiliated with the cen-

tre

Table 2.5 Number of published peer-reviewed articles by journal (excluding publications of Danish Cochrane Re-

view Groups affiliated with the centre)

Journal name \ year

Journal 1 [name of journal]

Journal 2 [name of journal]

…

Journal N [name of journal]

Total number of peer-re-

viewed articles

2014

2015

2016

2017

2018

Total for

all years

Please describe which international journals you consider as the top journals for the centre’s research

and provide information on potential strategies underlying the distribution of articles in the table above

and/or for future publications in peer-reviewed journals.

Please provide the same information for publications by employees at Danish Cochrane Review Groups

affiliated with the centre

Table 2.6 Number of published peer-reviewed articles by employees at Danish Cochrane Review Groups affiliated

with the centre

Journal name \ year

Journal 1 [name of journal]

Journal 2 [name of journal]

…

Journal N [name of journal]

Total number of peer-re-

viewed articles

2014

2015

2016

2017

2018

Total for

all years

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

2.2.2

Authorship order as indicator of involvement in research production

In health sciences, the first and last authorships are often considered the most prestigious. Please re-

flect whether this sort of reasoning is reflecting the way the authors are listed in the peer-reviewed arti-

cles published by your centre or other principles of authorship order are prominent.

2.3

2.3.1

PhD. activity

Phd Supervision responsibility

Please provide information whether the centre is expected to provide supervision for PhDs and if so,

how this supervision is provided

2.3.2

The number of PhD. projects that are initiated, ongoing, and defended dissertations

Please count the activity of PhDs for the last five years according to the three categories: PhDs that

have been initiated, PhDs that are ongoing and PhDs that are publicly defended (dissertations). Please

provide the number for each year in a table like the one below.

Table 2.7 PhD projects (excluding Danish Cochrane Review Groups affiliated with the centre)

Category \ year

PhDs initiated

PhDs ongoing

PhD Dissertations

Total

2014

2015

2016

2017

2018

Total

2.3.3

Affiliation of PhD fellows following the dissertation

Please provide the number of PhD fellows that are employed at the centre following the public defence

and PhDs that are no longer affiliated with the centre. This should be provided in table as the one be-

low. Please also provide a list of the specific affiliations of the PhDs that are not affiliated at the centre

after the PhD defence.

Table 2.8 Affiliation of PhD fellows (excluding Danish Cochrane Review Groups affiliated with the centre)

Category \ year

PhDs employed at the

centre following the de-

fence

PhDs employed other

places following the de-

fence

2014

2015

2016

2017

2018

Total

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Table 2.9 Employment of PhD fellows not affiliated with the centre after the Ph.D. defence (excluding Danish

Cochrane Review Groups affiliated with the centre)

Name of employment place following the public defence

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

3 Impact and relevance

3.1.1

Who are the centre’s primary audiences/target groups in the Danish society?

Please list the organizations that the centre considers its primary target group.

3.1.2

Please describe the ways in which the centres’ research and activities are used by and creates

benefits in various parts of

the Danish society.

Include at least the following topics:

•

Examples of potential impact on clinical practices and clinical guidelines within the past

five years

Collaborations with public authorities within the past three years

Collaboration with hospitals and healthcare professionals within the past three years

Participation in consulting, commissions, boards etc. within the past three years

Organization of conferences, symposiums or other knowledge sharing initiatives within

the past three years

Other ways the centres’ research and activities are used by and creates benefits in the

Danish society within the past three years

For each of these topics please provide examples or documentation.

3.1.3

Visitors, secondments etc. at the centre in the past three years?

Please provide information on the number of visitors, secondments etc. at the centre in the past

three years.

3.1.4

Web hits, downloads of Cochrane reviews and media activity

Please provide information on



3.1.5

the number of web hits for your centre per year in the past three years

the number of Cochrane reviews, that have been downloaded by Danish IP-addresses

(.dk) in the past three years

the number of times employees at the centre have been cited in Danish media in the

past three years

User-friendly summaries of reviews and other tailored research knowledge

Please reflect upon the centre’s policy for making user-friendly summaries of reviews or other

tailored written versions of publications to communicate research knowledge to the primary tar-

get groups in the Danish society

3.1.6

The centre’s impact on the public debate in Denmark?

Please reflect upon the centre’s impact on the public debate in Denmark and give examples

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

4 Collaborations and partnerships

4.1

4.1.1

National collaborations and partnerships

The centre’s formal collaborations and partnerships in Denmark

Please list the centre’s national collaborations during the last five years.

Please describe the role(s) of the centre in each of its national collaborations.

Please describe the achieved output of each of the national collaborations.

4.2

4.2.1

International collaborations and partnerships

The centre’s formal collaborations and partnerships internationally

Please list the centre’s international collaborations during the last five years

Please describe the role(s) of the centre in each of its international collaborations.

Please describe the achieved output of each of the international collaborations.

4.3

4.3.1

Own assessment of national and international collaborations

How does the centre’s collaborations cohere with the centre’s goals/vision and strategy?

Please reflect upon the importance of the centre’s collaborations for accomplishing the centre’s

goals/vision and for succeeding with its strategy.

Please reflect upon which areas of improvement you see, e.g. should the balance between na-

tional/international be changed; should output, focus, size of collaborations be different?

Collaborations and partnerships refer to arrangements where there is a formalized written agreement

on e.g. the content of the collaboration, the division of labor and the duration of the collaboration.

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Bilag 3

Evaluation Report Nordic Cochrane Cen-

tre

Dette bilag indeholder den evalueringsrapport, som det internationale evalueringspanel har

udarbejdet for det Nordiske Cochrane Center (NCC).

Ud over evalueringspanelets evaluering af det Nordiske Cochrane Center indeholder evalue-

ringsrapporten også panelets evalueringsperspektiver på de to danske Cochrane Review-

grupper, som er tættest knyttet til det Nordiske Cochrane Center:



The Colorectal Group (CCG),

The Anaesthesia Group and the Emergency and Critical Care Group (ACE).

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Evaluation of the Nordic Cochrane Centre

Final Report

Evalueringspanelets evalueringsrapport for Nordic Cochrane Centre

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Amanda Sowden

(Chair)

Professor and Deputy Director and leader of the University of York’s Centre for Reviews and Dissemination’s

research programme in public health, United Kingdom

Terje P. Hagen

(Co-chair)

Professor and Head at the Institute of Health and Society, University of Oslo, Norway

Diederick Grobbee

Professor of Clinical Epidemiology and founder of the Julius Center, University Medical Center Utrecht, The

Netherlands

Eva Swahn

Professor at the Department of Medical and Health Sciences, Division of Cardiology, Linköping University &

Department of Cardiology, University Hospital, Linköping, Sweden

Merete Osler

Clinical Professor at the Department of Public Health, University of Copenhagen & consultant at the Center for

Clinical Research and Prevention, Bispebjerg and Frederiksberg Hospital, Denmark

Mickael Bech

Professor in healthcare leadership at the Crown Prince Frederik Center for Public Leadership, Aarhus

University, Denmark

September 2019

Evalueringspanelets evalueringsrapport for Nordic Cochrane Centre

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Table of Contents

Executive summary ............................................................................................ 1

Background of the evaluation ................................................................................. 1

Conclusions of the Evaluation Panel ...................................................................... 1

Introduction ............................................................................................... 3

1.1

1.2

1.3

1.4

1.5

Background ................................................................................................... 3

The evaluation .............................................................................................. 3

The evaluation panel .................................................................................... 4

Evaluation protocol ....................................................................................... 5

The evaluation process and methods ........................................................... 6

Management and organisation .................................................................. 7

2.1

2.2

2.3

2.4

2.5

Background, current position and funding .................................................... 7

Purpose and strategy .................................................................................... 8

Organisation and management .................................................................... 9

Governance arrangements ........................................................................... 9

The Panel’s assessment of the overall management and organisation ..... 10

Research production and quality ............................................................. 12

3.1

3.2

Research production and impact of publications ........................................ 12

The Panel’s overall assessment of research productivity and quality ........ 13

3.2 PhD supervision .............................................................................................. 13

Relevance and impact ............................................................................ 15

4.1

4.2

4.3

4.4

4.5

Primary audiences and target groups (stakeholders) ................................. 15

Setting priorities .......................................................................................... 15

Activities that lead to impact ....................................................................... 15

Contributions to public debate .................................................................... 16

The Panel’s overall assessment of relevance and impact.......................... 17

Collaborations and partnerships.............................................................. 18

5.1

5.2

5.3

Overview of national and international collaborations and partnerships .... 18

Cochrane partnerships ............................................................................... 18

The Panel’s overall assessment of collaborations and partnerships .......... 19

Options for organising Cochrane activities .............................................. 20

6.1

6.2

Summary of the evaluation ......................................................................... 20

Options ........................................................................................................ 20

Cochrane Review Groups (CRGs) .......................................................... 22

7.1

7.2

7.3

7.4

History and background .............................................................................. 22

Organizational arrangements ..................................................................... 23

Research production ................................................................................... 23

The Panel’s overall reflections on Cochrane Review Groups (CRGs) ....... 26

Evalueringspanelets evalueringsrapport for Nordic Cochrane Centre

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Executive summary

Background of the evaluation

The Nordic Cochrane Centre (NCC) and the Copenhagen Trial Unit (CTU) receive funding from

the Ministry of Health via the National Finance Act. This funding has been in place since 1993

and, until now, no formal evaluation of the work of the Centres has taken place. Such an

evaluation was requested by the Ministry of Health and was organised and overseen by VIVE,

the Danish Center for Social Science Research. The aim of the evaluation was to assess

whether the purpose of the two centres as stated in the National Finance Act grant has been

fulfilled. The Finance Act states:

“The activities in the Cochrane area include preparation of systematic reviews of

health care interventions, prevention of diseases, diagnostics, treatment and care.

CTU supports and performs scientifically relevant clinical trials, i.a. as part of the

Cochrane Collaboration.”

(the National Finance Act 2018, translation by VIVE).

An international Panel of six people carried out the evaluation. The evaluation focused on i)

management and organisation of the centres; ii) research production and quality; iii) impact

and relevance of the centres’ activities and iv) collaborations and partnerships.

The results of the evaluation are presented in separate reports for each of the two centres. This

report presents the results of the evaluation of the NCC. The report also includes perspectives

on the Cochrane Anaesthesia, Cochrane Emergency and Critical Care and Cochrane

Colorectal Groups, which are also funded under the Finance Act.

Conclusions of the Evaluation Panel

The Panel concludes that the NCC’s overall purpose of informing health care decision-making

through the provision of high-quality synthesised evidence has been fulfilled. The Panel notes

the significant contribution made to improving the quality of evidence available to decision-

makers, including patients and the public. This has been achieved by developing robust

methods and advocating for their use through publication in high-impact journals and through

training. The NCC’s work is of clear relevance to Danish health care practices and there are

examples of significant influence on policy.

Some of the NCC’s work has received considerable media interest and has without doubt

informed public debate in Denmark and internationally. In turn, this is likely to bring benefits to

society by helping to drive science and clinical practice forward. Stakeholder feedback

suggests that the NCC has a good reputation, which is based on the quality, independence

and importance of its work and its anchoring to Cochrane’s overall mission. The Panel notes

the controversy surrounding some of the scientific and public debates, which have split the

opinions of policy-makers and the research community. The Panel see that there is a need for

increased engagement with stakeholders and to re-establish the Advisory Board, with a

refreshed membership, to include people engaged with the health policy and practice agenda

in Denmark.

Active collaboration with researchers nationally and internationally has resulted in high-quality

publications and outputs, notably reporting standards, Cochrane reviews, and clinical

Evalueringspanelets evalueringsrapport for Nordic Cochrane Centre

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

guidelines. By comparison, collaboration with the Denmark-based Cochrane Review Groups

(CRGs) appears to be very limited. Given the increasing complexity of Cochrane methods, the

need to adhere to methodological standards and editorial processes, and to ensure stakeholder

involvement in priority setting, the Panel concludes that the NCC and the Denmark-based

CRGs should develop closer working relationships, with opportunities for shared learning as a

priority.

Despite the NCC’s considerable achievements, the Panel questions the sustainability of its

current operating model, given its small size, dependence on Ministry funding, positioning in

Rigshospitalet and complex governance structures. The Panel concludes that there is an

urgent need to consider the options for a revised governance structure that would strengthen

the NCC’s current position and facilitate its future growth. Closer alignment with a university

would seem desirable, in bringing further potential for collaboration and increased opportunities

to secure research funding and recruit PhD students, as well as signalling its national scope,

while maintaining the research integrity and independence that is central to its operation and

strategy.

Evalueringspanelets evalueringsrapport for Nordic Cochrane Centre

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

1.1

Introduction

Background

The Ministry of Health has asked VIVE, The Danish Center for Social Science Research, to

evaluate the Nordic Cochrane Centre (NCC) and the Copenhagen Trial Unit (CTU).

The NCC and CTU receive a grant each year through the National Finance Act. In the National

Finance Act for 2018, it is stated:

”In 2018, DKK 17.8 million were allocated for financing of the Cochrane and

Copenhagen Trial Unit (CTU). The activities in the Cochrane area include

preparation of systematic reviews of health care interventions, prevention of

diseases, diagnostics, treatment and care. CTU supports and performs scientifically

relevant clinical trials, i.a. as part of the Cochrane Collaboration. The allocated funds

from 2009 onwards include DKK 0.5 million for the financing of free access for all to

the Cochrane Library”

(the National Finance Act 2018, translation by VIVE).

According to information from the centres, the 17.8 million DKK in 2018 were distributed as

follows:

•

The Nordic Cochrane Centre: 7,335,000 DKK

Copenhagen Trial Unit, including the Cochrane Hepato-Biliary Group: 7,735,000

The Cochrane Review Groups: 2,730,000 DKK

Herlev Hospital: 1.316.000 DKK (The Cochrane Anaesthesia and Cochrane

Emergency and Critical Care Groups)

Bispebjerg Hospital: 1,413,000 DKK (The Cochrane Colorectal Group).

1.2

The evaluation

The results of the evaluation are presented in separate reports for each of the two centres. This

report presents the results of the evaluation of the Nordic Cochrane Centre (NCC). In addition

to the NCC, the report includes perspectives on the Cochrane Anaesthesia and Cochrane

Emergency and Critical Care Groups and the Cochrane Colorectal Group.

The main aim of the evaluation is to assess whether the purpose of the National Finance Act

grant has been fulfilled, by evaluating the scale of research production and its quality as well

as the societal impact of the centres and their external collaborations and partnerships.

Accordingly, the evaluation includes both formative elements focusing on broadening the

understanding of the institute's characteristics for the purpose of continuous improvement and

summative elements focusing on accountability to the entity that funds the running of the

centre.

Specifically, the evaluation addresses the following four topics:

•

Evalueringspanelets evalueringsrapport for Nordic Cochrane Centre

Management and organisation of the centres

Research production and quality

Impact and relevance of the centres’ activities

Collaborations and partnerships.

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

These topics are presented in further details below.

1.3

The evaluation panel

An international evaluation panel has carried out the evaluation and is responsible for writing

this report. The appointment of the chair of the evaluation panel was done by The Independent

Research Fund Denmark [Danish name: Danmarks Frie Forskningsfond, DFF] based on a

nomination by VIVE. VIVE appointed the additional members of the evaluation panel based on

nominations by DFF. VIVE requested DFF to nominate three internationally well-acknowledged

researchers from the medical sciences council under DFF and two researchers from the social

science council under DFF. The candidates from the medical sciences council were to have

insight into the processes and methods of randomised studies, systematic reviews and

experience with assessing the quality and impact of health care research. The candidates from

the social sciences council under DFF were to have insight into the Danish health care system

and management and organisation of health research. In order to ensure the impartiality of the

evaluation, a basic criterion in the nomination of candidates, in addition to the professional

skills, was that they must not have had collaborations or other close relations with the NCC,

CTU or their international counterparts in the last five years. DFF, in accordance with the

Danish Gender Equality Act, nominates at least one male and one female candidate for each

seat in the panel. After a process of nomination and invitation of nominees by DFF, VIVE

received a list from DFF of five researchers from the medical sciences council and two

researchers from the social sciences council, all of whom accepted the invitation. The

appointment of the additional members was done by VIVE, after consulting the chair of the

evaluation panel. Based on an overall assessment of the individual and complementary

qualifications, VIVE appointed three of the five researchers who accepted the invitation from

the medical sciences council and the two researchers that accepted the invitation from the

social sciences council. All panel members were appointed in their own personal capacity. The

composition of panel members has been subject to a hearing by the NCC and the CTU.

The panel consists of six members with complementary competencies in reviews in public

health, clinical research, trials and management, and organisation of health research:

•

Amanda Sowden (Chair). Professor and Deputy Director and leader of the University

of York’s Centre for Reviews and Dissemination’s research programme in public

health, United Kingdom

Terje P. Hagen (Co-chair). Professor and Head at the Institute of Health and Society,

University of Oslo, Norway

Diederick Grobbee. Professor of Clinical Epidemiology and founder of the Julius

Center, University Medical Center Utrecht, The Netherlands

Eva Swahn. Professor at the Department of Medical and Health Sciences, Division of

Cardiology, Linköping University & the Department of Cardiology, University Hospital,

Linköping, Sweden

Merete Osler. Clinical Professor at the Department of Public Health, University of

Copenhagen & consultant at the Center for Clinical Research and Prevention,

Bispebjerg and Frederiksberg Hospital, Denmark

Mickael Bech. Professor in healthcare leadership at Crown Prince Frederik Center for

Public Leadership, Aarhus University, Denmark.

•

Evalueringspanelets evalueringsrapport for Nordic Cochrane Centre

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

1.4

Evaluation protocol

The evaluation protocol elaborated by VIVE and the chair of the panel covers four topics:

•

Management and organisation

Research production and quality

Impact and relevance

Collaborations and partnerships.

The protocol contains the questions guiding the evaluation panel’s assessment and is

presented in further detail below.

Management and organisation

The evaluation panel was asked to give a general description and assessment of the

governance structure and the management and organizational structure of each of the two

centres. The description and assessment were to address the following:

•

An overall description and assessment of the governance structure of the centre,

including reflections on the governance structure’s ability to hold the management of

the centre accountable

A reflection on the strategy of the centre in relation to the purpose and mission of the

centre

A description and assessment of the management and organizational structure of the

centre, including reflections on the organizational and management structures’

adequacy for accomplishing the centre´s purpose and strategy, and the expediency of

the decision-making structure and the division of labour between the different sections

of the centre

An assessment of the organizational support for researchers

An assessment of the centre’s ability to attract external resources from research

councils, research funds, EU etc.

A reflection on the expediency of governance structure, the management and

organizational structure compared to similar types of institutions.

•

Research production and quality

The panel was asked to give an assessment of the quantity and quality of research from each

of the two centres. The assessment was to include the following elements:

•

A count of publications over the past five years divided into main types of publications

The average number of annual publications per researcher for the centre at large

An assessment of the quality of research based on bibliometric indicators

Reflections on the level and quality of output compared to economic resources

An assessment of the centres’ production of PhD candidates, including reflections on

the centres’ involvement in PhD programmes

An overall assessment of the productivity of the centre and the importance and quality

of the research.

Evalueringspanelets evalueringsrapport for Nordic Cochrane Centre

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Impact and relevance

The evaluation panel was asked to give its assessment of the impact and relevance of each of

the two centres’ activities for the Danish society. The assessment was to include the following

elements:

•

An assessment of the impact of the centre on clinical practice and clinical guidelines.

Reflections on how the centres’ research and other activities create value in various

parts of the Danish society

Reflections on the centres’ impact on the public debate in Denmark

Collaborations and partnerships

The evaluation panel was asked to give its assessment of each of the two centre’s national

and international collaborations and partnerships. The assessment was to include the following

elements:

•

An assessment of the level of national and international collaboration of the centre,

including reflections on the level of national vs. international collaborations, and the

role of the centre in the collaborations

An assessment of the output of the collaborations

Reflections on importance of the collaboration for accomplishing the centre´s purpose

and strategy.

•

1.5

The evaluation process and methods

The evaluation of the two centres was carried out in parallel and is based on both quantitative

and qualitative data. The data include:

•

Self-evaluation reports by the NCC and CTU

Site-visit by the evaluation panel in Copenhagen 20

to 22

May 2019

Documentation and additional data collected by VIVE

Both the self-evaluation reports, and the documentation and additional data collected by VIVE,

provide information on each of the four topics of the evaluation. The site-visit included meetings

with management, employees and a selected number of stakeholders of each centre as well

as a meeting with the management of the hosting organisation, The Danish National Hospital

– ‘Rigshospitalet’ and a teleconference with the Chief Executive Officer of Cochrane.

Evalueringspanelets evalueringsrapport for Nordic Cochrane Centre

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Management and organisation

This section provides a brief overview of the history of the Nordic Cochrane Centre (NCC), its

current position and funding arrangements, its purpose and strategy, and its organizational and

governance structures.

2.1

Background, current position and funding

The NCC was established in 1993 by Professor Gøtzsche at Rigshospitalet in Copenhagen.

The international Cochrane Collaboration (now Cochrane) was itself founded in the same year.

Cochrane is an international organisation, with over 13,000 members from more than 130

countries, whose mission is to promote evidence-informed health decision-making by

producing high-quality, relevant and accessible systematic reviews and other forms of

synthesised research evidence.

The NCC was originally funded via the organisation Hovedstadens Sygehusfællesskab (HS:

The hospital community of the Capital Region). As part of the 2006 structural reforms in

Denmark, the organisation HS was merged into one of five regions, Capital Region of Denmark,

and the financial support for the NCC was transferred to the national Danish Finance Act, which

has continued to provide funding. In 2018, the amount of funding from the Danish Government

was DKK 7.335 million. Funding from other sources (e.g. research councils and research

funds) for 2018 amounted to DKK 0.71 million. In 2018, funding from the Danish Government

comprised 87% of the NCC’s total budget. In earlier years, this percentage ranged from 58

(2014) to 99 (2017).

Several Associate Centres have been set up, which are directly affiliated with the NCC. A

Norwegian Branch was established in 1997, and in 2017 Polish and Russian Branches were

established. Cochrane Finland was established in 1997, and Cochrane Sweden in 2017. In

addition, several Cochrane groups also refer to the NCC:

•

the Cochrane Hepato-biliary Group (DK)

the Cochrane Colorectal Group (DK)*

the Cochrane Anaesthesia Group (DK)*

the Cochrane Emergency and Critical Care Group (DK)*

the Occupational Health Group (Fin)

the Norwegian Satellite of the Effective Practice and Organization of Care Group (No)

the Cochrane Bias Methods Group (DK).

Three of these groups* are funded via the Danish Finance Act. In 2018, this funding amounted

to DKK 2.729 million, which is in addition to the funding for the NCC.

The NCC operates as a research department within Rigshospitalet, and all financial actions go

through the finance office and administrative issues through the Human Resources Department

at Rigshospitalet. Payments by the Danish Government are made directly to Rigshospitalet.

Funding for the three* Cochrane groups is administered by Herlev Hospital and Bispebjerg

Hospital, which host the editorial bases for these groups.

The NCC and the Copenhagen Trial Unit (CTU) are connected through their collective

government funding and are housed next door to each other at Rigshospitalet, but their

Evalueringspanelets evalueringsrapport for Nordic Cochrane Centre

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

activities are completely independent of each other. The CTU hosts the editorial base of the

Cochrane Hepato-Biliary Group, including its Co-ordinating and Managing Editors and

Information Specialist (see Evaluation of the Copenhagen Trial Unit – Report).

Also based at Rigshospitalet, and co-located with the NCC, is a small group who are part of

Cochrane’s Informatics and Technology Services. This service is centrally managed via the

Cochrane Central Executive team based in London, UK.

2.2

Purpose and strategy

The overall aim of the NCC is “to help citizens, patients, health care professionals and payers

of health care services to choose – or to avoid use of – interventions rationally, in an evidence-

based fashion, and with a focus on benefits, harms and costs.”

This aligns with Cochrane’s

Strategy to 2020

, which aims to put Cochrane evidence at the heart of health decision-making

all over the world and has the following goals:

1. Producing evidence –

to produce high-quality, relevant, up-to-date systematic reviews

and other synthesised research evidence to inform health decision-making.

2. Making our evidence accessible –

to make Cochrane evidence accessible and useful

to everybody, everywhere in the world.

3. Advocating for evidence –

to make Cochrane the ‘home of evidence’ to inform health

decision-making, build greater recognition of our work, and become the leading

advocate for evidence-informed health care.

4. Building an effective sustainable organisation –

to be a diverse, inclusive and

transparent international organisation that effectively harnesses the enthusiasm and

skills of our contributors, is guided by our principles, governed accountably, managed

efficiently and makes optimal use of its resources.

The NCC has five main areas of work, as set out in their

Strategic Plan 2016-2020:

1. Research

– priority is given to research that makes a difference to many people.

Specific areas of interest are psychiatric drugs and screening. Methodological research

that aims to elucidate the sources of bias is also a priority. Work will continue to improve

the reliability of clinical and observational studies and to develop and update reporting

guidelines.

2. Dissemination of research results

– priority is given to the translation of review

findings into understandable formats, including for patients and the public.

3. Associate centres and review groups

– the focus is on providing support to and close

working with affiliated centres and groups (currently in Denmark, Norway, Finland,

Sweden, Russia and Poland).

4. Partnerships

– priority is given to informal partnerships to ensure full academic

freedom.

5. Workshops and courses

– priority is given to supporting individuals in Denmark,

Sweden, Norway, Poland and Russia to complete Cochrane protocols and reviews.

Taken from

Strategic Plan 2016-2020: Nordic Cochrane Centre.

Taken from

Strategy to 2020 (Cochrane).

Evalueringspanelets evalueringsrapport for Nordic Cochrane Centre

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

2.3

Organisation and management

Full-time research positions are currently held by the Acting Director and a Senior Researcher.

In addition, the NCC employs a communications consultant, an administrator and a secretary,

plus four full-time PhD students and early career researchers, and two PhD students who are

shared with the Centre for Evidence-based Medicine (CEBMO) at the University of Southern

Denmark and Odense University Hospital. In addition, the Centre hosts visiting scholars and

students. NCC staff view their core strengths as being research synthesis and methodological

expertise.

In order to supervise PhD students and contribute to teaching, it is necessary for one or more

members of NCC staff to have a university affiliation. The PhD students are employed at the

NCC and they are enrolled in university PhD programmes. The former Director was affiliated

with the University of Copenhagen, through a Professorship and had the overall responsibility

for the NCC-funded PhD students. None of the NCC’s senior staff currently have a university

affiliation. Interim arrangements for the current PhD students are in place.

2.4

Governance arrangements

Cochrane

– The NCC holds the status of a ’Cochrane Centre’. Cochrane and the NCC have

signed a Collaboration Agreement which sets out the terms and conditions for both parties and

commits the NCC to support Cochrane’s mission, principles, organisational strategies and

goals, as defined by Cochrane’s Governing Board, and to fulfil the core functions of a Cochrane

Centre, as set out in

“Implementing Strategy to 2020: Cochrane Centres, Branches & Networks

– New functions and Structures”.

The Cochrane Governing Board is responsible for overseeing

the development and implementation of Cochrane’s strategic direction. The Acting Director of

the NCC is an elected member of the Cochrane Governing Board.

Cochrane Centres are directly accountable to the CEO of Cochrane to ensure adherence to

Cochrane’s overall strategy, and Centre Directors meet quarterly with Cochrane’s CEO to

review progress in relation to their own strategic plans.

Centres have coordinating

responsibility for national (and sometimes regional) work plans, and the NCC represents

Cochrane in the Nordic countries. The groups and affiliated centres in the Nordic Region report

directly to the NCC.

NCC

– The Centre is characterised by an informal culture, partly due to its small size, and

decisions about work plans are made in dialogue with individuals, rather than through formal

meetings or pre-allocation of resources based on full-time equivalents (FTEs). Prioritisation of

research activity is in line with the direction set out in the centre’s

Strategic Plan 2016-2020,

but ultimately it is the Director who decides which projects to initiate.

The NCC does not have a board of directors, as the Cochrane Governing Board sets the

agenda for the organisation overall. The NCC does have an advisory board; members are

consulted infrequently and on an ad hoc basis.

Danish Government

– The NCC is funded through the Danish Finance Act via the Ministry of

Health. The Danish Finance Act does not pose any requirements in terms of governance or

NCC’s Strategic plan 2015-2020 is available here - https://nordic.cochrane.org/strategic-

plan

Evalueringspanelets evalueringsrapport for Nordic Cochrane Centre

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

accountability. Since its inception in 1993, the NCC has not been formally evaluated, beyond

the monitoring of annual scientific and financial reports.

Rigshospitalet

– The NCC is hosted by Rigshospitalet and is organised as a hospital research

department. Rigshospitalet ensures formal rules and regulations are met, but no explicitly

defined governance structure is in place and the NCC is not accountable to Rigshospitalet for

its activities or decision-making.

University of Copenhagen

– The NCC had an affiliation with the University through the former

Director’s Professorship. The affiliation was a prerequisite for the centre to supervise PhD

students. None of the senior staff currently have affiliations with the University, and senior

researchers outside the NCC (with University affiliations) act as the main supervisors for

current PhD students.

2.5

The Panel’s assessment of the overall management and

organisation

The NCC’s overall purpose to inform health care decision-making through the provision of high-

quality synthesised evidence has been fulfilled. The Centre has made a significant contribution

to the quality of evidence available to decision-makers, including patients and the public, by

developing robust methods and advocating for their use through publication in prestigious

journals and through training.

Given the small size of the NCC, its achievements are impressive. Currently, there are two

senior researchers to develop, lead, oversee and manage the work of the Centre and its staff.

The Panel noted that due to its positioning in Rigshospitalet, the NCC appears to be relatively

isolated from the academic community and this, coupled with its small size, may be

disadvantageous for PhD students and early career researchers, who appear to have limited

opportunities to engage in wider academic pursuits or for career development. Although, it is

noted that PhD students can visit Cochrane Centres and partnering research units in other

countries. PhD students are formally enrolled at the University of Copenhagen, but there seem

to be few interactions with other academic environments. The Panel suggests that a more

formal affiliation with the University of Copenhagen, and/or with the CEBMO at the University

of Southern Denmark, would enhance possibilities for collaboration and joint working. The

Panel notes that Cochrane Centres in other countries are often based in or affiliated with

Universities and therefore benefit from wider academic engagement and collaboration.

The Panel notes that the NCC is almost exclusively dependent on funding from the Danish

Ministry, which could threaten its future sustainability – particularly if, at any point, this grant is

reduced or removed. Additional research grants have been secured, but for relatively small

amounts compared with the Ministry funding. The success in attracting external funding seems

to vary, and the NCC does not appear to have a strategy for actively pursuing research income.

The NCC’s governance arrangements appear intricate, with differing levels of accountability to

Cochrane, Rigshospitalet and the Ministry of Health. The NCC’s affiliation to the University of

Copenhagen (via a Professorship) creates an additional complexity. The Panel’s impression

was one of a somewhat vague governance structure, lacking in accountability and with weak

organisational anchoring. The NCC is answerable to Rigshospitalet for the legal use of the

government grant, but no formal evaluation of its work and activities has been made beyond

the annual monitoring of accounts and scientific outputs. Discussions have taken place

Evalueringspanelets evalueringsrapport for Nordic Cochrane Centre

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

between the Chief Executive Officer (CEO) of Cochrane and the Acting Director of the NCC

about the governance structure, and both acknowledged that clear lines of accountability and

regular communication are important and necessary. In a meeting with the Panel,

Rigshospitalet highlighted the importance of a clear and strong governance structure, while

noting the need for a Cochrane Centre to be an independent entity, which led them to conclude

that an alternative governance structure was needed.

Although the NCC has an advisory board, the Panel felt that the full potential of this group to

influence and shape the Centre’s activities has not been realised. The Panel questioned

whether the membership of the advisory board was sufficiently close to, and engaged with, the

health policy and practice agenda, in Denmark, and whether membership could be refreshed.

Evalueringspanelets evalueringsrapport for Nordic Cochrane Centre

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Research production and quality

This section provides an overview and assessment of the research production and quality.

Various bibliometric indicators were used to inform the assessment. The indicators capture the

involvement and productivity of the NCC in any of the peer-reviewed publications, as well as

the performance of the publications. The analyses are restricted to publications from 2012/14

to 2018 and include only publications indexed in the Web of Science (WoS) database.

This section also includes an overview of the NCC’s involvement in PhD supervision.

3.1

Research production and impact of publications

From 2014 to 2018, the NCC has registered 140 publications, of which 55 (39% of total

publications) are journal articles and 18 (12% of total publications) are Cochrane articles (Table

3.1).

Table 3.1 Number and publication classification of the publications from NCC

WoS -Classification

2012

No.

Article

Cochrane

Editorial

Letter

Review

Total

2013

No.

2014

No.

2015

No.

2016

No.

2017

No.

2018

No.

2014-2018

Total

No.

140

2012-2018

Total

No.

215

Abbreviations: WoS: Web of Science, No.: Number of observations.

Des:

The table shows the NCC’s publications from 2012 to 2018, divided into the five publication categories used in the

analysis.

Between 2014 and 2018, NCC staff had 60 (43% of total publications) first authorships, 77

(55% of total publications) last authorships and 51 co-authorships (data not shown in tables).

Comparing the citations received by the publications to those of the average article published

in the field of clinical medicine, publications from the NCC received 2.93 times more citations

than the average publication (MNCS Clinical Medicine range 1.71 to 3.62, for individual years).

A similar result is found when the NCC publications are compared to an average publication in

all the WoS fields. NCC publications receive 2.88 times more citations than the average article

in all WoS fields (MNCS All fields range from 1.68 to 2.64, for individual years; Table 3.2).

Evalueringspanelets evalueringsrapport for Nordic Cochrane Centre

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Table 3.2 Number of citations and normalised citation score for publications from the

NCC

Number

Mean TC

Median

5.5

Max TC

MNCS

Clinical

Medicine

3.62

3.22

1.71

2.46

3.54

2.93

MNCS All

fields

3.64

3.25

1.68

2.30

3.27

2.88

Missing

Publi-

cations

2014

2015

2016

2017

2018

All years

127

1603

765

298

112

2816

42.18

28.33

10.28

6.22

2.53

22.17

608

571

608

Abbreviations: TC: Total number of citations for the period, MNCS: mean normalised citations score of the publications of a

unit

Des:

The table shows the number of publications included in the analysis. For each publication year, the total, mean,

median and maximum number of citations of the publications in that year are shown. The table further shows the

mean MNCS Clinical medicine and MNCS All fields scores, which indicate whether the publications received more

(scored above 1) or fewer (scored less than 1) citations than the average publications in the relevant field in the given

publication year. It was not possible to obtain citation data for all the NCC publications, for which reason there are

missing data. The most frequent missing publication type is Cochrane protocols.

Note: We apply weights based on the publication’s classification for both MNCS scores. Articles, Cochrane articles and

reviews are given a weight of 1 in the analysis, whereas letters, comments and editorials are given a weight of 0.25

When focusing on articles from the two publication categories ‘articles’ and ‘Cochrane articles

(excluding reviews, letters and editorials)’, there are 69 publications, which on average have a

MNCS Clinical Medicine score of 4.5 (MNCS Clinical Medicine range 2.81 to 5.73, for individual

years) and a MNCS All fields score of 4.44 (MNCS All fields range 2.75 to 5.77, for individual

years). Both results imply that the publications from the NCC receive more than twice as many

citations as an average publication in the two fields of WoS. The MNCS scores are slightly

higher than the scores obtained using all publications categories, which might be explained by

the exclusion of letters and editorials, which do not receive many citations.

3.2 PhD supervision

The NCC considers PhD supervision and mentoring to be a necessary obligation of any

research centre. In their self-evaluation report, the NCC recorded seven PhD students as being

registered in the period 2014-18. In the same period, one PhD Fellow was employed by the

NCC following their defence.

3.2

The Panel’s overall assessment of research productivity and

quality

The Panel concluded that the NCC has produced an impressive number of outputs, with many

judged to be of high quality, as evidenced by publication in high-impact journals, including

BMJ,

Lancet, Ann Intern Med, JAMA,

and

J Clin Epidemiol.

Based on the number of employees at

the NCC (FTE), ranging between 2.4 and 2.77 over the five-year period, this equates to

approximately 20 outputs (all) per year, per researcher (FTE), or approximately five peer-

reviewed journal articles per year, per researcher (FTE).

Many of the publications report methodological research, with the aim of expanding existing,

or developing new, approaches and methods for systematic review – for example, the inclusion

of clinical study reports from drug regulatory authorities. Publications include major reporting

Evalueringspanelets evalueringsrapport for Nordic Cochrane Centre

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

guidelines, such as CONSORT for randomised trials and PRISMA for systematic reviews,

developed as part of international collaborations. The NCC has played an important role in

developing the methods that underpin research synthesis and this is notably one of their main

strengths, as well as being a central aim of their overall strategy.

Of the 17 Cochrane reviews in production over the past five years, five are judged by the NCC

as being ‘complex’. Reviews are considered complex if they differ from standard systematic

reviews in terms of the nature and sources of data and the processes used for synthesis and

interpretation. These reviews often require a multidisciplinary review team with a particular skill

set, especially highly skilled methodological and statistical expertise.

Although the NCC does not have a specific remit to supervise PhD students, the importance

of doctoral research and students was highlighted. Over the five-year period, seven PhD

students were registered. In addition, staff contributed to PhD training at the University of Lund

in Sweden and to courses in evidence-based medicine at the University of Copenhagen, as

well as to workshops and events in Denmark and beyond. The Panel agreed that the NCC

potentially had more to offer with regard to boosting systematic review capacity, through

increased supervision of PhD students.

Evalueringspanelets evalueringsrapport for Nordic Cochrane Centre

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Relevance and impact

This section describes and assesses the relevance and impact of the NCC’s activities in the

context of Cochrane’s overall mission as an international organisation, whose combined

activities have a worldwide focus, which benefits many countries. As such, Danish society

benefits from Cochrane activities carried out in other countries, not just those undertaken in

Denmark.

4.1

Primary audiences and target groups (stakeholders)

The NCC’s primary audiences and target groups include patients, carers and their families;

health professionals; policy-makers and other decision-makers; and researchers and

academics, guideline producers, journalists, professional societies and librarians. Within

Cochrane itself, Danish authors and editors, and affiliated entities, are key target groups.

4.2

Setting priorities

Working with stakeholders to set priorities for systematic reviews is essential; not only to ensure

that important questions are addressed, but also to provide a foundation for knowledge

translation. The NCC’s overarching aim is to work in areas that affect many people, because

this is where research can have the most value. The NCC states that their research agenda is

set locally and developed with national and international collaborators. Examples of systematic

reviews that have addressed important questions with national and international relevance

include:

•

Screening for breast cancer with mammography. This review was requested by the

Danish Ministry of Health and has been updated regularly, most recently in 2013.

https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD001877.pub5/full

Treatment for women with post-partum iron deficiency anaemia. This review was

undertaken with Danish clinicians, who identified a clinically important question. Most

recent update; 2015.

https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD010861.pub2/full

General health checks in adults for reducing morbidity and mortality from disease. This

review was undertaken because of its potential societal impact, and the findings were

discussed with the Danish Minister of Health. Most recent update: 2019.

https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD009009.pub3/full

Screening for reducing morbidity and mortality in malignant melanoma. This review

was initiated because of its potential societal impact. Published in June 2019.

https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD012352.pub2/full

•

4.3

Activities that lead to impact

The NCC has been active in a range of activities that are considered important in achieving

impact (‘pathway to impact’). These activities include advocating for better quality evidence;

making evidence accessible to Danish audiences (now coordinated by a communications

consultant); promoting the use of evidence in decision-making; and working with public

authorities to develop clinical guidelines, using the best available research evidence. National

Evalueringspanelets evalueringsrapport for Nordic Cochrane Centre

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

and international guidelines can have a direct impact on healthcare practices and affect the

lives of thousands of patients. Recent examples of NCC involvement in the development of

national clinical guidelines include:

•

The national clinical guidelines project initiated by the Danish Health Authority (2014).

The NCC was instrumental in developing the underpinning methodology.

Physiotherapy and Ergotherapy for Children with Cerebral Palsy, March 2014.

National Clinical Guideline for Rehabilitation in patients with COPD, May 2014.

National Clinical Guideline for Treatment of ADHD in Adults, February 2015.

National Clinical Guideline for Treatment of Cervical Radiculopathy, May 2015.

National Clinical Guideline for Non-pharmacological Interventions for Depression, May

2016.

National Clinical Guideline for Surgical Interventions for Obesity, January 2017.

In addition, the NCC has played a key role in developing and promoting the methods of

evidence-based medicine in Denmark and beyond. International collaborations have resulted

in reporting guidelines, including the CONSORT statement for randomised trials; STROBE for

observational studies; PRISMA for systematic reviews and SPIRIT for study protocols, which

are used globally. Of particular note is the work to gain access to clinical study reports from the

European Medicines Agency, which led to full disclosure of clinical trial data in Europe. The

next step is to explore the inclusion of study reports in systematic reviews, and this is a priority

for Cochrane.

4.4

Contributions to public debate

The NCC reports that they receive frequent requests to evaluate the quality of the evidence on

a specific research topic, and this is related to their reputation of independence. They view this

as an important contribution to advancing the call for better evidence to inform decision-making

and to improve health information for the general public. The NCC gives several examples:

•

In 2017, the NCC was requested by journalists from the Danish Broadcasting

Corporation to evaluate the evidence behind the promotional claim that the pain

medications Tramadol and Tapentadol were rarely addictive. This resulted in a series

of TV documentaries, which were reported on national TV news. The TV programmes

raised awareness about opioid use in Denmark and led to new regulations ensuring

that both Tramadol and Tapentadol were regulated in the same way as other opioids.

The Danish Ministry produced a new report on adverse reactions to Tramadol, and

recent data show that the use of Tramadol has fallen.

In 2018, the NCC was requested to review the evidence for the effectiveness of an

intervention for childhood obesity, which resulted in articles in a national newspaper

about the drug’s limitations.

The NCC’s research on screening, particularly mammography screening, has informed

public policy and debate over many years. This has initiated an intense debate

between the NCC and the Danish Cancer Society and, although opinions have been

diverse, it is plausible that the debate will help drive science forward and encourage

more investigation of the beneficial effects and potential harms of screening. This work

has also had impact outside Denmark, most notably in France and Switzerland, with

both countries re-evaluating their recommendations on mammography screening.

More generally, the NCC has advocated for transparency and better documentation

on the efficacy and safety of healthcare interventions and conflicts of interest. This has

•

Evalueringspanelets evalueringsrapport for Nordic Cochrane Centre

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

contributed to new regulations governing postgraduate training of doctors in

collaboration with the pharmaceutical industry, and rules for conflicts of interest among

members of the Medical Council. The NCC’s agenda on this has been articulated in

Danish medical journals and major newspapers (see, for instance:

http://ugeskriftet.dk/debat/laegemiddelstyrelsen-og-laegemiddelindustrien-ligner-et-

lukket-kredsloeb,

https://finans.dk/erhverv/ECE11496437/chefer-i-laegemiddelstyrelsen-ejer-aktier-i-

novo-nordisk-nu-skal-det-stoppes/?ctxref=forside

https://laegemiddelstyrelsen.dk/en/news/2019/danish-medicines-agency-strengthens-

its-policy-on-conflicts-of-interest/).

4.5

The Panel’s overall assessment of relevance and impact

The NCC’s work is of clear relevance to Danish health care practice and policy, and there are

examples of significant influence on policy. These include the Centre’s contributions to national

clinical guidelines covering a range of topics, such as depression and obesity, plus the Centre’s

work on general health checks, which influenced the Government’s decision not to introduce

them. The findings from this review have been promoted internationally, and their implications

are being discussed in other countries, including the UK.

As noted in Section 4.4, some of the NCC’s work has received considerable media interest and

widespread coverage and has without doubt informed public debate in Denmark and beyond.

In turn, this is likely to bring benefits to society by helping to drive science and practice forward.

Public opinion on some of the NCC’s work has been diverse, and the depth of feeling has been

played out in the public domain, leading the Panel to question whether this has been potentially

detrimental to the organisations involved and the role of the NCC’s governing structures.

Stakeholder feedback suggests that the NCC has a good reputation, largely due to the quality,

independence and importance of its work and to the central mission of Cochrane. There was

strong support for re-establishing the Advisory Board, and for increased engagement with

stakeholders, to help determine research priorities. The Panel concurred that having stronger

stakeholder engagement and a refreshed Advisory Board would be beneficial.

Evalueringspanelets evalueringsrapport for Nordic Cochrane Centre

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Collaborations and partnerships

This section provides a short overview of the NCC’s collaborations and partnerships with other

organisations, together with the Panel’s reflections on these, and of the NCC as a

representative of Cochrane entities in the Nordic countries and as a contributor to the

international Cochrane organisation.

The Panel notes that the NCC states that they are independent of any political interests and

institutions, and have full academic freedom. This underpins their approach to collaboration

and partnerships outside of Cochrane.

5.1

Overview of national and international collaborations and

partnerships

The NCC has highlighted three national partners: the Danish Health Authority; the Centre for

Evidence-based Medicine Odense (CEBMO) and the Section for General Practice at the

University of Copenhagen.

Collaboration with the Danish Health Authority is long-standing and focuses on the production

of national clinical guidelines, where NCC staff have served as methods consultants and

contributed to national screening committees. Collaborations with researchers based in the

CEBMO and the Section for General Practice at the University of Copenhagen involve

supervision and mentoring of PhD students, as well as joint working on individual projects,

particularly on bias in research, the impact of conflicts of interest and screening.

The NCC has highlighted its collaboration with the international

Preventing Overdiagnosis

initiative. This initiative supports the development of a community of clinicians, researchers,

policy-makers and citizen advocates with an interest in overdiagnosis, organises international

conferences and encourages the generation of research. The NCC is leading a project

involving 30 researchers from institutions worldwide to develop an evidence-based approach

to re-evaluation of existing screening interventions. This aligns closely with the NCC’s ongoing

interests in screening.

NCC staff are currently exploring options to contribute to EVIPNet, a network established by

WHO to promote the systematic use of research evidence in health policy-making across

Europe.

Evidence presented in other sections of this report is relevant here, as many of the

achievements highlighted are dependent on successful collaboration.

5.2

Cochrane partnerships

The NCC serves as the local representative for Cochrane in Denmark and in the countries

where affiliated centres or groups exist. Staff from the NCC and Cochrane Sweden contribute

to PhD training in systematic reviews at the University of Lund, and these courses are available

to Danish PhD students. Workshops in systematic reviews are provided by NCC staff to

affiliated centres in Sweden, Norway, Poland and Russia. All Denmark-based Cochrane

Review Groups (CRGs) are invited to research meetings at the NCC.

Evalueringspanelets evalueringsrapport for Nordic Cochrane Centre

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

NCC staff worked with the Cochrane Editorial Unit in London to establish new coordinating

editors for two of the CRGs based in Denmark: Cochrane Colorectal Group and Cochrane

Anaesthesia, Critical and Emergency Care Group.

5.3

The Panel’s overall assessment of collaborations and

partnerships

The NCC has actively engaged in developing collaborations and partnerships within Denmark,

the other Nordic countries and internationally. These have mainly been with individual

researchers, as opposed to wider organizational collaborations. Collaborations have resulted

in high-quality research and publications, notably reporting standards, Cochrane reviews,

methodological development and other important outputs, such as clinical guidelines.

The NCC contributes to a range of Cochrane training activities, notably with Cochrane Sweden.

The Panel agreed that the NCC potentially had more to offer with regard to enhancing

systematic review capacity in Denmark and other Nordic countries, but that the relatively small

number of senior staff might be a restricting factor.

Collaboration with the CRGs based in Denmark appears to be limited, with the exception of the

Cochrane Bias Methods Group where a range of activities are ongoing, including joint

supervision of PhD students. The Panel noted that the CRGs appear to work independently of

the NCC and of each other. The NCC expressed their desire to the Panel to re-introduce regular

meetings with the CRGs and to establish closer working relationships. The Panel concurred

that stronger collaborations between the NCC and the Denmark-based CRGs would be

beneficial. In light of the increasing complexity of Cochrane methods, and the need to adhere

to methodological standards, opportunities for support and shared learning should be actively

pursued.

Evalueringspanelets evalueringsrapport for Nordic Cochrane Centre

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

6.1

Options for organising Cochrane activities

Summary of the evaluation

The Panel concluded that the NCC has achieved its main aims and objectives: it has i)

contributed to the overall goals of Cochrane; ii) pursued the activities outlined in its own

strategic plan (2016-2020) and iii) met the objectives set out in the contract with the Ministry of

Health (“The

activities in the Cochrane area include preparation of systematic reviews of health

care interventions, prevention of diseases, diagnostics, treatment and care”).

However, given

the lack of specificity in the contract with the Ministry, it is challenging for the Panel to make

precise assessments regarding the degree to which the NCC is meeting the expectations of its

core funding.

In considering the value of core funding, the Panel agreed that it provides a platform to:

Support and co-ordinate Cochrane activities in Denmark and other Nordic countries

Promote the work of Cochrane and the NCC and the value of evidence-informed

health-care

•

Engage with policy-makers, practitioners, patients and other stakeholders to identify,

define and address important research priorities in Denmark

•

Undertake high-quality, impactful research, underpinned by longer-term

methodological development

•

Build research capacity and expertise, and provide a unique source of advice and

bespoke training

•

Generate additional research income that expands the work of the Centre and

contributes to future sustainability.

Despite the NCC’s considerable achievements, the panel questions the viability of its current

operating model:

•

The NCC is small in size and seems isolated, given its positioning in Rigshospitalet.

These limitations might affect the NCC’s ability to secure additional research funding,

which is necessary to foster growth and reduce dependency on the ministry grant. An

organisational affiliation that facilitates collaboration beyond the local academic

community should increase opportunities as well as bringing other benefits, especially

for early career researchers.

The NCC’s governing structure is complex and lacks clear lines of accountability. A

clearer governance structure should improve the transparency of decision-making and

clarify to whom and how the NCC is accountable for its national grant and activities. In

light of Rigshospitalet’s steer to the Panel that an alternative governance structure is

needed, clarification around future arrangements requires urgent consideration.

•

6.2

Options

The Panel concludes that there is a need to consider the options that would strengthen the

NCC’s current position and facilitate its future growth. Closer alignment with a university would

seem desirable, bringing further potential for collaboration, and increased opportunities to

secure research funding and recruit PhD students.

Evalueringspanelets evalueringsrapport for Nordic Cochrane Centre

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Option 1

– The NCC was previously affiliated with the University of Copenhagen (through its

Professorship) and hosts several PhD students enrolled at the University of Copenhagen. One

option is to align the NCC within a cognate Department or Faculty at the University of

Copenhagen to provide potential synergy. The national funding from the Finance Act would be

an earmarked grant to the University, which would allow Cochrane activities to continue at the

current level. This could be supplemented by research grants and possible investment from

the University to foster growth, which should improve academic sustainability while maintaining

the research integrity and independence that is central to the NCC’s operation and strategy.

Option 2

– The NCC works collaboratively with the Centre for Evidence-based Medicine

Odense (CEBMO) at the University of Southern Denmark, through joint supervision of PhD

students and via the Cochrane Bias Methods Group. Another option would be to align NCC

with existing research groups working in the area of evidence-based medicine or in the National

Institute of Public Health at the University of Southern Denmark. As outlined in Option 1,

funding would be from an earmarked grant from the Ministry, which could be supplemented by

research grants and possible investment from the University of Southern Denmark. Again, this

should improve academic sustainability while maintaining the research integrity and

independence that is central to the NCCs operation and strategy.

Given existing collaborations, either option seems viable, and both options would offer

opportunities to build critical mass and for cross-fertilisation of ideas with researchers working

in cognate areas. Either option would signal the national scope of the NCC, which, given its

funding source, is important. Similar arrangements are in place in other countries.

See VIVE supplementary data and materials: The Nordic Cochrane Centre.

Evalueringspanelets evalueringsrapport for Nordic Cochrane Centre

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Cochrane Review Groups (CRGs)

This section provides a short overview of the history of the CRGs funded under the grant from

the Ministry of Health, their organizational arrangements (including links with the NCC),

research productivity and the Panel’s overall reflections.

7.1

History and background

Three Cochrane review groups, based in Copenhagen, are funded via the grant from the

Ministry of Health. These are:

1. The Colorectal Group

The Colorectal Cancer Group was registered as a Cochrane Review group in 1998. The

coordinating editor of this group is Jacob Rosenberg, Professor and Director of the

Center for Perioperative Optimizaton (CPO) at Herlev Hospital in the greater

Copenhagen area. Rosenberg was appointed coordinating editor in 2018 and is

currently working on re-establishing an infrastructure and new strategic direction for the

group.5 The group is now known as the Cochrane Colorectal Group (CCG).

2. The Anaesthesia Group and the Emergency and Critical Care Group (one group

until 2018)

The Anaesthesia, Critical and Emergency Care Group was established in 2000 at

Bispebjerg Hospital, in Copenhagen.6 Since 2014, the Group has been based at Herlev

Hospital in Copenhagen. Professor Ann Merete Møller has been Coordinating Editor of

both groups for 20 years, until 2018. Now, Andrew Smith from the Royal Lancaster

Infirmary, UK, is the coordinating editor of the Anaesthesia Group and Harald Herkner,

Medical University of Vienna, Austria, is coordinating editor of the Emergency and

Critical Care Group. Both groups are affiliated with the anaesthetic research unit at

Herlev Hospital, Capital Region, Denmark. For the purposes of this report, the two

groups will be considered as a whole (ACE).

3. The Cochrane Hepato-Biliary Group

The Cochrane Hepato-Biliary Group was formed in 1996, and the coordinating editor is

Christian Gluud, director of the Copenhagen Trials Unit (CTU) in Copenhagen. This

Group is discussed in more detail in the Evaluation Report for the CTU.

Further details about the history of the Colorectal Group can be found on the Cochrane website:

https://colorectal.cochrane.org/more-about-us

Further details about the Anaesthesia Group and the Cochrane Emergency and Critical Care Groups can be found on the

Cochrane website: https://community.cochrane.org/news/news-cochrane-anaesthesia-critical-and-emergency-care-

group

Evalueringspanelets evalueringsrapport for Nordic Cochrane Centre

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

7.2

Organizational arrangements

The financial accounting for the Colorectal Group (CCG), the Anaesthesia Group and the

Emergency and Critical Care Group (ACE) takes place at the host hospitals:

•

The Cochrane Review Groups: TOTAL 2,730,000 DKK per year

ACE: 1,316,000 DKK

CCG: 1,413,000 DKK

ACE: the managing editor, information specialist and administrative co-ordinator (0.2 full-time

equivalent) are paid directly from the Danish Ministry grant. Editors and reviewers contribute

on a voluntary basis.

CCG: the managing editor and administrative co-ordinator are paid directly from the Danish

Ministry grant. Editors and reviewers contribute on a voluntary basis. A few non-profit

organisations have provided funds over the past ten years.

The remit of these groups is to prepare and update Cochrane reviews. Review production is

managed within each group by co-ordinating and managing editors. In 2018, Cochrane

implemented its ‘CRG Transformation Programme’ with the aim of creating a more sustainable

review production system.

The Programme introduced Cochrane networks and new

governance structures for CRGs. Each CRG is nested within one of eight networks and the

networks are led by a Senior Editor. The CRGs work with their Senior Editor to ensure

consistent quality of reviews and efficient editorial processes. CRGs are accountable to the

editor-in-chief of the Cochrane Library via their senior editor. Each CRG is required to sign a

Memorandum of Understanding, every five years, with the editor-in-chief, which describes the

expectations and responsibilities of Cochrane and the CRG. Senior Editors and the editor-in-

chief are responsible for ensuring that accountability is aligned with any requirements of the

funder.

7.3

CCG

Research production

From 2014 to 2018, the CCG registered 56 publications, of which 54 (96% of total publications)

are Cochrane articles and two (4% of total publications) are reviews (Table 7.1). There was no

information on 22 (39% of total publications) publications from 2014 to 2018 in WoS databases

(mostly Cochrane protocols).

Further details are available at: https://community.cochrane.org/organizational-info/resources/resources-groups/crg-

networks-portal/crg-network-resources

Evalueringspanelets evalueringsrapport for Nordic Cochrane Centre

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Table 7.1

Number and publication classification of the publications from CCG

2012

No.

2013

No.

2014

No.

2015

No.

2016

No.

2017

No.

2018

No.

2014-2018

Total

No.

2012-2018

Total

No.

101

103

WoS -Classification

Article

Cochrane

Editorial

Letter

Review

Total

Abbreviations: WoS: Web of Science, No.: Number of observations.

Des:

The table shows CCG’s publications from 2012 to 2018, divided into the five publication categories used in the

analysis.

The 34 publications by CCG gained 253 citations from 2014 to 2018. Most citations were for

articles published in 2014 and 2015, which is expected as articles from those years have had

longer exposure time than articles published in 2017 or 2018. The average article received

seven citations for all years (Mean TC range is 0 to 47, for individual years).

Table 7.2 Number of citations and normalised citation score for publications from

CCG

Numb

2014

2015

2016

2017

2018

All

years

Mean

Median

Max

MNCS Clinical

Medicine

4.08

1.52

1.93

1.83

0.20

1.46

MNCS All

fields

4.10

1.53

1.89

1.71

0.19

1.40

Missing

Publications

Abbreviations: TC: Total number of citations for the period, MNCS: Mean Normalised Citations Score of the publications of a

unit.

Des:

The table shows the number of publications included in the analysis. For each publication year, the total, mean,

median and max. number of citations the publications in that year are shown. The table further shows the mean

MNCS Clinical medicine and MNCS All fields scores, which indicate whether the publications received more (scored

above 1) or fewer (scored less than 1) citations than the average publication in the relevant field in the given

publication year. It was not possible to obtain citation data for all the CCG publications, for which reason there are

missing data. The most frequent missing publication type is Cochrane protocols.

Note: We apply weights based on the publication’s classification for both MNCS scores. Articles, Cochrane articles and

reviews are given a weight of 1 in the analysis, whereas letters, comments and editorials are given a weight of 0.25.

When comparing the citations received by the 34 publications to those of the average article

published in the field of clinical medicine, publications from CCG received 1.46 times more

citations than the average publication in the field of clinical medicine (MNCS Clinical Medicine

range is 0.20 to 4.08, for individual years). In other words, they received more citations than an

average publication in the field. Similarly, when CCG publications are compared to an average

publication in all the WoS fields, the CCG publications received 1.40 times more citations than

the average article in all WoS fields (MNCS All fields range is 0.19 to 4.10, for individual years,

see Table 7.2).

Evalueringspanelets evalueringsrapport for Nordic Cochrane Centre

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

ACE

From 2014 to 2018, ACE registered 399 publications, of which 160 (40% of total publications)

are journal articles and 231(58% of total publications) are Cochrane articles (Table 7.3). There

was no information on 42 (10% of total publications) publications from 2014 to 2018 in WoS

databases (mostly Cochrane protocols).

Table 7.3 Number and publication classification of the publications from ACE

WoS -Classification

2012

No.

Article

Cochrane

Editorial

Letter

Review

Total

2013

No.

2014

No.

2015

No.

2016

No.

2017

No.

2018

No.

2014-2018

Total

No.

160

231

399

2012-2018

Total

No.

176

279

466

Abbreviations: WoS: Web of Science, No.: Number of observations.

Des:

The table shows ACE’s publications from 2012 to 2018, divided into the five publication categories used in the

analysis.

Note: There are an unknown number of reviews in the Article category.

The 357 publications by ACE gained 3,846 citations from 2014 to 2018 (Table 7.4). Most

citations were for articles published in 2014 and 2015, which is expected as articles from those

years have had a longer exposure time than articles published in 2017 or 2018. The average

article received 10.77 for all years (Mean TC range is 1.04 to 21.13, for individual years).

Table 7.4 Number of citations and normalised citation score for publications from

ACE

Number

Mean TC

Median

Max TC

MNCS

Clinical

Medicine

1.84

1.92

1.76

2.57

2.12

2.02

MNCS All

fields

1.85

1.94

1.72

2.40

1.96

1.95

Missing

Publica-

tions

2014

2015

2016

2017

2018

All years

357

1,437

1,137

808

385

3,846

21.13

16.72

9.62

6.31

1.04

10.77

140

120

140

Abbreviations: TC: Total number of citations for the period, MNCS: Mean Normalised Citations Score of the publications of a

unit.

Des:

The table shows the number of publications included in the analyses. For each publication year, the total, mean,

median and maximum number of citations in the publications in that year are shown. The table further shows the

mean MNCS Clinical medicine and MNCS All fields score, which indicate whether the publications received more

(score above 1) or fewer (score less than 1) citations than the average publication in the relevant field in the given

publication year. It was not possible to obtain citation data for all the ACE’s publications, for which reason there are

missing data. The most frequent missing publication type is Cochrane protocols.

Note: We apply weights based on the publication’s classification for both MNCS scores. Articles, Cochrane articles and

reviews are given a weight of 1 in the analysis, whereas letters, comments and editorials are given a weight of 0.25.

When comparing the citations received by the 357 publications to the average article published

in the field of clinical medicine, publications from ACE receive 2.02 times more citations than

the average publication in the field of clinical medicine (MNCS Clinical Medicine range is 1.76

to 2.57, for individual years). In other words, they receive more than twice as many citations as

Evalueringspanelets evalueringsrapport for Nordic Cochrane Centre

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

an average publication in the field. A similar state of affairs is seen, when ACE publications are

compared to an average publication in all of the WoS fields, which shows that the ACE

publications receive 1.95 times more citations than the average article in all WoS fields (MNCS

All fields range 1.72 to 2.40, for individual years; Table 7.4).

7.4

The Panel’s overall reflections on Cochrane Review Groups

(CRGs)

As can be seen from the tables above, there are differences between CRGs in the number of

reviews published over the period. Several factors need to be considered when assessing

productivity. The breadth of the research question usually impacts on the number of studies to

be included in the review, meaning that the effort and resource required to produce reviews is

not evenly distributed. The complexity of the review question is also a factor: some reviews are

methodologically challenging to conduct and require advanced statistical and reviewing skills,

which are not always immediately available within the team. Another factor is the importance

and relevance of the review question to stakeholders. Reviews will only be used if they address

meaningful questions, and therefore priority setting is an important aspect of review production.

Quality is a key factor, and Cochrane has documented the inconsistent quality of its reviews,

and its new strategy focuses on ‘fewer, better reviews’.

The Panel notes that the CRGs appear to work independently of each other and of the NCC.

There appears to be enthusiasm for re-introducing regular meetings and for working together

more closely. In light of the increasing complexity of Cochrane methods, the need to adhere to

methodological standards and to ensure stakeholder involvement in priority setting, the Panel

strongly supports the need for closer working and shared learning between all Cochrane

entities in Denmark. This aligns with the new arrangements and governance structures that

Cochrane has recently implemented.

As part of these new arrangements, all CRGs sign a

Collaboration Agreement with Cochrane, which outlines expectations, requirements and

responsibilities. Each CRG needs to be re-accredited every five years to ensure that core

functions, including review production, editorial activities organisational and collaboration are

carried out.

Further

details

are

available:

‘The

structure

and

function

Cochrane

Review

Groups’.

https://community.cochrane.org/sites/default/files/uploads/inline-

files/The%20Structure%20and%20Function%20of%20Cochrane%20Review%20Groups_16Aug17_2.pdf

Further details are available: https://community.cochrane.org/organizational-info/resources/resources-groups/crg-networks-

portal/crg-network-resource

Evalueringspanelets evalueringsrapport for Nordic Cochrane Centre

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Bilag 4

Evaluation Report Copenhagen Trial Unit

Dette bilag indeholder den evalueringsrapport, som det internationale evalueringspanel har

udarbejdet for Copenhagen Trial Unit (CTU).

Ud over evalueringspanelets evaluering af Copenhagen Trial Unit indeholder evalueringsrap-

porten også panelets evalueringsperspektiver på den danske Cochrane Review-gruppe, som

er tættest

knyttet

til Copenhagen Trial Unit:



The Cochrane Hepato-Biliary Group (CHBG).

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Evaluation of the Copenhagen Trial Unit

Final Report

Evalueringspanelets evalueringsrapport for Copenhagen Trial Unit

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Amanda Sowden (Chair)

Professor and Deputy Director and leader of University of York’s Centre for Review and Dissemination’s

research programme in public health, United Kingdom

Terje P. Hagen (Co-chair)

Professor and Head at Institute of Health and Society, University of Oslo, Norway

Diederick Grobbee

Professor of Clinical Epidemiology, Founder of the Julius Center, University Medical Center Utrecht, The

Netherlands

Eva Swahn

Professor, Department of Medical and Health Sciences, Division of Cardiology, Linköping University &

Department of Cardiology, University Hospital, Linköping, Sweden.

Merete Osler

Clinical Professor at Department of Public Health, UC and consultant, Center for Clinical Research and

Prevention, Bispebjerg and Frederiksberg Hospital, Denmark

Mickael Bech

Professor in healthcare leadership at Crown Prince Frederik Center for Public Leadership, Aarhus University,

Denmark

September 2019

Evalueringspanelets evalueringsrapport for Copenhagen Trial Unit

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Table of contents

Executive summary ............................................................................................ 1

Background of the evaluation ................................................................................. 1

Conclusions of the Evaluation Panel ...................................................................... 1

Introduction ............................................................................................... 3

1.1

1.2

1.3

1.4

1.5

Background ................................................................................................... 3

The evaluation .............................................................................................. 3

The evaluation panel .................................................................................... 4

Evaluation protocol ....................................................................................... 5

The evaluation process and methods ........................................................... 6

The Copenhagen Trial Unit – management and organisation ................... 7

2.1

2.2

2.3

2.4

Background, organizational placement and funding..................................... 7

Purpose and strategy .................................................................................... 8

Organisation and management .................................................................... 9

The Panel’s assessment of the overall management and organisation ....... 9

Research production and quality ............................................................. 11

3.1

3.2

3.3

3.4

Research production and impact – the CTU .............................................. 11

Research production and impact – CHBG ................................................. 12

PhD candidates and the CTU’s involvement in PhD programmes ............. 13

The Panel’s overall assessment of research productivity and quality ........ 14

Impact and relevance .............................................................................. 15

4.1

4.2

4.3

Target groups and examples of trials with significant relevance ................ 15

Interventions in the public debate ............................................................... 16

The Panel’s assessment of the impact of the centre on clinical practice and

clinical guidelines ........................................................................................ 17

Evaluation of collaborations and partnerships ......................................... 18

5.1

5.2

Overview of collaborations and partnerships .............................................. 18

Assessment of collaboration and partnerships ........................................... 18

Options for organising the activities of the CTU and the CHBG............... 20

6.1

6.2

6.3

Summary of the assessment ...................................................................... 20

Options for the CTU .................................................................................... 20

Options for the CHBG ................................................................................. 21

Evalueringspanelets evalueringsrapport for Copenhagen Trial Unit

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

Executive summary

Background of the evaluation

The Nordic Cochrane Centre (NCC) and the Copenhagen Trial Unit (CTU) receive funding from

the Ministry of Health via the National Finance Act. This funding has been in place since 1993

and, until now, no formal evaluation of the work of the Centres has taken place. Such an

evaluation was requested by the Ministry of Health and was organised and overseen by VIVE,

the Danish Centre for Social Science Research. The intent of the evaluation was to assess

whether the purpose of the two centres as stated in the National Finance Act grant has been

fulfilled. The Finance Act states:

“The activities in the Cochrane area include preparation of systematic reviews of

health care interventions, prevention of diseases, diagnostics, treatment and care.

CTU supports and performs scientifically relevant clinical trials, i.a. as part of the

Cochrane Collaboration.”

(the National Finance Act 2018, translation by VIVE).

An international Panel of six people carried out the evaluation. The evaluation focused on i)

management and organisation of the centres; ii) research production and quality; iii) impact

and relevance of the centres’ activities and iv) collaborations and partnerships.

The results of the evaluation are presented in separate reports for each of the two centres. This

report presents the results of the evaluation of the CTU. The report also includes perspectives

on The Cochrane Hepato-Biliary Group (CHBG), which is also funded under the Finance Act.

Conclusions of the Evaluation Panel

In the Panel’s view the CTU has achieved its main goals: support, coordinate and conduct

randomized clinical trials; participate in the development of methods for randomised clinical

trials and meta-analyses; educate students and researchers in evidence-based medicine,

randomised clinical trials, meta-analyses and trial sequential analysis; and support, coordinate,

and conduct systematic reviews of the literature.

Furthermore, the CTU has published its research in a range of high impact medical journals.

The impact has been substantial as CTU-studies have influenced clinical practice and

stimulated public debate and discussion.

Despite the CTU’s considerable achievements, the Panel questions the sustainability of the

current operating model, given its small size, dependency on Ministry funding, positioning in

Rigshospitalet and complex governance structures. The Panel concludes that there is an

urgent need to consider the options for a revised governance structure. The Panel has

discussed several alternatives, including reorganising the CTU as a regional trials unit,

incorporating the CTU into a university environment and creating a broader Center for

Biostatistics and Epidemiology.

In light of the increasing complexity of Cochrane methods, the need to adhere to

methodological standards and to ensure stakeholder involvement in priority setting, the Panel

suggest a closer collaboration and shared learning between all Cochrane entities in Denmark.

Evalueringspanelets evalueringsrapport for Copenhagen Trial Unit

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

This aligns with the new arrangements and governance structures that Cochrane has recently

implemented.

Evalueringspanelets evalueringsrapport for Copenhagen Trial Unit

SUU, Alm.del - 2019-20 - Bilag 109: Orientering om evalueringen af Nordic Cochrane Centre og Copenhagen Trial Unit, fra sundheds- og ældreministeren

1.1

Introduction

Background

The Ministry of Health has asked VIVE, The Danish Center for Social Science Research, to

evaluate the Nordic Cochrane Centre (NCC) and the Copenhagen Trial Unit (CTU).

The NCC and CTU receive a grant each year through the National Finance Act. In the National

Finance Act for 2018, it is stated:

”In 2018, DKK 17.8 million were allocated for financing of the Cochrane and

Copenhagen Trial Unit (CTU). The activities in the Cochrane area include

preparation of systematic reviews of health care interventions, prevention of

diseases, diagnostics, treatment and care. CTU supports and performs scientifically

relevant clinical trials, i.a. as part of the Cochrane Collaboration. The allocated funds

from 2009 onwards include DKK 0.5 million for the financing of free access for all to

the Cochrane Library”

(the National Finance Act 2018, translation by VIVE).

According to information from the centres, the 17.8 million DKK in 2018 were distributed as

follows:

•

The Nordic Cochrane Centre: 7,335,000 DKK

Copenhagen Trial Unit, including the Cochrane Hepato-Biliary Group: 7,735,000

The Cochrane Review Groups: 2,730,000 DKK

Herlev Hospital: 1,316,000 DKK (The Cochrane Anaesthesia and Cochrane

Emergency and Critical Care Groups)

Bispebjerg Hospital: 1,413,000 DKK (The Cochrane Colorectal Group).

1.2

The evaluation

The results of the evaluation are presented in separate reports for each of the two centres. This

report presents the results of the evaluation of the Copenhagen Trial Unit (CTU). In addition to

the CTU, the report also includes perspectives on the Cochrane Hepato-Biliary Group.