ORGANISATION FOR ECONOMIC CO-OPERATION AND DEVELOPMENTThe OECD is a unique forum where the governments of 34 democracies work together to addressthe economic, social and environmental challenges of globalisation. The OECD is also at the forefrontof efforts to understand and to help governments respond to new developments and concerns, such ascorporate governance, the information economy and the challenges of an ageing population. TheOrganisation provides a setting where governments can compare policy experiences, seek answers tocommon problems, identify good practice and work to co-ordinate domestic and international policies.The OECD member countries are: Australia, Austria, Belgium, Canada, Chile, the CzechRepublic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Israel,Italy, Japan, Korea, Luxembourg, Mexico, the Netherlands, New Zealand, Norway, Poland, Portugal,the Slovak Republic, Slovenia, Spain, Sweden, Switzerland, Turkey, the United Kingdom and theUnited States. The Commission of the European Communities takes part in the work of the OECD.

Table of ContentsExecutive Summary .................................................................................................................................... 5Assessment and Conclusions ...................................................................................................................... 7List of Acronyms and Abbreviated Terms ............................................................................................. 17Chapter 1: Introduction ........................................................................................................................... 191.1 Purpose of the OECD Review ........................................................................................................... 201.2 The participation of Denmark ........................................................................................................... 201.3 Acknowledgements ........................................................................................................................... 221.4 Structure of the Country Note ........................................................................................................... 22Chapter 2: The Context of Evaluation and Assessment in Denmark .................................................. 232.1 National context ................................................................................................................................ 242.2 Main features of compulsory education in Denmark ........................................................................ 252.3 Main trends and concerns .................................................................................................................. 262.4 Main developments ........................................................................................................................... 28Chapter 3: The Evaluation and Assessment Framework...................................................................... 313.1 Context and features .......................................................................................................................... 323.2 Strengths and challenges ................................................................................................................... 343.3 Pointers for future policy development ............................................................................................. 41Chapter 4: Student Assessment ............................................................................................................... 454.1 Context and features .......................................................................................................................... 464.2 Strengths and challenges ................................................................................................................... 504.3 Pointers for future policy development ............................................................................................. 59Chapter 5: Teacher Appraisal ................................................................................................................. 715.1 Context and features .......................................................................................................................... 725.2 Strengths and challenges ................................................................................................................... 765.3 Pointers for future policy development ............................................................................................. 82Chapter 6: School Evaluation .................................................................................................................. 896.1 Context and features .......................................................................................................................... 906.2 Strengths and challenges ................................................................................................................... 916.3 Pointers for future policy development ............................................................................................. 99Chapter 7: System Evaluation ............................................................................................................... 1097.1 Context and features ........................................................................................................................ 1107.2 Strengths and challenges ................................................................................................................. 1147.3 Pointers for future policy development ........................................................................................... 121OECD REVIEWS OF EVALUATION AND ASSESSMENT IN EDUCATION: DENMARK � OECD 2011

TablesTable 4.1 Common Objectives, national tests and final examinations in Danish compulsory education... 48FiguresFigure 4.1 Common Objectives guiding curriculum, teaching, learning and assessment .......................... 60BoxesBox 2.1 A 360 degree of theFolkeskole(2010) ......................................................................................... 30Box 4.1 Feedback to teachers on student performance in the national tests ............................................... 52Box 4.2 Performance standards in the United States and Canada .............................................................. 61Box 4.3 Professional development related to student assessment .............................................................. 65Box 5.1 The teaching profession in Denmark – Main features .................................................................. 74Box 6.1 Assessing and improving school quality in Odense ...................................................................... 93Box 6.2 Designing and evaluating the process of school self-evaluation ................................................. 101Box 6.3 Quality criteria used in the Singapore School Excellence Model (SEM) ................................... 102Box 6.4 Leadership roles that make a difference in improving school outcomes .................................... 104Box 6.5 The leadership framework in Ontario, Canada ........................................................................... 107Box 7.1 Publication of national outcome data on line .............................................................................. 112Box 7.2 National tests: design, purpose and use of results for accountability .......................................... 125Box 7.3 The proposed publication of national test results for schools: some reporting considerations ... 126

The major responsibility for the quality of compulsory education in Denmark lieswith the school providers – the 98 municipalities for the public schools (Folkeskole) andparent-elected boards for the private schools. An evaluation and assessment framework,therefore, plays a key role for central authorities to promote and monitor sufficient qualityand focus on improvement. Denmark holds high ambitions to improve student outcomesand deserves credit for gaining broad agreement from all major stakeholders in efforts tostimulate an assessment and evaluation culture in compulsory education. Over a shortperiod of time, Denmark has introduced new national bodies to monitor and evaluatequality in compulsory education, new national measures on student outcomes incompulsory education and requirements for municipalities to produce annual qualityreports on their school systems. However, the suite of compulsory measures does not yetform a coherent framework for evaluation and assessment. In developing a strategic planto complete the evaluation and assessment framework, the evaluation of teaching andlearning quality should be at the core. Top priorities are to:•Integrate teacher and school principal appraisal in the framework.Teachersare trusted professionals who increasingly work in teams and benefit from thesupport of special advisors. However, there is no shared understanding of whatcounts as accomplished teaching in Denmark and teacher appraisal is notsystematic. A framework of teaching standards would provide a common basis forboth teacher appraisal and a career structure for teachers. An external certificationprocess would determine both teachers’ career advancement and professionaldevelopment plans. Danish teachers are generally keen to receive feedback fortheir professional development, but while some school principals hold a formaldialogue with teachers on an annual basis, it is not wide-spread practice for schoolprincipals to observe teaching. School principals should be held accountable forproviding adequate developmental teacher appraisal and such appraisal should belinked with both teacher professional development and school improvement.Refine key elements in the framework and clarify their purposes.TheCommon Objectives that ‘teaching should lead towards’ in compulsory educationprovide a common basis for the evaluation and assessment of student learningprogress in all schools. However, teachers and schools report difficulty intranslating them into instructional and assessment plans. Refining these anddeveloping performance standards against them would promote more consistentimplementation and a more active engagement of students in their own assessment.At the same time, it would be important to review the purpose, procedures andcontent of the final examinations in Grade 9 to ensure they reflect both the breadthof outcomes and the type of skills desired at the end of compulsory education. Thecommon set of indicators in the municipal quality reports does not sufficiently

6– EXECUTIVE SUMMARYaddress the quality of teaching and learning. Developing an agreed set of formalresearch-based criteria of school quality would make the internal and externalevaluation of schools more coherent and relevant to school improvement. Further,at this pivotal stage it is critical to clarify the monitoring purpose of the nationaltests, further validate these and to develop a strategy to complement them withbroader measures of outcomes, including stakeholder views on the quality ofteaching and learning.•Invest in evaluation and assessment capacity development at all levels.Whilethere have been both central and municipal efforts to promote evaluation andassessment activities, implementation varies among schools and municipalities.Developing evaluation and assessment capacity throughout the compulsoryeducation system includes further efforts to: build teacher assessmentcompetencies, by ensuring adequate attention to this in initial teacher education,providing sufficient professional development and making more use of specialisedevaluation advisors at schools; develop competency profiles for school principalsand municipal education directors; engage schools in more systematic self-evaluation, by training school leaders to implement an authentic evaluation ofteaching and learning, feedback and objective setting at their schools and preparingsenior school staff for particular evaluation responsibilities; replenish centralevaluation expertise to support capacity development at the municipal level; buildon successful municipal partnerships to develop evaluation capacity; andstrengthen the monitoring of municipal evaluation frameworks and ensure theseinclude an evaluation of the quality of teaching and learning.Promote and support the optimal use of evaluation and assessment results.The new national tests offer a powerful pedagogical tool to teachers, and effortsshould continue to promote the use of results by teachers to adjust teachingstrategies and the possibility for teachers to re-administer the tests to follow up onstudent progress in the discrete areas tested. Similarly, national test results and theIndividual Student Plans should be key elements in teachers’ communication withstudents’ parents. Outcome data and evaluation results should form a core part ofthe municipal monitoring system and discussion and follow-up with schools forimprovement. Well led schools benefit from effective use of central or municipalself-evaluation guidelines, plus the rapid availability of results from the nationaltests. Devising an optimal system to feedback key results held at the national levelto municipalities for their monitoring purposes will limit the repetition of basicstatistical tasks at the municipal level. In general, the reporting and analysis ofinformation from the national monitoring system could be further exploited toinform system improvement.

The Danish Government’s competitiveness strategy, in tandem with political and publicdebate on the ‘mediocre’ performance of Danish students on international assessments,has increased policy focus on improving student learning outcomes. While Denmark isproud of international evidence that its students are leaders in terms of civic knowledge,recent results from the OECD’s Programme for International Student Assessment (PISA2009) confirmed Denmark’s average academic performance at the end of compulsoryeducation and, importantly, a shortage of Danish students at the highest performancelevels. Largely influenced by a review of theFolkeskolecommissioned by the PrimeMinister in early 2010, the government proposes a reform aiming to strengthen academicperformance by giving more freedom to schools in return for an increased focus onresults, in particular, the publication of national test results for schools.An increased focus on evaluation, assessment andaccountability since 2006, including new national bodies andquality assurance systems

Denmark holds high ambitions to improve student outcomes and deserves credit forgaining broad agreement from all major stakeholders and stimulating an assessment andevaluation culture in compulsory education. Over a short period of time, Denmark hasintroduced a suite of compulsory measures of student learning, a system of qualityreporting involving municipalities and schools, and a national structure to monitoroutcomes and evaluate priorities in compulsory education. Further, these measures forstudent assessment, school evaluation and system evaluation were largely designed as acoherent set and a process of ongoing dialogue and evaluation seeks to maximise theireffectiveness and adjust them where necessary. However, they are not yet fully developedand do not yet form a coherent framework for evaluation and assessment. Importantly, theframework does not include the key components of teacher and school principalappraisal. Further, the private sector is not fully integrated.There is a common basis for evaluation and assessment andcapacity building efforts, but activities vary among schoolsand municipalities

Binding national Common Objectives specify the skills and knowledge that ‘teachingshould lead towards’ by the end of compulsory education in a given subject (endobjectives) as well as at different stages of compulsory education (form-level objectives)and must be used in all schools. These form a common basis for the evaluation andassessment of student learning progress. Further, central and municipal efforts to promoteevaluation and assessment activities include centrally developed tools for teachers,schools and municipalities, training for municipalities on drafting quality reports, plusconferences and partnerships to share and build municipal efforts. However, theimplementation of the Common Objectives varies due to limited detail in some subjectsand a lack of assessment exemplars beyond those for the final examinations. Further, thelevel of municipal oversight and support to ensure that schools achieve these variessignificantly and quality reports include no information on this apart from student resultsin the final examinations.OECD REVIEWS OF EVALUATION AND ASSESSMENT IN EDUCATION: DENMARK � OECD 2011

Teachers and students report using an admirable mix of different assessment methods. Assuch, there is very strong potential for effective formative assessment practice,i.e.the useof frequent assessments to identify learning needs and adapt teaching. However, thereported lack of clarity of the Common Objectives makes it difficult for teachers andschools to translate the content into instructional and assessment plans. Teachers carry themajor responsibility for student summative assessment at the end of compulsoryeducation allowing a broad overview of student achievement. In Form 9, all students mustcomplete a mandatory project assignment in addition to written and oral standardexaminations. Common marking guidelines and moderation procedures provide anequitable way to judge whether students have achieved the Common Objectives.However, the final examinations are criticised by students and others as being ‘outdated’and students are not able to sit examinations in all subjects that they study.National tests are powerful diagnostic tools for teachers, butnot all teachers use these effectively

Municipal quality reports are intended to be the basis of further action in managing thesystem at the level of the municipality. First, they provide an agenda for dialoguebetween the municipality and the school principal and an opportunity to set aspirationaltargets which, replicated on a large scale, could contribute to the improvement ofeducational performance nationally. Second, municipalities are required to produce actionplans for schools that are underachieving, which should be an important lever for schoolimprovement provided the school has the capacity to take the necessary steps. However,the degree of follow-up by municipalities varies and is not always rigorous and objective.Without adequate follow-up of schools’ subsequent action, the municipal quality reportscan have little impact on school improvement.Schools benefit from the availability of more information andtools for self-evaluation, but need to develop expertise to usethese effectively

Since 2001, the publication of final examination results in Forms 9 and 10, alongsideteacher-awarded final grades, serve as the major national indicators of overall quality inDanish compulsory education, as they cover theFolkeskoleand the majority of privateproviders. These complement international measures (e.g. PISA), to inform debate on theoverall productivity agenda. The introduction of the national tests also offers monitoringinformation on theFolkeskoleat different stages in compulsory education and the firstreal opportunity to reliably monitor progress in educational outcomes over time againstthe national Common Objectives. However, the lack of inclusion of the private sectorlimits their national monitoring value. Plus a lack of clarification of how results will beused to hold schools accountable runs the risk that results will not reflect real progress inoutcomes, but rather just increased focus in instruction on the discrete areas measured inthe tests. Further, it is not clear to what extent current national measures are assessinghigher-order thinking skills and cross-curricular competencies – a serious concern if theyare to signal the expected outcomes of compulsory education. Also, there is a lack ofinformation on the quality of the teaching and learning environment,e.g.views ofstudents, teachers and parents.There is a strengthened national structure to monitorcompulsory education, but no overview of municipal qualityassurance systems

Now that the major centrally-designed evaluation tools have all been introduced to theFolkeskole(and mandatory examinations to private schools), there is considerable need tostrengthen central support to ensure that these tools are linked to effective classroompractices. The top priority is to significantly invest in capacity development across thecompulsory education system to ensure the effective use of these and other evaluation andassessment measures by stakeholders. The effectiveness of the overall evaluation andassessment framework depends to a large extent on whether those who evaluate and thosewho use evaluation results at the different levels of the system have the appropriatecompetencies. Further, it should be clearly communicated that the purpose of suchevaluation tools is to improve educational outcomes and that stakeholders should activelyuse these to develop strategic improvement or action plans at all levels. In going forwardthere is a need to refine and expand the Common Objectives including a set ofperformance standards to make them a powerful basis for student assessment and schoolself-evaluation, to define evidence-based teaching standards, and to further developcentral evaluation and assessment tools,e.g.making quality reports relevant to schoolself-evaluation and ensuring the final examinations reflect the knowledge and skillsexpected at the end of compulsory education. TheEvaluation portalremains vital andshould be expanded to offer more evaluation tools that are aligned with the CommonObjectives.Develop performance standards, engage students inassessment and review the final examinations

The Common Objectives articulate the knowledge and skills that ‘teaching should leadtowards’, but not what students are expected to learn at key stages in each of the mainsubjects. Common Objectives with specific performance standards could guideinstruction and assessment more effectively. As such, there is room to further refine andexpand the Common Objectives, develop a set of specific performance standards againstthese and provide relevant support materials for teachers to mobilise the performancestandards. It is important to build on democratic traditions and to ensure that students areactively involved in assessment. Specifying what will be assessed and how is the key tothis and teachers can use performance standards to develop specific scoring rubrics withstudents and to stimulate student self- and peer-assessment processes. At the same time asclarifying the Common Objectives, there should be a review of the content of the finalexaminations to ensure they are adequately aligned with expected outcomes andperformance standards set in the Common Objectives. Further, if the final examinationsOECD REVIEWS OF EVALUATION AND ASSESSMENT IN EDUCATION: DENMARK � OECD 2011

It is important to continue efforts to validate and further develop the national tests byensuring that all Common Objectives and subject areas are given certain forms ofattention and the tests are adequately aligned with the Common Objectives and includeperformance tasks. It is critically important to engage teachers in working effectively withthe national test results as one means to diagnose student learning needs and to adjusttheir teaching strategies accordingly. In general, teachers need to be actively involved indeveloping data-driven professional learning communities where assessment data areused in non-threatening ways and teachers develop assessment competencies. This takestime and it is crucial that assessment literacy is adequately covered in initial teachereducation. Further, professional development will play a pivotal role in realising aparadigm shift so that teachers view assessment as an integral part of their teaching andnot as an additional burden on their teaching responsibilities.Develop teaching standards as a basis for a career structurewith progression determined by certification

Developing a framework of teaching standards as a reference for teacher appraisal is a toppriority. A clear and concise profile needs to reflect the sophistication and complexity ofwhat effective teachers are expected to know and be able to do and should be based onthe objectives for student learning (the Common Objectives), be informed by research,and benefit from the ownership and responsibility of the teaching profession. Suchstandards should provide the common basis to organise a career structure for teachers,expressing different career stages, such as competent teacher, established teacher, andaccomplished/expert teacher, with distinct roles and responsibilities in schools associatedwith given levels of teaching expertise. Access to each of the key stages could beassociated with formal processes of appraisal through a system of teacher certification.New teachers should only access the first stage after successful completion of amandatory probationary period. The absence of career opportunities for effective teachersundermines the role of teacher appraisal. Teacher appraisal for certification would aim tohold teachers accountable for their practice and determine both their career advancementand professional development plans.Strengthen developmental teacher appraisal and link this withprofessional development and school improvement

The internal and external evaluation of schools, including the municipal quality reports,should be based on an agreed set of formal criteria of school quality,e.g.the quality ofteaching and learning, teacher professional development, pedagogical leadership, schoolcurriculum, vision and expectations, plus the assessment of student learning progress andoutcomes. Without this, the school evaluation framework lacks coherence. The qualityreports should be further developed in ways which encourage and take greater account ofschool self-evaluation and teacher appraisal, and that put the quality of teaching andlearning at the heart of the process. A requirement for schools to produce an annualquality report would be a stimulus for many schools to further their self-evaluationpractices and holds strong potential for school improvement, if the process adequatelyengages the school community and the report is based on sound school quality criteria.School principals are pivotal in developing a school self-evaluation culture. This arguesfor a shift in the role of school principal from one who administers and manages theschool and organises its staffing, students and programmes, to one who is the pedagogicalleader of the school.Strengthen municipal and school follow-up on schoolevaluation results and support school evaluation capacitydevelopment

The school evaluation culture will not be endemic until evaluation is shared, followed upand reviewed to see what difference it has made both internally by schools and externallyby municipalities. Outcome data and evaluation results should form a core part of themunicipal monitoring system and discussion and follow-up with schools forimprovement. In particular, nationally comparable information, including national testresults, transition statistics and student final grades in Form 9, provide comparativeinformation across schools that can be used by municipalities most constructively toidentify improvement and share best practice among schools. Municipalities and schoolsneed to go further to ensure constructive use of these outcome data and strive tocomplement them with other measures. To promote internal and external schoolevaluation capacity, it would be useful to establish an authoritative centre for schoolevaluation at arm’s length from schools and municipalities to develop evaluationframeworks and criteria and model good practice. Given their key influence in furtheringthe effective internal and external evaluation of schools, there should be competencyprofiles for both municipal education directors and school principals.

It is important to develop a strategy to complement existing national monitoringinformation with broader measures of outcomes, including stakeholder views on thequality of teaching and learning. There is strong support among key stakeholders todevelop broader measures of student learning,e.g.creativity and innovation, and it wouldbe useful to take stock of current efforts to develop such measures at the municipal leveland to evaluate to what extent these could be supported and extended throughout thesystem. The current national tests could be developed to measure the progression of agiven cohort through compulsory education and to include open-ended questions. Longer-term efforts could include administering a light monitoring sample survey to providestable trend information on a broader range of student knowledge and skills. Critically,there should be a careful review of strategies to maximise the monitoring potential of thenational tests at the system level, in particular the proposal to publish national test resultsat the school level. The priorities would be to continue to validate the national tests and togo further in supporting and promoting capacity building to ensure the effective use ofnational test results by school principals and municipal directors as a core part of theirquality monitoring systems.Strengthen efforts to both monitor and promote municipalevaluation capacity

Clearly formulated objectives and performance management at the municipal leveltogether with strong school leadership has proven to be an effective partnership forimprovement in Denmark. Therefore, central monitoring of municipal evaluation capacityshould be strengthened, as it is of key importance to identify municipalities where realprogress is being made in student outcomes and to share this knowledge throughout thesystem. In subsequent years, one helpful indicator will be student progress as measured inthe national tests and it will be important to invest in efficient systems to report andanalyse this to feed results into the central monitoring of municipalities. It is of criticalimportance to devise an optimal feedback system of key results held at the national levelto municipalities for their monitoring purposes, so as to minimise the repetition of basicstatistical and reporting tasks at the municipal level. At the same time, there is room forfurther central and municipal collaboration to build municipal monitoring capacity,including the effective use of national test results and other performance indicators, andalso to design ways to further stimulate horizontal collaborations among municipalities.

1.1 Purpose of the OECD ReviewThis Country Note for Denmark forms part of the OECDReview on Evaluation andAssessment Frameworks for Improving School Outcomes.This Review is designed torespond to the strong interest in evaluation and assessment issues evident at national andinternational levels. It provides a description of design, implementation and use ofassessment and evaluation procedures in countries; analyses strengths and weaknesses ofdifferent approaches; and provides recommendations for improvement. The OECDReview looks at the various components of assessment and evaluation frameworks thatcountries use with the objective of improving student outcomes. These include studentassessment, teacher appraisal, school evaluation and system evaluation. The OECDReview focuses on primary and secondary education.1The overall purpose is to explore how systems of evaluation and assessment can beused to improve the quality, equity and efficiency of school education.2The overarchingpolicy question is “How can assessment and evaluation policies work together moreeffectively to improve student outcomes in primary and secondary schools?” The Reviewfurther concentrates on five key issues for analysis:(i)Designing a systemic frameworkfor evaluation and assessment;(ii)Ensuring the effectiveness of evaluation andassessment procedures;(iii)Developing competencies for evaluation and for usingfeedback;(iv)Making the best use of evaluation results; and(v)Implementing evaluationand assessment policies.Twenty-five education systems are actively engaged in the Review. These cover awide range of economic and social contexts, and among them they illustrate quitedifferent approaches to evaluation and assessment in school systems. This will allow acomparative perspective on key policy issues. These countries prepare a detailedbackground report, following a standard set of guidelines. Countries can also opt for adetailed review, undertaken by a team consisting of members of the OECD Secretariatand external experts. Ten OECD countries have opted for a country review. The finalcomparative report from the OECD Review, bringing together lessons from all countries,will be completed in 2012.The project is overseen by the Group of National Experts on Evaluation andAssessment, which was established as a subsidiary body of the OECD Education PolicyCommittee in order to guide the methods, timing and principles of the Review. Moredetails are available from the website dedicated to the Review:www.oecd.org/edu/evaluationpolicy.

Denmark requested for the review to focus on compulsory education (primary andlower secondary education) and primarily on the public provision of compulsoryeducation in theFolkeskole.Private compulsory education providers are also included inthe review. The review does not pay attention to upper secondary education.An important part of Denmark’s involvement was the preparation of a comprehensiveand informative Country Background Report (CBR) on evaluation and assessmentpolicy.3The review team is very grateful to the authors of the CBR, and to all those whoassisted them for providing an informative document. The CBR is an important outputfrom the OECD activity in its own right as well as an important source for the reviewteam. Unless indicated otherwise, the data for this report are taken from the DanishCountry Background Report. The CBR follows guidelines prepared by the OECDSecretariat and provides extensive information, analysis and discussion in regard to thenational context, the organisation of the education system, the main features of theevaluation and assessment framework and the views of key stakeholders. In this sense,the CBR and this Country Note complement each other and, for a more comprehensiveview of evaluation and assessment in Denmark, should be read in conjunction.The review visit to Denmark took place on 5-12 October 2010 and covered visits toCopenhagen, Odense, Hedensted and Glostrup. The itinerary is provided in Annex 1.The visit was designed by the OECD in collaboration with the Danish authorities. Thereviewers comprised two OECD Secretariat members and two experts external to boththe OECD and Denmark. The composition of the OECD review team is provided inAnnex 2.During the review visit, the OECD review team held discussions with a wide rangeof national, regional and local authorities; officials from the Ministry of Education;relevant agencies outside the Ministry which deal with evaluation and assessment issues;teacher unions; parents’ organisations; representatives of schools; students’organisations; and researchers with an interest in evaluation and assessment issues. TheOECD review team also visited a range of schools, interacting with school management,teachers and students. The intention was to provide a broad cross-section of informationand opinions on evaluation and assessment policies and how their effectiveness can beimproved.This Country Note is the report from the OECD review team. The report provides,from an international perspective, an independent analysis of major issues facing theevaluation and assessment framework in Denmark, current policy initiatives, and possiblefuture approaches. The Country Note serves three purposes:•••Provide insights and advice to the Danish education authorities,Help other OECD countries understand the Danish approach, andProvide input for the final comparative report of the project.

1.3 AcknowledgementsThe OECD review team wishes to record its grateful appreciation to the many peoplewho gave time from their busy schedules to inform the OECD review team of their views,experiences and knowledge. The meetings were open and provided a wealth of insights.Special words of appreciation are due to the National Co-ordinator, Ms. Charlotte Rotbøllfrom the Danish Quality and Supervision Agency, for sharing her expertise andresponding to the many questions of the review team. The courtesy and hospitalityextended to us throughout our stay in Denmark made our task as a review team aspleasant and enjoyable as it was stimulating and challenging.The OECD review team is also grateful to colleagues at the OECD, especially toStefanie Dufaux for preparing the statistical annex to this Country Note (Annex 3) and toHeike-Daniela Herzog for editorial support.While this report benefitted from the Danish CBR and other documents as well as themany discussions in Denmark, any errors or misinterpretations in this Country Note areour responsibility.

1.4 Structure of the Country NoteThe remainder of this report is organised in six chapters. Chapter 2 provides thenational context, with information on the Danish compulsory education system and recentdevelopments. Chapter 3 looks at the overall evaluation and assessment framework andanalyses how the different components of the framework play together and can be mademore coherent to effectively improve student learning. Then Chapters 4 to 7 present eachof the components of the evaluation and assessment framework – student assessment,teacher appraisal, school evaluation and system evaluation – in more depth, presentingstrengths, challenges and policy suggestions.The policy suggestions intend to build on reforms that are already underway inDenmark. The suggestions should take into account the difficulties that face any visitinggroup, no matter how well briefed, in grasping the complexity of the Danish compulsoryeducation system and fully understanding all the issues.

The decentralised Danish system places the major responsibility for quality assurance incompulsory education with the education providers, that is, the 98 municipalities for thepublic schools (Folkeskole) and parent-elected boards for private schools. Municipalitiessupervise the Folkeskole and the Ministry of Education supervises private schools. TheDanish Government’s competitiveness strategy plus the ‘mediocre’ performance ofDanish students on international assessments, have increased policy focus on improvingstudent learning outcomes. To this end, there have been sustained central efforts since2006 to stimulate evaluation and assessment activities in compulsory education,including new national bodies to monitor and evaluate quality in compulsory education,new national measures on student outcomes in compulsory education and requirementsfor municipalities to produce annual quality reports on their school systems. A currentproposal for reform aims to strengthen academic performance by giving more freedom toschools in return for an increased focus on results.

2.1 National contextDemographic contextDenmark has a population of 5.5 million people, with over one million living inCopenhagen and just over half a million living in the three other major cities Aarhus,Odense and Aalborg. The public sector was significantly reorganised in 2007 into fiveregions with 98 municipalities. This has seen administrative mergers of many of theformer, smaller municipalities (prior to 2007, there were 274 in total) and in turn thecreation of many larger schools. Ten per cent of all students in compulsory educationhave a migrant background, with over 60% of these students coming from Turkey, Iraq,the Lebanon, Somalia, Pakistan, Afghanistan, Bosnia-Herzegovina, Yugoslavia, Vietnamor Sri Lanka (The Danish Ministry of Education, 2010). The school population has grownin diversity also in terms of students’ socio-economic background. Such administrativeand demographic changes pose new opportunities and challenges to the delivery of high-quality compulsory education in Denmark.

2.2 Main features of compulsory education in DenmarkStructure of compulsory education in DenmarkThere are ten years of compulsory education in Denmark. Children usually begin theircompulsory schooling in August of the year they turn 6 years of age. This reflects a recentchange (August 2009) to make the ‘pre-school class’ year part of the compulsoryeducation,i.e.compulsory education starts a year earlier. The ten years of compulsoryeducation, therefore, run from the typical ages of 6 to 15 years and comprise one year ofearly childhood education in Form 0 (ISCED 0), Forms 1 to 6 of primary education(ISCED 1) and Forms 7 to 9 of lower secondary education (ISCED 2). There is also avoluntary Form 10 year offered as part of lower secondary education. In 2008-09, 49% ofstudents attending Form 9 also attended Form 10.Compulsory education is offered by both public and private schools, plus children canbe taught at home. The vast majority of children follow their primary education in thepublic school (Folkeskole) – in 2008, 88% of children in Form 1 was in theFolkeskole.However, the proportion of children attending private lower secondary schools is higher,notably due to attendance at independent boarding schools. Overall, therefore,approximately 80% of all children enrolled in compulsory education are in theFolkeskole.There are compulsory school-leaving examinations at the end of Form 9. Studentsuccess on such examinations does not determine their access to upper secondaryeducation. In theFolkeskole,students will also sit national tests at different stages of theircompulsory education (see Chapter 4).

Distribution of responsibilitiesTheFolkeskoleare governed by the 98 municipalities in Denmark. Municipalresponsibility for theFolkeskolewas made more explicit in the 2006 revision of theFolkeskoleAct. As such, municipalities are responsible for the overall quality of theirschools and for setting local objectives and conditions, including the goals and scope forschool activities, as well as the supervision of theFolkeskole.Municipalities need to draftand publish on their websites annual mandatory quality reports. They are the employersof all school staff and should be responsible for the appraisal of school principals.Private independent schools are governed by a parent-elected board which isresponsible for school quality, in particular for ensuring that educational content matchesacademic standards in theFolkeskoleand that students are prepared for life in the Danishfree and democratic society. There is external supervision of the quality of education inprivate schools. Traditionally, the Ministry of Education, by means of the Quality andSupervision Agency (see below), supervises the private school sector and can advise tostop public funding in cases of significant quality concerns. While the Ministry ofEducation has primary responsibility for supervising the quality of compulsory educationoffered at private schools, a new supervision possibility was introduced in August 2010,whereby parents in a given private school can elect a supervisor to conduct school self-evaluation. Such a supervisor must follow and successfully complete Ministry-approvedtraining.

FinancingEducation is prioritised and generously funded in Denmark in comparison to in otherOECD countries. Denmark allocates a slightly higher proportion of public spending togeneral social protection and education, and a lower proportion to general public services(OECD, 2009c). Denmark’s spending on primary and lower secondary education perstudent is high compared to other OECD countries (OECD, 2010a). Further, Denmarkprioritises local funding to more flexibly meet local needs. A much larger share ofspending is at the local government level in Denmark compared to in other OECDcountries (OECD, 2009c). The Danish government provides block grants to themunicipalities, but it is up to each municipality to determine the funding and level ofservice provided to theFolkeskole(Rambøll, 2011). Private schools also receivesignificant levels of government funding – this is equivalent to 75% of the averageexpenditure in theFolkeskole.This supports the tradition of offering private alternativesto theFolkeskole.

2.3 Main trends and concernsPolitical urgency to improve student learning outcomesIncreased policy focus has been given to the performance of Danish students ininternational assessments of reading, mathematics and science skills. At the top politicallevels, this is judged to be ‘mediocre’ and the political discourse aspires for Danishstudents to be among the best in the world. The OECD’s PISA 2009 results released inDecember 2010 confirmed Denmark’s average position internationally in terms of studentmean performance in reading at the end of compulsory education (OECD, 2010b).4International evidence on student performance in primary education (Form 4) is slightlymore encouraging: mathematics and science results from the TIMSS 2007 assessmentwere above the international average (IEA, 2008a and b). At the same time, Denmark isjustifiably proud that its students lead the international pack in terms of civic knowledge(IEA, 2010).International evidence indicates that the key challenge is to stimulate averageperforming students to excellence. While in primary education, a higher proportion ofDanish students achieved the advanced benchmark in mathematics than the internationalmedian, in science the proportion was the same as the international median (IEA, 2008aand b). At the end of compulsory education, there is a comparatively small proportion ofDanish students demonstrating excellence as measured by the PISA tests: 4.7% of Danishstudents performed in the top two performance levels in the PISA reading tests, comparedto 7.6% on average in the OECD countries (OECD, 2010b). Further, Denmark has lostground here: while internationally there was a small decline in the proportion of studentsperforming in the top two reading levels between 2000 and 2009, this decline was morepronounced in Denmark (OECD, 2010c)5. Similarly, there was a decline in the proportionof Danish students performing at the top level in the mathematics test between 2003 and4.5.An assessment of 15-year-olds, PISA sheds light on what students demonstrate they knowand can do at the end of compulsory education.On average across the 26 OECD countries with comparable results for both the PISA 2000and 2009 assessments, the combined percentage of students performing at Level 5 or 6 inreading was 9.0% in 2000 and decreased to 8.2% in 2009 (OECD, 2010c).OECD REVIEWS OF EVALUATION AND ASSESSMENT IN EDUCATION: DENMARK � OECD 2011

Demands to effectively integrate all students into the FolkeskoleTheFolkeskoleoffers integrated, comprehensive education and aims to differentiateteaching to individual student needs. Denmark enjoys comparative success internationallyin limiting the proportion of low performing students at the end of compulsory education.PISA 2009 confirmed that compared to other countries, there are fewer students inDenmark who are unable to perform the most basic reading tasks and the proportion oflow performing students has decreased since 2000 (OECD, 2010b; OECD, 2010c). Thisis a strong success factor for Danish compulsory education and it is commendable that thegovernment wants to go even further in efforts to ensure all students have basic skills.International and national evidence reveal that there is a wide range of student abilitywithin each school (e.g. OECD, 2010d; Wandall, 2010). Evidence from PISA 2009 showsa decrease in the variation in reading performance among students within schools inDenmark since 2000. However, there are stronger calls to go further in integrating allstudents successfully in theFolkeskole,in particular by reducing the proportion ofstudents receiving special educational provision by integrating them in theFolkeskole.Further, international evidence and national research has shown a strong and persistentaverage performance disadvantage for students with a migrant background (e.g. OECD,2010e; OECD, 2006; Egelund and Tranæs, 2008).The between-school variation of performance in Denmark remains lower than theOECD average (OECD, 2010d), which indicates that the specific school a student attendshas less of an impact on how the student performs in Denmark than is the caseinternationally.

2.4 Main developmentsAn increased focus on evaluation, assessment and accountabilityThe OECD (2004a) review of theFolkeskoleemphasised the importance anddifficulty of establishing an evaluation culture in Danish compulsory education: “… theestablishment of a new culture of evaluation is bound to be difficult. Yet this isprobably the most important single change that needs to be achieved if other measuresare to be effectively implemented and standards are to rise.” Following the review, arevision of theFolkeskoleAct in 2006 saw the introduction of a raft of nationalmeasures aiming to stimulate an evaluation culture in theFolkeskoleand to introducean element of accountability to compulsory education providers. These included therequirement for municipalities to draft and publish annual quality reports on the schoolsin their jurisdiction, the introduction of national tests (with a provision to publish anational performance profile on average performance in these tests), plus theintroduction of mandatory Individual Student Plans (ISPs) to document student learningprogress. Further, the final examinations in Form 9 were made mandatory and thepublication of these results by school and municipality was assured by the 2002transparency law. As such, the revision of theFolkeskoleAct in 2006 is described as ‘ashift towards a more centralised approach to evaluation and assessment’ in response toconcerns of Danish students’ performance in international assessments (Rambøll,2011).

New national structure to promote evaluation and assessment in compulsoryeducationSince 2006, the creation of a new national structure has aimed to signal and establishthe importance of evaluation and assessment in compulsory education, including: anadvisory body with representatives from all the major stakeholder groups to inform theMinister of Education on the quality of theFolkeskole(the Council for Evaluation andQuality Development of Primary and Lower Secondary Education or ‘the SchoolCouncil’) and an agency to monitor and develop quality in compulsory education (theformer School Agency)6. As of 1 March 2011, in addition to the Ministry of Educationand the School Council, there are three national bodies with a role in the evaluation andassessment of compulsory education: the Education Agency; the Quality and SupervisionAgency; and the Danish Evaluation Institute (EVA).•The Education Agency has responsibility for drawing up legislation for, as well asthe management and operation of compulsory education and upper secondaryeducation. The secretariat of the Chairmanship of the School Council now sits inthe new Education Agency.

Proposal for reform in compulsory education (December 2010)Since the OECD review visit was conducted in October 2010, the government haspublished a proposal for reform entitled ‘Professionalism and Freedom’ (Regeringen,2010). The proposal makes clear the increased academic expectations of students incompulsory education, including that all children are able to read at the end of Form 2and that there are significant improvements in Danish, mathematics, English and science.Accordingly, the proposal includes more instructional hours in these core subjects, plusthe ability for schools to create talented and elite classes. The government also proposesmore training for teachers and more money for research, plus ‘character requirements’ forteacher education to increase the prestige of the teaching profession. Importantly, theproposal argues that ‘to match the greater freedom to schools’ the results of the nationaltests in compulsory education should be published for each school.The proposal draws on the findings of a 360 degree review of theFolkeskolethat wascommissioned by the Prime Minister in early 2010 (see Box 2.1).

Box 2.1 A 360 degree of theFolkeskole(2010)The Prime Minister commissioned the 360 degrees review in early 2010. The review team(the ‘Flying Squad’) comprised six members with different competencies, including experiencewith reviews of theFolkeskole(a member of the OECD 2003 review team and members of theSchool Council) and educators (two teachers and two school principals). The research base wasstrengthened by support from School Council staff and ready access to work already conductedby the School Council. Plus, the review team had funds to commission research reports onspecific areas,e.g.a review of teacher education in internationally high performing countries. Thereview team produced a report with recommendations in seven thematic areas of equalimportance, although representatives informed the OECD review team that teacher quality is atthe core of the recommendations.1.Lift teacher, school leader and municipal education director competency:Increaseadmission requirements to teacher training and make this research-based, schools tostrategically use teacher training and competence development; offer school leaders andmunicipal education directors special management training to increase targets andresults focus, identify and train up prospective leaders.2.Increase school development research:Establish a research institute for schooldevelopment and fund research on learning and teaching.3.Clarify student learning goals:Modernise and prioritise theFolkeskoleobjectives tostrengthen reading throughout schooling and to ensure broad student development.4.Focus on results:Schools to produce annual school results reports; government toensure easy access to results data and create fora for schools to share knowledge;municipal education directors to increase focus on results and school development;school results considered in school leader and municipal education directorpay/progression.5.Ensure schools cater to all students:Change legislation to reduce to a minimumnumber of students in special education provision; ensure teacher access to educationalpsychologists; prioritise parent-teacher co-operation; further integrate school and leisuretime in schools with socially deprived children.6.Increase local freedom:Scrap barriers to making up classes or groups; ease currentrequirements regarding minimum numbers of lessons; allow schools to employ peoplewith different professional qualifications and to extend opening hours; give studentsmore choice during the final years of compulsory schooling; use IT consistently;modernise theFolkeskoleleaving examinations and make them count for students’access to upper secondary education.7.Improve school structure:Ensure in future school size optimises teaching andmanagement competences and effective utilisation of resources.The review heavily influenced the government’s new seven objectives for theFolkeskole,which include: reducing the number of students in special education; recruiting teachers from thebest graduates; ensuring teaching is knowledge-based; clarifying objectives and making resultstransparent to lessen monitoring needs.Source:Danish School Agency (2010).

Denmark deserves credit for stimulating an assessment and evaluation culture incompulsory education. However, the suite of compulsory measures for studentassessment, school evaluation and system evaluation does not yet form a coherentframework for evaluation and assessment. Importantly, the framework does not includeteacher and school principal appraisal. In developing a strategic plan to complete theevaluation and assessment framework, the evaluation of teaching and learning qualityshould be at the core. There is a common basis for the evaluation and assessment ofstudent learning progress (the Common Objectives in compulsory education), plus therehave been considerable central and municipal efforts to promote evaluation andassessment activities. However, implementation and activities vary among schools andmunicipalities. A priority now is to develop evaluation and assessment capacitythroughout the compulsory education system. There is also room to clarify evaluationpurposes and to refine and update the Common Objectives and other central measures.

32– 3. THE EVALUATION AND ASSESSMENT FRAMEWORKThis chapter looks at the overall framework for evaluation and assessment inDenmark,i.e.its various components such as student assessment, teacher appraisal,school evaluation and system evaluation, the coherence of the whole as well as thearticulation between the different components. Following this overview, the succeedingchapters (4-7) will analyse the issues relevant to each individual component in moredepth.This report differentiates between the terms “assessment”, “appraisal” and“evaluation”. The term “assessment” is used to refer to judgements on individual studentperformance and achievement of learning goals. It covers classroom-based assessments aswell as large-scale, external tests and examinations. The term “appraisal” is used to referto judgements on the performance of school-level professionals,i.e.teachers and schoolleaders. Finally, the term “evaluation” is used to refer to judgements on the effectivenessof schools, school systems and policies. This includes school inspections, school self-evaluations, evaluation of municipalities, system evaluation and targeted programmeevaluations.

Key componentsIn a nutshell, the Danish approach can be described as consisting of the followingfour components (adapted from Rambøll, 2011):•Student assessment:The long-established approach to student assessment inDanish compulsory education involves school and/or teacher developedassessments, strengthened in 1993 by the legal requirement for teachers to ensurea continuous assessment of student learning. The national Common Objectives forcompulsory education (2003) form the basis of student assessment and compriseend objectives (in Form 9) as well as objectives for different forms and subjectsthroughout compulsory education. Policy initiatives since 2006 have sought tointroduce summative assessments for students in compulsory education,including: mandatory school-leaving examinations in Form 9; and national tests(2010) for different subjects in Forms 2, 3, 4, 6, 7 and 8 based on the respectiveform-level Common Objectives. Further, mandatory ‘Individual Student Plans’aim to document student learning progress more systematically.Teacher appraisal:There are no national requirements for teacher appraisal.This is conducted on a voluntary basis and practices are defined locally, usuallyby the school. The major tradition is for teacher self-appraisal and also feedbackfrom school principals. Teacher appraisal remains very much an internal schoolmatter. The former School Agency presented tools for teacher appraisal to helpstimulate this practice. According to Danish teacher reports in an internationalstudy (TALIS), appraisal by an individual or body external to the school is notcommon practice and is much rarer than in other countries.School evaluation:External school evaluation is the responsibility of themunicipalities for public schools and the Quality and Supervision Agency forprivate schools. There was a major initiative to introduce a quality assurancesystem in 2006 with the requirement for municipalities to produce an annualquality report, including information on schools in their jurisdiction compiled bythe schools. Each report must include a standard set of indicators, but can includeadditional, locally specified indicators also. Little is known about school internalevaluation practices in Denmark and this is reported to vary significantly amongdifferent municipalities and schools. Tools for use in school self-evaluation areprovided centrally by the Quality and Supervision Agency.System evaluation:Evaluation of compulsory education as a whole in Denmarkhas been heavily reliant on information provided via international assessments.Results from such external studies – notably the OECD’s PISA 2000 and 2003surveys – have led to increased demand for information on the compulsoryeducation system. Since 2006, significant attempts have been made to producenational information on outcomes in compulsory education. The results ofmandatory school-leaving examinations and final grades in Form 9 are publishedby school and municipality. December 2010 saw the publication of the first‘national profile’ showing average results for Danish public-school students in thenational tests and designed to measure national progress over time.

3.2 Strengths and challengesStrengthsIncreased focus on improving educational outcomes and the role of compulsoryeducationThe Prime Minister firmly believes that the continuous improvement of skills is thekey to growth in the Danish economy and in particular highlights the crucial role for theFolkeskolein achieving this. The Prime Minister’s Office issued goals and an agenda for‘Denmark 2020’ in which one of the ten major goals is for Danish children to be amongthe best performing in the world – specifically for them to be in the top five in PISAsurveys and also in ‘English measured in relation to non-English speaking countries’(Danish Government, 2010). In October 2010 he presented seven objectives for theFolkeskolewith an aim to ensure that Danish students achieve these goals. In December2010, these were set out in a reform package, which highlights the key role of schoolleadership and teachers in improving results.Much of this heightened political focus on improving educational outcomes stemsfrom the Globalisation Council (established in 2005) recognising the key role ofeducation in confronting the challenges of globalisation and keeping Denmarkinternationally competitive. The most recent Competitiveness Report (Danish Ministry ofEconomic and Business Affairs, 2009) presents 16 areas of significance tocompetitiveness, including ‘primary and lower secondary education’, ‘upper secondaryeducation’ and ‘higher education’. Key indicators for ‘primary and lower secondaryeducation’ include results from PISA, both average performance in reading, mathematicsand science, and a focus on the proportions of weak and capable students and studentattitude to collaboration.

Strong political will to establish an evaluation and assessment frameworkDenmark deserves credit for the high ambitions it holds for the school system and thepolicies and strategies adopted in the last decade to improve the performance of schoolsand the standards achieved by children and young people. The raft of measures intendedto improve assessment and evaluation at all levels from the student to the system itselfhave done much to stimulate public awareness of assessment and evaluation and beginthe process of embedding an evaluation culture throughout the teaching profession. Inparticular, the creation of a specific national authority to monitor compulsory education(the Quality and Supervision Agency, formerly the School Agency) and an advisory bodyto evaluate priorities in compulsory education (the School Council) sent a strong signalthat evaluation is a top priority.At the local political level (the municipalities) there is also growing support forestablishing an evaluation culture. Notably, Local Government Denmark (KL) ran apartnership study with 34 municipalities on the evaluation culture, management andprofessionalism in inclusive education during the period 2007-2009 and has placedstudent performance and results on the political agenda (KL, 2009). The overall aim ofthe partnership was to strengthen students’ learning outcomes and as such the study wasbased on research and studies to investigate which factors affect student learning. Thepartnership noted heightened political engagement in school communities.OECD REVIEWS OF EVALUATION AND ASSESSMENT IN EDUCATION: DENMARK � OECD 2011

Common Objectives for compulsory education provide a basis for studentassessment and evaluationThe overall objectives of compulsory education in Denmark are specified in therespective legal acts for public and private schools. These are: to provide students withprofessional skills, to help students develop as independent individuals, and to preparethem for their role as citizens in a democratic society. Binding national CommonObjectives were developed in 2003 and must be used in all schools. Common Objectivesspecify the skills and knowledge that ‘teaching should lead towards’ by the end ofcompulsory education in a given subject (end objectives) as well as at different stages ofcompulsory education (form-level objectives). Even in private schools where there isfreedom to adapt these, there is a requirement for each school to set clear end andform-level objectives and for there to be a comparable education to students in theFolkeskoleto ensure equitable access and preparation for upper secondary education.Thus, it can be said that there is a basis for common expectations of outcomes fromcompulsory education in Denmark and – it follows – a common basis for evaluation andassessment of student learning progress against these.

Central and municipal efforts to guide schools on evaluation and assessmentactivitiesNational bodies have made concerted efforts to build up a knowledge base andguidelines on evaluation and assessment activities. The Ministry of Education’s CommonObjectives include guiding curriculum for all courses and subjects in compulsoryeducation. The former School Agency’sEvaluation portalprovides a plethora ofevaluation and assessment tools for use by teachers, schools and municipalities. There areefforts to stimulate capacity building from EVA via its ‘EVA days’ during whichmunicipalities learn about how to draw up quality reports and put these to best use andfrom the Quality and Supervision Agency’s work with facilitating knowledge flow onexperiences and approaches to evaluation among the municipalities.In addition, some municipalities send out local guidance materials on good evaluationpractice to schools. Local Government Denmark (KL) launched a partnership with34 municipalities to promote and evaluate efforts on developing an evaluation culture,among other things (see above). Many municipalities have provided information to theQuality and Supervision Agency’s central repository on municipal quality assurance.

Increased support for the use of evidence and research to inform evaluation andassessment policiesFor more than a decade, EVA has conducted evaluations of compulsory educationwhich have often led to increased political debate about issues identified (Rambøll,2011). Notably, the EVA (2005) evaluation identifying that municipalities did notsystematically use school performance data or follow-up on school performance,highlighted the need to introduce a mandatory evaluation instrument for municipalities.The School Council, as part of its mandate to monitor the academic level in theFolkeskole,commissions research and evaluation studies. The latter often include studieson the implementation and use of new national evaluation and assessment tools,e.g.Individual Student Plans and municipal quality reports and have led to considerationsand pilots of how to make such tools most relevant to local needs. One of the ten majorchallenges identified by the School Council for the ‘Folkeskole 2020’ is to strengthen theOECD REVIEWS OF EVALUATION AND ASSESSMENT IN EDUCATION: DENMARK � OECD 2011

Students are increasingly at the heart of the evaluation and assessment frameworkThe major focus of many national evaluation and assessment initiatives is theassessment of student progress. The high political goals are to increase overall andrelative student performance in international assessments. Students should benefit fromcontinuous assessment from their teachers (legal requirement since 1993) and teachersshould use the Individual Student Plans to document their assessment of student progressagainst the Common Objectives in compulsory education. Such plans, furthermore,should create a more structured dialogue between students and their teachers and offer thepossibility to open up student self-assessment on a more regular basis. The plans alsosupport discussions with students’ parents on their learning progress.The national tests are designed to offer robust measures and rapid feedback toteachers on student knowledge and skills in discrete, testable areas of the CommonObjectives. The nature of the test is adaptive, which means that the test adapts to theindividual student to heighten the relevance and diagnosis for that particular student’slearning. Each student would have a different test experience answering differentquestions depending on his/her ability.Students have the right to choose their pathway into upper secondary education andthe final examinations and their overall final grades in Form 9 serve to document theirlearning. In the final certificate, students can request that other information be includedon the certificate,e.g.the descriptive assessment of their final project and comments fromteachers in subjects where there are no final examinations offered.The Association of Danish Students has an official seat on the School Council and itsviews are well respected at the political level. Further, the ‘Student Council’ is aprominent feature in the DanishFolkeskoleand offers students an official platform toexpress their views on what’s happening in their schools. Seventy per cent of theseStudent councils are represented by the Association of Danish Students.

A culture of trust in the professionalism of all actors, consultation and opennessto flexible, local solutionsAt the heart of the Danish compulsory education system is a strong trust in theprofessionalism of all actors and a culture of consultation and dialogue. Following theOECD (2004a) recommendations on the need to establish an evaluation culture, all majorstakeholder groups formed broad agreement on the importance of working to this end.Stakeholders worked together in a number of groups set up by the Minister of Educationto come to agreement on how to follow up on the OECD recommendations and thesewere documented in “TheFolkeskole’sresponse to the OECD”. The School Councilcomprises members of all major stakeholders that meet twice a year officially to discussand contribute to the School Council Chairmanship’s deliberations. During one suchmeeting, the School Council Chairmanship’s annual report is discussed and stakeholderviews expressed during the discussion are documented in an annex to the report.

In spring 2008 a national agreement was made between Local Government Denmarkand the Danish Union of Teachers and the Confederation of Teachers Unions to introducemore flexibility in working hours for teachers in theFolkeskole.The philosophy behindthe agreement is that ‘teachers should be responsible for independently and professionallycarrying out the overall teaching task, and that leaders should have space to lead, set goalsand frameworks, and to show the way for teachers’ independent, professional work’.However, a common feature in manyFolkeskoleis the use of teacher teams to promotecollective planning, learning and knowledge sharing (Rambøll, 2011). Plus, theFolkeskoleoften include staff with particular expertise and roles to offer targeted supportand/or build school capacity in certain fields (EVA, 2010).Finally, the Ministry of Education relies on both dialogues with stakeholders andevaluation to verify the implementation and use of various central policies. For example,based on stakeholder feedback, the Ministry of Education is open to reducing theperceived bureaucratic burden of some of the national evaluation tools,i.e.the IndividualStudent Plans and the municipal quality reports, in order to maximise their effective useby stakeholders for school improvement.

Use of information technology to gain efficiency in student assessmentThe World Economic Forum (2009) ranked Denmark as the leading informationtechnology nation in the world. The new national tests are entirely computer-based andcapitalise on the efficiency of automatically scored student answers and rapid diagnosticfeedback to students’ teachers. The use of an electronic platform to communicate thenational test results to teachers and schools also enables teachers to set up their ownanalytical tools within the platform and gives greater possibility for professional use ofthe results to follow individual students, groups of students, classes, the effect ofparticular teaching strategies, etc. (see Chapter 4). Digital examinations are also beingintroduced in selected science subjects for the school-leaving examinations in Form 9. Infact, Denmark was one of three OECD countries to pilot a computer-based assessment inscience as part of the PISA 2006 study (along with Iceland and Korea) and as such helpedlead the way in developing this medium in international studies (OECD, 2010f).

ChallengesThe evaluation and assessment framework is incompleteWhile the measures introduced in 2006 have helped to kick start an evaluation culturethroughout the public compulsory education (e.g. Skoleradet, 2010), the evaluation andassessment framework is not yet complete. Some of the key components of theframework are underdeveloped:•Evaluation of municipalities:Municipalities have responsibility for publicschools, including setting the local framework and goals, and monitoring andpromoting school quality. Following the introduction of municipal quality reports,there is a common set of information (‘quality indicators’) available to the Qualityand Supervision Agency for compliancy checking, but the content of such reportsdoes not sufficiently speak to quality issues (Chapter 6). Further, there is nosystematic overview of the existing municipal quality assurance systems andmunicipal follow-up with underperforming schools and the Agency’s supervisionrole could be strengthened (Chapter 7).

38– 3. THE EVALUATION AND ASSESSMENT FRAMEWORK•School internal and external evaluation:Municipalities are responsible for theexternal evaluation of theFolkeskole.Municipal approaches and capacity toundertake school evaluation vary considerably and central expertise on schoolevaluation is limited to the private sector. Similarly, there is great variation inschool use of self-evaluation, despite efforts to promote this by providing training(e.g. EVA days) and self-evaluation tools on theEvaluation portal.There are nolegal requirements for schools to conduct internal evaluations. The Chairmanshipof the School Council (Skolerådet, 2009) reported that 70% of school leadersexpressed the need for increased competence development in evaluation, strategicdevelopment and quality assurance and development.Teacher appraisal:Teacher appraisal is entirely determined at the local level andthere are no national requirements for the evaluation of teacher performance orteacher professional development (see Chapters 5 and 6).School principal appraisal:There is no systematic approach to the appraisal ofschool principals and the OECD review team found little expectation that schoolprincipals are accountable for the quality of teaching and learning in their schools(see Chapter 6). In the absence of national requirements, this varies significantlyamong differentFolkeskole(Rambøll, 2011). Although there is evidence thatschool principals are being held increasingly accountable for their school quality.For example, some municipalities have started to introduce results-oriented short-term contracts for school principals (Rambøll, 2011).

The OECD review team also noted some areas where complementarities among thekey components of evaluation and assessment could be established or strengthened moresystematically:•Teacher appraisal and school evaluation:No attention is paid to the coreprocesses of the quality of teaching and learning in the current municipal qualityreports (see Chapter 6), so there is no guarantee that external school evaluation bymunicipalities addresses teacher appraisal practices. Equally, school self-evaluation does not necessarily place adequate emphasis on how appropriateteacher appraisal and follow-up mechanisms are (see Chapter 5).School evaluation and school improvement:The municipal quality reports serveas a major tool in external school evaluation, but do not sufficiently speak toquality and improvement, with for example, a lack of focus on school self-evaluation and teacher appraisal (see Chapter 6).National student assessment tools and formative assessment:The national testsprovide rapid diagnostic feedback to teachers on student performance in discreteareas against the Common Objectives. The key is to ensure that such informationfeeds into a discussion with students on their future learning plans. Equally, ISPsserve to document student learning, but should also feed into a discussion ofstudent future learning needs and goals (see Chapter 4).Teacher appraisal and school improvement:School development needs should besystematically linked to teacher professional development activities. Considerablebenefits can be obtained through a better alignment of teacher professionaldevelopment in areas of priority in overall school improvement plans (seeChapter 5).

Private schools are not integrated in the evaluation and assessment frameworkThere is specific legislation for private and public schools. Most of the 2006 policyinitiatives to strengthen the evaluation culture in compulsory education do not applydirectly to private schools. While private schools have to demonstrate similar conditionsfor student assessment in as much as they must provide end objectives and educationaldescriptions of how students will reach these, evaluate the student’s learning outcomesand communicate this with parents and evaluate the school as a whole on a regular basis,they have considerable freedom in how they do so. Private schools are not required to usethe Common Objectives, ISPs or national tests. Further, they may opt out ofadministering final examinations in Form 9 by officially informing the Ministry ofEducation. The parents hold the primary responsibility for supervising the educationalquality in private schools. Private schools choose between self-evaluation and a parent-elected certified supervisor. The private schools are accountable to supervision by theMinistry of Education, specifically, the Quality and Supervision Agency. The risk of alimited integration of the private sector in the framework is that there is little guaranteethat their evaluation and assessment procedures are sufficiently aligned with nationalstudent learning objectives.

The implementation and assessment of Common Objectives varies among schoolsand municipalitiesCurrently, Common Objectives vary in their coverage in different subjects and levelof detail provided for teachers. The form-level objectives are few in some subjects –indeed in some there are only end objectives provided. Despite the revision of CommonObjectives in 2009, teachers still report concerns with translating these into curriculumand assessment plans (see Chapter 4). This reported lack of shared understanding of theCommon Objectives leads inevitably to a variation in their implementation and use byteachers. Currently, beyond the final examinations in Form 9, there are no gradingcriteria/assessment exemplars for teachers in understanding how to assess the studentunderstanding of knowledge and skills specified in the Common Objectives. While theavailability of national tests in Danish, English, mathematics, geography, biology andphysics/chemistry will prove a considerable asset to teachers in assessing student progressagainst the Common Objectives, these form only a minor part of expected assessment ofstudent progress in compulsory education.The level of oversight and support by municipalities in ensuring that schoolseffectively achieve the Common Objectives varies significantly (see Chapter 6). Not allschools draft guidelines on evaluation for their schools and only a minority employevaluation advisors to offer specific support to schools in developing their evaluation andassessment practices. The basic template for municipal quality reports may include someOECD REVIEWS OF EVALUATION AND ASSESSMENT IN EDUCATION: DENMARK � OECD 2011

A need to radically improve evaluation and assessment competencies throughoutthe systemWhilst there have been considerable national efforts to stimulate an evaluation cultureby introducing assessment and evaluation activities, guidelines and materials, as well asproviding competency-building seminars in some cases, the OECD review team assessesthat there are still limited evaluation and assessment competencies throughout thecompulsory education system. At the national level there is under-used capacity tomonitor effectively municipal approaches to evaluation and assessment (Chapter 7). Anexchange of practices via the Quality and Supervision Agency reveals that there is greatvariation in the capacity for municipalities to develop and effectively use qualityassurance systems. To shed more light on this issue, the School Council hascommissioned EVA to evaluate municipalities’ capacity to follow up particular schoolswith action plans as determined in the quality reports. Similarly, there is a need toincrease school leader competencies in evaluation and assessment, in particular withregard to ensuring a school self-evaluation process, including teacher appraisal andteacher observation by school leaders and/or peers (Chapters 5 and 6). Initial teachertraining lacks a specific focus on student assessment and given teacher reports on the lackof clarity of Common Objectives, it is probable that there is great variation in theircapacity/approach in awarding students their overall Form 9 grades upon completion ofcompulsory education, plus in their capacity to work effectively with the national testsresults (Chapters 4, 5 and 7).

Need to engage all stakeholders and achieve consensus on the evaluation andassessment frameworkIn going further with building the evaluation culture in compulsory education, it iscritical to engage all stakeholders in this effort. While there was general agreement withthe OECD (2004a) diagnosis of a need to improve educational outcomes and to ensurecontinuous assessment of students, key stakeholder groups were not in completeagreement with some of the national initiatives introduced in 2006 (Rambøll, 2011).Ongoing evaluations of municipal quality reports and ISPs aim to address some concernsraised by key stakeholder groups and evidence from EVA evaluations indicates that thesetools are widely accepted and used by stakeholders. Certainly, there is strong supportfrom students and parents for the ISPs (Rambøll, 2011; OECD review interviews). TheOECD review team highlights the need to heighten efforts to engage teachers in theeffective use of national test results and identifies a lack of clarity on the purpose of thenational tests as a significant risk to both their pedagogical and monitoring value (seeChapters 4 and 7).

Develop a strategic plan to complete the evaluation and assessment frameworkThe OECD review team commends the national and local efforts to establish anevaluation culture in Denmark. To go further, it would be important to develop a strategicplan to complete the evaluation and assessment framework. An effective evaluation andassessment framework would: include key components that are currently missing (theevaluation of municipalities, the evaluation of school leaders and teacher appraisal);achieve proper articulation between the different evaluation components (e.g. schoolevaluation and teacher appraisal); ensure that the different elements within an evaluationcomponent are sufficiently linked (e.g. ISPs and national test results are linked toformative assessment); and ensure processes are in place to guarantee the consistentapplication of evaluation and assessment procedures (e.g. consistency of teacher gradingin Form 9 and consistency of municipal quality assurance systems).For example, it would be critical to ensure that the evaluation of teaching and learningquality is central in the evaluation framework. To this end, there is room to better definethe articulations between school evaluation and teacher appraisal, teacher developmentalappraisal and school principal appraisal, teacher professional development and schooldevelopment, etc. This indicates that school evaluation should comprise the monitoring ofthe quality of teaching and learning, possibly include the external validation of school-based processes for teacher appraisal (holding the school principal accountable asnecessary), and school development processes should explore links to the evaluation ofteaching practice (see Chapters 5 and 6). In the context of school self-evaluation, it is alsoimportant to ensure the centrality of the evaluation of teaching quality and the feedback toindividual teachers.Examples of linkages within single evaluation components which need to bereinforced include the association between teacher appraisal and teacher professionaldevelopment (see Chapter 5), the articulation between school self-evaluation and externalschool evaluation (see Chapter 6) and the relationship between ISPs and national tests andstudent formative assessment (see Chapter 4).Finally, moderation processes are vital to ensure the consistency of the applicationof evaluation and assessment procedures. Priority should be given to ensuring that thereis heightened consistency among municipal quality assurance systems and that thesefocus adequately on the quality of teaching and learning (see Chapters 6 and 7).Further, there is room to build on the existing system of moderation of finalOECD REVIEWS OF EVALUATION AND ASSESSMENT IN EDUCATION: DENMARK � OECD 2011

Strengthen central support and clarify key purposes and objectives of evaluationand assessmentNow that the major centrally-designed evaluation tools have all been introduced totheFolkeskole(and mandatory examinations to private schools), there is considerableneed to strengthen central support to ensure that these tools are linked to effectiveclassroom practices. It should be clearly communicated that the purpose of suchevaluation tools is to improve educational outcomes and that stakeholders should activelyuse these to develop strategic improvement or action plans at all levels.The Ministry of Education should start work with key stakeholders to:•Refine and expand the Common Objectives and to develop a set of performancestandards against these (see Chapter 4). These refined Common Objectives andperformance standards would serve as the basis of student assessment and schoolself-evaluation.Develop evidence-based teaching standards to aid teacher appraisal, developmentand career progression (see Chapter 5).Draw up competency profiles for school leaders and municipal education officers(see Chapter 6).

Significantly invest in evaluation and assessment capacity development acrosscompulsory educationSince 2006, Denmark has introduced a raft of evaluation and assessment tools intocompulsory education – all of which are mandatory for public schools. In this context, thetop priority is to significantly invest in capacity development across the compulsoryeducation system to ensure the effective use of these and other evaluation and assessmentmeasures by stakeholders. The effectiveness of the overall evaluation and assessmentframework depends to a large extent on whether those who evaluate and those who useevaluation results at the different levels of the system have the appropriate competencies.It is natural that the introduction of new evaluation and assessment requirementsnecessitates training and capacity building at different levels – and Denmark has madesome efforts to this end. However, there is room to significantly develop capacity, asfollows:•Develop teacher capacity to assess and engage their students in assessment:Prioritise professional development and teacher networking to build teachercapacity in both formative and summative assessment; develop, with teachers andteacher educators, a set of performance standards against the Common Objectivesand provide sample assessment materials aligned with these; continue efforts toensure teachers’ effective use of results from the national tests and promote theuse of ISPs and classroom assessments to engage students in their self-assessmentand assessment of peers; go further with current moderation approaches bybuilding wider professional networks with an aim to improving teacher capacityto award students’ final grades in Form 9 against the Form 9 end objectives; at thesame time, strengthen and make more explicit the development of teacher skills informative and summative student assessment in initial teacher educationprogrammes.Develop school leader and leadership team capacity in school self-evaluation andteacher appraisal:Train school leaders to implement an authentic evaluation ofteaching and learning, feedback and objective setting at their schools, includingtechniques in teacher observation; strengthen teacher developmental appraisal atschools by preparing the school leader, management teams and lead teachers forparticular evaluation responsibilities and engaging evaluation advisors.Develop municipal capacity in school evaluation, with particular focus on theevaluation of teaching and learning:Replenish central evaluation expertise inEVA to support capacity development at the municipal level; promote municipalpartnerships to develop evaluation capacity (in the mode of KL’s 2007-2009partnership); identify examples of effective municipal quality monitoring systems

Teachers and students report using an admirable mix of different assessment methods inDanish classrooms. However, teachers and schools report difficulty in translating theCommon Objectives into instructional and assessment plans. There is room to developperformance standards against these to promote more consistent implementation and amore active engagement of students in their own assessment. New national tests offer apowerful pedagogical tool to teachers, but they are not yet used effectively by allteachers. To maximise their pedagogical value, they should be further validated, whilebuilding teacher assessment competencies. Teachers carry the major responsibility forstudent summative assessment at the end of compulsory education which allows a broadoverview of student achievement. The written and oral standard examinations in Form 9are criticised by students and others as being ‘outdated’ and students are not able to sitexaminations in all subjects that they study. Accordingly, there should be a review of thepurpose, procedures and content of the final examinations.

4.1 Context and featuresThe approach to student assessment in Danish compulsory educationThe major tradition for student assessment in Denmark has been on school and/orteacher developed assessments and ongoing assessments at the class level (Rambøll,2011). Since 1993, public schools have been legally obliged to provide an ongoingassessment of student learning outcomes. But student assessment has very much beendetermined by the individual school and therefore, there has not been a typical Danishapproach to student assessment in compulsory education. Prior to 2006, the tradition forsummative assessment was very weak. A first significant attempt to introduce a commonbasis for student assessment in compulsory education was the introduction of mandatorynational Common Objectives in 2003 (these were revised in 2009). Such objectives detailthe knowledge and skills that ‘teaching should lead towards’ by the end of compulsoryeducation (Form 9) and also progressively through key stages of compulsory education(see Form level objectives in Table 4.1). They are not a national curriculumper se.Rather, they are intended to provide a framework for the curriculum. The absence of anational curriculum in Denmark reflects the traditional belief that qualified teachersshould own their curricula and syllabi (Egelund, 2005). The legislation states thatmunicipalities should specify and supervise learning targets and frameworks forpedagogical activities (Rambøll, 2011).There were major efforts in 2006 to strengthen educators’ use of Common Objectivesfor ongoing student assessment by introducing externally-determined elements of studentassessment into Danish classrooms:•Form 9 leaving examinations were made mandatory:Upon the completion ofForm 9, Danish students are required to take mandatory school leavingexaminations. The examinations assess the extent to which students haveachieved the end objectives for compulsory education, cover a wide range ofsubjects taught, and use multiple assessment methods including oral, written andproject-based assignments. Each student must sit seven examinations: five fixed(written and oral in Danish, written in mathematics, oral in English andphysics/chemistry) and two randomly selected from each of the major studyblocks of humanities (written in English, history, social studies, Christian studies,and written and oral in the optional subjects of French or German) and science(geography and biology). All written examinations are marked by the students’teachers using national scoring guidelines and an external censor from the State(or in the randomly selected subjects of English, French or German usually acensor from a different school or municipality). All oral examinations are marked

by the students’ teachers using national scoring guidelines and an external censorfrom a different school or municipality or – as a sample – from the State.9•Computer-based adaptive national tests:these were officially implemented in theFolkeskolein March 2010 after a long period of development and pilot tests since2006. A number of compulsory multiple-choice tests are included in this IT-basedtesting system: Danish (Forms 2, 4, 6 and 8), mathematics (Forms 3 and 6),English (Form 7), biology, geography and physics/chemistry (Form 8). The testsystem is adaptive in that the items are tailored to students’ latent ability levels.Test items are selected sequentially according to a student’s performance on theprevious test items: if the student answers the question correctly, he/she gets amore difficult question; if the student’s answer is incorrect, he/she gets an easierquestion next, etc. These efficient national tests provide rapid feedback of testresults to teachers the next day, which can greatly facilitate teachers’ use of thetest results.Individual student plans (ISPs):These are compulsory working tools for teachersto document their evaluation of each student’s learning outcomes in all subjects,including a summary of each student’s test and evaluation results (althoughnational test results are not included, only notes on how these will be followedup). Teachers are obliged to write individual student plans at least once a year forall students in Forms 1 to 7. ISPs are combined with student learning plans inForms 8 and 9.

However, attempts to introduce a national influence on student assessment havemainly impacted the public schools, as in Denmark, the legal framework – and thereforepractices – for student assessment differ between public and private schools. There aredistinct legal acts for theFolkeskole,Private Independent Schools and Lower SecondaryBoarding Schools. Private schools can choose to develop their own end objectives(in similar areas to the national Common Objectives): they must develop curricula thatdescribe student development towards these in each Form and subject, as well as thestudent’s personal development; they must specify the form-level objectives at certaintimes in the lesson plan for Danish, mathematics, English, geography, biology andphysics/chemistry. TheFolkeskoleare obliged to conduct both the national tests and theschool leaving examinations in Form 9. Private schools are obliged to conductevaluations of student learning outcomes, however, they enjoy a higher freedom thanpublic schools in how they do so. For example, private schools can inform the Ministry ofEducation that they will not conduct the mandatory school leaving examinations inForm 9 – 28 schools have currently chosen not to administer Form 9 examinations(Rambøll, 2011) – or can opt out of particular subjects (Christianity and history) if thesedo not match the school values, but students instead must sit another examination in thehumanities block.10However, in practice the vast majority of private schools doadminister the Form 9 school leaving examinations. With regard to the national teststhese are not compulsory for private schools.

National initiatives and summative assessmentThe major summative assessment of each student comes at the end of compulsoryeducation. At the end of Form 9 each student receives a Diploma containing the followingkey assessment information: most recent teacher ongoing assessment mark (the student’sattainment level); results on the school leaving examinations in Form 9 (five fixed andtwo selected); and teacher assessment in the subject ‘Physical Education’. Further, thestudent can request for additional information to be included in the Diploma, for example:the written teacher remarks and/or grades for the mandatory project assignment and forthe optional assignment; the written teacher reviews on other subjects not tested in thefinal examinations.Individual Student Plans serve a summative function (in addition to a formativefunction) as they record student achievement in a systematic way (Madaus and Kellaghan,1993). ISPs serve as a tool to communicate to parents a teacher assessment of where eachstudent stands in relation to the Common Objectives. The ISPs also serve as a record ofstudent achievement in each Form which can be shared with teachers in subsequent Forms.Such records are enhanced by the inclusion of student results on classroom tests (althoughdo not include information from national tests). The national tests hold a summative aspectfor students as their results serve as an indicator of how much they demonstrate they knowon expected knowledge and skills in discrete, testable areas against the national CommonObjectives. Beyond this, national test results provide some summative information at thenational level (to be published as a national profile) and can be used to monitor outcomesat the municipal and school levels (see Chapters 6 and 7).

Support for student assessment available to teachersTeachers use a number of standardised reading and mathematics tests for diagnosticpurposes throughout the school year. Danish teachers have access to support andexpertise on various aspects of schooling from resource individuals (usually teachers)employed at school (Rambøll, 2011). However, a study in 2009 revealed that specialassessment/evaluation advisors are rare (only reported in 8% of schools) and typicallylend support at the school level as well as to individual teachers (EVA, 2009a). There arealso several support materials available to Danish teachers, including many different freeassessment and evaluation tools offered by the Quality and Supervision Agency in itsnationalEvaluation portal.For example, guiding materials on how to assess students inthe final examinations in Form 9 prepared by the examiners and subject advisors of thefinal examinations, including advice, guidance and ideas for classroom teaching and alsocriteria for student evaluation and assessment in classroom activities. Support materialsare also provided by municipalities and the Danish Evaluation Institute (EVA). Thesesystems are in place to ensure both the efficiency and effectiveness of student assessmentfor formative and summative purposes.OECD REVIEWS OF EVALUATION AND ASSESSMENT IN EDUCATION: DENMARK � OECD 2011

4.2 Strengths and challengesStrengthsStrong potential in teacher culture to optimise formative assessment practiceThe OECD review team gained the impression that the predominant classroomassessment culture in theFolkeskoleemphasises the diagnostic role of assessment inteaching and learning. This holds very strong potential for effective formative assessmentpractice, defined by the OECD (2005a) as the frequent assessment of student progress toidentify learning needs and adapt teaching. Indeed, there is a legal requirement forcontinuous assessment of students’ learning as a basis for further planning andorganisation of teaching (Folkeskole Act § 13 paragraph 2). Traditionally, teacherfeedback emphasises qualitative, descriptive comments on student learning progress,given the absence of grades before Form 8. Research evidence supports teachersproviding descriptive comments rather than numerical or letter grades to young children,as students engage more productively in improving their work – students tend to ignoreteachers’ comments when numerical marks are also given (Blacket al.,2004). During theOECD review, educators described several ways they make use of student assessment,including reports that: they assess students to gather information about learning needs andtailor instruction to meet the needs for students at various proficiency levels; they usedaily instructional activities to identify students’ strengths and areas of improvement;assessments provide opportunities to engage other teachers, parents and students indialogue about future learning. Further, teachers voiced appreciation for the availabilityof various assessment tools, especially, for reading, spelling, and mathematics, andstudents voiced appreciation for teachers’ feedback on their task performance.

Wide range of student assessment methods used by teachersThe inclusion of a mandatory project assignment in Form 9 is a positive signal toconsider different methods of student evaluation and it is useful that students can requestfor this to be included in their final diploma. In addition, the OECD review team notedvarious admirable features of classroom assessment in Denmark as reported by theteachers and students, notably a mix of different assessment methods. Besides the use of anumber of formal standardised tests for diagnostic, placement, and evaluation purposes inreading, spelling, and mathematics, teachers use various forms of alternative assessmentto make students demonstrate the skills and knowledge they have mastered. For example,teachers reported the use of written essays and laboratory reports (‘products’), role plays,experiments and presentations (‘performance’) and collections of student work(‘portfolios’).11These alternative assessment methods are easily integrated intoinstruction and can be used to assess both processes and outcomes of learning.Alternative assessments can assess creative and critical thinking skills that are necessaryfor knowledge transfer to real life.

A good mix of teacher judgement and standard examinations in the final diplomaTeachers are at the heart of student assessment in Danish compulsory education: thereis legal provision for their role of continuous assessment of students throughoutcompulsory education and this continuous assessment represents the students’ overallattainment mark in the final diploma at the end of Form 9. While other information isincluded in the final diploma, notably results on the school leaving examinations, theteacher’s professional judgement carries most weight in summative assessment at the endof compulsory education. At the same time the final examinations are taken seriously byteachers and students. While students’ teachers are still at the heart of the markingprocess, the standard format of the examinations, the common marking guidelines and thefact that a sample of examinations are marked by an external censor provides an equitableway to judge whether students have achieved the national Common Objectives.

National tests provide rapid feedback to educators on student performanceagainst Common ObjectivesOne of the most significant changes in theFolkeskolesince the 2004 OECD review isthe national tests. They are now fully functional and impressively computer-basedadaptive tests. The national tests assess students’ academic achievement in select subjectareas across Forms 2 to 8, and align the test content with prioritised form-level CommonObjectives (see Table 4.1). The OECD review team assesses that the Quality andSupervision Agency’s communication strategy shows a responsible and healthy attitudetowards what the national tests can cover and what their strengths are. Officialinformation on the national tests produced by the former School Agency clearly repeatsthe message that the national tests only measure a discrete area of student knowledge andskills and teachers should use a range of other tests to gauge student progress. Forexample, it is stressed that the Danish test only measures students’ proficiency in readingand a wide range of key knowledge and skills in Danish teaching (e.g., spelling, grammar,punctuation, cultural understanding, literary knowledge, ability to express oneself) is nottested. Educators are aware that the tests provide only a snapshot of students’achievement levels in select learning targets and subjects (Wandall, 2010).At the same time, within a discrete area, the adaptive nature of the national testsprovides a very accurate diagnosis of student performance and of course, this has theadvantage of being aligned to the form-level Common Objectives. In a regular test, allstudents would be confronted with the same range of questions with varying difficulty.Depending on the student’s ability, this could offer only limited information to teacherson their learning misconceptions or strengths and weaknesses in particular areas tested,e.g.a student may simply not answer many questions or give wrong answers. However,adaptive tests can provide much more meaningful diagnostic information to teachers aseach student sits a different test including questions that are adapted to his/her abilitylevel. This allows a more thorough feedback on student performance and sheds more lighton the types of tasks they can perform. Indeed, initial pilots of the national tests revealedmajor performance differences among students in each Form12. National tests can offer

52– 4. STUDENT ASSESSMENTextremely useful diagnostic information to teachers in the context of increased demand onDanish classrooms to be inclusive.The major value added for Danish educators is that they can receive rapid feedbackfrom the national tests which fosters the use of the test results for adapting teaching andlearning for individual student progress (see Box 4.1). This is in strong contrast to severalnational test systems where educators receive student test results several months after thetest was administered. Further, there is potential for teachers and schools to track studentprogress in the test areas over time, as teachers can decide to administer the test to theirstudents up to three times – one mandatory and two voluntary. Teacher questionnaireswere sent to one-third of the schools in June 2010 and results indicate that four out of fiveteachers find the results useful in some way (e.g. for planning instruction, communicationwith parents, informed analysis) and a minority of teachers reported they had alreadyused results to plan instruction (Wandall, 2010).

Overview for teacher: an overview of the available results for the teacher’s classesand student groups. Results appear as an overall score for each class within eachprofile area as well as an overall score – an assessment across the profile areas.All students: a summary of results and status for each student’s scores in each profilearea and a comprehensive assessment of each student.Individual students: information for individual students on their response(right/wrong/not answered/length of time taken to answer the task) on the test tasks ineach profile area. For each task, general information is given on: task difficulty (on ascale of 1 to 5, with 5 being the hardest); topic area (core academic content); typicaltime students take to answer this task; where the task fits on the overall assessmentscale (scale scores from 1 to 100).Teacher-specified groups: teachers can specify particular groups of students and seean overview of their results,e.g.for boys and girls, or for students following aparticular teaching strategy/programme.

Availability of assessment resources and supportsThe former School Agency invested significantly to build an on line resource forteachers offering a range of assessment and evaluation tools (theEvaluation portal).Further, there is much information related to the national tests including articles on howteachers can interpret results in these tests and use these most constructively to adaptstudents’ further instruction. Many of the official booklets on the Common Objectives gowell beyond a presentation of the end and form-level objectives and suggested teachingplans and provide stimulus information on student evaluation. As such, they try topromote a teaching approach incorporating evaluation and assessment against theCommon Objectives. For example: for natural science and technology, sample problemsand associated teacher evaluation questions are provided for different stages of the classinstruction, including learning goals, study plan and completion and evaluation; forDanish and mathematics general suggestions are provided on differentiating instructionand ensuring continuous assessment of students and inviting teachers to access materialson theEvaluation portal.13TheEvaluation portalprovides teachers with access tovarious assessment materials. These external resources are valuable material for teachers’classroom assessment. While these materials may not be aligned with the CommonObjectives, they can contribute to establishing consistency in assessment practice in theFolkeskole.The OECD review team commends the former School Agency’s efforts tocreate this rich support system for student assessment. Some teachers reported to theOECD review team that they found resources useful, especially when alternative project-based assessments were necessary to evaluate students’ ability to transfer skills to othersituations (e.g. science). Teachers’ use of these resources is not yet consistently observedacross classrooms and schools.Recent policy initiatives to build teacher capacity in evaluation and assessment arecommendable. The availability of resource teachers at schools provides important supportto teachers. Although, assessment and evaluation advisors are few they have the potentialto offer critical support to the majority of teachers whose initial training did not giveparticular emphasis to student assessment and evaluation (see Chapter 5). Thus, providingtraining to special resource teachers who are expected to consult with teachers inassessment practice appears to be an effective short-term strategy to build teachercapacity in this area. Test consultants and supervisors can collaborate with classroomteachers in planning, interpreting and using assessment.

54– 4. STUDENT ASSESSMENTThe ISPs are well in accordance with theFolkeskoleact that students and parents mustbe informed of student progress toward their learning targets on a regular basis. The OECDreview team judges that the ISPs contribute to formalising Danish assessment practice bydocumenting students’ learning progress for dialogue with key stakeholders. The positivefeature of the Danish ISPs is their emphasis on the student’s future learning rather thansummative learning outcomes. Official evaluations, strong support from national levelparent organisations and student associations (see Rambøll, 2011) and stakeholder feedbackduring the OECD review confirm that the ISPs are well received by parents and teachers. Inshort, parents appreciate a written summary of their child’s progress because they feel thatthey are better prepared for their meeting with teachers. Teachers perceive benefit intransferring documented information on student achievement to subsequent teachers and assuch ISPs play a crucial role in tracking individual students’ developmental growth overtime. Teachers recognise the role of ISPs in easing communication with parents. The addedworkload ISPs entail for teachers is a bone of contention, but there is a current pilot to alloweducators more flexibility in determining and prioritising the content of ISPs. Depending onthe evaluation of this pilot this may lead to a modified approach to drawing up ISPs.

Assessment for equity and inclusionThe Danish assessment culture is driven by the view of education for equity andinclusion. In this view, all children have their right to education and through this toachieve their learning potential. Inclusion is sought through ‘responsive’ education tovarious learning needs in optimal learning environments. Students are not treateddifferently based on their ability measured by tests. TheFolkeskolemust embrace allstudents regardless of their background and ability. During the OECD review, moststakeholders expressed their support for equity and inclusion. Research supports theDanish view that students who feel included and connected to the school communityperform better in their academic tasks (Schargel, Thacker and Bell 2007). Someaccommodations have been made in the national tests to make them more inclusive:alternative versions of the national tests were prepared for students with special needs anda special Danish test for bilingual students is offered. Denmark’s focus on equity andinclusion should continue by implementing strategies to identify barriers and biasesagainst fair student assessment.

ChallengesMany teachers struggle in translating Common Objectives into curriculum andassessment plansThe Common Objectives set the national educational goals and values by specifyingthe knowledge and skills that ‘teaching should lead towards’ by the end of compulsoryeducation. They also include form-level objectives for the majority of subjects incompulsory education (although for some subjects there are only end objectives). Thecurrent Common Objectives were refined in the light of the new objective of theFolkeskolein 2009; however, during the OECD review several teachers still expressedconcern about the lack of clarity and specificity in guiding curriculum development. Thisechoes a finding in the report by the ‘Flying Squad’ (Danish School Agency, 2010; seealso Box 2.1) that ‘the national objectives for the contents of the teaching are unclear, andit can therefore be difficult for schools and for the individual teachers to translate theseinto lesson plans and learning objectives’.OECD REVIEWS OF EVALUATION AND ASSESSMENT IN EDUCATION: DENMARK � OECD 2011

Common Objectives are not provided for every Form meaning that teachers need totranslate the content of form-level objectives into an instructional plan over severaldifferent Forms. For example, in mathematics objectives for the Form 3 serve for Forms 1to 3, objectives for Form 6 serve for Forms 4 to 6 and objectives for Form 9 serve forForms 7 to 9. While this has the advantage of giving more ownership to teachers andbringing different Form teachers together to establish a coherent instructional plan, thereseems to be a need for more structure for several teachers. The Common Objectivesshould play a key role in establishing the interdependent relationship among curriculum,instruction, assessment, and local instructional policies (Allington and Cunningham,2002). National tests are aligned with the Common Objectives, but it seems reasonable tospeculate that given reports from some teachers on the lack of clarity of CommonObjectives, there would equally be a lack of alignment regarding the other much widerareas that should be regularly assessed. Research shows that misalignment amongcurriculum, instruction and assessment compromises student achievement (Baker andLinn, 2000).In general, the Common Objectives provide core content standards that articulate theknowledge and skills that ‘teaching should lead towards’, but appear to lack performancestandards that describe concrete learning goals that students should meet. The OECDreview team is of the opinion that classroom-based assessment has not yet reached itsfullest potential because rubrics that detail evaluation criteria for clear expectations andachievable targets for students are not common place. Clear rubrics can make teachers’assessment transparent and fair and prompt students’ metacognitive reflection on theirown learning process. They are used to define what constitutes excellent work in studentassessment. In classrooms where teachers increasingly use ‘3P assessments’(performance, product, portfolios), teachers should be able to explain to students howgood is good enough using rubrics that entail clear evaluative criteria and qualitydefinitions (Popham, 1997).

Weak differentiated instruction and increased demand to make classrooms moreinclusiveWhile there is a strong basis for formative assessment in Danish compulsoryeducation, evidence on outcomes indicates that teachers may struggle with the secondpart of adapting their instructional strategies after diagnosis of student learning status.Crucially, evidence from the OECD’s PISA assessments has repeatedly shown acomparatively low proportion of Danish students able to perform the most demandingassessment tasks (see Chapter 2). This suggests that high achievers are not adequatelychallenged and stimulated to bring forth their full academic potential in theFolkeskole.This is a key challenge if Denmark is to achieve its ambitious goal to be among the topfive international performers and indicates a significant need to increase educators’ use ofassessment for planning and evaluating differentiated instruction.At the same time there will be increased demands on teachers to make classroomsmore inclusive and to effectively integrate many students who are currently offeredspecial educational provision (e.g. Skolerådet, n.d.). While international evidenceshows that Denmark has comparatively fewer weaker performers (see Chapter 2) andthis would appear to reflect theFolkeskole’semphasis on equity and inclusion ineducation, there is evidence of significant performance disadvantage for some students.In particular, Denmark is well aware of the challenge of increasing the academicperformance of students with a migrant background (see for example OECD, 2006;OECD REVIEWS OF EVALUATION AND ASSESSMENT IN EDUCATION: DENMARK � OECD 2011

56– 4. STUDENT ASSESSMENTNuscheet al.,2010; Danish Ministry of Economics and Business Affairs, 2009). Forexample, the School Council judged that the aspect of teaching Danish as a secondlanguage was not adequately included as a dimension of the Common Objectives acrossdifferent subjects (Skolerådet, 2008). The ‘Flying Squad’ (see Box 2.1) asserts that theFolkeskolelacks ability to cope with all children, especially those from sociallydisadvantaged family backgrounds. The national tests serve as one tool to check thatstudents have gained the basic skills in key subjects, however – beyond the special testoffered for Danish as a Second Language – they may not give an accurate diagnosis ofbilingual students’ cognitive ability. For example, although students have masteredmathematical concepts, they may have difficulty understanding some of the morelinguistically complex mathematics problems in the national tests. Research shows thatteachers can assess these students’ content knowledge by simplifying the linguisticfeatures of mathematics problems. Such careful modifications can also provide similaradvantages to students with disabilities (Abedi, 2004; Abedi, Hofstetter and Lord,2004).

Inconsistency in classroom assessment practice across schools and municipalitiesThe OECD review team shares the School Council’s assessment that the evaluationculture has been significantly improved following the launch of key national initiativesin 2004 (Skolerådet, 2010). However, good student assessment practice is not spreadevenly across schools and municipalities. The availability of in-house specialists and useof test materials varies from school to school – indeed a recent evaluation revealed thatassessment specialists are only in 8% of schools (EVA, 2009a). The OECD review alsobrought to light – albeit it based on only a small sample of schools – that there wasvaried practice in use of free tools available on theEvaluation portal.Though ISPs andthe obligation for municipalities to produce a quality report provide a means to formaliseassessment practice, they have not yet reached the fullest potential. Though manyschools – often as part of a municipal policy – choose to use commercially availablediagnostic tools for reading and mathematics skills, teachers reported to the OECDreview team that they need both a range of evaluation tools, including subject-specificassessment materials, guidelines, and – importantly – they also need professionaltraining.The OECD review team interviews with pre-service teacher educators revealed thatthe pre-service teacher education programmes presently offer little training in studentassessment for teacher candidates. The ‘Flying Squad’ (see Box 2.1) also concurred onthis matter stating that not all teacher education colleges promote a culture where teachersand students continuously evaluate themselves and each other and reflect on suchevaluation (Danish School Agency, 2010).This heightens the importance of effective municipal support to develop teacherassessment capacity. During the OECD review, some teachers reported theirapprehension regarding assessment activities and their perception that these increasedworkloads. This indicates that assessment is not an integral aspect of their teaching. Infact, effective teachers integrate assessment into their teaching and do not see it as anadditional burden on their teaching responsibilities (Stiggins, 1995).

Lack of engagement of students in their self-assessment and peer assessmentStudent involvement in the assessment process is crucial for developing a sense ofownership in learning. The Assessment Reform Group (2002) in England emphasises theinvolvement of students in effective formative assessment, defining it as the ‘process ofseeking and interpreting evidence for use by learners and their teachers to decide wherethe learners are in their learning, where they need to go and how best to get there’. TheOECD review team gained the impression that the involvement of students in theirassessment is not wide-spread practice in theFolkeskole.In most schools the teamvisited, students and their teachers did not report that students participate in settinglearning goals, reflecting on their progress, and evaluating their learning outcomes,indicating a rather passive role for students in assessment activities. Of course, there werealso examples where teachers and students reported they were jointly involved inassessment activities. Teachers should solicit students’ input when creating assessmentrubrics and should provide students with opportunities to use these rubrics for self- andpeer-evaluation purposes (Ross, Rolheiser and Hogaboam-Gray, 1999). The ISPs provideone mechanism to actively engage students in assessment of their progress at regularintervals in a more systematic way. However, the current requirement is for ISPs to bedrawn up at least once a year – this is clearly not often enough to engage students in aserious ongoing self-evaluation.

Engaging teachers to effectively use the results of the national testsThe Ministry of Education clearly states that the intended purposes of the nationaltests are two-fold: to provide teachers with diagnostic information about students’strengths and areas of improvement and to monitor school performance. While during theOECD review, some teachers reported that they found the test results useful andconsistent with their own assessment, others raised concerns about using the test resultsfor accountability. The Danish Union of Teachers (DLF) reported to the OECD reviewteam that it perceives that the tests were designed as a control mechanism and not as adiagnostic tool (see also Rambøll, 2011). DLF believes that the timing of testadministration should be set by the teachers and that this would significantly increasetheir use of the national tests. (It is of note, that in future teachers will be able to scheduleadministration of the national tests anytime during the period 1stFebruary to 30thApril[Wandall, 2010]). Such concerns are not unexpected because a nationwide standardisedtest is unprecedented in theFolkeskole.Also, the initial implementation issues inadministering the computer-based tests damaged their credibility among someeducators14.The challenge is to continue open dialogue to establish evidence-based credibility forthe national tests among teachers. The OECD review team met with teachers who wereeffectively working with the test results and saw these as a free source of extra diagnosticmaterial to confirm their own professional judgement. However, the OECD review teamgained the impression that due to their ideological opposition, many teachers do notcurrently use the national tests well which renders them of limited value. The OECDreview team sees considerable value in rapid feedback to teachers that the national testsoffer, however, it is pivotal, at this early stage of the system implementation, to build14.Although the 2010 tests were largely successful, there was a two-week period when testscould not be conducted due to technical issues and not all students who should have sat thetests were able to (Wandall, 2010).

58– 4. STUDENT ASSESSMENTsupport among teachers and to promote the effective pedagogical use of test results. Thetesting system requires research evidence on the extent to which the interpretations anduse of the test results are appropriate, meaningful, and useful (Messick, 1989). Denmarkshould avoid the pitfalls faced by other countries when introducing national tests. Tensionbetween formative and summative assessments arises when teachers are responsible forboth. In classroom-based formative assessment, teachers offer assistance and providemultiple opportunities for students to demonstrate their proficiency. This role is at oddswith their role as a test administrator in external testing (Haertel, 1999). When teachersexperience such tension along with changes in national assessment policies, they becomeconfused, counter-productive and resistant to changing their practice despite substantialtraining (Black and Wiliam, 1998; Gippset al.,1997; Shepardet al.,1996).The main concerns for teachers reported during the OECD review were associatedwith negative consequences resulting from potential future high stakes attached to thetests, including narrowing the curriculum by teaching to the tests and emphasising basicknowledge and skills that are measurable by multiple-choice test items. This indicateslack of clarity on the purpose of the national tests and the types of skills they assess. Suchconcerns stem from teachers’ awareness of research evidence surrounding the misuse ofhigh stakes national tests by educators in other systems. Madaus and Kellaghan (1993) inan analysis of differentiated teacher views on standardised assessments in the UnitedKingdom identifies the important influence that the use of test results for multiplepurposes exerts on teachers’ views. The use of standardised assessment results inEngland, Wales and Scotland, emphasised different purposes of assessment such asformative, diagnostic, summative, and evaluative.15In England and Wales, despite thedesign of the tests to provide diagnostic information for teachers and formative feedbackto students, the publication of test results for schools caused teachers to perceive them as‘high stakes’. In Scotland, teachers viewed these as ‘low stakes’ as results were used byteachers to provide diagnostic and formative information to students and could becommunicated to parents and the school board only. However, Mooset al.(2008) findthat school leaders in countries with tighter accountability (e.g., the UK, the USA,Canada, China and Australia) tend to accept standards-based testing as is, but considerthat results from national tests alone are insufficient indicators of school effectiveness andthat the goal of education is more than teaching basic skills.

Final examinations in compulsory education are criticised as ‘outdated’While the OECD review team noted that both teachers and students took the finalexaminations in Forms 9 and 10 seriously, there were some criticisms voiced on theactual content and nature of the examinations, plus on their limited coverage of subjectsoffered in theFolkeskole –this would mainly refer to the Practical/Music subjects, butalso not every student sits an examination in all the subjects he/she follows in thehumanities and sciences (see Table 4.1). The ‘Flying Squad’ (see Box 2.1) calls to‘modernise theFolkeskoleleaving examinations and make them count for students’15.Madaus and Kellaghan (1993) define: formative use as ‘the positive achievements of astudent may be recognized and discussed and the appropriate next steps may be planned’;diagnostic use as ‘learning difficulties may be scrutinized and classified so that appropriateremedial help and guidance can be provided’; summative use as ‘the recording of theoverall achievement of a student in a systematic way’; and evaluative use as ‘some aspectsof the work of a school, a local education authority or other discrete part of the educationalservice can be assessed and/or reported upon’.OECD REVIEWS OF EVALUATION AND ASSESSMENT IN EDUCATION: DENMARK � OECD 2011

4.3 Pointers for future policy developmentDenmark has a strong basis for formative assessment and has introduced severalmeasures to strengthen the summative assessment of students over recent years. Thepreceding discussion of strengths and ongoing challenges within the Danish assessmentsystem suggest a number of potential directions for policy makers to consider. In allcases, these strategies involve teachers and aim to develop their student assessmentcapacity. These include:•••••••Collaborate with educators to develop performance standards for the CommonObjectives;Conduct research on effective assessment practices in the classroom;Develop teacher assessment literacy and competencies;Ensure students are actively engaged with and proficient in assessment;Maximise the pedagogical value of the national tests;Further validate and develop the national tests;Review the purpose, procedures and content of the final examinations in Forms 9and 10.

Collaborate with educators to develop performance standards for the CommonObjectivesThe OECD review team commends the use of national Common Objectives in Danishcompulsory education and the efforts made in 2009/10 to tighten and clarify the CommonObjectives so that they can better guide curriculum development and assessment.However, various stakeholders stated that they are still not clear enough. The currentCommon Objectives articulate the knowledge and skills that ‘teaching should leadtowards’, but not what students are expected to learn at key stages in each of the mainsubjects. The next step is to establish links among curriculum, teaching and learning, andassessment based on these Common Objectives. This is key to strengthening the Danishevaluation and assessment culture. A lack of alignment among objectives, curriculum,teaching and learning, and assessment compromises the quality of education and leaveseducational goals unfulfilled. The Common Objectives are the anchor that links the threemajor pedagogical elements in theFolkeskole(see Figure 4.1).

Box 4.2 Performance standards in the United States and CanadaIn the United States and Canada, States and Provinces make use of performance standards to guide a morerigorous and transparent assessment of students. Regarding the United States, Perie (2008) compares theperformance level descriptors used across different States and suggests best practice for developing these.1First,policy makers should determine the number of performance levels to use (typically no more than four levels) andcarefully name these levels so as to reflect policy makers’ values (e.g. does not meet standards/partially meetsstandards/meets standards/exceeds standards). Second, policy makers, possibly in collaboration with contentexperts, write generic policy definitions for each performance level that would apply and guide performance leveldescriptors for all subjects and all Grades. Third, once policy definitions have been set, content descriptions fordifferent subjects and grade levels are developed. An example from Oregon (below) illustrates this: the policydefinitions set the standard for performance in all subjects; a general achievement level definition is provided foreach subject (reading in this example); then for each subject, a series of specific achievement level descriptors areprovided for each Grade on defined knowledge and skills categories (e.g. for Grade 8 in reading: Read to performa task (see below); Vocabulary; Demonstrate general understanding; Develop an interpretation; Examine contentand structure/Informational text; and Examine content and structure/Literary text).Extract from Grade 8 reading performance standards in Oregon, the United StatesStandardsPolicy definitionsDoes not yet meetStudents do notdemonstrate masteryof grade-levelknowledge and skillsrequired forproficiency.Students do notdemonstrate masteryof grade levelreading/ Literatureknowledge and skills.They have a limitedcomprehension ofgrade level text andcannot makemeaningfulinterpretations or ananalysis of text.Misuse or ignoreaspects of practicaltext (headings, boldprint, numbering) todecipher text whenreading to perform atask.Nearly meetsStudents demonstratepartial mastery of grade-level knowledge and skillsrequired for proficiency.MeetsStudents demonstratemastery of the grade-levelknowledge and skillsrequired for proficiency.ExceedsStudents demonstratemastery of grade-levelknowledge and skillsexceeding the requirementfor proficiency.

Box 4.2 Performance standards in the United States and Canada (continued)The introduction of performance standards can also promote a set of highly valued skills/competencies thatteachers should help students to develop in all subjects. For example, in Ontario, Canada, teachers are required touse an ‘achievement chart’ comprising four common categories used in all subject areas and disciplines:knowledge and understanding; thinking; communication; and application. These four categories are interrelated,reflect ‘the wholeness and interconnectedness of learning’ and help teachers to focus on students’ development ofthinking, communication and application skills, in addition to their acquisition of knowledge (Ontario, Ministry ofEducation, 2010). Thus, teachers are required to plan instruction, learning, assessment and evaluation carefully soas to address students’ acquisition of content knowledge in balance with their development of skills to think,communicate and apply their acquired knowledge, in an appropriate way over the course of the year/term.Finally, some systems offer examples of student work collected when compiling and developing thestandards, to help teachers apply these in their classrooms. In British Columbia, Canada, specific performancedescriptors are provided for different subjects at each grade level (against the four standard performance levels)and include helpful examples of either student responses at each performance level or teacher observations ofstudent performance and their eventual performance level rating (see:www.bced.gov.bc.ca/perf_stands/).Illinois,the United States provides samples of classroom assessments in each subject demonstrating the performancestandards (see:www2.isbe.state.il.us/ils/html/descriptors.htm).1.Note that such performance standards generally describe student performance as measured in standardised assessments,following the requirement of the federal policy (the No Child Left Behind Act, 2001) that any assessment developedunder the act must include at least three performance levels.

Refined Common Objectives and common performance standards can guide studentassessment more effectively. Such explicit definitions of the critical skills and knowledgeat each stage of student learning enables teachers to select, develop and use the mosteffective assessments to determine where the teaching and learning process needs to beadjusted and how to do this to ensure student progress (Popham, 2008). First, teacherswill benefit from a more obvious translation of the Common Objectives to curriculum.Second, teachers can explain to students how good is good enough, using theperformance indicators included in the Common Objectives. Performance indicators canhelp students understand how their essays, laboratory demonstrations, or role-plays willbe evaluated. Based on the common performance standards, teachers can develop generalrubrics for each subject in teacher teams drawing from materials provided on theEvaluation portal,including assessment materials, sample rubrics and examples ofstudent work at each performance level.Further, teachers can engage students in constructing specific rubrics and use these tofacilitate peer assessment, self-assessment as well as teacher feedback. The co-construction of rubrics promotes both student learning and reflective teaching practice(Andrade, 2005; Jonsson and Svingby, 2007). As students internalise the criteria forevaluating their work, they are better able to connect their performance with theirpreparation, and develop an internally oriented sense of self-efficacy (Stiggins, 2005).Teachers can use classroom assessment as the vehicle for helping students develop,practice, and become comfortable with reflection and with critical analysis of their ownlearning (Earl and Katz, 2008). These evaluation criteria and quality definitions should bealigned with the Common Objectives and common performance standards. In this way,the Common Objectives can positively influence consistency and equity of teachers’assessment practice across schools and municipalities.

Conduct research on effective assessment practices in the classroomMore research is necessary to understand what works for Danish students in theclassroom. Student assessment should be informed and guided by various perspectives oflearning. Different perspectives hold different views of learning, accordingly resulting indifferent methods of assessment (Greeno, Collins and Resnick, 1996; Pellegrino,Chudowsky and Glaser, 2001). Cognitive perspectives view learning as a process thatinvolves individual students in structuring knowledge and skills and transferring them tonew situations through socially mediated interactions. On the other hand, situativeperspectives attribute a learning process to ways in which individuals interact with andparticipate in communities, and develop their identities (Lave and Wenger, 1991).Evaluation should be guided by such theories of learning and should assess students’growth in problem solving, reasoning, communication, and social participation (Andersonet al.,2000). The Danish student assessment framework should be based on a soundtheoretical foundation of learning. It should guide teachers in deciding what to assess,how to assess and how to use information. Assessment policies without research evidencecan result in well-intended but ill-informed instructional practice. This recommendationresonates well with the ‘Flying Squad’s call for more rigorous research in building aknowledge base for theFolkeskole(Danish School Agency, 2010; see Box 2.1).In 2007, the Norwegian Ministry of Education and the Norwegian Directorate forEducation and Training launched the ‘Better Assessment Project’ to shed more light oneffective formative assessment (Norwegian Directorate for Education and Training,2011). This large-scale research project explored whether criteria developed in differentways for different subjects could give a more subject-related and fairer assessment ofstudents’ competencies. The Norwegian Directorate provided guidance and support to the77 participating schools (primary, lower and upper secondary). Teachers in primary andlower secondary schools are testing out the use of assessment criteria in the subjectsNorwegian, mathematics, social science and home economics in Forms 2, 4, 7 and 10.Results from the research will feed directly into policy and competence building.

Develop teacher assessment literacy and competenciesInstructional decisions should be driven by assessment data that are systematicallyaccumulated, analyzed and evaluated in both short and long terms (Hamiltonet al.,2009).Over recent years, there have been attempts to launch theFolkeskoleon the road tobecoming a data rich environment. There is now student data available to teachers in theFolkeskolefrom the national tests, the school leaving examinations, individual studentplans, and standardised diagnostic tests. Strong assessment competencies and the abilityto effectively interpret data are central not only in making the right diagnosis, but also inmaking the right decisions (Barber, 2009; Heritageet al.,2009). Teachers’ professionaldevelopment in student assessment is of the utmost importance in Denmark. If newteachers enter the teaching profession without basic knowledge and skills about how toassess student learning, it is likely to prolong the pathway to excellence in theFolkeskoleeducation. Building teachers’ assessment literacy takes time and it is crucial that this isadequately covered in initial teacher education. Faced with a similar need to enhance theinclusion of student assessment in initial teacher education, the Norwegian Directorate ofEducation funds a ‘Network For Assessment’. The network links various teachereducation institutions and aims to develop expertise within each institution to improve thecompetence of teacher educators in assessment.

64– 4. STUDENT ASSESSMENTTeachers who are assessment literate and competent can distinguish high-qualityassessment from poor and can apply that ability to improve their own assessment practice(Stiggins 1991, 1995). In the United States, teachers’ assessment competence ischaracterised in terms of seven principles in the standards for teacher competence ineducational assessment of students (American Federation of Teachers, the NationalEducation Association and the National Council on Measurement in Education, 1990):1. Teachers should be skilled in choosing assessment methods appropriate forinstructional decisions.2. Teachers should be skilled in developing assessment methods appropriate forinstructional decisions.3. The teacher should be skilled in administering, scoring and interpreting the resultsof both externally-produced and teacher-produced assessment methods.4. Teachers should be skilled in using assessment results when making decisionsabout individual students, planning teaching, developing curriculum, and schoolimprovement.5. Teachers should be skilled in developing valid student grading procedures whichuse student assessments.6. Teachers should be skilled in communicating assessment results to students,parents, other lay audiences, and other educators.7. Teachers should be skilled in recognising unethical, illegal, and otherwiseinappropriate assessment methods and uses of assessment information.There needs to be a paradigm shift so that teachers view assessment as an integralpart of their teaching and not as an additional burden on their teaching responsibilities.Professional development will play a pivotal role here. On-going professionaldevelopment should not be considered optional but essential for teachers’ professionalgrowth. Today’s students are immersed in an ever expanding digital and multi-modallearning environment. Accordingly, teachers need to continually adjust and renew theirassessment practice to accurately assess new ways of learning. The OECD review teamcommends the practice of some municipalities in providing in-service specialist trainingfor special teams of teachers so that they can lead and support other teachers’assessment activities. If offering external in-service teacher training is costly and time-consuming, assessment specialists should play a more active role on teacher trainingwithin schools. Teachers’ professional growth in assessment can begin with reflectivequestions such as:••••••Should this assessment be part of normal classroom activities?Does it clearly match the targeted learning goal?What level of knowledge and skills are being assessed? Does it assess basicknowledge or creative and deep thinking?Are there clear criteria against which performance is judged?Does it provide equal opportunities for students to demonstrate what they can do?What feedback will be most useful for students and how should it be provided,orally or in writing?

Differentiated instruction is key to academic success for all students. Principledassessment that evaluates students’ progress toward high yet attainable targets is pivotalfor differentiated instruction. High-quality assessment can provide diagnostic data fordeveloping differentiated instructional plans, providing accommodations for studentsrequiring additional support, and evaluating the effectiveness of teaching (Jordan,Lindsay and Stanovich, 1997; Pellegrino, Chudowski and Glaser, 2001).In New Zealand, the Ministry of Education has implemented a specific strategy topromote the ongoing professional development of teachers and specifically to build theirassessment capacity. One part of this two-fold strategy (Gilmore, 2008) involves nationalprofessional development programmes and evaluations of the major programmes offeredhave shown positive impact on student learning (see Box 4.3 for examples). The secondpart of the strategy involves continuous professional development at school. This can bethrough teachers’ involvement in school activities where assessment plays an importantrole, and increasingly via student feedback as they develop their own assessment capacity(Timperleyet al.,2007; Absolum,et al.,2009). The third major way to build teachers’assessment capacity is by engaging them in the moderation of the national samplemonitoring tests.

Box 4.3 Professional development related to student assessmentAssess to Learn (AtoL)is a whole-school professional development programme that hasbeen offered to primary and secondary schools since 2002. Schools can apply for participation inthe programme and typically participate for two years. The annual budget for AtoL is $3.17million and currently involves 155 schools. The programme intends to support teachers inchoosing adequate assessment tools and analysing assessment information so as to furtheradvance student learning. A 2008 evaluation of the AtoL programme reported a significant impactof the programme on teacher professional practice and important improvements in studentlearning, especially for students with initially low achievement level. Monitoring data showed thatschools participating in AtoL had achieved up to 4.5 times greater improvements in writingachievements in Years 4 to 9 than the nationally expected rate of progress.The Literacy Professional Development Project (LPDP)was a six-year programmeinvolving 323 schools and over 3 000 teachers. The programme provided whole-staff school-based literacy professional development running for over two years. It focused on improvingstudent literacy achievement through an evidence-based inquiry model focused on qualityteaching and development of professional learning communities. This was not a programmeparticularly focused on assessment, but collecting and interpreting data was a key component ofit. The evaluation showed that schools participating in LPDP had significantly improved studentprogress and achievement in reading and writing, and especially so for the students most at risk ofunderachieving.Source:McDowallet al.(2007); Poskitt and Taylor (2008); Gilmore (2008); New Zealand Ministry ofEducation (forthcoming).

Ensure students are actively engaged and proficient in assessmentStudents are at the heart of student assessment and should be actively involved inassessment. Today’s classrooms are experimenting with collaborative real-life tasks tobetter prepare students for the future. As a result, students are observed working oncomplex problems in groups. To engage students in self- and peer-assessment processesin such co-operative learning and assessment contexts, students should first learn todevelop both leadership and responsibility for learning. Hargreaves and Fullan (1998)suggest that classrooms and schools can turn into democratic practices through activeinvolvement of students and parents in assessment and decision-making processes. Asnoted above, assessment schemes, including the specification of what will be assessedand how it will be appraised, must be made transparent to students (Ross, Rolheiser andHogaboam-Gray, 1999).In Sweden, students are involved in goal setting and self-assessment which forms acore part of their formative assessment – in addition to regular development talks betweenteachers, students and parents and the use of individual development plans (IDP).Teachers in Sweden use the individual development plans to engage students in settinggoals for learning and encourage students to develop skills for self- and peer-assessment.Teachers are generally more likely to focus on formative assessment when they havetools and guidelines to support the process (OECD, 2005a). Similar to in Sweden,teachers in Denmark could use the ISPs to focus both teachers and students on identifyingindividual learning goals and developing strategies to address any shortcomings,i.e.focusing on the future development of the students’ learning. Used effectively, ISPscould be a powerful tool for developing students’ own assessment skills.In Finland, ‘learning-to-learn’ skills are considered to be central to each student’sdevelopment (Finnish Department for Education and Science Policy, forthcoming). Theseare actively promoted as core elements in achieving lifelong learning and include astudent’s capacity for independent and self-motivated learning, problem-solving and theability to evaluate his/her own learning and related strategies. There is a clear pedagogicalgoal in all compulsory education subjects for students to develop ‘learning-to-learn’skills. To evaluate and promote the importance of such skills, national sample assessmentswere developed by the Centre of Educational Assessment at the University of Helsinki toevaluate ‘learning-to-learn’ skills in Years 3, 6 and 9 of compulsory education.

Maximise the pedagogical value of the national testsThe political discourse on publication of national test results seems to haveovershadowed the significant pedagogical value that the national tests can bring. It iscritically important to engage teachers in working effectively with the national test resultsas one means to diagnose student learning needs and to adjust their teaching strategiesaccordingly. The national tests offer teachers access to a sophisticated analytical tool toplan instruction both for individual students, for the class and for particular groups ofstudents following a particular programme. Teachers as professionals are responsible forensuring the clear communication of the nature of the national tests. Some students andteachers expressed confusion during the OECD review regarding the adaptive feature ofthe national tests, for example, some of the more proficient students felt that theyperformed poorly on the test because they found the test difficult. Such concerns arelinked to the implementation issues and should no longer be an obstacle as teachers andstudents become familiar with the format of the national tests. While the former SchoolOECD REVIEWS OF EVALUATION AND ASSESSMENT IN EDUCATION: DENMARK � OECD 2011

Agency has provided clear information on the national test features – specifically itsadaptive function – and has offered conferences and other outreach activities on thenature of the tests, the Quality and Supervision Agency must continue to be proactive andwork together with the Teacher Union, KL and Municipalities to ensure that teachersunderstand and communicate to students that in adaptive testing, students’ perceived itemdifficulty does not necessarily reflect poor test performance. Students’ test-takingexperience should not affect their self-esteem in any way. Municipalities need to ensurethat teachers explain both the purpose of the national tests and how the tests work clearlyto students and parents. Indeed, evidence from the OECD’s trial of a computer-basedscience test in Denmark, Korea and Iceland, revealed high motivation for the computermedium in Denmark: 87% of girls and 85% of boys agreed or strongly agreed that thecomputer-based test was enjoyable; in contrast 44% of girls and 36% of boys agreed orstrongly agreed that the paper-and-pencil test was enjoyable (OECD, 2010f).The national tests offer teachers the possibility to re-administer the tests (outside of themandatory testing period) up to twice more with their students. This offers a powerfulopportunity to teachers to track student learning progress on the discrete areas assessed.Effective teachers adopt a systematic and cyclical approach to using assessment data(Hamiltonet al.,2009). High-quality external tests (e.g., national and standardisedassessments) have the potential to provide useful data for teachers to guide and improveteaching and learning (Chudowsky and Pellegrino, 2003). Often teachers find it difficult tointerpret and use student data on external tests because the data do not always provide timelydiagnostic information and lack strong curricular links (Militello, Schweid and Sireci, 2010).In the case of the Danish national tests, teachers are provided with rapid diagnostic feedbackand these are in selected profile areas within the national Common Objectives.Without the active engagement of the Teacher Union and teachers in using resultsfrom the national tests, Denmark can face the dilemma found in other systems that thelearning environment can be ‘both data rich and information poor’ (Wayman andStringfield, 2006). Educators need to be actively involved in developing data-drivenprofessional learning communities where assessment data are used in non-threateningways and teachers develop assessment competencies.

Further validate and develop the national testsAny new and existing tests require comprehensive validity evidence. The OECDreview team commends the former School Agency’s efforts for the development andinitial validation of the national tests through large-scale field trials (Wandall, 2010). Thenational tests include an item bank of 7 200 items – 10% of which will be renewed eachyear. Each test item was trialled on 500-700 students for functionality, the test items werecarefully developed and reviewed by content experts to ensure construct validity andpsychometric scales were successfully established through 1PL Rasch modelling(Wandall, 2010).Validation is a long-term process of accumulating, interpreting, refining, andcommunicating multiple sources of evidence about appropriate interpretations and use oftest information. The OECD review team strongly encourages the Ministry of Educationand the Quality and Supervision Agency to continue to refine and validate the testingsystem with short- and long-term strategic plans. We are confident that the national testswill gain credibility over time among teachers. With sufficient validity evidence from on-going research, teachers and administrators will grow comfortable with the system, andthe system will fulfil the intended purposes of both accountability and pedagogy.OECD REVIEWS OF EVALUATION AND ASSESSMENT IN EDUCATION: DENMARK � OECD 2011

68– 4. STUDENT ASSESSMENTIn the process of validation, attention should be paid to the following issues:•The breadth of curriculum should be maintained in student assessment byensuring that all objectives and subject areas are given certain forms of attention.Other diagnostic tools and activities should complement the national tests tomeasure students’ learning progress and outcomes in all subjects and objectives.When the Danish national tests serve the high-stakes accountability purpose bypublishing the results to the public, instructional time is more likely to be devotedto what will be measured. To prevent teachers from teaching to the tests andthereby narrowing the curriculum, multiple measures of student achievementshould be used to determine the quality of school performance and studentoutcomes.The degree to which the national tests authentically reflect what is taught andlearned in classrooms may have significant impact on the pedagogical usefulnessof the test results. Considering a lack of curricular standards in theFolkeskole,thenational tests should be monitored and validated through on-going research onissues related to the alignment between Common Objectives prioritised by thetests, local curriculum, and the test content. The assertion by the Association ofDanish Students that the tests do not reflect ‘the complicity or concrete themes ofthe teaching in the class’ (Rambøll, 2011) is worrying. Although the tests are onlyin discrete areas of the Common Objectives, it would be important to establishwhether such reported phenomena are due to sequencing of class instruction,inadequate implementation of the Common Objectives by schools or inadequatealignment of the tests with the Common Objectives.The item bank of the national tests needs to be expanded by includingperformance tasks in addition to the multiple-choice test format. We areimpressed that the national test system has developed a large bank of items whichwere psychometrically calibrated and validated through field testing and expertpanel review. While the multiple-choice test items allow for timely and efficientscoring by the central system, relying solely on multiple-choice test items riskslosing the potential of deep learning that involves critical reasoning and problemsolving skills beyond factual knowledge, which is crucial for certain core subjectareas, such as science and mathematics, in preparing Danish students to becompetent citizens nationally and internationally. As noted earlier, performancetasks are more common than multiple-choice tests in student assessment inDenmark. Therefore, including performance test items in the national tests mayenhance the authenticity of the national test scores by better reflecting ways inwhich Danish students learn in their classrooms. Engaging teachers in the scoringand moderation of performance tasks is also an effective way to build assessmentcapacity throughout the system (see below). Alternatively, some performance testitems could be developed and added to the item bank in the computer adaptivetests. From a psychometric point of view, although computer adaptive teststypically use multiple-choice test items that are scored as either correct orincorrect (i.e., binary), they increasingly include performance test items, such asshort answer test items, oral speeches, short and long essays (a.k.a. constructedresponse items) that are scored in multiple categories (i.e., polytomous).Currently, much research is underway to understand the validity and reliability ofautomated computer scoring of performance tasks. Therefore, it is possible(technologically) and valuable (pedagogically) to consider constructed response

Review the purpose, procedures and content of the final examinations inForms 9 and 10The OECD review team recommends a review of the purpose, procedures and contentof the current suite of final examinations in Forms 9 and 10. The ‘Flying Squad’recommends that the purpose of the final examinations should change to make themcount for students’ access to upper secondary education (see Box 2.1). If the finalexaminations are to carry higher stakes for students, then:•It would be imperative to ensure the comparability of assessment practices amongteachers, schools and municipalities and current moderation procedures for oralexaminations should be reviewed and if necessary strengthened;Consideration should be given to changing current procedures whereby studentsonly sit examinations in a few core subjects and are randomly selected forexaminations in the humanities and sciences blocks, to ensure that students areable to sit examinations in all subjects that are relevant to their futureeducational/career pathways. This would mean developing final examinations inother subjects,e.g.in technology, to reflect the full suite of subjects offered in thefinal Form levels of theFolkeskole.

Teachers are given considerable scope to exercise their professionalism and benefit fromgood levels of trust among students, parents, and the community. Schools increasinglystructure their work around teaching teams and engage special support advisors.However, teacher appraisal is not systematic and there is no shared understanding ofwhat counts as accomplished teaching in Denmark. It is a top priority to develop aframework of teaching standards to provide the common basis to organise a careerstructure for teachers. Teacher appraisal for certification would determine both teachers’career advancement and professional development plans. Danish teachers are generallykeen to receive feedback for their professional development, but while some schoolprincipals hold a formal dialogue with teachers on an annual basis, it is not wide-spreadpractice for school principals to observe teaching. Developmental teacher appraisalshould be strengthened and linked with teacher professional development and schoolimprovement.

5.1 Context and featuresTeacher appraisal proceduresTeacher appraisal is not regulated by law and no national requirements exist toevaluate the performance of teachers. Actual teacher appraisal practices are determinedlocally (i.e. at the school level) with the possible influence of municipalrequirements/guidelines. According to theFolkeskoleAct, the school principal isresponsible for the quality of teaching at the school as well as the overall administrativeand pedagogical management of the school, including the professional development ofteachers. As a result, the main responsibility for designing, introducing and organisingteacher appraisal procedures within the school lies with the school principal. Actualteacher appraisal practices in Danish schools are poorly documented but they seemed tobe based on a culture where school leaders show confidence in their teachers, appraisal istaken as a school-teacher or teacher-teacher dialogue, and procedures are defined incollaboration with the teachers.Many municipalities require that all school employees engage in a typically annualprofessional dialogue with their leader/manager. This employee dialogue often results ina professional development plan for the employee and is usually organised in a formalreport/follow-up model. The basis for the discussion and the set of criteria used aredefined at the school level possibly following a framework defined at the municipal leveland so can vary considerably. At the present time, the implementation of employeedialogues differs considerably across those municipalities and schools which use them,depending on local capacities and the evaluation ethos of schools. It is not guaranteed thatevery school principal assesses each teacher annually.Occasionally, teacher appraisal external to the school can also be conducted as whenmunicipal consultants, the Danish Evaluation Institute or a third party carry out teacherappraisals to ensure the quality of the schools and their work.There is little guidance provided at the central level on how to evaluate teacherperformance. No performance criteria and reference standards exist at the national level.The idea is that each school defines its own evaluation criteria linked to local objectives,possibly following the municipal requirements or guidelines. Nonetheless, theEvaluationportalprovides tools for teacher appraisal (see below). This reflects a national approachbased on encouraging voluntary teacher appraisal through the provision of informationand tools for its application in a school context rather than making it a requirement in auniform structure.

Teacher accountability through market mechanismsIt is important to stress that there are features in the Danish education system that leadto strong competitive forces on schools and teachers to perform well in order to: justifymunicipal spending on schools; attract students; and attract/retain effective teachers.These relate to school funding and reflect the decentralised nature of the Danisheducation system and the extent of school choice (e.g. with private independent schoolsreceiving a public subsidy per student equivalent to 75% of that provided to publicschools). First, “funding follows the student” as when a student moves school, theoperating grant that applies to that student is reallocated to their new school (regardless ofit being a municipal or a private independent school). Second, the municipalities are ableto choose the amount of funding that they allocate to schools provided that they complywith their legislative obligations and meet the national objectives. Third, most funds areallocated to schools in a block grant, and school principals are able to determine thedivision of funds between different categories of expenditure, including different types ofteachers and non-teaching staff.

Box 5.1 The teaching profession in Denmark – Main featuresEmployment statusTeachers working in the public sector are salaried employees of municipalities. Conditions ofservice are governed by two-year agreements between Local Government Denmark, the Ministryof Finance and the Confederation of Teacher Unions. These stipulate basic salaries and generalworking conditions. More specific salary supplements and working conditions are determined atthe local level. Teachers working in the private independent sector are salaried employees ofindependent schools’ organisers.Most teachers are employed on indefinite term contracts, which means that they can only bedismissed on grounds covered by legislation such as redundancy, disciplinary reasons orunderperformance. According to TALIS,1796.6% of Danish teachers of lower secondaryeducation are permanently employed (2ndhighest figure among TALIS countries, against anaverage of 84.5%). No probationary period for newly qualified teachers exists in theFolkeskole.Where there are not enough qualified applicants local authorities can employ other persons on afixed-term contract.

Salary and career structureIn Denmark career progression and salary are almost entirely dependent on length of serviceand qualifications. There is a single salary scale (basic salary system), incremental on the basis oftenure, and the top is reached after 8 years. Teachers may earn more within their pay scale inspecific instances. There is a centrally-agreed function supplement for teaching over and above300 annual hours (there is a function wage for work and responsibility areas linked to theindividual position above those covered by the basic salary). Teachers may also receive a‘qualification’ wage associated with objective conditions such as education and experience, whichis agreed at local level. Plus, teachers may be eligible to receive an area supplement depending onthe location of theFolkeskolewhere they teach. Finally, teachers can access a ‘seniority salarysystem’ once they have reached the top of the ‘basic salary system’. Opportunities for promotionare limited to access to school management roles within schools.

Professional developmentProfessional development for Danish teachers is not regulated by law and there is nominimum requirement. Participation in professional development activities has no direct effect onpay levels or the career of the teacher (e.g. promotion is not conditional upon having taken part inprofessional development activities). However, professional development takes place at theinitiative of teachers and schools. According to TALIS, in 2007-08, 75.6% of Danish teachers oflower secondary education undertook some professional development in the previous 18 months(against an average of 88.5 among TALIS countries). Professional development for teachers intheFolkeskoleis primarily organised by the Danish University of Education, university collegesand municipalities. Specialised State training institutions, teachers’ associations and the Ministryof Education also offer in-service training activities. Regional committees for teacher in-servicetraining have been established to align municipal and school training needs with the supply ofprogrammes by professional development providers.

5.2 Strengths and challengesStrengthsTeachers are trusted professionals with a high degree of autonomyThe OECD review team formed the view that Danish teachers are generally perceivedas trusted professionals among the different stakeholders which also results from theirextensive autonomy in the exercise of their duties. Teachers decide on the teachingcontent within the framework provided by the Common Objectives and the possiblerefinement of these developed by the municipality in which they teach (although in realitythe majority adopt the Common Objectives). They choose teaching materials andmethods of instruction to achieve learning objectives. They also have fairly good levels ofautonomy in student assessment, including in the oral and project components of school-leaving examinations. Teachers have room to develop practices that fit local needs. Mostimportantly, they function as learning facilitators for their students as these are takingmore responsibility for their learning, student learning becomes more individualised andcommunication with students’ parents is strengthened. Overall, teachers are givenconsiderable scope to exercise their professionalism and benefit from good levels of trustamong students, parents, and the communities in general.

Teamwork provides opportunities for peer learningWork in Danish schools is increasingly organised in a way that grants opportunitiesfor teamwork. Schools more and more are structuring work around teams of teachers(e.g. Class team, Form team, Section team, Subject team) which share responsibility fororganising their work. This recent development has led to growing co-operation amongteachers and a more formal dialogue between the school leaders and teams of teachers.This also provides a context in which some schools organise teacher appraisal mostlywithin teams. In this situation teachers are to co-operate on promoting the quality of theteaching in the school. It is a widespread practice in theFolkeskolethat planning, learningand knowledge sharing takes place in teacher teams in school. Other typical activitiesamong teachers include supervising each other within a team and discussing the progressand development of a single student together. Denmark is among the countries in which,in lower secondary education, the feedback from other teachers or members of the schoolmanagement team can be more frequent. According to TALIS, and for the followingfrequency, the proportion of teachers who reported having received such type of feedbackon their work is: 27.4% three or more times per year (3rdhighest figure, against a TALISaverage of 19.3%); 12.5% monthly (8thhighest figure, TALIS average of 10.4%); and11.7% more than once a month (5thhighest figure, TALIS average of 9.1%).

Evaluation advisors have the potential to foster professional feedback in schoolsAn interesting development in the effort to strengthen the evaluation culture inDanish schools is the introduction of the role of evaluation advisor among the so-called‘resource persons’ that Danish schools can hire. Resource persons are teachers whoundertake specific training and acquire expertise in a given domain who then perform thefunction of expert in that domain as part of their duties as a teacher (e.g. IT tutors, readingtutors, librarians). Evaluation advisors are still a limited resource in Danish schools: in astudy from the Danish Evaluation Institute, only 8% of schools examined in the study hadan evaluation advisor (EVA, 2009a). There is little information about their specific rolesand tasks. These broadly consist of supporting the school effort in developing evaluationpractices and an evaluation culture. It might involve the guidance and coaching ofcolleagues and school management on self-appraisal of teaching practices, peer feedback(including classroom observation), new knowledge and/or initiatives in the educationalfield, implementation of educational policies, co-ordination of quality assurance withinthe school or simple individual advice to teachers. The study by the Danish EvaluationInstitute concludes that it is easier for resource persons to be more effective in schoolswhich have a tradition of peer feedback, team work and a culture of open classroom doors(EVA, 2009a).

Teacher appraisal is not systematic across the system and is not perceived asmeaningfulTeacher appraisal is not a consolidated practice in Danish schools. There is noexpectation that each teacher in theFolkeskolehas his or her practice appraised andreceives feedback for improvement. The existing teacher appraisal practices are theinitiative of individual schools (in some cases in the context of municipality’srequirements) and depend essentially on the endeavour of the school principal and theevaluation ethos created in the school. As such, there is great variation between schools inthe way teacher appraisal and feedback is conceptualised and carried out, from a verylight-touch approach to more elaborate processes in some schools. The OECD reviewteam saw examples of schools with established practices of formative teacher appraisal,including classroom observation and peer feedback, but also examples of schools whereteachers had few opportunities to receive professional feedback. Where it exists, theemployee dialogue is often limited in its ability to provide feedback for teacher’sdevelopment. Therefore there are no guarantees in Danish schools that approaches toteacher appraisal and feedback are addressing the real issues and complexities of teachingand learning. There is no mechanism to ensure minimum standards for teacher appraisalprocesses in schools and so there is no guarantee each teacher receives properprofessional feedback. This also means that in those schools where teacher appraisalprocesses are weak, it might be difficult to identify and address underperformance.Some form of appraisal of and feedback to teachers seems to take place. According toTALIS, only 14.2% of teachers of lower secondary education reported never receivingappraisal and/or feedback from the principal about their work (9thlowest figure, against aTALIS average of 22.0%). About 68% of teachers of lower secondary education reportedreceiving such appraisal and/or feedback at least once a year.However, there seems to be the perception that appraisal and feedback has littleimpact. According to TALIS, Danish teachers of lower secondary education are the mostnegative in their perceptions of the impact of teacher appraisal and/or feedback. In fact,the proportion of lower secondary teachers who reported that the appraisal and/orfeedback they received led to or involved moderate or large changes is the lowest amongTALIS countries across a range of practices: 18.2% for classroom management practices(against a TALIS average of 37.6%); 10.9% for knowledge or understanding of theteacher’s main subject field (against a TALIS average of 33.9%); and 11.1% forknowledge or understanding of instructional practices (against a TALIS average of37.5%). Also, 60.8% of teachers of lower secondary education agree or strongly agreethat the review of teachers’ work has little impact upon the way teachers teach in theclassroom (4thhighest figure, against a TALIS average of 49.8%).In general, there seems to be an issue about how teacher appraisal by the schoolprincipal is perceived by teachers. A study by the Danish Evaluation Institute indicatesthat often the involvement of school management in teacher appraisal is perceived as anattempt of control rather than a tool for quality development (EVA, 2007).

Teachers have few opportunities for formal recognitionTeacher appraisal at the school level is not perceived as an instrument to rewardteachers, which is not surprising as teacher appraisal procedures essentially have aformative purpose in Denmark. For instance, according to TALIS, only 15.0% of teachersof lower secondary education agree or strongly agree that in the school the most effectiveteachers receive the greatest monetary or non-monetary rewards (13thlowest figure,against a TALIS average of 26.2%). Similarly, only 8.3% of teachers of lower secondaryeducation agree or strongly agree that in the school if they improve the quality of theirteaching they receive increased monetary or non-monetary rewards (5thlowest figure,against a TALIS average of 25.8%). The principle of associating good performance tocareer progression is not in place in Denmark.The same seems to occur in relation to more informal means of recognition.According to TALIS, 25.3% of teachers of lower secondary education reported that theappraisal and/or feedback they received led to a moderate or large change in the publicrecognition from the principal and/or their colleagues (7thlowest figure, against a TALISaverage of 36.4%).

Teacher appraisal could be more effective in addressing underperformanceThere are some indications that teacher appraisal is not effectively fulfilling itsfunction of addressing underperformance. On the one hand, teachers’ identifiedweaknesses seem to be relatively well addressed through support measures provided toteachers. The following proportion of Danish lower secondary teachers are in schoolswhere the principal reported that the following measures are always taken to addressweaknesses in their teaching as identified by teacher appraisal:(i)The principal ensuresthat measures to remedy the weakness in their teaching are discussed with the teacher:61.0% (13thhighest figure against a TALIS average of 58.9%);(ii)The principal, orothers in the school, establishes a development or training plan for the teacher to addressthe weakness in their teaching: 20.8% (10thhighest figure against a TALIS average of20.6%); and(iii)The principal ensures that the teacher has more frequent appraisals oftheir work: 17.1% (9thhighest figure against a TALIS average of 15.2%).On the other hand, there seems to be the perception that sustained underperformanceis not as well addressed. According to TALIS, 40.7% of teachers of lower secondaryeducation agree or strongly agree that in the school the sustained poor performance of ateacher would be tolerated by the rest of the staff (8thhighest figure, against a TALISaverage of 33.8%). In addition, only 6.6% of teachers of lower secondary education agreeor strongly agree that in the school the school principal takes steps to alter the monetaryreward of the persistently underperforming teacher (3rdlowest figure, against a TALISaverage of 23.1%). Similarly, 35.0% of teachers of lower secondary education agree orstrongly agree that in the school teachers will be dismissed because of sustained poorperformance (6thhighest figure, against a TALIS average of 27.9%).

The extent of externality in teacher appraisal is limitedTeacher appraisal, when it is organised, is school-based and rarely involves agentsexternal to the school. According to TALIS, 69.7% of teachers of lower secondaryeducation reported having received no appraisal and/or feedback from an externalindividual or body (e.g. external reviewer) about their work in the school (5thhighestfigure, against a TALIS average of 50.7%). The municipal quality reports rarely engagein a closer look at the quality of the teaching and involve little interaction with individualteachers. Practices are somewhat different in the private independent sector: externalteacher appraisal is slightly more common given that parent’s exert responsibility tosupervise schools and are required to recruit individuals with expertise to oversee thequality of the teaching in school.The limited extent of externality in teacher appraisal raises a number of challenges.Teachers are appraised according to local judgements and appraisal criteria. Teachers arealso entirely dependent on local capacity and willingness to benefit from opportunities toimprove their practice, see their professional development recognised and gain greaterresponsibility as they evolve in the profession. The involvement of some externality inteacher appraisal can provide an element of distance and rigour which can be particularlyvaluable in validating school-based approaches to teacher appraisal.

Teachers have few opportunities for feedback and could benefit from morepedagogical leadershipDanish teachers have relatively few opportunities for professional feedback. The mainopportunity to receive feedback on their practices is the dialogue with the schoolprincipal, sometimes in the formal setting provided by the employee dialogue. However,school principals are overwhelmed with tasks at the school and, in general they do notseem to have the time to engage properly in the coaching, monitoring, and appraisal ofteachers. For example, classroom observations by school principals seem to be relativelyoccasional. In a study from the Danish Evaluation Institute, 82% of the teachers in the 20participating schools indicated that their school principal had not observed their teachingin the previous year (EVA, 2007).According to TALIS, the following proportion of Danish teachers of lower secondaryeducation reported that the following were considered with high or moderate importanceas a criterion in the appraisal and/or feedback they received:(i)Direct appraisal ofclassroom teaching: 40.7% (lowest figure against a TALIS average of 73.5%);(ii)Classroom management: 61.6% (lowest figure against a TALIS average of 79.7%);(iii)Innovative teaching practices: 35.7% (lowest figure against a TALIS average of70.7%);(iv)Student feedback on the teaching they receive: 60.7% (7thlowest figureagainst a TALIS average of 72.8%); and(v)Feedback from parents: 56.4% (4thlowestfigure against a TALIS average of 69.1%). According to the PISA survey, 32.0% of15-year-old students are in schools where the principal reported that teacher peer review(of lesson plans, assessment instruments, lessons) has been used the previous year tomonitor the practices of teachers at their school (6thlowest figure against an OECDaverage of 56.3%). Overall, there is scope for improvement in areas such as classroomobservation, peer discussion, coaching, or self-critical analysis. Also, instructionalleadership in schools is not a system-wide expectation (see Chapter 6).

Missing links between teacher appraisal, professional development and schooldevelopmentThe OECD review team formed the view that the provision of professionaldevelopment appears not thoroughly planned, fragmented and not systematically linked toteacher appraisal. According to TALIS, only 12.4% of teachers of lower secondaryeducation reported that the appraisal and/or feedback they received directly led to orinvolved moderate or large changes in a teacher development or training plan to improvetheir teaching (lowest figure, against a TALIS average of 37.4%). Also, only 25.6% ofteachers of lower secondary education reported that the appraisal and/or feedback theyreceived led to a moderate or large change in opportunities for professional developmentactivities (10thhighest figure, against a TALIS average of 23.7%). In most cases, theidentification of professional development needs is not a requirement of establishedteacher appraisal practices. In Denmark, there is no consistent means to base professionaldevelopment needs on a thorough assessment of teaching practice. Without a clear link toprofessional development opportunities, the appraisal process is not sufficient to improveteacher performance, and as a result, often becomes a meaningless exercise thatencounters mistrust – or at best apathy – on the part of teachers being evaluated(Danielson, 2001; Milanowski and Kimball, 2003; Margoet al.,2008). The shortcomingsin the provision of professional development in theFolkeskolewere also noted by thereport of the ‘Flying Squad’ which reviewed practices in theFolkeskole(Danish SchoolAgency, 2010; see Box 2.1).There is also scope to better link professional development to school development. Inthe OECD review team’s view, school development could better explore its links to theevaluation of teaching practice. This is in part due to the limited time school principalshave for pedagogical leadership and the limited extent to which professional developmentactivities are linked to the results of teacher appraisal. But it also stems from the fact thatprofessional development activities are mostly an individual teacher’s choice and areoften not associated with school development needs.

5.3 Pointers for future policy developmentThe development of meaningful teacher appraisal in Denmark is a vital step in thedrive to improve the effectiveness of teaching and learning and to raise educationalperformance. The effective monitoring and appraisal of teaching is central to thecontinuous improvement of the effectiveness of teaching in a school. It is essential toknow the strengths of teachers and those aspects of their practice which could be furtherdeveloped. In order to make teacher appraisal more effective in the DanishFolkeskole,the OECD review team proposes the following approach:••••••••Develop teaching standards to guide teacher professional development andappraisal;Create a career structure with key stages;Introduce a system of teacher certification to determine career progression;Introduce a mandatory probationary period for new teachers;Strengthen developmental teacher appraisal and hold school principalsaccountable for this;Ensure links between developmental appraisal and appraisal for certification;Reinforce linkages between teacher appraisal, professional development andschool development;Ensure appropriate articulation between school evaluation and teacher appraisal.

In recognition of the variety of tasks and responsibilities in today’s schools and theteaching expertise developed while on the job, teaching standards should expressdifferent levels of performance such as competent teacher, established teacher, andaccomplished/expert teacher. These should reflect teachers’ tasks in schools and theknowledge and skills that they need to acquire to be effective at the different stages oftheir careers to achieve student learning objectives. They need to reflect the sophisticationand complexity of what effective teachers are expected to know and be able to do; beinformed by research; and benefit from the ownership and responsibility of the teachingprofession. It also needs to be ensured that the teaching standards provide the commonbasis to organise the key elements of the teaching profession such as initial teachereducation, teacher certification (see below), teachers’ professional development, careeradvancement and, of course, teacher appraisal.

Create a career structure with key stagesThe OECD review team has noted that the absence of career opportunities foreffective teachers undermines the role of teacher appraisal. Schools and teachers couldbenefit from a career structure for teachers that comprised (say) three key stages:competent teacher; established teacher, and accomplished/expert teacher. The differentstages in the career should be associated with distinct roles and responsibilities in schoolsassociated with given levels of teaching expertise. Access to each of the key stages couldbe associated with formal processes of appraisal through a system of teacher certification(see below).The career structure for teachers should match the different levels of expertisereflected in teaching standards. Such alignment would reflect the principle of rewardingteachers for accomplishing higher levels of expertise through career advancement andwould strengthen the linkages between roles and responsibilities in schools (as reflectedin career structures) and the levels of expertise needed to perform them (as reflected inteaching standards). A career structure for teachers reflecting different levels of expertiseis likely to enhance the links between teacher appraisal, professional development andcareer development.

Introduce a system of teacher certification to determine career progressionThe teaching profession in Denmark would benefit from teacher appraisal at keystages in the teaching career to formalise the principle of advancement on meritassociated with career opportunities for effective teachers. Such appraisals, which aremore summative in nature, need to have a stronger component external to the school andmore formal processes. They could be organised through a system of teacher certificationwith (say) access to three key stages: competent teacher (following a probationary period– see below), established teacher; and accomplished/expert teacher. It could be a mostlyschool-based process led by the school principal (or another member of the managementgroup) but it should include an element of externality such as an accredited externalevaluator, typically a teacher from another school with expertise in the same area as theteacher being appraised. Examples of consolidated teacher certification/registrationmodels are those of several states in Australia (Australian Government Department ofEducation, Employment and Workplace Relations, 2011).

Reference criteriaThe appraisal system associated with the certification process should be founded onthe national framework of teaching standards. A reference contribution in this area is theDanielson’sFramework for Teaching(1996, 2007), which is articulated to provide at thesame time “a ‘road map’ to guide novice teachers through their initial classroomexperiences, a structure to help experienced professionals become more effective, and ameans to focus improvement efforts”. TheFrameworkgroups teachers’ responsibilitiesinto four major areas further divided into components:•Planning and Preparation:demonstrating knowledge of content and pedagogy;demonstrating knowledge of students; selecting instructional goals; designingcoherent instruction; assessing student learning.The Classroom Environment:creating an environment of respect and rapport;establishing a culture for learning; managing classroom procedures; managingstudent behaviour and organising physical space.Instruction:communicating clearly and accurately; using questioning anddiscussion techniques; engaging students in learning; providing feedback tostudents; demonstrating flexibility and responsiveness.Professional Responsibilities:reflecting on teaching; maintaining accuraterecords; communicating with families; contributing to the school and community;growing and developing professionally; showing professionalism.

This framework has influenced a large number of teacher appraisal systems aroundthe world. An example can be found in theProfessional Standards for TeachersinEngland (TDA, 2007). These standards cover all aspects grouped into ‘professionalattributes’ – including relationships with children and young people, ‘professionalknowledge and judgement’ and ‘professional skills’. Moreover, the standards differentiatein several stages from what can be expected of the newly qualified teacher to the standardexpected of excellent and advanced skills teachers (see Santiagoet al.,2009, for furtherdetails).It is important that teacher appraisal for certification takes account of the schoolcontext, and includes the views of the school principal. Schools have to respond todifferent needs depending on the local context and face different circumstances,especially in a system as decentralised as Denmark. Hence it is desirable that individualteachers are evaluated against reference standards with criteria that account for theirschool’s objectives and context.

InstrumentsTeacher appraisal for certification could rely on three core instruments: classroomobservation, self-appraisal and documentation of practices in a simplified portfolio. Itshould be firmly rooted in classroom observation. Most key aspects of teaching aredisplayed while teachers interact with their students in the classroom. It should alsoinvolve an opportunity for teachers to express their own views about their performance,and reflect on the personal, organisational and institutional factors that had an impact ontheir teaching. In this respect, the tools available in theEvaluation portalfor teacher self-appraisal are particularly instrumental. The portfolio should allow teachers to mentionspecific ways in which they consider that their professional practices are promotingstudent learning, and could include elements such as: lesson plans and teaching materials,samples of student work and commentaries on student assessment examples, teacher’sself-reported questionnaires and reflection sheets (see Isoré, 2009). Given the high-stakesof appraisal for certification, decisions must draw on several types of evidence, rely onmultiple independent evaluators and should encompass the full scope of the teacher’swork.

TrainingExternal evaluators would receive specific training for this function, in particular instandards-based methods for assessing evidence of teacher performance, and would needto be accredited by the proper organisation. Evaluators need be trained to assess teachersaccording to the limited evidence they gather, the criteria of good teaching and thecorresponding levels to attain certification. Second, evaluators should be trained to alsoprovide constructive feedback to the teacher for further practice improvement.18Also,substantial activities for professional development on how to best use appraisal processesshould be offered to teachers. It is vitally important that teachers are provided withsupport to understand the appraisal procedures and to benefit from appraisal results. It isalso expected that appraisal and feedback become core aspects offered in teacher initialteacher education. Finally, if teacher certification is essentially school-based, it wouldalso be desirable to establish moderation processes to ensure consistency of schoolapproaches to appraisal for teacher certification.

Strengthen developmental teacher appraisal and hold school principalsaccountable for thisThe OECD review team is of the view that there needs to be a stronger emphasis onteacher appraisal for improvement purposes (i.e. developmental appraisal). Given thatthere are risks that the improvement function is hampered by the high-stakes teacherappraisal associated to the certification process, we propose that a componentpredominantly dedicated to developmental appraisal, fully internal to the school, becreated. This suggestion is in line with the recommendation by the Chairmanship of theSchool Council for school management (and other teachers) to engage in the regularprofessional appraisal of individual teachers in the school (Skolerådet, 2010).This development appraisal would have as its main purpose the continuousimprovement of teaching practices in the school. It would be an internal process carriedout by line managers, senior peers, and the school principal (or members of themanagement group). The reference standards would be the teaching standards but withschool-based indicators and criteria. This appraisal should also take account of the schoolobjectives and context. The main outcome would be feedback on teaching performance aswell as on the overall contribution to the school which would lead to a plan forprofessional development. It can be low-key and low-cost, and include self-appraisal,peer appraisal, classroom observation, and structured conversations and regular feedbackby the school principal and experienced peers. It could be organised once a year for eachteacher, or less frequently depending on the previous assessment by the teacher. The keyaspect is that it should result in a meaningful report with recommendations forprofessional development. To be effective, appraisal for improvement requires a culturein which there is developmental classroom observation, professional feedback, peerdiscussion and coaching opportunities.There are advantages to having the principal and/or other teachers as the assessors indevelopmental appraisal given their familiarity with the context in which teachers work,their awareness of the school needs and their ability to provide quick and informedfeedback to the teacher. However, it might prove difficult for principals to undertake thethorough assessment of each teacher in the school. In addition, most principals have noprior training in evaluation methods and might not have the content expertise relevant tothe teaching areas of the teacher being evaluated. Hence, it might prove valuable to buildcapacity in appraisal methods at the school level by preparing members of themanagement group or accomplished/expert teachers to undertake specific evaluationfunctions within the school. In the context of developmental appraisal, evaluationadvisors could have a reinforced and clearer role: the position could be formalised andschools could benefit from resources to create such positions.OECD REVIEWS OF EVALUATION AND ASSESSMENT IN EDUCATION: DENMARK � OECD 2011

Reinforce linkages between teacher appraisal, professional development andschool developmentThe linkages between teacher appraisal, professional development and schooldevelopment need to be reinforced. Teacher appraisal is unlikely to produce effectiveresults if it is not appropriately linked to professional development which, in turn, needsto be associated with school development if the improvement of teaching practices is tomeet the school’s needs. Schools that associate the identified individual needs with theschool priorities, and that also manage to develop the corresponding professionaldevelopment activities, are likely to perform well (Ofsted, 2006). Schools can learn fromthe strengths of effective teachers and implement professional development programmesthat respond to their weaknesses. This is in line with the recommendation by the ‘FlyingSquad’ for engaging in a more strategic approach to professional development focused onschools’ needs and the vision of school leaders, in a context of extra support andresources from the municipalities (Danish School Agency, 2010; see Box 2.1).Effective operation of teacher appraisal and its contribution to school developmentwill depend to a great extent on the pedagogical leadership of school principals. Given thecentral role of principals in Denmark’s decentralised system, it is difficult to envisageeither productive teacher appraisal or effective school development without suchleadership. Other education systems have increasingly recognised the importance ofschool leadership in raising standards, as substantiated in an OECD report (Pontet al.,2008). Teacher appraisal will only succeed in raising educational standards if schoolprincipals take direct responsibility for exerting pedagogical leadership and for assumingthe quality of education in their schools. School principals are also more likely to provideinformal continuing feedback to the teacher throughout the year and not only during theformal appraisal process. More generally, they are essential to make performanceimprovement a strategic imperative, and to promote teacher appraisal as beingindispensable to teacher and school broader policies (Henemanet al.,2007; Robinson,2007; Pontet al.,2008). Therefore the recruitment, training, professional developmentand evaluation of school leaders should be given great importance. This is in line with the

Ensure appropriate articulation between school evaluation and teacherappraisalAnalysis from TALIS (OECD, 2009d) suggests that school evaluations can be anessential component of an evaluative framework which can foster and potentially shapeteacher appraisal and feedback. Given that the systems of school evaluation and teacherappraisal and feedback have both the objective of maintaining standards and improvingstudent performance, there are likely to be great benefits from the synergies betweenschool evaluation and teacher appraisal. To achieve the greatest impact, the focus ofschool evaluation should either be linked to or have an effect on the focus of teacherappraisal (OECD, 2009d). This indicates that the external review of schools shouldcomprise the monitoring of the quality of teaching and learning (see Chapter 6). Also, asindicated above, school review should comprise the external validation of the processes inplace to organise developmental appraisal, holding the school principal accountable asnecessary. Linkages between school review and teacher appraisal would also greatlybenefit from the improvement of skills and competencies for evaluation withinmunicipalities.In the context of school self-evaluation, it is also important to ensure the centrality ofthe appraisal of teaching quality and the appraisal of individual teachers. The quality ofteaching and the learning results of students are predominantly regarded as aresponsibility of groups of teachers or of the school as a whole. In this light, school self-evaluation needs also to put emphasis on assessing the appropriateness of mechanismsboth for internal developmental appraisal and for following up on the results of appraisalfor certification.

The introduction of mandatory municipal quality reports has been accompanied bycentral efforts to build municipal capacity. The common set of indicators in the municipalquality reports does not sufficiently address the quality of teaching and learning. Theinternal and external evaluation of schools should be based on an agreed set of formalcriteria of school quality. Municipal quality reports provide an agenda for dialoguebetween the municipality and the school principal, plus municipalities are required toproduce action plans for schools that are underachieving. However, the degree of follow-up by municipalities varies and is not always rigorous and objective. Outcome data andevaluation results should form a core part of the municipal monitoring system anddiscussion and follow-up with schools for improvement. Well led schools benefit fromeffective use of central or municipal self-evaluation guidelines, plus the rapid availabilityof results from the national tests. However, this is not the predominant culture. Arequirement for schools to produce an annual quality report could be an effectivestimulus for school self-evaluation.

Supervision and evaluation of private independent schoolsIn private independent schools, parents are responsible for supervising the generalschool activities. The parents are obliged to choose a person (the school principal) withprofessional and pedagogical qualifications to supervise the teaching of the school. Inaddition, the Ministry of Education conducts inspections of the quality of teaching. TheMinistry of Education can also undertake an enhanced inspection of a school. TheMinistry may conclude that the school is subject to exclusion from the scheme for publicfinancial support for private primary and lower secondary schools. On 1 August 2010,new rules of inspection came into force. Schools can now choose to implement self-evaluation as an alternative to a visit by an inspector elected by the parents. Furthermore,the inspector must have taken part in a special training programme focused on inspectingeducation and must be certified by the Ministry of Education. The involvement of thestate in inspection of private schools is a consequence of these schools receivingsubstantial government subsidies for the majority of their expenditure.

Supervision and evaluation of the FolkeskoleIn Denmark, each of the 98 municipalities is responsible for running the publicschools –Folkeskole– in their areas. The municipality defines the goals and scope for theschool activities within a framework of objectives set at national level. The chain ofaccountability and responsibility leads to the school principal. The school principal isemployed by the municipality and is responsible, both administratively andpedagogically, for the school activities in relation to both the objectives and policiesimposed by the municipal council and the principles set out by the school board. Inparticular, the school principal manages and distributes the work between the school staff,drafts proposals for the school ‘principles’ (aims and values), is responsible for the schoolbudget and takes decisions concerning the students. The school principal must workclosely with the school staff, although teachers are formally employed by the municipaldistrict council, following a recommendation from the school board, in practice theschool principal recruits the teachers.Since on the one hand the municipality establishes its own objectives and scope forthe schools, determines local guidelines and special initiatives, manages the expenditureand formally appoints the teaching staff of schools, and on the other hand the schoolOECD REVIEWS OF EVALUATION AND ASSESSMENT IN EDUCATION: DENMARK � OECD 2011

In practice, quality and development reports are generally prepared by the schools andforwarded to the municipal council after taking account of comments from the schoolboard. The municipal council then combines all the reports into a quality report for theschools in the municipality. The schools and municipalities with whom the quality reportwas discussed responded positively to this development, but thought that the qualityreports could be further developed in the future. It is reported that “many localmunicipalities now make use of the quality report in developing their school system”(EVA, 2009b), although current evidence suggests wide variation in practice. Wherethere is dialogue about the report between schools and education officers resulting insome specific goals for schools, there may be little subsequent follow-up. Schools placetheir quality and development report on their websites, rather as an advertisement for theschool. Although the major proportion of the quality report is prescribed by regulation,schools and municipal councils have some freedom to decide what else they include. Thegeneration and discussion of quality reports has been beneficial to promoting dialogue

The expectation that quality reports will lead to positive actionIt is an important feature of quality reports that they are intended to be the basis offurther action in managing the system at the level of the municipality. First, they areintended to provide an agenda for dialogue between the municipality and the schoolprincipal. At its best, such discussion is based on robust analysis of the school’sefficiency and performance, resulting in an objective consideration of the schoolprincipal’s performance and redefining personal and school objectives. There is anopportunity to set aspirational targets which, replicated on a large scale, couldcontribute to the improvement of educational performance nationally. Such dialogueshould be both challenging and supportive, but at present there is much variation inwhether and how it is conducted and what impact it has. Box 6.1 shows the approachused in Odense.Second, the quality reporting process requires municipalities to produce action plansfor schools that are underachieving. This should be an important lever for schoolimprovement provided the school has the capacity to take the necessary steps.International experience shows that this is not necessarily the case. Indeed it often takes achange of leadership to turn a school around (Matthews and Sammons, 2005). The SchoolCouncil has commissioned a specific study by EVA to shed more light on howmunicipalities follow up on action plans for schools (Skolerådet, 2010). Representativesfrom EVA informed the OECD review team that they are closely examining threemunicipalities that had included clear follow-up plans in their quality reports and thatresults would be available in June 2011.

94– 6. SCHOOL EVALUATIONsome schools, the meeting has been held once only; students have not always beeninvolved in dialogue about the ISPs; and some teachers criticised the time it takes toprepare them. The plans do however provide a mechanism for periodically reviewingeach student’s progress and involving students and parents in dialogue about it.The OECD review also revealed that where schools use form-level tests and the newnational tests well, teachers use the results to inform and cause them to reflect on theimpact of their teaching. The rapid availability (next day) of results from the nationalcomputer-based tests is welcomed, particularly by teachers in well-led schools wherethere has been dialogue about the use that can be made of the tests. The national testsappeared to concern some teachers more than their students, for a variety of reasonsincluding: fear of the tests including content that has not (yet) been taught and the factthat they only cover parts of the curriculum; and some opposition to the perceivedintention of the government to publish the results on a school-by-school basis.

These are all important and worthwhile steps which are beginning to impact on schoolaccountability and improvement. In particular, the OECD review team agrees with theassessment of the Chairmanship of the School Council that the introduction of the qualityreports has strengthened the evaluation culture in Danish schools (Skolerådet, 2010).However, while the implementation of the quality reports has introduced a moresystematic approach to the documentation of school quality, the extent to whichindividual schools actively use the results of the quality reports in development activitiesis yet to be documented. The OECD review team identifies the following as particularstrengths of the ongoing production of quality reports as part of the evaluationframework:•The process of generating annual quality reports benefits from being developedco-operatively, by requiring schools and school boards to play an active part, bothin providing data and in shaping the report. This gives schools and their boards asense of ownership over the quality report, provided they see value in it and thedata and information it contains. The opportunity to decide and shape some of thecontents of the report is an important incentive to counteract the bureaucraticburden of collecting the ‘framework data’ required for inclusion in the report. Asa result, there is a growing commitment to school review and self-review.Production of quality reports is resulting from the development of capacity inmunicipalities and schools to undertake school evaluation, although this is not yetapplied extensively to the quality of teaching and learning, the core processes ofthe school. Having a national template, locally interpreted, involves considerationof local needs and context, promotes dialogue between different players and givesschools and their boards some ownership of the reports.

There are many initiatives to build evaluation capacity at school and municipal levels.For example at the municipal level, the umbrella organisation for municipalities (KL) haslaunched several initiatives including notably the partnership of 35 municipalities (2007-2009) with a focus on management, the evaluation culture and professionalism ininclusive education, as well as producing various policy papers and conferences onevaluation. The former School Agency organised three annual conferences on qualityassurance for municipalities and developed an on-line evaluation resource showingexamples of different approaches that have been contributed by different municipalitiesthroughout Denmark. This serves as a central knowledge base on municipal qualityassurance development and monitoring and aims to promote collaboration and exchangeamong municipalities. Since 2008 EVA has offered ‘EVA days’ which offermunicipalities the opportunity to share information on how they develop and use resultsof quality reports (EVA, 2010). The self-evaluation guidelines and other tools offered byEVA are valued by some school principals who feel empowered and confident to be moreaccountable for the effectiveness and performance of their schools. EVA reported to theOECD review team that the climate is fertile to strengthen and update these self-evaluation tools and to include more critical and reflective components. Armed withinstruments that support evaluation, school principals have a great opportunity to takeownership of the evaluation agenda and re-professionalise their role as leaders.

ChallengesDefining what makes a good schoolThere does not appear to be an accepted model of school effectiveness in Denmark.Such a model would provide clear criteria for effective schools and provide a robust,research-based foundation for all internal and external evaluation. It would provideschools with criteria and benchmarks that would allow them to consider the evidenceneeded to rate their own effectiveness. ‘How good is our school?’ is a central question notonly for students and parents the world over but for those who lead and work in schools.Similarly, ‘How good are our schools?’ is the question for municipalities. ‘Good’ in thiscontext is synonymous with ‘effective’. The school cannot give an account of itseffectiveness unless it evaluates regularly ‘how students benefit from teaching and otheractivities. Such an evaluation should form the basis for guidance for the individualstudent and for further planning of the organisation and form of teaching, includingspecial interventions for the student. Similarly, the evaluation is also the basis forinforming parents about the school’s view of how the students are benefiting from theschool, and their wellbeing at school in order to enhance further the ongoing dialoguebetween the school and home’ (Christensenet al.,2007). There are no doubt many goodschools in Denmark, but as Collins (2001) said: “Good is the enemy of the great”. . . “Wedon’t have great schools principally because we have good schools.” Equally,‘satisfactory’ is the enemy of the good. In ambitious schools and municipalities, theanswer to the questions posed above must be ‘Not good enough!’OECD REVIEWS OF EVALUATION AND ASSESSMENT IN EDUCATION: DENMARK � OECD 2011

This limited set of indicators is mainly concerned with efficiency measures: levels ofprovision and other input factors, with the exception of student absence. It is useful inproviding input data at the level of the school, allowing provision to be monitored and ifnecessary re-distributed. But they are not quality indicators.It cannot be said, either, thatdescriptionsof pedagogical processes which comprisethe second set of mandatory indicators in the municipal quality report say much about thequalityof these processes. For example, “description of the continuous assessment ofstudent outcomes” may describe the assessment processes but not what use is made of theassessments, for example, to advance children’s learning, trigger intervention andsupport, and cause re-evaluation of the curriculum and pedagogy.Only the third group of indicators, “results”, has real meaning in giving the readerclear information about the effect of schooling. These results are appropriate outcomeindicators, especially in the case of those indicators that have national validity andreliability.Some of these shortcomings in the mandatory content of quality reports may indeedbe addressed by the local indicators adopted by individual municipalities, although wehave no evidence on the extent to which these include process indicators which reflectcore processes like the quality of teaching and learning.Given that municipal quality reports are currently the main instrument and product ofschool evaluation, internal or external (and their use is a combination of both), they arenot sufficiently scientific or focused to diagnose the strengths and weaknesses of theschool’s core processes relating to teaching and learning and leadership. The reportscontain indicators that are secondary to the pursuit of quality and that, in any case, shouldbe available to municipalities without the schools having to find and supply the data. Thereports also appear to lack critical independence and objectivity. Despite raising theprofile of school accountability, they have little to say about quality of education at thisstage of development.OECD REVIEWS OF EVALUATION AND ASSESSMENT IN EDUCATION: DENMARK � OECD 2011

Developing expertise in school self-evaluationThe OECD review revealed little evidence of school self-evaluation or observation-based appraisal of teachers. Professional dialogue is an important and deeply embeddedpart of school culture and often cited in answering questions about school self-evaluation.Dialogue is indeed an important part of a learning community, but runs the danger of beinga medium without a message. Southworth (2009) summarised the leadership of learning interms of “modelling, monitoring and dialogue”. Dialogue, when part of this trilogy, has theauthority that stems from demonstrating good practice and monitoring the effect ofpractice when discussing its strengths and how it could be improved. Joint planning andteamwork are strengths of many Danish schools, but the observation and evaluation ofteaching and learning by managers or peers – followed by feedback, discussion andpossibly coaching – is the exception rather than the rule. The OECD review team notesthat KL has recognised the need for class observation by school leaders.A professional culture of evaluation will only be embedded when professionalpractice (the craft of teaching) is openly shared among teachers and school leaders.Indeed, in schools where evaluation is encouraged, the students themselves providefeedback, even on occasion asking for harder books or more challenging work. But anevaluation culture which rests solely on dialogue – which the OECD review teamassesses to still be the predominant culture in Denmark – lacks the discipline involved inarriving at an assessment of the quality or impact of practice through the collection ofrelevant evidence and analysis against a framework of principles, criteria or benchmarksfor school improvement.

Embedding municipal feedback to schools and follow-up of schools for schoolimprovementThe feedback to schools based on the annual municipal quality reports is an importantmechanism for school improvement. This is not always rigorous and objective, and thedegree of follow-up is variable. Without robust evaluation, evidence-based feedback anda mechanism for monitoring and following up subsequent action, the municipal qualityreports can have little impact on school improvement. If quality reports are to becomepart of the evaluation culture, they must lead to feedback and follow-up since evaluationin the Danish system has been described as “an important stepping stone for developingand improving school performance” (Regeringen, 2010). The OECD review team foundthat practice varied between municipalities. While many municipalities make use of thequality assurance reports in developing their school system, it has been suggested thatOECD REVIEWS OF EVALUATION AND ASSESSMENT IN EDUCATION: DENMARK � OECD 2011

98– 6. SCHOOL EVALUATIONsome municipalities may lack the capacity to take on the school improvement function. Ifso, there is a case for joining forces and sharing expertise.An independent review, commissioned by the School Council and carried out by theDanish Evaluation Institute (EVA, 2009b), indicated that many municipalities:••Need support in the active use of the quality reports, especially in relation to thefollow-up processes on poor performing schools (the action plans).Face challenges in meeting the formal requirements of the quality reports,emphasising the need for tools and resources in the form of written guidelines onhow to develop and report key figures, how to report on good practice, and how touse existing data from central databases.Prefer test results to take account of the socio-economic context of the schools toprovide a more accurate measurement of their performance.Need clearer and more specified role assignments for the different actors involvedin developing the quality reports – who delivers what and when?Prefer that the quality reports should include strengths and weaknesses of theindividual schools in order to use the reports as an instrument for further schooldevelopment. In addition, many municipalities suggest that this entails a sharedunderstanding among local actors on how to use school assessment in developingand ensuring quality in schools.Develop follow-up or action plans for the school system rather than the individualschools.

The EVA review also reported that “in addition, we found little evidence ofcommunity involvement in the preparation of quality reports.” This was borne out by oralevidence presented to the OECD review team by schools and municipalities.The Quality and Supervision Agency is responsible for monitoring municipalities’quality reports and giving feedback, but describes its approach as being “very lighttouch”. There are no reports on how the current approach to school assessment achievesthe objectives of improvement and accountability, but the introduction of the qualityreports is judged to be beneficial (Skolerådet, 2010). However, there is still room forimprovement in relation to the strategic and systematic use of action plans by themunicipalities. While the implementation of the quality reports has introduced a moresystematic approach to the documentation of school quality, the extent to whichindividual schools actively use the results of the quality reports in development activitiesis yet to be documented. These weaknesses point to the need for thorough training ofmany municipal education directors and school principals in how to develop and makeactive use of quality reports.

6.3 Pointers for future policy developmentThe OECD review team notes the progress that has been made in establishing anassessment and evaluation framework while identifying some of the challenges inembedding the framework within an evaluative culture. The challenge of creating aculture where accountability and responsibility are accepted, where dialogue is replacedby reflection and evidence-based enquiry into what works best and why, and whereprofessional practice and knowledge are shared, is considerable. Kotter (e.g. 1996) andothers have written persuasively on leading change. Two of his observations are relevanthere, the first being that cultural change comes at the end of the process not at thebeginning. The second is about the importance of personal example – modelling – in“anchoring new approaches in an organisation’s culture”. “A particularly importantfactor”, he writes, “is a conscious attempt to show people how specific behaviours andattitudes have helped improve performance. When people are left on their own to makethe connections . . . they can easily create inaccurate links” (Kotter, 1996). On the basis ofthe analysis of strengths and challenges in this chapter, the OECD review team proposesthe following directions for policy development:••••••Define formal criteria of school quality;Radically improve the value of quality reports to school self-evaluation andimprovement;Review the role of school leaders and select, train and retrain them for that role;Identify the change leaders and replicate their practice;Strengthen the follow-up on school evaluation results;Promote and support capacity development in the external and internal evaluationof schools.

Define formal criteria of school qualityThe internal and external evaluation of schools, including the municipal qualityreports, should be based on a model or rationale for school effectiveness. Without this,the school evaluation framework lacks coherence. The characteristics for effectiveschools are well understood (Sammons, Hillman and Mortimore, 1995) and are broadlycommon to many national systems and school cultures. They relate to the quality ofteaching and learning – which has much to do with the calibre of teachers (Barber andMourshed 2007); the way teachers are developed and helped to become more effectivethroughout their careers (e.g. Barber and Mourshed 2007; Robinsonet al.,2008); thequality of instructional leadership in schools (Leithwoodet al.,2006) as well as factorsconcerning the curriculum, vision and expectations, assessment for learning, the rate ofprogress of students and their educational outcomes. Factors such as these are generallyassociated with the quality and standards of schools.For example, key quality indicators for student outcomes and their rate of progresscould include the extent to which:••••Every student in a school is making better than expected progress given theirearlier attainment;Every student is pleased with the education at their school;Every student feels safe and happy at school;Every student gains the knowledge, skills, understanding and attitudes necessaryfor lifelong fulfilment, etc.

Radically improve the value of quality reports to school self-evaluation andimprovementThe OECD review team encourages Denmark to continue to develop the qualityreports which have established a platform for accountability and improvement. Werecommend that this should be done in ways which encourage and take greater account ofschool self-evaluation and teacher appraisal, and that put the quality of teaching andlearning at the heart of the process. The OECD review team notes the recommendation bythe ‘Flying Squad’ (see Box 2.1) for schools to publish their own quality reports, plusreports from many municipalities that they would prefer quality reports to includestrengths and challenges for individual schools (see above). The process of producing anannual quality report would be a stimulus for many schools to further their self-evaluationpractices and holds strong potential for school improvement, if: the quality report payssufficient attention to key processes of teaching and learning and a broad range ofoutcomes; the process of drawing up the report adequately engages the schoolcommunity. In Sweden, the involvement of school staff, students and parents inproducing annual quality reports and their focus on monitoring school improvement(NAE, 2005a) has contributed to establishing a school self-evaluation culture. In fact,based on stakeholder feedback, the OECD judged that teachers had a sense of ownershipover the school self-evaluation and it emphasised democratic dialogue and that these were‘highly valuable asset(s) which should be preserved’ (Nuscheet al.,2011). Van Hoof andVan Petegem (in press) offer some principles and indicators to maximise the developmentof an effective school self-evaluation process (see Box 6.2).

Is prepared to engage in systematic reflection,e.g.team members are prepared toexamine their own professional practice critically and confident that results will notbe used improperly, there is sufficient openness and trust, etc.Works towards shared objectives,e.g.there are clear objectives for the self-evaluation and these are set out in writing, the team understands the criteria that willbe used to test the success of the self-evaluation, etc.Uses shared leadership as a means of creating involvement,e.g.self-evaluation innot just a matter for the school management, decision making procedures aretransparent and team members have the chance to be involved, the degree ofacceptance by team members is considered when making choices, school uses allpossible in-house expertise when making decisions, etc.Communicates effectively,e.g.communication is carried out in a co-ordinatedmanner, all those involved are properly informed about the objectives of the self-evaluation, there is open communication about the results, etc.Seeks to create supportive relationships and collaboration,e.g.the team hassufficient backing to carry out the self-evaluation successfully, collaboration avoidsduplication of tasks, there is mutual trust between school management and teammembers, etc.Integrates the self-evaluation process into existing school policy,e.g.theobjectives of self-evaluation are linked to other initiatives in the school, self-evaluation looks at policy aspects as well as educational aspects, make use of existingcommittees/work groups, etc.Is responsive with regard to internal and external expectations concerning theself-evaluation process,e.g.the expectations of external actors are taken into accountfrom the start, the self-evaluation process involves a critical friend, the self-evaluationactively seeks the input of the local community and takes account of general socialissues, etc.

Box 6.3 Quality criteria used in the Singapore School Excellence Model (SEM)Leadership:How school leaders and the school’s leadership system address values and focuson student learning and performance excellence; and how the school addresses itsresponsibilities towards society.Strategic Planning:How the school sets clear stakeholder-focused strategic directions,develops action plans to support its directions, deploys the plans and tracks performance.Staff Management:How the school develops and utilises the full potential of its staff to createan excellent school.Resources:How the school manages its internal resources and its external partnershipseffectively and efficiently in order to support its strategic planning and the operation of itsprocesses.Student-Focused Processes:How the school designs, implements, manages and improves keyprocesses to provide a holistic education and works towards enhancing student well-being.Administrative and Operational Results:What the school is achieving in relation to theefficiency and effectiveness of the school.Staff Results:What the school is achieving in relation to the training and development, andmorale of its staff.Partnership and Society Results:What the school is achieving in relation to its partners andthe community at large.Key Performance Results:What the school is achieving in the holistic development of itsstudents, in particular the extent to which the school is able to achieve the Desired Outcomes ofEducation.Source:Ng (2007).

For each quality criterion, evaluation in the SEM requirescompelling evidence.TheSEM is a self-evaluation system, which serves as a mechanism for school leaders to driveschool improvement. An external team from the Ministry of Education validates the self-evaluation results using the same criteria approximately once in five years. Theevaluation process is explicit in requiring evidence to justify a certain score. So, evenwhen a school is thought to perform well against a particular criterion, if there is noevidence of this, the model permits no score beyond that forad hocperformance.Moreover, to score well, a school, in addition to having explicit evidence relating to acriterion, must also have evidence of continuous improvement through trend analysis.Closely associated with the SEM is the Masterplan of Awards for schools. There are fourawards: Achievement Awards; Best Practice Awards, Sustained Achievement Awardsand the School Excellence Award (SEA), which gives recognition to schools forexcellence in education processes and outcomes. Schools may also apply for theSingapore Quality Award (SQA) just like any other industrial or commercial sectorOECD REVIEWS OF EVALUATION AND ASSESSMENT IN EDUCATION: DENMARK � OECD 2011

Review the role of school leaders and select, train and retrain them for that roleIn Denmark, a top-down approach to educational reform is confronted by a relativelyautonomous teaching force whose classrooms are sovereign. The key agents of changeare not to be found in the ministry or the municipalities but in the leadership of schools.School principals are pivotal in developing an assessment and evaluation culture with anemphasis on high-quality provision and the best outcomes possible for students. Thisargues for a shift in the role of school principal from one who administers and managesthe school and organises its staffing, students and programmes to one who is thepedagogical leader of the school. Findings from the International Successful SchoolPrincipalship Project (Mooset al.,2008) showed that, for example in Ontario, Canada,new public management policies had stimulated school principals to set goals for theirschools drawing on both provincial tests and initiatives, as well as their own broader viewof learning, setting high expectations for student achievement, while offering support andacting as role models to their staff. This new shift to a pedagogical leader included‘planning and supervising instructions that often include monitoring teachers’ practiceand modifying school structures, like the school day, to maximise learning’ (see alsoBox 6.5).The OECDImproving School Leadershipreport (Pontet al.,2008) has identified fourcore responsibilities of school leadership based on an empirical analysis of which rolesmake a difference in improving school outcomes. The first two of these are directlyconcerned with the quality of teaching and learning and evaluation and accountability(Box 6.4).The recent international study of school leadership in high performing systems(Barberet al.,2010) found a consensus on the importance of school leadership and howto improve it which recognised, among other things that:••Leadership focused on teaching, learning, and people is critical to the current andfuture success of schools;High-performing school principals focus more on instructional leadership and thedevelopment of teachers.

Managing the curriculum and teaching programme:Most countries establish a core curriculum atthe national level. National policy is often further specified at regional or municipal level. It is theleader’s job to implement the school curriculum within these policy boundaries in a manner thatachieves the intended curriculum objectives for their specific context. School leaders generally have ameasure of discretion in how they design curriculum content and sequencing, organise teaching andinstructional resources and monitor quality. Giving schools a greater say in curricular decision-makingallows for tailoring education and making it significant to different cultural and regional groups, andthus seems to be positively related to student performance, provided schools have the confidence andcapacity to make this type of decision.Teacher monitoring and evaluation:The OECD Teaching and Learning International Survey(TALIS) showed that of 18 countries reporting on teacher monitoring and evaluation there wereformal provisions for teacher evaluation in 14, although the form, rigour, content and consequences ofteacher evaluation varied widely. Most of the countries indicated that teacher monitoring andevaluation are important responsibilities carried out by school leaders. Several research studiesindicate that school leader involvement in classroom observation and feedback seems to be associatedwith better student performance.Supporting teacher professional development:School leadership also plays a vital part inpromoting and participating in professional learning and development of teachers. The balancebetween school-based and out-of-school professional development has moved strongly in favour ofschool-based professional development in recent years. The OECD (2005b) noted that school-basedprofessional development activities involving the entire staff or significant groups of teachers werebecoming much more common, and teacher-initiated personal development probably less so. The mostpersuasive evidence of the impact of school leaders’ involvement in promoting and participating inteacher learning and development is probably that of Robinson’s 2008 meta-analysis of six researchstudies. She identified the participation of the school leader as the “leading learner” in staffdevelopment as being strongly associated with improved student outcomes.Supporting collaborative work cultures:This is an increasingly important and recognisedresponsibility of school leaders in several countries and involves fostering teamwork among teachersand creating environments in which student learning is the central focus. Policy makers can promoteand encourage teamwork among school staff by explicitly recognising the core role of school leadersin building collaborative cultures and by disseminating and sharing best practice in this aspect.

Identify the change leaders and replicate their practiceDuring the short time of the OECD review, we identified school leaders who reflectedthe principles of effective instructional leadership described above, in contrast to schoolprincipals who had never observed teaching and learning taking place as a matter ofroutine. One of the school principals in the first category was sharing his approach with acluster of schools in the Odense municipality and helping them to develop and improvetheir outcomes. He was in turn subject to robust appraisal by the education director of themunicipality. It should be a priority to identify the school principals and municipaldirectors who exemplify good practice so as to set standards for leadership practice anddisseminate their example to an expanding cadre of receptive school principals.Ultimately, Denmark will need to decide who is accountable for the performance ofindividual schools. If it is the school principal or the school board, then they will need thedecision-making powers to be able to choose the resources they need to do the job withinthe limitations of the budget. If the municipalities, then they may need to be reconfiguredso that directors of education become, in effect, executive school principals for a clusteror chain of schools, each headed by a head of school.There is considerable evidence that in other administrations, including England, Finlandand Sweden, school-school partnerships, clusters and networks can provide mechanisms forsharing effective leadership as well as effective practice in a way that contributes to raisingthe performance of the member schools (Pontet al.,2008, p. 56). In England, executiveleadership across partner schools has proven to be a very effective mechanism for raisingthe performance of underachieving schools (Hill and Matthews, 2010).

Strengthen the follow-up on school evaluation resultsThe school evaluation culture will not be endemic until evaluation is shared, followedup and reviewed to see what difference it has made both internally by schools andexternally by Municipalities. Outcome data and evaluation results should form a core partof the municipal monitoring system and discussion and follow-up with schools forimprovement. In particular, nationally comparable information, including national testresults, transition statistics and student final grades in Form 9, provide comparativeinformation across schools that can be used by municipalities most constructively toidentify improvement and share best practice among schools. In light of theGovernment’s proposal to publish results from the national tests for individual schools,municipalities and schools need to go further to ensure constructive use of these outcomedata and strive to complement them with other measures. Copenhagen’s annual studentsurvey is a good example of a systematic collection of information on broader schoolingoutcomes to complement student academic results.A School Council study (Mehlbye, 2010) revealed that top performing schools inDenmark had strong management and clear objectives and a strong culture of academicachievement for all students – the study included successful schools with socio-economically disadvantaged student populations. This is an example of how evaluationand outcome data can together provide the evidence and examples to challenge pre-conceived attitudes and say what works. It is critical that schools do not perceive studentsocio-economic disadvantage as an immutable reason for low educational attainment.Recent reports by the Office for Standards in Education in England showcase schools thatrefuse to accept this assumption and achieve exceptionally high results despite working invery challenging circumstances (Ofsted, 2009).OECD REVIEWS OF EVALUATION AND ASSESSMENT IN EDUCATION: DENMARK � OECD 2011

Promote and support capacity development in the external and internalevaluation of schoolsAs evaluation of teaching and learning and appraisal of teachers takes root, the OECDreview team sees a strong case for having a centre of expertise in school evaluation atarm’s length from schools and municipalities to develop evaluation frameworks andcriteria, model good practice and even evaluate – through sampling – the real quality ofDanish schools. EVA is well placed to undertake such a remit. Central underdevelopedand underused expertise in school and municipality evaluation can still be found withinEVA. The OECD review team considers it would be prudent to nourish and refresh thisexpertise in order that the system has an authoritative centre for school evaluation. Oneuseful and informative way of disseminating best practice would be for EVA to becharged with evaluating the effectiveness and capacity for evaluation of a sample ofschools, one in each of a number of municipalities which is regarded by the municipalityas leading the way locally in self-evaluation and teacher appraisal. Equally, the Qualityand Supervision Agency should do serious analysis of the value and impact of qualityreports with a view to identifying and disseminating best practice (see Chapter 7).The seeds of an evaluation culture have been planted in Danish compulsory schools,but there is much further to go. The Ministry of Education could initiate a process withkey stakeholders to draw up competency profiles for both municipal education directorsand school principals – each hold influential positions in furthering the effective externaland internal evaluation of schools. For example, the Ontario Ministry of Education hasestablished a ‘Leadership Framework’ in collaboration with school leaders and schooldistrict supervisors (see Box 6.5). Further, school self-evaluation can be promoted byretraining school principals in school effectiveness and its evaluation, including thetechniques of observing and assessing teaching and learning and giving developmentalfeedback. School subject supervisors should be trained as the next step, with theexpectation that they will take responsibility for performance in their fields. Teachereducators should be engaged much more closely in the practice of school and classroomevaluation, and teacher education and the appraisal of teaching would benefit from thedevelopment of evidence-based standards showing minimum benchmarks for good andeffective teaching (see Chapter 5).

Denmark has developed national measures on outcomes, including the publication offinal examination results in Forms 9 and 10. The new national tests offer the first realopportunity to reliably monitor progress in educational outcomes over time against thenational Common Objectives. However, the lack of inclusion of the private sector plus alack of clarification of how results will be used to hold schools accountable, limits theirmonitoring value and there should be a careful review of strategies to maximise this.Further, it is important to develop a strategy to complement existing national monitoringinformation with broader measures of outcomes, including stakeholder views on thequality of teaching and learning. The Quality and Supervision Agency has the mandate tomonitor school providers and should identify municipalities where real progress is beingmade in student outcomes and share this knowledge throughout the system, plus devise anoptimal system to feedback key results held at the national level to municipalities for theirmonitoring purposes.

7.1 Context and featuresResponsibilities for evaluation of the Danish compulsory educationEstablishing a system evaluation frameworkThe Ministry of Education is responsible for working with the Minister to draw upany necessary laws and to ensure a clear legal framework for compulsory education.Regarding the evaluation framework, the Ministry of Education sets the legalrequirements for continuous evaluation and compulsory testing, ensures the frameworkfor supervision of private schools and defines the common basis for evaluation incompulsory education – the Common Objectives to be achieved at key stages and by theend of compulsory education. The Common Objectives were revised in 2009 to match thenew objectives for theFolkeskole.The Ministry of Education lead these revisions, whichwere prepared by a series of working groups formed of researchers and teacherrepresentatives, drawing on recommendations by several subject expert committees.There follows a consultation process with stakeholders before any amendment to theFolkeskoleAct.

The major collection, processing and presentation of education data is conducted byUNI-C (the Danish IT-Centre for Education and Research), an agency under the DanishMinistry of Education. UNI-C develops and maintains the educational databases on theMinistry of Education’s website (Databanken) which include all major benchmarks forcompulsory education (outcome data, transition to secondary education, number ofstudents enrolled). Plus, UNI-C calculates the ‘profile model’ that is a statisticalprojection of the course of study the current youth cohort will take over the next 25 yearsafter completing the Form 9. Schools are responsible for directly reporting their Form 9and 10 examination results to UNI-C.The Danish Evaluation Institute (EVA) – an independent state institution under theMinistry of Education established in 1999 – has a mandate to evaluate all levels ofeducation and conducts research and evaluations on its own initiative. However, as of2006, the School Council commissions official evaluations in compulsory education andthese are often large-scale evaluations.The Danish Council for Strategic Research recently gave financial support to theCentre for Strategic Educational Research to bring together researchers for targetedresearch on priority areas in theFolkeskole(Rambøll, 2011).

Major tools to measure performance in compulsory educationParticipation in international student surveysDenmark has shown heightened interest in international benchmarks of studentperformance over recent years. Participating in the OECD’s Programme for InternationalStudent Assessment of 15-year-old students since its inception in 2000, Denmark hasrecently administered tests to younger students (Grade 4) by the International Associationfor Educational Achievement’s (IEA), including the Progress in Reading Literacy Skills(PIRLS) survey and the Trends in Mathematics and Science Skills (TIMSS). Denmarkalso supports international comparisons on non-cognitive outcomes, including itsparticipation in the recent IEA International Civic and Citizenship Education Study(ICCS 2009). Plus, Denmark has participated in international surveys on ICT use and theOECD’s Teaching and Learning International Survey (TALIS).Outcome measures offered by participation in international studies have been themajor indicators of performance in compulsory education in Denmark. In particular, thePISA results were very much driving educational policy deliberations in the absence ofnational measures (Rambøll, 2011).

Performance on school-leaving examinations in Forms 9 and 10 and transition tofurther educationThe average results in Forms 9 and 10 are available on the Ministry of Education’swebsite (for both public and private schools)20. These include final grades awarded byteachers (Standpunkt), as well as results in final examinations (written and oral) in bothcompulsory core subjects (bundneprøvefag)and randomly selected subjects (Prøvefagtiludtræk).Schools are required to submit student results on Forms 9 and 10 examinationsand final grades each year to UNI-C. Results are presented in a selection of thematicstatistical tables, for example showing average results by municipality or by individualschools, or for Denmark by student migrant background. Plus, a written report in pdfform is available for each year since 2006 (this is produced by UNI-C). Results arereported on a 7-point scale that follows the European Credit Transfer System (ECTS)scale developed for higher education institutions (see Box 7.1).Transition statistics on student academic or labour market destination after completionof Form 9 are also published on the Ministry of Education’s website (see Box 7.1).

Thematic evaluations of different aspects of compulsory educationDenmark increasingly makes use of special thematic evaluations or studies to bringmore information at the system level. A notable example is the nation-wide researchproject in 2005 involving a special administration of the PISA test on a sample of 4 000students with an overrepresentation of students with a migrant background (PISA Ethnic).The study aimed to further elucidate on factors associated with observed performancegaps in the PISA 2000 and 2003 surveys. Subsequently, Denmark also participated in theOECD Review of Migrant Education (see Nuscheet al.,2010). For the PISA 2009survey, Denmark chose to oversample students with a migrant background and, therefore,was able to conduct an in-depth examination of their performance and learning profilescompared to those of native Danes based on the main survey outcomes (see AKF, 2011).The School Council decides on national large-scale evaluations to be conducted incompulsory education. These include major evaluations of national initiatives that areconducted by EVA, plus research studies conducted by EVA and other partners. A recentexample is a qualitative study on factors contributing to sustained, high academicperformance in schools, including schools with students from disadvantaged socio-economic backgrounds (Mehlbye, 2010). The study included 12 schools, of which 8demonstrated sustained high grades in Form 9, including 4 with less advantaged studentpopulations. The schools were selected by a screening of register data.

7.2 Strengths and challengesStrengthsStrengthened national structure to monitor and evaluate the FolkeskoleThe heightened priority given to monitoring and evaluation of compulsory educationcan be seen with the decision to create both the Quality and Supervision Agency (and theformer School Agency) and the School Council. The Quality and Supervision Agency hasthe mandate to monitor, evaluate and promote quality in the Danish school system. Theformer School Agency – primarily through the development of national tests –considerably strengthened the ability to monitor the average outcomes of students in theFolkeskole.This work continued by the Quality and Supervision Agency results in thenational profile that will be published each year as a good indicator of studentperformance in key subjects at different stages of their compulsory schooling and willallow analysis of student progress over time. Importantly, the Quality and SupervisionAgency also has the mandate to monitor school providers (see below). In addition, theSchool Council has introduced a more systematic evaluation of theFolkeskolebycommissioning high-quality evaluations on a large scale in different priority areas.

schools the Quality and Supervision Agency has identified as having sustained qualityconcerns. For public school providers – the municipalities – such supervision is conducted‘at arm’s length’ via the analysis of the annual quality reports published on line by eachmunicipality. Currently, the Quality and Supervision Agency monitors four key aspects ofthe quality reports: whether all required points have been addressed by municipalities;whether the reports provide enough information for municipalities to gain a solid overviewof their school system and to intervene in a timely manner where necessary; whether thereport includes a description of action taken by the municipalities to improve school quality,notably ‘action plans’ for underperforming schools; whether the report was published intime on the municipality’s homepage (Danish School Agency, 2009). Plus, the Quality andSupervision Agency screens all schools on basic performance indicators each year(e.g. Form 9 results). If the Agency identifies schools with persistent poor performance, itcan use the quality report as a basis for dialogue with the municipality on possible solutionsand action plans. The quality report should provide good indication on whether/to whatextent the municipality is aware of the problems identified in the given school(s). Themunicipalities are legally required to draw up an ‘action plan’ for underperforming schools.The Minister has the right to intervene if the municipal follow-up is judged inadequate(although the representatives from the former School Agency informed the OECD reviewteam that this had never happened). For private schools the supervision can involve a visitfrom members of the Quality and Supervision Agency, as part of a regular thematicevaluation or in the case that there are quality concerns in that school. While there is a newpossibility for private schools to elect their own Ministry-approved evaluator, members ofthe Quality and Supervision Agency would still visit such schools in the event there werequality concerns (see Chapter 6).

Inclusion of national outcome data for compulsory education in the monitoringsystemSince 2006, the national monitoring system for compulsory education has beenconsiderably strengthened by the inclusion of national data on student outcomes. Thepublication of final examination results in Forms 9 and 10 alongside teacher-awardedfinal grades serve as a the major indicators of overall quality in theFolkeskoleand arebolstered by transition statistics showing student destination 3 months after completingForm 9 of theFolkeskole.The advantage of such data is that they cover all the publicsector and the majority of private providers of compulsory education, which allows amonitoring of the situation for Denmark and informs debate on the overall productivityagenda. Despite limitations of such data for tracking trends (see below), these are themost comprehensive national indicators available for monitoring compulsory educationand as such play a key role in broadening the national debate beyond results ininternational assessments.The introduction of the national tests also offers monitoring information on publicschools at early stages in compulsory education. The OECD review team commends theformer School Agency’s efforts for the development and initial validation of the nationaltests. The first national profile was published in December 2010. The national tests aredesigned to offer indicators of how performance changes over time and are conductedwith varying subject intensity in Forms 2, 3, 4, 6, 7 and 8. In Danish, the national tests areconducted in Forms 2, 4, 6 and 8 and in mathematics in Forms 3 and 6. The national testdesign offers the potential to link items in these subjects across a common scale, thusallowing a measure of student progress across Forms (Wandall, 2010).

Serious commitment to use of evidence and evaluation to monitor compulsoryeducationThe availability of system-level information has both sparked intense nationaldebate on schooling and fuelled the need for more key outcome measures. Results frominternational assessments (PISA 2000 and PISA 2003) generated intense debate amongpolicy makers with a major focus on ‘value for money’ given Denmark’s comparatively‘mediocre’ performance and comparatively expensive public education (Danish SchoolAgency 2009). Denmark has looked to the OECD for external evaluation of key aspectsof the education system, including the commissioned review of theFolkeskolein 2003,participation in OECD reviews on School Leadership and Migrant Education and thepresent review of Evaluation and Assessment Frameworks. System results showinglarge performance discrepancies between native Danes and students with a migrantbackground led to an in-depth statistical study of the key competencies of the schoolpopulation with a migrant background (see Egelund and Tranæs, 2008). The review byEVA in 2005 on the municipal supervision of schools was a key catalyst to introducingthe municipal quality reports. In general, there is a considerable strength andcompetency in monitoring of national initiatives undertaken by EVA, for example, theongoing evaluations commissioned by the School Council on monitoring the impact ofthe municipal quality report system. Similarly, as teachers criticised the amount of workinvolved with drafting student plans, the government is currently conducting anexperiment in 360 schools (as of September 2010) to allow teachers more freedom inwriting the student plan (Rambøll, 2011). Most recently, the Prime Ministercommissioned a review of theFolkeskoleto feed into the government’s reform proposalfor early 2011 (see Box 2.1).

Principle of transparency at the national levelThe availability of evaluation and assessment information to key stakeholders is afirst step in ensuring their engagement with the results. Since 2002, it has been arequirement for schools to make performance data and any external evaluation resultsavailable to the public (usually on the school website). The government in its latestreform proposal has pledged to go further in transparency of national reporting. TheOECD review team supports this principle and encourages the constructive use ofcomparative information for school improvement; however, as noted below, cautions onthe importance of maximising the benefits and minimising the potential negative impactswith regard to the government’s proposal to publish the national test results for schools,given that the original purpose and design of the national tests was as a tool to monitornational outcomes and to provide diagnostic information for teachers.Major national outcomes results are publically available at the national level. Resultsfor all schools on key outcome data such as school average results in Forms 9 and 10 andtransition data to further education/employment are available in the Ministry ofEducation’sDatabankenwebsite – results are also presented by municipality (seeBox 7.1). Further, past examinations are also available in theEvaluation portal.Anannual national performance profile is published on the Quality and Supervision Agencywebsite for each subject and Form level tested in the national tests and currently resultsare made available to key stakeholders for their interest group,e.g.school leaders see allresults for students in their school, municipal education leaders see results for all schoolsin their jurisdiction.

Recognition of the importance of looking at broader outcomesAt the political level there is justified pride that Danish students are internationalleaders in civic and citizenship education as assessed by the IEA’s ICCS. These areconsidered to be important outcomes of compulsory education. There is also awarenessthat parental choice is often more heavily influenced by school environment factors thanby school academic outcomes (Nuscheet al.,2010). The Competitiveness Reportincludes an indicator on student attitudes to collaboration (e.g. Danish Ministry ofEconomics and Business Affairs, 2009), which appears an attempt to signal theimportance of looking at wider outcomes, but at the same time may indicate the limitedinformation currently available.Within the system, there is strong support from major stakeholders for developingmeasures on broader outcomes. For example, the KL partnership from 2007-2009included a focus on student well-being and skills in creativity, innovation, problemsolving and collaboration as reported by students, both at the start and end of thepartnership (KL, 2009). Further, there has been a pilot large-scale parent surveyconducted in five municipalities in 2010 to determine suitability as a benchmark measureof parent satisfaction across municipalities (Rambøll, 2011) and Copenhagen already runsan annual student survey on their school experiences (the ‘Copenhagen Barometer’).

ChallengesUnder use of system-level data – in particular on outcomesThere have been considerable efforts over the past five years to provide reliableinformation on outcomes at the system level, notably by making Form 9 examinationscompulsory and introducing national tests. While results for Form 9 and 10 finalexaminations are made available in theDatabanken,these are not systematicallypresented in key national reports,e.g.the Ministry of Education’sFacts and Figuresreport series. Currently, it is not possible to gain an overview of these outcomes alongwith new ‘national profile’ information available from the national tests. Further, thecurrent presentation of results from Form 9 and 10 examinations is not accompanied byan analytical component and this simple ‘benchmark’ style is also mirrored in theCompetitiveness Report. The tabular presentation of Form 9 results impedes an overallcomparison: the user can see individual school results, results for that school’smunicipality, results for Denmark (but all displayed as individual tables).The OECD review team sees more room for the Ministry of Education to consult withUNI-C when drawing up policy priorities, in particular by making more use of theAgency’s statistical analytical competency. The Agency’s micro-simulation model is oneexample of an effective collaboration to track progress toward reaching the political goalOECD REVIEWS OF EVALUATION AND ASSESSMENT IN EDUCATION: DENMARK � OECD 2011

118– 7. SYSTEM EVALUATIONof 95% of the youth cohort completing upper secondary education in 2015. The currentanalytical approach appears to be to commissionad hocspecial thematic studies,e.g.in2009 on average cost for different student groups (migrant background, bilingual, specialeducational needs, etc). Statistical analysis could be more proactive for example byestablishing a systematic early warning system on key indicators. In general, the OECDreview team had the impression that there was limited feedback to municipalities on keyoutcome data that are centrally available and could feed into the analysis of municipalschool systems. Representatives from EVA reported to the OECD review team that theirreview of the implementation of municipal quality reports indicated that municipalitiescould greatly benefit from a centralised data provision on key performance indicators –77% of municipalities surveyed lacked easy access to data already provided for othercentral databases. This cumbersome information collection was perceived to be a barrierto municipalities making use of the possibility to include additional local indicators.

Measures of higher-order thinking skills and cross-curricular competenciesFirst, it is not clear to what extent current national measures are assessing higher-order thinking skills and cross-curricular competencies. Importantly, the current finalexaminations offered in Forms 9 and 10 are perceived by the Danish Student Associationto be ‘outdated’ in the skills they assess – a criticism echoed by the ‘Flying Squad’ in itsrecommendations to modernise the final examinations (see Box 2.1 and Chapter 4). Thisis a serious concern if final examinations serve to signal the expected outcomes ofcompulsory education in Denmark. Further, it would stand in contrast to the politicalambition to be in the top five performers internationally in the OECD’s PISA assessment– an assessment of student key competencies and their ability to apply their knowledgeand skills in real life contexts. In-depth analysis of the PISA 2003 mathematicsassessment showed that the most demanding questions not only required students toanswer with little or no guidance (and a number of answers might be acceptable), but alsorequired students to write an explanation of their conclusion or justify their results(OECD, 2009e).

Concerns on the comparability of measures of output from compulsory educationThe comparability of the final grades awarded by teachers can be called into questiongiven the reported lack of clarity of the current criteria and Common Objectives.Although national guidelines and examples are offered via theEvaluation portalforteachers21, there is much room for interpretation by teachers in awarding final grades tostudents. It would seem prudent to evaluate to what extent the awarding of teacher gradesvaries across schools and municipalities. This is important given the publication of resultsat the national and school levels and the current use of such results to compare schoolsand to inform on national outcomes. There is no guarantee that final grades provide areliable measure of school or national improvement over time.Whilst the grading of written and oral examinations is moderated by an externalcensor, the final examinations serve a summative purpose and therefore are not designedto test a common set of items to allow comparison of performance over time. Indeed,representatives from the former School Agency advised the OECD review team thatobserved fluctuations in final examination results may not necessarily reflect realperformance changes due to the change of content area assessed from year to year.Further, national reporting of students’ final grades and final examinations results byschool makes no allowance for the average composition of students at the school in termsof their socio-economic background.

Misconception of what skills the national tests measureAs yet, there has been no independent evaluation of how the national tests functionand the skills set that they assess. The evaluation of the national tests conducted by theNational Audit Office focused on the implementation of the testing system (Rambøll,2011). The design of the national tests capitalises on rapid feedback to teachers of studentperformance in discrete areas of the national objectives. It would be important to evaluateto what extent the use of multiple choice format compromises the types of skills assessed.Well designed multiple choice questions can measure quite complex cognitive processes.For example, the PISA mathematics assessment includes some complex multiple-choicequestions that require students to demonstrate a degree of sustained thought and exposethe thoroughness of the students’ understanding of the mathematical concepts and skillsinvolved in solving the problem – a few of which are among the most difficult tasks in theassessment (e.g. OECD, 2009e). However, generally questions with a simple multiplechoice format were among the easiest in the test (idem). Indeed, findings from an OECDpilot of a computer-based science assessment in Denmark, Iceland and Korea revealed the21.Readers can access theEvaluation portalvia the homepage of the Quality and SupervisionAgency atwww.ktst.dk.

Ensuring results reflect real progress in outcomesAt this stage, the national tests do have potential as an indicator to gauge progressfrom year to year in theFolkeskole.Student scores on the tests are automaticallycalculated and generated and therefore results have high scoring reliability. But theOECD review team sees a significant challenge in ensuring that the national tests providea valid measure of student progress due to the lack of clarity over the purpose of the testsas communicated by different stakeholders during the OECD review, specificallyeducators’ fears that results will be used to hold them directly accountable. Wandall(2010) states that the national tests were designed for both regulation and control and as apedagogical tool and to serve both purposes the test ‘has to be low stake’ and that is why‘test results are made strictly confidential by law’. In this context, it will be crucial for theMinistry of Education to clarify the major purpose of the announced future publication ofresults from the national tests. Research from the United States has shown that if nationaltests are considered to be ‘high stakes’ for teachers and schools, teaching to the test caneasily lead to an artificial over-inflation of results and thus render the results useless as ameasure of real progress (e.g. Koretz, 2005). While the computer medium of the nationaltests in Denmark avoids concerns over intentional erroneous marking by teachers(a considerable advantage), there is potential risk to their value as both a monitoring tooland a pedagogical tool through teachers’ under-use of the test results or their over focuson the discrete content areas that are assessed. Wandall (2010) defines ‘teaching to thetest’ for the Danish national tests as ‘too much focus on tested profile areas and too littlefocus on creative, innovative and oral skills (which play a significant role in thecurriculum of theFolkeskole)’.

Optimise the reporting and use of system-level dataThere could be attempts to more effectively communicate results from the nationalmonitoring system to encourage their use by different stakeholders. The reporting of finalgrades and examination results in Forms 9 and 10 would benefit considerably from ananalytical component to aid interpretation of results and, in the case of average results byschools, the addition of contextual data and adjusted measures to show the ‘value added’(see Box 7.3). With the new set of data available from the national tests – and respectingthe current policy of only publishing results at the national level – the value of thenational tests in monitoring national progress in discrete areas could be enhanced by:••Linking the test items across different Form levels to show progress of givencohorts at different stages of compulsory education (see Box 7.3);Reporting the national performance profile by gender and by student background(migrant background and socio-economic background) to allow the tracking ofimprovement for these key groups over time;Reporting a distribution of municipality and school results (without identifyingparticular municipalities or schools) to monitor performance variation amongmunicipalities and schools over time.

Consider ways to further complement the new national monitoring systemBroadening national measures on student outcomesDenmark has recently invested in a computer-based adaptive testing system and thishas significantly strengthened the availability of information on outcomes of students incompulsory education to many stakeholders throughout the system. Consideration shouldbe given to further developing the national tests by introducing performance type tasks(see Chapter 4). The aim would be to have national measures of student higher-orderthinking skills to monitor progress in stimulating more students to excellence. In thelonger term, Denmark may wish to consider introducing a light monitoring sample survey– such surveys can provide stable trend information and monitor a broader range ofstudent knowledge and skills compared to a full cohort test – to supplement the nationalmonitoring with information on broader student outcomes. For example, in Finland asurvey is used to monitor students’ ‘learning-to-learn’ skills (see Chapter 4). Somesystems only use monitoring surveys to inform system monitoring needs and do not makeuse of full-cohort student tests, but ensure that these cover a wide range of the nationalcurricula (e.g. New Zealand and the Flemish Community of Belgium). In Australia, thereis a suite of national assessments comprising both full cohort student tests in numeracyand literacy, and cyclical sample surveys to monitor student outcomes in science, ICT,civics and citizenship (Santiagoet al.,2011). The sample surveys draw on a statisticallyrepresentative sample of students at target form levels (equivalent to about 5% of thecorresponding population). Each area surveyed represents an agreed national priority andis tested once every three years. The first survey was run in 2003 for science, in 2004 forcivics and citizenship and in 2005 for ICT. Each assessment results in a national reportshowing student average performance and proportion of students at the set ‘proficientstandard’ for each state and territory, each school sector (e.g. government and non-government) and for selected student subgroups (e.g. by Indigenous and socio-economicbackground) and allows a reporting of progress over time, as each subject is assessedevery three years (see for example MCEECDYA, 2010). Australia also capitalises oncomplementary information from international assessments by only administering thescience survey in Form 6 (using PISA science results to inform progress in Form 10). Forboth ICT and civics and citizenship students are assessed in Forms 6 and 10.The Ministry of Education should take stock of existing efforts in municipalities todevelop measures of creativity, problem solving, collaboration and innovation (KL, 2009)and evaluate to what extent these could be supported and extended throughout the system.Indeed, there is strong support from key stakeholders to examine how to best clarify

Developing national measures on teaching and learning qualityDenmark has recently piloted a parent survey which will be evaluated and consideredas a potential national monitoring tool. At the heart of the compulsory education systemare the students. Their perspective and opinions can be incredibly enlightening in formingpolicies for school improvement. They could offer insightful feedback on thedevelopment of the evaluation culture, for example, by reporting on the use of evaluationat their schools, their opportunity to give feedback to teachers and school leaders on theirlearning and educational priorities, their regular assessment activities and theirperspective on the usefulness of these. Copenhagen’s experience with its annual survey ofstudents in Forms 4 to 9 shows both high response rates (in 2010 this was 76% – onlystudents who full completed the survey were included) and useful analytical informationon broader schooling outcomes (Københavns Kommune, 2010).23Norway introduced astudent survey in 2005 and this forms a key part of the national reporting on the educationsystem. In the annual summative report on education in Norway (TheEducation Mirror)there is always a clear presentation and analysis of results from the survey and these feedinto the national policy debate (Nuscheet al.,forthcoming). This is one way to ensure thesystematic inclusion of student perceptions at the political level. The Quality andSupervision Agency could also give consideration to the collection of some studentfeedback on key issues during the administration of the national tests. Certainly, thecollection of information from students during the administration of international surveyshas led to informed analysis of how different reported factors relate to studentperformance,e.g.classroom climate factors such as discipline and student-teacherrelations have shown strong correlation with student achievement (e.g. OECD, 2004b).Results from the KL partnership study showed that during the period of increased focuson learning outcomes students remained happy and confident and in fact the proportionreporting this had increased by the end of the study (KL, 2009). Further, KL identifies theneed to build on new measures such as membership, community, happiness andwellbeing.

Further validate and clarify the monitoring role of the national testsThe OECD review team strongly advocates a careful review of strategies to maximisethe monitoring potential of the national tests at the system level without compromisingtheir reliability as a monitoring tool (i.e. avoiding the artificial inflation of test scores thatdo not reflect real improvement) and their use as a pedagogical tool (see Chapter 4). TheOECD review team believes that the current government proposal to publish national testresults at the school level is premature. The priorities would be to continue to validate the22.This is included in the joint policy paper by the Confederation of Danish Industry, DanishTeachers Union, Danish School Leaders Union, Local Government Denmark, DanishStudents Association and the Parents and Society Association, as part of the ‘Vores Skole’project.The 2010 survey include 53 questions on students’ experience in school regarding security,welfare, health, democratic education, happiness, motivation to learn, recreation and habits.Response rates among Copenhagen schools varied from 52% to 97% (KøbenhavnsKommune, 2010).

124– 7. SYSTEM EVALUATIONnational tests (see Chapter 4) and to go further in supporting and promoting capacitybuilding to ensure the effective use of national test results by key stakeholders: byteachers as diagnostic tools to assess individual student, student group and class progressand to monitor the impact of different instructional interventions; by municipal educationdirectors and school leaders as a key part of their own quality monitoring systems.Rosenkvist (2010) conducted a detailed review of different uses of student test results inOECD countries and highlights that to bring about positive effects of national studenttests ‘necessitates that schools and teachers have the capacity to interpret and use studenttest results’ (see also Box 7.2).It is critical that the Ministry of Education clarifies the purpose of the proposedpublication of national test results for schools and how this fits into an accountabilitysystem (see Box 7.2). Accountability systems should enable valid decisions that ‘reflectaccurate evaluation of what was intended to be measured’ and ‘justification of theinterpretations and uses (especially consequences formally specified as part of theaccountability system)’ (Perie and Park, 2007). The national tests were originallyconceived for dual purposes: to provide a powerful pedagogical tool to teachers againsttestable areas of the national Common Objectives; to monitor national progress over timevia a ‘national performance profile’ and to allow municipalities to monitor their schoolresults against this. To satisfy both purposes, the government strategy at the time was tokeep the results confidential and for internal management and improvement purposes –with the exception of the publication of a national performance profile. Denmark wouldstrongly benefit from stable and robust national measures of student outcomes that wouldallow the monitoring of changes over time. The national tests represent a significantinvestment and do offer the possibility to track overall progress – at this stage at least inthe public schools – on national measures to complement evidence from internationalstudies, and also importantly, at different stages throughout compulsory education.In particular, the Ministry of Education will need to clarify and clearly communicateto all stakeholders the use of national test results for:•Monitoring improvement:With confidential results, municipalities and schoolsthemselves can use the results for monitoring their outcomes and promotingimprovement – the challenge here would be in developing adequate capacity atthese levels to effectively use these results to this end. If the intention is tomonitor municipal and school improvement, it would be imperative to presentresults showing changes from year to year and in such a way as to allow a faircomparison among schools and municipalities (see Box 7.3).Holding school providers accountable:The publication of results usually aims tointroduce an external accountability to schools and education providers. However,if this is the aim, then the OECD review team is unsure why private schools areallowed to opt out of the national tests. In light of the significant public funding toprivate schools, both public and private schools should be held accountable in thesame way – especially given the increased number of private schools offeringcompulsory education. This argument is even stronger, if the publication of testresults aims to inform parental choice of schools.

Box 7.2 National tests: design, purpose and use of results for accountabilityMost OECD countries use student test results for accountability and improvement, but contexts and testingtraditions vary considerably. For example, Australia recently designed and introduced national tests with the clearpurpose to test basic skills in core areas that all Australian students should acquire and the publication of resultswas to hold all Australian schools accountable (Australian Government Department of Education, Employmentand Workplace Relations, 2011). This is in the context of a culture where States and Territories had offered formany years standardised testing systems and many school districts and schools had already invested in analyticalinformation systems and related training programmes to support educators’ use of test results (Santiagoet al.,2011). Countries that do not have a well-developed tradition for using student test results for improving instructionmay need to enforce and support this practice with some measures of accountability (Rosenkvist, 2010). Inintroducing a system of accountability, it is essential to have a clear understanding of what student tests can (andcannot) be used for: ‘teachers and schools should only be accountable for factors that they can influence. This isimportant for the fairness and legitimacy of the accountability system’ (Rosenkvist, 2010). Perie and Park (2007)conducted a review of accountability systems and related literature in the United States and caution that a welldesigned and effective accountability system pays attention to seven core components:•••••••

Sweden – like Denmark – is a country where comparatively small performancedifferences are observed between schools in the PISA performance results. However,between-school variation in the Swedish PISA results did increase significantly between2000 and 2009 and is now higher than in the other Nordic countries (OECD, 2010d). InSweden, results of national tests are published by municipality (Nuscheet al.,2011).However, in the national reports the major focus is on the national level and by subgroupsof key analytical interest (gender and immigrant background). Descriptive results arepresented at the municipal level in each of the Swedish Education Agency’s (NAE) threemajor annual reports using nine municipal groupings established by the SwedishUmbrella organisation for municipalities (SALAR). At the same time, SALAR alsopublishes its own league tables of municipal results. There is transparent reporting ofstudent results at the school level in both national tests and final school grades in theNAE’s online databases SIRIS (observed school averages) and SALSA (school resultsadjusted with a proxy value added measure). However, the NAE reports that thepublication of these results ‘has attracted very little attention and there has been relativelylittle public debate on the question’ (NAE, 2005b). However, in the event of reportingstudent results at the school level, it is usually considered better practice to also adjustresults for the particular school population (see Box 7.3).OECD REVIEWS OF EVALUATION AND ASSESSMENT IN EDUCATION: DENMARK � OECD 2011

Box 7.3 The proposed publication of national test results for schools:some reporting considerationsIf Denmark plans to publish the national test results by municipality and school for accountability purposes, itshould examine various approaches to reporting used in other countries that try to maximise the fairness in schoolcomparisons. Compared to other countries, Denmark has the major performance differences within schools andresults from national test pilots confirm the huge performance differences among students in each Form (e.g.OECD 2010c; Wandall, 2010). Differentiating instruction within classes and schools and ensuring adequateprogress of individual students is, therefore, rightly identified as the major challenge in theFolkeskole.However,there are still some performance differences observed between schools (e.g. OECD, 2010d) and in attempting todraw conclusions on the extent to which these reflect differences in the effectiveness of teaching and schoolpolicies, it is important to consider characteristics of the student population.

Focus reporting on progressThe reporting of results should capitalise on the ability of national tests to show changes over time. Denmarkshould consider ways to present school results with a focus on school-level gains or losses on the existing suite ofnational tests. The items in the national tests could be linked across Forms in different content areas to showstudent progress in a given measure across Forms within the school – currently for Danish and mathematics andfor other subjects if further tests are developed. This could be reported against the national average progress ofstudents in each content area across different Forms. This would have the advantage also of promoting educatortake up of the possibility to re-administer the national tests a maximum of two more times for each student. InAustralia (see Santiagoet al.,2011), the National Assessment Plan – Literacy and Numeracy (NAPLAN) testsinclude items linked on a common scale to allow documentation of student progression in each of the core areas(reading, writing, language conventions [spelling, grammar and punctuation]) across the four key educationalstages that each student sits the test (Years 3, 5, 7 and 9). In this way, it is possible to gauge student progress in thenational tests on a subsequent year, for example, it will be possible to see how well a student performs on thecommon NAPLAN reading scale at four different stages of his or her schooling (in Years 3, 5, 7 and 9). Resultsfrom 2010 on will be aggregated to show progress at the State and Territory and national levels.

Consider the progress achieved by a school for its particular student populationIn England, schools are expected to meet targets for student expected progress between specified key stages ofschooling (see Rosenkvist, 2010). Such progress measures are complemented by a statistical indicator of‘Contextual Value Added (CVA) score’. Such scores show the progress made by students from the end of a keystage to the end of another key stage using their test results. CVA takes into account the varying starting points ofeach students’ test results, and also adjusts for factors which are outside a school’s control (such as gender,mobility and levels of deprivation) that have been observed to impact on student results. Several systems in theUnited States also attempt to measure ‘adequate yearly growth’. Various models have been researched and used inpractice. In value-added models, students’ actual test scores are often compared to the projected scores, andclassroom and school scores that exceed the projected values are considered as positive evidence of instructionaleffectiveness. In this way, value-added models can be used to identify teachers and schools that have met aboveexpected growth despite various challenging circumstances. It is important to note that value-added models arestill under development, and therefore they are prone to error (Koretz, 2008).

Strengthen efforts to both monitor and promote municipal evaluation capacityDenmark has made considerable progress in establishing a quality monitoring system:municipalities draw up quality reports based on information submitted to them byschools; the Quality and Supervision Agency monitors these and major indicators ofschool performance and checks and evaluates whether municipalities follow upadequately schools where there are quality concerns. While it is important that theQuality and Supervision Agency in its monitoring capacity continues to ensure aminimum quality and conduct a basic risk assessment of all schools, it is also of keyimportance to identify municipalities where real progress is being made in studentoutcomes. The Quality and Supervision Agency should be in a position to identifymunicipalities that are producing and sustaining improved student performance and to beable to learn from these examples and share this knowledge throughout the system. Insubsequent years, one helpful indicator will be student progress as measured in thenational tests and it will be important for the Quality and Supervision Agency to invest inefficient systems to report and analyse this to feed results into their monitoring ofmunicipalities. As identified above, it would be useful to complement performanceinformation with data on the teaching and learning environment.Given the pivotal role that municipalities play in validating curriculum against theCommon Objectives and the basis that such objectives form for the evaluation culture inDenmark, it would be of significant interest for the Quality and Supervision Agency toexplore ways to evaluate the adequacy and effectiveness of such municipal curriculumplans and follow up. Especially in the light of the OECD review team and the ‘FlyingSquad’ recommendations (see Box 2.1) to further clarify the national CommonObjectives. There may be room to draw on both EVA’s capacity, the content expertise ofeducators and KL’s partnership experience.Similarly, the Quality and Supervision Agency could work with KL to buildmunicipal monitoring capacity and effective use of national test results and otherperformance indicators. Results from the School Council’s study of top performingschools revealed the importance of clearly formulated objectives and performancemanagement at the municipal level with strong school leadership (Mehlbye, 2010):school goals are formulated in quality reports at the municipal and school levels and themunicipal officers conduct close and continuous dialogue with the schools on their work.Great progress has already been made around the implementation of quality reports inengaging municipal officials in information meetings at the former School Agency and insharing municipal experiences with establishing new quality systems on theEvaluationportal.The KL partnership revealed positive effects of horizontal collaboration andknowledge share at the political levels with respect to evaluation issues and in generalmany participants felt like they were ‘part of something bigger’ which gave both leverageand motivation to go further with their monitoring and evaluation systems. The Qualityand Supervision Agency and KL could design ways to further stimulate suchcollaborations. The KL partnership (KL, 2009) made use of a ‘status analysis’ tool – aquestionnaire administered to school principals, teachers, parents and students at both thestart and end of the partnership, including concrete measures of reading progress andstakeholder reports on the use of targets, goals and related discussions – and identified thefollowing effects: put greater focus and follow up on results – both easy and hard-to-measure results; made municipal quality reports operational; strengthened qualitydevelopment; and provided profiles of schools that were not so well known on existingmanagement areas.

128– 7. SYSTEM EVALUATIONAustralia presents an example of the effective use of national monitoring results byschool providers to evaluate and manage their school system performance (Santiagoet al.,2011). There is both a clear focus on national goals and monitoring of progresstowards achieving these and a focus on locally relevant priorities and the monitoring ofthese – often including broader information collected locally. All State and Territorygovernment departments – the equivalent role to municipalities in Denmark as schoolproviders – produce an annual report on major activities, including both financial andperformance information. In performance reporting, the major focus is on performanceoutcomes for the government school sector, although the reports also usually includeminimal reporting on the non-government school sectors (e.g. enrolment figures, newschools registered, proportion of schools meeting agreed requirements). A commonfeature in the 2009/10 government reports is the prominence of data from the newnational literacy and numeracy tests in the performance monitoring. The exact format forreporting of national test results varies according to the emphasis on different monitoringgoals for each of the providers and demonstrates the way that the same data results can beused to monitor different local level goals.

ReferencesAbedi, J. (2004), “Will You Explain the Question?”,Principal Leadership,Vol. 4, No. 7,pp. 27–31.Abedi, J., C.H. Hofstetter and C. Lord (2004), “Assessment Accommodations for EnglishLanguage Learners: Implications for Policy-Based Empirical Research”,Review ofEducational Research,Vol. 74, No. 1, pp. 1–28.Absolum, M., L. Flockton, J. Hattie, R. Hipkins and I. Reid (2009),Directions forAssessment in New Zealand (DANZ) Report: Developing Students’ AssessmentCapabilities,Ministry of Education, Wellington.AKF (2011),PISA Etnisk 2009 – Etniske og danske unges resultater i PISA 2009,Anvendt KommunalForskning (AKF), Copenhagen, available at:www.akf.dk.Allington, R.L. and P.M. Cunningham (2002),Schools That Work: Where All ChildrenRead and Write,Boston, MA: Allyn and Bacon.American Federation of Teachers, National Council on Measurement in Education &National Education Association (1990), “Standards for Teacher Competence inEducational Assessment of Students”,Educational Measurement: Issues and Practice,Vol. 9, pp. 30-32.Anderson, J.R., J.G. Greeno, L.M. Reder and H.A. Simon (2000), “Perspectives onLearning, Thinking, and Activity”,Educational Researcher,Vol. 29, No. 4, pp. 11-13.Andrade, H. (2005), “Teaching with Rubrics: The Good, the Bad, and the Ugly”,CollegeTeaching,Vol. 53, No. 1, pp. 27-30.Assessment Reform Group (2002),Assessment is for Learning: 10 Principles,Assessment Reform Group,www.assessment-reform-group.org/CIE3.PDF.Australian Government Department of Education, Employment and Workplace Relations(2011),Country Background Report for Australia,prepared for theOECD Review onEvaluation and Assessment Frameworks for Improving School Outcomes,Canberra,available fromwww.oecd.org/edu/evaluationpolicy.Baker, E.L. and R.L. Linn (2000),Alignment: Policy Goals, Policy Strategies, and PolicyOutcomes,The CRESST Line, UCLA, Los Angeles.Barber, M. (2009), “From System Effectiveness to System Improvement”, inA. Hargreaves and M. Fullan (eds.),Change Wars,Solution Tree, Bloomington, IN,71-94.Barber, M. and S. Mourshed (2007),How the World’s Best Performing School SystemsCome out on Top,McKinsey & Company.

130– REFERENCESBarber, M., F. Whelan and M. Clarke (2010),Capturing the Leadership Premium: Howthe World’s Top School Systems are Building Leadership Capacity for the Future,McKinsey & Co. with the National College for Leadership of Schools and Children’sServices, England.Black, B., C. Harrison, C. Lee, B. Marshall and D. Wiliam (2004), “Working Inside theBlack Box: Assessment for Learning in the Classroom”,The Phi Delta Kappan,Vol. 86, No. 1, pp. 8-21.Black, P. and D. Wiliam (1998), “Assessment and Classroom Learning”,Assessment inEducation,Vol. 5, No. 1, pp. 7-74.Christensen, A.B., A. Birkebeak, H. Pedersen, T.D. Petersen, C. Hjortdal and J. Faerk (2007),“How we Move On”,Working Paper – Pupils, Parents, Social Pedagogues, Teachersand Leaders Working Together to Meet the Challenges Facing Public Basic Schools,Copenhagen, Denmark.Chudowski, N. and J.W. Pellegrino (2003), “Large-Scale Assessments that SupportLearning: What Will it Take?”Theory into Practice, 42(1),75-83.Collins, J. (2001),Good to Great,Random House Business Books, London.Dahler-Larsen, P. (2006),Evaueringskultur:Universitetsforlag, Odense.Etbegrebbliverti,Syddansk

Danielson, C. (1996, 2007),Enhancing Professional Practice: A Framework forTeaching,1stand 2ndeditions, Association for Supervision and CurriculumDevelopment (ASCD), Alexandria, Virginia.Danielson, C. (2001), “New Trends in Teacher Evaluation”,Educational Leadership,Vol. 58, No. 5, pp. 12-15.Danish Government (2010),Denmark 2020 – Knowledge > Growth > Prosperity >Welfare,The Danish Government, Copenhagen,www.stm.dk/publikationer/arbprog_10_uk/Denmark_2020_knowledge_growth_prosperity_welfare.pdf.Danish Ministry of Economic and Business Affairs (2009),Denmark in the GlobalEconomy – Competitiveness Report 2009,Danish Ministry of Economic and BusinessAffairs, Copenhagen.Danish Ministry of Education (2010),Facts and Figures 2009 – Key Figures inEducation 2009,Statistical Publication No. 2 – 2010, Danish Ministry of Education.Danish School Agency (2009), “Quality Reports as a Municipal Management Tool”,Article by J. Hess, L. Weinreich Jakobsen and H. Holmsgaard, Office of QualityAssurance, Danish School Agency.Danish School Agency (2010),The Folkeskole of the Future: One of the Best in theWorld: Recommendations,360 Degree Review of theFolkeskoleperformed by theFlying Squad of theFolkeskole,Danish School Agency, Copenhagen.Earl, L. and S. Katz (2008), “Getting to the Core of Learning: Using Assessment for Self-Monitoring and Self-Regulation”, in S. Swaffield (2008),Unlocking Assessment:Understanding for Reflection and Application,Routledge/Taylor and Francis, London.Egelund, N. (2005), “Educational Assessment in Danish Schools”,Assessment inEducation: Principles, Policy & Practice,Vol. 12, No. 2, pp. 203-212.OECD REVIEWS OF EVALUATION AND ASSESSMENT IN EDUCATION: DENMARK � OECD 2011

Egelund, N. and T. Tranæs (2008),PISA Etnisk 2005 – Kompetencer hos danske ogetniske elever I 9. Klasse i Danmark 2005,Rockwool Fonden.EPPI (2002),A Systematic Review of the Impact of Summative Assessment and Tests onStudents’ Motivation for Learning,Review conducted by the Assessment andLearning Synthesis Group, Evidence for Policy and Practice Information andCo-ordinating Centre, London.EVA (2005),Kommunernes kvalitetssikring af folkeskolen,Danmarks Evalueringsinstitut,Copenhagen.EVA (2007),Kvalitetsarbejde: erfaringer fra 20 skoler,Danmarks Evalueringsinstitut,Copenhagen.EVA (2008),Arbejdet med elevplaner,Danmarks Evalueringsinstitut, København.EVA (2009a),Særlige ressourcepersoner i folkeskolen,Danmarks Evalueringsinstitut,Copenhagen.EVA (2009b),Kommunernes arbejde med kvalitetsrapporter,Danmarks Evalueringsinstitut,Copenhagen.EVA (2010),ICSEI 2010: Educational Evaluation,Danmarks Evalueringsinstitut,Copenhagen.Faubert, V. (2009), “School Evaluation: Current Practices in OECD Countries and aLiterature Review”,OECD Education Working Papers,No. 42, OECD Publishing.Finnish Department for Education and Science Policy (forthcoming),OECD Review onEvaluation and Assessment Frameworks for Improving School Outcomes: CountryBackground Report Finland.Gilmore, A. (2008),Assessment Review Paper 8: Professional Learning in Assessment,Ministry of Education, Wellington.Gipps, C., B. McCallum and M. Brown (1997), “Models of Teacher Assessment amongPrimary School Teachers in England”,The Curriculum Journal,Vol. 7, pp. 167-183.Greeno, J.G., A.M. Collins and L.B. Resnick (1996), “Cognition and Learning”, in D.C.Berliner and R.C. Calfee (eds.),Handbook of Educational Psychology,Macmillan,New York, pp. 15-16.Haertel, E. (1999), “Performance Assessment and Education Reform”,Phi Delta Kappan,80, pp. 662-666.Hamilton, L., R. Halverson, S. Jackson, E. Mandinach, J. Supovitz and J. Wayman (2009),Using Student Achievement Data to Support Instructional Decision Making(NCEE2009-4067), Washington, DC: National Center for Education Evaluation and RegionalAssistance, Institute of Education Sciences, U.S. Department of Education,http://ies.ed.gov/ncee/wwc/publications/practiceguides/.Hanushek, E.A. and M.E. Raymond (2005), “Does School Accountability Lead toImproved Student Performance?”Journal of Policy Analysis and Management,24(2).Hargreaves, A. and M. Fullan (1998),What’s Worth Fighting for Out There?,PublicSchool Teachers’ Federation, Mississauga, Ontario.

132– REFERENCESHarvey-Beavis, O. (2003), “Performance-Based Rewards for Teachers: A LiteratureReview”, paper distributed at the third workshop of participating countries on OECDActivity “Attracting, Developing and Retaining Effective Teachers”, 4-5 June, Athens,Greece, available fromwww.oecd.org/edu/teacherpolicy.Heneman, H., A. Milanowski and S. Kimball (2007),Teacher Performance Pay:Synthesis of Plans, Research, and Guidelines for Practice,Consortium for PolicyResearch in Education (CPRE) Policy Briefs RB-46.Heritage, M., J. Kim, T. Vendlinski and J. Herman (2009), “From Evidence to Action: ASeamless Process in Formative Assessment?”Educational Measurement: Issues andPractice,Vol. 28, No. 3, 24-31.Hill, R. and P. Matthews (2010),Schools Leading Schools II: The Growing Impact ofNational Leaders of Education,National College for Leadership of Schools andChildren’s Services, Nottingham, England.Hong Kong Education and Manpower Bureau (2003),Enhancing School Developmentand Accountability through School Self Evaluation and External Self Review,CircularNo. 23/2003,www.edb.gov.hk/FileManager/EN/Content_6470/circular0612_eng.pdf.Hovde, K. and S.B. Mahfooz (2010), “Supervision and Support of Primary andSecondary Education: A five-Country Comparison”,Knowledge Brief2010, 33, TheWorld Bank.IEA (2008a),TIMSS 2007 International Mathematics Report: Findings from IEA’sTrends in International Mathematics and Science Study at the Fourth and EighthGrades,TIMSS & PIRLS International Study Center, Lynch School of Education,Boston College.IEA (2008b),TIMSS 2007 International Science Report: Findings from IEA’s Trends inInternational Mathematics and Science Study at the Fourth and Eighth Grades,TIMSS & PIRLS International Study Center, Lynch School of Education, BostonCollege.IEA (2010),Initial Findings from the IEA International Civic and Citizenship EducationStudy,International Association for the Evaluation of Educational Achievement,Amsterdam,http://iccs.acer.edu.au/uploads/File/Reports/ICCS_10_Initial_Findings.pdf.Isoré, M. (2009), “Teacher Evaluation: Current Practices in OECD Countries and aLiterature Review”,OECD Education Working Papers,No. 23, OECD, Paris,available fromwww.oecd.org/edu/workingpapers.Jonsson, A. and G. Svingby (2007), “The Use of Scoring Rubrics: Reliability, Validityand Educational Consequences”,Educational Research Review,2, pp. 130-144.Jordan, A., L. Lindsay and P. Stanovich (1997), “Classroom Teachers’ InstructionalInteractions with Students who are Exceptional, at Risk, and Typically Achieving”,Remedial and Special Education, 18(2),82-93.KL (2009),Partnerskab om Folkeskolen – Kort og godt,Kommuneforlaget A/S,www.kl.dk/ImageVault/Images/id_41939/ImageVaultHandler.aspx.Københavns Kommune (2010),Kvalitetsrapport 2010 – Teknisk bilag,Børne- ogUngdomsforvaltningen, Københavns Kommune, Københavns.

Koretz, D. (2005),Alignment, High Stakes, and the Inflation of Test Scores,University ofCalifornia, National Center for Research on Evaluation, Standards, and StudentTesting (CRESST), Los Angeles.Koretz, D. (2008), “A Measured Approach: Maximizing the Promise, and Minimizing thePitfalls of Value-Added Models”,American Educator,Vol. 32, No. 2, pp. 18-39.Kotter, J. (1996),Leading Change,Harvard Business School Press.Lave, J. and E. Wenger (1991),Situated Learning: Legitimate Peripheral Participation,Cambridge, England.Leithwood, K., C. Day, P. Sammons, A. Harris and D. Hopkins (2006),Seven StrongClaims about Effective School Leadership,The National College for Leadership ofSchools and Children’s Services, Nottingham, England.Levitt, R., B. Janta and K. Wegrich (2008),Accountability for Teachers: A LiteratureReview,Rand Cooperation, Santa Monica, CA.Madaus G.F. and T.K. Kellaghan (1993), “The British Experience with ‘Authentic’Testing”,The Phi Delta Kappan,Vol. 74, No. 6, pp. 458-469.Madaus, G.F. and L.M. O’Dwyer (1999), “A Short History of Performance Assessment:Lessons Learned”,The Phi Delta Kappan,Vol. 80, No. 9, pp. 688-695.Margo, J., M. Benton, K. Withers and S. Sodha (2008),Those Who Can?,Institute forPublic Policy Research (IPPR) Publications.Matthews, P. and P. Sammons (2005), “Survival of the Weakest? The DifferentialImprovement of Schools Causing Concern in England”,London Review of Education,Vol. 3, No. 2, pp. 159-176.McDowall, S., M. Cameron, R. Dingle, A. Gilmore and L. MacGibbon (2007),Evaluation of the Literacy Professional Development Project,New Zealand Ministryof Education, Wellington.MCEECDYA (2010),National Assessment Program – ICT Literacy Years 6 & 10 Report– 2008,Ministerial Council for Education, Early Childhood Development and YouthAffairs,www.mceecdya.edu.au/verve/_resources/NAP-ICTL_2008_report.pdf.Mehlbye, J. (2010),Den højt præsterende skole – Hvordan kan skolen løfte elever medsvag social baggrund,AKF, Anvendt KommunalForskning, Copenhagen.Messick, S. (1989), “Validity”, in R. Linn (ed.),Educational Measurement,3rdEdition,American Council on Education, Washington D.C.Milanowski, A. and S. Kimball (2003), “The Framework-Based Teacher PerformanceAssessment Systems in Cincinnati and Washoe”,CPRE Working Paper SeriesTC-03-07.Militello, M., J. Schweid and S.G. Sireci (2010), “Formative Assessment Systems:Evaluating the Fit between School Districts’ Needs and Assessment Systems’Characteristics”,Educational Assessment, Evaluation and Accountability,22(1),29–52,www.springerlink.com/content/l77872u56713t736/.Moos, L., J. Krejsler and K. Kofod (2008), “Successful Principals: Telling or Selling? Onthe Importance of Context for School Leadership”,International Journal ofLeadership in Education,Vol. 11, No. 4, pp. 341-352.

134– REFERENCESNAE (2005a),Educational Inspection 2004: Summary of Inspection results,SwedishNational Agency for Education (Skolverket), Stockholm.NAE (2005b),National Assessment and Grading in the Swedish School System,NationalAgency for Education (Skolverket), Stockholm.New Zealand Ministry of Education (forthcoming),OECD Review on Evaluation andAssessment Frameworks for Improving School Outcomes: Country BackgroundReport New Zealand,available atwww.oecd.org/edu/evaluationpolicy.Ng, P.T. (2007), “Quality Assurance in the Singapore Education System in an Era ofDiversity and Innovation”,Education Research Policy and Practice,6, 235-247.Norwegian Directorate for Education and Training (2011),OECD Review on Evaluationand Assessment Frameworks for Improving School Outcomes: Country BackgroundReport Norway,available atwww.oecd.org/dataoecd/7/10/47088605.pdf.Nusche, D., G. Wurzburg and B.Naughton (2010),OECD Reviews of Migrant Education:Denmark,OECD, Paris,www.oecd.org/dataoecd/54/17/44855206.pdf.Nusche, D., G. Hálasz, J. Looney, P. Santiago and C. Shewbridge (2011),OECD Reviewsof Evaluation and Assessment in Education: Sweden,OECD, Paris,www.oecd.org/dataoecd/38/42/47169533.pdf.Nusche, D., L. Earl, W. Maxwell and C. Shewbridge (forthcoming),OECD Reviews ofEvaluation and Assessment in Education: Norway,OECD, Paris.Odense Kommune (2010),KIS Kvalitet i Skolerne – En guide til arbejdet med evalueringog kvalitetsudvikling på skoleområdet I Odense Kommune,www.odense.dk/Topmenu/Borger/Uddannelse/Folkeskole/~/media/BUF/BornOgUngeforvaltningen/Skoleafdelingen/KIS_guide.ashx.OECD (2004a),Review of National Policies for Education. Denmark: Lessons from PISA2000,OECD, Paris.OECD (2004b),Learning for Tomorrow’s World – First Results from PISA 2003,OECD,Paris.OECD (2005a),Formative Assessment: Improving Learning in Secondary Classrooms,OECD, Paris.OECD (2005b),Teachers Matter: Attracting, Developing and Retaining EffectiveTeachers,OECD, Paris.OECD (2006),Where Immigrant Students Succeed – A Comparative Review ofPerformance and Engagement in PISA 2003,OECD, Paris.OECD (2009a),OECD Review on Evaluation and Assessment Framework for ImprovingSchool Outcomes: Design and Implementation Plan for the Review,OECD, Paris,available fromwww.oecd.org/edu/evaluationpolicy.OECD (2009b),Economic Survey of Denmark 2009,OECD, Paris.OECD (2009c),Government at a Glance 2009,OECD, Paris.OECD (2009d),Creating Effective Teaching and Learning Environments: First Resultsfrom TALIS,OECD, Paris.OECD (2009e),Learning Mathematics for Life: A Perspective from PISA,OECD, Paris.

OECD (2010a),Education at a Glance 2010,OECD, Paris.OECD (2010b),PISA 2009 Results: What Students Know and Can Do – StudentPerformance in Reading, Mathematics and Science (Volume I),OECD, Paris.OECD (2010c),PISA 2009 Results: Learning Trends: Changes in Student PerformanceSince 2000 (Volume V),OECD, Paris.OECD (2010d),PISA 2009 Results: Overcoming Social Background – Equity in LearningOpportunities and Outcomes (Volume II),OECD, Paris.OECD (2010e),Closing the Gap for Immigrant Students: Policies, Practice andPerformance,OECD, Paris.OECD (2010f), PISA Computer-Based Assessment of Student Skills in Science, OECD,Paris.http://browse.oecdbookshop.org/oecd/pdfs/browseit/9810041E.PDF.OECD (2011),Economic Policy Reforms 2011 – Going for Growth,OECD, Paris.Office for Standards in Education (2006), “The Logical Chain: Continuing ProfessionalDevelopment in Effective Schools”,OFSTED PublicationsNo. 2639, UnitedKingdom.Office for Standards in Education (2009) (Ed. P. Matthews),Twelve OutstandingSecondary Schools: Excelling against the Odds; Twenty Outstanding PrimarySchools: Excelling against the Odds; Fifteen Outstanding Special Schools: Inclusionagainst the Odds,Office for Standards in Education and Children’s Services andSkills, England.Ontario Ministry of Education (2010),Growing Success – Assessment, Evaluation, andReporting in Ontario Schools 2010: First Edition, Covering Grades 1 to 12,OntarioMinistry of Education, Toronto,www.edu.gov.on.ca/eng/policyfunding/growSuccess.pdf.Pellegrino J., N. Chudowsky and R. Glaser (2001),Knowing What Students Know: TheScience and Design of Educational Assessment,National Academies Press,Washington D.C.Perie, M. (2008), “A Guide to Understanding and Developing Performance-LevelDescriptors, National Council on Measurement in Education”,EducationalMeasurement: Issues and Practice,Vol. 27, Issue 4.Perie, M. and J. Park (2007),Key Elements for Educational Accountability Models– Apaper commissioned by the Council of Chief State School Officers AccountabilitySystems and Reporting State Collaborative, The Council of Chief State SchoolOfficers, Washington D.C.Pont B., D. Nusche and H. Moorman (2008),Improving School Leadership – Volume 1:Policy and Practice,OECD, Paris.Popham, W.J. (2008),Transformative Assessment,Association for Supervision andCurriculum Development, Alexandria, VA.Popham, W.J. (1997), “What’s Wrong and What’s Right with Rubrics”,EducationalLeadership,Vol. 55, pp. 72-75.Poskitt, J. and K. Taylor (2008),National Education Findings of Assess to Learn (AtoL)Report,Education Group – Report, Palmerston North.

136– REFERENCESRambøll (2011),Country Background Report for Denmark,prepared for theOECDReview on Evaluation and Assessment Frameworks for Improving School Outcomes,Aarhus, available fromwww.oecd.org/edu/evaluationpolicy.Regeringen (2010),Faglighed og frihed – Regeringens udspil til en bedre folkeskole,Regeringen December 2010:40, Copenhagen,www.uvm.dk/~/media/Files/Udd/Folke/PDF10/101208_Folkeskolereform_web.ashx.Robinson, V. (2007) “School Leadership and Student Outcomes: Identifying What Worksand Why”, Australian Council for Educational Leaders,ACEL Monograph SeriesNo. 41.Robinson, V.M.J., C. Lloyd and K.J. Rowe (2008), “The Impact of Leadership on StudentOutcomes: An Analysis of the Differential Effects of Leadership Type”,EducationAdministration Quarterly,44(5), 635-674.Rosenkvist, M.A. (2010), “Using Student Test Results for Accountability andImprovement: A Literature Review”,OECD Education Working Papers,No. 54,OECD Publishing, doi:http://dx.doi.org/10.1787/5km4htwzbv30-en.Ross, J.A., C. Rolheiser and A. Hogaboam-Gray (1999), “Effects of Self-EvaluationTraining on Narrative Writing”,Assessing Writing,Vol. 6, pp. 107-132.Sammons, P., J. Hillman and P. Mortimore (1995),Key Characteristics of EffectiveSchools: A Review of School Effectiveness Research,Institute of Education, Londonand the Office for Standards in Education.Santiago, P. and F. Benavides (2009),Teacher Evaluation: A Conceptual Framework andExamples of Country Practices,paper presented at the OECD-Mexico Workshop‘Towards a Teacher Evaluation Framework in Mexico: International Practices,Criteria and Mechanisms, Mexico City 1-2 December, OECD, Paris, available fromwww.oecd.org/edu/evaluationpolicy.Santiago, P., D. Roseveare, G. van Amelsvoort, J. Manzi and P. Matthews (2009),Teacher Evaluation in Portugal: OECD Review,OECD, Paris.Santiago, P., G. Donaldson, J. Herman and C. Shewbridge (forthcoming),OECD Reviewsof Evaluation and Assessment in Education: Australia,OECD, Paris.Schargel, F.P., T. Thacker and J. Bell (2007),From At-Risk to Academic Excellence:What Successful Leaders Do,Eye on Education, Larchmont, NY.Shepard, L.A.,et al.(1996), “Effects of Introducing Classroom Performance Assessmenton Student Learning”,Educational Measurement Issues and Practice,Vol. 15,pp. 7-18.Skolerådet (2008),Beretning om Evaluering og Kvalitetsudvikling af Folkeskolen 2008,Formandskabet for Rådet for Evaluering og Kvalitetsudvikling af Folkeskolen,Copenhagen.Skolerådet (2009),Beretning om Evaluering og Kvalitetsudvikling af Folkeskolen 2009,Formandskabet for Rådet for Evaluering og Kvalitetsudvikling af Folkeskolen,Copenhagen.Skolerådet (2010),Beretning om Evaluering og Kvalitetsudvikling af Folkeskolen 2010,Formandskabet for Rådet for Evaluering og Kvalitetsudvikling af Folkeskolen,Copenhagen.

Skolerådet (n.d.),Folkeskolen 2020 – De 10 vigtigste udfordringer,Skolerådet,www.skoleraadet.dk/~/media/Raadet/VidenOm/Beretninger/2010/Folkeskolen2020.ashx.Skolestyrelsen(2009),TALIS,Laerereogskoleledereomundervisning,kompetenecudvikling og evaluering – i et internationalt perspektiv,af Eglund, N.,Danmarks Pedagogiske Universitetsskole, Aarhus Universitet, (Red) Rotbøll, C.,Skolestyrelsen, Copenhagen.Skolestyrelsen (2010),Fremtidens folkeskole – Én af verdens bedste: anbefalinger -360-graders-eftersyn af folkeskolen gennemført af Skolens rejsehold– Juni 2010, Styrelsenfor Evaluering og kvalitetsudvikling af Folkeskolen (Skolestyrelsen) for Skolensrejsehold,www.skolensrejsehold.dk/skolens%20rejsehold/~/media/SkolensRejsehold/Rapporter/RapportA_enkeltsider.ashx.Southworth, G. (2009), “Learning-centred Leadership” in B. Davies (ed.),The Essentialsof School Leadership,Sage, London, California, New Delhi.Stiggins, R.J. (1987), “Design and Development of Performance Assessments”,Education Measurement: Issues and Practice,Vol. 6 No. 3, pp. 33-42.Stiggins, R.J. (1991), “Assessment Literacy”,The Phi Delta Kappan,Vol. 72, No. 7,pp. 534-539.Stiggins, R.J. (1995), “Assessment Literacy for the 21st Century”,The Phi Delta Kappan,Vol. 77, No. 3, pp. 238-245.Stiggins, R. (2005),From Formative Assessment to Assessment for Learning: A Path toSuccess in Standards-based Schools,Phi Delta Kappan, 87(4), 324-328.TDA (Training and Development Agency for Schools, 2007),Professional Standards forTeachers,The Training and Development Agency for Teachers, London,www.tda.gov.uk/upload/resources/pdf/s/standards_a4.pdf.Timperley, H.,et al.(2007),Teacher Professional Learning and Development BestEvidence Synthesis(report to the New Zealand Ministry of Education), New ZealandGovernment, Wellington.Van Hoof, J. and P. Van Petegem (in press), “Designing and Evaluating the Process ofSchool Self-evaluations”,Improving Schools.Wandall, J. (2010),National Tests in Denmark: CAT as a Pedagogical Tool,DanishNational School Agency, Copenhagen.Wayman, J. and S. Stringfield (2006), “Data Use for School Improvement: SchoolPractices and Research Perspectives,American Journal of Education,112(4),463-468.World Economic Forum (2009),The Global Information Technology Report 2008-2009,S. Dutta, INSEAD, and I. Mia (eds.), World Economic Forum, Geneva.

Annex 1: Visit ItineraryTuesday 5 October09.00-10.45Ministry of Education11.00-12.00The Danish School Agency12.00-13.00UNI-C13.00-14.00Working lunch with Rambøll – authors of the Country Background Report14.00-15.00Group of teacher educators15.00-16.00School and Parents Association16.15-17.15Municipal children and culture authority – HerlevWednesday 6 October09.00-09.30Ministry of Economic and Business Affairs – the ‘Competitiveness Report’09.30-10.00Audit of State Accounts10.00-11.00The Danish Evaluation Institute (EVA)11.00-12.00Private Schools Organisations12.00-13.00The Confederation of Danish Employers14.00-16.30School visit 1 – Bordings Friskole17.50-21.00Travel to HedenstedThursday 7 October08.30-09.30Municipal educational authority – Hedensted10.00-12.30School visit 2 – Stjernevejskolen13.00-16.00School visit 3 – Stoubyskole16.43-18.00Travel to OdenseFriday 8 October08.30-11.00School visit 4 – Sct. Hans Skole11.30-12.30Municipal educational authority – Odense13.07-15.00Travel to Copenhagen15.15-16.00The Association of Danish StudentsMonday 11 October09.00-09.45Minister of Education10.30-13.00School visit 5 – Skovvangsskolen in Glostrup13.30-16.00School visit 6 – Katrinedals Skole in Vanløse16.30-17.30Municipal educational authority – CopenhagenTuesday 12 October09.00-09.45The Danish Association of School Leaders09.45-10.45Local Government Denmark11.00-12.00The Danish Union of Teachers12.00-14.00Research seminar14.00-15.00The Chairmanship of the Council for Evaluation and Quality Development ofPrimary and Lower Secondary Education15.00-16.00Representatives from the “360 degrees review of theFolkeskole”‘Flying Squad’16.00-17.00Final meeting (Ministry of Education and the Danish School Agency)OECD REVIEWS OF EVALUATION AND ASSESSMENT IN EDUCATION: DENMARK � OECD 2011

Eunice Jang,a Korean national, is an Associate Professor at the Department ofCurriculum, Teaching and Learning in the Ontario Institute for Studies in Education,University of Toronto. Her research interests lie at the intersection between educationalassessment and educational and social program evaluation. Her research seeks toadvance practices through alternative assessment approaches that promote students’self-regulated learning while informing instructional planning. She has collaboratedwith various stakeholders on a large-scale study that examined multidimensional factorsinfluencing school improvement despite challenging circumstances. She currently leadsa longitudinal evaluation study that examines the feasibility of a new performance-based classroom assessment system developed by the Ontario Ministry of Educationand teachers. She also examines the role of community support workers in inner-cityschools. She has served on the Advisory Board of the Education Quality andAccountability Office (EQAO) provincial literacy and numeracy assessments as well asthe International Language Testing Association Nominating Committee and theAmerican Educational Research Association (AERA) Significant ResearchContribution Award Committee.Peter Matthews,a British national, is an Education Consultant and VisitingProfessorial Fellow at the Institute of Education, University of London. His researchinterests include school and system leadership, evaluation and improvement. Hespecialises in the evaluation of national policies for schools and works mainly forgovernments or national organisations in the UK and internationally. In England, he isadviser to the National College of School Leadership on the appointment andeffectiveness of National Leaders of Education, and is contributing to the revision ofthe national professional qualification for headteachers. He is also a consultant to theOffice for Standards in Education (Ofsted), leading work on schools in challengingcircumstances. Peter was previously a senior civil servant, Her Majesty’s Inspector andHead of School Inspections in Ofsted from 1993 to 2003 where he led the developmentand operation of the school inspection system in England from its inception. He hasalso been a Chief Adviser and deputy chief officer in local government and is PastPresident of the National Association of Educational Inspectors, Advisers andConsultants and the Society of Chief Inspectors and Advisers. He has chaired nationalcommittees ranging from the introduction of autonomous schools to science education,and has worked in schools and teacher education. His publications include research inboth science and education. In 2003, he received a State honour for his contribution toeducation.

Paulo Santiago,a Portuguese national, is a Senior Analyst in the OECD Directoratefor Education, where he has been since 2000. He is currently the co-ordinator of theOECD Review on Evaluation and Assessment Frameworks for Improving SchoolOutcomes. He has previously assumed responsibility for two major cross-countryreviews, each with the participation of over twenty countries: a review of teacher policy(between 2002 and 2005, leading to the OECD publication “Teachers Matter”) and thethematic review of tertiary education (between 2005 and 2008, leading to the OECDpublication “Tertiary Education for the Knowledge Society”). He has also led reviewsof teacher policy and tertiary education policy in several countries. He holds a PhD inEconomics from Northwestern University, United States, where he also lectured. Witha background in the economics of education, he specialises in education policy analysis.Claire Shewbridge,a British national, is an Analyst in the OECD Directorate forEducation and is currently working for the OECD Review on Evaluation andAssessment Frameworks for Improving School Outcomes. She most recently worked onthe OECD Review on Migrant Education working on country-specific analysis for theNetherlands, Austria and Norway and co-authored the OECD report “Closing the Gapfor Immigrant Students” (2010). For five years, Claire co-ordinated the PISA thematicreport series. She also led analysis of student attitudes towards science learning and theenvironment in the PISA 2006 survey. Her earlier statistical work with the OECDincluded educational enrolment, graduation and financial statistics published inEducation at a Glance, labour force survey statistics published in the OECDEmployment Outlook and financial statistics in the OECD’s Development AssistanceCommittee. She co-ordinated the review of Denmark and acted as Rapporteur for theOECD review team.

DenmarkEDUCATIONAL ATTAINMENTSource: Education at a Glance (OECD, 2010a)3% of population that has attained at least upper secondary education, by age group(excluding ISCED 3C short programmes)4(2008)Ages 25-64Ages 25-34Ages 35-44Ages 45-54Ages 55-64% of population that has attained tertiary education, by age group(2008)Ages 25-64Ages 25-34Ages 35-44Ages 45-54Ages 55-64Upper secondary graduation rates(2008)% of upper secondary graduates (first-time graduation) to the population at the typicalage of graduationSTUDENT PERFORMANCEMean performance in PISA(Programme for International Student Assessment)(15-year-olds) (2006) Source: PISA 2009 Results (OECD, 2010d)3Reading literacyMathematics literacyScience literacySCHOOL SYSTEM EXPENDITURESource: Education at a Glance (OECD, 2010a)3Expenditure on primary, secondary and post-secondary non-tertiary institutions asa % of GDP,from public and private sources199520002007Public expenditure on primary, secondary and post-secondary non-tertiaryeducationas a % of total public expenditure (2008)5Total expenditure on primary, secondary and post-secondary non-tertiaryeducationfrom public sources (2007) (%)Annual expenditure per student by educational institutions,(2007) (US$)6PrimaryLower secondaryUpper secondaryAll secondaryChange in expenditure per student by educational institutions,primary, secondaryand post-secondary non-tertiary education, index of change between 1995, 2000 and2007 (2000 = 100)19952007Current expenditure – composition,primary, secondary and post-secondary non-tertiary education (2007)7Compensation of teachersCompensation of other staffCompensation of all staffOther current expenditure

144– ANNEX 3DenmarkSCHOOL STAFF NUMBERSRatio of students to teaching staff(2008) Source: Education at a Glance(OECD, 2010a)3, 8PrimaryLower SecondaryUpper SecondaryAll SecondaryCHARACTERISTICS OF THE TEACHER WORKFORCE(lower secondary education, 2007-08) Source: TALIS (OECD, 2009b)9Age distribution of teachersTeachers aged under 25 yearsTeachers aged 25-29 yearsTeachers aged 30-39 yearsTeachers aged 40-49 yearsTeachers aged 50-59 yearsTeachers aged 60 years and moreGender distribution of teachers(% of females)Teachers’ educational attainment% of teachers who completed an ISCED 5A qualification or higher4Employment status of teachers% of teachers permanently employedTEACHER SALARIESin public institutions, Source: Education at a Glance(OECD, 2010a)3Annual teacher salaries(2008)6Primary – starting salary (US$)Primary – 15 years experience (US$)Primary – top of scale (US$)Primary – ratio of salary after 15 years experience to GDP per capitaLower secondary – starting salary (US$)Lower secondary – 15 years experience (US$)Lower secondary – top of scale (US$)Lower secondary – ratio of salary after 15 years experience to GDP per capitaUpper secondary – starting salary (US$)Upper secondary – 15 years experience (US$)Upper secondary – top of scale (US$)Upper secondary – ratio of salary after 15 years experience to GDP per capitaNumber of years from starting to top salary(lower secondary education) (2008)Decisions on payments for teachers in public schools(2008)10Criteria for base salary and additional payments awarded to teachers in public institutions●Base salary/■ Additional yearly payment /∆ Additional incidental paymentYears of experience as a teacherManagement responsibilities in addition to teaching dutiesTeaching more classes or hours than required by full-time contractSpecial tasks (career guidance or counselling)Teaching in a disadvantaged, remote or high cost area (location allowance)Special activities (e.g. sports and drama clubs, homework clubs, summer schools etc.)Teaching students with special educational needs (in regular schools)Teaching courses in a particular fieldHolding an initial educational qualification higher than the minimum qualificationrequired to enter the teaching professionHolding a higher than minimum level of teacher certification or training obtained duringprofessional lifeOutstanding performance in teachingSuccessful completion of professional development activitiesReaching high scores in the qualification examinationHolding an educational qualification in multiple subjectsFamily status (married, number of children)Age (independent of years of teaching experience)Other

DenmarkTEACHER PROFESSIONAL DEVELOPMENT(lower secondary education)Source: TALIS (OECD, 2009b)9Teacher participation in professional development(2007-08)% of teachers who undertook some prof. development in the previous 18 monthsAverage days of professional development across all teachersAverage days of professional development among those who received someAverage % of professional development days taken that were compulsoryTypes of professional development undertaken by teachers(2007-08)Courses and workshopsEducation conferences and seminarsQualification programmesObservation visits to other schoolsProfessional development networkIndividual and collaborative researchMentoring and peer observationReading professional literatureInformal dialogue to improve teachingImpact of different types of professional development undertaken by teachers(2007-08)% of teachers reporting that the professional development undertaken had a moderate orhigh impact upon their development as a teacherCourses and workshopsEducation conferences and seminarsQualification programmesObservation visits to other schoolsProfessional development networkIndividual and collaborative researchMentoring and peer observationReading professional literatureInformal dialogue to improve teachingTeachers’ high professional development needs(2007-08)% of teachers indicating they have a ‘high level of need’ for professional development inthe following areasContent and performance standardsStudent assessment practicesClassroom managementSubject fieldInstructional practicesICT teaching skillsTeaching special learning needs studentsStudent discipline and behaviour problemsSchool management and administrationTeaching in a multicultural settingStudent counsellingTEACHER PERCEPTION OF SELF-EFFICACY(lower secondary education)Source: TALIS (OECD, 2009b)9% of teachers who ‘strongly agree’ or ‘agree’ with the statement “Teachers feel that theyare making a significant educational difference” (2007-08)% of teachers who ‘strongly agree’ or ‘agree’ with the statement “Teachers feel thatwhen they try really hard, they can make progress with even the most difficult andunmotivated students” (2007-08)SYSTEM EVALUATIONExamination regulations,public schools only (2008)11Primary education (Yes/No)A standard curriculum or partially standardised curriculum is requiredMandatory national examination is required12Mandatory national assessment is required13Lower secondary education (Yes/No)A standard curriculum or partially standardised curriculum is requiredMandatory national examination is requiredMandatory national assessment is required

146– ANNEX 3DenmarkPotential subjects of assessment at national examinations12(lower secondaryeducation) (2006) Source: Education at a Glance (OECD, 2008)3, 11National examinations exist (Yes/No)MathematicsScienceNational language or language of instructionOther subjectsCompulsory for schools to administer national examinations (Yes/No)Year/Grade of national examinationPotential subjects of assessment at national periodical assessments13(lowersecondary education) (2006) Source: Education at a Glance (OECD, 2008)3, 11National periodical assessments (Yes/No)MathematicsScienceNational language or language of instructionOther subjectsCompulsory for school to administer national assessment (Yes/No)Year/Grade of national assessmentPossible influence of national examinations(lower secondary education) (2006)Source: Education at a Glance (OECD, 2008)3None/Low/Moderate/High14Performance feedback to the schoolPerformance appraisal of the school managementPerformance appraisal of individual teachersThe school budgetThe provision of another financial reward or sanctionThe assistance provided to teachers to improve their teaching skillsRemuneration and bonuses received by teachersLikelihood of school closurePublication of results (Yes/No)11Publication of tables that compare school performance (Yes/No)Possible influence of national periodical assessments(lower secondary education)(2006) Source: Education at a Glance (OECD, 2008)3None/Low/Moderate/High14Performance feedback to the schoolPerformance appraisal of the school managementPerformance appraisal of individual teachersThe school budgetThe provision of another financial reward or sanctionThe assistance provided to teachers to improve their teaching skillsRemuneration and bonuses received by teachersLikelihood of school closurePublication of results (Yes/No)11Publication of tables that compare school performance (Yes/No)Existence of national tests(2008-09) Source: Eurydice (2009)15Number of national tests(2008-09) (primary and lower secondary education)Source: Eurydice, (2009)15Compulsory testsSample testsOptional tests16Years of testingNumber of subjects covered in national tests17Main aims of nationally standardised tests(2008-09) (primary and lower secondaryeducation) Source: Eurydice (2009)11, 15(Yes/No)Taking decisions about the school career of pupilsMonitoring schools and/or the education systemIdentifying individual learning needsBodies responsible for setting national tests(2008-09) (primary and lower secondaryeducation) Source: Eurydice (2009)10, 15●Testsfor taking decisions about the school career of pupils/■Tests for otherpurposes/∆No national testsA unit/agency within the ministry of education without external playersA unit/agency within the ministry of education with external playersA public body distinct from the ministry, which specialises in education or educational evaluationA private body or university department

DenmarkPeople in charge of administering national tests(2008-09)(primary and lower secondary education)Source: Eurydice (2009)10, 15●Testsfor taking decisions about the school career of pupils/■Testsfor other purposes /∆Nonational testsClass teachersClass teachers + external peopleOther teachers from the same schoolOther teachers from the same school + external peopleExternal people alonePersons in charge of marking national tests(2008-09)(primary and lower secondary education)Source: Eurydice (2009)10, 15●Testsfor taking decisions about the school career of pupils/■Tests for otherpurposes/∆No national testsClass teachersClass teachers + external peopleOther teachers from the same schoolOther teachers from the same school + external personsExternal persons aloneStandardisation of test questions(2008-09) (primary and lower secondary education)Source: Eurydice (2009)11, 15(Yes/No)Questions are the same for all pupils taking one national testQuestions are not the same for all pupils taking one national testWhether test questions are standardised or not varies depending on type of testData not availableUse of ICT in national testing(2008-09) (primary and lower secondary education)Source: Eurydice (2009)11, 15(Yes/No)ICT is currently used in national testsUse of ICT for on-screen testingUse of ICT for marking testsParticipation of students with special educational needs (SEN) in national testing(2008-09) (primary and lower secondary education)Source: Eurydice (2009)11, 15(Yes/No)Pupils with SEN may take part in national testingParticipation in national testing for pupils with SEN is compulsoryParticipation in national testing for pupils with SEN is optionalParticipation varies depending on type of test, level of education or type of schoolData not availableCommunication of the results of national tests to local authorities(2008-09)(primary and lower secondary education) Source: Eurydice (2009)11, 15(Yes/No)Local authorities have access to aggregated results for their own areaUse of achievement data for accountability(2009) (15-year-olds)Source: PISA Compendium for the school questionnaire (OECD, 2010c)3% of students in schools where the principal reported that achievement data is used inthe following proceduresPosted publiclyUsed in evaluation of the principal’s performanceUsed in evaluation of teachers’ performanceUsed in decisions about instructional resource allocation to the schoolTracked over time by an administrative authoritySCHOOL EVALUATIONRequirements for school evaluations by an inspectorate(lower secondary education)(2006) Source: Education at a Glance (OECD, 2008)3None/1 per 3+ years/1 per 3 years/1 per 2 years/1 per year/1+ per yearPossible influence of school evaluation by an inspectorate(lower secondaryeducation) (2006) Source: Education at a Glance (OECD, 2008)3None/Low/Moderate/High14Influence on performance feedbackPerformance feedback to the schoolPerformance appraisal of the school managementPerformance appraisal of individual teachers

148– ANNEX 3DenmarkFinancial and other implicationsThe school budgetThe provision of another financial reward or sanctionThe assistance provided to teachers to improve their teaching skillsRemuneration and bonuses received by teachersLikelihood of school closurePublication of results (Yes/No)11Publication of tables that compare school performance (Yes/No)Requirements for school self-evaluations(lower secondary education) (2006)Source: Education at a Glance (OECD, 2008)3None/1 per 3+ years/1 per 3 years/1 per 2 years/1 per year/1+ per yearPossible influence of school self-evaluations(lower secondary education) (2006)Source: Education at a Glance (OECD, 2008)3None/Low/Moderate/High14Influence on performance feedbackPerformance feedback to the schoolPerformance appraisal of the school managementPerformance appraisal of individual teachersFinancial and other implicationsThe school budgetThe provision of another financial reward or sanctionThe assistance provided to teachers to improve their teaching skillsRemuneration and bonuses received by teachersLikelihood of school closurePublication of results (Yes/No)11Publication of tables that compare school performance (Yes/No)Frequency and type of school evaluations(lower secondary education) (2007-08)Source: TALIS (OECD, 2009b)9% of teachers working in schools where school evaluations were conducted with thefollowing frequency over the last five yearsFrequency of school self-evaluationNeverOnce2-4 timesOnce per yearMore than once per yearFrequency of external evaluationNeverOnce2-4 timesOnce per yearMore than once per yearNo school evaluation from any sourceCriteria of school evaluations(lower secondary education) (2007-08) Source: TALIS(OECD, 2009b)9% of teachers whose school principal reported that the following criteria were consideredwith high or moderate importance in school self-evaluations or external evaluationsStudent test scoresRetention and pass rates of studentsOther student learning outcomesStudent feedback on the teaching they receiveFeedback from parentsHow well teachers work with the principal and their colleaguesDirect appraisal of classroom teachingInnovative teaching practicesRelations between teachers and studentsProfessional development undertaken by teachersTeachers’ classroom managementTeachers’ knowledge and understanding of their main subject field(s)Teachers’ knowledge and understanding of instructional practices in their main subjectfield(s)Teaching of students with special learning needsStudent discipline and behaviourTeaching in a multicultural settingExtra-curricular activities with students (e.g. school plays and performances, sporting activities)mmmmmmm

DenmarkImpacts of school evaluations upon schools(lower secondary education) (2007-08)Source: TALIS (OECD, 2009b)9% of teachers whose school principal reported that school evaluations (external or self-evaluation) had a high or moderate level of influence on the followingLevel of school budget or its distribution within schools22.3Performance feedback to the school52.9Performance appraisal of the school management58.5Performance appraisal of teachers32.5Assistance provided to teachers to improve their teaching44.3Teachers’ remuneration and bonuses9.0Publication of school evaluations(lower secondary education) (2007-08)Source: TALIS (OECD, 2009b)9% of teachers in schools where school evaluation results were :Published; or84.5Used in school performance tables54.8Use of student test results in school evaluation(2008-09) (primary and lowersecondary education) Source: Eurydice (2009)11, 15(Yes/No)Test results may be used for evaluationNoTest results used for external evaluationNoRecommendations or support tools for the use of results during internal evaluationNoUse varies depending on type of test, level of education or type of schoolNoPublication of individual school results in national tests(2008-09) (primary andlower secondary education) Source: Eurydice (2009)11, 15(Yes/No)Individual school results may be publishedYesPublication organised, or required of schools, by central/local governmentsYesPublication at the discretion of schoolsNoAccountability to parents(2009) (15-year-olds)Source: PISA Compendium for the school questionnaire (OECD, 2010c)3% of students in schools where principals reported that their school provides parents with information on:This child’s academic performance relative to other students in the school-This child’s academic performance relative to national or regional benchmarks37.430.4This child’s academic performance of students as a group relative to students in thesame grade in other schoolsTEACHER APPRAISALFrequency and source of teacher appraisal and feedback(lower secondaryeducation) (2007-08) Source: TALIS (OECD, 2009b)9% of teachers who reported having received appraisal and/or feedback on their workwith the following frequency from the following sourcesFeedback received from the principalNeverLess than once every two yearsOnce every two yearsOnce per yearTwice per year3 or more times per yearMonthlyMore than once per monthFeedback received from other teachers or members of the school management teamNeverLess than once every two yearsOnce every two yearsOnce per yearTwice per year3 or more times per yearMonthlyMore than once per monthFeedback received from an external individual or body (e.g. external inspector)NeverLess than once every two yearsOnce every two yearsOnce per yearTwice per year3 or more times per yearMonthlyMore than once per monthOECD REVIEWS OF EVALUATION AND ASSESSMENT IN EDUCATION: DENMARK � OECD 2011

150– ANNEX 3DenmarkCriteria for teacher appraisal and feedback(lower secondary education) (2007-08)Source: TALIS (OECD, 2009b)9% of teachers who reported that the following criteria were considered with high ormoderate importance in the appraisal and/or feedback they receivedStudent test scoresRetention and pass rates of studentsOther student learning outcomesStudent feedback on the teaching they receiveFeedback from parentsHow well they work with the principal and their colleaguesDirect appraisal of classroom teachingInnovative teaching practicesRelations with studentsProfessional development undertakenClassroom managementKnowledge and understanding of their main subject field(s)Knowledge and understanding of instructional practices in their main subject field(s)Teaching of students with special learning needsStudent discipline and behaviourTeaching in a multicultural settingExtra-curricular activities with students (e.g. school performances, sporting activities)Outcomes of teacher appraisal and feedback(lower secondary education) (2007-08)Source: TALIS (OECD, 2009b)9% of teachers who reported that the appraisal and/or feedback they received let to amodest or large change in the following aspects of their work and careersA change in salaryA financial bonus or another kind of monetary rewardA change in the likelihood of career advancementPublic recognition from the principal and/or their colleaguesOpportunities for professional development activitiesChanges in work responsibilities that make the job more attractiveA role in school development initiatives (e.g. curriculum development group)Actions undertaken following the identification of a weakness in a teacher appraisal(lower secondary education) (2007-08) Source: TALIS (OECD, 2009b)9% of teachers whose school principal reported that the following occurs if an appraisalof teachers’ work identifies a specific weaknessThe principal ensures that the outcome is reported to the teacherNeverSometimesMost of the timeAlwaysThe principal ensures that measures to remedy the weakness in their teaching arediscussed with the teacherNeverSometimesMost of the timeAlwaysThe principal, or others in the school, establishes a development or training plan for theteacher to address the weakness in their teachingNeverSometimesMost of the timeAlwaysThe principal, or others in the school, imposes material sanctions on the teacher (e.g.reduced annual increases in pay)NeverSometimesMost of the timeAlwaysThe principal, or others in the school, report the underperformance to another body totake action (e.g. governing board, local authority, school inspector)NeverSometimesMost of the timeAlways

DenmarkThe principal ensures that the teacher has more frequent appraisals of their workNeverSometimesMost of the timeAlwaysTeacher perceptions of the appraisal and/or feedback they received(lowersecondary education) (2007-08) Source: TALIS (OECD, 2009b)9% of teachers who reported the following about the appraisal and/or feedback they hadreceived in their schoolAppraisal and/or feedback contained a judgement about the quality of the teacher’s workAppraisal and/or feedback contained suggestions for improving certain aspects ofteacher’s workAppraisal and/or feedback was a fair assessment of their work as a teacher in this schoolStrongly disagreeDisagreeAgreeStrongly agreeAppraisal and/or feedback was helpful in the development of their work as teachers inthis schoolStrongly disagreeDisagreeAgreeStrongly agreeTeacher perceptions of the personal impact of teacher appraisal and feedback(lower secondary education) (2007-08) Source: TALIS (OECD, 2009b)9% of teachers who reported the following changes following the appraisal and/orfeedback they received in their schoolthe following personal impact from appraisal and feedbackChange in their job satisfactionA large decreaseA small decreaseNo changeA small increaseA large increaseChange in their job securityA large decreaseA small decreaseNo changeA small increaseA large increaseImpact of teacher appraisal and feedback upon teaching(lower secondary education)(2007-08) Source: TALIS (OECD, 2009b)9% of teachers who reported that the appraisal and/or feedback they received directly ledto or involved moderate or large changes in the followingClassroom management practicesKnowledge or understanding of the teacher’s main subject field(s)Knowledge or understanding of instructional practicesA development or training plan for teachers to improve their teachingTeaching of students with special learning needsStudent discipline and behaviour problemsTeaching of students in a multicultural settingEmphasis placed by teachers on improving student test scores in their teachingTeacher appraisal and feedback and school development(lower secondaryeducation) (2007-08) Source: TALIS (OECD, 2009b)9% of teachers who agree or strongly agree with the following statements about aspects ofappraisal and/or feedback in their schoolIn this school, the school principal takes steps to alter the monetary reward of thepersistently underperforming teacherIn this school, the sustained poor performance of a teacher would be tolerated by the restof the staffIn this school, teachers will be dismissed because of sustained poor performanceIn this school, the principal uses effective methods to determine whether teachers areperforming well or badlyIn this school, a development or training plan is established for teachers to improve theirwork as teachers5.342.934.717.1

152– ANNEX 3DenmarkIn this school, the most effective teachers receive the greatest monetary or non-monetaryrewardsIn this school, if I improve the quality of my teaching I will receive increased monetaryor non-monetary rewardsIn this school, if I am more innovative in my teaching I will receive increased monetaryor non-monetary rewardsIn this school, the review of teacher’s work is largely done to fulfil administrativerequirementsIn this school, the review of teacher’s work has little impact upon the way teachers teachin the classroomOfficial methods for the individual or collective evaluation of teachers(2006-07)Source: Eurydice (2008)11, 15Teacher evaluation existsTeacher inspection on an individual or collective basisSchool self-evaluationIndividual evaluation by school headsIndividual evaluation by peersMethods used to monitor the practice of teachers(2009) (15-year-olds)Source: PISA Compendium for the school questionnaire (OECD, 2010c)3% of students in schools where the principal reported that the following methods havebeen used the previous year to monitor the practice of teachers at their schoolTests of assessments of student achievementTeacher peer review (of lesson plans, assessment instruments, lessons)Principal or senior staff observations of lessonsObservation of classes by inspectors or other persons external to the schoolSTUDENT ASSESSMENTThe influence of test results on the school career of pupils(2008-09)(primary and lower secondary education)Source: Eurydice (2009)10, 15ISCED 1/ ISCED 24Award of certificatesStreamingProgression to the next stage of educationNo national tests, or no impact on progressionCompletion requirements for upper secondary programmesSource: Education at a Glance (OECD, 2009a)3, 10●Final examination /■ Series of examinations during programme /∆ Specified numberof course hours and examination /♦Specified number of course hours onlyISCED 3A4ISCED 3BISCED 3CStudent grouping by ability(2009) (15-year-olds)Source: PISA Compendium for the school questionnaire (OECD, 2010c)3% of students in schools where principals reported the following on student grouping byabilityStudent are grouped by ability into different classesFor all subjectsFor some subjectsNot for any subjectStudent are grouped by ability within their classesFor all subjectsFor some subjectsNot for any subjectGroups of influence on assessment practices(2009) (15-year-olds)Source: PISA Compendium for the school questionnaire (OECD, 2010c)3% of students in schools where the principal reported the following groups exert a directinfluence on decision making about assessment practicesRegional or national education authorities (e.g. inspectorates)The school’s governing boardParent groupsTeacher groups (e.g. staff association, curriculum committees, trade union)Student groups (e.g. student association, youth organisationExternal examination boards15.08.39.048.160.8

DenmarkResponsibility for student assessment policies(2009) (15-year-olds)Source: PISA Compendium for the school questionnaire (OECD, 2010c)3% of students in schools where the principal reported the following groups haveconsiderable responsibility in establishing student assessment policiesEstablishing student assessment policiesPrincipalsTeachersSchool governing boardRegional or local education authorityNational education authorityFrequency of student assessment by method(2009) (15-year-olds)Source: PISA Compendium for the school questionnaire (OECD, 2010c)3% of students in schools where the principal reported the student assessment methodsbelow are used with the indicated frequencyStandardised testsNever1-2 times a year3-5 times a yearMonthlyMore than once a monthTeacher-developed testsNever1-2 times a year3-5 times a yearMonthlyMore than once a monthTeachers’ judgmental ratingsNever1-2 times a year3-5 times a yearMonthlyMore than once a monthStudent portfoliosNever1-2 times a year3-5 times a yearMonthlyMore than once a monthStudent assignments/projects/homeworkNever1-2 times a year3-5 times a yearMonthlyMore than once a month% of students reporting the following on the frequency of homework(2000)Source: PISA Student Compendium (Reading) (OECD, 2000) (15-year-olds)Teachers grade homeworkNeverSometimesMost of the timeAlwaysTeachers make useful comments on homeworkNeverSometimesMost of the timeAlwaysHomework is counted as part of markingNeverSometimesMost of the timeAlways

154– ANNEX 3DenmarkUse of student assessments(2009) (15-year-olds)Source: PISA Compendium for the school questionnaire (OECD, 2010c)3% students in schools where the principal reported that assessments of students are usedfor the following purposesTo inform the parents about their child’s progressTo make decisions about students’ retention or promotionTo group students for instructional purposesTo compare the school to district or national performanceTo monitor the school’s progress from year to yearTo make judgements about teachers’ effectivenessTo identify aspects of instruction or the curriculum that could be improvedTo compare the school with other schools% of students repeating a grade in the previous school year according to reports byschool principals in the following levels(2009) (15-year-olds)Source: PISA Compendium for the school questionnaire (OECD, 2010c)3ISCED24ISCED3Parents’ perception of school’s monitoring of student progress(2009) (15-year-olds)Source: PISA Compendium for the parent questionnaire (OECD, 2010b)3% of parents who agree or strongly agree with the following statements18My child’s progress is carefully monitored by the schoolStrongly agreeAgreeDisagreeStrongly disagreeMy child’s school provides regular and useful information on my child’s progressStrongly agreeAgreeDisagreeStrongly disagreeLevel of school autonomy regarding the criteria for the internal assessment ofpupils(2006-07) (primary and lower secondary education)Source: Eurydice (2008)11, 15Full/Limited/No autonomySchool decision-makers involved in determining the criteria for the internalassessment of pupils(2006-07) (primary and lower secondary education)Source: Eurydice (2008)11, 15School responsibility involvedSchool headTeachers individually or collectivelySchool management bodyResponsibilities vary depending on level of educationSchool autonomy in preparing the content of examinations for certifiedqualifications(2006-07) (primary and lower secondary education)Source: Eurydice (2007)11, 15School responsibility involved/ examinations for certified qualifications existFull/Limited/No autonomySchool decision-makers who may be involved in preparing the content ofexaminations for certified qualifications(ISCED 2)4(2006-07)Source: Eurydice (2007)11, 15School responsibility involved/ examinations for certified qualifications existSchool headTeachers individually or collectivelySchool management bodyResponsibilities vary depending on level of education

Sources:Eurydice (2008),Levels of Autonomy and Responsibilities of Teachers in Europe,Eurydice, Brussels.Eurydice (2009),National Testing of Pupils in Europe: Objectives, Organisation and Use of Results,Eurydice, Brussels.OECD (2000),PISA Student Compendium (Reading),OECD,http://pisa2000.acer.edu.au/downloads.php/OECD (2008),Education at a Glance, OECD Indicators 2008,OECD, Paris.OECD (2009a),Education at a Glance, OECD Indicators 2009,OECD, Paris.OECD (2009b),Creating Effective Teaching and Learning Environments: First Results from TALIS,OECD, Paris.OECD (2010a),Education at a Glance, OECD Indicators 2010,OECD, Paris.OECD (2010b),PISA 2009 Compendium for the parent questionnaire,OECD,http://pisa2009.acer.edu.au/downloads.phpOECD (2010c),PISA 2009 Compendium for the school questionnaire,OECD,http://pisa2009.acer.edu.au/downloads.phpOECD (2010d),PISA 2009 Results: What Students Know and Can Do, Volume I,OECD, Paris.Data explanation:mData is not availableaData is not applicable because the category does not apply~Average is not comparable with other levels of education=At least one other country has the same rankThe report Eurydice (2009) includes all 32 member countries/education areas of the European Union as well as the members ofthe European Economic Area (Iceland, Liechtenstein and Norway).TALIS is the OECD’s Teaching and Learning International Survey which was implemented for the first time in 2007-08. Thedata provided concerns 23 countries. The results derived from TALIS are based on self-reports from teachers and principals andtherefore represent their opinions, perceptions, beliefs and their accounts of their activities. Further information is available atwww.oecd.org/edu/talis.PISA is the OECD’s Programme for International Student Assessment, which was undertaken in 2000, 2003, 2006 and 2009.15-year-old students worldwide are assessed on their literacy in reading, mathematics and science. The study included 27 OECDcountries in 2000, 30 in 2003 and 2006, and 34 in 2009. Data used in this appendix can be found atwww.pisa.oecd.org.General notes:1.2.The country average is calculated as the simple average of all countries for which data are available.“Denmark’s rank” indicates the position of Denmark when countries are ranked in descending order from thehighest to lowest value on the indicator concerned. For example, on the first indicator “population that hasattained at least upper secondary education”, for the age group 25-64, the rank 14/30 indicates that Denmarkrecorded the 14th highest value of the 30 countries that reported relevant data.The column “country average” corresponds to an average across OECD countries.ISCED is the “International Standard Classification of Education” used to describe levels of education (andsubcategories).ISCED 1 - Primary educationDesigned to provide a sound basic education in reading, writing and mathematics and a basic understanding of some othersubjects. Entry age: between 5 and 7. Duration: 6 yearsISCED 2 - Lower secondary educationCompletes provision of basic education, usually in a more subject-oriented way with more specialist teachers. Entry follows6 years of primary education; duration is 3 years. In some countries, the end of this level marks the end of compulsoryeducation.ISCED 3 - Upper secondary educationEven stronger subject specialisation than at lower-secondary level, with teachers usually more qualified. Students typicallyexpected to have completed 9 years of education or lower secondary schooling before entry and are generally around the ageof 15 or 16.ISCED 3A - Upper secondary education type APrepares students for university-level education at level 5AISCED 3B - Upper secondary education type BFor entry to vocationally oriented tertiary education at level 5BISECD 3C - Upper secondary education type CPrepares students for workforce or for post-secondary non tertiary education

ISCED 4 - Post-secondary non-tertiary educationProgrammes at this level may be regarded nationally as part of upper secondary or post-secondary education, but in terms ofinternational comparison their status is less clear cut. Programme content may not be much more advanced than in uppersecondary, and is certainly lower than at tertiary level. Entry typically requires completion of an upper secondaryprogramme. Duration usually equivalent to between 6 months and 2 years of full-time study.ISCED 5 - Tertiary educationISCED 5 is the first stage of tertiary education (the second – ISCED 6 – involves advanced research). At level 5, it is oftenmore useful to distinguish between two subcategories: 5A, which represent longer and more theoretical programmes; and5B, where programmes are shorter and more practically oriented. Note, though, that as tertiary education differs greatlybetween countries, the demarcation between these two subcategories is not always clear cut.ISCED 5A - Tertiary-type A“Long-stream” programmes that are theory based and aimed at preparing students for further research or to give accessto highly skilled professions, such as medicine or architecture. Entry preceded by 13 years of education, studentstypically required to have completed upper secondary or post-secondary non-tertiary education. Duration equivalent toat least 3 years of full-time study, but 4 is more usual.ISCED 5B - Tertiary-type B“Short-stream” programmes that are more practically oriented or focus on the skills needed for students to directlyenter specific occupations. Entry preceded by 13 years of education; students may require mastery of specific subjectsstudied at levels 3B or 4A. Duration equivalent to at least 2 years of full-time study, but 3 is more usual.

Public expenditure includes public subsidies to households for living costs (scholarships and grants to students/households and students loans), which are not spent on educational institutions.Expressed in equivalent US$ converted using purchasing power parities.Expenditure on goods and services consumed within the current year which needs to be made recurrently to sustain theproduction of educational services – refers to current expenditure on schools and post-secondary non-tertiary educationalinstitutions. The individual percentage may not sum to the total due to rounding.Public and private institutions are included. Calculations are based on full-time equivalents. “Teaching staff” refers toprofessional personnel directly involved in teaching students.The column “country average” corresponds to an average across TALIS countries.The column “country average” indicates the number of countries/systems, in which a given criterion is used, for example,regarding the indicator “Decision on payments for teachers in public schools”. In the row “Management responsibilities inaddition to teaching duties”,●12 ■18 ∆7indicates that this criterion is used to determine the base salary in 12countries/systems, to determine an additional yearly payment in 18 countries/systems and to determine an additionalincidental payment in 7 countries/systems.The column “country average” indicates the number of countries for which the indicator applies. For example, for theindicator “mandatory national examination is required” 4/29 means, that 4 countries out of 29 for which data is availablereport that mandatory national examinations are required in their countries.By “national examination” we mean those tests, which do have formal consequences for students.By “national assessment” we mean those tests, which do not have formal consequences for students.These measures express the degree of influence on the indicator: None: No influence at all, Low: Low level of influence,Moderate: Moderate level of influence, High: High level of influence. The column “country average” indicates the numberof countries/systems, in which one of the given criteria is used.For this indicator, the column “country average” refers to Eurydice member countries/areas.“Compulsory tests” have to be taken by all pupils, regardless of the type of school attended, or by all students in publicsector schools. “Optional tests” are taken under the authority of schools.Austria, Belgium-Flemish Community, Ireland, Lithuania, Latvia, Sweden, England, Northern Ireland and Scotland applyseveral tests at the national level each with a distinct number of subjects. Thus, for these countries no exact number ofsubjects tested can be provided.Results are based on reports from parents of the students who were assessed and reported proportionate to the number of15-year-olds enrolled in the school.

Denmark

Quality in education is increasingly judged on the basis of effective learning outcomes. Countriesuse a range of techniques for the evaluation and assessment of students, teachers, schools andeducation systems with the objective of assessing whether the school system is delivering goodperformance and providing feedback for improvement in student outcomes. Many countries testsamples and/or all students at key points and also formally review the work of individual teachersand schools. However, countries often face challenges in bringing the different elements ofevaluation and assessment together into a coherent and comprehensive strategy, within whicheach element is fit for purpose and contributes effectively to improving learning outcomes.In this context, the OECD launched theReview on Evaluation and Assessment Frameworks forImproving School Outcomesto provide analysis and policy advice to countries on the followingoverarching policy question: “How can assessment and evaluation policies work together moreeffectively to improve student outcomes in primary and secondary schools?” The Review coversstudent assessment, teacher appraisal, school evaluation and education system evaluation.Twenty-five education systems are participating in the project. Denmark is one of the countrieswhich opted to host a Country Review, in which a team of external reviewers carried out anin-depth analysis of evaluation and assessment in education. This report includes:An overview of Denmark’s educational evaluation and assessment framework;An account of trends and developments in evaluation and assessment in Denmark;An analysis of the strengths and challenges in evaluation and assessment in Denmark; andPointers for future policy development.