Home > Your ideas > Submissions > Anonymous

Anonymous

Submission: 

Please see attached.

Document: 
PDF icon Download (7.43 MB)
Automatic Transcription: 
Independent Panel

Department of the Prime Minister and Cabinet

PO Box 6500

CANBERRA ACT 2600

Dear Chair and esteemed Panel

INDEPENDENT REVIEW OF THE APS

It is with great pleasure that I formally lodge my submission in response to the

opportunity to contribute to the Independent Review of the APS (the Review).

I have always been passionate about the Australian Public Service (APS) and both

how and what it delivers for all Australians, as well as the wider domestic and global

community.

Both my parents have had distinguished careers within and supporting the public

service and I am proud of their service to an entity that is the foundation of good

government and ultimately a better standard of living and future for country.

My private sector background has seen me work for some of the largest and

smallest organisations across a number of industries. A long history supporting the

APS in this capacity ultimately saw me move to work within the public sector with a

drive to make a difference. I have long been driven by the want and need to make

difference to how and what we deliver to see the APS position itself to support our

country’s leaders build a better future for citizens today and in the decades to come.

I would like to once again express my sincere thanks for being afforded the

opportunity to contribute to your independent review of the APS.

I look forward to seeing the review’s findings and to contributing to the holistic and

structural reform and change required to see the APS proactively transformed to be

fit-for-purpose for the future ahead.

Yours sincerely

5 (July 2018
Preamble

The APS has long been recognised for its expertise, flexibility and capacity to adapt

and respond to change. However the rate of change within its operating environment

is unprecedented - driven by digital transformation, and significant geo-political,
economic and demographic shifts.

This response considers how these shifts will create new challenges and

opportunities for the APS. No organisation can afford to be reactive and in order to

remain relevant and continue to add value to citizens and our community, it is

essential the purpose, role, contribution and approach all be reviewed with the

purpose of ensuring we are positioned to deliver valued services in a future context.

Further to this, it is imperative that any improvements see the APS positioned to flex

and adjust to the significant, fast-paced and ongoing change to voters and

taxpayers, staff, suppliers, clients, policy makers and partners are experiencing.
They must also be cognisant of the tight fiscal environment in which the service

operates and the continuing expectation by taxpayers that monies are used

efficiently, effectively and ultimately to the betterment of all Australians.

This response provides insights across the key topic areas the Review panel will be

exploring (as per the Terms of Reference), to provide insights into some areas which

may not have previously been identified, or from this perspective.

I appreciate the complexity and significant challenge posed to the Review panel

within the tight timeframes and I wish you all the best in your endeavours to analyse

and present practical recommendations to ensure the APS is ready, over the coming

decades, to best serve Australia.
Attachments:
Please see the attached documents which are referred to within this response and

which I believe will add to your greater understanding of the key issues with the

evidence to support.

  1. The Mandarin - Tom Burton: Canberra Clones itself
  2. The International Civil Service Effectiveness (InCiSE) Index 2017
  3. The International Civil Service Effectiveness (InCiSE) Index - Technical
    Report
  4. The International Civil Service Effectiveness (InCiSE) Index - Country
    Rankings

Key Observations:

Recruitment
• Pathway programs and partnerships with high schools, trade schools and
universities
• Secondments with professional and industry partners - especially in
alignment with portfolio aims
• Recruitment that utilises the best approach to capturing the right person
based on the profession and industry (specialist role based recruiters,
advertising and processes.)
• Recruitment processes that reflect advances in assessing staff for jobs based
on the roles (eg. skills based testing, alignment with private sector
approaches to ensure we do not miss out on a large section of those who are
intimidated by the APS process, behavioural testing to see if the mental
pedigree also aligns with the role and service).
• Promoting the real benefits of a career in the APS that target the aspects that
attract the role types.
• Considering numerous pathway to entry based on the role by creating
streams (to include a new management stream as a technical skill in itself)
• As we move to a more technical based and flexible based environment, look
to attract those in all parts of Australia through this approach that does not
miss out on a large section of those who are would be disinclined to apply due
to a primarily Canberra based requirement)

Assessing and investing in talent
• Opportunities to invest in staff once they find their calling - moving away from
disqualifying attributes roles which may prevent real talent from coming
through (eg. degrees or experiences unless these are core to the role).
• Creating technical line with opportunities for those who want to progress in
their career who don’t have the passion or attributes to lead people, (more
than just remuneration)
• Creating a technical line for leadership so that this stream reflects the
attributes of what we want to see in our leaders of today and tomorrow as
they shape the future.

Driving merit
• Bring greater accountability to all roles of the APS that require KPIs that are
aligned with better practice, strategy, culture, equivalent private sector
counterparts and policy outcomes, (despite departments and agencies not
being challenged by a bottom line, look to ways to really assess outcomes).
• Moving away from detailed old type recruitment processes that were originally
created to prevent cronyism and nepotism, however have led to this being the
primary attributes into today’s APS “merit” process.
• Improving the performance management approach with APS employees and
ensuring that it allows leaders and managers the ability to deal with
underperformance rather than driving a culture of failing to invest in staff and
rather transferring or promoting underperformers so as not to have to deal
with a 12 month process that might ultimately not lead to a resolution.
• Provider leaders and managers the tools and training to manage staff and
underperformance and high performers (this is no just e-modules or one day
face to face training).
• Creating mentoring and buddy programs to drive more cost effective staff
development and collaboration.
• Incentivise high performers who are disincentivised by the “treat everyone the
same” principles and managers and leaders not wanting to look like they are
playing favourites.

Equity, equality and diversity
• Move away from affirmative action and look at ways to drive merit based
diversity
• Although gender diversity is finally being managed, it is insulting to think that
affirmative action is needed to bring gender diversity when there are plenty of
intelligent, qualified and suitable women, (this requires significant changes in
the way we recruit and how we capture talent).
• Gender diversity through affirmative action is breeding a bad culture where
merit has been lost given it alienates other minorities who are far more
disadvantaged (eg. non-Anglo-Saxons make up the majority of the APS
senior leadership even the current independent review panel). It is also
alienating those in younger generations who were always pro-diversity who
now feel vilified and are penalising women who had no intention of being
given this greater advantage. Also indicators like university graduation by
gender and also most departments seeing a majority of women in levels
below SES seem to fly in the face of the approach being taken under the
banner of equality and what should also include equity. This is an issue of an
older generation who have made it the “boys club” and this should not be the
group that the rules are written against for everyone else.
• Investing in real diversity where merit is the primary driver. Please see
attached article by Tom Burton: Canberra clones itself from the Mandarin that
shows that the APS’s leadership is incestuous and fails to allow for alternative
ways of thinking. For policies, strategy and culture and issues to be
considered from all sides, one must have various perspectives. (Anglo-Saxon
males and females aged 45+ who all went to a top 8 university and are
primarily Canberra based bureaucratic lifers does not achieve this end).
Many SES have advised the fact that the SES bands are a club, if you don’t
have a “advocate” and don’t look, sound and think the same, you’re unlikely to
ever make the ranks, regardless of your performance.

Political vs apolitical
• The APS and the Review ToR both indicate that being apolitical is a key value
of the APS now and expected to continue as a key value into the future.
Despite this, it is evident using the most basic of indicators, that with a change
in government and the resulting changing of the guard at the Secretary level,
(the top level which ultimately shape all aspects of the APS) that the APS is
fundamentally very political. Although one can appreciate that within the
private sector senior leadership are hired or fired for a particular capability,
strategy or track record to deliver on an aligned strategy that the Board wants
to pursue, this is not ordinarily based on one’s political affiliation.
• Cuts in FTE eg. ASL caps cuts which has seen a marked increase in total
bodies (which includes contractors and consultants) is purely a politically
driven agenda. It is worrying to see that despite aggregate costs increasing
with no corresponding value proposition, this approach is applied across the
APS. There should be mechanisms in place to insulate department from
being impacted by political agendas, but being accountable to align with
accountability principles that reflect quantitative indicators of efficiency and
effectiveness.

Risk taking and frank and fearless.
• The APS has seen itself degraded in its ability to remain apolitical in giving
sound, evidenced based advice. In days gone by, the role was to be the
apolitical guardians of good evidenced based policy based on robust
discussions with the political apparatus and the key stakeholders with
considerations for benefit realisation. It is well known now that the APS has
moved away from providing frank and fearless advice and moved to a “yes
minister” type approach, based on reactive, policy implementation on the run,
far more focused on the process, political agendas and reputation rather than
actually delivering the best outcomes for Australians, men and women" who
again realise an outcome where the way one person thinks is the way all
problems are considered and resolved, meaning you can never considers all
the risks and opportunities that exist.
• The lack of understanding or acumen in true risk management that is focused
on all enterprise risks both financial and non-financial (especially culture and
conduct as seen in the royal commission into the banks), is significantly
lacking the public service. When SES members are more focused on
reputation and political agendas than taking risks in alignment with the
rewards proposed to them, this will always prevent the APS from taking the
next step forward. They also then breed a culture of “yes men and women”
who no longer looks to exploit opportunity but rather become indoctrinated in
risk averse thinking. Further to this, failure to manage real risks by focusing
on issues leads to a very reactive APS rather than understanding that there
can be proactive approaches that will assist with mitigating issues and risks.

Lessons learned
• The APS continues to show that it doesn’t learn from the past. Lessons
learned is one of the cheapest ways to innovate and improve and yet, in most
departments we undertake the same approach to doing things and continue to
engage suppliers to undertake the same reviews. If we cannot learn from our
mistakes, any chance or learning anything new is close to impossible.

Revising approach to procurement that drives real value
• The APS continues to identify value for money as the key approach to
procurement and contract management, however in most cases this boils
down to cheapest price, cronyism/nepotism. Although there is should be no
problem selecting a supplier that you may know, or selecting a supplier is cost
effective, there should be better ways from upskilling and incentivising our
staff and improving our assessment process from picking the cheapest or
failing to provide evidence as to why a particular supplier should be used.

Fiscal management
• In the international civil service effectiveness index, the APS is scored poorly
for fiscal management. The lack of real fiscal acumen among its SES with
primarily theoretical university driven knowledge which is out of date after a
decade is reflected. Key issues of budget management are so poor, that it is
a common fact that unused budget at April should be spent to ensure same or
greater appropriations in the future. In a time of austerity, this is unacceptable
when no benefit realisation is required to ensure that spends can be assessed
for return on investment or other relevant indictors such as alignment with
program or policy delivery objectives. It would be good to consider alternative
budget approaches including incentivising effective fiscal management (this

does not mean slash and burn) but rather not incentivising this current

approach.
The old management principle of more with less as we know has been thrown

out the door and is now less with less. This however should be considered

from more than one angle and should be the avenue where agency heads

request short term funding to enact efficiency and capability building with an

expectation to show full benefit realisation. This could be improvements via

R&D, innovation, automation, process improvement, outsourcing, better

procurement and contract management or improving capability of workforce.
Productivity indicators should be applied to all departments based on profile.
This should consider program size and complexity vs funding and outcomes.
The lack of standardised KPIs is leading to ambiguous outcomes that are

leaving tax payers questioning more and more what happens with their

money. This should be more important than focusing on reputation.
APS leaders and staff continue to believe that we can outsource our risks by

hiring consultants. As an ex-consultant I am well aware of the planning

sessions that go on at consulting houses, where they laugh at how easy it is

to convince APS leaders to engage them to undertake the same work over

and over again utilising junior staff which tax payers ultimately pay to train.
APS staff and leaders who are so risk averse, lack accountability for how they

spend on consultancies and who do not have the capacity or capability

continuously hire consultants for exorbitant rates regardless of need or

outcome.
Tom Burton: Canberra clones itself | The Mandarin - The Mandarin https://www.themandarin.com.au/83422-tom-burton-canberra-clones/

Tom Burton: Canberra Author Bio

clones itself
By Tom Burton (/author/tburton) • 08/09/2017

DNA replicates and so this week did Canberra’s
mandarin class. Half the existing federal
departments will get new secretaries in a major
shuffle of the bureaucratic deck, including two
new appointments. One of those, new
Inf rastructure and Regional Development boss
Steven Kennedy, has rightly had “secretary”
stamped on his personal f ile for several years.
He is a real loss to PM&C where he has led
much of that department’s new work around
cities.

But an eyeball of the bios of the 18 portfolio
secretaries, reveals a sameness, that says much
about the 155,000 strong Australian Public
Service.

Without exception, the secretaries are
bureaucratic lifers, almost all having spent their
career in Canberra, promoted through the
senior executive service as high performance
individuals. No plumbers or small business
owners, nor big corporate, industrial, or NGO
experience to be seen.

Treasury boss, John Fraser, worked in Treasury
before spending his mid career at investment
bank, UBS. Education chief, Michele Bruniges,
began life as a teacher rising through the NSW
and Canberra education bureaucracy. And the
journos remember new Employment secretary,
Kerri Hartland, as a Toowoomba rural journo and
then in the Canberra press gallery for four years
with various News Corp publications. Finance
secretary, Rosemary Huxtable, moved west
when then Prime Minister, John Howard,
downsized the bureaucracy in 1996. And
Kennedy started life as a nurse.

But these dalliances aside, every secretary is
very much a product of the Canberra system,
the Commonwealth government aided and
abetted by its local breeding ground, the ACT
government. All have worked in plenty of gigs
across the wide diaspora of domestic and
international functions the federal government
spans, but the complete lack of outside blood is
startling.

Compare this with NSW, where over a third of
the top bureaucracy is now f rom the
non-government world, a f ive year change
program that has injected much needed new
thinking and experience into that government.

All are white bread. I say that with no personal
disrespect, but to make the point it passes
strange that at the elite level of the Australian
government there is not an Asian, Af rican,

1 of 5 REDACTED 9:15 PM
Tom Burton: Canberra clones itself | The Mandarin - The Mandarin https://www.themandarin.com.au/83422-tom-burton-canberra-clones/

South American, North American to be seen.
Nor an indigenous person. This suggests a
cultural narrowness that common sense says is
not healthy, if you want to be kind. Less kindly,
would be to describe it as a tightly
self-controlled, in-bred world, cocooned f rom
the broader realities of modern diverse
Australia. This is equally true of the political
class (us* in the press gallery included) that
works in and around Capitol Hill. Anglo-Saxon
culture is alive and well with our ruling
Canberra elite.

But an educated elite
All the secretaries are products of the
post-Whitlam higher education system. All have
been products of the top-tier G8 university
system, liberal arts alumni mixed with post
grad law and economics qualif ications.
Economics is by far the dominant pedigree,
more precisely neo classical economics, taught
through conventional post-Keynesian
institutions.

The leader of this pack is the Australian
National University, which has long acted as
f inishing school for the top end of the APS.

No bomb throwers at that university. Its
impressive schools of strategy, economics and
public policy score well on almost any measure.
But there is a sameness and comfortable
acquiescence with the core mission of the big
APS agencies, that feels unhealthy when
observed f rom the grandstand. The ANU is the
only university funded directly f rom the
Commonwealth government. In addition ANU
through its iron grip on its APS alumni, wins a
lot of federal money in the form of research
grants, executive programs, and numerous
pilots and collaborative institutions. This means
there is no incentive to bite the hand that
feeds. Indeed the opposite.

And it shows. At times this borders on self
congratulation and a dangerous master-of-the
universe attitude, that might be warranted if
the APS was an exemplar for modern
government, but delusional if it isn’t. Sit in on
ANU Chancellor Gareth Evan’s annual Crawford
School policy confab and you will hear a lot of
intelligent people ruminating about “Vatican
City” type issues, that feels very distant f rom
the ferment that is deeply disrupting the real
world.

There are no dumb-dumbs in this latest
secretarial cohort, with lots of f irst-class
honours, masters degrees and PhDs, but other
than new Social Services boss Kathryn
Campbell, I could not f ind any technical,
engineering, marketing, f inancial or science
credentials. If the next 20 years of the digital
revolution is as disruptive as the f irst 20 then
this again reveals a dangerous narrowness —
and lack of capability — for the group charged
with designing policy, regulation and reform for
this era.

Apologies to Game of Thrones fans, but with
public administration ripe for automation and
an inevitable transformation into an Uber-like
mass data algorithm, the APS feels like the
citizens of Westeros waiting for the white
walkers to come.

In the meantime the work practices have barely
changed since the large typing pools of the
’80s gave way to PCs and local networks.
Stifling and deeply bureaucratic, there is little
incentive for agencies to radically reinvent their
work places and practices for the collaborative
fast moving world typif ied in many newer
businesses and NGO’s.

2 of 5 REDACTED 9:15 PM
The International

Civil Service

Effectiveness
(InCiSE) Index

2017
Users are free to copy, download and print InCiSE content and findings for their own use. Excerpts

from InCiSE reports and publications may also be used, but must be appropriately referenced.
Please cite any uses of this content as: “International Civil Service Effectiveness (InCiSE) Index, 2017”.

The Index is comprised of a variety of other data sources. Reuse is not applicable to data subject to

intellectual property rights of third parties. Please refer to other organisations’ corresponding websites

and data licensing restrictions to ensure compliance with their data limitations. Details of data sources

can be found in the InCiSE Technical Report. Every effort has been made to adhere to third party

data regulations.

Please contact incise@instituteforgovernment.org.uk for any queries.

Acknowledgements: Our thanks go to those who have given their time to shape this inaugural pilot

publication. This has included substantial contributions made by academics, think-tanks, international

organisations (particularly the OECD), and civil servants past and present. Our thanks also to those

organisations who have allowed use of their data in the Index.
The International Civil Service Effectiveness Index 1

Contents

Foreword from InCiSE founding organisations 3

Executive summary 4

Chapter 1: Introduction and background 7

1.1 Why we need civil service effectiveness indicators 7

1.2 How InCiSE has been developed 8

1.3 What InCiSE is not aiming to do 8

1.4 Who InCiSE is for 8

Chapter 2: The InCiSE framework and approach 9

2.1 Defining the InCiSE framework 9

2.2 Measuring core functions 9

2.3 Measuring attributes 9

2.4 Indicator definitions 11

2.5 Data availability and limitations 12

2.6 Data sources 13

2.7 Country coverage 13

2.8 Country metrics issues 13

Chapter 3: InCiSE index results 15

3.1 Overview 15

3.2 InCiSE overall scores and rankings 17

3.3 Top 5 country rankings by indicator 17

Chapter 4: InCiSE index country results summary 23

Chapter 5: Next steps and conclusion 55

5.1 Next steps 55

5.2 Conclusion 56
2 The International Civil Service Effectiveness Index

Appendices 57

Annex A: InCiSE indicator definitions 58

Core functions 58

Attributes 58

Annex B: InCiSE indicator data sources 59

Annex C: InCiSE country rankings data 62

Annex D: InCiSE indicator scores summary by country 63

Annex E: Summary graphs of country scores and rankings for each indicator 64

Core functions 64

Attributes 72

Annex F: Reader’s guide 76

F.1 Abbreviations 76

F.2 Country codes 77
The International Civil Service Effectiveness Index 3

Foreword from InCiSE

founding organisations

We are pleased to come together to launch the InCiSE is not claiming at this stage to be

International Civil Service Effectiveness (InCiSE) a comprehensive measure of civil service

Index – an exciting new initiative to help countries performance. Some countries and data are

determine how their central civil services are missing which prevent the Index from being as

performing and learn from each other. robust and comparative as we would wish. We
hope partners will help us strengthen and refine

An effective civil service plays an important this inaugural Index over time. We will encourage

role in driving forward a country’s progress and further data collection in areas where it is lacking

prosperity. An ineffective one can act as a brake and will actively support these efforts.
on these things. Yet it is not a straightforward

task to assess whether a civil service is Our long term goal is to broaden the scope of

performing well or how it might improve. InCiSE and establish a robust Index which can
be produced year on year, as well as expand

In an attempt to address this, InCiSE provides its country coverage. We have committed to

a realistic set of comparative information supporting its development for a further four

drawn together from the wealth of existing years.
data available globally. The first inCiSE results,
covering 31 countries, are set out in this report. InCiSE has already brought together a significant
volume of data and insights. We hope the

InCiSE is primarily a performance improvement launch of this report will stimulate wide-ranging

tool, enabling senior decision makers to see discussions globally about how civil services can

which countries perform best in which areas and improve their effectiveness, as well as how to

learn from them. We hope it will also serve as an improve the usefulness of the InCiSE data. We

accountability tool, allowing citizens, government would welcome feedback; contact details can be

officials and politicians to establish in a concise found at the front of this report.
way how well their civil service is functioning.

InCiSE has been developed following a literature

review and in consultation with many experts.
It has also been the subject of an independent

peer review which scrutinised the methodology

without knowing the individual country results.
We are grateful to all those who have given

their time to shape our approach and helped

to produce this inaugural report. We are also

grateful to the many organisations who have

made the Index possible by allowing us to use The International Civil Service Effectiveness

their data. (InCiSE) Index project is a collaboration between
the Blavatnik School of Government and the
Institute for Government. The project has been
supported by the UK Civil Service and is funded
by the Open Society Foundations.
4 The International Civil Service Effectiveness Index

Executive summary

Why and how InCiSE has InCiSE has been the subject of extensive
development and consultation. This has included a

been developed
literature review and discussions with many experts

The International Civil Service Effectiveness and international organisations. InCiSE has also
(InCiSE) Index has been created to help determine been the subject of an independent, international

whether civil services globally are performing peer review process (during which country

effectively and in which areas, relative to their results were provided in an unnamed format).
international counterparts. This matters because

civil services have an important role to play in Scope of the InCiSE index

helping their countries to prosper.
InCiSE is focused on the central government civil

Previous initiatives have sought to develop service only in the countries covered. It does not

measurement tools of this kind but there is seek to measure service delivery outcomes for

currently no other global index available which citizens, for example healthcare and education,
provides a comprehensive assessment of civil because effectiveness is often driven by other

service effectiveness. There are, however, many parts of the public sector as well.
existing data surveys and indexes available

globally that could be pulled together to provide InCiSE is not claiming at this stage to be a

a realistic set of information on an annual basis. robust, comparative measure of civil service

This is what InCiSE aims to do. performance, mainly because of the limited
availability of some data. This inaugural Index

By providing a set of indicators (each measuring should therefore be seen as a pilot product

a different dimension of civil service effectiveness) which will be refined and improved over time.
and then an overall score and ranking for each

country, InCiSE can serve as:

• a performance improvement tool to enable
civil service decision makers to find out which
countries perform best in which areas and
learn from them.

• an accountability tool which allows citizens,
government officials and politicians to find
out how well their civil service is performing.
The International Civil Service Effectiveness Index 5

How the InCiSE framework has Only two countries currently have available data
that covers the full set of metrics – Norway

been compiled
and UK. However, several have close to the

A detailed explanation of the InCiSE framework full complement and any missing data has

is set out in a separate Technical Report. Its been estimated using standard methods.
starting point is to define the core characteristics Countries with the highest missing data points

of an effective, central government civil service. inevitably have a larger proportion of estimated

To do this, InCiSE assesses effectiveness based metrics and this should be borne in mind when

on two interrelated components: interpreting results.

• Core functions: these are the core things
that civil services deliver in each country
Pilot Index results
(‘what’). There are 11 core functions, detailed Canada is ranked top overall of this inaugural
in Annex A. Index, followed by New Zealand and Australia
respectively. When scores are adjusted for GDP
• Attributes: these are the main characteristics
per capita (to take account of countries’ relative
across every part of a civil service which are
wealth and thus the potential resources available
important drivers of the ways in which core
for civil service operations) then Estonia is ranked
functions are delivered (‘how’). There are 6
top overall, followed by Mexico and New Zealand
attributes, detailed in Annex A.
respectively.
Most, but not all, of the InCiSE indicators have
No country consistently appears in the top

been measured to produce this inaugural Index.
5 positions for every indicator, although there

This is largely because of gaps in existing
are some strong all-round performers and

data. So far 8 out of the 11 core functions
these are highlighted in the individual country

proposed have been measured, and 4 out of
assessments. There are some stand-out

the 5 attributes. The quality of data also varies
scores for specific indicators which have been

from country to country. Nevertheless, the initial
highlighted as well.
results already provide some useful insights in the

countries that the InCiSE Index currently covers. When analysing results, it is important to keep in
mind that all country scores are relative to others

Many of the datasets which InCiSE draws on are
included in the Index, not an absolute measure.
updated annually. This should enable the InCiSE
A country which scores well against a particular

Index to be revised on a regular basis to reflect
indicator or theme may still have performance

recent country developments.
issues. The Index provides an important
opportunity to learn from other countries.
Country coverage
Some interesting trends can be observed

To achieve its goals and maximise impact, already. For example, some countries generally

InCiSE aims to cover a diverse and growing score more strongly against attribute indicators

range of countries over time. The speed of rather than core functional ones, and vice versa.
expansion will largely be determined by the Nevertheless, of the top 10 countries performing

availability of data sets and collections which well against core functions, 8 also feature in the

currently vary considerably. This pilot Index top 10 for attributes.
therefore comprises 31 countries where at least

75% of the data needed is available.
6 The International Civil Service Effectiveness Index

Next steps Data collection plans by other organisations
may help to fill some of the data gaps in future,
The founding organisations have committed while in other areas additional data collection

to supporting the development of the InCiSE may be needed. The project will strive to close

framework and Index for a further four these gaps as InCiSE evolves, as well as actively

years if this inaugural Index is well received. support others’ efforts.
An International Advisory Panel will also be

established to guide this work. One of the Changes to the InCiSE framework will be

founders, the Blavatnik School of Government determined mainly through feedback from
(at Oxford University in the UK), will host an a wide range of interested partners. The

international conference in September 2017 new International Advisory Panel will also

to discuss the pilot Index results and future be consulted. InCiSE contact details can be

direction of the project. found at the front of this document to provide
feedback.
Priority tasks over the next 12 months to improve

InCiSE will include: strengthening data collection

and filling gaps; refining the InCiSE methodology

and framework; expanding the country coverage;
examining the potential to include non-OECD

and developing countries over time; developing

an interactive website; and increasing the InCiSE

partner institutions and network through advice,
expertise and funding.
The International Civil Service Effectiveness Index 7

Chapter 1: Introduction

and background

1.1 Why we need civil service The creation of a new and concise set of
indicators would serve as:
effectiveness indicators
• An accountability tool: allowing citizens,
An effective civil service can play an
government officials and politicians to

important role in determining a country’s
establish in a clear and concise way how

progress and prosperity. But what
well their civil service is performing.
constitutes an “effective civil service” in

the 21st Century? And once a consensus • A performance improvement tool: enabling

has been reached on defining this, how senior decision makers to see which

do civil service leaders know whether their countries perform best in which areas,
organisations are effective, and in which and learn from them.
areas are they performing more strongly

than others? The need to develop a new and more
comprehensive set of indicators has grown

The InCiSE Index seeks to help answer following the launch of the new United Nations

these questions. Although a comprehensive (UN) Sustainable Development Goals. InCiSE

set of international indicators of civil service has the potential to help countries report on

effectiveness does not currently exist, various progress towards some aspects of Goal 16

organisations have sought to develop reliable which focuses on a range of governance issues,
measurement systems. including promoting stronger government
institutions. The World Bank’s 2017 World

This subject area is well recognised in
Development Report on Governance and the

academic, international and practitioner
Law also highlights the need for a greater focus

communities as a highly complex area
on improving critical government functions.
for analysis. This is partly because of

data limitations and different views on The InCiSE project is a collaboration between

the definitions of “civil service” and the Blavatnik School of Government and the
“effectiveness”, as well as the need to take Institute for Government. It has been supported

account of country context factors when by the UK Civil Service and is funded by the

looking at performance. Nevertheless, Open Society Foundations. All have a strong

there are many existing data surveys and interest in civil service performance issues. The

indexes available globally that could be long term goal is to broaden the scope of InCiSE

pulled together to provide a realistic set of and establish a sound Index which is capable of

information on an annual basis. This is what being produced year on year, as well as expand

the InCiSE framework aims to do. the country coverage.
8 The International Civil Service Effectiveness Index

1.2 How InCiSE has been developed 1.3 What InCiSE is not aiming to do

Although the InCiSE framework covers many Given the limited availability of some data and

aspects of civil service performance which complexity of the subject area, InCiSE is not

earlier initiatives of this type have sought to claiming at this stage to be a robust, comparative

cover, it does not duplicate them. InCiSE aims measure of civil service performance. It is

to define “effectiveness” more extensively than therefore important to view the initial InCiSE

previously. It draws on a wide range of existing framework and Index as pilot products only, to

international data sources and brings together be refined and improved over time. It is hoped

a set of indicators – each measuring a different that the launch of InCiSE will encourage further

dimension of civil service effectiveness – and data collection in areas where it is lacking and the

then produces an overall score. project will actively support these efforts.

Whilst there are alternative ways to define civil It is also important to be clear about the scope

service effectiveness, the InCiSE framework, with of InCiSE:
its various themes and measurements, has the

potential to make valid judgments about whether • InCiSE is focused on the central government

a country’s civil service is performing well relative civil service in the countries covered – not the

to its international counterparts. public service more generally. It is recognised
that this is not without challenges, particularly

InCiSE has been developed following a given the varying sizes and shapes of public

literature review and in consultation with many administrations internationally, as well as

experts, including academics from schools of different systems of government. It is also

government, think-tanks that monitor government recognised that well performing civil servants

effectiveness, international organisations, are a necessary but not sufficient condition

senior civil servants (past and present) from for better government.
several countries, as well as subject experts.
• InCiSE is not seeking to measure service

InCiSE has also been the subject of an delivery outcomes for citizens, for example in

independent, international peer review process. areas like healthcare and education. Although

Three peer reviewers were selected: a senior in many countries the central government civil

academic from a major European governance service plays an important role in determining

research institute; a recently retired top civil the performance of key public services,
servant with practical experience of civil service their effectiveness is driven by other parts of

performance issues; and a senior governance the sector as well – teachers, doctors, the

expert in two major international institutions. police etc. Local government effectiveness is

Country results were provided in an anonymous also excluded.
form for the review. Between them, the

peer reviewers were asked to examine the • InCiSE does not aim to be definitive. It is one

measurement framework of ‘effectiveness’, of a wide range of tools currently available to

the methodology and approach used to produce measure civil service effectiveness globally.
the indicators, and the data being used.
1.4 Who InCiSE is for
InCiSE is expected to be of interest to a wide
audience – governments (civil servants and
ministers), Parliamentarians, think-tanks,
academics, civil society organisations, the media
– and of course citizens.
The International Civil Service Effectiveness Index 9

Chapter 2: The InCiSE

framework and approach

This chapter sets out the overall approach taken • Service delivery functions where central

to develop the InCiSE framework and produce the government civil services interact more

pilot Index, as well as country coverage issues. directly with citizens (eg. tax and social

Further details about the choice of indicators, their security administration, digital services).
definitions, data availability and quality issues, can

be found in a separate Technical Report. • Mission support functions which enable a
civil service to do its job (eg. finance, human
resource management (HRM), information

2.1 Defining the InCiSE framework technology (IT), procurement).
The InCiSE framework starts by defining the core
By looking across at all three types of function,
characteristics of an effective, central government
the aim is to measure how well civil services

civil service. To do this, it assesses effectiveness
deliver the core parts of what they do. Figure 1

based on two interrelated components:
shows the eleven core functions included in the
• Core functions: the core things that civil InCiSE framework.
services deliver in each country (‘what’).
2.3 Measuring attributes
• Attributes: the characteristics across every
part of a civil service which are important Every civil service has an underlying set of
drivers of the ways in core functions are behavioural characteristics or traits which
delivered (‘how’). are important drivers of how effectively core
functions are delivered, for example levels of

2.2 Measuring core functions openness, integrity and inclusiveness. These
attributes should apply to all parts of the civil

Civil services across the world vary widely in their service and should not be limited to specific

shape and size and are responsible for different core functions.
things. However, there are certain core functions

which they all deliver for the governments Cultivating and displaying these attributes is

and citizens that they serve. The approach commonly (though not necessarily universally)
InCiSE takes is to focus on three interrelated understood to be good practice. They are also

types of core function to measure civil service key determinants of an organisation’s overall

effectiveness more comprehensively: effectiveness. Figure 2 shows the six attributes
included in the InCiSE framework.
• Central executive functions for ministers,
the effects of which are felt by citizens (eg.
policy making, fiscal management, regulation,
crisis/risk management).
10 The International Civil Service Effectiveness Index

Figure 1: InCiSE Core Functions

Central executive Mission support Direct service delivery
Policy making Procurement Tax administration

Fiscal and financial HR management Social security
management administration

Regulation Information technology Digital services

Crisis/risk management Finance

Figure 2: InCiSE Attributes

  1. Integrity 4. Inclusiveness
  1. Openness 5. Staff engagement
  1. Capabilities 6. Innovation

There are some concepts which could be considered attributes but which are particularly relevant

to some core functions. For example, the pursuit of value for money (VFM) is a key feature of the

procurement function, while the use of evidence plays an important role in effective policy making.
Where feasible, these concepts have been captured through the core functions of the InCiSE

framework instead.
The International Civil Service Effectiveness Index 11

2.4 Indicator definitions

A brief definition of each of the InCiSE indicators can be found at Annex A of this report. Further

details, including justifications for including these indicators in the InCiSE framework, can be found

in the Technical Report.

A visual summary of the InCiSE framework and its context is displayed in Figure 3 below.

Figure 3: The InCiSE Framework

Overall Measurement framework Outcomes
inputs (process and output focussed) for citizens

  1. Total human Civil Service
    resource
  2. Total financial Central Mission Direct
    resource
    executive support service
    functions functions delivery
    Outputs functions Outputs
    to ministers to citizens
  3. Policy making 1. Procurement 1. Tax administration
  4. Fiscal and 2. HR (at the central/
    financial management federal level)
    management
  5. IT 2. Social security
  6. Regulation administration
  7. Finance
    Ministers 4. Crisis/Risk (at the central/ Citizens
    management federal level)
  8. Digital services

Attributes

  1. Integrity 2. Openness 3. Capabilities 4. Inclusiveness
    Policy effects 5. Staff engagement 6. Innovation Policy effects

    (‘Value for money’ and ‘use of evidence’ are other key attributes
    identified. However, these concepts are already captured on the
    functions side of the framework to the extent that their inclusion
    as attribute indicators is considered unnecessary.)
    12 The International Civil Service Effectiveness Index

2.5 Data availability and limitations

Most – but not all – of the InCiSE indicators have been measured to produce the pilot Index.
This is because of gaps in existing data. Eight out of the eleven core functions proposed have been

measured and included, and four out of the six attributes. These are set out below in Figure 4.

Figure 4: InCiSE core functions and attributes being measured for pilot Index

Measured and included in pilot index Not yet measured or included in pilot index

Central executive functions Mission support functions
Policy making Procurement
Fiscal and financial management (FFM) Information technology (IT)
Regulation Finance
Crisis/risk management
Attributes
Mission support functions Staff engagement
Human resource management (HRM) Innovation

Direct service delivery functions
Tax administration
Social security administration
Digital services

Attributes
Integrity
Openness
Capabilities
Inclusiveness

Data collection plans by other organisations may help to fill some of the gaps in future. In other areas,
additional data collection may need to be initiated to measure the relevant indicator. The project will

strive to close these data coverage gaps as InCiSE evolves. Specific actions are set out in the final

chapter of this report.
The International Civil Service Effectiveness Index 13

2.6 Data sources 2.8 Country metrics issues

A wide range of existing data sources have Whilst only two countries currently cover the full

been used to compile the pilot Index and a set of metrics – Norway and UK – several have

detailed summary is provided at Annex B. close to the full complement and any missing

Further details on how these data sources have data has been estimated. Countries with more

been developed to measure each indicator are missing data points inevitably have a larger

available in the Technical Report. proportion of estimated metrics and this factor
should be borne in mind when interpreting

A wealth of data underlies the 12 indicators results. Further details about the methodology

measured so far. Each indicator has one or more used for estimated data can be found in the

broad themes and these are then assessed via Technical Report.
a specific set of performance measurements or

metrics. In total, 76 metrics are spread across Table 1 highlights the availability of InCiSE

the indicators – although some metrics are metrics for each of the 31 countries in this

themselves indices, meaning that the actual pilot index, data availability for each of the 12

number of ‘total metrics’ is far higher. indicators, and where data has been estimated.

Many of the datasets from which the metrics are The development of the InCiSE Index has involved

drawn are updated annually, enabling the Index stages where subjective judgements have had

to be revised on a regular basis to reflect country to be made, for example in aggregating the

developments. This iteration includes data up results and dealing with gaps in data. The impact

until January 2017. on Index results of changing some of these
judgements is explored in the Technical Report.
2.7 Country coverage

To realise its goals and maximise impact, InCiSE

aims to cover a diverse and growing range of

countries over time. However, InCiSE is reliant

on the availability of a wide range of data sets

and collections – and this currently varies

considerably from country to country. Expanding

the range of countries too quickly would require

a large amount of data estimation, or a reduction

in the scope of the framework, or greater reliance

on civil service proxy indicators, which may

undermine the results.

To take account of these issues, any country

with less than 75% of the data that the InCiSE

Index needs has been excluded. The pilot Index

therefore comprises 31 countries where data

availability met this condition.
14 The International Civil Service Effectiveness Index

Table 1: Country coverage and availability of InCiSE metrics

Green squares indicate data was available for all metrics within the indicator for a specific country;
amber squares that data was available for some metrics only; and red squares where no data was

available and so all metrics were estimated.
Country Metric Attributes Direct Service Delivery functions Mission Central Executive functions
account (34 metrics) 26 metrics Support (11 metrics)
(out of 76) functions
(5 metrics)
Integrity Openness Capabilities Inclusiveness Policy Fiscal & Regulation Risk Man. Human Tax Admin Social Digital
(16) (9) (4) (5) making Fin. Man. 6 9 Resources (6) Security Services
8 3 (5) Admin (4)
(1)

NOR 76

GBR 76

AUT 75

FRA 75

ITA 75

NLD 75

FIN 74

DEU 74

DNK 73

POL 73

SWE 73

CZE 72

ESP 72

SVN 70

PRT 69

SVK 69

HUN 68

TUR 68

BEL 67

AUS 66

NZL 66

EST 65

MEX 65

CHE 65

GRC 64

CAN 62

IRL 60

JPN 59

KOR 59

CHL 58

USA 57
The International Civil Service Effectiveness Index 15

Chapter 3: InCiSE

Index results

This chapter presents the pilot Index results 3.1 Overview

in several ways:
Developing a comprehensive range of indicators
• Overall scores and rankings – presented by means that there is often a wide variation in
use of graphs. how countries perform against each of them.
• Top 5 country rankings – these tables rank No country consistently appears in the top
the top 5 countries for each indicator. 5 positions for every indicator, although there
are some strong all-round performers and
• A visual summary of individual country these are highlighted in the individual country
scores – presented via radar diagrams. assessments. There are some stand-out scores
against specific indicators which have been

Annex C of this report provides a summary table
highlighted as well.
of overall country rankings, as well as rankings

for each indicator. Annex D contains a summary There are some already interesting trends to be

of indicator scores for each country. observed. For example, some countries generally
score more strongly against attribute indicators

Annex E contains summary graphs of country
rather than functional ones, and vice versa.
scores and rankings for each indicator.
Nevertheless, of the top 10 countries performing
well against core functions, 8 also feature
in the top 10 for attributes. GDP per capita
adjustments to the overall scores result in some
shifts in the rankings as well.

As yet there are no clear regional patterns
emerging, although the current dominance of
European countries in the Index does enable
some direct comparison. As the methodology gets
refined and the InCiSE country coverage begins
to expand it may be possible to analyse regional
patterns and other trends in future reports.
16 The International Civil Service Effectiveness Index

Figure 5: InCiSE index overall scores and rankings

Figure 6: InCiSE index overall scores and rankings (adjusted for GDP per capita)
The International Civil Service Effectiveness Index 17

3.2 InCiSE overall scores 3.3.1 Top 5 rankings: core functions

and rankings Few patterns emerge within this set of indicators

The graph in Figure 5 shows the overall score and a wide range of countries appear across

and ranking for each country. All scores are the tables.
relative, not absolute, and so the maximum score
a) Policy making

that the top country can achieve is 1.0 while the

lowest score is zero. The same scoring system This indicator currently has four themes: the

applies for each indicator. The methodology quality of policy advice; the role of civil servants

used to obtain these scores is explained in the in setting strategic policy direction; policy

Technical Report. proposal coordination across government;
and monitoring policy implementation.
The graph in Figure 6 adjusts each overall

country score in line with GDP per capita to Some proxy metrics have been used for

take account of its relative wealth, and thus the measuring the quality of policy advice. A fifth

potential resources available for civil service theme, assessing the timeliness and accuracy

operations. The overall levels of civil service of policy delivery, will be added when data

resources invested, human and financial, may becomes available. All data for this indicator

influence the performance of most core functions is drawn from the Bertelsmann Sustainable

and attributes. Governance Indicators (SGI).

Once the GDP per capita adjustment is made,
the top 10 country overall scores and rankings Country Rank

shift, although not dramatically. Only three new UK 1

countries appear in the top 10, most notably USA 2

Turkey, Mexico and Chile who were previously
Denmark 3

in the second half of the Index.
Finland 4

Regional location does not appear to be the
Canada 5

deciding factor in determining levels of civil

service effectiveness, with the first 5 countries
The policy making scores for all 5 countries are

coming from diverse parts of the world, including
very high and close. The UK does particularly

when adjusted for GDP per capita.
well on the quality of policy advice, including
whether it is evidence based, as does the USA.
3.3 Top 5 country rankings The USA also does particularly well on the policy

by indicator coordination theme.
This section provides a brief summary of For the other three countries in this table, all

each indicator and how the top five countries score well on strategic policy direction (Denmark,
performed (prior to GDP adjustment). Finland and the UK are joint top), also on the
quality of policy advice and policy coordination

The top 5 scores across most core function
themes. There is greater variation in the policy

indicators are relatively close, while there are
monitoring scores for these countries.
wider variations in the attribute scores. This

difference may reflect some of the data gaps

and limitations described earlier. Where there

is a greater variation in scores, rankings are more

significant in terms of helping countries to learn

from each other.
18 The International Civil Service Effectiveness Index

b) Fiscal and financial management (FFM) Country Rank

This indicator has four themes: economic Mexico 1

appraisal; economic evaluation; medium-
UK 2

term budgeting processes; and performance

budgeting. Australia 3
Canada 4

Proxy measures have been used for some

aspects. Both the economic appraisal and Switzerland 5

evaluation themes are measured by the World

Economic Forum Global Competitiveness Index Mexico’s top position for this indicator may in
(GCI). The two budgeting themes are measured part stem from the country’s sustained focus on

by two relevant OECD indexes. regulatory policy reforms over the last decade.
Using the OECD data, Mexico scores very highly
across all themes, coming top on both metrics
Country Rank for stakeholder engagement, second for impact
Switzerland 1 assessment, and third for evaluation.
Netherlands 2 The UK achieves the top position for the impact
Sweden 3 assessment theme while Australia is top for
evaluation. Canada and Switzerland score
New Zealand 4
strongly against both the impact assessment
Republic of Korea 5 and stakeholder engagement themes.

Switzerland performs well across all four d) Crisis/risk management

themes, with its highest position (4th) being for This indicator has five themes relating to disaster

performance budgeting. It is worth noting that risk reduction and management: integrated risk

Switzerland tops the Global Competitive Index planning; risk monitoring; public information and
(GCI) and has done so for eight consecutive awareness raising; international cooperation and

years. It also does well in the OECD indexes risk coordination; and post-disaster assessment

used for this indicator. This consistently strong methodology.
performance may in part reflect the Swiss

government’s decision over a decade ago A sixth theme on preparedness for disaster

to overhaul its accounting system in view of response will be added when data becomes

growing financial management demands. available. The data source for all themes is the
UN Hyogo Framework for Action.
Looking at the other country positions in this

table, it is worth noting that the Netherlands ranks

4th overall on the GCI and achieves the top Country Rank

position for the OECD medium-term budgeting Turkey 1

metric. Sweden also does well against this
Finland 2

metric, while the Republic of Korea tops the

OECD performance budgeting index. Slovenia 3
Australia 4

c) Regulation
Sweden 5

This indicator has three themes relating to

regulation policies and management: method, The top 5 country scores for this indicator are

use and quality of regulatory impact assessments; all very high. This may relate to the way in which

stakeholder engagement; and evaluation work. some data relates to adherence to best practice

The sole data source is the OECD’s Indicators in crisis management (with most countries being

of Regulatory Policy and Governance. compliant already).
The International Civil Service Effectiveness Index 19

Turkey’s top ranking for this indicator reflects Japan’s high scores for the meritocratic theme

its well recognised leadership and experience confirm its highly competitive civil service

of disaster risk planning and management. entry systems, including the use of formal

It comes joint top for the integrated risk planning examinations (it came second for this metric).
theme (i.e. the extent to which disaster risk is Both countries also do well against two other

incorporated in national policy planning). Turkey metrics for this theme which assess the extent

also does well on risk monitoring, demonstrating to which personal or political connections

a consistent and systematic approach to determine who gets a job.
reporting. In addition, Turkey comes top for

two of the metrics used to measure the public Both Ireland and Japan do well on attracting and

information dissemination and public awareness retaining talent as well. A single metric is used to

strategies theme. assess the extent to which senior officials have
salaries that are comparable with similar jobs in

e) Human resource management (HRM) the private sector. Ireland ranks 4th and Japan

This indicator currently measures two themes: 6th for this metric.
the extent to which civil service recruitment For other countries in the top 5, Canada scores

systems are meritocratic; and attracting and strongly against the metric covering the skills

retaining talent. and merits of job applicants. Belgium and the

InCiSE hopes to assess four additional themes Republic of Korea do well on the extent to which

when data is available: talent deployment; public sector employees are hired using a formal

performance management; the quality of examination system (Belgium comes top for

learning and development; and the level of civil this metric).
servant satisfaction with HR services. For the f) Tax administration

two themes measured, the data source is the

Quality of Government survey. This indicator currently measures three themes:
the overall efficiency of tax collection; the extent
to which services are user focused; and the
Country Rank extent and the quality of digital provision.
Ireland 1 Two additional themes will be assessed when

Japan 2 data becomes available: the prevention of tax

Canada 3 evasion; and the level of tax gap measurement.
The existing data sources are the OECD’s Tax

Belgium 4
Administration Survey and The World Bank’s

Republic of Korea 5 Doing Business Index (DBI).

Ireland only just beats Japan for the top position,
edging ahead on the attracting and retaining Country Rank

talent theme. Ireland’s overall top position Estonia 1

may reflect its centrally administered systems, 2
Norway

including at relatively low grade levels, plus its

strong regulation of civil service appointments Denmark 3

and increased use of external recruitment. Ireland 4
UK 5

Estonia’s top position reflects a strong all-round
performance against the three themes measured
and there may be a connection between
Estonia’s top ranking for this indicator and the
one for digital services.
20 The International Civil Service Effectiveness Index

Estonia’s scores are very high for the OECD h) Digital services

sourced e-filing metrics and the World Bank This indicator measures four themes: the

and OECD sourced metrics covering cost of extent to which services are user focused; the

tax collection ratios. transparency of services; cross-border mobility

All five countries perform well across of services; and the availability of key enablers

all the metrics for the three themes and their (technical advances made possible through,
overall scores are very close. It is also notable for example, infrastructure investment). It does

that all appear in the top 10 of the World Bank not assess all the services which governments

DBI measure on the time it takes businesses typically provide digitally.
to pay taxes. The sole data source is the European

g) Social security administration Commission’s E-Government Benchmark.
Estimated data was used for most non-EU

This indicator is currently measured via one
countries.
theme/metric only: assessing administration

costs as a proportion of total expenditure on

social security, to capture the overall efficiency Country Rank

of the system. Estonia 1

Additional metrics are planned Austria 2

for future reports as more data becomes Denmark 3

available, focused on the extent to which
Australia 4

services are user focused, the extent and

quality of digital provision, and the prevention Finland 5

of fraud. The existing data source is the

European Commission (Eurostat). Estimated Estonia’s overall score for this indicator is some

data was used for most non-EU countries. way ahead of the other countries in the top
5 table and it does well across all four themes.
This could reflect the country’s strong digital
Country Rank policies and investment in digital infrastructure.
UK 1 Estonia comes top for two themes (key enablers
2
and transparency of services), joint top for user

Hungary
focused services, and second for cross-border

Estonia 3 mobility.
Portugal 4
The other top 5 countries’ scores are close

Slovenia 5 together. They generally perform well across all
four themes and all are particularly strong on the

The top 5 scores for this indicator are very close.
degree to which services are user focused.
This could reflect the narrow way in which this

core function is currently measured, benefiting

those countries where administrative costs 3.3.2 Top 5 rankings: attributes

as a proportion of social security expenditure The attribute rankings show some clear country

are low. This is also an indicator where further patterns emerging, with Canada appearing in

investigation may be needed on the value of four out of the five tables and Finland and New

comparing social security systems internationally Zealand in three of them. There are also some

as they can vary widely. These issues should be stand-out top positions, for example Japan for

borne in mind when interpreting results. the capabilities attribute. Poland and Slovenia’s
high inclusiveness scores are notable as well.
The International Civil Service Effectiveness Index 21

a) Integrity b) Openness

This indicator covers 6 themes: corruption level This indicator has six themes: the degree

perceptions; adherence to rules and procedures; and quality of consultation with society;
work ethics; fairness and impartiality; striving the existence and quality of complaint

to serve citizens and ministers; and processes in mechanisms; government data availability and

place to preserve integrity and prevent conflicts accessibility; government data impact; the right

of interest. to information; and the publication of laws.

There are four data sources: Transparency There are six data sources: the World Justice

International’s Global Corruption Barometer Project’s Open Government Index (OGI); the
(GCB); the World Economic Forum (WEF) Global UN E-participation Index (EPI); Bertelsmann

Competitiveness Index (GCI); the Quality of SGIs; the World Wide Web Foundation: Open

Government (QoG) Expert Survey Data; and the Data Barometer (ODB); The Open Knowledge

OECD’s “Government at a Glance” (GaaG) data. Foundation: Open Data Index (GODI); and the
OECD’s OURdata index.
Fifteen countries have some missing data, partly

because of the large number of metrics for

this indicator (16), and so estimated data has Country Rank

been used. UK 1
New Zealand 2
Country Rank Norway 3

New Zealand 1 Denmark 4

Sweden 2 Finland 5

Norway 3
4
All the top 5 countries for this indicator feature

Canada
in the top 10 of Transparency International’s

Denmark 5 Corruptions Perception Index 2016 (Denmark,
New Zealand and Finland are in the top 3 places

New Zealand’s overall score is well ahead of respectively, Norway is 6th and the UK is in

the others in this top 5 table, reflecting a strong 10th place). It is worth noting that this is the only

performance for all metrics. New Zealand does attribute where Canada does not appear in the

particularly well on the corruption perceptions top 5, although it still performed well against

and work ethic themes. It comes top or joint most of the relevant metrics.
top in 6 out of the 16 metrics for this indicator,
and top in at least one metric in 4 out of the 6 The UK and New Zealand’s overall scores for

themes. this attribute are very close. The UK scores
more highly on two themes – government data

For all other countries in this table the overall availability and accessibility, plus government

scores are very close. data impact. New Zealand comes top on the
publishing laws theme and 2nd on the right
to information theme. Both countries score
particularly well on the public consultation theme,
as do others in the top 5 table.
22 The International Civil Service Effectiveness Index

c) Capabilities d) Inclusiveness

This attribute currently measures two themes: This indicator currently has only two themes:
core capability (eg. problem solving, numeracy the proportion of women represented in the

and literacy skills); and the educational civil service; and ethnic/religious minority

attainment of the workforce. representation. InCiSE hopes to broaden the
coverage over time once data becomes available

Additional themes are planned for this indicator for other key representation categories.
once data becomes available to include

leadership, commercial, analytical and digital There are two data sources: the OECD’s

capabilities. The sole data source is the OECD Government at a Glance (GaaG) survey; and

Programme for the International Assessment of the Quality of Government (QoG) survey.
Adult Competencies (PIAAC) survey. A high level

of estimated data was used for some countries.
Country Rank
Poland 1

Country Rank
Slovenia 2

Japan 1
Canada 3

Finland 2
Finland 4

Canada 3
Australia 5

New Zealand 4

Switzerland 5 Poland’s top position reflects its strong score
for the metric which assesses the proportion of

Japan’s capabilities score is strikingly ahead women in central government senior positions,
of the other countries in this top 5 table. as well as ethnic minority representation. Slovenia

This reflects its scores in the OECD survey also scores well against one of the metrics

concerning the proportion of the public sector concerning senior positions and tops the ethnic

with high literacy, numeracy and problem solving minority representation metric. Canada scores

skills, where it tops the three rankings by a well across all four metrics for this attribute.
considerable margin. Japan’s score also reflects

its strong position in the same survey concerning

the proportion of the public sector with tertiary

education (ranked 8th – Canada is top). It is

notable that Japan scores highly against the

HRM indicator as well.
The International Civil Service Effectiveness Index 23

Chapter 4: InCiSE index

country results summary

This section presents a visual summary of each More detailed information about country scores,
country’s indicator scores via the use of radar including the themes and metrics which underlie

diagrams. These highlight where a country each indicator result, can be found on the

performs particularly well compared to other InCiSE website.
countries in the Index and where further attention

or analysis may be beneficial. Further details about the methodological
approach, and examples of interpretations,
Indicator results are presented on a scale of can be found in the Technical Report.
0 to 1, with 0 representing the weakest

performing country in the Index and 1 the best Annex E of this main report shows the country

performing country. Assessment of a country is scores for each indicator, in order of ranking.
therefore relative to others included in the index The key for the country radar graphs is

only, not an absolute measure. Weaker scores as follows:
do not reflect a view on prioritisation within a

country, but rather opportunities to learn from Key

other countries.
Country scores

Average (mean) scores
24 The International Civil Service Effectiveness Index

4.1 Australia

5/26/2017 Radar chart

Tax
administration
Policy
making Inclusiveness

0.75

Social
security Capabilities
0.50

0.25

Digital
services Openness

Fiscal
and Integrity
financial
management

Regulation Human
Crisis resource
management management

Australia is ranked 3rd

overall on the Index and

achieves above average

scores in most areas.
Australia’s regulation score is strong (ranked

3rd overall) and they should have some

useful lessons to share, particularly on the

evaluation theme where they achieve the top

score. Australia is ranked 4th for crisis/risk

management, scoring well across all themes.

Within the attribute indicators, Australia’s

https://incise­radar­vis.cloudapps.digital/ 2/6

strongest ranking is for inclusiveness (5th

overall), suggesting its civil service has a good

representation of women, ethnic and religious

groups relative to most other countries.

The main indicator which requires attention is

fiscal and financial management (FFM), where

most thematic scores are just below average.
The International Civil Service Effectiveness Index 25

4.2 Austria

5/26/2017 Radar chart

Tax
administration
Policy
making Inclusiveness

0.75

Social
security Capabilities
0.50

0.25

Digital
services Openness

Fiscal
and Integrity
financial
management

Regulation Human
Crisis resource
management management

Austria is ranked 16th overall The key core functions which require
further analysis are policy making, crisis/risk

on the Index. The country’s management and HRM where scores are

strongest score is for digital below average.
services (ranked 2nd overall). On attributes, the metrics used to measure the
Austria scores strongly across most digital capabilities indicator suggest that literacy skills
service themes, coming joint top (with Estonia) and educational attainment within the workforce
on the extent to which services are user focused. are relatively low, and there may be some
Austria also scores well on tax and social lessons to learn from high scoring countries.
security administration. There may be a link There may also be a connection between
between Austria’s high digital services score and Austria’s capabilities and HRM scores given the
some aspects of the digital theme under the tax latter indicator includes metrics on attracting and
administration indicator, for example the use of

https://incise­radar­vis.cloudapps.digital/
retaining talent. 2/6

e-filing. Austria scores are above average for the
integrity and openness attributes as well.
26 The International Civil Service Effectiveness Index

4.3 Belgium

5/26/2017 Radar chart

Tax
administration
Policy
making Inclusiveness

0.75

Social
security Capabilities
0.50

0.25

Digital
services Openness

Fiscal
and Integrity
financial
management

Regulation Human
Crisis resource
management management

Belgium is ranked 13th overall Belgium also scores well on policy making,
its integrity score is above average too.
on the Index and achieves

above average scores for The main core function where Belgium

many indicators. performs relatively less well is fiscal and
financial management (FFM), particularly the

Belgium’s strong HRM score (ranked medium-term and performance budgeting

4th overall) suggests it has a relatively themes. Belgium’s openness score also

meritocratic recruitment system and that indicates a below average performance.
the civil service can attract and retain talent.
Belgium comes top for the metric on the

extent to which recruitment is carried out

via a formal examination system.
https://incise­radar­vis.cloudapps.digital/ 2/6
The International Civil Service Effectiveness Index 27

4.4 Canada

5/25/2017 Radar chart

Tax
administration
Policy
making Inclusiveness

0.75

Social
security Capabilities
0.50

0.25

Digital
services Openness

Fiscal
and Integrity
financial
management

Regulation Human
Crisis resource
management management

Canada tops the InCiSE Index Tax administration scores are close to the
average. Canada’s score for overall efficiency

overall and scores consistently of tax collection suggests there may be potential

well above the average for for improvement through greater use of digital

most indicators. processes for tax administration. This would
be in line with high scoring countries for this

Canada is ranked 4th for the regulation indicator, indicator.
with high scores for the stakeholder engagement

and impact assessment themes. It is ranked Canada’s openness score, although well above

3rd for HRM (scoring strongly for the theme the average, suggests there may be some

on the extent to which the civil service has a lessons to learn from the leading countries

meritocratic system) and 5th for policy making. concerning the right to information theme,
as well as the availability and accessibility of

On attributes, Canada appears in the top 5 for government data.
https://incise-radar-vis.cloudapps.digital/ 2/6

all but one indicator (openness). It comes top

for the capabilities metric on the proportion of

the public sector with tertiary education. On

inclusiveness, Canada scores well across most

metrics, suggesting its civil service has a good

representation of women, ethnic and religious

groups relative to other countries.
28 The International Civil Service Effectiveness Index

4.5 Chile

5/26/2017 Radar chart

Tax
administration
Policy
making Inclusiveness

0.75

Social
security Capabilities
0.50

0.25

Digital
services Openness

Fiscal
and Integrity
financial
management

Regulation Human
Crisis resource
management management

Chile is ranked 22nd overall on On attributes, Chile’s integrity score is at
the average for all countries included in the

the Index. Its position rises to Index; it performs particularly well on the

8th when scores are adjusted metric measuring the degree of whistleblower

for GDP per capita. protection.

The core function where Chile scores relatively

Chile scores well on policy making (ranked weakly is regulation, particularly the impact

9th), particularly on the quality of policy advice assessment and evaluation themes and metrics.
theme where it is ranked joint top position. These could be initial areas for attention, learning

Chile also scores well on crisis/risk management from the high scoring countries for this indicator.
(ranked 8th).

https://incise­radar­vis.cloudapps.digital/ 2/6
The International Civil Service Effectiveness Index 29

4.6 Czechia

5/26/2017 Radar chart

Tax
administration
Policy
making Inclusiveness

0.75

Social
security Capabilities
0.50

0.25

Digital
services Openness

Fiscal
and Integrity
financial
management

Regulation Human
Crisis resource
management management

Czechia is ranked 28th overall The main core function where performance
is weak relative to other countries is tax

on the Index. Its strongest administration, particularly the themes concerning

score is for fiscal and financial the degree to which services are user focused,
management (ranked 11th). as well as the extent of digital provision.

Czechia also has above average scores Czechia’s HRM and digital services scores

for regulation, scoring well on the impact also warrant attention across most themes.
assessment and stakeholder engagement All attributes scores require further analysis

themes (ranked 9th-13th for two of the as well.
four metrics).

https://incise­radar­vis.cloudapps.digital/ 2/6
30 The International Civil Service Effectiveness Index

4.7 Denmark

5/26/2017 Radar chart

Tax
administration
Policy
making Inclusiveness

0.75

Social
security Capabilities
0.50

0.25

Digital
services Openness

Fiscal
and Integrity
financial
management

Regulation Human
Crisis resource
management management

Denmark is ranked 11th overall On policy making, Denmark ranks joint top for
the strategic policy direction theme and scores

on the Index. Its strongest highly on the quality of policy advice and policy

attribute scores are for coordination.
integrity where it is ranked A core function which may warrant attention, by

5th and openness where it is considering whether lessons can be learnt from

ranked 4th. high scoring countries, is crisis/risk management,
focusing initially on the metrics within the
Denmark is ranked 3rd for three core function integrated risk planning theme. Denmark’s social
indicators – policy making, tax administration security administration score also merits further
(scoring well across all themes) and digital analysis as the sole metric used for this indicator
services (again scoring well across all themes, suggests lower than average systems efficiency.
particularly the degree to which services are

https://incise­radar­vis.cloudapps.digital/
The main attribute where 2/6
Denmark performs less
user focused).
well relative to other countries is inclusiveness
where scores for the main themes vary.
The International Civil Service Effectiveness Index 31

4.8 Estonia

5/26/2017 Radar chart

Tax
administration
Policy
making Inclusiveness

0.75

Social
security Capabilities
0.50

0.25

Digital
services Openness

Fiscal
and Integrity
financial
management

Regulation Human
Crisis resource
management management

Estonia is ranked 7th overall Estonia is ranked 3rd for social security
administration, suggesting comparatively low

on the Index. It rises to the administrative costs relative to sector expenditure.
top position when scores are
The main indicator requiring attention, to

adjusted for GDP per capita. consider whether lessons can be learnt from
Some indicator scores are especially noteworthy. other countries, is inclusiveness. The metrics
First, digital services where Estonia is ranked overall suggest lower than average women’s and
top and their overall score stands out above other group representation in the civil service.
the rest. It scores highly against all themes
and Estonia’s overall performance could
reflect its strong IT policies and investment in
IT infrastructure. Estonia is also ranked top
for the tax administration indicator. There is a

https://incise­radar­vis.cloudapps.digital/ 2/6

potential connection between these two core
function scores as one of the tax administration
metrics relates to the extent and quality of digital
service provision.
32 The International Civil Service Effectiveness Index

4.9 Finland

5/26/2017 Radar chart

Tax
administration
Policy
making Inclusiveness

0.75

Social
security Capabilities
0.50

0.25

Digital
services Openness

Fiscal
and Integrity
financial
management

Regulation Human
Crisis resource
management management

Finland is ranked 5th overall Finland’s strongest core functions scores are
in crisis/risk management (ranked 2nd overall),
on the Index and achieves policy making (ranked 4th) and digital services

above average scores for most (ranked 5th). For the latter, Finland scores

indicators. well across all themes particularly the extent
to which services are user focused. On policy

Finland’s attributes scores are consistently high. making, Finland is in joint top position for the

It is ranked 2nd for capabilities – suggesting strategic policy direction theme. It also scores

strong technical skills and educational attainment highly for the quality of policy advice and policy

across the workforce relative to other countries coordination themes.
– and 4th on inclusiveness. Finland is ranked 5th The main indicator requiring further attention is

on openness, where it scores particularly well on regulation, focusing on impact assessment as

the public consultation theme. this score was relatively2/6weak compared with the

https://incise­radar­vis.cloudapps.digital/
other themes.
The International Civil Service Effectiveness Index 33

4.10 France

5/26/2017 Radar chart

Tax
administration
Policy
making Inclusiveness

0.75

Social
security Capabilities
0.50

0.25

Digital
services Openness

Fiscal
and Integrity
financial
management

Regulation Human
Crisis resource
management management

France is ranked 17th France’s social security administration score also
merits further analysis as the sole metric used

overall on the Index. It has for this indicator suggests lower than average

above average scores for systems efficiency.
the integrity and openness France’s crisis/risk management scores warrant

attributes. further analysis. It scores well on some themes,
for example integrated risk planning and post

France’s fiscal and financial management (FFM),
disaster risk assessments. Improvements in

tax administration, digital services and HRM core
other themes, for example public information

functions scores are also all above average.
and awareness strategies, may help improve the
On HRM, France scores particularly well on overall score in future.
some aspects of the meritocratic recruitment
theme, for example the use of formal

https://incise­radar­vis.cloudapps.digital/ 2/6

examination systems to hire staff (ranked 4th).

The main attributes requiring attention are

inclusiveness and capabilities, initially by

considering the scope to learn from the leading

countries.
34 The International Civil Service Effectiveness Index

4.11 Germany

5/26/2017 Radar chart

Tax
administration
Policy
making Inclusiveness

0.75

Social
security Capabilities
0.50

0.25

Digital
services Openness

Fiscal
and Integrity
financial
management

Regulation Human
Crisis resource
management management

Germany is ranked 24th overall Germany’s policy making score also requires
further analysis. Its thematic scores vary,
on the Index. It achieves on or whereas high ranking countries for this indicator

above average scores for all tend to do well across most themes. Specific

attribute indicators. themes to consider further include strategic
direction (this is based on an assessment of the

Germany is ranked 8th for integrity and 9th for extent to which strategic planning units influence

capabilities. The latter score suggests a civil government decision making), and policy

service with relatively good technical skills, monitoring.
alongside a high level of educational attainment It is worth noting that Germany has a federal

compared with other countries. government system. An issue for the future is
On core functions, Germany scores highly on the need to ensure the InCiSE framework can
regulation (ranked 7th overall) and does particularly

https://incise­radar­vis.cloudapps.digital/
capture the strengths of2/6different systems.
well on the evaluation theme, ranking 4th and
5th for the relevant metrics. The main indicator
requiring attention is crisis/risk management,
although it should be noted that estimated data
was used for a number of countries.
The International Civil Service Effectiveness Index 35

4.12 Greece

5/26/2017 Radar chart

Tax
administration
Policy
making Inclusiveness

0.75

Social
security Capabilities
0.50

0.25

Digital
services Openness

Fiscal
and Integrity
financial
management

Regulation Human
Crisis resource
management management

Greece is ranked 29th overall Greece’s crisis/risk management score is around
the average, although it performs well on aspects

on the Index. The country’s of the integrated risk planning theme.
high inclusiveness score
The core functions where performance is

stands out (ranked 6th). relatively weaker are policy making, digital

Within the inclusiveness indicator, Greece does services, regulation, and fiscal and financial

particularly well on the metrics for the women’s management. On the latter, Greece’s

representation theme. Greece also scores above performance is only slightly below average for

average for the metric on ethnic and religious the medium-term budgeting theme. However,
groups representation. The remaining attributes its performance was weaker compared with

scores require further analysis to prioritise areas other countries for the performance budgeting,
for improvement. economic appraisal and evaluation themes.
https://incise­radar­vis.cloudapps.digital/
Greece’s social security administration score On digital services, Greece’s
2/6
performance is

is above average as well, suggesting it has assessed to be relatively weak across all themes

reasonable administration costs relative to and there is scope to learn from the practices of

sector expenditure. the leading countries.
36 The International Civil Service Effectiveness Index

4.13 Hungary

5/26/2017 Radar chart

Tax
administration
Policy
making Inclusiveness

0.75

Social
security Capabilities
0.50

0.25

Digital
services Openness

Fiscal
and Integrity
financial
management

Regulation Human
Crisis resource
management management

Hungary is ranked 30th overall Some thematic scores are worth noting.
On policy making, Hungary comes top for

on the Index. The country’s two of the three metrics concerning policy

very high social security monitoring. The coordination of policy proposals

administration score is thematic score is also above average. Hungary
also achieves above average thematic scores

noteworthy. within the integrity indicator. On regulation,
Hungary has above average scores for the

Hungary is ranked 2nd overall for this indicator,
impact assessment theme.
suggesting it has a comparatively efficient social

security system with low administration costs The main core function indicators requiring

relative to sector expenditure. attention are HRM, plus digital services and fiscal
and financial management. All attribute indicators

https://incise­radar­vis.cloudapps.digital/
merit further attention, initially
2/6
by considering the
scope to learn from the leading countries.
The International Civil Service Effectiveness Index 37

4.14 Ireland

5/26/2017 Radar chart

Tax
administration
Policy
making Inclusiveness

0.75

Social
security Capabilities
0.50

0.25

Digital
services Openness

Fiscal
and Integrity
financial
management

Regulation Human
Crisis resource
management management

Ireland is ranked 20th overall Ireland’s social security administration score
merits further attention as the sole metric used

on the Index and generally for this indicator suggests weaker systems

performs more strongly on efficiency compared with other countries. Ireland’s

core functions. regulation score warrants further investigation
as well as it performs better in some thematic
Ireland’s scores are high across all the HRM areas than others. Themes for potential analysis
metrics. This may reflect Ireland’s centrally include stakeholder engagement and evaluation.
administered systems, including at relatively
low grade levels, plus its strong regulation of The main attribute which would benefit from
civil service appointments and increased use further analysis is openness. However, estimated
of external recruitment. Ireland also scores well data was used for some metrics and this should
against the metrics concerning attracting and be borne in mind when analysing results.
retaining talent, notably the extent to which

https://incise­radar­vis.cloudapps.digital/ 2/6

senior officials’ salaries are comparable with
similar jobs in the private sector.

On tax administration, Ireland is ranked 4th,
scoring consistently well across all themes,
and it is worth noting that the top 5 countries’
overall scores are all close for this indicator.
38 The International Civil Service Effectiveness Index

4.15 Italy

5/26/2017 Radar chart

Tax
administration
Policy
making Inclusiveness

0.75

Social
security Capabilities
0.50

0.25

Digital
services Openness

Fiscal
and Integrity
financial
management

Regulation Human
Crisis resource
management management

Italy is ranked 27th overall with the underlying data reflecting perceptions
of the composition and efficiency of public

on the Index. It generally spending.
achieves better scores for
Italy’s tax administration score is also relatively

core function indicators. low but with a wide variation in thematic scores.
It leads for the metrics on the extent and quality

Italy’s social security administration score is of digital provision theme but does less well in the

above average, suggesting it has relatively low other themes concerning the overall efficiency of

administration costs relative to sector expenditure. tax collection and the extent to which services are
Italy’s crisis/risk management scores are user focused.
above average as well. It achieves the joint All attributes scores require further analysis,
highest score for the post-disaster assessment although some positive metrics scores are worth
methodology theme (with 9 other countries).
https://incise­radar­vis.cloudapps.digital/
noting. Italy scores well2/6relative to other countries
Italy is also joint top for one of the two metrics on the whistleblower protection metrics within
concerning risk monitoring. the integrity indicator, as well as the openness

The core function where Italy performs less well metrics on the degree and quality of consultation

compared with other countries is fiscal and with society. On inclusiveness, Italy scores well

financial management (FFM). Scores are below on two gender representation metrics relating to

average for the budgeting themes, and also for the proportion of women in senior positions in

the economic appraisal and evaluation themes, the civil service.
The International Civil Service Effectiveness Index 39

4.16 Japan

5/26/2017 Radar chart

Tax
administration
Policy
making Inclusiveness

0.75

Social
security Capabilities
0.50

0.25

Digital
services Openness

Fiscal
and Integrity
financial
management

Regulation Human
Crisis resource
management management

Japan is ranked 15th overall The core functions where Japan’s performance
is weaker compared with other countries are

on the Index. It is ranked top tax administration and regulation. The overall

for the capabilities attribute efficiency of tax collection and the extent to which

with a score well ahead of services are user focused are themes where future
improvements could be informed by the high

other countries. scoring countries.
The metrics used for the capabilities indicator
For regulation, Japan’s performance is below

suggest Japan has a civil service with very high
the average for each of the three themes

literacy, numeracy and problem solving skills,
concerning appraisal, evaluation and stakeholder

plus strong educational attainment levels.
engagement, and relatively low for the metrics
Japan is ranked a close 2nd for the HRM within the stakeholder engagement theme.
indicator as well. Japan’s high scores for the

https://incise­radar­vis.cloudapps.digital/
On attributes, Japan’s inclusiveness
2/6
scores
meritocratic recruitment theme confirm its
are comparatively low. Learning lessons from
competitive civil service entry systems, including
the leading countries on increasing women’s
the use of formal examinations (ranked 2nd).
representation in the civil service could be an
Japan also scores well on attracting and
initial area of focus.
retaining talent.
40 The International Civil Service Effectiveness Index

4.17 Republic of Korea

5/26/2017 Radar chart

Tax
administration
Policy
making Inclusiveness

0.75

Social
security Capabilities
0.50

0.25

Digital
services Openness

Fiscal
and Integrity
financial
management

Regulation Human
Crisis resource
management management

The Republic of Korea is The Republic of Korea’s high HRM scores across
most metrics suggests its civil service has a

ranked 9th overall on the Index relatively meritocratic recruitment system. It does

and its position rises to 4th particularly well on the metric concerning the use

following adjustments for GDP of formal examinations systems to recruit staff.
The country’s scores suggest that it is also able

per capita. to attract and retain talent, although it is ranked
just below average for the metric on the extent
The Republic of Korea scores above average
to which senior staff salaries compare favourably
for all core functions, notably HRM (ranked 5th
with similar jobs in the private sector.
overall) and fiscal and financial management
(ranked 5th again). On the latter, one of the The Republic of Korea’s inclusiveness score
main data sources for this indicator is the OECD merits further analysis. Based on the available
performance budgeting index where the country data, the representation2/6of women in the civil

https://incise­radar­vis.cloudapps.digital/
is ranked top. service is assessed to be weak relative to other
countries.
The International Civil Service Effectiveness Index 41

4.18 Mexico

5/26/2017 Radar chart

Tax
administration
Policy
making Inclusiveness

0.75

Social
security Capabilities
0.50

0.25

Digital
services Openness

Fiscal
and Integrity
financial
management

Regulation Human
Crisis resource
management management

Mexico is ranked 19th overall methodology and integrated risk planning) they
are less strong for others, for example the risk

on the Index and its position monitoring and international cooperation themes

rises to 2nd when scores are where there may be opportunities to learn from

adjusted for GDP per capita. best practice in future.

Mexico achieves the top position for regulation There is a similar pattern of thematic variation
which could reflect the country’s sustained for the HRM indicator. Mexico performs relatively
focus on regulatory policy reforms over the last strongly against the attracting and retaining
decade. It scores strongly across all themes talent theme where it is ranked top on the
and comes top for both metrics on stakeholder extent to which senior officials’ salaries compare
engagement. Mexico’s strong performance favourably with private sector counterparts.
against this indicator suggests they have some However, scores are relatively low for the
positive lessons to share with countries who wish

https://incise­radar­vis.cloudapps.digital/
meritocracy of recruitment
2/6
theme and there may
to improve in this area. be lessons to learn from the leading countries.

When assessing relative performance, crisis/risk All attribute scores merit further analysis to

management is a core function for further prioritise areas for improvement. Estimated data

consideration as Mexico’s scores vary significantly was used for some metrics within the capabilities

across the themes measured. Although scores and inclusiveness indicators which should be

compare favourably for two themes (post disaster borne in mind when interpreting these results.
42 The International Civil Service Effectiveness Index

4.19 Netherlands

5/26/2017 Radar chart

Tax
administration
Policy
making Inclusiveness

0.75

Social
security Capabilities
0.50

0.25

Digital
services Openness

Fiscal
and Integrity
financial
management

Regulation Human
Crisis resource
management management

The Netherlands is ranked 14th Regulation is another core function where the
Netherlands’ performance is less strong relative

overall on the Index. It performs to other countries. There may be lessons to

very strongly on fiscal and learn from the leading countries concerning

financial management (ranked the stakeholder engagement and impact
assessment themes. For the policy making

2nd overall). indicator, there may be gains through focusing
on the policy monitoring theme as relatively

The Netherlands scores highly on tax
weak performance in this area reduced the

administration and crisis/risk management as
overall score.
well. Its openness score is strong too (ranked

6th), also capabilities (ranked 7th). Looking at the attribute indicators, the
Netherlands’ inclusiveness score merits

The Netherlands’ score for the social security
attention, although it does better on some

administration indicator is relatively weak,
https://incise­radar­vis.cloudapps.digital/ 2/6
gender metrics than on others. Efforts to boost

suggesting there may be opportunities
women’s representation in senior management

for improvement in systems efficiency.
positions in the civil service, learning from the
practices of the high scoring countries, could be
an initial focus area.
The International Civil Service Effectiveness Index 43

4.20 New Zealand

5/26/2017 Radar chart

Tax
administration
Policy
making Inclusiveness

0.75

Social
security Capabilities
0.50

0.25

Digital
services Openness

Fiscal
and Integrity
financial
management

Regulation Human
Crisis resource
management management

New Zealand is ranked 2nd There are two core functions where New
Zealand’s performance is relatively weaker: crisis/
overall on the Index. Its risk management and tax administration. On the

high integrity and openness former, scores vary and the themes which merit

rankings are noteworthy. further analysis concern integrated risk planning
and risk monitoring.
New Zealand’s top integrity score is well

ahead of all other countries, reflecting a strong On tax administration, learning from the practices

all-round performance against most metrics. of the leading countries on some aspects of the

It does particularly well on the corruption extent and quality of digital provision could help

perceptions and work ethic themes, achieve a higher overall indicator score in future.
and comes top or joint top in 6 out of the

16 metrics for this indicator.
https://incise­radar­vis.cloudapps.digital/ 2/6

New Zealand comes a close 2nd on

openness, achieving the top score for the

publishing laws theme and ranking 2nd on

the right to information.
44 The International Civil Service Effectiveness Index

4.21 Norway

5/26/2017 Radar chart

Tax
administration
Policy
making Inclusiveness

0.75

Social
security Capabilities
0.50

0.25

Digital
services Openness

Fiscal
and Integrity
financial
management

Regulation Human
Crisis resource
management management

Norway is ranked 8th overall Norway’s inclusiveness score is just below
average, although the thematic scores vary.
on the Index and its scores For example, while it scores relatively well against

are above average for most some metrics for the gender representation

indicators. theme its performance for the other theme is
below average. An initial focus on the latter,
Norway is currently one of two countries where learning from best practice in the leading

data is available for all 76 InCiSE metrics. countries, may help to achieve a stronger
overall indicator score in future.
Norway scores particularly well on tax
administration (ranked 2nd) and across all On core functions, Norway’s performance for
metrics for this indicator. Norway’s scores are the regulation indicator merits further attention
also very high for two attributes: integrity and as scores were below the average across the
openness (both ranked 3rd). On the latter,
https://incise­radar­vis.cloudapps.digital/
themes, particularly impact
2/6
assessment work.
Norway does particularly well on the public
consultation theme.
The International Civil Service Effectiveness Index 45

4.22 Poland

5/26/2017 Radar chart

Tax
administration
Policy
making Inclusiveness

0.75

Social
security Capabilities
0.50

0.25

Digital
services Openness

Fiscal
and Integrity
financial
management

Regulation Human
Crisis resource
management management

Poland is ranked 21st overall Poland scores well on policy making (ranked 11th
overall) and is in the top 10 for metrics concerning

on the Index and its position the quality of policy advice theme. It also does well

rises to 12th when scores are on the policy monitoring theme. Poland’s above

adjusted for GDP per capita. average score for social security administration
suggests that it has relatively low administration

Poland is ranked top for the inclusiveness costs relative to sector expenditure.
indicator. This reflects very strong scores for

the metrics on the proportion of women in One core function where Poland’s performance

central government management and senior is weaker compared with other countries is

management positions, as well as levels of tax administration. Scores are below average

ethnic and religious minority representation. across the themes, particularly those covering
the overall efficiency of tax collection and the

For the capabilities indicator, while Poland’s extent and quality of digital provision. It is worth

https://incise­radar­vis.cloudapps.digital/ 2/6

relative performance is above average for noting that Poland’s score for the digital services

the educational attainment theme, a focus indicator is below average as well and an initial

on the numeracy and problem solving skills focus on this area could have a positive effect on

metrics within the core capability theme, other core functions in future.
learning from the high scoring countries, may

help achieve a higher overall score in future.
46 The International Civil Service Effectiveness Index

4.23 Portugal

5/26/2017 Radar chart

Tax
administration
Policy
making Inclusiveness

0.75

Social
security Capabilities
0.50

0.25

Digital
services Openness

Fiscal
and Integrity
financial
management

Regulation Human
Crisis resource
management management

Portugal is ranked 26th overall For the HRM indicator, Portugal’s performance
relative to other countries is lower across most

on the Index and its position metrics within the meritocracy of recruitment

rises to 23rd when scores are theme, the main exception being the extent to

adjusted for GDP per capita. which formal examination systems are used
for recruitment purposes. Within the attracting

Portugal scores highly on social security and retaining talent theme, Portugal’s score is

administration (ranked 4th overall), suggesting relatively low on the extent to which salaries for

that administrative costs as a proportion of sector senior officials compare favourably with similar

expenditure are relatively low. It is ranked 6th for jobs in the private sector. Learning from the

digital services and is in the top 5 for three out leading countries may help to achieve improved

of the four themes within this indicator, including scores in future.
the extent to which services are user focused. On attributes, while Portugal’s inclusiveness

https://incise­radar­vis.cloudapps.digital/ 2/6

The core functions where Portugal’s performance score is around the average its performance

is assessed to be weaker compared with other against the other indicators is relatively lower.
countries are regulation and HRM. On regulation, Further analysis would help prioritise areas for

the main themes which could be considered improvement.
for further attention relate to stakeholder

engagement and impact assessment.
The International Civil Service Effectiveness Index 47

4.24 Slovakia

5/26/2017 Radar chart

Tax
administration
Policy
making Inclusiveness

0.75

Social
security Capabilities
0.50

0.25

Digital
services Openness

Fiscal
and Integrity
financial
management

Regulation Human
Crisis resource
management management

Slovakia is ranked 31st overall The main core functions where Slovakia’s
performance is weaker relative to other countries

on the Index. It is ranked 14th are policy making, HRM and digital services. On

for the regulation indicator, policy making, it scores below the average for

with particularly strong all thematic metrics, particularly the coordination
of policy proposals. On HRM, learning from

scores for the stakeholder the leading countries might help achieve

engagement metrics. improvements across the themes. Further

Within the regulation indicator, Slovakia also analysis of Slovakia’s digital services indicator

scores well on the impact assessment theme. score would also help to identify areas for
improvement, as well as all attribute indicators.
Slovakia’s crisis/risk management score is
around the average mark. It is in the top 10
countries for both risk monitoring metrics,
https://incise­radar­vis.cloudapps.digital/ 2/6

although its performance is less strong for
the integrated risk planning theme. An initial
focus on this theme, learning from best practice,
could help to improve Slovakia’s overall indicator
score in future.

Slovakia’s social security administration score is

around the average mark as well.
48 The International Civil Service Effectiveness Index

4.25 Slovenia

5/26/2017 Radar chart

Tax
administration
Policy
making Inclusiveness

0.75

Social
security Capabilities
0.50

0.25

Digital
services Openness

Fiscal
and Integrity
financial
management

Regulation Human
Crisis resource
management management

Slovenia is ranked 23rd 5th overall), suggesting that administrative
costs as a proportion of sector expenditure are

overall on the Index and its comparatively low.
position rises to 16th following
The main core function indicators where Slovenia

adjustments for GDP per capita. performs weakly relative to other countries are
policy making and HRM. On policy making, all

Slovenia’s very high inclusiveness score – ranked themes could be reviewed to establish priority

2nd overall – is noteworthy, reflecting a strong areas for improvement. Both themes within the

score for the metric assessing the proportion HRM indicator merit further analysis, although

of women in central government management Slovenia’s performance is above average for the

and senior management positions. It also tops metric concerning the use of formal examinations

the ethnic and religious minority representation systems for recruitment purposes.
metric for this indicator.
https://incise­radar­vis.cloudapps.digital/
Slovenia’s performance2/6for all other attribute

Slovenia is ranked 3rd overall for crisis/risk indicators is below the average and they may

management and it achieves consistently good benefit from learning from the high scoring

scores across most metrics. Its scores are countries. However, a degree of estimated data

particularly high for the risk monitoring theme was used for the capabilities indicator which
(joint top on both metrics) and it also achieves should be borne in mind when analysing results.
strong scores for the post disaster assessment

methodology theme. Slovenia’s social security

administration scores are also high (ranked
The International Civil Service Effectiveness Index 49

4.26 Spain

5/26/2017 Radar chart

Tax
administration
Policy
making Inclusiveness

0.75

Social
security Capabilities
0.50

0.25

Digital
services Openness

Fiscal
and Integrity
financial
management

Regulation Human
Crisis resource
management management

Spain is ranked 18th overall on assessed to be below the average, particularly the
stakeholder engagement and evaluation themes.
the Index and its position rises

to 15th following adjustments On inclusiveness, whilst some scores for the
gender representation theme are above average,
for GDP per capita. a review of the leading countries’ scores would

Spain is ranked 7th overall for the tax help prioritise areas to focus on in future.
administration indicator, achieving a particularly

good score for the extent and quality of digital

provision. Spain is ranked 9th for the social

security administration indicator, suggesting

reasonable administrative costs as a proportion

of total sector expenditure.
https://incise­radar­vis.cloudapps.digital/ 2/6

The main core functions where Spain’s

performance is weak relative to other

countries are regulation and fiscal and financial

management. On the latter, Spain achieves

a stronger performance for the medium term

budgeting theme and could learn from the high

scoring countries for the performance budgeting

theme. On regulation, Spain’s performance is
50 The International Civil Service Effectiveness Index

4.27 Sweden

5/26/2017 Radar chart

Tax
administration
Policy
making Inclusiveness

0.75

Social
security Capabilities
0.50

0.25

Digital
services Openness

Fiscal
and Integrity
financial
management

Regulation Human
Crisis resource
management management

Sweden is ranked 6th overall On inclusiveness, there is some variation in
scores by theme. For example, Sweden mostly

on the Index and achieves scores well compared with other countries on

above average scores in the gender representation theme. High scoring

most areas. countries for this indicator tend to perform well
across both themes and learning from their
Sweden’s integrity score is noteworthy (ranked practices may be beneficial.
2nd overall). It achieves especially strong scores For the HRM indicator, Sweden performs
for the work ethic theme and for the levels/ relatively well against most of the meritocracy of
perceptions of corruption theme. Sweden recruitment metrics. Lessons could be learned
is ranked 3rd overall for fiscal and financial from the leading countries for the attracting and
management, scoring particularly well against retaining talent theme, particularly the extent to
the medium term budgeting, performance which senior staff salaries compare favourably
appraisal and evaluation themes.
https://incise­radar­vis.cloudapps.digital/ 2/6
with similar jobs in the private sector.
Indicator scores that are relatively weaker Sweden performs well relative to others in some

and which may merit further analysis include aspects of regulation, although its stakeholder

inclusiveness and HRM (where Sweden’s engagement theme score is below average and

performance is around the average). this could be an area for development in future.
The International Civil Service Effectiveness Index 51

4.28 Switzerland

5/26/2017 Radar chart

Tax
administration
Policy
making Inclusiveness

0.75

Social
security Capabilities
0.50

0.25

Digital
services Openness

Fiscal
and Integrity
financial
management

Regulation Human
Crisis resource
management management

Switzerland is ranked Switzerland scores well on regulation (ranked
5th overall), particularly the stakeholder

12th overall on the Index engagement and impact assessment themes.
and achieves the top Its crisis/risk management indicator scores are

position for fiscal and also above average.
financial management. Switzerland’s attributes scores are all on or
above average.
Switzerland performs well across all the fiscal
and financial management themes. It is worth Switzerland’s social security administration score
noting that Switzerland has topped the Global may warrant further investigation as the sole
Competitive Index, which is used for this metric used for this indicator suggests that there
indicator, for eight consecutive years. It also does could be opportunities to improve the efficiency
well in the OECD indexes used for this indicator. of the system.
Switzerland’s high performance may in part

https://incise­radar­vis.cloudapps.digital/ 2/6

reflect the government’s decision over a decade Switzerland’s scores for the digital services and
ago to overhaul its accounting system because policy making indicators are also less strong
of growing financial management demands. compared with other countries. On the latter,
Switzerland could focus initially on improving its
scores for the quality of advice, strategic policy
direction and coordination themes, learning from
the practices of the high scoring countries.
52 The International Civil Service Effectiveness Index

4.29 Turkey

5/26/2017 Radar chart

Tax
administration
Policy
making Inclusiveness

0.75

Social
security Capabilities
0.50

0.25

Digital
services Openness

Fiscal
and Integrity
financial
management

Regulation Human
Crisis resource
management management

Turkey is ranked 25th overall The main core functions where Turkey’s
performance is relatively weaker are regulation

on the Index and its position and HRM. On the latter, a focus on the

rises to 10th when scores are meritocracy of recruitment theme could be

adjusted for GDP per capita. beneficial, although it is ranked 10th for the
metric on the use of formal examination systems
Turkey achieves the top position for the crisis/ to hire staff. Turkey’s regulation indicator scores
risk management indicator, scoring consistently show significant variation in relative performance.
well across all themes. It comes joint top for the It could focus initially on the impact assessment
integrated risk planning theme (i.e. the extent and evaluation thematic metrics, learning from
to which disaster risk is incorporated in national best practice in the high scoring countries.
policy planning). Turkey also scores well on Turkey’s performance against all the attribute
risk monitoring, demonstrating a consistent indicators merits further2/6analysis, especially the
and systematic approach to reporting. Turkey’s

https://incise­radar­vis.cloudapps.digital/
openness indicator score which is low relative to
performance against this indicator reflects its well other countries.
recognised leadership and experience of disaster
risk planning and management.

Turkey’s social security administration score

is also high (ranked 6th) suggesting that

administrative costs as a proportion of sector

expenditure are relatively low.
The International Civil Service Effectiveness Index 53

4.30 United Kingdom

5/26/2017 Radar chart

Tax
administration
Policy
making Inclusiveness

0.75

Social
security Capabilities
0.50

0.25

Digital
services Openness

Fiscal
and Integrity
financial
management

Regulation Human
Crisis resource
management management

The UK is ranked 4th overall The UK scores highly on regulation as well
(ranked 2nd overall), coming top on the impact

on the Index. It is ranked assessment theme, and on tax administration

top for policy making, (ranked 5th overall), with consistently good

openness and social scores across all metrics.
security administration. The main core function where further attention
may be warranted is crisis/risk management

The UK is currently one of two countries where
where the UK’s performance is just above

data is available for all 76 InCiSE metrics.
average. On digital services, the UK scores
On policy making, the UK performs particularly relatively highly for the cross-border mobility
well on the quality of policy advice theme, of services theme (ranked 4th overall) but less
including whether it is evidence based. For the well against other themes considered. The UK’s
openness indicator, the UK scores highly on

https://incise­radar­vis.cloudapps.digital/
scores for the integrity and
2/6
capabilities indicators
two themes – government data availability and may also benefit from further analysis, learning
accessibility, plus government data impact – from the leading countries.
coming top in two of the metrics used. It also
scores well on the public consultation theme.
The UK’s social security administration ranking
suggests that system costs as a proportion of
sector expenditure are comparatively low.
54 The International Civil Service Effectiveness Index

4.31 United States of America

5/26/2017 Radar chart

Tax
administration
Policy
making Inclusiveness

0.75

Social
security Capabilities
0.50

0.25

Digital
services Openness

Fiscal
and Integrity
financial
management

Regulation Human
Crisis resource
management management

The USA is ranked 10th overall The USA’s attributes scores are all above
average, although its performance for the

on the Index. The country’s capabilities indicator (ranked 12th overall)
strongest indicator score is could merit further analysis, particularly the

for policy making (ranked a core capabilities theme.
close 2nd).
Within the policy making indicator the USA

scores particularly well on the quality of advice

theme and metrics, including whether it is

evidence based, also on the coordination of

policy proposals.

The USA performs less well relative to other

https://incise­radar­vis.cloudapps.digital/ 2/6

countries on fiscal and financial management.
A focus on the economic appraisal and

evaluation themes and metrics could be

beneficial, learning from the best practices of

the high scoring countries.
The International Civil Service Effectiveness Index 55

Chapter 5: Next steps

and conclusion

5.1 Next steps b) Refining the InCiSE framework
Framework tasks will include: exploring the

InCiSE is a long term project and the founding
potential to add new civil service functions (for

organisations have committed to supporting
example, security, foreign affairs, defence and

its development for a further four years. This
justice); ensuring each indicator in the framework

will include publishing an annual report and
is a unique concept, to avoid duplication or

developing an interactive website. An International
overlap between the themes being measured;
Advisory Panel has also been established to
and ensuring that framework does not favour

provide strategic advice to the project.
certain types of systems.
As described earlier, this initial Index is a pilot
One issue emerging from the InCiSE pilot is the

only which requires further work to refine the
need to look at additional ways of capturing the

methodology and make the data more robust.
strengths that can arise from federal government

The main issues that the project will focus on
systems and ensure that the framework

over the next 12 months are set out below.
can measure and compare the respective

a) Strengthening data collection core functions.
Tasks will include: determining how best to Changes to the InCiSE framework will be

collect data on the 5 omitted indicators (staff determined mainly through user feedback and

engagement, innovation, IT, procurement, discussions with data owners to learn lessons

finance); exploring how data collection for from their own experiences. The International

existing indicators can be strengthened, Advisory Panel will also be consulted.
including thematic gaps; expanding data sources

and looking at ways of avoiding over reliance c) Expanding country coverage

on a single survey; reducing instances of public InCiSE will explore the scope to expand the

sector performance being used as a proxy for Index’s current country coverage over time,
civil service performance; and cross-country including the potential to include non-OECD and

applicability issues. developing countries. Country coverage in future
will largely be determined by feedback from

Complementary work by other institutions may
countries on the usefulness of having their own

help to fill some data gaps over time. Additional
set of indicators, as well as the availability of data

data collection, for example through the
to produce results.
commissioning of specific country surveys, could

also help to provide valuable insights and the Several countries were excluded from the pilot

project will actively support this work. Index because they had fewer than 75% of
the metrics available. However, three countries
(Bulgaria, Romania and Croatia) had over 70% of
data available. A small increase in data collection
may help them meet the threshold for inclusion in
the next Index and this issue will be explored.
56 The International Civil Service Effectiveness Index

d) Increasing InCiSE partners and network 5.2 Conclusion

The founding institutions will actively encourage
The InCiSE pilot Index has already brought

more partners to join them and support the
together a rich volume of data and insights from

development of the InCiSE Index over time,
many international sources. Its launch provides

through advice, expertise and funding. The
an important opportunity to stimulate wide-
project’s new International Advisory Panel will
ranging discussions at country level, as well as

also play a key role in promoting InCiSE and
globally, about civil service effectiveness issues

encouraging more partners.
generally, as well as the relevance and usefulness

The Blavatnik School of Government will host of some data.
an international conference in September 2017
Feedback is vitally important to help improve

to discuss the results of the pilot Index with a
InCiSE and the project team would welcome

wide range of interested players, as well as the
responses from country governments, as well

future direction of the project. This event will
as policy and learning networks, think-tanks and

additionally provide a key opportunity to build
academia. InCiSE contact details can be found

support for InCiSE and increase its network.
at the front of this report.

The InCiSE project team will continue to
coordinate closely with other institutions who are
engaged in similar efforts to measure civil service
effectiveness. The team will also coordinate
closely with the many organisations who have
made a vital contribution to InCiSE so far by
permitting use of their research or survey data.
Continued collaboration will be essential to help
strengthen InCiSE in the coming years.
The International Civil Service Effectiveness Index 57

Appendices

Annex A: InCiSE indicator definitions

Annex B: InCiSE indicator data sources

Annex C: InCiSE country rankings data

Annex D: InCiSE indicator scores summary by country

Annex E: Summary graphs of country scores
and rankings for each indicator

Annex F: Reader’s guide
58 The International Civil Service Effectiveness Index

Annex A: InCiSE indicator

definitions

Core functions Tax administration: The efficiency and
effectiveness of tax collection (at the central/
Policy making: The quality of the policy making federal level).
process, including how policy is developed and

coordinated across government and monitored Social security administration: The efficiency

during implementation. and effectiveness of social security administration
(at the central/federal level).
Fiscal and financial management (FFM):
The quality of the budgeting process and the Digital services: The user-centricity,
extent to which spending decisions are informed transparency and cross-border mobility of

through economic appraisal and evaluation. digitally-provided public services and the
availability of ‘key enablers’.
Regulation: The extent and quality of regulatory

impact assessments and the degree of
Attributes

stakeholder engagement involved in them.
Integrity: The extent to which civil servants

Crisis/risk management: The effectiveness with
behave with integrity, make decisions impartially

which the government engages the whole of
and fairly, and strive to serve both citizens

society to better assess, prevent, respond to and
and ministers.
recover from the effects of extreme events.
Openness: The regular practice and degree

Procurement: The extent to which the
of consultation with citizens to help guide

procurement process is efficient, competitive,
decision-making and extent of transparency

fair, and pursues value for money.
in decision-making.
Human resource management (HRM): The
Capabilities: The extent to which the workforce

meritocracy of recruitment and extent to which
has the right mix of skills.
civil servants are effectively attracted, managed,
and developed. Inclusiveness: The extent to which the civil
service is representative of the citizens it serves.
Information technology: The extent to

which civil servants have the digital tools to Staff engagement: Staff levels of pride,
work efficiently. attachment and motivation to work for their
organisation.
Finance: The extent to which operations are

supported by well-managed, efficient finance Innovation: The degree to which new ideas,
systems, particularly on the alignment of finance policies, and ways of operating are able to

with the business strategy and the level of civil freely develop.
servant satisfaction with finance support.
The International Civil Service Effectiveness Index 59

Annex B: InCiSE indicator

data sources

Country coverage

Indicator Data source and frequency Data content

Policy making Bertelsmann Sustainable 41 countries. Updated The SGIs assess three
Governance Indicators annually. pillars of governance:
(SGI), 2016. policy; performance; and
democracy.

Fiscal and financial The World Economic 138 countries. Updated The GCI combines 114

management Forum (WEF) Global annually. indicators (grouped into
Competitiveness Index 12 pillars) that capture
(GCI), 2016. concepts that matter for
productivity and long-
term prosperity.

OECD medium-term 32 countries. Last The OECD indexes cover
budgeting index, (2012) survey 2012. Next date range of public financial
and performance unknown. management areas.
budgeting index, (2011).

Regulation OECD Indicators of 34 countries. Updated The OECD survey
Regulatory Policy and every 3-4 years. assesses 3 aspects of
Governance, 2014. the process towards
creating effective
regulation.

Crisis/risk UN Hyogo Framework Covers 168 countries, Country reports track

management for Action (HFA) 2005- rolled out gradually over progress towards
2015, (2015). 10 years from 2005. implementing UN
priorities for disaster risk
reduction action.

Human resource Quality of Government 159 countries. Last The QoG Survey gathers

management (QoG) Expert Survey survey 2015. data on structure and
Data, University of behaviour of public
Gothenburg, Sweden, administrations.
2015.
60 The International Civil Service Effectiveness Index

Country coverage
Indicator Data source and frequency Data content

Tax administration OECD Tax Administration 56 countries. Repeat The OECD survey
2015 survey, (2013 planned mid 2017. provides comprehensive
data). 190 countries. assessment of tax
The World Bank Doing administration systems.
Updated annually.
Business Index (DBI), The DBI measures
2016. regulations affecting
11 areas of the life of a
business.

Social security European Commission 33 European countries. Eurostat data includes
administration data (via Eurostat), 2014. Updated annually. administration costs
as a proportion of total
expenditure on social
security. This data is
used to capture overall
efficiency of social
security administration
systems.

Digital services European Commission 33 European countries. The E-Government
E-Government Updated annually. Benchmark assesses
Benchmark, 2014/15. quality of digital service
delivery across four key
areas.

Integrity Transparency GCB: 100 countries. GCB: corruption
International Global Updated annually. perceptions survey.
Corruption Barometer GCI: 138 countries. CGI: See above.
(GCB), 2013. Updated annually. QoG: See above.
World Economic QoG: 159 countries. GaaG: wide ranging
Forum (WEF) Global Last survey 2015. set of indicators on
Competitiveness Index
GaaG: 35 countries. governance and public
(GCI), 2016.
Updated every five years. sector performance.
Quality of Government
(QoG) Expert Survey
Data, University of
Gothenburg, Sweden,
2015.
OECD “Government at
a Glance” (GaaG) data,
2013 and 2014.
The International Civil Service Effectiveness Index 61

Country coverage

Indicator Data source and frequency Data content

Openness a) World Justice Project: a) 102 countries. Last a) The OGI measures
Open Government survey 2015. government openness
Index (OGI), 2015. b) 193 countries. based on citizen
b) UN E-participation Updated biennially perceptions and
Index (EPI), 2016. experiences.
c) 41 countries. Updated
c) Bertelsmann SGIs, annually b) The EPI measures
2016. the use of online
d) 92 countries. Updated
services to support
d) World Wide Web annually.
government
Foundation: Open e) 122 countries. information sharing
Data Barometer Updated annually. and engagement
(ODB), 2015.
f) 29 countries. Last with citizens.
e) The Open Knowledge report 2014. c) SGIs: see above.
Foundation: Open
Data Index (ODI), d) The ODB ranks
2015. government open
data policies,
f) OECD OURdata
planning, delivery and
index, 2014.
impact.
e) The ODI measures
the status of open
government data.
f) The OURdata Index
measures government
efforts to implement
the G8 Open Data
charter.

Capabilities OECD Programme 24 countries. Data PIACC survey measures
for the International collected 2011/12. adults’ proficiency in key
Assessment of Adult information-processing
Competencies (PIAAC) skills – literacy, numeracy
survey data, 2012. and problem solving;
and gathers data on how
adults use their skills at
home, work and in the
wider community.

Inclusiveness OECD Government at See above. See above.
a Glance (GaaG) and
Quality of Government
(QoG) data, 2015.
62 The International Civil Service Effectiveness Index

Annex C: InCiSE country

rankings data

The table below covers three sets of country rankings: an overall ranking based on

a composite score; rankings against each set of core functions and attributes; and then

a ranking against each of the indicators.
Table 2: Country rankings (without GDP per capita adjustments)
Country Fiscal and Human Social

code Policy financial Crisis/risk resource Tax security Digital

3-digit Composite Integrity Openness Capabilities Inclusiveness Attributes making man. Regulation man. man. admin. admin. services Functions

AUS 3 9 7 6 5 5 7 19 3 4 7 10 22 4 4

AUT 16 10 13 29 12 14 28 21 17 26 22 9 11 2 15

BEL 13 11 20 10 9 15 13 30 10 =8 4 18 25 18 13

CAN 1 4 9 3 3 2 5 6 4 =8 3 20 14 7 1

CHL 22 21 19 11 20 20 9 17 31 =8 24 23 24 20 23

CZE 28 27 24 25 26 25 27 11 12 20 28 31 23 28 28

DNK 11 5 4 17 23 8 3 10 19 28 10 3 26 3 11

EST 7 15 17 15 28 19 20 7 6 =8 18 1 3 1 3

FIN 5 6 5 2 4 3 4 8 21 2 14 14 19 5 8

FRA 17 12 12 22 27 18 18 13 18 27 12 19 28 16 20

DEU 24 8 15 9 15 13 26 18 7 31 13 24 27 19 26

GRC 29 28 29 23 6 26 29 28 29 21 25 27 13 29 30

HUN 30 25 30 30 31 31 24 31 20 24 30 25 2 30 29

IRL 20 16 26 20 13 22 16 20 28 =8 1 4 31 21 21

ITA 27 29 22 31 24 27 21 29 22 17 20 26 16 25 27

JPN 15 18 14 1 29 12 17 16 26 =8 2 29 20 13 17

KOR 9 20 11 16 25 17 12 5 11 =8 5 13 17 8 7

MEX 19 31 23 27 30 30 14 15 1 29 27 21 7 14 12

NLD 14 14 6 7 18 9 23 2 25 7 15 6 29 15 19

NZL 2 1 2 4 7 1 6 4 9 22 6 15 12 9 5

NOR 8 3 3 8 19 4 8 12 24 23 8 2 10 12 9

POL 21 19 21 19 1 16 11 22 16 30 19 30 15 26 24

PRT 26 24 27 24 17 24 22 26 30 25 29 22 4 6 25

SVK 31 30 28 28 21 29 30 25 14 = 18 31 28 18 31 31

SVN 23 23 25 21 2 21 31 23 13 3 23 16 5 22 22

ESP 18 22 18 26 22 23 15 27 23 = 18 21 7 9 17 16

SWE 6 2 10 13 16 6 10 3 15 5 16 8 8 11 6

CHE 12 7 16 5 11 11 25 1 5 6 9 11 30 27 14

TUR 25 26 31 18 14 28 19 9 27 1 26 12 6 24 18

GBR 4 17 1 14 10 7 1 14 2 16 11 5 1 23 2

USA 10 13 8 12 8 10 2 24 8 =8 17 17 21 10 10
The International Civil Service Effectiveness Index 63

Annex D: InCiSE indicator

scores summary by country

The table below covers three sets of country scores: an overall composite score; scores against each

set of core functions and attributes; and then individual scores for each of the indicators.

Table 3: Country indicator scores (without GDP per capita adjustments)
Country Fiscal and Human Social

code Policy financial Crisis/risk resource Tax security Digital

3-digit Composite Integrity Openness Capabilities Inclusiveness Attributes making man. Regulation man. man. admin. admin. services Functions

AUS 0.91 0.74 0.9 0.66 0.68 0.84 0.88 0.55 0.91 0.92 0.73 0.8 0.65 0.86 0.93

AUT 0.56 0.74 0.76 0.22 0.51 0.63 0.36 0.53 0.41 0.59 0.42 0.81 0.83 0.87 0.53

BEL 0.6 0.72 0.55 0.56 0.55 0.61 0.75 0.1 0.59 0.77 0.85 0.69 0.6 0.67 0.59

CAN 1 0.85 0.86 0.72 0.93 0.95 0.94 0.86 0.84 0.77 0.88 0.68 0.79 0.82 1

CHL 0.44 0.59 0.58 0.51 0.37 0.51 0.84 0.6 0 0.77 0.37 0.59 0.6 0.6 0.42

CZE 0.16 0.29 0.43 0.25 0.22 0.21 0.38 0.69 0.5 0.72 0.21 0 0.62 0.27 0.16

DNK 0.73 0.84 0.93 0.44 0.25 0.75 0.98 0.72 0.3 0.57 0.69 0.93 0.44 0.86 0.71

EST 0.81 0.66 0.65 0.45 0.16 0.52 0.57 0.81 0.69 0.77 0.54 1 0.99 1 0.95

FIN 0.87 0.83 0.91 0.73 0.8 0.93 0.97 0.78 0.26 0.98 0.58 0.76 0.69 0.85 0.82

FRA 0.5 0.67 0.77 0.28 0.16 0.54 0.6 0.66 0.34 0.58 0.68 0.68 0.38 0.69 0.48

DEU 0.4 0.74 0.68 0.59 0.45 0.67 0.42 0.56 0.65 0 0.67 0.52 0.41 0.62 0.29

GRC 0.04 0.19 0.24 0.27 0.65 0.18 0.29 0.27 0.07 0.71 0.31 0.32 0.79 0.05 0.01

HUN 0 0.38 0.02 0.21 0 0 0.49 0 0.29 0.63 0.1 0.48 0.99 0 0.05

IRL 0.46 0.66 0.42 0.34 0.5 0.47 0.66 0.54 0.1 0.77 1 0.93 0 0.59 0.47

ITA 0.21 0.18 0.47 0 0.25 0.13 0.53 0.2 0.25 0.74 0.49 0.42 0.77 0.46 0.28

JPN 0.57 0.65 0.74 1 0.16 0.68 0.61 0.6 0.14 0.77 0.99 0.26 0.68 0.72 0.52

KOR 0.78 0.6 0.77 0.44 0.22 0.55 0.8 0.87 0.55 0.77 0.82 0.77 0.71 0.82 0.88

MEX 0.47 0 0.45 0.24 0.14 0.07 0.71 0.64 1 0.54 0.22 0.68 0.85 0.7 0.68

NLD 0.58 0.67 0.9 0.62 0.44 0.74 0.5 0.94 0.18 0.84 0.58 0.86 0.11 0.7 0.51

NZL 0.95 1 0.98 0.7 0.65 1 0.93 0.88 0.63 0.69 0.77 0.71 0.81 0.8 0.91

NOR 0.81 0.87 0.96 0.6 0.41 0.85 0.84 0.66 0.23 0.68 0.73 0.98 0.83 0.78 0.78

POL 0.44 0.6 0.52 0.37 1 0.61 0.82 0.5 0.42 0.53 0.51 0.18 0.77 0.46 0.37

PRT 0.31 0.42 0.4 0.26 0.44 0.31 0.51 0.34 0.06 0.61 0.15 0.61 0.93 0.83 0.33

SVK 0 0.11 0.31 0.24 0.32 0.09 0.22 0.37 0.47 0.73 0 0.3 0.69 0 0

SVN 0.44 0.56 0.42 0.3 0.93 0.51 0 0.46 0.48 0.93 0.37 0.7 0.92 0.5 0.42

ESP 0.49 0.58 0.59 0.24 0.3 0.44 0.68 0.31 0.24 0.73 0.45 0.85 0.84 0.68 0.53

SWE 0.86 0.89 0.82 0.48 0.45 0.77 0.83 0.92 0.47 0.9 0.57 0.84 0.84 0.79 0.89

CHE 0.61 0.77 0.66 0.66 0.53 0.71 0.47 1 0.7 0.88 0.71 0.79 0.01 0.33 0.56

TUR 0.37 0.29 0 0.4 0.45 0.1 0.58 0.72 0.12 1 0.23 0.77 0.86 0.46 0.52

GBR 0.91 0.65 1 0.46 0.54 0.77 1 0.66 0.94 0.76 0.69 0.9 1 0.49 0.96

USA 0.74 0.67 0.86 0.48 0.62 0.73 0.98 0.44 0.65 0.77 0.55 0.7 0.68 0.79 0.73
64 The International Civil Service Effectiveness Index

Annex E: Summary graphs

of country scores and

rankings for each indicator

Core functions

a) Policy making

This indicator currently has four themes: the quality of policy advice; the role of civil servants in setting

strategic policy direction; policy proposal coordination across government; and monitoring policy

implementation. Some proxy metrics have been used for measuring the quality of policy advice. A fifth

theme, assessing the timeliness and accuracy of policy delivery, will be added when data becomes

available. All data for this indicator is drawn from the Bertelsmann Sustainable Governance Indicators (SGI).
The International Civil Service Effectiveness Index 65

b) Fiscal and financial management (FFM)
This indicator has four themes: economic appraisal; economic evaluation; medium-term budgeting

processes; and performance budgeting. Proxy measures have been used for some aspects.
Both the economic appraisal and evaluation themes are measured by the World Economic Forum

Global Competitiveness Index (GCI). The two budgeting themes are measured by two relevant

OECD indexes.
66 The International Civil Service Effectiveness Index

c) Regulation

This indicator has three themes relating to regulation policies and management: method, use and

quality of regulatory impact assessments; stakeholder engagement; and evaluation work. The sole

data source is the OECD’s Indicators of Regulatory Policy and Governance.
The International Civil Service Effectiveness Index 67

d) Crisis/risk management

This indicator has five themes regarding disaster risk reduction and management issues most relevant

for the civil service: integrated risk planning; risk monitoring; public information and awareness raising;
international cooperation and risk coordination; and post-disaster assessment methodology. A sixth

theme on preparedness for disaster response will be added when data becomes available. The data

source for all themes is the UN Hyogo Framework for Action.
68 The International Civil Service Effectiveness Index

e) Human resource management (HRM)
This indicator currently measures two themes: the extent to which civil service recruitment systems

are meritocratic; and attracting and retaining talent. InCiSE hopes to assess four additional themes

when data is available: talent deployment; performance management; quality of learning and

development; and level of civil servant satisfaction with HR services. For the two themes measured,
the data source is the Quality of Government survey.
The International Civil Service Effectiveness Index 69

f) Tax administration

This indicator currently measures three themes: the overall efficiency of tax collection; the extent to

which services are user focused; and the extent and the quality of digital provision. Two additional

themes will be assessed when data becomes available: the prevention of tax evasion; and the level of

tax gap measurement. The existing data sources are the OECD’s Tax Administration Survey and The

World Bank’s Doing Business Index (DBI).
70 The International Civil Service Effectiveness Index

g) Social security administration

This indicator is currently measured via one theme/metric only: assessing administration costs as

a proportion of total expenditure on social security, to capture the overall efficiency of the system.
Additional metrics are planned for future reports as more data becomes available, focused on

the extent to which services are user focused, the extent and quality of digital provision, and the

prevention of fraud. The existing data source is the European Commission (Eurostat). Estimated data

was used for most non-EU countries.
The International Civil Service Effectiveness Index 71

h) Digital Services

This indicator measures four themes: the extent to which services are user focused; the transparency

of services; cross-border mobility of services; and availability of key enablers (technical advances

made possible through, for example, infrastructure investment). It does not assess all the services

which governments typically provide digitally. The sole data source is the European Commission’s

E-Government Benchmark. Estimated data was used for most non-EU countries.
72 The International Civil Service Effectiveness Index

Attributes

a) Integrity

This indicator covers 6 themes: corruption level perceptions; adherence to rules and procedures;
work ethics; fairness and impartiality; striving to serve citizens and ministers; processes in place

to preserve integrity and prevent conflicts of interest. There are four data sources: Transparency

International’s Global Corruption Barometer (GCB); the World Economic Forum (WEF) Global

Competitiveness Index (GCI); the Quality of Government (QoG) Expert Survey Data; and the OECD’s
“Government at a Glance” (GaaG) data.

Fifteen countries have some missing data, partly because of the large number of metrics for this

indicator (16), and so estimated data has been used.
The International Civil Service Effectiveness Index 73

b) Openness

This indicator has six themes: the degree and quality of consultation with society; the existence and

quality of complaint mechanisms; government data availability and accessibility; government data

impact; the right to information; and the publication of laws. There are six data sources: the World

Justice Project’s Open Government Index (OGI); the UN E-participation Index (EPI); Bertelsmann

SGIs; the World Wide Web Foundation: Open Data Barometer (ODB); The Open Knowledge

Foundation: Open Data Index (GODI); and the OECD’s OURdata index.
74 The International Civil Service Effectiveness Index

c) Capabilities

This attribute currently measures only two themes: core capability (eg. problem solving, numeracy

and literacy skills); and the educational attainment of the workforce. InCiSE aims to broaden

the themes for this indicator once data becomes available to include leadership, commercial,
analytical and digital capabilities. The sole data source is the OECD Programme for the International

Assessment of Adult Competencies (PIAAC) survey. A high level of estimated data was used for

some countries.
The International Civil Service Effectiveness Index 75

d) Inclusiveness

This indicator currently has only two themes: the proportion of women represented in the civil service;
and ethnic/religious minority representation. InCiSE hopes to broaden the data coverage over time

once data becomes available on other key representation categories. There are two data sources: the

OECD’s Government at a Glance (GaaG) survey; and the Quality of Government (QoG) survey.
76 The International Civil Service Effectiveness Index

Annex F: Reader’s guide

F.1 Abbreviations

InCiSE International Civil Service Effectiveness

OECD Organisation for Economic Co-operation and Development

UN United Nations

VFM Value for money

GCI Global Competitiveness Index

WEF World Economic Forum

QoG Quality of Government

SGI Sustainable Governance Indicators

EPI E-participation index

ODI Open Data Index

ODB Open Data Barometer

PIAAC Programme for the International Assessment of Adult Competencies

GaaG Government at a Glance

SDG Sustainable Development Goals

OURdata Open, Useful, Reusable data
The International Civil Service Effectiveness Index 77

F.2 Country codes

The following ISO country codes are used in some tables of this report1:

ISO alpha- ISO alpha-
Country 3 digit country code Country 3 digit country code

Australia AUS Republic of Korea KOR

Austria AUT Mexico MEX

Belgium BEL Netherlands NLD

Canada CAN New Zealand NZL

Chile CHL Norway NOR

Czechia CZE Poland POL

Denmark DNK Portugal PRT

Estonia EST Slovakia SVK

Finland FIN Slovenia SVN

France FRA Spain ESP

Germany DEU Sweden SWE

Greece GRC Switzerland CHE

Hungary HUN Turkey TUR

Ireland IRL United Kingdom GBR

Italy ITA United States of America USA

Japan JPN

1
https://www.iso.org/obp/ui/#home
The International Civil Service Effectiveness
(InCiSE) Index project is a collaboration between

the Blavatnik School of Government and the

Institute for Government. The project has been

supported by the UK Civil Service and is funded

by the Open Society Foundations.
The International

Civil Service

Effectiveness
(InCiSE) Index

Technical Report
Users are free to copy, download and print InCiSE content and findings for their own use. Excerpts from
InCiSE reports and publications may also be used, but must be appropriately referenced. Please cite any
uses of this content as: “International Civil Service Effectiveness (InCiSE) Index, 2017”.
The Index is comprised of a variety of other data sources. Reuse is not applicable to data subject to
intellectual property rights of third parties. Please refer to other organisations’ corresponding websites
and data licensing restrictions to ensure compliance with their data limitations. Details of data sources can
be found in Chapter 4 of this technical report. Every effort has been made to adhere to third party data
regulations.
Please contact InCiSE@instituteforgovernment.org.uk for any queries.

Acknowledgements: Our thanks go to those who have given their time to shape this

inaugural pilot publication. This has included substantial contributions made by academics,
think-tanks, international organisations (particularly the OECD), and civil servants past

and present. Our thanks also to those organisations who have allowed use of their data in

the Index.
The International Civil Service Effectiveness Index | Technical Report 1

Contents

Chapter 1: Introduction 3

1.1 Why we need civil service effectiveness indicators 3

1.2 What InCiSE is aiming to do 3

1.3 Structure of this report 4

Chapter 2: Defining the civil service 5

2.1 Alternative definitions of the civil service 5

2.2 Scope of the InCiSE Index 11

Chapter 3: Defining the measurement framework 12

3.1 Purpose of the framework 12

3.2 Principles for the framework 12

3.3 Approach to assessing performance 13

3.4 The InCiSE framework 15

3.4.1 Definitions and justification of the attributes and functions 18

Chapter 4: Measuring against the framework 27

4.1 Data availability 27

4.2 Data underpinning the indicators 31

4.2.1 Functions 31

4.2.2 Attributes 44

Chapter 5: Index country coverage 53

5.1 Countries included in the Index 53

5.2: Imputation method 57

5.3: Imputation approach for indicators 59

5.3.1 Functions 59

5.3.2 Attributes 60

Chapter 6: Index results 62

6.1 Normalisation 62

6.2 Reporting of results 64
2 The International Civil Service Effectiveness Index | Technical Report

Chapter 7: Composite 68

7.1 Strengths and weaknesses of composites 68

7.2 The InCiSE composite 69

Chapter 8: Sensitivity Analysis 72

8.1 Introduction to sensitivity analysis 72

8.2 Sensitivity analysis 1: Adjusting for GDP per capita 72

8.3 Sensitivity analysis 2: Aggregation method 75

8.4 Sensitivity analysis 3: Exclusion of input data 78

8.4.1 Subjective assessment 78

8.4.2 Out of date data 81

8.4.3 Public sector proxy 83

8.5 Sensitivity analysis 4: Imputation method 85

Chapter 9: Next steps 87

9.1 Identified limitations of the Index 87

9.2 Priorities for the next 12 months 88

Appendices 90

Annex A: Data limitations 91

Annex B: Reference list 93

Annex C: Reader’s Guide 96

C.1 Abbreviations 9
7

C.2 Country codes 98
The International Civil Service Effectiveness Index | Technical Report 3

Chapter 1: Introduction

This chapter provides a brief background to the InCiSE Index, and sets out the structure of

the report.

1.1 Why we need civil service The creation of a new and concise set of civil
service effectiveness indicators would serve as:
effectiveness indicators
• An accountability tool: allowing citizens,
An effective civil service can play an important
government officials and politicians to

role in determining a country’s progress and
establish in a clear and concise way how

prosperity. But what constitutes an “effective
well their civil service is performing.
civil service” in the 21st Century? And once a

consensus has been reached on defining this, • A performance improvement tool: enabling

how do civil service leaders know whether their senior decision makers to see which

organisations are effective, and in which areas countries perform best in which areas,
are they performing more strongly than others? and learn from them.
The InCiSE Index seeks to help answer these

questions.
1.2 What InCiSE is aiming to do

A comprehensive set of international indicators
InCiSE aims to define “effectiveness” more

of civil service effectiveness does not currently
extensively than previously. It draws on a wide

exist. This subject area is also well recognised
range of existing international data sources

in academic, international and practitioner
and brings together a set of indicators each

communities as a highly complex area for
measuring a different dimension of civil service

analysis. This is partly because of data
effectiveness – and then produces a composite

limitations, different views on the definitions of
(an overall) score.
“civil service” and “effectiveness”, as well as

the need to take account of country context InCiSE has been developed following

factors when looking at performance issues. a literature review and in consultation

Nevertheless, there are many existing surveys with many experts, including academics

and data collections available globally that can from schools of government, think-tanks

be pulled together to provide a view on civil that monitor government effectiveness,
service effectiveness on an annual basis. international organisations, senior civil servants
(past and present) and subject experts.
4 The International Civil Service Effectiveness Index | Technical Report

InCiSE has also been the subject of an Including this introductory chapter, there are 9

independent, international peer review process. chapters of this report:
Three peer reviewers were selected: a senior

academic from a major European governance • Chapter 2 – Defining the civil service

research institute; a recently retired top civil considers the object of assessment, the

servant with practical experience of civil service “civil service” and establishes a definition

performance issues; and a senior governance of the unit of measurement that allows for

expert with a distinguished career in two major international comparison.
international institutions. Country results were • Chapter 3 – Defining the measurement

provided in an anonymous form for the review. framework sets out a common approach for

Between them, the peer reviewers were asked assessing the effectiveness of a civil service.
to examine the measurement framework of
‘effectiveness’, the methodology and approach • Chapter 4 – Measuring against the

used to produce the indicators, and the data framework describes the data included in the

being used. Index and the weighting of data.

The International Civil Service Effectiveness Index • Chapter 5 – Index country coverage lists the
(InCiSE) project is a collaboration between the countries included in the Index, and explains

Blavatnik School of Government and the Institute our approach to handling missing data.
for Government. The project has been supported
• Chapter 6 – Index results describes how

by the UK Civil Service, and is funded by the
country scores are produced and how results

Open Society Foundations.
are presented.
Despite some current limitations, the Index
• Chapter 7 – Composite discusses the pros

has already brought together a rich volume of
and cons of producing a composite and

data. The launch of this pilot Index will provide
explains how the InCiSE composite Index is

an opportunity to engage with a wide variety of
produced.
stakeholders on a range of issues, stimulating

discussions and feedback. • Chapter 8 – Sensitivity analysis describes
some of the uncertainties associated with

1.3 Structure of this report the modelling process and the subjective
choices, and the consequent impact on the

This Technical Report on the InCiSE Index is Index results.
intended to describe the methodology, data,
and limitations of the approach used. The results • Chapter 9 – Next steps highlights limitations

of the Index can be found in the accompanying of the pilot Index and areas for future

Main Report. consideration to develop the Index.

Given the 2017 publication is a pilot release, this

technical paper will be updated as the Index is

developed and as feedback is received. Should

you have comment on this paper please email

InCiSE@instituteforgovernment.org.uk.
The International Civil Service Effectiveness Index | Technical Report 5

Chapter 2: Defining

the civil service

This chapter considers the object of assessment, the “civil service”, and establishes a

definition of the unit of measurement that allows for international comparison.

2.1 Alternative definitions of the In the 2001 version of the International Monetary
Fund’s Government Financial Statistics Manual1

civil service
and in Eurostat’s European System of Accounts2,
The scope and responsibilities of the civil service the classification of functions of government

varies across countries. Moreover, given that civil is used as an internationally-agreed statistical

services across the world provide a wide-range grouping of government expenses.
of functions and are organised in a variety of
A second definition could take a national

ways, there is a need to establish exactly which
accounts perspective and again, taking a narrow

parts are being assessed, and how.
view, identify civil service entities as those which

In defining the civil service there are a number are owned by government and whose financial

of approaches that can be taken. First, the reporting places them within the System of

civil service can be defined by function. Taking National Accounts (SNA) category of General

a narrow view this could refer to the central, Government. The OECD have undertaken work
“upstream” agencies which set policy direction to establish a terminology and a new definition

and procedural regulation for the “downstream” of what has been called the “public domain”.3

agencies. This could include imposing regimes The new classification is now consistent with the

of transparency or accountability on them and system of national accounts (SNA).
marshalling budgetary and human resources for

them. Those upstream agencies also produce

policy outputs of their own for regulating

economic behaviour and through changing

tax and revenue, expenditure, state-owned-
enterprises and investment policies. A broader

functional perspective encompasses those

agencies which are responsible for service

delivery (although noting that services can be

commissioned or funded, as well as provided

by government).

1 IMF, 2001, Government Finance Statistics Manual (Washington: International Monetary Fund).
2 Eurostat, 2007, Manual on Sources and Methods for the Compilation of COFOG Statistics: Classification of the Functions of
Government (COFOG), Eurostat Methodologies and Working Papers (Luxembourg: Office for Official Publications of the European
Communities).
3 Pilichowski, E. and E. Turkisch (2008). Employment in Government in the Perspective of the Production Costs of Goods and Services in
the Public Domain. OECD, Paris; OECD (2009), Measuring Government Activity, Paris, OECD.
6 The International Civil Service Effectiveness Index | Technical Report

The darkest shaded areas in Table 1 delineate An even broader SNA perspective would

those institutions which might be regarded pull in state-owned enterprises selling goods

as part of the civil service, namely the or services at an economically significant

administrative units of central governments price, and financial and non-financial public
(and state governments, if a wider view of what corporations – to the extent that these are

constitutes the civil service is taken) and the controlled by government as set out in the SNA.
typically very small (in terms of employment, Although this category would not be included

not fiscal impact) social security funds at each as part of the civil service by practitioners or

level of government. Entities in government often researchers, some have argued that the core

lack a legal personality and are hence unable public sector, broader than the civil service

to own assets and incur liabilities which are as usually understood, should include market

reliant on central financial authorities. As Table producers and non-profits which are to all intents

1 shows, for the SNA, General Government and purposes entirely reliant on the public sector

agencies include Ministries and departments in as monopsony customer, and private enterprises

central government, along with state and local which have been granted a distinctive and

governments if taking a wider view. Since it is an statutorily privileged market position.
accounting and not a management perspective,
This is on the basis that while there may be little

project implementation unit arrangements are
direct public funding for these agencies there is

covered to the extent that they are included in often an assumption that government will meet

the budget (even if using consultants outside any implicit contingent liability that arises, with

of any civil service regime). A less often noted the probability that government would, in the

but occasionally significant group of agencies event of major operational failure, underwrite

within General Government encompasses the debts of these entities. They are thus

those organisations that are largely funded and underpinned by an implicit guarantee.
controlled by state or provincial government

but not owned by government. In some

countries, this includes schools and hospitals

which are fully funded by government and

where government determines the employment

and financial regimes, but where there is

an independent owner who is acting on the

government’s behalf, often a charity or other

not-for-profit.
The International Civil Service Effectiveness Index | Technical Report 7

Table 1: The public sector from a National Accounts perspective

Institutional domain How transactions are Examples
recorded in the national
accounts

The Public General Central Administrative units in Ministries and
“public sector government5 government central government departments in

domain”4 central government

All non-market non- Schools, hospitals,
profit institutions that are etc. that are
controlled and more than 50 largely funded
percent financed by central and controlled by
government units central government
but not owned by
government

State Administrative units in state Departments in
governments government states, provinces

All non-market non- Schools, hospitals,
profit institutions that are etc. that are
controlled and more than 50 largely funded
percent financed by state and controlled by
government units state or provincial
government but
not owned by
government

Local Administrative units in local Departments
governments government in counties,
municipalities

All non-market non- Schools, hospitals,
profit institutions that are etc. that are
controlled and more than largely funded
50 percent financed by local and controlled by
government units local government
but not owned by
government

Social All social security funds at Health fund,
security each level of government unemployment fund,
funds pension fund

Other public sector Market producers, controlled Publicly owned
by government, selling banks
goods or services at an Publicly owned
economically significant harbors, airports
price (“public enterprises”):
• Public financial (quasi-)
corporations
• Public non-financial
(quasi-) corporations
As defined by S.11 and S.12
in the SNA.
8 The International Civil Service Effectiveness Index | Technical Report

Table 1: The public sector from a National Accounts perspective

Private sector in the public domain Market producers, whose Profit or non-profit
indirect public funding private hospitals
comprises more than 50 accessible to
percent of total revenue: publicly insured
• Non-profit institutions clients
• Profit institutions
As defined by S.11, S.12 in
the SNA

Non-profit institutions Schools, hospitals,
serving households, etc. that are
financed more than 50 largely funded
percent by government, by government
but not controlled by but not owned
government: nor controlled by
• Non-profit institutions government
serving households
As defined by S.15 in the
SNA

Private enterprises with a Energy companies,
distinctive and statutorily local public transport
privileged market position: companies
• Private sector utilities National train
licensed to operate in company
very limited markets
(water, energy,
sewage, waste
disposal, post, but not
telecommunication)
Legal monopolies
As defined by S.11 in the
SNA

Source: Developed largely from Pilichowski and Turkisch6 with OECD7

Note: The darkest shaded area highlights those institutions which might be regarded as part of the civil service, namely

the administrative units of central governments. The medium shaded area covers those categories, in addition to the

darkest shaded area, considered part of General Government under the SNA. The lighter shaded area covers state-owned

enterprises selling goods or services at an economically significant price, and financial and non-financial public corporations
– to the extent that these are controlled by government as set out in the SNA.

4 This description is devised for this purpose and is not a recognized SNA term.
5 As defined by section S.13 in the System of National Accounts).
6 Pilichowski, E. and E. Turkisch (2008). Employment in Government in the Perspective of the Production Costs of Goods and Services in
the Public Domain. OECD, Paris.
7 OECD (2009), Measuring Government Activity, Paris, OECD.
The International Civil Service Effectiveness Index | Technical Report 9

The final approach explored was of a civil service This can present practical difficulties as, although

determined on the basis of employment regimes generally the civil service is understood to
(Figure 1). The most stringent definition limits the constitute a distinct body of staff within the

selected entities to those which are required to public sector, staff that are commonly referred

hire most employees under the civil service law, to as “civil servants” do not always have legally

excluding those using other legal employment distinct employment contracts. When it exists

regimes. formally, the essence of civil servant status is that
the legal basis for employment – the laws and
regulations that shape the nature of employment
contracts – is different from that found elsewhere
in the economy as defined by the general labour
law. It also is generally different from that found
elsewhere in the public sector, such as in the
health or education sectors or in state-owned
enterprises.

Figure 1: Typical legal employment regimes in general government

TOTAL PUBLIC EMPLOYMENT

SOE employees General Government

total civilian total subnational
Armed Forces central government government

total education
employees { Education Education

total health
employees { Health Health

total police { Police Police

Civilian Central Subnational
Government Government
excluding excluding
education, education,
health and health and
police police

often known as
“civil servants” { permanent
employees
permanent
employees

daily paid temporary temporary
employees employees employees

Source: World Bank 20078

8 World Bank (2007). The World Bank’s Administrative and Civil Service Reform Website. World Bank.
10 The International Civil Service Effectiveness Index | Technical Report

Historically, civil service employment was The three methods offered here for defining

not a formal agreement between two equal the centre of government based on functions,
parties, but rather a decision of the State, national accounts or employment regimes omit

with appointments made by an authorised other less useful criteria that could be applied,
public institution in accordance with the civil such as:
service law, with tight regulations that shape

the appointment process, and with many • A legal definition of the centre of government

constraints on dismissal. Today, civil service could comprise those entities that are

employment tends to share some features that created under the authority of the constitution

are typical of a voluntary arrangement between or by public law.12

an employer and employee in the private sector. • Alternatively, the centre of government could

There can be other arrangements, particularly be defined as the set of all entities that

in the health, education, military and police operate directly under the authority of the

sectors, that provide civil service-like protections political executive.
and responsibilities for public employees who

are not, in fact, civil servants. Subnational • Ownership is a further set of ideas for

government employment is considered a defining the institutional units that belong

separate, legally defined civil service in many to the public sector, drawing on accounting

countries.9,10 A number of organisations including standards for criteria concerning which body

the International Labour Organisations and the has the power to govern the financial and

OECD have considered the legal basis for civil operating policies of another entity.13

servant status.11
The problem with each of these alternative
conceptions is that they result in a large and
unwieldy group of agencies which, most
importantly, is rarely consistent with professional
or other working definitions.

9 Gow, J. I. and M. d. C. Pardo (1999), ‘Comparing Different Civil Services’, International Review of Administrative Sciences, 65 (4), 527-
550.
10 Cardona, F. (2000). Scope of Civil Services in European Countries: Trends and Developments. Seminar at the European Institute of
Public Administration. Maastricht, Sigma/OECD

11 SIGMA (1996a), Civil Service Legislation Contents Checklist – Sigma Paper No. 5, Paris, OECD.
12 Gill, D. (2002), ‘Signposting the Zoo – from Agencification to a More Principled Choice of Government Organisational Forms’, OECD
Journal on Budgeting, 2 (1), 27-80

13 Lienert, I. (2009). Where Does the Public Sector End and the Private Sector Begin? . IMF, Washington DC.
The International Civil Service Effectiveness Index | Technical Report 11

2.2 Definition in the InCiSE Index

The approach taken for the InCiSE Index is to define the scope of ‘civil services’ by outlining and

measuring performance on the core functions of civil services; the parts which can generally be

classified as civil service in every country. This approach leads to a focus on (i) functions which deliver

services or affect citizens directly and (ii) public management and policy functions carried out at the

centre of government.

Table 2 gives more detail about what is included and excluded in the InCiSE Index:

Table 2: Public sector areas included in the InCiSE framework

Parts/functions of the public sector Degree of inclusion in measurement framework

Civil service functions which deliver services A primary focus of the InCiSE Index with each
to citizens directly (e.g. tax and social security function assessed in an individual indicator.
administration at the central/federal level).

Central, public administration-type civil service A primary focus of the Index with each function
functions (e.g. fiscal management, policy making, assessed in an individual indicator.
regulation).

The ‘mission support’ functions which support these A primary focus of the Index.
core service delivery and central administration-type
functions.

The parts of civil services which direct and support Performance captured by indicators on the functions
the wider public sector on specific policy areas (e.g. above which cut across most of the policy areas
Health Ministry, Education Ministry, Environment governments typically deal with. For example
Ministry), but may not deliver public services to regulation, and policy making more generally, cut
citizens directly. across a number of policy areas including health,
education, environment, and so on. Performance
on specific policy areas not individually assessed
because:

  1. Data is unlikely to exist which sufficiently isolates
    the ‘oversight’ specific ministries provide over
    different policy areas from the service delivery the
    wider public sector provides in these areas; and
  2. Governments in all countries deal with a large
    number of policy areas and there is a need to limit
    the scope of the measurement framework.

    The wider public sector itself (e.g. workers in public Out of scope, although occasionally public sector
    hospitals, schools and police departments). data is used to proxy for civil service performance.

    Local government. Out of scope. Whilst public servants working in
    sub-national governments may in some countries be
    technically classified as ‘civil servants’, the scope of
    this Index is primarily concerned with the civil service
    at central government level.

The unit of analysis of interest for the InCiSE Index is the civil service, not the public sector more

generally. Having said that, isolating civil service performance with currently available data is difficult,
particularly given the varying sizes and shapes of civil services internationally. This issue is discussed

further in Chapter 9 and Annex A which highlight some limitations of the Index.
12 The International Civil Service Effectiveness Index | Technical Report

Chapter 3: Defining the

measurement framework

Chapter 3 considers the purpose of designing a framework for the Index, the principles

specified, the framework derived and the justification for its components.

3.1 Purpose of the framework This framework, and the resultant Index, provides
civil services across the world with a powerful

The purpose of the framework is to define tool for (i) accountability, allowing citizens,
a common approach for assessing the commentators and ministers to establish how

effectiveness of a civil service, in a way which well their country’s civil service is performing and

could realistically enable international data to (ii) performance improvement, for example by

be collected to measure against it. Whilst there allowing senior decision makers to see which

are many alternative ways one could define civil countries perform best in which areas and learn

service effectiveness, the framework outlined from them. This learning potential would be

here, and the themes and dimensions therein, enhanced if the Index could connect with and

is informed by evidence such that if a civil service inform existing learning and research initiatives in

were to score highly against it, it is reasonable the field.
to conclude that the civil service would be

high-performing relative to its international 3.2 Principles for the framework

counterparts.
Prior to developing the framework we specified

Our approach to deriving a common framework a list of general principles for our indicators.
was to: These principles are shown in Table 3.
• Specify and adhere to a set of principles to
inform the development of the framework
(3.2);

• Draw on evidence (3.4.1) to identify key
features of a draft framework (chapter 4),
which was then extensively tested through
consultation;

• Build on existing indicators and data where
possible, while striving to develop a more
comprehensive framework capturing all
aspects of civil service effectiveness;

• Refine the framework through consultation
with a number of experts, including
academics, think-tanks, international
organisations, civil servants (past and
present) and subject experts.
The International Civil Service Effectiveness Index | Technical Report 13

Table 3: Principles for the InCiSE framework14

• Coherent framework of what are the key elements and drivers of an effective public administration.
• Comprehensive, meaning that it covers all relevant aspects and drivers of the performance of public
administration.
• Actionable, offering genuine policy insights into what drives excellent public administration performance
that can be implemented.
• Transparent in its methodology and assessment process to ensure credibility, robustness and
replicability.
• Feasible to collect from a large group of countries at reasonable cost.
• Replicable to allow for both time and cross country comparison.

The pilot Index provides actionable insights, 3.3 Approach to assessing

and is built on a coherent framework that is
performance

transparent, feasible and replicable:
The standard approach for assessing civil
• The indicator draws on literature to determine
service effectiveness would be to think in terms
the key elements of an effective civil service
of inputs, outputs and outcomes. This is the
and builds on the work of other indicators
approach taken by some recent studies15.
and data collection.
However, when looking specifically at civil
• All source data is specified and the services and the public administration-type
methodology to derive the Index set out functions they provide, this approach seems less
in this report, promoting transparency and attractive. While output and outcome measures
replicability. may seem to cut through the conceptual
uncertainty and simply ask what got done, in
• The Index is feasible to produce for a large practice they are likely to be problematic for
group of countries on a regular basis, three reasons:
largely due to its draw on existing data,
use of imputation for missing scores, and 1. They can be affected by exogenous factors,
normalisation of data to allow comparison. making it difficult to isolate the contribution of
the civil service.
Section 3.4 sets out the range of functions and

attributes which the Index aims to measure. 2. Measuring output is itself problematic

Work will be required post pilot publication to methodologically; difficulties include defining

develop the framework further. To this end we the units of output and obtaining information

encourage feedback as to how the indicator as to the value of this output, due to public

framework can be further strengthened. sector output being provided for free or for
an economically insignificant amount.

  1. Normative and procedural concerns
    (i.e. how the output or outcome was achieved)
    are relevant to effectiveness.16

14 A number of these principles were also used to develop World Bank ISMPS project http://www.worldbank.org/en/topic/governance/
brief/indicators-of-the-strength-of-public-management-systems

15 Goderis et al (2015), ‘Public Sector achievement in 36 countries: a comparative assessment of inputs, outputs and outcomes’,
The Netherlands Institute for Social Research.
16 Wieland J. (2014) Governance Ethics: Global value creation, economic organization and normativity, Springer International Publishing,
p205
14 The International Civil Service Effectiveness Index | Technical Report

Ascribing causality between public administration However, certain procedural measures remain

performance and outcomes is too difficult: “It is at the core of any measure of effectiveness,
reasonable to conclude that because causality in both where there is evidence to support the

achieving public policy outcomes is notoriously relationship between such procedures and

hard to assign, the quality of outputs is very hard positive outcomes, and because procedures

to determine.17” Without the certainty that there may have a benefit in themselves. For example,
is a causal relationship between a government meritocracy of recruitment procedures in the

output and the achievement of a stated policy civil service are important because there is a

objective, we cannot know the quality of the considerable evidence base to support the

output because quality, in this context, is a relationship between such procedures and

measure of how well the output is contributing to outcomes associated with an effective civil

the objective. service. However, the extent to which recruitment
processes reward merit is also important for the

Given this difficulty, the preferred approach principle of fairness which is valued in itself.
here is to focus on the effectiveness of the

procedures within the civil service which (often Aggregate inputs, such as the total human and

indirectly) affect those outcomes. The approach financial resource put in, are not measured at

deployed is therefore more process and output this stage. It may be important to compare

focused. Beyond just feasibility, one could argue performance on the framework against those

an advantage of process-based indicators aggregate inputs (for example this could be

is that they are more instructive for potential done in the form of a civil service efficiency ratio;
performance improvements as it is processes the ratio of the score for an indicator to the

which are ultimately changed to increase overall inputs). We have however examined the

effectiveness. We acknowledge the problem with sensitivity of the Index when taking into account

all procedural definitions of effectiveness that the the relative wealth of countries as measured

procedures, however defined, may not actually by GDP per capita (see Chapter 9 for more

correlate with the positive outcomes. information).

17 Holt J. and Manning N. (2014), ‘Fukuyama is right about measuring state quality: now what?’, Governance: An International Journal of
Policy, Administration, and Institutions, 27(4).
The International Civil Service Effectiveness Index | Technical Report 15

Figure 2 below shows what is captured in the every organisation, mission support functions

indicator set. support these core external functions (on both
sides). By looking across all 3 types of function,
The logic model highlights the focus on the the aim is to measure how well civil services

effectiveness of the procedures within the civil deliver the core elements of their roles.
service rather than outcomes, leading to the

more process and output focused approach Figure 3 shows the functions currently included

deployed. in the InCiSE framework.

Section 3.4.1 sets out the definition of each of

3.4 The InCiSE framework these functions, along with a justification for their

The InCiSE framework starts by defining the inclusion in the framework.
core characteristics of an effective civil service.
Attributes

To do this, it assesses effectiveness on the basis

of two inter-related dimensions, 1) the delivery Every civil service also has an underlying set of

of its core functions and 2) an underlying set attributes which are important drivers of how

of attributes which are important drivers of effectively they deliver core functions. These

effectiveness across all parts of the civil service. attributes should apply to all parts of the civil
service; they are not specific to particular parts

Functions or functions. The inclusion of attributes in the

Civil services across the world are of different framework is based on both a normative and

shapes and sizes, but there are certain core positive judgement: civil services should aim

functions which all of them deliver. On one side, to cultivate and demonstrate these attributes,
they deliver a set of central executive functions as they are commonly (but not necessarily

for ministers. These may help to formulate policy universally) understood as aspects of best

for the country (the effects of which are borne by practice, and the included attributes should

citizens). On the other side, civil services interact generally be determinants of performance across

more directly with citizens through the delivery of all functions. Figure 4 shows those attributes

services such as tax administration. As with currently included in the InCiSE framework.

Figure 2: Logic model of the civil service
Inputs Activities/ Outputs Outcomes Impact
procedures
Human and financial These inputs can be These activities These services These outcomes
resources are used to complete and procedures affect outcomes can lead to changes
needed to operate activities and help the civil service for citizens in communities
the civil service. procedures the civil deliver the services (e.g. educational and organisations
service undertakes it provides to outcomes), across the country.
ministers and ministers (e.g. trust
citizens. in government),
and the civil service
(e.g. trust in the
civil service).

Captured in parts Captured in parts Not captured in the
of the indicator of the indicator indicator set.
set but not the Primary focus of the indicator set. set but not the
primary focus. primary focus.
16 The International Civil Service Effectiveness Index | Technical Report

Section 3.4.1 sets out the definition of each of these attributes, along with a justification for their

inclusion in the framework.

Figure 3: InCiSE Core Functions

Central executive Mission support Direct service delivery
Policy making: The quality Procurement: The extent Tax administration: The
of the policy making to which the procurement efficiency and effectiveness
process, including how process is efficient, of tax collection (at the
policy is developed competitive, fair, and central/federal level).
and coordinated across pursues value for money.
government and monitored
during implementation.

Fiscal and financial HR management: The Social security
management: The quality meritocracy of recruitment administration: The
of the budgeting process and extent to which civil efficiency and effectiveness
and the extent to which servants are effectively of social security
spending decisions are attracted, managed and administration (at the
informed through economic developed. central/federal level).
appraisal and evaluation.

Regulation: The extent Information technology: Digital services: The user-
and quality of regulatory The extent to which civil centricity, transparency
impact assessments and servants have the digital and cross-border mobility
the degree of stakeholder tools to work efficiently. of digitally-provided public
engagement involved in services and the availability
them. of ‘key enablers’.

Crisis/risk management: Finance: The extent
The effectiveness with to which operations
which the government are supported by well-
engages the whole of managed, efficient finance
society to better assess, systems, particularly on the
prevent, respond to and alignment of finance with
recover from the effects of the business strategy and
extreme events. the level of civil servant
satisfaction with finance
support.

Figure 4: InCiSE Attributes

  1. Integrity: The extent to which civil 4. Inclusiveness: The extent to which
    servants behave with integrity, make the civil service is representative of the
    decisions impartially and fairly, and citizens it serves.
    strive to serve both citizens and
    ministers.
  2. Openness: The regular practise and 5. S
    taff engagement: Staff levels of pride,
    degree of consultation with citizens attachment and motivation to work for
    to help guide the decisions we make their organisation.
    and extent of transparency in our
    decision‑making.
  3. Capabilities: The extent to which the 6. Innovation: The degree to which new
    workforce has the right mix of skills. ideas, policies, and ways of operating
    are able to freely develop.
    The International Civil Service Effectiveness Index | Technical Report 17

The attributes and functions identified are brought together to form the InCiSE framework shown

in Figure 5.

Figure 5: InCiSE Index framework

Overall Measurement framework Outcomes
inputs (process and output focussed) for citizens

  1. Total human Civil Service
    resource
  2. Total financial Central Mission Direct
    resource
    executive support service
    functions functions delivery
    Outputs functions Outputs
    to ministers to citizens
  3. Policy making 1. Procurement 1. Tax administration
  4. Fiscal and 2. HR (at the central/
    financial management federal level)
    management
  5. IT 2. Social security
  6. Regulation administration
  7. Finance
    Ministers 4. Crisis/Risk (at the central/ Citizens
    management federal level)
  8. Digital services

Attributes

  1. Integrity 2. Openness 3. Capabilities 4. Inclusiveness
    Policy effects 5. Staff engagement 6. Innovation Policy effects

    (‘Value for money’ and ‘use of evidence’ are other key attributes
    identified. However, these concepts are already captured on the
    functions side of the framework to the extent that their inclusion
    as attribute indicators is considered unnecessary.)

This framework shows the attributes which drive and support the successful delivery of the civil

service across the three categories of functions. The ‘mission support’ functions (Procurement,
HR Management, IT and Finance) underpin both the ‘central executive functions’ providing outputs

to ministers and the ‘direct service delivery’ functions providing outputs to the public. For the reasons

described in section 3.3, citizen outcomes are not currently included directly in the framework,
although it is important to bear in mind that all procedures and outputs within the framework are

delivered with a view to influence them.
18 The International Civil Service Effectiveness Index | Technical Report

3.4.1 Definitions and justification of Policy Making

the attributes and functions Definition: The quality of the policy making
process, including how policy is developed and

In this section we set out the definition for coordinated across government and monitored

each attribute and function specified in Figures during implementation.
2 and 3, and the justification for its inclusion

in the framework from our literature review. Justification: Policy making remains a

A bibliography of references in this section central role of a civil service and the quality

can be found in Annex C. of evidence and appraisal are central to the
success of policy. Kaufmann et al. outline
three functions of good governance, including
‘the capacity of government to effectively
formulate and implement sound policies’
(1999). Policymakers need to ‘receive rigorous
analyses of comprehensive background
information and evidence, and of the options
for actions,’ according to the Office for Public
Management (OPM) and the Chartered Institute
of Public Finance and Accountability (CIPFA)
(2004). This paper also advises that ‘good
quality information and clear, objective advice
can significantly reduce the risk of taking
decisions that fail to achieve their objectives
or have serious unintended consequences’.
The necessity of sound evidence and reliability
in policymaking is echoed by Bovaird and Löffler:
‘The choice of a particular policy direction should
be informed by existing evidence on what has
been tried elsewhere and whether it has been
demonstrated to deliver the desired benefits,’
and ‘evidence can be used both to facilitate
accountability and to promote improvement in
policy-making, programme development and
service delivery’ (2003).
The International Civil Service Effectiveness Index | Technical Report 19

Fiscal and Financial Management Regulation

Definition: The quality of the budgeting process Definition: The extent and quality of regulatory

and the extent to which spending decisions impact assessments and the degree of

are informed through economic appraisal stakeholder engagement involved in them.
and evaluation.
Justification: The appropriate appraisal and

Justification: Fiscal and financial management evaluation of regulatory changes accompanied

is an important measure of every system of by sufficient stakeholder engagement is

public administration. The Indicators of the crucial to ensuring that any introductions are

Strength of Public Management Systems fully considered and fair, involving various
(ISPMS) from the World Bank state `Public stakeholders. This scrutiny is endorsed by

sector management arrangements must also many; the OECD, for instance, ‘recognis[es]
encourage fiscal and institutional sustainability that regulations are one of the key levers by

as less tangible but equally critical outcomes’ which governments act to promote economic

and ‘Reforms of budgetary and financial prosperity, enhance welfare and pursue

management systems… are often crucial for the public interest,’ and that ‘well-designed

development outcomes’ (2012). Holt and regulations can generate significant social and

Manning also consider that ‘public administration economic benefits which outweigh the costs of

practitioners break down the functioning of the regulation, and contribute to social well-being,’
central agencies into five management systems,’ (2012). The International Monetary Fund (IMF)
including fiscal and financial management which acknowledges the importance of regulatory

is made up of: ‘planning and budgeting; financial frameworks to successful governance: ‘From

management; and accounting, fiscal reporting the perspective of the IMF, countries with good

and audit.’ The OECD’s recommendation paper governance have strong legal and regulatory

on budgetary governance sets out ten principles frameworks in place,’ (2016). Additionally,
for good budgetary governance which include in promoting best practice, ‘[the] Regulatory
‘ensur[ing] that performance, evaluation and Impact Analysis (RIA) is a multiple stakeholder

value for money are integral to the budget assessment of the economic, environmental

process’ and ‘manag[ing] budgets within clear, and social impact of regulations. The OECD and

credible and predictable limits for fiscal policy,’ European Union have strongly promoted this
(OECD, 2015a). evidence-based approach towards legislation’
(Bovaird and Löffler, 2003).
20 The International Civil Service Effectiveness Index | Technical Report

Crisis/Risk Management Procurement

Definition: The effectiveness with which the Definition: The extent to which the procurement

government engages the whole of society to process is efficient, competitive, fair, and pursues

better assess, prevent, respond to and recover value for money.
from the effects of extreme events.
Justification: ‘Government procurement

Justification: The OECD Strategic Crisis accounts for an average of 15 per cent or more

Management report highlights crisis of a country’s GDP,’ (World Trade Organisation,
management as central to government’s role and 2015). As procurement makes up such a

a ‘fundamental element of good governance’ large proportion of countries’ GDP, it must be
(Baubion, 2013). New Zealand measures their managed appropriately. Effective procurement

public sector outputs against eight identified management can streamline contracts and

categories, of which two encompass contingent reduce outgoings, contributing to improved

and emergency capabilities: ‘so that an efficiencies in civil services. Phillips et al. also

adequate response will be available in time observe that: ‘effective procurement practices

to minimise loss, damage or injury,’ cited by provide governments with a means of bringing

Van Dooren et al. (2006). Studies have shown about social, environmental and economic

that credibility and trust in governments to reform’ (2007). On public procurement, the

deal with crises is vital both to reassure and World Bank states it ‘is a key variable in

encourage support from the private sector and determining development outcomes and,
general public, as outlined by Christensen et when carried out in an efficient and transparent

al. (2011). In the OECD’s recommendation of manner, it can play a strategic role in delivering

the council on the governance of critical risks, more effective public services. It can also act as

it is ‘recognising that effective risk governance a powerful tool for development with profoundly

is a means of maintaining or achieving national positive repercussions for both good governance

competitive advantage against a backdrop of and more rapid and inclusive growth’ (2016).
numerous geopolitical, environmental, societal

and economic uncertainties as it represents an

opportunity to invest in safer and better lives for

the future’ and ‘recognising that citizens and

businesses expect governments to be prepared

for a wide range of possible crises and global

shocks and to handle them effectively should

they arise,’ (OECD, 2014).
The International Civil Service Effectiveness Index | Technical Report 21

Human Resource Management Information Technology

Definition: The meritocracy of recruitment and Definition: The extent to which civil servants

extent to which civil servants are effectively have the digital tools to work efficiently.
attracted, managed and developed.
Justification: It is important for the public

Justification: ‘The public sector is very labour sector to keep up with IT developments in

intensive – around 70 per cent of the budgets order to maintain optimum efficiency. Advanced

of most public organisations are spent on IT can also allow for improvements in flexible

staff’ (Bovaird and Löffler, 2003), so good HR working patterns for civil servants. Madzova

management is key to the successful functioning et al. make the point that improved IT ‘can

of an exemplary civil service. Performance enhance the speed and efficiency of operations,
management can help to create incentives by streamlining processes, lowering costs,
for personal development in the civil service. improving research capacities and record

Fukuyama (2013) recognises that recruitment keeping,’ (2013), highlighting the cost saving

and reward ‘remain at the core of any measure which could be achieved. Magno and Serafica

of quality of governance… whether bureaucrats go further to identify three ways in which IT

are recruited and promoted on the basis of promotes good governance: ‘(1) by increasing

merit’. ‘If the HR policies are not right then transparency, information, and accountability;
public organisations will not attract the human (2) by facilitating accurate decision-making and

resources they need to perform the functions public participation; and (3) by enhancing the

of government and deliver the services that efficient delivery of public goods and services,’
government has promised the electorate’ (2001). However, skills must be in place to
(Bovaird and Löffler, 2003). harness this capability, and Fukuyama (2013)
acknowledges the technical expertise of civil
servants ‘remain[s] at the core of any measure of
quality of governance’.
22 The International Civil Service Effectiveness Index | Technical Report

Finance Tax Administration

Definition: The extent to which operations are Definition: The efficiency and effectiveness of

supported by well-managed, efficient finance tax collection (at the central/federal level).
systems, particularly on the alignment of finance

with the business strategy and the level of civil Justification: Effective tax systems can be

servant satisfaction with finance support. viewed as a critical building block for increased
domestic resource mobilisation which is

Justification: Managing government finance in essential for civil service effectiveness and good

the context of business operations contributes governance. ‘Successful tax extraction provides

towards value for money for the taxpayer to resources that enable the government to

the same extent as fiscal spending. ‘Taxpayers operate in other domains,’ Fukuyama highlights

are entitled to receive assurance that Public ‘it is a necessary function of all states, and

Administrations take due care in managing one for which considerable data exist’ (2013).
funds,’ and ‘the issue of adequate internal The role of tax administration as the basis of

control is at the heart of sound financial government operations in made clear by the

management of the national budget.’ (European OECD, ‘Strong tax administrations and sound

Commission, 2006). Bouchard and McCrae public financial management help maximise

point out that the ‘Control of money and the domestic resources that are necessary for

budgets is a core role of any finance function,’ government to function, to sustain social safety

and they find that ‘previous work on improving nets, to maintain long-term fiscal sustainability,
decision making has highlighted the importance and to free up fiscal space for pursuing socio-
of financial leadership and strengthened economics objectives,’ (n.d.). Although priorities

performance management at the top of and circumstances vary widely across countries,
government departments’ (2013). the drive to elevate the collective standard of
tax administration is of great importance. Holt
and Manning highlight the importance of tax
administration in measuring the effectiveness
of public administration and it is one of the
key functions highlighted by the World Bank
Indicators of the Strength of Public Management
Systems (2012).
The International Civil Service Effectiveness Index | Technical Report 23

Social Security Administration Digital Services

Definition: The efficiency and effectiveness Definition: The user-centricity, transparency and

of social security administration (at the central/ cross-border mobility of digitally-provided public

federal level). services and the availability of ‘key enablers’.

Justification: Social security administration Justification: A changing world and digital

plays an important role in civil services. environment provide the impetus for a civil

The OECD study Towards Better Measurement service to ensure modernity and remain user-
of Government, for example, highlights the centric for the public. In doing so, efficiencies

importance of social security administration. should be achieved to enable cost savings

Chalam says that ‘Social security is a human in processes while also allowing for further

right as well as a social and economic necessity. accessibility of services. The OECD has

All successful societies and economies have supported this view of potential benefits:
employed developmental strategies where ‘ICT is increasingly used to support broader

social security systems played an important public sector development objectives… by

role to alleviate poverty and provide economic changing service delivery approaches by

security that helps people to cope with life’s creating personalised, high quality services to

major risks or the need to quickly adapt to users, thereby increasing user satisfaction and

changing economic, political, demographic and effective service delivery; facilitating major work

societal circumstances’ (2014). McKinnon (2011) organisation and management changes creating

supports this view: ‘Several legal instruments back-office coherence and efficiency gains;
adopted by the United Nations recognize social increasing transparency of government activities,
security as a basic human right. The State and increasing citizen engagement.’ (Lonti and

has the responsibility to create the enabling Woods, 2008).
environment that would allow citizens to exercise

this right,’ as well as pointing out ‘the role of

social security as a requisite economic and social

stabilizer’. In summary, ‘Social security systems

play a central role in the efforts of every country

to promote and ensure the social and economic

well-being of its citizens’ (McKinnon, 2011).
24 The International Civil Service Effectiveness Index | Technical Report

Integrity Openness

Definition: The extent to which civil servants Definition: The regular practise and degree

behave with integrity, make decisions impartially of consultation with citizens to help guide the

and fairly, and strive to serve both citizens and decisions we make and extent of transparency in

ministers. our decision-making.

Justification: Integrity is one of the core values Justification: The need for transparency within

associated with a civil service. The International a civil service is imperative for the public to trust

Civil Service Commission highlights the and feel empowered to hold the government

importance of integrity to the work of United accountable for their actions, whilst reducing

Nations (UN) common systems staff: ‘The corruption. The World Bank ‘supports efforts to

concept of integrity… embraces all aspects encourage open and transparent government’
of behaviour of an international civil servant, in their advice for helping governments to

including… honesty, truthfulness, impartiality strengthen institutions against corruption (The

and incorruptibility. These qualities are as basic World Bank, n.d.). The United Nations outlines

as those of competence and efficiency.’ (Civil the need for transparency and accountability in

Service Commission, 2002). The World Bank governance; ‘[this] implies a proactive effort to

states: ‘A well-performing civil service resists make information accessible to citizens’ and it

petty corruption and provides the staff for is ‘one indicator of a government that is citizen-
many of the institutions that protect integrity in focussed and service-oriented’. (United Nations,
government’ (n.d.) while setting out the role of 1999). Graham et al. also make reference to

the World Bank in helping countries combat the United Nations Development Program’s

corruption. Indeed, the Charter of the United five principles of good governance, in which

Nations lists integrity as key to the qualities transparency is identified as a key characteristic.
employees must show: ‘The paramount The OECD summarises the importance of

consideration in the employment of the staff openness in that ‘access to information and

and in the determination of the conditions of proactive transparency help build citizens’ trust

service shall be the necessity of securing the in government.’ (n.d.).
highest standards of efficiency, competence,
and integrity.’ (United Nations, 1945). This paper

also outlines the need for impartiality as key to

independence of a good civil service as well as

Kaufmann et al. (1999) and Huther and Shah
(1999). Numerous studies aiming to establish

good governance have utilised similar metrics

in their analyses, for instance Muiithi et al. at the

LSE (2015). Therefore, the inclusion of integrity in

the InCiSE is deemed necessary and crucial for

the assessment of an effective civil service.
The International Civil Service Effectiveness Index | Technical Report 25

Capabilities Inclusiveness

Definition: The extent to which the workforce Definition: The extent to which the civil service is

has the right mix of skills. representative of the citizens it serves.

Justification: The need for a variety of certain Justification: A modern civil service should be

strong skills is vital for the successful operation representative of the public it stands to serve.
of any organisation, civil services included. In order to do so, institutions must be inclusive in
‘Public service organisations need people nature. ‘Governments are increasingly concerned

with the right skills to direct and control them about the importance of diversity in public

effectively’ (OPM and CIPFA, 2004). The institutions, to ensure that the needs, aspirations

standards for good governance set out by and experiences of a diverse range of citizens

the Office for Public Management (OPM) and are reflected in the decision-making process,’
the Chartered Institute of Public Finance and (OECD, 2015b). In their Post-2015 Millennium

Accountability (CIPFA) which outline seven Development Goal reflections, the OECD

principles for people in public life (known as the outlines the greater success felt by inclusive

Nolan principles), include leadership as a core public bodies: ‘Inclusive governments and an

skill. It goes on to list necessary skills as ‘the active civil society put forward more responsive,
ability to scrutinise and challenge information… equitable policies’ and that these ‘build trust in

including skills in financial management and the government and help create… public services

ability to recognise when outside expert advice that are better suited to diverse needs,’ (n.d.).
is needed,’ (2004). Fukuyama acknowledges The guiding principles to the international

the importance of educational attainment of civil civil service, set out by the International Civil

servants: ‘Another critical measure of capacity is Service Commission, support the claim

the level of education and professionalization of that civil servants must ‘respect the dignity,
government officials,’ along with the importance worth and equality of all people’ and have: ‘a

of digital capability: ‘what level of technical willingness to work without bias with persons

expertise they are required to possess’ (2013). of all nationalities, religions and cultures’ (2002).
The OECD Government at a Glance report
makes the point that opinion on the groups in
need of representation in public administration
has widened ‘and now includes a range of
dimensions such as women; racial, ethnic and
religious minorities; the poor; the elderly; the
disabled; and other minority groups such as
indigenous populations,’ and goes on to say that
‘a more representative public administration can
better access previously overlooked knowledge,
networks and perspectives for improved policy
development and implementation,’ (2015b).
A paper by the Office for Public Management
(OPM) and the Chartered Institute of Public
Finance and Accountability (CIPFA) highlights
the potential benefits of this view; ‘To enrich
governance deliberations by bringing together
a group of people with different backgrounds
– governing bodies need to recruit governors
from different parts of society. Public trust
and confidence in governance will increase if
governance… [is] done by a diverse group of
people who reflect the community.’ (2004).
27 The International Civil Service Effectiveness Index | Technical Report

Staff Engagement Innovation

Definition: Staff levels of pride, attachment and Definition: The degree to which new ideas,
motivation to work for their organisation. policies, and ways of operating are able to
freely develop.
Justification: Staff who feel that their roles are

making a real contribution, as well as a genuine Justification: The opportunities for government

interest and pride in their work will be more innovation are vast and numerous; ‘Governments

motivated and engaged. The OECD defines are operating in a new landscape. The public

engagement as: ‘Engaged employees are those sector faces economic, social and environmental

who are “committed to their organisation’s challenges; technology is revolutionizing how

goals and values, motivated to contribute to citizens interact with government; individuals

organisational success, and are able at the same and organisations across society are forming

time to enhance their own sense of wellbeing”’ new kinds of partnerships; and citizens are more
(2015d). The paper goes on to state that informed and connected than ever. Together
‘engaged employees are critical to successfully these factors create opportunities for new ways

manage change in the public administration, of thinking about government and how it works’
to enhance service orientation and to ensure (OECD, 2015c). The necessity for innovation has

customer satisfaction,’ (OECD, 2015d). A report been highlighted by Nesta: ‘For public sectors

for the UK Government by MacLeod and Clarke to become more adept at innovation they need

points out that ‘Employee engagement strategies to treat it with the same seriousness they deal

enable people to be the best they can at work, with handling risk, financial controls or regulatory

recognising that this can only happen if they enforcement’ (Mulgan, 2014).
feel respected, involved, heard, well led and

valued by those they work for and with’ (2009).
The paper also highlights the wider benefits

associated: ‘they are motivated and able to give

of their best to help it succeed – and from that

flows a series of tangible benefits for organisation

and individual alike.’ (MacLeod and Clarke, 2009).

Classification of value for money and use of evidence within the framework

There are concepts, such as the pursuit of value for money (VFM) and use of evidence, which could

be considered attributes but which are particularly relevant to some functions and are therefore

captured through this side of the framework.

By utilising the deep knowledge and experience of the founder organisations and their networks,
and by undertaking extensive consultation, we have identified a range of themes relevant to

each indicator, and metrics to measure each theme. These themes and metrics are described in

Chapter 4. There are a number of indicators and themes for which appropriate metrics could not be

identified and as such measurement of these indicators and themes has not been possible; they are

thus omitted from this edition. There are naturally a number of potential limiting factors relevant to the

metrics included in the Index and these are described in further detail in Annex A.
The International Civil Service Effectiveness Index | Technical Report 28

Chapter 4: Measuring

against the framework

Chapter 4 explores the data used to measure against the framework specified in Chapter 3,
and the weighting given to data to produce the score for each indicator.

4.1 Data availability Of the eleven functions proposed in the InCiSE
framework, eight are measured and included in

The InCiSE measurement framework, as the pilot InCiSE composite Index (explained in

outlined in Chapter 3, is such that if a civil Chapter 7). Of the six attributes in the InCiSE

service were to score highly against it, it is framework, four are measured and included in

reasonable to conclude that the civil service the composite.
would be high-performing relative to its

international counterparts. This chapter of the

report focuses on what is measured under

each indicator, and the data used to do so.
However, it should first be noted that existing

data does not enable measurement against

all of the InCiSE framework. Nevertheless,
the Index results provide valuable insights for

accountability and performance improvement

and we will strive to close gaps in data coverage

as it is developed further. The metrics included

in the Index represent a wide range of data

sources and are derived from a combination of

administrative data, survey findings and expert

assessments. Although every effort has been

made to ensure data is current and measures

the subject of civil service effectiveness closely,
due to limited data availability, some metrics may

measure wider public sector, rather than civil

service performance. While all data used is the

most recent available, not all of the data used

is updated every year. Several countries were

excluded from the pilot Index because they had

fewer than 75% of the metrics available.
29 The International Civil Service Effectiveness Index | Technical Report

Table 4: Indicators included in the InCiSE composite

Measured and included in the InCiSE composite. Not yet measured or included in the InCiSE
composite due to data availability.

Central executive functions Mission support functions
Policy making Procurement
Fiscal and Financial Management (FFM) Information Technology (IT)
Regulation Finance
Crisis/risk management
Attributes
Mission support functions Staff engagement
Human Resource Management (HRM) Innovation

Direct service delivery functions
Tax administration
Social security administration
Digital services

Attributes
Integrity
Openness
Capabilities
Inclusiveness

In some of the areas where data was not available we are aware of new data collection that may help

to fill gaps over time. In other areas, new data collection may need to be initiated over time to fulfill

the measurement of this indicator.
The International Civil Service Effectiveness Index | Technical Report 30

Figure 6: Functions and attributes included in the pilot InCiSE composite

Central executive functions Mission support functions Direct service
delivery functions
Policy making HR management Tax administration
(at the central/federal level)
Fiscal and financial management
Social security administration
Regulation
(at the central/federal level)
Crisis/Risk management
Digital services

Attributes
• Integrity • Openness • Capabilities • Inclusiveness

(‘Value for money’ and ‘use of evidence’ are other key attributes identified. However, these concepts are already captured on the
functions side of the framework to the extent that their inclusion as attribute sub-indicators is considered unnecessary.)

A wealth of data, from survey respondents, expert opinion and administrative data underlies the 12

indicators measured in the Index. In total, 76 metrics are spread across these indicators – although

some metrics are themselves indices meaning that the actual number of ‘total metrics’ is far higher.
34 metrics underlie the attributes, while 42 metrics underlie the core functions. The quality of this

data varies – in Figure 7 we set out an indicative assessment of the quality of data supporting

each indicator.
31 The International Civil Service Effectiveness Index | Technical Report

Figure 7: Indicative data quality for framework indicators

Rag rating summary
of data quality
Integrity

Openness

Attributes Capabilities
Weight: 1/3
Metrics: 34 Inclusiveness

Staff Engagement N/A

Innovation N/A

Policymaking

Fiscal and financial
Central Executive
Regulation

Risk/Crisis Management

Procurement N/A
Core functions
Weight: 2/3 HR
Metrics: 42 Mission Support
IT N/A

Finance N/A

Tax Administration

Direct Service Delivery Social Security Administration

Digital Services

The data quality was assessed according to the following criteria:

• Green – The indicator contains a large number of metrics, which seem to give a detailed, and
relatively (but not necessarily completely) comprehensive picture of performance on the indicator.
The metrics generally have few limitations. For example they get close to measuring civil service,
rather than wider public sector, performance, and they are gathered from a regularly updated data
source, so are up-to-date. Whilst metrics based on subjective expert assessments have potential
drawbacks (see Chapter 8 covering sensitivity analysis for further details), indicators which utilise
these where measurement by other means is particularly difficult are still given green ratings.

• Yellow – The indicator contains a number of metrics, which seem to give a fairly detailed picture
of performance on the indicator. However there are themes which should be measured under the
indicator, which currently are not. The metrics included generally have limitations, but these are
not prohibitive. Some may be based on subjective expert assessment. Some may measure wider
public sector, rather than civil service performance, and some may be out of date.

• Red – the indicator contains a small number of metrics, which do not generally give a detailed
or comprehensive picture of performance on the indicator, but do give some partial information
about the effectiveness of the civil service in this area.
The International Civil Service Effectiveness Index | Technical Report 32

• N/A denotes those indicators for which severe data limitations have prevented their inclusion in
the pilot Index.

4.2 Data underpinning the indicators

Many of the datasets from which the metrics are drawn are updated annually, enabling the Index to

be revised on a regular basis to reflect country developments. This iteration of the index includes data

up until January 2017.

In the following tables we set out the metrics underpinning each of the indicators and the weighting

given to each to make up the indicator score. Metrics were not identified for all the themes we would

like to measure – we identify where this is the case in the tables by use of ticks (✓), crosses (X) and

tildes (~).

✓ A tick indicates the theme is closely represented by one or more metrics utilised in the
framework

X A cross indicates that data has not yet been identified to represent this theme of an indicator
in the framework

~ A tilde indicates that the theme is only approximately represented by one or more metrics in
the framework

In each table the shorthand for the metrics, used in the Index interactive graphics, which are available

online, are shown in italics.

4.2.1 Functions

Central Executive

Policy Making

The score for Policy Making is calculated based on eight metrics from the Bertelsmann Sustainable

Governance Indicators which mainly assess the role civil servants play in setting strategic policy

direction, coordination of policy across government, and the monitoring of policy implementation.
The Bertelsmann Sustainable Governance Indicators are updated annually and cover 41 countries.
Data included is from 2016.

Bertelsmann Sustainable Governance Indicators:
“As a cross-national comparative survey designed to identify and foster successes in effective policymaking,
the SGI explores how governments target sustainable development. We advocate for more sustainable
governance, which is built on three pillars:
– Policy Performance
– Democracy
– Governance
Driven by evidence-based analyses, the SGI helps a variety of stakeholders throughout the OECD and EU
navigate the complexity of effective governance.18”
Further information about the Bertelsmann Sustainable Governance Indicators can be found at http://www.
sgi-network.org/2016/.

18 Sustainable Governance Indicators. Bertelsmann Stiftung. [Online]. Available at: http://www.sgi-network.org/2016/About. [Accessed on
27 April 2017].
33 The International Civil Service Effectiveness Index | Technical Report

Table 5 shows the metrics measured in the policy making indicator, the source of data, and the

weighting given within the indicator.

Table 5: Policy making indicator

Theme to be Currently Weighting Metrics to capture theme and the weighting each receives within
measured captured in in indicator the metric (shown in brackets).
indicator? (%)

The quality ~ 25 • Bertelsmann ‘Scholarly Advice’ sub-indicator is used as a
of policy proxy for the evidence base to policy decisions by assessing
advice the degree of civil servant to academic coordination: ‘How
influential are non-governmental academic experts for
government decision-making?’ (50%).
• Bertelsmann ‘Government Office Expertise’ sub-indicator
assesses: ‘Does the government office/prime minister’s office
(GO/PMO) have the expertise to evaluate ministerial draft bills
substantively?’ (50%).

Degree of ✓ 25 • Bertelsmann ‘Strategic Planning’ sub-indicator assesses:
strategic ‘How much influence do strategic planning units and bodies
policy have on government decision-making?’ (100%).
direction

Coordination ✓ 25 • Bertelsmann ‘Ministerial Bureaucracy’ sub-indicator
of policy assesses: ‘How effectively do ministry officials/civil servants
proposals coordinate policy proposals?’ (50%).
• Bertelsmann ‘Line Ministries’ sub-indicator assesses ‘To
what extent do line ministries involve the government office/
prime minister’s office in the preparation of policy proposals?’
(50%).

Timeliness X – No data identified
and
accuracy
of policy
delivery

Degree ✓ 25 • Bertelsmann ‘Monitoring Ministries’ sub-indicator assesses:
of policy ‘How effectively does the GO/PMO monitor line ministry
monitoring activities with regard to implementation?’ (33%).
• Bertelsmann sub-indicator ‘Monitoring Agencies’, assesses:
‘How effectively do federal and subnational ministries monitor
the activities of bureaucracies/executive agencies with regard
to implementation?’ (33%).
• Bertelsmann ‘National Standards’ sub-indicator assesses:
‘To what extent does central government ensure that
subnational self-governments realise national standards of
public services?’ (33%).

The weighting within the policy making indicator is equally split between the four themes where

metrics were identified. Within the themes, weighting was equally split between the metrics.
The International Civil Service Effectiveness Index | Technical Report 34

Fiscal and financial management

The score for Fiscal and Financial Management is calculated based on two sources:

• The World Economic Forum (WEF) Global Competitiveness Index (GCI)

• OECD medium-term and performance budgeting data

The WEF GCI covers 150 countries, and is updated annually. Survey data included is from 2016.
The OECD data covers 32 countries. The survey data included here is from 2011 (performance

budgeting) and 2012 (medium-term budgeting), and was published in the 2013 Government at

a Glance report.

Global Competitiveness Index:
“The GCI combines 114 indicators that capture concepts that matter for productivity and long-term
prosperity…. These indicators are grouped into 12 pillars...: institutions, infrastructure, macroeconomic
environment, health and primary education, higher education and training, goods market efficiency,
labor market efficiency, financial market development, technological readiness, market size, business
sophistication, and innovation. These pillars are in turn organised into three subindexes: basic requirements,
efficiency enhancers, and innovation and sophistication factors. The three subindexes are given different
weights in the calculation of the overall Index, depending on each economy’s stage of development,
as proxied by its GDP per capita and the share of exports represented by raw materials19.”
OECD medium-term budgeting:
“Data... draw upon country responses to questions from the 2012 OECD Survey on Budgeting Practises
and Procedures. Responses represent the countries’ own assessments of current practices and
procedures. The composite index… contains 10 variables that cover information on the existence of
medium-term perspective in the budget process, the number of years the estimate covers, the types
of expenditures included in the frameworks, the possibility of carry over unused funds from one year
to another and how they are monitored20.”
OECD performance budgeting:
“Data refer to 2011 and draw upon country responses to questions from the 2011 OECD Survey on
Performance Budgeting. Responses represent countries’ own assessments of current practices and
procedures. For EU member countries, results exclude any EU funding. The composite index… contains
11 variables that cover information on the availability and type of performance information developed,
processes for monitoring and reporting on results and whether (and how) performance information
is used on budget negotiations and decision making by the central budget authorities, line ministries
and politicians21.”

19 Global Competitiveness Index. World Economic Forum. [Online]. Available at: http://reports.weforum.org/global-competitiveness-
report-2015-2016/methodology/. [Accessed on 27 April 2017].
20 Government at a Glance, Medium-term expenditure frameworks. OECD. [Online]. Available at: http://www.oecd-ilibrary.org/docserver/
download/4213201ec027.pdf?expires=1493288249&id=id&accname=guest&checksum=F1942DF8725DA66F41F53532D47200D1.
[Accessed on 27 April 2017].
21 Government at a Glance 2013, Performance Budgeting. OECD. [Online]. Available at: http://www.oecd-ilibrary.org/docserver/
download/4213201ec029.pdf?expires=1493288108&id=id&accname=guest&checksum=C9DAFD442CFD772363B3F79047EE4F65.
[Accessed on 27 April 2017].
35 The International Civil Service Effectiveness Index | Technical Report

Table 6 shows the metrics measured in the fiscal and financial management indicator, the source of

data, and the weighting given within the indicator.

Table 6: Fiscal and financial management indicator

Themes to Currently Weighting Metrics to capture theme and the weighting each receives within
be measured captured in in indicator the metric (shown in brackets)
Indicator? (%)

Economic ~ 33 • ‘Public Spending’: WEF GCI business executive opinion:
appraisal ‘How would you rate the composition of public spending in
(use, quality, your country?’ (1) extremely wasteful; (7) highly efficient in
guidance) providing necessary goods and services (100%).

Economic ~
evaluation
(use, quality,
guidance)

Medium- ✓ 33 • OECD ‘Medium-Term Budgeting’ index (100%).
term
budgeting

Performance ✓ 33 • OECD ‘Performance Budgeting’ index (100%).
budgeting

The weighting within the Fiscal and Financial Management indicator gives a one third share to the

combined ‘economic appraisal’/’economic evaluation’ themes, as between these themes there is

one metric to approximately capture them, and one third each to ‘medium-term budgeting’ and
‘performance budgeting’.

Regulation

The score for Regulation is calculated based on six metrics from the OECD’s ‘Indicators of Regulatory

Policy and Governance’ to assess 3 parts of the process behind creating regulation. The OECD data

covers 34 countries and is updated every 3-4 years. Survey data included is from 2014.

Indicators of Regulatory Policy and Governance:
“Indicators of Regulatory Policy and Governance (iREG) present up-to-date evidence of OECD member
countries’ regulatory policy and governance practices advocated in the 2012 Recommendation of
the Council on Regulatory Policy and Governance. They cover in detail three principles of the 2012
Recommendation: stakeholder engagement, Regulatory Impact Assessment (RIA) and ex post evaluation,
and provide a baseline measurement to track countries’ progress over time and identify areas for reform.22”

22 Indicators of Regulatory Policy and Governance 2015. OECD. [Online]. Available at: http://www.oecd.org/gov/regulatory-policy/
indicators-regulatory-policy-and-governance.htm. [Accessed 27 April 2017].
The International Civil Service Effectiveness Index | Technical Report 36

Table 7 shows the metrics measured in the regulation indicator, the source of data, and the weighting

given within the indicator.

Table 7: Regulation indicator

Themes to be Currently Weighting Metrics to capture theme and the weighting each receives within
measured captured in in indicator the metric (shown in brackets)
Indicator? (%)

Ex ante ✓ 33 • OECD i-reg indicators – Regulatory ‘Impact Assessment –
appraisal: Primary Laws’ (50%);
application, • OECD i-reg indicators – Regulatory ‘Impact Assessment –
quality, Secondary Laws’ (50%).
sustainability,
Stakeholder engagement and ex post evaluation assessments
transparency,
are made on four areas of performance: methodology,
oversight.
systematic adoption, transparency and oversight/quality control.

Stakeholder ✓ 33 • OECD i-reg indicators – ‘Stakeholder Engagement –
engagement: Primary Laws’ (50%);
application, • OECD i-reg indicators – ‘Stakeholder Engagement –
quality, Secondary Laws’ (50%).
transparency,
oversight

Ex post ✓ 33 • OECD i-reg indicators – Ex-post ‘Evaluation – Primary Laws’
evaluation: (50%);
application, • OECD i-reg indicators – Ex-post ‘Evaluation – Secondary
quality, Laws’ (50%).
sustainability,
transparency,
oversight

The weighting within the Regulation indicator is equally split between the three themes. Within the

themes, the weighting was equally split between metrics.

For each of the 3 parts of the process, assessments are made for both primary and secondary laws,
giving 6 separate composite indicators in total. Data was collected through surveys of government

officials. Countries were asked to support responses with evidence.
37 The International Civil Service Effectiveness Index | Technical Report

Crisis/risk management

This indicator covers nine areas related to disaster risk management. All metrics are based on the

parts of the Hyogo Framework for Action (HFA) country progress reports, which track each country’s

progress towards implementing the UN priorities for disaster risk reduction action, most relevant to

the civil service. These progress reports contain scores for each country, but these are based on self-
assessment so are seen as less robust and are not included in our indicator. Scores on each of the

nine metrics here are instead calculated by ‘counting yeses’ to a number of binary questions which

are factual in nature (only examples are given for each area in the table but forty-three questions are

used in total). Survey data used here is from 2015.

Hyogo Framework for Action:
“The World Conference on Disaster Reduction was held from 18 to 22 January 2005 in Kobe, Hyogo,
Japan, and adopted the present Framework for Action 2005-2015: Building the Resilience of Nations and
Communities to Disasters (hereafter referred to as the “Framework for Action”). The Conference provided
a unique opportunity to promote a strategic and systematic approach to reducing vulnerabilities and
risks to hazards. It underscored the need for, and identified ways of, building the resilience of nations and
communities to disasters.23”

23 Hyogo Framework for Action 2005 - 2015. World Conference on Disaster Reduction. [Online]. Available at: http://www.unisdr.org/2005/
wcdr/intergover/official-doc/L-docs/Hyogo-framework-for-action-english.pdf. [Accessed 27 April 2017].
The International Civil Service Effectiveness Index | Technical Report 38

Table 8 shows the metrics measured in the crisis/risk management indicator, the source of data,
and the weighting given within the indicator.

Table 8: Crisis/risk management indicator

Themes to be Currently Weighting Metrics to capture theme and the weighting each receives within
measured captured in in indicator the metric (shown in brackets)
Indicator? (%)

Integrated ✓ 22 • ‘Risk Planning Extent’: The extent to which disaster risk
risk planning (2/9)24 is integrated into national policy planning is measured by
counting the types of policy planning into which risk is
integrated (out of 7, e.g. national development plans) (50%).
• ‘Disaster Spending Appraisal’: A score for economic
appraisal looks at whether the costs and benefits of
spending related to disaster spending are considered (50%).

Risk ✓ 22 • ‘Risk Assessment Quality’: The quality of multi-hazard risk
monitoring (2/9) assessment is measured by assessing the processes behind
risk assessments (out of 5, e.g. whether risk assessments
have agreed national standards) (50%);
• The ‘Degree of Risk Monitoring’ is measured by assessing the
extent and use of reports and databases of this type (50%).

Public ✓ 33 • ‘Early Warning Systems’: The quality of early warning
information (3/9) systems is assessed by looking at the processes in place
dissemination (out of 3, e.g. whether protocols are used and applied) (33%).
and public • ‘Public Information’ dissemination is scored by looking at the
awareness types of communications in place (out of 4, e.g. is a national
strategies disaster information system publicly available?) (33%).
• ‘Public Awareness Strategy’ looks at the relevant
workstreams in place (out of 5, e.g. whether public
education campaigns for risk awareness exist) (33%).

International ✓ 11 • ‘International Cooperation’ is measured by assessing the
cooperation (1/9) number of processes/activities in place for international
and risk cooperation and international risk management (out of 6,
coordination e.g. whether transboundary protocols are in place) (100%).

Preparedness X – No data identified
for disaster
response

Post disaster ✓ 11 • ‘Post Disaster Assessment’: A score is given for post
assessment (1/9) disaster damage and loss assessment by considering,
methodology for example, whether a specified methodology for this exists
(out of 3) (100%).

Note – percentages do not sum due to rounding

24 For this indicator, weightings between themes are rounded in the table to the nearest percentage point and are based on multiples of
1/9 fractions.
39 The International Civil Service Effectiveness Index | Technical Report

The weighting within the Crisis/risk management indicator relates to the availability of metrics

under each theme; there are nine metrics in total underlying this indicator across its six themes.
For the ‘integrated risk planning’ and ‘risk monitoring’ themes, both have 2/9 of the overall indicator

weighting, with weighting split equally across the two metrics for each theme. For ‘public information

dissemination and public awareness strategies’, this theme has 3/9 (1/3) of the weight, split equally

across its three metrics. Both ‘international cooperation and risk coordination’ and ‘post disaster

assessment methodology’ are measured by a single metric and are given 1/9 of the total weighting.
No suitable data to measure ‘preparedness for disaster response’ could be found so it is not

currently captured.

Mission support

Human resource management

The score for Human Resource Management is calculated based on Quality of Government (QoG)
expert assessments to determine two important parts of HR management: the meritocracy of

recruitment and attracting talent. Data included is from 2015.

Quality of Government:
The Quality of Government Institute at the University of Gothenburg have developed the QoG Expert
Survey Data.
“The QoG Survey is a unique data set with information on the structure and behavior of public
administration in a range of different countries. The data covers 159 countries and is based on a web
survey of 1294 experts.
The dataset covers different dimensions of Quality of Government, such as, politicisation, professionalisation,
openness, and impartiality.25”

25 Expert Survey Data. The Quality of Government Institute, University of Gothenburg. [Online]. Available at: http://qog.pol.gu.se/data/
datadownloads/qogexpertsurveydata. [Accessed 27 April 2017].
The International Civil Service Effectiveness Index | Technical Report 40

Table 9 shows the metrics measured in the human resource management indicator, the source of

data, and the weighting given within the indicator.

Table 9: Human resource management indicator

Themes to be Currently Weighting Metrics to capture theme and the weighting each receives within
measured captured in in indicator the metric (shown in brackets)
Indicator? (%)

Meritocracy ✓ 75 • ‘Applicant Skills’: QoG expert assessment of extent to which
of recruitment the skills and merits of the applicants decide who gets the
job when recruiting public sector employees (25%);
• ‘Connections Bias in Recruitment: Political’: QoG expert
assessment of extent to which political connections of the
applicants decide who gets the job (25%);
• ‘Connections Bias in Recruitment: Personal’: QoG expert
assessment of extent to which personal connections of the
applicants decide who gets the job (25%);
• ‘Recruitment via Formal Exam System’: QoG expert
assessment of extent to which public sector employees are
hired using a formal examination system (25%).

Attracting ✓ 25 • ‘Comparable Salaries’: QoG expert assessment of extent to
and retaining which senior officials have salaries that are comparable with
talent the salaries of private sector managers with roughly similar
training and responsibilities (100%).

Talent X – No data identified
deployment
(i.e.
minimising
skills gaps)

Performance X – No data identified
management

Quality of X – No data identified
learning and
development

Level of X – No data identified
customer (i.e.
civil servant)
satisfaction

The weighting within the Human Resource Management indicator awards 75% of the total weight

to the ‘meritocracy of recruitment’ theme. This is split equally among the four metrics which underlie

this theme. The ‘attracting and retaining talent’ theme has the remaining 25% with a single metric

capturing this theme. The remaining four themes are not currently captured in the indicator set

because of a lack of suitable data.
41 The International Civil Service Effectiveness Index | Technical Report

Direct service delivery

Tax Administration

The score for Tax Administration is calculated based on metrics mostly taken from ‘OECD Tax

Administration 2015’ which is a fairly comprehensive assessment of OECD tax administration

systems. Metrics included cover efficiency and the extent of digital use in tax collection made up

of 2013 data. In addition, some metrics are taken from the World Bank’s Doing Business Index

with data from 2016.

OECD Tax Administration 2015:
“Tax Administration 2015, produced under the auspices of the Forum on Tax Administration, is a unique
and comprehensive survey of tax administration systems, practices and performance across 56 advanced
and emerging economies (including all OECD, EU, and G20 members). Its starting point is the premise that
revenue bodies can be better informed and work more effectively together given a broad understanding
of the administrative context in which each operates. However, its information content is also likely to be
of interest to many external parties (e.g. academics, external audit agencies, regional tax bodies, and
international bodies providing technical assistance).
The series identifies some of the fundamental elements of national tax system administration and uses
data, analyses and country examples to identify key trends, comparative levels of performance, recent and
planned developments, and good practices.26”
Doing Business index:
“Doing Business presents quantitative indicators on business regulations and the protection of property
rights that can be compared across 190 economies – from Afghanistan to Zimbabwe – and over time.
Doing Business measures regulations affecting 11 areas of the life of a business. Ten of these areas are
included in this year’s ranking on the ease of doing business: starting a business, dealing with construction
permits, getting electricity, registering property, getting credit, protecting minority investors, paying taxes,
trading across borders, enforcing contracts and resolving insolvency.27”

26 Tax Administration 2015. OECD. [Online]. Available at: http://www.oecd.org/ctp/administration/tax-administration-23077727.htm.
[Accessed 27 April 2017].
27 Doing Business 2017. The World Bank. [Online]. Available at: http://www.doingbusiness.org/reports/global-reports/doing-
business-2017. [Accessed 27 April 2017].
The International Civil Service Effectiveness Index | Technical Report 42

Table 10 shows the metrics measured in the Tax Administration indicator, the source of data, and the

weighting given within the indicator.

Table 10: Tax administrator indicator

Themes to be Currently Weighting Metrics to capture theme and the weighting each receives within
measured captured in in indicator the metric (shown in brackets)
Indicator? (%)

Overall ✓ 33 • ‘Collection Cost’: Data from OECD Tax Administration on:
efficiency of cost of collection ratios (administrative costs/net revenue
collection collected) (50%);
• ‘Tax Debt’ ratios (total year end tax debt (excl. disputed)/net
revenue collected) (50%).

User ✓ 33 • ‘Time to Pay Taxes – Business’: Data from the World Bank’s
centricity of ‘Doing Business’ Index on the time it takes businesses to
services pay taxes (100%).

Extent and ✓ 33 • ‘E-Filed Tax Returns – Personal’: Data from OECD Tax
quality Administration on: the percentage of tax returns e-filed
of digital during the last fiscal year for personal taxes (33%);
provision • ‘E-Filed Tax Returns – Corporate’: Data from OECD Tax
Administration on: the percentage of tax returns e-filed
during the last fiscal year for corporate taxes (33%);
• ‘E-Filed Tax Returns – VAT’: Data from OECD Tax
Administration on: the percentage of tax returns e-filed
during the last fiscal year for VAT taxes (33%).

Prevention of X – No data identified
tax evasion

Level of X – No data identified
tax gap
measurement

The weighting within the Tax Administration indicator is currently equally split between the three

themes where relevant data was found. Within the themes, weighting is equally split between

the metrics.
43 The International Civil Service Effectiveness Index | Technical Report

Social Security Administration

The score for Social Security Administration is calculated based on metrics mostly taken from

data from the European Commission (via Eurostat) on administration costs as a proportion of total

expenditure on social security, capturing the overall efficiency of the social security administration

system. Total expenditure includes: social protection benefits, admin costs, sickness/health care

payments, disability payments, pensions, child benefits, unemployment benefits, housing benefits,
social exclusion benefits and other expenditure. The comparison of heterogeneous systems is difficult

so the conclusions which can be drawn from the indicator may need further investigation. Only a

single metric is included at this point.

Table 11 shows the metrics measured in the social security administration indicator, the source of

data, and the weighting given within the indicator.

Table 11: Social security administration indicator

Themes to be Currently Weighting Metrics to capture theme and the weighting each receives within
measured captured in in indicator the metric (shown in brackets)
Indicator? (%)

Overall ✓ 100 • Administration costs as a percentage of total expenditure on
efficiency of social security (100%).
distribution

User X – No data identified
centricity of
services

Extent and X – No data identified
quality
of digital
provision

Prevention of X – No data identified
fraud

The Social Security indicator currently wholly relies on one data source, given 100% of the indicator

weighting, which relates to the theme of overall efficiency of distribution.
The International Civil Service Effectiveness Index | Technical Report 44

Digital services

The score for digital services is calculated based on metrics taken from the European Commission’s

E-Government Benchmarks to assess the quality of digital services provided across four main

dimensions. A detailed assessment of the quality of services across seven life events, according to

the four dimensions, is made. The benchmark is usually published annually and covers 33 European

countries. Data is an average of Mystery Shopping exercises conducted in 2014 and 2015.

E-Government benchmarks:
“E-Government services were assessed in 34 participating countries, including all of the EU28. The
benchmark makes use of Mystery Shopping, where the quality and quantity of online public services
is measured by assessors acting as a user. The subject of the benchmark is a set of seven life events.
Together, these life events represent virtually all domains of government. Each life event is reviewed once
every two years.28”

Table 12 shows the metrics measured in the digital services indicator, the source of data, and the

weighting given within the indicator.

Table 12: Digital service indicator

Themes to be Currently Weighting Metrics to capture theme and the weighting each receives within
measured captured in in indicator the metric (shown in brackets)
Indicator? (%)

User ✓ 25 • European Commission E-Government benchmark: ‘User
centricity of Centricity’ (100%), indicates to what extent (information
services about) a service is provided online.

Transparency ✓ 25 • European Commission E-Government benchmark:
of service ‘Transparency’ (100%), indicates to what extent
governments are transparent as regards a) their own
responsibilities and performance, b) the process of service
delivery and c) personal data involved.

Cross-border ✓ 25 • European Commission E-Government benchmark: ‘Cross
mobility of Border Mobility’ (100%), indicates to what extent European
services users can use online services in another country;

The ✓ 25 • European Commission E-Government benchmark: ‘Key
availability of Enablers’ (100%), indicates the extent to which five technical
‘key enablers’ pre-conditions for eGovernment are used.

The weighting within the Digital Services indicator is equally split between the four themes. As only

one metric was used within each theme, this was given 100% of the theme weighting.

28 E-Government Benchmark 2016. European Commission. [Online]. Available at: https://www.egovernment.ch/index.php/download_file/
force/991/3343/. [Accessed on 27 April 2017].
45 The International Civil Service Effectiveness Index | Technical Report

4.2.2 Attributes

Integrity

The score for Integrity is calculated based on metrics covering six main themes related to the integrity

and other key values of civil servants. Metrics capturing perceptions of these values from country

experts, citizens and businesses make up 80% of the indicator. The remaining 20% assesses the

laws and procedures in place to preserve integrity and prevent conflicts of interest, such as the

degree of whistleblower protection. The indicator uses a range of data sources:

• Transparency International’s Global Corruption Barometer, which covers 100 countries.
Data from 2013.

• World Economic Forum’s (WEF) Global Competitiveness Index (GCI), which covers over 150
countries. Survey data from 2016.

• Quality of Government (QoG), which covers over 100 countries. Expert survey data from 2015.

• OECD private interest (2013), conflict of interest and whistleblower data (2014), which covers 36,
26 and 30 countries respectively. Survey data from 2013 and 2014.

Global Corruption Barometer:
“In the Global Corruption Barometer we ask people to tell us how corrupt they think different powerful groups
in their country are – that is whether they think “none”, “some”, “most” or “all” of them are corrupt. The nine
different groups that we ask about include high-level political actors (the president/prime minister’s office,
members of parliament and government officials); key public sector employees who interact with citizens (tax
officials, the police, judges/magistrates and local government councillors); and those who are not part of the
public sector, but are influential in political life (business executives and religious leaders).29”
Global Competitiveness Index:
See Policy Making above.
Quality of Government:
See Human Resource Management above.
OECD conflict of interest, private interest, and whistleblower data:
“Government at a Glance provides readers with a dashboard of key indicators assembled with the goal
of contributing to the analysis and international comparison of public sector performance. Indicators on
government revenues, expenditures, and employment are provided alongside key output and outcome data
in the sectors of education, health and justice.
Government at a Glance also includes indicators on key governance and public management issues,
such as transparency in governance, regulatory governance, public procurement and the implementation
of employment and remuneration reforms since 2009.30”

29 Global Corruption Barometer. Transparency International. [Online]. Available at: http://transparencia.org.es/wp-content/
uploads/2016/11/informe_barometro-2016-ingles.pdf. [Accessed 27 April 2017].
30 Government at a Glance 2015. OECD. [Online]. Available at: http://www.oecd-ilibrary.org/governance/government-at-a-glance-2015_
gov_glance-2015-en. [Accessed 27 April 2017].
The International Civil Service Effectiveness Index | Technical Report 46

Table 13 shows the metrics measured in the integrity indicator, the source of data, and the weighting

given within the indicator.

Table 13: Integrity indicator

Themes Currently Weighting Metrics to capture theme and the weighting each receives within
to be captured in the metric (shown in brackets)
measured in indicator
Indicator? (%)

Levels/ ✓ 30 • ‘Corruption Perceptions’: Global Corruption Barometer – % of
perceptions people viewing public officials/civil servants as corrupt (25%);
of corruption • ‘Public Officials Stealing’: QoG – expert opinion of how often
public sector officials steal or embezzle public funds (25%);
• ‘Public Officials Favours for Bribes’: QoG – expert opinion of
how often public sector officials grant favours for bribes (25%);
• ‘Government Favouritism of Business’: WEF GCI – business
executive opinion of extent government officials show
favouritism to well-connected firms (25%).

Fairness and ✓ 12.5 • ‘Fair Treatment by Public Officials’: QoG expert assessment of
impartiality extent to which public sector employees treat some groups in
society unfairly (50%);
• ‘Public Officials Act Impartially’: QoG expert assessment of
extent to which public sector employees act impartially when
deciding how to implement policy (50%).

Adherence ✓ 12.5 • ‘Public Officials Follow Rules’: QoG expert assessment of
to rules and extent to which public sector employees strive to follow rules
procedures (100%).

Striving ✓ 12.5 • ‘Public Officials Strive to Help Citizens’: QoG expert
to serve assessment of extent to which public sector employees strive
citizens and to help citizens (33%);
ministers • ‘Public Officials Strive to Implement Policies’ QoG expert
assessment of extent to which public sector employees strive
to implement policies decided by political leaders (33%);
• ‘Public Officials Strive to Fulfill Ideology’: QoG expert
assessment of extent to which public sector employees strive
to fulfil ideology of party in government (33%).

Work ethic ✓ 12.5 • ‘Employee Absences’: QoG expert assessment of extent to
which public sector employees are absent without permission
(50%);
• ‘Employee Efficiency’: QoG expert assessment of extent to
which public sector employees strive to be efficient (50%).

Processes ✓ 20 • ‘Post Employment Cooling Off’: OECD data on whether a
in place to post-employment cooling off period exists and whether it’s
preserve paid (25%);
integrity • ‘Lobbyist Protection’: OECD data on degree of protection
and prevent against lobbyists and other private interests influencing
conflicts of advisory groups (25%);
interest
• ‘Whistleblower Protection: Coverage’: OECD data on number
of groups who receive whistleblower protection (25%);
• ‘Whistleblower Protection: Degree’: QoG expert assessment of
degree of whistleblower protection (25%).
47 The International Civil Service Effectiveness Index | Technical Report

The weighting within the Integrity indicator is split between the six themes as follows: Levels/
perception of corruption – 30%; fairness and impartiality – 12.5%; adherence to rules and procedures
– 12.5%; striving to serve citizens and ministers – 12.5%; work ethic – 12.5%; and processes in

place to preserve integrity and prevent conflicts of interest – 20%. Levels/perceptions of corruption

and Processes in place to preserve integrity and prevent conflicts of interest themes contain metrics

from a wider variety of data sources and were therefore weighted more heavily.

Within each theme, the weight is further split between the metrics which measure that theme.
For all the Integrity themes, this weight is split equally among the metrics, although the number of

metrics under each theme varies: 1 metric (adherence to rules and procedures); 2 metrics (fairness

and impartiality and worth ethic); 3 metrics (striving to serve citizens and ministers); and 4 metrics
(levels/perceptions of corruption and processes in place to preserve integrity and prevent conflicts

of interest).

Openness

The score for Openness assumes there are two distinct themes embedded within the concept of

open government: societal consultation and transparency. Each get 50% weight within the indicator.

The data on both sides consists of existing composite indicators, for example composites designed

to measure open government as a whole and others looking more specifically at open government

data. There are four metrics on the consultation side (the top two themes in the table) and five

metrics on the transparency side (the bottom four themes in the table).

The indicator uses a range of data sources:

• The World Justice Project’s Open Government Index, which covers 102 countries. Survey data
from 2015.

• The UN’s E-participation Index, covering 193 countries and updated biennially. Survey data from
2016.

• Bertelsmann’s Sustainable Governance Indicators (SGIs), covering 41 countries and updated
annually, expert survey data from 2016.

• The World Wide Web Foundation’s Open Data Barometer, covering 92 countries and updated
annually. Survey data from 2015.

• The Open Knowledge Foundation’s Open Data Index, covering 122 countries and updated
annually. Survey data used from 2015.

• OECD’s OURdata index, covering 29 countries. Survey data from 2014.
The International Civil Service Effectiveness Index | Technical Report 48

Open Government Index:
“The WJP Open Government Index is the first effort to measure government openness based on the general
public’s experiences and perceptions in 102 countries. The Open Government Index is composed of four
dimensions: publicised laws and government data, right to information, civic participation, and complaint
mechanisms.31”
E-participation Index:
“The e-participation index (EPI) is derived as a supplementary index to the UN E-Government Survey.
It extends the dimension of the Survey by focusing on the use of online services to facilitate provision
of information by governments to citizens (“e-information sharing”), interaction with stakeholders
(“e-consultation”), and engagement in decision-making processes (“e-decision making”).
E-Participation Framework:
• E-information: Enabling participation by providing citizens with public information and access to
information without or upon demand
• E-consultation: Engaging citizens in contributions to and deliberation on public policies and services
• E-decision-making: Empowering citizens through co-design of policy option and co-production of service
components and delivery modalities.32”
Sustainable Governance Indicators:
“The SGI is a platform built on a cross-national survey of governance that identifies reform needs in 41 EU
and OECD countries.
The SGI brings together a broad network of experts and practitioners aiming to understand what works best
in sustainable governance.33”
Open Data Barometer:
“Covering 92 countries in the present edition, the Barometer ranks nations on:
• Readiness: How prepared are governments for open data initiatives? What policies are in place?
• Implementation: Are governments putting their commitments into practice?
• Impact: Is open government data being used in ways that bring practical benefit?”
Open Data Index:
“The Global Open Data Index is an annual effort to measure the state of open government data around
the world. The crowdsourced survey is designed to assess the openness of specific government datasets
according to the Open Definition.34”
OURdata index:
“The OECD OURdata Index measures government efforts to implement the G8 Open Data charter based on
the availability, accessibility and government support to promote the reuse of data, focusing on the central
OGD portal in each country.35”

31 Open Government Index. World Justice Project. [Online]. Available at: https://worldjusticeproject.org/our-work/wjp-rule-law-index/wjp-
open-government-index-2015. [Accessed 27 April 2017].
32 E-Participation Index. United Nations. [Online]. Available at: https://publicadministration.un.org/egovkb/en-us/About/Overview/E-
Participation. [Accessed on 27 April 2017].
33 Sustainable Governance Indicators. Bertelsmann Stiftung. [Online]. Available at: http://www.sgi-network.org/2016/. [Accessed
27 April 2017].
34 Open Data Index. Open Knowledge International. [Online]. Available at: http://index.okfn.org/about/. [Accessed 27 April 2017].
35 Open, Useful, Reusable Data Index. OECD. [Online]. Available at: http://www.oecd-ilibrary.org/docserver/download/4215081e.
pdf?expires=1493300444&id=id&accname=guest&checksum=665A4CCF1038B2E655ECB2D663FA9D34. [Accessed on
27 April 2017].
49 The International Civil Service Effectiveness Index | Technical Report

Table 14 shows the metrics measured in the openness indicator, the source of data, and the

weighting given within the indicator.

Table 14: Openness indicator

Themes to be Currently Weighting Metrics to capture theme and the weighting each receives within
measured captured in in indicator the metric (shown in brackets)
Indicator? (%)

The degree ✓ 37.5 • The ‘Civic Participation’ component of the Open
and quality Government Index (33%), which “measures the effectiveness
of societal of civic participation mechanisms, including the protection of
consultation the freedoms of opinion and expression, and assembly and
association, and the right to petition the government. It also
measures whether people can voice concerns to various
government officers and members of the legislature, and
whether government officials provide sufficient information
and notice about decisions affecting the community,
including opportunities for citizen feedback”;
• ‘E-Government Engagement’: The UN’s E-Participation
Index, which reviews the quality and usefulness of
e-government programs for the purpose of engaging people
in public policy-making and implementation (33%);
• ‘Negotiating Public Support’ Bertelsmann sub-indicator
(33%), which “assesses how successfully the government
consults with societal actors such as trade unions,
employers’ associations, leading business associations,
religious communities, and social and environmental interest
groups in preparing its policy.”

The existence ✓ 12.5 • The ‘Complaint Mechanisms’ component of the Open
and quality Government Index (100%), which “measures whether
of complaint people are able to bring specific complaints to the
mechanisms government about the provision of public services or the
performance of government officers in carrying out their legal
duties in practice, and how government officials respond
to such complaints. It also measures whether people can
challenge government decisions before another government
agency or a judge.”

Government ✓ 30 • ‘Open Data Practice and Impact’: The Open Data Barometer
data (ODB) measures the implementation of open data practice
availability and is the only index to also measure the impact of open
and data (e.g. how many use it) (33%).
accessibility • ‘Government Datasets Openness’: The Open Data Index
(ODI) measures whether publicly held data across 13
Government ✓ areas is defined as open, with results crowdsourced from
data impact volunteers reviewing websites (33%).
and support
• ‘Data Availability and Government Support’: The OURdata
for re-use
index also aims to capture the availability and accessibility
of data but uniquely it also attempts to measure the level of
pro-active support governments provide to foster innovative
re-use of the data (33%).
The International Civil Service Effectiveness Index | Technical Report 50

Table 14: Openness indicator

Right to ✓ 10 • The ‘Rights to Information’ component of the Open
information Government Index (100%) which “measures whether
(e.g. FOIs) requests for information held by a government agency
are granted. It also measures whether these requests are
granted within a reasonable time period, if the information
provided is pertinent and complete, and if requests for
information are granted at a reasonable cost and without
having to pay a bribe. This dimension also measures
whether people are aware of their right to information,
and whether relevant records – such as budget figures of
government officials, ombudsman reports, and information
relative to community projects – are accessible to the public
upon request.”

Publicised ✓ 10 • The ‘Publicised Laws’ component of the Open Government
laws Index (100%), which “measures whether basic laws and
information on legal rights are publicly available, presented
in plain language, and are made accessible in all languages
used by significant segments of the population. This
dimension also measures the quality and accessibility of
information published by the government in print or online
(i.e. active transparency), and whether administrative
regulations, drafts of legislation, administrative decisions,
and high court decisions are made accessible to the public
in a timely manner.”

The weighting within the Openness indicator is split between six themes as follows: Levels/perception

of corruption – 30%; fairness and impartiality – 12.5%; adherence to rules and procedures – 12.5%;
striving to serve citizens and ministers – 12.5%; work ethic – 12.5%; and processes in place to

preserve integrity and prevent conflicts of interest – 20%. This choice of weighting relates to the

number of metrics available under each theme.

Within each theme the weight is split by the number of metrics.
51 The International Civil Service Effectiveness Index | Technical Report

Capabilities

The score for Capabilities is based on OECD PIAAC data, which was collected between August

2011 and March 2012 in most participating countries. Around 166,000 adults, representing 724

million adults aged 16 to 65, were surveyed in 24 countries. Given that not all included countries are

covered, some countries have all data imputed for this indicator (see table 18).

Programme for the International Assessment of Adult Competencies:
“The Programme for the International Assessment of Adult Competencies (PIAAC) developed and conducts
the Survey of Adult Skills. The survey measures adults’ proficiency in key information-processing skills –
literacy, numeracy and problem solving in technology-rich environments – and gathers information and data
on how adults use their skills at home, at work and in the wider community.36”

Table 15 shows the metrics measured in the capabilities indicator, the source of data, and the

weighting given within the indicator.

Table 15: Capabilities indicator

Themes to be Currently Weighting Metrics to capture theme and the weighting each receives within
measured captured in in indicator the metric (shown in brackets)
Indicator? (%)

Leadership X – No data identified
capability

Commercial X – No data identified
capability

Analytical X – No data identified
capability

Digital X – No data identified
capability

Core ✓ 75 • ‘Literacy Skills’: Taken from the OECD’s PIAAC (Survey of
capability Adult Skills) data (analysis of the microdata): the proportion of
(e.g. the public sector getting level 4 or 5 for literacy skills (33%);
problem- • ‘Numeracy Skills’: The proportion of the public sector getting
solving, level 4 or 5 for numeracy skills (33%);
numeracy,
• ‘Problem Solving Skills’: The proportion of the public sector
literacy skills)
getting level 3 for problem skills (33%).

Educational ✓ 25 • ‘Educational Attainment’: The proportion of the public sector
attainment of with tertiary education, taken from the OECD’s PIAAC (Survey
the workforce of Adult Skills) data (analysis of the microdata) (100%).

The weighting within the Capabilities indicator is split between the core capability and educational

attainment of the workforce themes; the two areas for which usable data has been found. For the

core capability theme, the weighting is equally split between the three metrics. Thus, although the

core capability theme has three quarters of the overall weighting and the educational attainment

theme the remaining quarter, the four metrics underpinning the indicator are equally weighted.

36 Survey of Adult Skills. OECD, Programme for the Assessment of Adult Competencies. [Online]. Available at: http://www.oecd.org/skills/
piaac/. [Accessed 27 April 2017].
The International Civil Service Effectiveness Index | Technical Report 52

Inclusiveness

The score for Inclusiveness is based on the difference in demographics within central government

compared to the country’s workforce overall. A high score is given for having a similar demographic

profile. The indicator used survey data published in Government at a Glance 2013 (GaaG) and

collected in 2010, and Quality of Government survey data from 2015.

Government at a Glance
“Government at a Glance provides readers with a dashboard of key indicators assembled with the goal
of contributing to the analysis and international comparison of public sector performance. Indicators on
government revenues, expenditures, and employment are provided alongside key output and outcome
data in the sectors of education, health and justice. Government at a Glance also includes indicators on key
governance and public management issues, such as transparency in governance, regulatory governance,
public procurement and the implementation of employment and remuneration reforms since 2009. While
measuring government performance has long been recognised as playing an important role in increasing the
effectiveness and efficiency of the public administration, following the economic crisis and fiscal tightening in
many member countries, good indicators are needed more than ever to help governments make informed
decisions regarding tough choices and help restore confidence in government institutions.37”

37 Government at a Glance. OECD. [Online]. Available at: http://www.oecd-ilibrary.org/governance/government-at-a-glance-2015_gov_
glance-2015-en. [Accessed 27 April 2017].
53 The International Civil Service Effectiveness Index | Technical Report

Table 16 shows the metrics measured in the inclusiveness indicator, the source of data, and the

weighting given within the indicator.

Table 16: Inclusiveness indicator

Themes to be Currently Weighting • Metrics to capture theme and the weighting each receives
measured captured in in indicator within the metric (shown in brackets)
Indicator? (%)

Proportionate ✓ 50 • ‘Gender: Central Government Share’: OECD data – the
gender absolute difference between the share of total central
representation government employment filled by women and the share of
women in the labour force (25%);
• ‘Gender: Public Sector Share’: QoG data – the absolute
difference between proportion of women in public sector
and in the labour force (25%);
• ‘Gender: Management Share’: OECD data – the absolute
difference between the share of employment in top
management positions within central government filled by
women and the share of women in the labour force (25%);
• ‘Gender: Senior Management Share’: QoG data – the
absolute difference between proportion of women in central
government senior positions and in the labour force (25%)

Proportionate ✓ 50 • ‘Ethnic and Religious Group Representation’: QoG data
ethnic minority – expert opinion as to whether ‘Key ethnic and religious
representation groups in society are proportionally represented among
public sector employees (1=hardly ever, 7=almost always)’
(100%)

Proportionate X – No data identified
disability
representation

Proportionate X – No data identified
socio-
economic
representation

Proportionate X – No data identified
Lesbian Gay
Bi-sexual
Transgender
Other
sexuality
representation

The weighting within the inclusiveness indicator is equally split between the gender representation

and proportionate ethnic minority representation theme; the two areas for which usable data

has been found. For the gender representation themes, the weighting is equally split between the

four metrics.

For transparency, we set out limitations associated with the above data in Annex A. InCiSE is a long

term project and we expect to include further data and refine the data currently included in the Index

as we receive feedback about how the indicators could be further developed.
The International Civil Service Effectiveness Index | Technical Report 54

Chapter 5: Index

country coverage

This chapter examines which countries have been included and excluded from the

pilot Index, and explains our approach where a country is included but some of their

data is missing.

5.1 Countries included in the Index Covering the widest range of countries in the
Index is tempered by availability of data. There is

The ambition of the InCiSE Index is that considerable variation in country coverage for the

the indicators may serve two purposes: data used in this index (described in chapter 4).
firstly, to serve as an accountability tool, Expanding the range of countries would lead to

and secondly, to provide a performance a greater degree of data imputation, a reduction

improvement tool. To meet this ambition and in the scope of the framework or a greater

achieve the greatest impact the Index aims to reliance on proxies for the civil service. Table 17

cover a wide and growing range of countries shows the proportion of metrics available for

as it is developed further. each country. Only two countries, Norway and
the United Kingdom, had no missing metrics.
55 The International Civil Service Effectiveness Index | Technical Report

Table 17: Proportion of metrics available for each country
(Note: dark red area represents those countries not included in the pilot release of the Index)

Country Total number of metrics Proportion of metrics available (%)
Norway 76 100

United Kingdom 76 100

Austria 75 99

France 75 99

Italy 75 99

Netherlands 75 99

Finland 74 97

Germany 74 97

Denmark 73 96

Poland 73 96

Sweden 73 96

Czech Republic 72 95

Spain 72 95

Slovenia 70 92

Portugal 69 91

Slovakia 69 91

Hungary 68 89

Turkey 68 89

Belgium 67 88

Australia 66 87

New Zealand 66 87

Estonia 65 86

Mexico 65 86

Switzerland 65 86

Greece 64 84

Canada 62 82

Ireland 60 79

Japan 59 78

Korea, Republic of 59 78

Chile 58 76

United States of America 57 75

Bulgaria 57 75

Croatia 54 71

Romania 53 70

Iceland 51 67

Israel 49 64

Colombia 46 61
The International Civil Service Effectiveness Index | Technical Report 56

To balance the issue of country coverage and Across the metrics underlying each indicator

data availability, it was decided that any country there is some variation in the countries covered

with less than 75% of the data available would by the data. Table 18 gives an overview of where

be excluded at this stage from the Index. data was available for each indicator and where

Countries below this point have larger amounts data imputation was required.
of missing data and a greater proportion of the

total data would need to be imputed. Bulgaria Green indicates data was available for all metrics

is excluded to keep the Index limited to OECD within the indicator for that country; amber

countries, for simplicity at this stage. As a result, indicates that data was available for some

we have 31 countries currently included in the metrics within the indicator; and red indicates

pilot Index. Our hope is that further countries will that no data was available for the indicator such

be included as data is identified for use or new that all metrics were imputed.
data collection is initiated.
57 The International Civil Service Effectiveness Index | Technical Report

Table 18: Indicator data availability by country

Country Metric Attributes Direct Service Delivery functions Mission Central Executive functions
account (34 metrics) (26 metrics) Support (11 metrics)
(out of 76) functions
(5 metrics)
Integrity Openness Capabilities Inclusiveness Policy Fiscal & Regulation Risk Man. Human Tax Admin Social Digital
(16) (9) (4) (5) making Fin. Man. (6) (9) Resources (6) Security Services
(8) (3) (5) Admin (4)
(1)

NOR 76

GBR 76

AUT 75

FRA 75

ITA 75

NLD 75

FIN 74

DEU 74

DNK 73

POL 73

SWE 73

CZE 72

ESP 72

SVN 70

PRT 69

SVK 69

HUN 68

TUR 68

BEL 67

AUS 66

NZL 66

EST 65

MEX 65

CHE 65

GRC 64

CAN 62

IRL 60

JPN 59

KOR 59

CHL 58

USA 57
The International Civil Service Effectiveness Index | Technical Report 58

Policy making, fiscal and financial management such as efficiency and validity, when a correct

and human resources are the only indicators model is specified for the imputation39. An

where data is available for all metrics across imputation represents one set of plausible values

all countries currently included in the Index. for missing data, and so multiple imputations

However, this does not necessarily mean there represent multiple sets of plausible values.
are no limitations with the available data and For studies with roughly 10-60% missing values,
we aspire to continually improve data across past research suggests that multiple imputation

all indicators. There may be drawbacks to data is the method of choice.40

quality stemming, for example, from data which

is less current than is desirable, data measuring A number of methods of imputation (including MI)
concepts which are not a perfect fit with the use other variables to predict the missing values.
indicator in question, data which is based on MI requires assumptions to be made about

subjective assessment and data which uses the distribution of the variable and its predictor

the wider public sector as a proxy for the civil variables. Each missing value is imputed several

service. More detail on potential data limitations times, therefore generating several independent,
we have identified are set out in Annex A. completed data sets. Each completed data
set is analysed and then combined. This

For all other indicators, some degree of imputation method assumes that the data are

data imputation was required. A statistical missing at random, a hypothesis that cannot

methodology was adopted to estimate/ be verified, since we have no knowledge of
‘impute’ missing data. Section 5.2 describes the unobserved data.41 Nevertheless, the more

the approach to imputing missing data. predictors included in the imputation model,
the assumption that data is missing at random

5.2 Imputation method is more likely to hold because the uncertainty
associated with missingness is reduced.42

Researchers modeling data often encounter

the problem of missing data regarding one When a continuous variable contains missing

or more of the variables under investigation. values, a linear regression imputation

The most common approach is to remove those method can be used to fill in missing values.43

observations with missing values, allowing for a The method of multiple imputation utilised in

complete analysis of those for which full data is developing the Index fills in missing values in

available. However, this approach wastes data variables iteratively by using chained equations,
and reduces power, and also produces biased a sequence of univariate imputation methods.
estimates when the values are not missing This requires that a predictive model of the

completely at random38. One alternative is to use variables with missing values can be specified;
one of the many methods available for imputing the predictive model can include other variables

the missing values. Of the available methods, with missing values, while taking into account the

multiple imputation (MI) is attractive with problems associated with predictors that have a

theoretical and simulation studies showing that it high degree of missing data. Variables correlated

yields estimates with good statistical properties, to the variable with missing data need to be
utilised in the predictive model.

38 Little R. and Rubin D. (1987), Statistical analysis with missing data.
39 Little R. and Rubin D. (1987), Statistical analysis with missing data.
40 Barzi F. and Woodward M. (2004), Imputations of Missing Values in Practice: Results from Imputations of Serum Cholesterol in
28 Cohort Studies, American Journal of Epidemiology 160 (1): 34-45.
41 Ibid.
42 Shafer J.L. (1997), Analysis of incomplete multivariate data.
43 Rubin D. B. (1987), Multiple Imputation for Nonresponse in Surveys; Schenker, N., and Taylor J.M.G. (1996) Partially parametric
techniques for multiple imputation. Computational Statistics & Data Analysis 22: 425–446
59 The International Civil Service Effectiveness Index | Technical Report

The technique used for undertaking multiple For variables where strong correlations with other

imputation in the Index is multivariate imputation datasets are not observed, median imputation

using chained equations (MICE). MICE is very is used, with values imputed independently

popular in practice. Its popularity stems from of any predictor. With unconditional median

the flexibility it offers for imputing the different imputation, the median value of the variable

types of data in observational studies. The in question for the countries where data is

variable-by-variable specification of MICE allows available is substituted for each missing value.
practitioners to simultaneously impute variables Since all imputations are the same, this method

of different types by choosing from different will underestimate the variance for the variable.
univariate imputation methods appropriate for For those metrics where median imputation has

each variable. Being able to specify a separate been used, and for the functions and attributes

model for each variable provides flexibility in these metrics feed into, caution should be taken

incorporating certain characteristics specific to when interpreting the results and less weight

each variable.44,45 should be allotted to the relative position of
countries. However for interpreting the overall

Data from a wider range of countries than those results of the Index we are confident the use

included in the pilot edition of the Index has of median imputation has not led to significant

been utilised for multiple imputation. While the bias, due to the small proportion of missing data

overall availability of data for some countries led overall (10%) and the small proportion of missing

to our judgement to exclude them from the first data imputed using this technique (4%).
release of the Index, for particular datasets which

covered these countries, this data was utilised A mixture of multiple and median imputation is

for imputing values for the 31 countries included used in the estimation of missing data for the

in the Index. InCiSE Index. Further details and discussion
on the implementation of MI are available from
a number of sources46 and Section 5.3 sets
out the method of imputation used to estimate
missing data for each metric that feeds into the
Index, documenting the correlations observed
between variables and explaining the predictive
models used for multiple imputation.

44 For more information about multivariate imputation using chained equations, see van Buuren, Boshuizen, and Knook (1999);
Raghunathan et al. (2001); van Buuren et al. (2006); van Buuren (2007); White, Royston, and Wood (2011); and Royston (2004, 2005a,
2005b, 2007, 2009), among others.
45 The number of iterations necessary for MICE to converge depends on, among other things, the fractions of missing information
and initial values. The higher the fractions of missing information and the farther the initial values are from the mode of the posterior
predictive distribution of missing data, the slower the convergence, and thus the larger the number of iterations required. Current
literature suggests that in many practical applications a low number of burn-in iterations, somewhere between 5 and 20 iterations,
is usually sufficient for convergence. In any case, examination of the data and missing-data patterns is highly recommended when
investigating convergence of MICE.
46 StataCorp LP, Stata Multiple-Imputation Reference Manual, Release 13, 2013. For more information about multivariate imputation using
chained equations, see van Buuren, Boshuizen, and Knook (1999); Raghunathan et al. (2001); van Buuren et al. (2006); van Buuren
(2007); White, Royston, and Wood (2011); and Royston (2004, 2005a, 2005b, 2007, 2009), among others. For more information about
the compatibility of conditional specifications, see Arnold, Castillo, and Sarabia (2001); van Buuren (2007); and Arnold, Castillo, and
Sarabia (1999) and references therein.
The International Civil Service Effectiveness Index | Technical Report 60

5.3 Imputation approach For this indicator, multiple imputation was used.
Our analysis found that a number of Bertelsmann

for indicators
Sustainable Governance Indicators metrics are

As discussed in section 5.2, the dataset utilised suitable predictors for OECD iREG metrics.
for generating imputed data contains data for Our imputation model utilised these metrics

countries which are not included in the Index. as complete predictors in addition to the iREG

Data was imputed for all countries in this larger imputation variables.
dataset, and as such the approach includes
Crisis/risk management

imputation for some metrics where no imputed

data is present in the Index, because this covers For the crisis/risk management indicator,
a more restricted set of countries. Separate all metrics are missing for 8 countries in the

imputation models were developed for each Index and have been imputed. No obvious

indicator and for some indicators a number of predictor variables were discovered, so simple

imputation models were devised for the different median imputation was used. This assigns the

themes. In a small number of cases, metrics median value for countries where data was

that are not featured directly in the indicator available as the estimate of data for countries

framework are used as predictors in imputation where it is missing.
models for metrics that do feature in the Index,
Human resources management

due to their correlation with these metrics; this is

highlighted where relevant. For the human resources management indicator,
for all the metrics which contribute to the score,
data was available for every country in the Index

5.3.1 Functions
and no imputation was required.
Policymaking
Tax administration

For the policymaking indicator, for all the metrics
For tax administration, data is missing for one

which contribute to the score, data was available
or more metrics for 11 countries in the Index,
for every country in the Index and no imputation
although there are no countries where all data

was required.
is missing.
Fiscal and financial management
We found significant correlations between the

For the fiscal and financial management cost of collection ratios metric and the World

indicator, for all the metrics which contribute to Bank’s ‘Doing Business’ Index on the time it

the score, data was available for every country takes businesses to pay taxes. We also found

in the Index and no imputation was required. significant correlation between the three digital

Regulation metrics on e-filing of tax returns and the cost
of collection ratios. Correlation was weaker

Data was available for the metrics included in between the digital metrics and the time it takes

this indicator across every country in the Index to pay business taxes metric.
but one. The score for Regulation is calculated

based on six metrics from the OECD’s. The tax debt ratio appears to be correlated
‘Indicators of Regulatory Policy and Governance’ with the time it takes businesses to pay taxes
(iREG). Data was missing for two metrics; but the correlation is not statistically significant.
Regulatory Impact Assessment – Primary laws With these observed relationships we proceeded

and Stakeholder Engagement – Primary Laws. with a multiple imputation approach with all
metrics included as predictors, except for the tax
debt ratio. The tax debt ratio metric was imputed
using simple median imputation.
61 The International Civil Service Effectiveness Index | Technical Report

Social security administration 5.3.2 Attributes

For social security administration only one
Integrity

metric is currently included in the indicator.
Data is missing for 8 countries in the Index and For integrity sixteen metrics are included in

data has been imputed for these countries the indicator and data is missing for four of

using multiple imputation. We considered and the metrics. The Global Corruption Barometer

examined a wide range of metrics outside of metric measuring the percentage of people

this indicator to obtain a suitable predictor viewing public officials/civil servants as corrupt is

and observed a correlation with one of the missing data for 5 countries in the Index. OECD

dimensions of the Quality of Government Expert data on whether a post-employment cooling off

Survey. We acknowledge that the observed period exists and whether it’s paid, data on the

correlation does not identify or imply a causal degree of protection against lobbyists and other

relationship between these metrics and we wish private interests influencing advisory groups,
to identify and develop data to strengthen this and data on the number of groups who receive

indicator as a priority. whistleblower protection are also missing for
1 country for the first metric and 7 countries for

Digital services the latter two metrics.
The score for Digital Services is calculated
The imputation approach taken considered these

based on metrics taken from the European
metrics in two groups. Excluding the three OECD

Commission’s E-Government Benchmarks.
metrics with missing data, we found all integrity

All metrics are missing for 8 countries
metrics are highly correlated, noting that the QoG

in the Index and have been imputed by
expert assessment of the extent to which public

multiple imputation. We found that the Online
sector employees strive to fulfil ideology of party

Service Index metric of the UN e-Government
in government is negatively correlated. On this

Development Index47 was generally correlated
basis we proceeded with multiple imputation for

with all the E-Government Benchmarks
missing data in this group of metrics with all as

metrics and so this metric was utilised as
predictors for each other.
a complete predictor.
For the second group, our examination of the
data showed that among the three OECD
metrics in this group with missing data, data for
each metric is fairly clustered around one point
and simple median imputation is used.

47 UN E-Government Development Index https://publicadministration.un.org/egovkb/en-us/Data-Center
The International Civil Service Effectiveness Index | Technical Report 62

Openness Inclusiveness

The approach to imputation for openness For capabilities, five metrics are included in the

treated metrics concerning societal consultation indicator and data is missing for two of these;
separately from those concerning transparency. the OECD data covering the share of total central

There are four metrics on the consultation side government employment filled by women and

with five metrics on the transparency side. the share of employment in top management
positions within central government filled by

For societal consultation, we observed women as absolute differences in comparison

correlation between the metrics with missing to each of those equivalent shares of women

data (the ‘complaint mechanisms’ and in the labour force as a whole. For these two
‘civic participation’ components of the Open metrics data is missing for 10 and 12 countries

Government Index) and the metrics with non- in the Index respectively. We proceeded with

missing data (UN’s E-Participation Index and multiple imputation for missing data with all
‘Negotiating public support’ Bertelsmann metric). metrics utilised as predictors for each other.
Multiple imputation was undertaken with these

metrics utilised as complete predictors.

For the transparency group of metrics, we found

all metrics are highly correlated. On this basis we

proceeded with multiple imputation for missing

data in this group, with all metrics utilised as

predictors for each other.

Capabilities

For capabilities, four metrics are included in the

indicator and data is missing for all of these.
For 3 of the metrics 10 countries were missing

data and for one of the metrics 13 countries

were missing.

We examined the relationship of these metrics

to others outside of the capabilities indicator.
The most suitable predictors observed were the

QoG expert assessment of the extent to which

the skills and merits of the applicants decide

who gets the job when recruiting public sector

employees and EU membership; both metrics

are positively associated with skills variables.
Our imputation model utilised these metrics as

complete predictors.
63 The International Civil Service Effectiveness Index | Technical Report

Chapter 6:
Index results

In this chapter we describe the approach to the normalisation and reporting of Index

results and advise on their interpretation.

6.1 Normalisation Worked Example
The normalisation method is illustrated below

The data sets which feed into the Index capture
for a metric underpinning the Integrity indicator;
a range of aspects of the civil service through
the Quality of Government Institute expert

survey respondents, expert opinion and
assessment of the extent to which public sector

administrative data. Additionally, the datasets
employees strive to help citizens.
utilise a variety of different measurement scales.
To enable comparison of scores and for the The first column of the table presents the country

purposes of aggregating the data to provide scores prior to normalisation.
scores for the different functions and attributes,
and for the composite Index, we transform the For each country score, the minimum score

data into comparable units with the same scale – observed for the metric (2.32) is subtracted

a process known as normalisation. from this score and the result is then divided
by the difference between the maximum (6.29)
The normalisation method we employ in our pilot and minimum scores observed. This provides

InCiSE Index is the Min-Max method. The Min- the normalised score for a country. For example

Max48 process of normalisation preserves the Australia’s score before normalisation is 5.48.
distribution of the data and scales all numeric Subtracting the minimum score of 2.32 from

variables in the range [0,1]. Scoring 1 means that this score gives 3.16. The difference between

country has the highest score on that metric, 0 the maximum and the minimum scores is

the lowest. An example of normalisation using 3.97. Dividing 3.16 by 3.97 gives Australia’s

the Min-Max methodology is as follows: normalised score of 0.80 to 2 decimal places.

48 Handbook on Constructing Composite Indicators, OECD
The International Civil Service Effectiveness Index | Technical Report 64

Table 19: Normalisation example

Country QoG Institute expert assessment Normalised score
of the extent to which public
sector employees strive to
help citizens

AUS 5.48 0.80

AUT 5.43 0.78

BEL 5.43 0.78

CAN 6.24 0.99

CHL 5.40 0.78

CZE 4.00 0.42

DNK 5.53 0.81

EST 4.63 0.58

FIN 5.33 0.76

FRA 5.09 0.70

DEU 5.03 0.68

GRC 3.60 0.32

HUN 3.73 0.36

IRL 5.00 0.68

ITA 3.70 0.35

JPN 5.75 0.87

KOR 4.74 0.61

MEX 2.32 0.00

NLD 5.21 0.73

NZL 6.29 1.00

NOR 5.57 0.82

POL 5.33 0.76

PRT 4.53 0.56

SVK 3.89 0.40

SVN 4.75 0.61

ESP 4.91 0.65

SWE 5.92 0.91

CHE 5.60 0.83

TUR 3.92 0.40

GBR 5.18 0.72

USA 5.25 0.74
65 The International Civil Service Effectiveness Index | Technical Report

In mathematical terms, each data point for a 6.2 Reporting of results

country and a time is transformed according

to the formula given below: For the pilot edition of the Index, results are
reported for each metric online, normalised
between 0 and 1 according to the method
set out in Section 6.1, following imputation
where required. Results for each indicator,
the 8 functions and 4 attributes, are also
reported online and within the Main Report.
where and are the These indicator results are aggregated from

minimum maximum value of across all the contributing metrics according to the

countries at time .49 weightings described in Section 4.2.
One issue with standardising the range but not The scores for the composite (the overall) Index

the variance can be that, if you have outliers in are built from the indicator scores according to

your data set, normalising your data will scale the the weighting approach set out and described
‘normal’ data to a very small interval. Variables in more detail in Section 7.2. These are provided

with higher variation will have more effect on the both online and within the Main Report.
final outcome than variables with lower variance.
However, analysis of our data has shown the The normalised results are presented by way of

standard deviations of the normalised variables radar graphs, an example is shown in Figure 8.
are all between 0.18 and 0.35 (the maximum

value for this standardised range would be 0.5).

Looking ahead to future editions of the Index,
another variant of the Min-Max method is,

where the minimum and maximum for each

variable are calculated across countries and time,
in order to take into account the evolution of the

indicators.50 In future editions of the index the

distinction between developments in absolute

and relative performance of countries may be a

pertinent analytical question. For example, it is

possible for absolute and relative performance to

move in opposite directions over time. Adopting

such a method may help to address the issue of

comparability between years.

This transformation is not stable when data for a

new time point becomes available. This implies

an adjustment of the time period analysed,
which may affect the minimum and maximum

for some variables and hence the values of .
Maintaining comparability between the existing

and new data would require the Index for the

previous years to be recalculated.51

49 Ibid.
50 Ibid.
51 Ibid.
The International Civil Service Effectiveness Index | Technical Report 66

5/26/2017 Radar chart

Figure 8 Radar Graph Example – Poland
Tax
administration
Policy
making Inclusiveness

0.75

Social
security Capabilities
0.50

0.25

Digital
services Openness

Fiscal
and Integrity
financial
management

Key
Regulation Human Poland
Crisis resource Average (means) scores
management management

The normalised data, and radar diagrams drawn from this, allow for an assessment of relative

performance of the group of countries included in the Index. So for example, it may allow statements

such as ‘Country X is assessed to perform more strongly on the Inclusiveness indicator than country

Y’ or ‘Country X is assessed to perform more strongly than the average for all countries in the Index’.

The indicator and composite Index results should be interpreted in the context of the quality and

depth of the underlying metrics, and the degree and method of imputation utilised to estimate

missing data; these are all described within this Technical report.

A low score does not necessarily mean that a country is performing badly, or that such improvements
are more pressing, than for an indicator which has a higher score. The nature of a normalised
comparative index means that it is showing a comparative assessment against other countries,
not an absolute assessment. As such, relatively weaker scores may aid identification of areas for
improvement or learning from more strongly performing countries, but will not indicate priorities
for reform and attention on their own. In Figure 8, while the relative score for Crisis Management
may be higher than that for Regulation, this does not imply that a country will rank more highly

https://incise­radar­vis.cloudapps.digital/ 2/6
for this indicator. The average relative performance (score) for Crisis Management is higher than
for Regulation, such that Poland’s score for Crisis Management is somewhat below the average,
whereas its score for Regulation is much closer to the average. The higher average score for Crisis
Management reflects the grouping of country results which are closer to the leading country than for
Regulation.

The distribution of results within the Index, indicators and underlying metrics shows the degree of

closeness of results in various parts of the ranking, which may indicate whether a particular country

is the clear leader (or follower) in performance or whether there is a wider group of relatively strong
(weak) countries with similar scores. Information covering the distribution of results is provided in the

Main Report, with further detail available in figures 9 and 10, which show the ranking and distribution

of indicator scores.
67 The International Civil Service Effectiveness Index | Technical Report

Figure 9 Bar Chart Examples

The bar chart examples demonstrate the variation observed in the distribution of results across the

function and attribute indicators. For example, the 10 strongest performing countries for Fiscal and

Financial Management are more tightly grouped in their relative performance than the strongest

performing countries for Regulation. At the other end of the distribution, scores among the weakest

performing countries for Regulation are in fact more tightly grouped than the weakest performing

group for Fiscal and Financial Management; this can be observed from the range of scores occupied

by the bottom 10 countries for each indicator. The composite Index weights the scores for indicators

rather than the ranks.
The International Civil Service Effectiveness Index | Technical Report 68

Figure 10 Distribution of Indicator scores

Figures 10 provides an alternative, summary view, of the distribution of indicator scores. Clustering of

scores can be seen for some indicators, while others are more equally spread.

Integrity Mexico New Zealand
Openness Turkey United Kingdom
Capabilities Italy Japan
Inclusiveness Hungary Poland
Policymaking Slovenia United Kingdom

Fiscal and Financial Management Hungary Switzerland
Regulation Chile Mexico
Crisis/Risk Management Germany Turkey
Human Resource Management Slovakia Ireland
Tax Administration Czech Republic Estonia
Social Security Administration Ireland United Kingdon
Digital Services Slovakia Estonia
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

Lowest score Highest score Median
68 The International Civil Service Effectiveness Index | Technical Report

Chapter 7: Composite

In chapter 7 we describe the advantages and disadvantages associated with the use of

composite measures. We then describe the construction and weighting of the InCiSE

composite and how our approach mitigates against some of the issues with composite

indicators that have been identified.

7.1 Strengths and weaknesses of composites

A composite index is the result of compiling individual indicators into a single index in accordance

with an underlying framework. The composite can measure multi-dimensional concepts which cannot

be captured by a single indicator.

Composite indicators which compare country performance are increasingly recognised as a useful

tool in policy analysis and public communication. However, composite indicators can send misleading

policy messages if they are poorly constructed or misinterpreted.

The main strengths and weaknesses of using composite indicators are included in Table 2052.

Table 20: Strengths and weaknesses of composite indicators.

Pros: Cons:

• Can summarise complex, multi-dimensional • May send misleading policy messages if poorly
realities with a view to supporting decision makers. constructed or misinterpreted.
• Are easier to interpret than a battery of many • May reduce complex things to simple
separate indicators. rankings which could lead to overly simplistic
• Can assess progress of countries over time. interpretations.
• Reduce the visible size of a set of indicators • May be misused, e.g. to support a desired
without dropping the underlying information base policy, if the construction process is not
transparent and/or lacks sound statistical or
• Thus make it possible to include more
conceptual principles.
information within the existing size limit; the
number of underpinning measures can be much • The selection of indicators and weights could be
higher than the number of composite measures. the subject of political dispute.
• By providing summary measures and visualisations, • May disguise serious failings in some dimensions
can place issues of country performance and and increase the difficulty of identifying proper
progress at the centre of the policy arena. remedial action, if the construction process is
not transparent.
• Facilitate communication with general public (i.e.
citizens, media, etc.) and promote accountability. • May lead to inappropriate policies if dimensions
of performance that are difficult to measure
• Help to construct/underpin narratives for lay and
are ignored.
literate audiences.
• Enable users to compare complex dimensions
effectively

52 Saisana M. and Tarantola S. (2002), State-of-the-art report on current methodologies and practices for
composite indicator development, EUR 20408 EN, European Commission-JRC: Italy
The International Civil Service Effectiveness Index | Technical Report 69

The table sets out some of the most important, To account for these limitations and to promote

and connected, benefits and drawbacks of transparency in deploying a composite index,
developing and utilising composite indicators. we have taken the following steps:
Our reasoning for utilising a composite indicator

can be summarised as follows: i) We describe our favoured approach
to constructing a composite Index in
“The key strength of aggregate indicators is their Section 7.2.
ability to convey information on many parameters

succinctly (Booysen, 2002; Hahn, 2008; Zhou & ii) We have tested and consulted on the chosen

Ang, 2009; Balica, 2012b). Therefore, composite approach widely, refining and adjusting the

indices are powerful and communicative tools method over time; seeking the views of a

because they present clear and concise results range of multilateral organisations, academic

to non-technical audiences such as scores institutions and other experts in the field of

or rankings (Kenney et al., 2012). That helps public governance, including those most

to promote a multi-stakeholder dialogue familiar with the usage and drawbacks of

in establishing common understanding of composite measures.
supranational concerns and overcoming socio- iii) To promote a fuller understanding of the

political barriers of decision making (Preston et ramifications of different approaches to

al., 2011: 183). The two main advantages of weighting of indicators, we have also

aggregate measures are: tested the impact of a range of alternative

  1. Variables that cannot be directly observed weightings on the Index results. More
    may be inferred by integrating multiple information about the sensitivity analysis
    indicators as part of a composite indicator. is provided in Chapter 8.
  2. Composite indices usage helps to overcome v) Alongside the composite Index scores
    the problems of precision, reliability and and rankings, we present ranks across all
    accuracy by reducing the influence of indicators in our Main Report. This provides
    measurement error as the number of a fuller picture of how the Index is built up
    observations from multiple sources increase from the indicators.
    (Kaufmann & Kraay, 2007; Maggino &
    Zumbo, 2012).53” 7.2 The InCiSE composite

Bearing these potential benefits in mind, The InCiSE Index brings together the indicators,
we remain aware that even the most carefully each measuring a different dimension of

constructed of these indices can lack effectiveness into an overall composite indicator

transparency and comparability over time, of civil service effectiveness.
suffer from selection bias and be of limited use

in helping countries to identify how effectively

to improve the quality of the civil service.54

53 Muriithi et al (2015), Quantifying Governance: An indicator-based approach, Report for DFID by MPA students at the LSE

54 Arndt C. and Oman C. (2006), Uses and Abuses of Governance Indicators, OECD Development Centre Studies
70 The International Civil Service Effectiveness Index | Technical Report

Weighting Our preference would be for each attribute

The measurement framework outlined in and function indicator to be given equal weight

Section 3 gives a comprehensive overview of within their respective shares of the Index

civil service effectiveness. However, existing data weighting. Due to poorer data quality on two of

does not enable one to measure against all of it. the attributes, Capabilities and Inclusiveness,
Of the 6 attributes proposed for measurement, their weighting is reduced. A similar adjustment

so far 4 are included with an individual indicator. could be made for Tax and Social Security

Of the 11 core functions proposed, 8 are Administration, but given their overwhelming

included with an individual indicator. There are importance as civil service functions it was

12 indicators in total. A number of data sources, judged that reducing their weight would not

most with multiple metrics contained within be appropriate. Figure 11 sets outs the Index

them, underlie the 12 indicators in the Index. weighting of scores. The Index is more heavily

In total, 76 variables are spread across these weighted towards functions as a greater number

indicators. of metrics were available, and these were
generally judged to be of better data quality

A weighted average of scores on each indicator for the purpose of our Index. As a result, core

then gives the Index; an overall composite score. functions were given two-thirds of the overall

Functions make up two-thirds of the overall composite weight, and attributes one-third.
weight and attributes one-third.
The International Civil Service Effectiveness Index | Technical Report 71

Figure 11: Data weightings in the InCiSE Index

A B C D

Integrity /
1 18 16 /
1 12 /
19

Openness /
1 18 9 /
1 12 /
19

Attributes Capabilities /
1 18 4 /
1 12 /
19

Weight: 1/3
Metrics: 34 Inclusiveness /
1 18 5 /
1 12 /
19

Staff Engagement /
1 18 – – –

Innovation /
1 18 – – –

Policymaking /
2 33 8 /
1 12 /
1 12

Central Fiscal and financial /
2 33 3 /
1 12 /
1 12

Executive Regulation /
2 33 6 /
1 12 /
1 12

Risk/Crisis Management /
2 33 9 /
1 12 /
1 12

Procurement /
2 33 – – –
Core functions
Mission HR /
2 33 5 /
1 12 /
1 12
Weight: 2/3
Metrics: 42 Support IT /
2 33 – – –

Finance /
2 33 – – –

Tax Administration /
2 33 6 /
1 12 /
1 12

Direct
Social Security /
2 33 1 /
1 12 /
1 12
Service
Administration
Delivery
Digital Services /
2 33 4 /
1 12 /
1 12

A Preferred weight if data on all-indicators was available (equal weight within attributes and functions)
B Number of data metrics found and included

C Preferred weight given missing data for some indicators (equal weight within attributes and functions)
D Data quality-adjusted weight (used in this Index)

At a further level of disaggregation, Section 4 describes how the individual metrics are weighted in

each of the indicators they support.
72 The International Civil Service Effectiveness Index | Technical Report

Chapter 8:
Sensitivity Analysis

In chapter 8 we describe the sensitivity analysis undertaken, and the resultant effect on the

pilot Index results.

8.1 Introduction to sensitivity In the sensitivity analysis undertaken for
the InCiSE Index we focus on four main

analysis
assumptions: adjusting for GDP per capita,
The development of the Index involves stages the method of aggregation, exclusion of one

where subjective judgements have to be limiting dataset at a time and imputation of

made: the selection of individual data sets, missing data.
the treatment of missing values, the choice

of aggregation model and the weights of the 8.2 Sensitivity analysis 1:
indicators, for example. Adjusting for GDP per Capita

The quality of the Index depends on the Despite its acknowledged shortcomings,
soundness of its assumptions. Good modelling GDP per capita is still the most commonly used

practice requires that we develop an evaluation summary indicator of living standards. Much

of the confidence in the model, assessing the of the policy advice provided by international

uncertainties associated with the modelling organisations is framed on cross-country

process and the subjective choices taken. This comparisons of per capita GDP.55 The framework

role is fulfilled by sensitivity analysis, scrutinising established by the InCiSE Index sets out to

aspects of the relationships between inputs into measure and compare effectiveness of civil

the Index and its final output. services through an approach that is comparable

The approach taken to sensitivity analysis shows across countries. However, the economic

how variation in the output can be apportioned, resources available to develop and maintain

qualitatively or quantitatively, to different sources the effective institutions and practices of civil

of variation in the assumptions, and of how the services, and governments’ ability to raise

Index depends upon the information fed into revenues through taxation, are dependent on

it. Sensitivity analysis can help to gauge the the level of economic development and income

robustness of the composite indicator ranking levels of the economy, as measured by GDP

and increases transparency, to identify which per capita.
countries are assessed more or less favourably

under certain assumptions and to help develop

a debate around the Index.

55 World Economic Outlook, April 2003 – Chapter 3: Growth and Institutions
The International Civil Service Effectiveness Index | Technical Report 73

Research on growth and institutions has • The method used here is as follows: the

sought to identify the deep structural country’s Index score (prior to normalisation)
determinants of countries’ level of development. is divided by its GDP per capita, scores

The various measures of institutional quality are then normalised between 0 and 1.
reflect the generally high correlations among For example; if two countries had obtained

themselves and measures of economic identical scores in the Index but one country

performance (Table 21). All appear closely (A) had a GDP per capita twice that of the

related to cross-country differences in GDP other (B), then after adjusting for GDP, B’s

per capita, as well as to other measures of score (and position) would be higher than A’s.
economic performance such as growth rates

and the volatility of growth. Building on the Secondly, investigating this sensitivity allows

close correlation between institutional quality for comparison of performance across smaller

and development, recent analyses attempt groups of countries with similar income/GDP

to address the possibility of reverse causality per capita characteristics. In future, this may

from development to institutions, and the enhance the possibility of the Index serving

relative significance of institutions compared as a tool to drive performance improvement,
with other influences on development, for example by allowing decision makers to see

such as trade openness, geographical factors, which countries perform best at a similar income

and economic policies.56 level, this may support adopting arrangements
and improvements of practices that are more

Those countries with higher GDP per capita transferable between countries of comparable

may also be historically more stable, with income levels.
greater social capital and more established

institutions. While these are recognised as Thirdly, this adjustment allows for what might be

aspects which may support a highly effective considered a ‘fairer’ approach to assessment;
civil service, in considering the reverse causality assessing effectiveness in light of varying income

from development to institutions, and the impact levels and resources among countries, and

of other influences – it is helpful to test the estimating whether countries are stronger or

sensitivity of adjusting the Index for GDP per weaker performers considering their relative

capita for several reasons. resource constraints. This adjustment may
provide insights into capacity for effectiveness

Firstly, it allows us to measure and estimate the or potential for improved effectiveness, and may

importance of income levels for Index scores by enhance the impact and take up of the Index

assessing how Index scores are affected when among those countries with lower incomes and

the following adjustments are made: perhaps lower scores in the core Index.

• Scores and relative position are revised
to reflect income levels, the extent of the
revision depends on the difference between
a country’s GDP per capita and the average,
and also its original score in the Index.

56 Kaufmann, Kraay, and Zoido-Lobatón (1999b); Heritage Foundation (2003); Gurr and Marshall (2000); and World Development
Indicators, World Bank (2002)
74 The International Civil Service Effectiveness Index | Technical Report

Table 21: Index rankings comparison after adjustment for GDP per capita

Country Index Score Rank GDP Adjusted Rank after
Index Score adjustment

Canada 1.00 1 0.76 5

New Zealand 0.95 2 0.87 3

Australia 0.91 3 0.67 9

United Kingdom 0.91 4 0.74 6

Finland 0.87 5 0.71 7

Sweden 0.86 6 0.62 11

Estonia 0.81 7 1.00 1

Norway 0.81 8 0.43 18

Korea, Republic of 0.78 9 0.76 4

United States of America 0.74 10 0.43 19

Denmark 0.73 11 0.52 13

Switzerland 0.61 12 0.32 24

Belgium 0.60 13 0.45 17

Netherlands 0.58 14 0.40 21

Japan 0.57 15 0.52 14

Austria 0.56 16 0.38 22

France 0.50 17 0.42 20

Spain 0.49 18 0.48 15

Mexico 0.47 19 0.95 2

Ireland 0.46 20 0.22 26

Poland 0.44 21 0.58 12

Chile 0.44 22 0.68 8

Slovenia 0.44 23 0.48 16

Germany 0.40 24 0.27 25

Turkey 0.37 25 0.66 10

Portugal 0.31 26 0.36 23

Italy 0.21 27 0.19 27

Czech Republic 0.16 28 0.16 28

Greece 0.04 29 0.05 29

Hungary 0.00 30 0.01 30

Slovakia 0.00 31 0.00 31
The International Civil Service Effectiveness Index | Technical Report 75

8.3 Sensitivity analysis 2: While our approach to aggregation is reasonable
and justified, we acknowledge there are a

Aggregation Method
range of possible approaches and weightings

The InCiSE index brings together the underlying which could be followed in building up the

data sets into ‘indicators’; attributes and composite from these component indicators,
functions, each measuring a different dimension with corresponding arguments in favour. Indeed,
of effectiveness. These attributes and functions there are also a range of possibilities for building

are then drawn into an overall composite up the indicators from their constituent datasets

indicator of civil service effectiveness. Attributes (Chapter 4 provides further detail as how the

contribute one third of the composite weighting indicators are weighted across the contributing

and functions make-up the remaining two thirds datasets). As part of the sensitivity analysis
(see Chapter 7 for further detail and justification undertaken, we investigated the impact of

for the aggregation approach adopted). alternative aggregation approaches to test the
impact on the composite results obtained from
the particular approach adopted.

We first looked at the impact of adjusting the
weighting between attributes and functions
such that each contributes half of the composite
weighting. Results are shown in Table 22.
76 The International Civil Service Effectiveness Index | Technical Report

Table 22: Comparison of scores and ranking under adjusted aggregation method. Attributes and
Functions each provide half the weighting.

Country Index score Rank Index score Rank
adjusted for
equal weighting
Canada 1.00 1 1.00 1
New Zealand 0.95 2 0.98 2
Australia 0.91 3 0.90 3
United Kingdom 0.91 4 0.89 5
Finland 0.87 5 0.89 4
Sweden 0.86 6 0.85 6
Estonia 0.81 7 0.75 8
Norway 0.81 8 0.83 7
Korea, Republic of 0.78 9 0.73 11
United States of America 0.74 10 0.75 9
Denmark 0.73 11 0.74 10
Switzerland 0.61 12 0.64 12
Belgium 0.60 13 0.61 15
Netherlands 0.58 14 0.63 13
Japan 0.57 15 0.61 14
Austria 0.56 16 0.59 16
France 0.50 17 0.51 17
Spain 0.49 18 0.48 19
Mexico 0.47 19 0.37 24
Ireland 0.46 20 0.47 21
Poland 0.44 21 0.49 18
Chile 0.44 22 0.46 22
Slovenia 0.44 23 0.46 23
Germany 0.40 24 0.48 20
Turkey 0.37 25 0.30 26
Portugal 0.31 26 0.31 25
Italy 0.21 27 0.19 27
Czech Republic 0.16 28 0.17 28
Greece 0.04 29 0.08 29
Hungary 0.00 30 0.00 31
Slovakia 0.00 31 0.02 30

Also considered is an alternative weighting where attributes contribute two thirds of the composite,
with functions making up the remaining third. Results for this sensitivity are shown in Table 23.
The International Civil Service Effectiveness Index | Technical Report 77

Table 23: Comparison of scores and ranking under adjusted aggregation method. Attributes two

thirds weighting and Functions one third.

Country Index score Rank Index score Rank
adjusted for two
thirds attributes
weighting

Canada 1.00 1 0.99 2

New Zealand 0.95 2 1.00 1

Australia 0.91 3 0.89 4

United Kingdom 0.91 4 0.86 5

Finland 0.87 5 0.92 3

Sweden 0.86 6 0.84 7

Estonia 0.81 7 0.68 11

Norway 0.81 8 0.85 6

Korea, Republic of 0.78 9 0.68 12

United States of America 0.74 10 0.75 9

Denmark 0.73 11 0.76 8

Switzerland 0.61 12 0.67 13

Belgium 0.60 13 0.62 15

Netherlands 0.58 14 0.68 10

Japan 0.57 15 0.64 14

Austria 0.56 16 0.61 16

France 0.50 17 0.53 19

Spain 0.49 18 0.47 23

Mexico 0.47 19 0.27 25

Ireland 0.46 20 0.47 22

Poland 0.44 21 0.54 18

Chile 0.44 22 0.49 20

Slovenia 0.44 23 0.48 21

Germany 0.40 24 0.55 17

Turkey 0.37 25 0.24 26

Portugal 0.31 26 0.31 24

Italy 0.21 27 0.17 28

Czech Republic 0.16 28 0.19 27

Greece 0.04 29 0.11 29

Hungary 0.00 30 0.00 31

Slovakia 0.00 31 0.05 30
78 The International Civil Service Effectiveness Index | Technical Report

From the results of this sensitivity analysis 8.4.1 Subjective assessment

it appears while there is an impact on the

Index scores and rankings obtained under As noted earlier, the InCiSE Index is based on

the two alternative aggregation methods, this a mixture of administrative data and subjective

impact is small in relation to the overall Index; or perceptions‐based measures, taken from

most countries obtain very similar scores and surveys of households and firms as well as

rankings under the alternative aggregation expert assessments produced by various

approaches when compared to the approach organisations. Our view is that perceptions data

followed in the Index. This is reassuring, and have particular value in the measurement of

is indicative of the high degree of correlation governance. First, perceptions matter because

between the indicators. However, naturally if agents base their actions on their perceptions.
the weighting possibilities were pushed further If citizens believe that the courts are inefficient

towards their limits or extended to the level of or the police are corrupt, they are unlikely to

the data sets underlying the indicators, more avail themselves of their services. Similarly,
significant impacts on the scores and rankings enterprises base their investment decisions

might be observed; although the merits of ‐ and citizens their voting decisions ‐ on their

such an alternative weighting would likely be in perceived view of the investment climate and

question. Our favoured approach to aggregation the government’s performance. Secondly, in

appropriately takes into account a broad many areas of the Index framework, there are

range of attributes and functions to allow for a few alternatives to relying on perceptions data.
reasonable assessment of overall civil service For instance, this has been particularly the case

effectiveness. for Integrity, an attribute in the framework, which
almost by definition leaves few records that

8.4 Sensitivity analysis 3: Exclusion can be captured by purely objective measures.
Additionally even when objective or fact‐based

of Input Data data are available, this may capture the notion of

In Chapter 4 we set out the data underpinning laws and arrangements “in theory”, which often

each of the indicators and the weighting differs substantially from the reality that exists

given to each to make up the indicator score. “in practice”. For example, in every one of the 70

Where we have identified limitations in the countries covered in the 2007 and 2008 waves

data used, these are specified. It is important of the Global Integrity Index, it is formally illegal

to examine the results of the Index and for a public official to accept a bribe. Yet, despite

investigate whether utilising data which has them being identical when measured in theory,
limitations has any skewed effect on the results there are large differences across these countries

obtained. We therefore examine the exclusion in perceptions of the frequency with which bribes

of three categories of data that have general are in fact accepted by public officials.
limiting qualities; data resulting from subjective
Despite these advantages, one might

assessment, data deemed to be out of date,
nevertheless reasonably be concerned about

and data where the wider public sector is used
various potential problems in the interpretation

to proxy the civil service.
of the subjective data we rely upon in the Index.
Broadly such concerns question the extent to
which perceptions data adequately capture the
relevant reality. A first basic issue is simply that
perceptions data on civil service effectiveness
are imprecise. This by itself is not surprising
– as we have argued above, all measures of
effectiveness are necessarily imprecise proxies
for the outcomes they are intended to measure,
but imprecision alone does not disqualify the use
of perceptions‐based data.
The International Civil Service Effectiveness Index | Technical Report 79

A potentially more serious concern is that there Yet another potential source of bias comes

are various systematic biases in perceptions from the possibility that different providers

data on civil service effectiveness. One possibility of governance perceptions data rely on

is that different types of respondents differ each other’s assessments, and as a result

systematically in their perceptions of the same make correlated perceptions errors. This

underlying reality. For example, it could be would undermine the information content in

the case that business people, represented such indicators. Assessing this concern is

by owners of the businesses covered in a difficult because the high correlation between

survey, or the expert assessments provided governance perceptions rankings from different

by commercial business information providers, sources could be due either to perception errors,
have different views of what constitutes an or due to the fact that these sources are in fact

effective civil service than other types of accurately measuring cross‐country corruption

respondents, such as households or public differences and so necessarily agree with

sector agencies. each other.

Another possibility is that biases are introduced Concern about these potential sources of bias

by the ideological orientation of the organisation will be moderated to the extent that excluding

providing the subjective assessments of underlying data based on subjective assessment

governance. An additional type of bias might affects the resulting Index scores and rankings.
be the possibility that subjective assessments The results of this aspect of the sensitivity

of governance are driven by factors other analysis are demonstrated in Table 24.
than governance itself, such as the level of

development or recent economic performance

of a country.
80 The International Civil Service Effectiveness Index | Technical Report

Table 24: Sensitivity analysis – Removal of subjective data

Country Index score Rank Adjusted Index Adjusted Rank
Score

Canada 1.00 1 0.94 4

New Zealand 0.95 2 0.95 3

Australia 0.91 3 0.93 5

United Kingdom 0.91 4 0.92 6

Finland 0.87 5 0.87 7

Sweden 0.86 6 0.98 2

Estonia 0.81 7 1.00 1

Norway 0.81 8 0.83 8

Korea, Republic of 0.78 9 0.76 9

United States of America 0.74 10 0.66 11

Denmark 0.73 11 0.70 10

Switzerland 0.61 12 0.62 13

Belgium 0.60 13 0.46 20

Netherlands 0.58 14 0.61 14

Japan 0.57 15 0.48 18

Austria 0.56 16 0.63 12

France 0.50 17 0.48 19

Spain 0.49 18 0.51 17

Mexico 0.47 19 0.57 15

Ireland 0.46 20 0.36 23

Poland 0.44 21 0.24 26

Chile 0.44 22 0.32 25

Slovenia 0.44 23 0.54 16

Germany 0.40 24 0.33 24

Turkey 0.37 25 0.44 21

Portugal 0.31 26 0.37 22

Italy 0.21 27 0.14 28

Czech Republic 0.16 28 0.21 27

Greece 0.04 29 0.02 30

Hungary 0.00 30 0.00 31

Slovakia 0.00 31 0.05 29
The International Civil Service Effectiveness Index | Technical Report 81

8.4.2 Out of date data The project aspires to utilise the best available
data and where possible, this data should be

The project aims to define a framework recent. Where data is not released annually and

comprising the necessary aspects constituting is less recent, the inclusion of data sets has

an effective civil service, in a way that can been based on a judgement of its relevance to

realistically enable international data to be the framework despite this drawback. However,
collected to measure against it. The framework inclusion of these datasets may reduce the

developed takes into account a range of extent to which the composite reflects recent

functions and attributes of civil services and developments towards greater or reduced

utilises data from a range of sources. Many of civil service effectiveness among the countries

these data sources are released and updated included in the Index. A country that has made

annually, such that on release of the Index, rapid progress along a number of fronts may

we can be confident that the data utilised be dragged back by the inclusion of a measure

presents a recent and highly relevant assessment which is somewhat out of date and drawn from

of the aspect of effectiveness this data a period before the effects of such initiatives

underpins. However, some of the data sources had been felt. Table 25 shows the impact on

are updated less frequently, were released the rankings of excluding data released prior to

with the scheduled update process still to be 2013, the year judged as a cut-off point in this

confirmed, or were published with no scheduled sensitivity analysis.
updates envisioned.
82 The International Civil Service Effectiveness Index | Technical Report

Table 25: Sensitivity analysis – Removal of data pre-2013

Country Index Score Rank Adjusted Index Adjusted Rank
Score

Canada 1.00 1 1.00 1

New Zealand 0.95 2 0.99 2

Australia 0.91 3 0.95 3

United Kingdom 0.91 4 0.92 5

Finland 0.87 5 0.94 4

Sweden 0.86 6 0.88 6

Estonia 0.81 7 0.80 9

Norway 0.81 8 0.88 7

Korea, Republic of 0.78 9 0.75 12

United States of America 0.74 10 0.80 10

Denmark 0.73 11 0.81 8

Switzerland 0.61 12 0.78 11

Belgium 0.60 13 0.73 13

Netherlands 0.58 14 0.71 14

Japan 0.57 15 0.63 16

Austria 0.56 16 0.61 17

France 0.50 17 0.57 18

Spain 0.49 18 0.53 20

Mexico 0.47 19 0.40 23

Ireland 0.46 20 0.69 15

Poland 0.44 21 0.48 22

Chile 0.44 22 0.56 19

Slovenia 0.44 23 0.38 24

Germany 0.40 24 0.52 21

Turkey 0.37 25 0.35 25

Portugal 0.31 26 0.32 26

Italy 0.21 27 0.24 28

Czech Republic 0.16 28 0.26 27

Greece 0.04 29 0.05 29

Hungary 0.00 30 0.04 30

Slovakia 0.00 31 0.00 31
The International Civil Service Effectiveness Index | Technical Report 83

8.4.3 Public sector proxy

The Index is intended to cover the effectiveness

of the civil service as defined in Chapter 2,
rather than the wider public sector. Occasionally,
however, due to availability of data, data covering

the wider public sector data is used to proxy

for the civil service where this is deemed to be

a reasonable proxy. To ensure that the use of

such proxies does not significantly distort the

Index results, we developed a version of the

Index where these proxy measures are excluded.
Table 26 compares the results and rankings for

this sensitivity.
84 The International Civil Service Effectiveness Index | Technical Report

Table 26: Sensitivity analysis – Removal of public sector proxy data

Country Index Score Rank Adjusted Index Adjusted Rank
Score

Canada 1.00 1 1.00 1

New Zealand 0.95 2 0.87 5

Australia 0.91 3 0.98 2

United Kingdom 0.91 4 0.92 3

Finland 0.87 5 0.84 7

Sweden 0.86 6 0.86 6

Estonia 0.81 7 0.83 8

Norway 0.81 8 0.78 9

Korea, Republic of 0.78 9 0.92 4

United States of America 0.74 10 0.70 11

Denmark 0.73 11 0.72 10

Switzerland 0.61 12 0.55 15

Belgium 0.60 13 0.37 19

Netherlands 0.58 14 0.34 21

Japan 0.57 15 0.25 24

Austria 0.56 16 0.50 16

France 0.50 17 0.60 13

Spain 0.49 18 0.63 12

Mexico 0.47 19 0.59 14

Ireland 0.46 20 0.33 22

Poland 0.44 21 0.22 25

Chile 0.44 22 0.31 23

Slovenia 0.44 23 0.48 17

Germany 0.40 24 0.18 27

Turkey 0.37 25 0.37 20

Portugal 0.31 26 0.46 18

Italy 0.21 27 0.18 26

Czech Republic 0.16 28 0.01 30

Greece 0.04 29 0.06 29

Hungary 0.00 30 0.12 28

Slovakia 0.00 31 0.00 31
The International Civil Service Effectiveness Index | Technical Report 85

8.5 Sensitivity analysis 4: A range of alternative approaches to imputation
are possible, and of course one response to

Imputation Method
missing data is list-deletion of those countries

Chapter 5 describes the statistical methodology with missing data, although this would not meet

adopted to estimate or ‘impute’ missing data. the objective of our chosen imputation approach

With imputed data included, a complete data set by allowing the Index to represent a range of

is estimated and the Index can be determined. countries despite missing data. To examine the

While the approach to imputation adopted is impact of the favoured imputation approach we

sensible, based on demonstrated relationships compared the Index calculated to an approach

between variables and the proportion of missing using simple mean imputation to estimate

data, it is important to consider the impacts of missing data. Mean imputation is a method in

alternative imputation methods. which the missing value on a certain variable is
replaced by the mean of the available cases.
This method maintains the sample size and
is easy to use, but the variability in the data is
reduced, so the standard deviations and the
variance estimates tend to be underestimated.
Table 27 documents the comparison of the Index
generated under the two imputation methods.
86 The International Civil Service Effectiveness Index | Technical Report

Table 27: Sensitivity analysis – Differing imputation methods

Country Multiple Multiple Mean Imputation Mean Imputation
Imputation Index Imputation Rank Score Rank
Score

Canada 1.00 1 1.00 1

New Zealand 0.95 2 0.94 4

Australia 0.91 3 0.91 6

United Kingdom 0.91 4 0.98 2

Finland 0.87 5 0.97 3

Sweden 0.86 6 0.91 5

Estonia 0.81 7 0.87 8

Norway 0.81 8 0.88 7

Korea, Republic of 0.78 9 0.75 10

United States of America 0.74 10 0.69 11

Denmark 0.73 11 0.79 9

Switzerland 0.61 12 0.65 12

Belgium 0.60 13 0.61 16

Netherlands 0.58 14 0.63 13

Japan 0.57 15 0.62 15

Austria 0.56 16 0.63 14

France 0.50 17 0.53 18

Spain 0.49 18 0.55 17

Mexico 0.47 19 0.42 22

Ireland 0.46 20 0.50 19

Poland 0.44 21 0.46 21

Chile 0.44 22 0.42 23

Slovenia 0.44 23 0.49 20

Germany 0.40 24 0.42 24

Turkey 0.37 25 0.37 25

Portugal 0.31 26 0.34 26

Italy 0.21 27 0.20 27

Czech Republic 0.16 28 0.18 28

Greece 0.04 29 0.07 29

Hungary 0.00 30 0.02 30

Slovakia 0.00 31 0.00 31
The International Civil Service Effectiveness Index | Technical Report 87

Chapter 9:
Next steps

In chapter 9 we note some limitations of the pilot Index, and planned next steps.

9.1 Identified limitations of the Index Data Updates
Whilst every effort has been made to make use

This paper has offered a summary of key
of the most up-to-date data, each iteration of

aspects of the methodology to construct the
InCiSE will, by necessity, have a cut-off for data

pilot Index. When interpreting the results of the
being included in that years Index. InCiSE reports

Index the limitations listed, which are common to
will specify the date up to which data released

many other international indicators, and planned
has been included. For instance, the inaugural

next steps should be noted. The development of
pilot report includes data released up to January

the pilot Index is the culmination of the first stage
2017. The InCiSE Index will be updated annually

of an ambitious and innovative effort to develop
with the most recent data available.
and measure against a framework of civil service

effectiveness. Wide-ranging and sustained Varying depth of data

engagement from inception to publication has
With some data metrics only representing,
offered challenge, creativity and expertise to
for example, one survey question, but others

support improvement of the Index up to the
representing scores on an entire index relevant

point of the pilot publication and beyond. As
to the indicator (e.g. Open Data Barometer

this Index develops, we hope to overcome or
score measuring open data), the depth of data in

reduce the impact of these limitations as well as
indicators varies.
strengthening the indicator in other areas.
Public sector performance being used as

The general limitations of the Index are set out
a proxy for civil service performance

below. Limitations associated with particular

datasets included in the Index are identified in Some of the metrics used in the Index explicitly

Annex A. try to measure performance of the public
sector, not just civil service bodies. Therefore

Missing data until similar data collections are completed for

Only two countries have non-missing values the civil service, these measures can only be

across all 76 metrics. This is because different considered proxies. The OECD handbook on

included data sets cover different sets of constructing composite indicators notes that:
countries. However, a number of countries have “Proxy measures can be used when the desired

close to the full complement of data, and missing data are unavailable or when cross-country

data for these countries has been imputed. The comparability is limited.”
imputation methods utilised are well supported

for the degree of missing data observed and

are informed by observed relationships between

metrics where possible.
88 The International Civil Service Effectiveness Index | Technical Report

Mixture of quantitative and qualitative/ a) Strengthening data collection

subjective data Tasks will include: determining how best

The Index mixes quantitative and qualitative to collect data on the 5 omitted indicators

data. The OECD handbook on constructing (staff engagement, innovation, IT, procurement,
composite indicators notes that “Given finance); exploring how data collection for

a scarcity of internationally comparable existing indicators can be strengthened,
quantitative (hard) data, composite indicators including thematic gaps; expanding data sources

often include qualitative (soft) data from surveys and looking at ways of avoiding over reliance

or policy reviews.” on a single survey; reducing instances of public
sector performance being used as a proxy for

Possible ‘spill over’ across indicators civil service performance; and cross-country

The types of things measured under some of applicability issues.
the indicators will occasionally be relevant to

others. Attempts have been made to minimise Complementary work, particularly by the OECD,
occurrences of this by adapting the framework may help to fill some data gaps over time.
to ensure each indicator is a unique concept. Additional data collection, for example through
the commissioning of specific country surveys,
Proxy measures of effectiveness may also help to provide valuable insights.
The nature of the true level of civil service b) Refining the InCiSE framework

effectiveness in a country is inherently

unobservable, and therefore any observed Framework tasks will include: exploring the

empirical measure of the aptitude or capacity potential to add new civil service functions

will be a proxy for the broader dimensions of (for example, security, foreign affairs, defence

effectiveness it reflects. One consequence of and justice); and ensuring each indicator in

this is that our estimates of effectiveness are the framework is a unique concept, to avoid

subject to uncertainty. Users should not over- duplication or overlap between the themes

interpret small differences in performance (across being measured.
countries or, in future editions, over time) in the An issue that has emerged through the pilot

aggregate Index and the underlying indicators. is the need to consider how best to adapt

The presence of uncertainty does not imply that the InCiSE framework to enable it to measure

the Index cannot be used to make meaningful and compare core functions within federal

comparisons of effectiveness across countries government systems where some aspects of

or over time. this work are often carried out by a lower tier.

9.2 Priorities for the next 12 months Changes to the InCiSE framework will mainly
be determined through user feedback and

InCiSE is a long term project and the discussions with data owners to learn lessons

founding organisations have committed to from their own experiences, as well as through

supporting its development for a further four guidance from an International Advisory Panel.
years. This will include publishing an annual

report and developing an interactive website. c) Expanding country coverage

An International Advisory Panel has also been InCiSE will explore the scope to expand the

established to guide this work. Index’s current country coverage over time,
including the potential to include some non-
As described earlier, this first edition of the Index OECD countries. Country coverage in future

is a pilot- further work is required to refine the will largely be determined by feedback from

methodology and make the data more robust. countries on the usefulness of having their own

The main issues that the project will focus on set of indicators, and the availability of data to

over the next 12 months include: produce results.
The International Civil Service Effectiveness Index | Technical Report 89

Several countries were excluded from the pilot Feedback is vitally important to help improve

Index because they had fewer than 75% of the InCiSE Index and the project team would

the metrics available. However, three countries welcome responses from country governments,
(Bulgaria, Romania and Croatia) had over 70% of as well as policy and learning networks, think

data available. A small increase in data collection tanks and academia.
may help them meet the threshold for inclusion in

the next Index. The InCiSE project team will continue to
coordinate closely with other institutions who

d) Increasing InCiSE partners and network are engaged in similar efforts to measure civil

The founding institutions will actively encourage service effectiveness, as well as with the many

more partners to join them and support organisations who have made a vital contribution

the development of the InCiSE Index over to InCiSE so far by permitting use of their

time, through advice, expertise and funding. research or survey data. Continued collaboration

The project’s International Advisory Panel will will be essential to help strengthen InCiSE in the

also play a key role in promoting InCiSE and coming years.
encouraging more partners.

The Blavatnik School of Government will host

an international conference in Autumn 2017

to discuss the pilot Index with a wide range of

interested players. This event will additionally

provide a key opportunity to build support for

InCiSe and increase its network.

The InCiSE pilot Index has already brought

together a rich volume of data and insights.
Its launch provides an important opportunity to

stimulate wide-ranging discussions at country

level, as well as globally, about civil service

effectiveness issues generally, as well as the

relevance and usefulness of some data.
90 The International Civil Service Effectiveness Index | Technical Report

Appendices

a) Annex A: Data limitations

b) Annex B: Reference list

c) Annex C: Reader’s Guide – Abbreviations and Country Codes
The International Civil Service Effectiveness Index | Technical Report 91

Annex A: Data limitations

Table 28 sets out the limitations of the data used within the Index. The data is described in chapter 4

of this report.

Table 28: Data limitations within the Index
Indicator Limitations

Policy making • The data may be capturing the wider public sector rather than the civil service,
as the indices are designed to measure government performance. For example,
‘Strategic planning’ may include the performance of policy-related ministerial
decisions and/or academics. The ‘Monitoring agencies, bureaucracies’ metric
includes subnational governments, so is only a proxy for the civil service.
• A large number of the metrics included are subjective. However, the
standardisation process embedded in the Bertelsmann methodology helps
to regulate scores across countries.
• The survey questions give only a partial picture of the quality of policy advice.
For example, they do not capture the quality of written and oral briefing,
the quality of draft legislation, or the extent to which policy advice is based
on evidence.

Fiscal and financial • Some of the data used in this indicator is from 2012.
management • More detailed data on the use and quality of economic appraisal and evaluation
would improve the robustness of this indicator. The current metrics only partially
measure what we would like to include in this indicator.

Regulation • The metrics are mainly based on information provided to the OECD by countries
directly. Whilst countries are asked to support the information they provide with
evidence, it may be possible that countries exaggerate claims about the quality
of their processes.

Crisis/risk • The Hyogo framework gives an overview of best practice in crisis/risk
management management, but many of the countries in this indicator set appear to already
be adhering to large parts of it. This leads to a lack of variation in scores.
• Ideally this indicator would also be able to measure the quality of response from
civil servants after risks have occurred. All metrics included so far only contain
details on what risk management strategies and activities exist, with less detail
on their quality.

Human Resources • Data on meritocracy is based on expert opinion, which may not necessarily
Management accurately reflect reality. Data on meritocracy also refers to ‘public sector
employees’.
• Attracting talent is measured using expert assessments, but hard quantitative
data would be preferable for assessing financial incentives provided.
• Important themes of Human Resources Management do not have data
available, for example the quality of learning and development opportunities.
Financial incentives do not give a full picture of how ‘attractive’ an organisation
is to work for.

Tax administration • Cost of collection ratios are problematic because amount of revenue collected
depends to a large extent on tax rates. Adjusting revenue collected for tax rates
may be a potential solution, but how feasible this might be is unknown.
• Further investigation may be needed into the validity of comparing across
different countries’ tax systems (e.g. into how it’s been decided which taxes
should be included in each country).
92 The International Civil Service Effectiveness Index | Technical Report

Table 28: Data limitations within the Index

Social security • Whilst the included metric may give a good overall picture of efficiency,
only having one metric does not give a comprehensive picture of system
performance.
• Further investigation may be needed into the validity of comparing across
different countries’ welfare systems.

Digital services • The data only covers mystery-shopper assessments of seven life events so
does not assess all of the services which governments typically provide digitally.

Integrity • QoG data on the core values and principles civil servants adhere to is for public
sector, not civil servants specifically. OECD data on whistleblower protection
also covers the public sector.
• A large amount of the data included is subjective. A lot of it relies on expert
opinion, which is subject to the choice of experts. Citizen and business
perceptions data have the usual limitations which surround perception-based
measurement.

Openness • The indices are designed to measure open government in general, so may
capture elements of ministerial and wider public sector performance.

Capabilities • Leadership capabilities within the organisation are important, but not currently
fully captured. At present this indicator does not cover more specific measures
of skills, for example capabilities in specific areas such as economics and
statistics. Measures of whether these skills are deployed in the right areas may
also be desirable but difficult to obtain.
• There is currently no data on the educational attainment of civil services across
the world. OECD PIAAC data gives an estimate of public sector educational
attainment. Industry codes can be used to get closer to civil service but sample
sizes at that level were too small to be reliable.

Inclusiveness • Data is not available on most demographics of interest.
• Most OECD data is from 2010. However for a small number of countries data is
for 2011 and in one case is for 2009.
• The QoG gender data asks experts to estimate gender representation (%),
which may not always be accurate. For ethnicity metrics they are asked
to score ethnic/religious inclusiveness on a scale of 1 to 7; if management
information data were available it would give a more accurate assessment.
• This indicator penalises countries for having over-representation of
‘disadvantaged’ groups. An alternative approach would be to give the same,
‘perfect’ score if a disadvantaged group is overrepresented.
The International Civil Service Effectiveness Index | Technical Report 93

Annex B: Reference list

Baubion, C. (2013). OECD Risk Management: Strategic Crisis Management. OECD. [Online].
Available from: http://www.mmc.com/content/dam/mmc-web/Files/Strategic-Crisis-Management-
paper-July-2013.pdf. [Accessed on 21 February 2017].

Bouchard, P. and McCrae, J. (2013). Financial Leadership for Government. [Online]. Institute for

Government and The Chartered Institute of Management Accountants (CIMA). Available from:
https://www.instituteforgovernment.org.uk/sites/default/files/publications/2739%20IFG%20-%20

Financial%20Leadership%20v10%20(online-version).pdf. [Accessed 10 March 2017].

Bovaird, T. and Löffler, E. ed. (2003). Public Management and Governance. 2nd edn. Routledge.

Centre for the Study of Governance Innovation (GovInn). (2014).What is Governance Innovation?
[Online]. Available from: http://governanceinnovation.org/what-is-governance/. [Accessed 7 March

2017].

Chalam, K.S. ed. (2014). Governance in South Asia: State of the Civil Services. Pp. 59-72. SAGE

Publications Ltd.

Christensen, T., Fimreite, A.L. and Lægreid, P. (2011). Crisis Management: The Perceptions of

Citizens and Civil Servants in Norway. Administration & Society, 43(5), pp.561-594.

European Commission. (2006). Welcome to the world of Public Internal Financial Control. [Online].
Available from: http://ec.europa.eu/budget/library/biblio/documents/control/brochure_pifc_en.pdf.
[Accessed 10 March 2017].

Fukuyama, F. (2013). What Is Governance? Governance: An International Journal of Policy,
Administration, and Institutions, 26(3), pp. 347-368.

Graham, J., Amos, B. and Plumptre, T. (2003). Principles for Good Governance in the 21st Century.
[Online]. Institute on Governance, Policy Brief No.15. Available from: http://unpan1.un.org/intradoc/
groups/public/documents/UNPAN/UNPAN011842.pdf. [Accessed 17 March 2017].

Herbert, N. and Healey, J. (2015). Time to Govern Up. [Online]. Govern Up. Available from: http://
www.governup.org/latest/. [Accessed 22 February 2017].

Holt, J. and Manning, N. (2014). Fukuyama is Right about Measuring State Quality: Now what?
Governance: An International Journal of Policy, Administration and Institution, 27(4), pp.717-728.

Huther, J. and Shah, A. (1999). Applying a Simple Measure of Good Governance to the Debate on

Fiscal Decentralization. [Online]. World Bank. Available from: http://siteresources.worldbank.org/
INTWBIGOVANTCOR/Resources/wps1894.pdf. [Accessed 1 March 2017].

International Civil Service Commission. (2002). Standards of conduct for the international civil

service. [Online]. Available from: http://icsc.un.org/resources/pdfs/general/standards.pdf. [Accessed

15 February 2017].

International Monetary Fund (IMF). (2016). Consultation on the IMF Guideline on Governance Issues.
[Online]. Available from: https://www.imf.org/external/np/exr/consult/2016/governance/.
[Accessed 9 March 2017].
94 The International Civil Service Effectiveness Index | Technical Report

Kaufmann, D., Kraay, A. and Zoido-Lobatón, P. (1999). Governance Matters. The

World Bank Development Research Group Macroeconomics and Growth and World

Bank Institute, Governance, Regulation and Finance. Policy Research Working

Paper 2196. [Online]. Available from: https://poseidon01.ssrn.com/delivery.php?
ID=492099070121007123122084103071079102064038020065064007072124071027074112085

110025086006103020004074093027019087120122087004002065107067088079116012120070

113003011075024126024116123112090078&EXT=pdf. [Accessed 20 February 2017].

Lonti, Z. and M. Woods (2008), Towards Government at a Glance: Identification of Core Data and

Issues related to Public Sector Efficiency, OECD Working Papers on Public Governance, No. 7,
OECD Publishing. doi:10.1787/245570167540

MacLeod, D. and Clarke, N. (2009). Engaging for Success: enhancing performance through

employee engagement. A report to Government. [Online]. Available from: http://dera.ioe.
ac.uk/1810/1/file52215.pdf. [Accessed 8 March 2017].

Madzova, V., Sajnoski, K. and Davcev, L. (2013). E-Government as an Efficient Tool towards Good

Governance (Trends and Comparative Analysis throughout Worldwide Regions and within West

Balkan Countries). Balkan Social Science Review, (1), pp.157-174.

Magno, F. and Serafica, R. (2001). Information Technology for Good Governance. De La Salle

University. [Online]. Available from: http://unpan1.un.org/intradoc/groups/public/documents/apcity/
unpan002708.pdf. [Accessed on 3 March 2017].

McKinnon, R. (2011). Good Governance in Social Security Administration. International Social

Security Review, 64(4), pp.3-8.

Mulgan, G. (2014). Innovation in the Public Sector – How can public organisations better create,
improve and adapt? [Online]. Nesta, version 1. Available from: http://www.nesta.org.uk/sites/default/
files/innovation_in_the_public_sector-how_can_public_organisations_better_create_improve_and

adapt.pdf. [Accessed 7 March 2017].

Muriithi, K., Jimenez, M., Jannin, N., Sajid, N., Singh, S. and Sharma, S. (2015). Quantifying

Governance: An indicator-based approach. The London School of Economics and Department for

International Development Capstone Team. [Online]. Available from: https://assets.publishing.service.
gov.uk/media/57a08971ed915d622c000209/61506_Capstone_Report_DFID_FINAL_Quantifying_

Governance__Indicators.pdf. [Accessed 21 February 2017].

OECD. (2007). Towards Better Measurement of Government. [Online]. OECD Working Papers on

Public Governance, 2007(1). OECD Publishing. Available from: http://www.oecd.org/governance/
digital-government/38134037.pdf. [Accessed 10 March 2017].

OECD. (2012). Recommendation of the Council on Regulatory Policy and Governance. The OECD

Regulatory Policy Committee. [Online]. Available from: http://www.oecd.org/governance/regulatory-
policy/49990817.pdf. [Accessed 9 March 2017].

OECD. (2014). Recommendations of the Council on the Governance of Critical Risks. Meeting of The

OECD Council at Ministerial Level. Paris 6-7 May 2014. [Online]. Available from: http://www.oecd.org/
gov/risk/Critical-Risks-Recommendation.pdf. [Accessed on 2 March 2017].

OECD. (2015a). Recommendation of the council on Budgetary Governance. Public Governance &
Territorial Development Directorate. [Online]. Available from: http://www.oecd.org/gov/budgeting/
Recommendation-of-the-Council-on-Budgetary-Governance.pdf. [Accessed on 3 March 2017].
The International Civil Service Effectiveness Index | Technical Report 95

OECD. (2015b). Inclusive government for a more inclusive society. Government at a Glance 2015,
OECD Publishing, Paris. [Online]. Available from: http://dx.doi.org/10.1787/gov_glance-2015-6-en.
[Accessed on 21 February 2017].

OECD. (2015c). The Innovation Imperative in the Public Sector: Setting an Agenda for Action, OECD

Publishing, Paris. [Online]. Available from: http://dx.doi.org/10.1787/9789264236561-en.
[Accessed 7 March 2017].

OECD. (2015d). Public Sector Leadership for Improved Employee Engagement and Organisational

Success. Lead-Engage Perform. OECD expert meeting, 21-22 January 2015. [Online]. Available

from: https://www.oecd.org/gov/pem/agenda.pdf. [Accessed on 3 March 2017].

OECD. (n.d.). Building more effective, accountable, and inclusive institutions for all. [Online].
Post 2015 Millennium Development Goals: Element 6, Paper 1. Available from: https://www.
oecd.org/dac/_POST-2015%20effective%20and%20accountable%20institutions.pdf. [Accessed

1 March 2017].

Office for Public Management Ltd (OPM) and The Chartered Institute of Public Finance and

Accountability (CIPFA). (2004). The Good Governance Standard for Public Services. The Independent

Commission on Good Governance in Public Services. Hackney Press Ltd.

Phillips, W., Caldwell, N. and Callender, G. (2007). Public Procurement – a pillar of good

governance? [Online]. Available from: https://www.researchgate.net/publication/272487779_Public_

procurement_A_pillar_of_good_governance. [Accessed on 22 February 2017].

The Committee on Standards in Public Life. (1996). Summary of the Nolan Committee’s First Report

on Standards in Public Life. [Online]. The National Archives. Available from: http://webarchive.
nationalarchives.gov.uk/20140131031506/http:/www.archive.official-documents.co.uk/document/
parlment/nolan/nolan.htm. [Accessed 17 March 2017].

The World Bank. (2012). Indicators of the Strength of Public Management Systems: A key

part of the Public Sector Management results story. PRMPS Discussion Paper: Draft. [Online].
Available from:http://siteresources.worldbank.org/PUBLICSECTORANDGOVERNANCE/
Resources/285741-1354024300711/ISPMS.pdf. [Accessed 21 February 2017].

The World Bank. (2016). Benchmarking Public Procurement 2016. [Online]. Available from:
http://bpp.worldbank.org/~/media/WBG/BPP/Documents/Reports/Benchmarking-Public-
Procurement-2016.pdf. [Accessed 22 February 2017].

The World Bank (n.d.). [Online]. Available from: http://www1.worldbank.org/publicsector/anticorrupt/
corruptn/cor02.htm. [Accessed 1 March 2017].

United Nations. (1945.) Charter of the United Nations. Article 101. [Online]. Available from: http://
www.un.org/en/sections/un-charter/un-charter-full-text/. [Accessed 15th February 2017].

United Nations Public Administration. (1999). Transparency in Government. Presentation for ILEA

Seminar. [Online]. Available from: http://unpan1.un.org/intradoc/groups/public/documents/un/
unpan012062.pdf. [Accessed 20 February 2017].

Van Dooren, W., Manning, N., Malinska, J., Kraan, D. J., Sterck, M. and Bouckaert, G. (2006). OECD

project on Management in Government: Comparative Country Data. Issues in Output Measurement

for “Government at a Glance”. OECD, Technical paper 2 (Second draft).

World Trade Organisation. (2015). Government Procurement Agreement: opening markets and

promoting good governance. [Online]. Available from: https://www.wto.org/english/thewto_e/20y_e/
gpa_brochure2015_e.pdf. [Accessed 1 March 2017].
96 The International Civil Service Effectiveness Index | Technical Report

Annex C – Reader’s Guide

C.1 Abbreviations

InCiSE International Civil Service Effectiveness

SNA System of National accounts

OPM Office for Public Management

CIPFA Chartered Institute of Public Finance and Accountability

ISPMS Indicators of the Strength of Public Management Systems

RIA Regulatory Impact Assessment

OECD Organisation for Economic Co-operation and Development

IMF International Monetary Fund

UN United Nations

VFM Value for money

GCI Global Competitiveness Index

WEF World Economic Forum

iREG Indicators of Regulatory Policy and Governance

QoG Quality of Government

SGI Sustainable Governance Indicators

EPI E-participation index

ODI Open Data Index

ODB Open Data Barometer

PIAAC Programme for the International Assessment of Adult Competencies

GaaG Government at a Glance

MI Multiple imputation

MICE Multivariate imputation using chained equations

OURdata Open, Useful, Reusable data
The International Civil Service Effectiveness Index | Technical Report 97

C.2 Country codes

The following ISO country codes are used in some tables of this report57:

Country ISO alpha – 3 digit Country ISO alpha – 3 digit
country code country code

Australia AUS Korea, Republic of KOR

Austria AUT Mexico MEX

Belgium BEL Netherlands NLD

Canada CAN New Zealand NZL

Chile CHL Norway NOR

Czech Republic CZE Poland POL

Denmark DNK Portugal PRT

Estonia EST Slovakia SVK

Finland FIN Slovenia SVN

France FRA Spain ESP

Germany DEU Sweden SWE

Greece GRC Switzerland CHE

Hungary HUN Turkey TUR

Ireland IRL United Kingdom GBR

Italy ITA United States of America USA

Japan JPN

57 https://www.iso.org/obp/ui/#home
98 The International Civil Service Effectiveness Index | Technical Report

This text has been redacted: Date redaction, Date redaction