Please see attached.
Download (7.43 MB)Department of the Prime Minister and Cabinet
PO Box 6500
CANBERRA ACT 2600
Dear Chair and esteemed Panel
INDEPENDENT REVIEW OF THE APS
It is with great pleasure that I formally lodge my submission in response to the
opportunity to contribute to the Independent Review of the APS (the Review).
I have always been passionate about the Australian Public Service (APS) and both
how and what it delivers for all Australians, as well as the wider domestic and global
community.
Both my parents have had distinguished careers within and supporting the public
service and I am proud of their service to an entity that is the foundation of good
government and ultimately a better standard of living and future for country.
My private sector background has seen me work for some of the largest and
smallest organisations across a number of industries. A long history supporting the
APS in this capacity ultimately saw me move to work within the public sector with a
drive to make a difference. I have long been driven by the want and need to make
difference to how and what we deliver to see the APS position itself to support our
country’s leaders build a better future for citizens today and in the decades to come.
I would like to once again express my sincere thanks for being afforded the
opportunity to contribute to your independent review of the APS.
I look forward to seeing the review’s findings and to contributing to the holistic and
structural reform and change required to see the APS proactively transformed to be
fit-for-purpose for the future ahead.
Yours sincerely
5 (July 2018
Preamble
The APS has long been recognised for its expertise, flexibility and capacity to adapt
and respond to change. However the rate of change within its operating environment
is unprecedented - driven by digital transformation, and significant geo-political,
economic and demographic shifts.
This response considers how these shifts will create new challenges and
opportunities for the APS. No organisation can afford to be reactive and in order to
remain relevant and continue to add value to citizens and our community, it is
essential the purpose, role, contribution and approach all be reviewed with the
purpose of ensuring we are positioned to deliver valued services in a future context.
Further to this, it is imperative that any improvements see the APS positioned to flex
and adjust to the significant, fast-paced and ongoing change to voters and
taxpayers, staff, suppliers, clients, policy makers and partners are experiencing.
They must also be cognisant of the tight fiscal environment in which the service
operates and the continuing expectation by taxpayers that monies are used
efficiently, effectively and ultimately to the betterment of all Australians.
This response provides insights across the key topic areas the Review panel will be
exploring (as per the Terms of Reference), to provide insights into some areas which
may not have previously been identified, or from this perspective.
I appreciate the complexity and significant challenge posed to the Review panel
within the tight timeframes and I wish you all the best in your endeavours to analyse
and present practical recommendations to ensure the APS is ready, over the coming
decades, to best serve Australia.
Attachments:
Please see the attached documents which are referred to within this response and
which I believe will add to your greater understanding of the key issues with the
evidence to support.
- The Mandarin - Tom Burton: Canberra Clones itself
- The International Civil Service Effectiveness (InCiSE) Index 2017
- The International Civil Service Effectiveness (InCiSE) Index - Technical
Report - The International Civil Service Effectiveness (InCiSE) Index - Country
Rankings
Key Observations:
Recruitment
• Pathway programs and partnerships with high schools, trade schools and
universities
• Secondments with professional and industry partners - especially in
alignment with portfolio aims
• Recruitment that utilises the best approach to capturing the right person
based on the profession and industry (specialist role based recruiters,
advertising and processes.)
• Recruitment processes that reflect advances in assessing staff for jobs based
on the roles (eg. skills based testing, alignment with private sector
approaches to ensure we do not miss out on a large section of those who are
intimidated by the APS process, behavioural testing to see if the mental
pedigree also aligns with the role and service).
• Promoting the real benefits of a career in the APS that target the aspects that
attract the role types.
• Considering numerous pathway to entry based on the role by creating
streams (to include a new management stream as a technical skill in itself)
• As we move to a more technical based and flexible based environment, look
to attract those in all parts of Australia through this approach that does not
miss out on a large section of those who are would be disinclined to apply due
to a primarily Canberra based requirement)
Assessing and investing in talent
• Opportunities to invest in staff once they find their calling - moving away from
disqualifying attributes roles which may prevent real talent from coming
through (eg. degrees or experiences unless these are core to the role).
• Creating technical line with opportunities for those who want to progress in
their career who don’t have the passion or attributes to lead people, (more
than just remuneration)
• Creating a technical line for leadership so that this stream reflects the
attributes of what we want to see in our leaders of today and tomorrow as
they shape the future.
Driving merit
• Bring greater accountability to all roles of the APS that require KPIs that are
aligned with better practice, strategy, culture, equivalent private sector
counterparts and policy outcomes, (despite departments and agencies not
being challenged by a bottom line, look to ways to really assess outcomes).
• Moving away from detailed old type recruitment processes that were originally
created to prevent cronyism and nepotism, however have led to this being the
primary attributes into today’s APS “merit” process.
• Improving the performance management approach with APS employees and
ensuring that it allows leaders and managers the ability to deal with
underperformance rather than driving a culture of failing to invest in staff and
rather transferring or promoting underperformers so as not to have to deal
with a 12 month process that might ultimately not lead to a resolution.
• Provider leaders and managers the tools and training to manage staff and
underperformance and high performers (this is no just e-modules or one day
face to face training).
• Creating mentoring and buddy programs to drive more cost effective staff
development and collaboration.
• Incentivise high performers who are disincentivised by the “treat everyone the
same” principles and managers and leaders not wanting to look like they are
playing favourites.
Equity, equality and diversity
• Move away from affirmative action and look at ways to drive merit based
diversity
• Although gender diversity is finally being managed, it is insulting to think that
affirmative action is needed to bring gender diversity when there are plenty of
intelligent, qualified and suitable women, (this requires significant changes in
the way we recruit and how we capture talent).
• Gender diversity through affirmative action is breeding a bad culture where
merit has been lost given it alienates other minorities who are far more
disadvantaged (eg. non-Anglo-Saxons make up the majority of the APS
senior leadership even the current independent review panel). It is also
alienating those in younger generations who were always pro-diversity who
now feel vilified and are penalising women who had no intention of being
given this greater advantage. Also indicators like university graduation by
gender and also most departments seeing a majority of women in levels
below SES seem to fly in the face of the approach being taken under the
banner of equality and what should also include equity. This is an issue of an
older generation who have made it the “boys club” and this should not be the
group that the rules are written against for everyone else.
• Investing in real diversity where merit is the primary driver. Please see
attached article by Tom Burton: Canberra clones itself from the Mandarin that
shows that the APS’s leadership is incestuous and fails to allow for alternative
ways of thinking. For policies, strategy and culture and issues to be
considered from all sides, one must have various perspectives. (Anglo-Saxon
males and females aged 45+ who all went to a top 8 university and are
primarily Canberra based bureaucratic lifers does not achieve this end).
Many SES have advised the fact that the SES bands are a club, if you don’t
have a “advocate” and don’t look, sound and think the same, you’re unlikely to
ever make the ranks, regardless of your performance.
Political vs apolitical
• The APS and the Review ToR both indicate that being apolitical is a key value
of the APS now and expected to continue as a key value into the future.
Despite this, it is evident using the most basic of indicators, that with a change
in government and the resulting changing of the guard at the Secretary level,
(the top level which ultimately shape all aspects of the APS) that the APS is
fundamentally very political. Although one can appreciate that within the
private sector senior leadership are hired or fired for a particular capability,
strategy or track record to deliver on an aligned strategy that the Board wants
to pursue, this is not ordinarily based on one’s political affiliation.
• Cuts in FTE eg. ASL caps cuts which has seen a marked increase in total
bodies (which includes contractors and consultants) is purely a politically
driven agenda. It is worrying to see that despite aggregate costs increasing
with no corresponding value proposition, this approach is applied across the
APS. There should be mechanisms in place to insulate department from
being impacted by political agendas, but being accountable to align with
accountability principles that reflect quantitative indicators of efficiency and
effectiveness.
Risk taking and frank and fearless.
• The APS has seen itself degraded in its ability to remain apolitical in giving
sound, evidenced based advice. In days gone by, the role was to be the
apolitical guardians of good evidenced based policy based on robust
discussions with the political apparatus and the key stakeholders with
considerations for benefit realisation. It is well known now that the APS has
moved away from providing frank and fearless advice and moved to a “yes
minister” type approach, based on reactive, policy implementation on the run,
far more focused on the process, political agendas and reputation rather than
actually delivering the best outcomes for Australians, men and women" who
again realise an outcome where the way one person thinks is the way all
problems are considered and resolved, meaning you can never considers all
the risks and opportunities that exist.
• The lack of understanding or acumen in true risk management that is focused
on all enterprise risks both financial and non-financial (especially culture and
conduct as seen in the royal commission into the banks), is significantly
lacking the public service. When SES members are more focused on
reputation and political agendas than taking risks in alignment with the
rewards proposed to them, this will always prevent the APS from taking the
next step forward. They also then breed a culture of “yes men and women”
who no longer looks to exploit opportunity but rather become indoctrinated in
risk averse thinking. Further to this, failure to manage real risks by focusing
on issues leads to a very reactive APS rather than understanding that there
can be proactive approaches that will assist with mitigating issues and risks.
Lessons learned
• The APS continues to show that it doesn’t learn from the past. Lessons
learned is one of the cheapest ways to innovate and improve and yet, in most
departments we undertake the same approach to doing things and continue to
engage suppliers to undertake the same reviews. If we cannot learn from our
mistakes, any chance or learning anything new is close to impossible.
Revising approach to procurement that drives real value
• The APS continues to identify value for money as the key approach to
procurement and contract management, however in most cases this boils
down to cheapest price, cronyism/nepotism. Although there is should be no
problem selecting a supplier that you may know, or selecting a supplier is cost
effective, there should be better ways from upskilling and incentivising our
staff and improving our assessment process from picking the cheapest or
failing to provide evidence as to why a particular supplier should be used.
Fiscal management
• In the international civil service effectiveness index, the APS is scored poorly
for fiscal management. The lack of real fiscal acumen among its SES with
primarily theoretical university driven knowledge which is out of date after a
decade is reflected. Key issues of budget management are so poor, that it is
a common fact that unused budget at April should be spent to ensure same or
greater appropriations in the future. In a time of austerity, this is unacceptable
when no benefit realisation is required to ensure that spends can be assessed
for return on investment or other relevant indictors such as alignment with
program or policy delivery objectives. It would be good to consider alternative
budget approaches including incentivising effective fiscal management (this
does not mean slash and burn) but rather not incentivising this current
approach.
The old management principle of more with less as we know has been thrown
out the door and is now less with less. This however should be considered
from more than one angle and should be the avenue where agency heads
request short term funding to enact efficiency and capability building with an
expectation to show full benefit realisation. This could be improvements via
R&D, innovation, automation, process improvement, outsourcing, better
procurement and contract management or improving capability of workforce.
Productivity indicators should be applied to all departments based on profile.
This should consider program size and complexity vs funding and outcomes.
The lack of standardised KPIs is leading to ambiguous outcomes that are
leaving tax payers questioning more and more what happens with their
money. This should be more important than focusing on reputation.
APS leaders and staff continue to believe that we can outsource our risks by
hiring consultants. As an ex-consultant I am well aware of the planning
sessions that go on at consulting houses, where they laugh at how easy it is
to convince APS leaders to engage them to undertake the same work over
and over again utilising junior staff which tax payers ultimately pay to train.
APS staff and leaders who are so risk averse, lack accountability for how they
spend on consultancies and who do not have the capacity or capability
continuously hire consultants for exorbitant rates regardless of need or
outcome.
Tom Burton: Canberra clones itself | The Mandarin - The Mandarin https://www.themandarin.com.au/83422-tom-burton-canberra-clones/
Tom Burton: Canberra Author Bio
clones itself
By Tom Burton (/author/tburton) • 08/09/2017
DNA replicates and so this week did Canberra’s
mandarin class. Half the existing federal
departments will get new secretaries in a major
shuffle of the bureaucratic deck, including two
new appointments. One of those, new
Inf rastructure and Regional Development boss
Steven Kennedy, has rightly had “secretary”
stamped on his personal f ile for several years.
He is a real loss to PM&C where he has led
much of that department’s new work around
cities.
But an eyeball of the bios of the 18 portfolio
secretaries, reveals a sameness, that says much
about the 155,000 strong Australian Public
Service.
Without exception, the secretaries are
bureaucratic lifers, almost all having spent their
career in Canberra, promoted through the
senior executive service as high performance
individuals. No plumbers or small business
owners, nor big corporate, industrial, or NGO
experience to be seen.
Treasury boss, John Fraser, worked in Treasury
before spending his mid career at investment
bank, UBS. Education chief, Michele Bruniges,
began life as a teacher rising through the NSW
and Canberra education bureaucracy. And the
journos remember new Employment secretary,
Kerri Hartland, as a Toowoomba rural journo and
then in the Canberra press gallery for four years
with various News Corp publications. Finance
secretary, Rosemary Huxtable, moved west
when then Prime Minister, John Howard,
downsized the bureaucracy in 1996. And
Kennedy started life as a nurse.
But these dalliances aside, every secretary is
very much a product of the Canberra system,
the Commonwealth government aided and
abetted by its local breeding ground, the ACT
government. All have worked in plenty of gigs
across the wide diaspora of domestic and
international functions the federal government
spans, but the complete lack of outside blood is
startling.
Compare this with NSW, where over a third of
the top bureaucracy is now f rom the
non-government world, a f ive year change
program that has injected much needed new
thinking and experience into that government.
All are white bread. I say that with no personal
disrespect, but to make the point it passes
strange that at the elite level of the Australian
government there is not an Asian, Af rican,
1 of 5 REDACTED 9:15 PM
Tom Burton: Canberra clones itself | The Mandarin - The Mandarin https://www.themandarin.com.au/83422-tom-burton-canberra-clones/
South American, North American to be seen.
Nor an indigenous person. This suggests a
cultural narrowness that common sense says is
not healthy, if you want to be kind. Less kindly,
would be to describe it as a tightly
self-controlled, in-bred world, cocooned f rom
the broader realities of modern diverse
Australia. This is equally true of the political
class (us* in the press gallery included) that
works in and around Capitol Hill. Anglo-Saxon
culture is alive and well with our ruling
Canberra elite.
But an educated elite
All the secretaries are products of the
post-Whitlam higher education system. All have
been products of the top-tier G8 university
system, liberal arts alumni mixed with post
grad law and economics qualif ications.
Economics is by far the dominant pedigree,
more precisely neo classical economics, taught
through conventional post-Keynesian
institutions.
The leader of this pack is the Australian
National University, which has long acted as
f inishing school for the top end of the APS.
No bomb throwers at that university. Its
impressive schools of strategy, economics and
public policy score well on almost any measure.
But there is a sameness and comfortable
acquiescence with the core mission of the big
APS agencies, that feels unhealthy when
observed f rom the grandstand. The ANU is the
only university funded directly f rom the
Commonwealth government. In addition ANU
through its iron grip on its APS alumni, wins a
lot of federal money in the form of research
grants, executive programs, and numerous
pilots and collaborative institutions. This means
there is no incentive to bite the hand that
feeds. Indeed the opposite.
And it shows. At times this borders on self
congratulation and a dangerous master-of-the
universe attitude, that might be warranted if
the APS was an exemplar for modern
government, but delusional if it isn’t. Sit in on
ANU Chancellor Gareth Evan’s annual Crawford
School policy confab and you will hear a lot of
intelligent people ruminating about “Vatican
City” type issues, that feels very distant f rom
the ferment that is deeply disrupting the real
world.
There are no dumb-dumbs in this latest
secretarial cohort, with lots of f irst-class
honours, masters degrees and PhDs, but other
than new Social Services boss Kathryn
Campbell, I could not f ind any technical,
engineering, marketing, f inancial or science
credentials. If the next 20 years of the digital
revolution is as disruptive as the f irst 20 then
this again reveals a dangerous narrowness —
and lack of capability — for the group charged
with designing policy, regulation and reform for
this era.
Apologies to Game of Thrones fans, but with
public administration ripe for automation and
an inevitable transformation into an Uber-like
mass data algorithm, the APS feels like the
citizens of Westeros waiting for the white
walkers to come.
In the meantime the work practices have barely
changed since the large typing pools of the
’80s gave way to PCs and local networks.
Stifling and deeply bureaucratic, there is little
incentive for agencies to radically reinvent their
work places and practices for the collaborative
fast moving world typif ied in many newer
businesses and NGO’s.
2 of 5 REDACTED 9:15 PM
The International
Civil Service
Effectiveness
(InCiSE) Index
2017
Users are free to copy, download and print InCiSE content and findings for their own use. Excerpts
from InCiSE reports and publications may also be used, but must be appropriately referenced.
Please cite any uses of this content as: “International Civil Service Effectiveness (InCiSE) Index, 2017”.
The Index is comprised of a variety of other data sources. Reuse is not applicable to data subject to
intellectual property rights of third parties. Please refer to other organisations’ corresponding websites
and data licensing restrictions to ensure compliance with their data limitations. Details of data sources
can be found in the InCiSE Technical Report. Every effort has been made to adhere to third party
data regulations.
Please contact incise@instituteforgovernment.org.uk for any queries.
Acknowledgements: Our thanks go to those who have given their time to shape this inaugural pilot
publication. This has included substantial contributions made by academics, think-tanks, international
organisations (particularly the OECD), and civil servants past and present. Our thanks also to those
organisations who have allowed use of their data in the Index.
The International Civil Service Effectiveness Index 1
Contents
Foreword from InCiSE founding organisations 3
Executive summary 4
Chapter 1: Introduction and background 7
1.1 Why we need civil service effectiveness indicators 7
1.2 How InCiSE has been developed 8
1.3 What InCiSE is not aiming to do 8
1.4 Who InCiSE is for 8
Chapter 2: The InCiSE framework and approach 9
2.1 Defining the InCiSE framework 9
2.2 Measuring core functions 9
2.3 Measuring attributes 9
2.4 Indicator definitions 11
2.5 Data availability and limitations 12
2.6 Data sources 13
2.7 Country coverage 13
2.8 Country metrics issues 13
Chapter 3: InCiSE index results 15
3.1 Overview 15
3.2 InCiSE overall scores and rankings 17
3.3 Top 5 country rankings by indicator 17
Chapter 4: InCiSE index country results summary 23
Chapter 5: Next steps and conclusion 55
5.1 Next steps 55
5.2 Conclusion 56
2 The International Civil Service Effectiveness Index
Appendices 57
Annex A: InCiSE indicator definitions 58
Core functions 58
Attributes 58
Annex B: InCiSE indicator data sources 59
Annex C: InCiSE country rankings data 62
Annex D: InCiSE indicator scores summary by country 63
Annex E: Summary graphs of country scores and rankings for each indicator 64
Core functions 64
Attributes 72
Annex F: Reader’s guide 76
F.1 Abbreviations 76
F.2 Country codes 77
The International Civil Service Effectiveness Index 3
Foreword from InCiSE
founding organisations
We are pleased to come together to launch the InCiSE is not claiming at this stage to be
International Civil Service Effectiveness (InCiSE) a comprehensive measure of civil service
Index – an exciting new initiative to help countries performance. Some countries and data are
determine how their central civil services are missing which prevent the Index from being as
performing and learn from each other. robust and comparative as we would wish. We
hope partners will help us strengthen and refine
An effective civil service plays an important this inaugural Index over time. We will encourage
role in driving forward a country’s progress and further data collection in areas where it is lacking
prosperity. An ineffective one can act as a brake and will actively support these efforts.
on these things. Yet it is not a straightforward
task to assess whether a civil service is Our long term goal is to broaden the scope of
performing well or how it might improve. InCiSE and establish a robust Index which can
be produced year on year, as well as expand
In an attempt to address this, InCiSE provides its country coverage. We have committed to
a realistic set of comparative information supporting its development for a further four
drawn together from the wealth of existing years.
data available globally. The first inCiSE results,
covering 31 countries, are set out in this report. InCiSE has already brought together a significant
volume of data and insights. We hope the
InCiSE is primarily a performance improvement launch of this report will stimulate wide-ranging
tool, enabling senior decision makers to see discussions globally about how civil services can
which countries perform best in which areas and improve their effectiveness, as well as how to
learn from them. We hope it will also serve as an improve the usefulness of the InCiSE data. We
accountability tool, allowing citizens, government would welcome feedback; contact details can be
officials and politicians to establish in a concise found at the front of this report.
way how well their civil service is functioning.
InCiSE has been developed following a literature
review and in consultation with many experts.
It has also been the subject of an independent
peer review which scrutinised the methodology
without knowing the individual country results.
We are grateful to all those who have given
their time to shape our approach and helped
to produce this inaugural report. We are also
grateful to the many organisations who have
made the Index possible by allowing us to use The International Civil Service Effectiveness
their data. (InCiSE) Index project is a collaboration between
the Blavatnik School of Government and the
Institute for Government. The project has been
supported by the UK Civil Service and is funded
by the Open Society Foundations.
4 The International Civil Service Effectiveness Index
Executive summary
Why and how InCiSE has InCiSE has been the subject of extensive
development and consultation. This has included a
been developed
literature review and discussions with many experts
The International Civil Service Effectiveness and international organisations. InCiSE has also
(InCiSE) Index has been created to help determine been the subject of an independent, international
whether civil services globally are performing peer review process (during which country
effectively and in which areas, relative to their results were provided in an unnamed format).
international counterparts. This matters because
civil services have an important role to play in Scope of the InCiSE index
helping their countries to prosper.
InCiSE is focused on the central government civil
Previous initiatives have sought to develop service only in the countries covered. It does not
measurement tools of this kind but there is seek to measure service delivery outcomes for
currently no other global index available which citizens, for example healthcare and education,
provides a comprehensive assessment of civil because effectiveness is often driven by other
service effectiveness. There are, however, many parts of the public sector as well.
existing data surveys and indexes available
globally that could be pulled together to provide InCiSE is not claiming at this stage to be a
a realistic set of information on an annual basis. robust, comparative measure of civil service
This is what InCiSE aims to do. performance, mainly because of the limited
availability of some data. This inaugural Index
By providing a set of indicators (each measuring should therefore be seen as a pilot product
a different dimension of civil service effectiveness) which will be refined and improved over time.
and then an overall score and ranking for each
country, InCiSE can serve as:
• a performance improvement tool to enable
civil service decision makers to find out which
countries perform best in which areas and
learn from them.
• an accountability tool which allows citizens,
government officials and politicians to find
out how well their civil service is performing.
The International Civil Service Effectiveness Index 5
How the InCiSE framework has Only two countries currently have available data
that covers the full set of metrics – Norway
been compiled
and UK. However, several have close to the
A detailed explanation of the InCiSE framework full complement and any missing data has
is set out in a separate Technical Report. Its been estimated using standard methods.
starting point is to define the core characteristics Countries with the highest missing data points
of an effective, central government civil service. inevitably have a larger proportion of estimated
To do this, InCiSE assesses effectiveness based metrics and this should be borne in mind when
on two interrelated components: interpreting results.
• Core functions: these are the core things
that civil services deliver in each country
Pilot Index results
(‘what’). There are 11 core functions, detailed Canada is ranked top overall of this inaugural
in Annex A. Index, followed by New Zealand and Australia
respectively. When scores are adjusted for GDP
• Attributes: these are the main characteristics
per capita (to take account of countries’ relative
across every part of a civil service which are
wealth and thus the potential resources available
important drivers of the ways in which core
for civil service operations) then Estonia is ranked
functions are delivered (‘how’). There are 6
top overall, followed by Mexico and New Zealand
attributes, detailed in Annex A.
respectively.
Most, but not all, of the InCiSE indicators have
No country consistently appears in the top
been measured to produce this inaugural Index.
5 positions for every indicator, although there
This is largely because of gaps in existing
are some strong all-round performers and
data. So far 8 out of the 11 core functions
these are highlighted in the individual country
proposed have been measured, and 4 out of
assessments. There are some stand-out
the 5 attributes. The quality of data also varies
scores for specific indicators which have been
from country to country. Nevertheless, the initial
highlighted as well.
results already provide some useful insights in the
countries that the InCiSE Index currently covers. When analysing results, it is important to keep in
mind that all country scores are relative to others
Many of the datasets which InCiSE draws on are
included in the Index, not an absolute measure.
updated annually. This should enable the InCiSE
A country which scores well against a particular
Index to be revised on a regular basis to reflect
indicator or theme may still have performance
recent country developments.
issues. The Index provides an important
opportunity to learn from other countries.
Country coverage
Some interesting trends can be observed
To achieve its goals and maximise impact, already. For example, some countries generally
InCiSE aims to cover a diverse and growing score more strongly against attribute indicators
range of countries over time. The speed of rather than core functional ones, and vice versa.
expansion will largely be determined by the Nevertheless, of the top 10 countries performing
availability of data sets and collections which well against core functions, 8 also feature in the
currently vary considerably. This pilot Index top 10 for attributes.
therefore comprises 31 countries where at least
75% of the data needed is available.
6 The International Civil Service Effectiveness Index
Next steps Data collection plans by other organisations
may help to fill some of the data gaps in future,
The founding organisations have committed while in other areas additional data collection
to supporting the development of the InCiSE may be needed. The project will strive to close
framework and Index for a further four these gaps as InCiSE evolves, as well as actively
years if this inaugural Index is well received. support others’ efforts.
An International Advisory Panel will also be
established to guide this work. One of the Changes to the InCiSE framework will be
founders, the Blavatnik School of Government determined mainly through feedback from
(at Oxford University in the UK), will host an a wide range of interested partners. The
international conference in September 2017 new International Advisory Panel will also
to discuss the pilot Index results and future be consulted. InCiSE contact details can be
direction of the project. found at the front of this document to provide
feedback.
Priority tasks over the next 12 months to improve
InCiSE will include: strengthening data collection
and filling gaps; refining the InCiSE methodology
and framework; expanding the country coverage;
examining the potential to include non-OECD
and developing countries over time; developing
an interactive website; and increasing the InCiSE
partner institutions and network through advice,
expertise and funding.
The International Civil Service Effectiveness Index 7
Chapter 1: Introduction
and background
1.1 Why we need civil service The creation of a new and concise set of
indicators would serve as:
effectiveness indicators
• An accountability tool: allowing citizens,
An effective civil service can play an
government officials and politicians to
important role in determining a country’s
establish in a clear and concise way how
progress and prosperity. But what
well their civil service is performing.
constitutes an “effective civil service” in
the 21st Century? And once a consensus • A performance improvement tool: enabling
has been reached on defining this, how senior decision makers to see which
do civil service leaders know whether their countries perform best in which areas,
organisations are effective, and in which and learn from them.
areas are they performing more strongly
than others? The need to develop a new and more
comprehensive set of indicators has grown
The InCiSE Index seeks to help answer following the launch of the new United Nations
these questions. Although a comprehensive (UN) Sustainable Development Goals. InCiSE
set of international indicators of civil service has the potential to help countries report on
effectiveness does not currently exist, various progress towards some aspects of Goal 16
organisations have sought to develop reliable which focuses on a range of governance issues,
measurement systems. including promoting stronger government
institutions. The World Bank’s 2017 World
This subject area is well recognised in
Development Report on Governance and the
academic, international and practitioner
Law also highlights the need for a greater focus
communities as a highly complex area
on improving critical government functions.
for analysis. This is partly because of
data limitations and different views on The InCiSE project is a collaboration between
the definitions of “civil service” and the Blavatnik School of Government and the
“effectiveness”, as well as the need to take Institute for Government. It has been supported
account of country context factors when by the UK Civil Service and is funded by the
looking at performance. Nevertheless, Open Society Foundations. All have a strong
there are many existing data surveys and interest in civil service performance issues. The
indexes available globally that could be long term goal is to broaden the scope of InCiSE
pulled together to provide a realistic set of and establish a sound Index which is capable of
information on an annual basis. This is what being produced year on year, as well as expand
the InCiSE framework aims to do. the country coverage.
8 The International Civil Service Effectiveness Index
1.2 How InCiSE has been developed 1.3 What InCiSE is not aiming to do
Although the InCiSE framework covers many Given the limited availability of some data and
aspects of civil service performance which complexity of the subject area, InCiSE is not
earlier initiatives of this type have sought to claiming at this stage to be a robust, comparative
cover, it does not duplicate them. InCiSE aims measure of civil service performance. It is
to define “effectiveness” more extensively than therefore important to view the initial InCiSE
previously. It draws on a wide range of existing framework and Index as pilot products only, to
international data sources and brings together be refined and improved over time. It is hoped
a set of indicators – each measuring a different that the launch of InCiSE will encourage further
dimension of civil service effectiveness – and data collection in areas where it is lacking and the
then produces an overall score. project will actively support these efforts.
Whilst there are alternative ways to define civil It is also important to be clear about the scope
service effectiveness, the InCiSE framework, with of InCiSE:
its various themes and measurements, has the
potential to make valid judgments about whether • InCiSE is focused on the central government
a country’s civil service is performing well relative civil service in the countries covered – not the
to its international counterparts. public service more generally. It is recognised
that this is not without challenges, particularly
InCiSE has been developed following a given the varying sizes and shapes of public
literature review and in consultation with many administrations internationally, as well as
experts, including academics from schools of different systems of government. It is also
government, think-tanks that monitor government recognised that well performing civil servants
effectiveness, international organisations, are a necessary but not sufficient condition
senior civil servants (past and present) from for better government.
several countries, as well as subject experts.
• InCiSE is not seeking to measure service
InCiSE has also been the subject of an delivery outcomes for citizens, for example in
independent, international peer review process. areas like healthcare and education. Although
Three peer reviewers were selected: a senior in many countries the central government civil
academic from a major European governance service plays an important role in determining
research institute; a recently retired top civil the performance of key public services,
servant with practical experience of civil service their effectiveness is driven by other parts of
performance issues; and a senior governance the sector as well – teachers, doctors, the
expert in two major international institutions. police etc. Local government effectiveness is
Country results were provided in an anonymous also excluded.
form for the review. Between them, the
peer reviewers were asked to examine the • InCiSE does not aim to be definitive. It is one
measurement framework of ‘effectiveness’, of a wide range of tools currently available to
the methodology and approach used to produce measure civil service effectiveness globally.
the indicators, and the data being used.
1.4 Who InCiSE is for
InCiSE is expected to be of interest to a wide
audience – governments (civil servants and
ministers), Parliamentarians, think-tanks,
academics, civil society organisations, the media
– and of course citizens.
The International Civil Service Effectiveness Index 9
Chapter 2: The InCiSE
framework and approach
This chapter sets out the overall approach taken • Service delivery functions where central
to develop the InCiSE framework and produce the government civil services interact more
pilot Index, as well as country coverage issues. directly with citizens (eg. tax and social
Further details about the choice of indicators, their security administration, digital services).
definitions, data availability and quality issues, can
be found in a separate Technical Report. • Mission support functions which enable a
civil service to do its job (eg. finance, human
resource management (HRM), information
2.1 Defining the InCiSE framework technology (IT), procurement).
The InCiSE framework starts by defining the core
By looking across at all three types of function,
characteristics of an effective, central government
the aim is to measure how well civil services
civil service. To do this, it assesses effectiveness
deliver the core parts of what they do. Figure 1
based on two interrelated components:
shows the eleven core functions included in the
• Core functions: the core things that civil InCiSE framework.
services deliver in each country (‘what’).
2.3 Measuring attributes
• Attributes: the characteristics across every
part of a civil service which are important Every civil service has an underlying set of
drivers of the ways in core functions are behavioural characteristics or traits which
delivered (‘how’). are important drivers of how effectively core
functions are delivered, for example levels of
2.2 Measuring core functions openness, integrity and inclusiveness. These
attributes should apply to all parts of the civil
Civil services across the world vary widely in their service and should not be limited to specific
shape and size and are responsible for different core functions.
things. However, there are certain core functions
which they all deliver for the governments Cultivating and displaying these attributes is
and citizens that they serve. The approach commonly (though not necessarily universally)
InCiSE takes is to focus on three interrelated understood to be good practice. They are also
types of core function to measure civil service key determinants of an organisation’s overall
effectiveness more comprehensively: effectiveness. Figure 2 shows the six attributes
included in the InCiSE framework.
• Central executive functions for ministers,
the effects of which are felt by citizens (eg.
policy making, fiscal management, regulation,
crisis/risk management).
10 The International Civil Service Effectiveness Index
Figure 1: InCiSE Core Functions
Central executive Mission support Direct service delivery
Policy making Procurement Tax administration
Fiscal and financial HR management Social security
management administration
Regulation Information technology Digital services
Crisis/risk management Finance
Figure 2: InCiSE Attributes
- Integrity 4. Inclusiveness
- Openness 5. Staff engagement
- Capabilities 6. Innovation
There are some concepts which could be considered attributes but which are particularly relevant
to some core functions. For example, the pursuit of value for money (VFM) is a key feature of the
procurement function, while the use of evidence plays an important role in effective policy making.
Where feasible, these concepts have been captured through the core functions of the InCiSE
framework instead.
The International Civil Service Effectiveness Index 11
2.4 Indicator definitions
A brief definition of each of the InCiSE indicators can be found at Annex A of this report. Further
details, including justifications for including these indicators in the InCiSE framework, can be found
in the Technical Report.
A visual summary of the InCiSE framework and its context is displayed in Figure 3 below.
Figure 3: The InCiSE Framework
Overall Measurement framework Outcomes
inputs (process and output focussed) for citizens
- Total human Civil Service
resource - Total financial Central Mission Direct
resource
executive support service
functions functions delivery
Outputs functions Outputs
to ministers to citizens - Policy making 1. Procurement 1. Tax administration
- Fiscal and 2. HR (at the central/
financial management federal level)
management - IT 2. Social security
- Regulation administration
- Finance
Ministers 4. Crisis/Risk (at the central/ Citizens
management federal level) - Digital services
Attributes
-
Integrity 2. Openness 3. Capabilities 4. Inclusiveness
Policy effects 5. Staff engagement 6. Innovation Policy effects(‘Value for money’ and ‘use of evidence’ are other key attributes
identified. However, these concepts are already captured on the
functions side of the framework to the extent that their inclusion
as attribute indicators is considered unnecessary.)
12 The International Civil Service Effectiveness Index
2.5 Data availability and limitations
Most – but not all – of the InCiSE indicators have been measured to produce the pilot Index.
This is because of gaps in existing data. Eight out of the eleven core functions proposed have been
measured and included, and four out of the six attributes. These are set out below in Figure 4.
Figure 4: InCiSE core functions and attributes being measured for pilot Index
Measured and included in pilot index Not yet measured or included in pilot index
Central executive functions Mission support functions
Policy making Procurement
Fiscal and financial management (FFM) Information technology (IT)
Regulation Finance
Crisis/risk management
Attributes
Mission support functions Staff engagement
Human resource management (HRM) Innovation
Direct service delivery functions
Tax administration
Social security administration
Digital services
Attributes
Integrity
Openness
Capabilities
Inclusiveness
Data collection plans by other organisations may help to fill some of the gaps in future. In other areas,
additional data collection may need to be initiated to measure the relevant indicator. The project will
strive to close these data coverage gaps as InCiSE evolves. Specific actions are set out in the final
chapter of this report.
The International Civil Service Effectiveness Index 13
2.6 Data sources 2.8 Country metrics issues
A wide range of existing data sources have Whilst only two countries currently cover the full
been used to compile the pilot Index and a set of metrics – Norway and UK – several have
detailed summary is provided at Annex B. close to the full complement and any missing
Further details on how these data sources have data has been estimated. Countries with more
been developed to measure each indicator are missing data points inevitably have a larger
available in the Technical Report. proportion of estimated metrics and this factor
should be borne in mind when interpreting
A wealth of data underlies the 12 indicators results. Further details about the methodology
measured so far. Each indicator has one or more used for estimated data can be found in the
broad themes and these are then assessed via Technical Report.
a specific set of performance measurements or
metrics. In total, 76 metrics are spread across Table 1 highlights the availability of InCiSE
the indicators – although some metrics are metrics for each of the 31 countries in this
themselves indices, meaning that the actual pilot index, data availability for each of the 12
number of ‘total metrics’ is far higher. indicators, and where data has been estimated.
Many of the datasets from which the metrics are The development of the InCiSE Index has involved
drawn are updated annually, enabling the Index stages where subjective judgements have had
to be revised on a regular basis to reflect country to be made, for example in aggregating the
developments. This iteration includes data up results and dealing with gaps in data. The impact
until January 2017. on Index results of changing some of these
judgements is explored in the Technical Report.
2.7 Country coverage
To realise its goals and maximise impact, InCiSE
aims to cover a diverse and growing range of
countries over time. However, InCiSE is reliant
on the availability of a wide range of data sets
and collections – and this currently varies
considerably from country to country. Expanding
the range of countries too quickly would require
a large amount of data estimation, or a reduction
in the scope of the framework, or greater reliance
on civil service proxy indicators, which may
undermine the results.
To take account of these issues, any country
with less than 75% of the data that the InCiSE
Index needs has been excluded. The pilot Index
therefore comprises 31 countries where data
availability met this condition.
14 The International Civil Service Effectiveness Index
Table 1: Country coverage and availability of InCiSE metrics
Green squares indicate data was available for all metrics within the indicator for a specific country;
amber squares that data was available for some metrics only; and red squares where no data was
available and so all metrics were estimated.
Country Metric Attributes Direct Service Delivery functions Mission Central Executive functions
account (34 metrics) 26 metrics Support (11 metrics)
(out of 76) functions
(5 metrics)
Integrity Openness Capabilities Inclusiveness Policy Fiscal & Regulation Risk Man. Human Tax Admin Social Digital
(16) (9) (4) (5) making Fin. Man. 6 9 Resources (6) Security Services
8 3 (5) Admin (4)
(1)
NOR 76
GBR 76
AUT 75
FRA 75
ITA 75
NLD 75
FIN 74
DEU 74
DNK 73
POL 73
SWE 73
CZE 72
ESP 72
SVN 70
PRT 69
SVK 69
HUN 68
TUR 68
BEL 67
AUS 66
NZL 66
EST 65
MEX 65
CHE 65
GRC 64
CAN 62
IRL 60
JPN 59
KOR 59
CHL 58
USA 57
The International Civil Service Effectiveness Index 15
Chapter 3: InCiSE
Index results
This chapter presents the pilot Index results 3.1 Overview
in several ways:
Developing a comprehensive range of indicators
• Overall scores and rankings – presented by means that there is often a wide variation in
use of graphs. how countries perform against each of them.
• Top 5 country rankings – these tables rank No country consistently appears in the top
the top 5 countries for each indicator. 5 positions for every indicator, although there
are some strong all-round performers and
• A visual summary of individual country these are highlighted in the individual country
scores – presented via radar diagrams. assessments. There are some stand-out scores
against specific indicators which have been
Annex C of this report provides a summary table
highlighted as well.
of overall country rankings, as well as rankings
for each indicator. Annex D contains a summary There are some already interesting trends to be
of indicator scores for each country. observed. For example, some countries generally
score more strongly against attribute indicators
Annex E contains summary graphs of country
rather than functional ones, and vice versa.
scores and rankings for each indicator.
Nevertheless, of the top 10 countries performing
well against core functions, 8 also feature
in the top 10 for attributes. GDP per capita
adjustments to the overall scores result in some
shifts in the rankings as well.
As yet there are no clear regional patterns
emerging, although the current dominance of
European countries in the Index does enable
some direct comparison. As the methodology gets
refined and the InCiSE country coverage begins
to expand it may be possible to analyse regional
patterns and other trends in future reports.
16 The International Civil Service Effectiveness Index
Figure 5: InCiSE index overall scores and rankings
Figure 6: InCiSE index overall scores and rankings (adjusted for GDP per capita)
The International Civil Service Effectiveness Index 17
3.2 InCiSE overall scores 3.3.1 Top 5 rankings: core functions
and rankings Few patterns emerge within this set of indicators
The graph in Figure 5 shows the overall score and a wide range of countries appear across
and ranking for each country. All scores are the tables.
relative, not absolute, and so the maximum score
a) Policy making
that the top country can achieve is 1.0 while the
lowest score is zero. The same scoring system This indicator currently has four themes: the
applies for each indicator. The methodology quality of policy advice; the role of civil servants
used to obtain these scores is explained in the in setting strategic policy direction; policy
Technical Report. proposal coordination across government;
and monitoring policy implementation.
The graph in Figure 6 adjusts each overall
country score in line with GDP per capita to Some proxy metrics have been used for
take account of its relative wealth, and thus the measuring the quality of policy advice. A fifth
potential resources available for civil service theme, assessing the timeliness and accuracy
operations. The overall levels of civil service of policy delivery, will be added when data
resources invested, human and financial, may becomes available. All data for this indicator
influence the performance of most core functions is drawn from the Bertelsmann Sustainable
and attributes. Governance Indicators (SGI).
Once the GDP per capita adjustment is made,
the top 10 country overall scores and rankings Country Rank
shift, although not dramatically. Only three new UK 1
countries appear in the top 10, most notably USA 2
Turkey, Mexico and Chile who were previously
Denmark 3
in the second half of the Index.
Finland 4
Regional location does not appear to be the
Canada 5
deciding factor in determining levels of civil
service effectiveness, with the first 5 countries
The policy making scores for all 5 countries are
coming from diverse parts of the world, including
very high and close. The UK does particularly
when adjusted for GDP per capita.
well on the quality of policy advice, including
whether it is evidence based, as does the USA.
3.3 Top 5 country rankings The USA also does particularly well on the policy
by indicator coordination theme.
This section provides a brief summary of For the other three countries in this table, all
each indicator and how the top five countries score well on strategic policy direction (Denmark,
performed (prior to GDP adjustment). Finland and the UK are joint top), also on the
quality of policy advice and policy coordination
The top 5 scores across most core function
themes. There is greater variation in the policy
indicators are relatively close, while there are
monitoring scores for these countries.
wider variations in the attribute scores. This
difference may reflect some of the data gaps
and limitations described earlier. Where there
is a greater variation in scores, rankings are more
significant in terms of helping countries to learn
from each other.
18 The International Civil Service Effectiveness Index
b) Fiscal and financial management (FFM) Country Rank
This indicator has four themes: economic Mexico 1
appraisal; economic evaluation; medium-
UK 2
term budgeting processes; and performance
budgeting. Australia 3
Canada 4
Proxy measures have been used for some
aspects. Both the economic appraisal and Switzerland 5
evaluation themes are measured by the World
Economic Forum Global Competitiveness Index Mexico’s top position for this indicator may in
(GCI). The two budgeting themes are measured part stem from the country’s sustained focus on
by two relevant OECD indexes. regulatory policy reforms over the last decade.
Using the OECD data, Mexico scores very highly
across all themes, coming top on both metrics
Country Rank for stakeholder engagement, second for impact
Switzerland 1 assessment, and third for evaluation.
Netherlands 2 The UK achieves the top position for the impact
Sweden 3 assessment theme while Australia is top for
evaluation. Canada and Switzerland score
New Zealand 4
strongly against both the impact assessment
Republic of Korea 5 and stakeholder engagement themes.
Switzerland performs well across all four d) Crisis/risk management
themes, with its highest position (4th) being for This indicator has five themes relating to disaster
performance budgeting. It is worth noting that risk reduction and management: integrated risk
Switzerland tops the Global Competitive Index planning; risk monitoring; public information and
(GCI) and has done so for eight consecutive awareness raising; international cooperation and
years. It also does well in the OECD indexes risk coordination; and post-disaster assessment
used for this indicator. This consistently strong methodology.
performance may in part reflect the Swiss
government’s decision over a decade ago A sixth theme on preparedness for disaster
to overhaul its accounting system in view of response will be added when data becomes
growing financial management demands. available. The data source for all themes is the
UN Hyogo Framework for Action.
Looking at the other country positions in this
table, it is worth noting that the Netherlands ranks
4th overall on the GCI and achieves the top Country Rank
position for the OECD medium-term budgeting Turkey 1
metric. Sweden also does well against this
Finland 2
metric, while the Republic of Korea tops the
OECD performance budgeting index. Slovenia 3
Australia 4
c) Regulation
Sweden 5
This indicator has three themes relating to
regulation policies and management: method, The top 5 country scores for this indicator are
use and quality of regulatory impact assessments; all very high. This may relate to the way in which
stakeholder engagement; and evaluation work. some data relates to adherence to best practice
The sole data source is the OECD’s Indicators in crisis management (with most countries being
of Regulatory Policy and Governance. compliant already).
The International Civil Service Effectiveness Index 19
Turkey’s top ranking for this indicator reflects Japan’s high scores for the meritocratic theme
its well recognised leadership and experience confirm its highly competitive civil service
of disaster risk planning and management. entry systems, including the use of formal
It comes joint top for the integrated risk planning examinations (it came second for this metric).
theme (i.e. the extent to which disaster risk is Both countries also do well against two other
incorporated in national policy planning). Turkey metrics for this theme which assess the extent
also does well on risk monitoring, demonstrating to which personal or political connections
a consistent and systematic approach to determine who gets a job.
reporting. In addition, Turkey comes top for
two of the metrics used to measure the public Both Ireland and Japan do well on attracting and
information dissemination and public awareness retaining talent as well. A single metric is used to
strategies theme. assess the extent to which senior officials have
salaries that are comparable with similar jobs in
e) Human resource management (HRM) the private sector. Ireland ranks 4th and Japan
This indicator currently measures two themes: 6th for this metric.
the extent to which civil service recruitment For other countries in the top 5, Canada scores
systems are meritocratic; and attracting and strongly against the metric covering the skills
retaining talent. and merits of job applicants. Belgium and the
InCiSE hopes to assess four additional themes Republic of Korea do well on the extent to which
when data is available: talent deployment; public sector employees are hired using a formal
performance management; the quality of examination system (Belgium comes top for
learning and development; and the level of civil this metric).
servant satisfaction with HR services. For the f) Tax administration
two themes measured, the data source is the
Quality of Government survey. This indicator currently measures three themes:
the overall efficiency of tax collection; the extent
to which services are user focused; and the
Country Rank extent and the quality of digital provision.
Ireland 1 Two additional themes will be assessed when
Japan 2 data becomes available: the prevention of tax
Canada 3 evasion; and the level of tax gap measurement.
The existing data sources are the OECD’s Tax
Belgium 4
Administration Survey and The World Bank’s
Republic of Korea 5 Doing Business Index (DBI).
Ireland only just beats Japan for the top position,
edging ahead on the attracting and retaining Country Rank
talent theme. Ireland’s overall top position Estonia 1
may reflect its centrally administered systems, 2
Norway
including at relatively low grade levels, plus its
strong regulation of civil service appointments Denmark 3
and increased use of external recruitment. Ireland 4
UK 5
Estonia’s top position reflects a strong all-round
performance against the three themes measured
and there may be a connection between
Estonia’s top ranking for this indicator and the
one for digital services.
20 The International Civil Service Effectiveness Index
Estonia’s scores are very high for the OECD h) Digital services
sourced e-filing metrics and the World Bank This indicator measures four themes: the
and OECD sourced metrics covering cost of extent to which services are user focused; the
tax collection ratios. transparency of services; cross-border mobility
All five countries perform well across of services; and the availability of key enablers
all the metrics for the three themes and their (technical advances made possible through,
overall scores are very close. It is also notable for example, infrastructure investment). It does
that all appear in the top 10 of the World Bank not assess all the services which governments
DBI measure on the time it takes businesses typically provide digitally.
to pay taxes. The sole data source is the European
g) Social security administration Commission’s E-Government Benchmark.
Estimated data was used for most non-EU
This indicator is currently measured via one
countries.
theme/metric only: assessing administration
costs as a proportion of total expenditure on
social security, to capture the overall efficiency Country Rank
of the system. Estonia 1
Additional metrics are planned Austria 2
for future reports as more data becomes Denmark 3
available, focused on the extent to which
Australia 4
services are user focused, the extent and
quality of digital provision, and the prevention Finland 5
of fraud. The existing data source is the
European Commission (Eurostat). Estimated Estonia’s overall score for this indicator is some
data was used for most non-EU countries. way ahead of the other countries in the top
5 table and it does well across all four themes.
This could reflect the country’s strong digital
Country Rank policies and investment in digital infrastructure.
UK 1 Estonia comes top for two themes (key enablers
2
and transparency of services), joint top for user
Hungary
focused services, and second for cross-border
Estonia 3 mobility.
Portugal 4
The other top 5 countries’ scores are close
Slovenia 5 together. They generally perform well across all
four themes and all are particularly strong on the
The top 5 scores for this indicator are very close.
degree to which services are user focused.
This could reflect the narrow way in which this
core function is currently measured, benefiting
those countries where administrative costs 3.3.2 Top 5 rankings: attributes
as a proportion of social security expenditure The attribute rankings show some clear country
are low. This is also an indicator where further patterns emerging, with Canada appearing in
investigation may be needed on the value of four out of the five tables and Finland and New
comparing social security systems internationally Zealand in three of them. There are also some
as they can vary widely. These issues should be stand-out top positions, for example Japan for
borne in mind when interpreting results. the capabilities attribute. Poland and Slovenia’s
high inclusiveness scores are notable as well.
The International Civil Service Effectiveness Index 21
a) Integrity b) Openness
This indicator covers 6 themes: corruption level This indicator has six themes: the degree
perceptions; adherence to rules and procedures; and quality of consultation with society;
work ethics; fairness and impartiality; striving the existence and quality of complaint
to serve citizens and ministers; and processes in mechanisms; government data availability and
place to preserve integrity and prevent conflicts accessibility; government data impact; the right
of interest. to information; and the publication of laws.
There are four data sources: Transparency There are six data sources: the World Justice
International’s Global Corruption Barometer Project’s Open Government Index (OGI); the
(GCB); the World Economic Forum (WEF) Global UN E-participation Index (EPI); Bertelsmann
Competitiveness Index (GCI); the Quality of SGIs; the World Wide Web Foundation: Open
Government (QoG) Expert Survey Data; and the Data Barometer (ODB); The Open Knowledge
OECD’s “Government at a Glance” (GaaG) data. Foundation: Open Data Index (GODI); and the
OECD’s OURdata index.
Fifteen countries have some missing data, partly
because of the large number of metrics for
this indicator (16), and so estimated data has Country Rank
been used. UK 1
New Zealand 2
Country Rank Norway 3
New Zealand 1 Denmark 4
Sweden 2 Finland 5
Norway 3
4
All the top 5 countries for this indicator feature
Canada
in the top 10 of Transparency International’s
Denmark 5 Corruptions Perception Index 2016 (Denmark,
New Zealand and Finland are in the top 3 places
New Zealand’s overall score is well ahead of respectively, Norway is 6th and the UK is in
the others in this top 5 table, reflecting a strong 10th place). It is worth noting that this is the only
performance for all metrics. New Zealand does attribute where Canada does not appear in the
particularly well on the corruption perceptions top 5, although it still performed well against
and work ethic themes. It comes top or joint most of the relevant metrics.
top in 6 out of the 16 metrics for this indicator,
and top in at least one metric in 4 out of the 6 The UK and New Zealand’s overall scores for
themes. this attribute are very close. The UK scores
more highly on two themes – government data
For all other countries in this table the overall availability and accessibility, plus government
scores are very close. data impact. New Zealand comes top on the
publishing laws theme and 2nd on the right
to information theme. Both countries score
particularly well on the public consultation theme,
as do others in the top 5 table.
22 The International Civil Service Effectiveness Index
c) Capabilities d) Inclusiveness
This attribute currently measures two themes: This indicator currently has only two themes:
core capability (eg. problem solving, numeracy the proportion of women represented in the
and literacy skills); and the educational civil service; and ethnic/religious minority
attainment of the workforce. representation. InCiSE hopes to broaden the
coverage over time once data becomes available
Additional themes are planned for this indicator for other key representation categories.
once data becomes available to include
leadership, commercial, analytical and digital There are two data sources: the OECD’s
capabilities. The sole data source is the OECD Government at a Glance (GaaG) survey; and
Programme for the International Assessment of the Quality of Government (QoG) survey.
Adult Competencies (PIAAC) survey. A high level
of estimated data was used for some countries.
Country Rank
Poland 1
Country Rank
Slovenia 2
Japan 1
Canada 3
Finland 2
Finland 4
Canada 3
Australia 5
New Zealand 4
Switzerland 5 Poland’s top position reflects its strong score
for the metric which assesses the proportion of
Japan’s capabilities score is strikingly ahead women in central government senior positions,
of the other countries in this top 5 table. as well as ethnic minority representation. Slovenia
This reflects its scores in the OECD survey also scores well against one of the metrics
concerning the proportion of the public sector concerning senior positions and tops the ethnic
with high literacy, numeracy and problem solving minority representation metric. Canada scores
skills, where it tops the three rankings by a well across all four metrics for this attribute.
considerable margin. Japan’s score also reflects
its strong position in the same survey concerning
the proportion of the public sector with tertiary
education (ranked 8th – Canada is top). It is
notable that Japan scores highly against the
HRM indicator as well.
The International Civil Service Effectiveness Index 23
Chapter 4: InCiSE index
country results summary
This section presents a visual summary of each More detailed information about country scores,
country’s indicator scores via the use of radar including the themes and metrics which underlie
diagrams. These highlight where a country each indicator result, can be found on the
performs particularly well compared to other InCiSE website.
countries in the Index and where further attention
or analysis may be beneficial. Further details about the methodological
approach, and examples of interpretations,
Indicator results are presented on a scale of can be found in the Technical Report.
0 to 1, with 0 representing the weakest
performing country in the Index and 1 the best Annex E of this main report shows the country
performing country. Assessment of a country is scores for each indicator, in order of ranking.
therefore relative to others included in the index The key for the country radar graphs is
only, not an absolute measure. Weaker scores as follows:
do not reflect a view on prioritisation within a
country, but rather opportunities to learn from Key
other countries.
Country scores
Average (mean) scores
24 The International Civil Service Effectiveness Index
4.1 Australia
5/26/2017 Radar chart
Tax
administration
Policy
making Inclusiveness
0.75
Social
security Capabilities
0.50
0.25
Digital
services Openness
Fiscal
and Integrity
financial
management
Regulation Human
Crisis resource
management management
Australia is ranked 3rd
overall on the Index and
achieves above average
scores in most areas.
Australia’s regulation score is strong (ranked
3rd overall) and they should have some
useful lessons to share, particularly on the
evaluation theme where they achieve the top
score. Australia is ranked 4th for crisis/risk
management, scoring well across all themes.
Within the attribute indicators, Australia’s
https://inciseradarvis.cloudapps.digital/ 2/6
strongest ranking is for inclusiveness (5th
overall), suggesting its civil service has a good
representation of women, ethnic and religious
groups relative to most other countries.
The main indicator which requires attention is
fiscal and financial management (FFM), where
most thematic scores are just below average.
The International Civil Service Effectiveness Index 25
4.2 Austria
5/26/2017 Radar chart
Tax
administration
Policy
making Inclusiveness
0.75
Social
security Capabilities
0.50
0.25
Digital
services Openness
Fiscal
and Integrity
financial
management
Regulation Human
Crisis resource
management management
Austria is ranked 16th overall The key core functions which require
further analysis are policy making, crisis/risk
on the Index. The country’s management and HRM where scores are
strongest score is for digital below average.
services (ranked 2nd overall). On attributes, the metrics used to measure the
Austria scores strongly across most digital capabilities indicator suggest that literacy skills
service themes, coming joint top (with Estonia) and educational attainment within the workforce
on the extent to which services are user focused. are relatively low, and there may be some
Austria also scores well on tax and social lessons to learn from high scoring countries.
security administration. There may be a link There may also be a connection between
between Austria’s high digital services score and Austria’s capabilities and HRM scores given the
some aspects of the digital theme under the tax latter indicator includes metrics on attracting and
administration indicator, for example the use of
https://inciseradarvis.cloudapps.digital/
retaining talent. 2/6
e-filing. Austria scores are above average for the
integrity and openness attributes as well.
26 The International Civil Service Effectiveness Index
4.3 Belgium
5/26/2017 Radar chart
Tax
administration
Policy
making Inclusiveness
0.75
Social
security Capabilities
0.50
0.25
Digital
services Openness
Fiscal
and Integrity
financial
management
Regulation Human
Crisis resource
management management
Belgium is ranked 13th overall Belgium also scores well on policy making,
its integrity score is above average too.
on the Index and achieves
above average scores for The main core function where Belgium
many indicators. performs relatively less well is fiscal and
financial management (FFM), particularly the
Belgium’s strong HRM score (ranked medium-term and performance budgeting
4th overall) suggests it has a relatively themes. Belgium’s openness score also
meritocratic recruitment system and that indicates a below average performance.
the civil service can attract and retain talent.
Belgium comes top for the metric on the
extent to which recruitment is carried out
via a formal examination system.
https://inciseradarvis.cloudapps.digital/ 2/6
The International Civil Service Effectiveness Index 27
4.4 Canada
5/25/2017 Radar chart
Tax
administration
Policy
making Inclusiveness
0.75
Social
security Capabilities
0.50
0.25
Digital
services Openness
Fiscal
and Integrity
financial
management
Regulation Human
Crisis resource
management management
Canada tops the InCiSE Index Tax administration scores are close to the
average. Canada’s score for overall efficiency
overall and scores consistently of tax collection suggests there may be potential
well above the average for for improvement through greater use of digital
most indicators. processes for tax administration. This would
be in line with high scoring countries for this
Canada is ranked 4th for the regulation indicator, indicator.
with high scores for the stakeholder engagement
and impact assessment themes. It is ranked Canada’s openness score, although well above
3rd for HRM (scoring strongly for the theme the average, suggests there may be some
on the extent to which the civil service has a lessons to learn from the leading countries
meritocratic system) and 5th for policy making. concerning the right to information theme,
as well as the availability and accessibility of
On attributes, Canada appears in the top 5 for government data.
https://incise-radar-vis.cloudapps.digital/ 2/6
all but one indicator (openness). It comes top
for the capabilities metric on the proportion of
the public sector with tertiary education. On
inclusiveness, Canada scores well across most
metrics, suggesting its civil service has a good
representation of women, ethnic and religious
groups relative to other countries.
28 The International Civil Service Effectiveness Index
4.5 Chile
5/26/2017 Radar chart
Tax
administration
Policy
making Inclusiveness
0.75
Social
security Capabilities
0.50
0.25
Digital
services Openness
Fiscal
and Integrity
financial
management
Regulation Human
Crisis resource
management management
Chile is ranked 22nd overall on On attributes, Chile’s integrity score is at
the average for all countries included in the
the Index. Its position rises to Index; it performs particularly well on the
8th when scores are adjusted metric measuring the degree of whistleblower
for GDP per capita. protection.
The core function where Chile scores relatively
Chile scores well on policy making (ranked weakly is regulation, particularly the impact
9th), particularly on the quality of policy advice assessment and evaluation themes and metrics.
theme where it is ranked joint top position. These could be initial areas for attention, learning
Chile also scores well on crisis/risk management from the high scoring countries for this indicator.
(ranked 8th).
https://inciseradarvis.cloudapps.digital/ 2/6
The International Civil Service Effectiveness Index 29
4.6 Czechia
5/26/2017 Radar chart
Tax
administration
Policy
making Inclusiveness
0.75
Social
security Capabilities
0.50
0.25
Digital
services Openness
Fiscal
and Integrity
financial
management
Regulation Human
Crisis resource
management management
Czechia is ranked 28th overall The main core function where performance
is weak relative to other countries is tax
on the Index. Its strongest administration, particularly the themes concerning
score is for fiscal and financial the degree to which services are user focused,
management (ranked 11th). as well as the extent of digital provision.
Czechia also has above average scores Czechia’s HRM and digital services scores
for regulation, scoring well on the impact also warrant attention across most themes.
assessment and stakeholder engagement All attributes scores require further analysis
themes (ranked 9th-13th for two of the as well.
four metrics).
https://inciseradarvis.cloudapps.digital/ 2/6
30 The International Civil Service Effectiveness Index
4.7 Denmark
5/26/2017 Radar chart
Tax
administration
Policy
making Inclusiveness
0.75
Social
security Capabilities
0.50
0.25
Digital
services Openness
Fiscal
and Integrity
financial
management
Regulation Human
Crisis resource
management management
Denmark is ranked 11th overall On policy making, Denmark ranks joint top for
the strategic policy direction theme and scores
on the Index. Its strongest highly on the quality of policy advice and policy
attribute scores are for coordination.
integrity where it is ranked A core function which may warrant attention, by
5th and openness where it is considering whether lessons can be learnt from
ranked 4th. high scoring countries, is crisis/risk management,
focusing initially on the metrics within the
Denmark is ranked 3rd for three core function integrated risk planning theme. Denmark’s social
indicators – policy making, tax administration security administration score also merits further
(scoring well across all themes) and digital analysis as the sole metric used for this indicator
services (again scoring well across all themes, suggests lower than average systems efficiency.
particularly the degree to which services are
https://inciseradarvis.cloudapps.digital/
The main attribute where 2/6
Denmark performs less
user focused).
well relative to other countries is inclusiveness
where scores for the main themes vary.
The International Civil Service Effectiveness Index 31
4.8 Estonia
5/26/2017 Radar chart
Tax
administration
Policy
making Inclusiveness
0.75
Social
security Capabilities
0.50
0.25
Digital
services Openness
Fiscal
and Integrity
financial
management
Regulation Human
Crisis resource
management management
Estonia is ranked 7th overall Estonia is ranked 3rd for social security
administration, suggesting comparatively low
on the Index. It rises to the administrative costs relative to sector expenditure.
top position when scores are
The main indicator requiring attention, to
adjusted for GDP per capita. consider whether lessons can be learnt from
Some indicator scores are especially noteworthy. other countries, is inclusiveness. The metrics
First, digital services where Estonia is ranked overall suggest lower than average women’s and
top and their overall score stands out above other group representation in the civil service.
the rest. It scores highly against all themes
and Estonia’s overall performance could
reflect its strong IT policies and investment in
IT infrastructure. Estonia is also ranked top
for the tax administration indicator. There is a
https://inciseradarvis.cloudapps.digital/ 2/6
potential connection between these two core
function scores as one of the tax administration
metrics relates to the extent and quality of digital
service provision.
32 The International Civil Service Effectiveness Index
4.9 Finland
5/26/2017 Radar chart
Tax
administration
Policy
making Inclusiveness
0.75
Social
security Capabilities
0.50
0.25
Digital
services Openness
Fiscal
and Integrity
financial
management
Regulation Human
Crisis resource
management management
Finland is ranked 5th overall Finland’s strongest core functions scores are
in crisis/risk management (ranked 2nd overall),
on the Index and achieves policy making (ranked 4th) and digital services
above average scores for most (ranked 5th). For the latter, Finland scores
indicators. well across all themes particularly the extent
to which services are user focused. On policy
Finland’s attributes scores are consistently high. making, Finland is in joint top position for the
It is ranked 2nd for capabilities – suggesting strategic policy direction theme. It also scores
strong technical skills and educational attainment highly for the quality of policy advice and policy
across the workforce relative to other countries coordination themes.
– and 4th on inclusiveness. Finland is ranked 5th The main indicator requiring further attention is
on openness, where it scores particularly well on regulation, focusing on impact assessment as
the public consultation theme. this score was relatively2/6weak compared with the
https://inciseradarvis.cloudapps.digital/
other themes.
The International Civil Service Effectiveness Index 33
4.10 France
5/26/2017 Radar chart
Tax
administration
Policy
making Inclusiveness
0.75
Social
security Capabilities
0.50
0.25
Digital
services Openness
Fiscal
and Integrity
financial
management
Regulation Human
Crisis resource
management management
France is ranked 17th France’s social security administration score also
merits further analysis as the sole metric used
overall on the Index. It has for this indicator suggests lower than average
above average scores for systems efficiency.
the integrity and openness France’s crisis/risk management scores warrant
attributes. further analysis. It scores well on some themes,
for example integrated risk planning and post
France’s fiscal and financial management (FFM),
disaster risk assessments. Improvements in
tax administration, digital services and HRM core
other themes, for example public information
functions scores are also all above average.
and awareness strategies, may help improve the
On HRM, France scores particularly well on overall score in future.
some aspects of the meritocratic recruitment
theme, for example the use of formal
https://inciseradarvis.cloudapps.digital/ 2/6
examination systems to hire staff (ranked 4th).
The main attributes requiring attention are
inclusiveness and capabilities, initially by
considering the scope to learn from the leading
countries.
34 The International Civil Service Effectiveness Index
4.11 Germany
5/26/2017 Radar chart
Tax
administration
Policy
making Inclusiveness
0.75
Social
security Capabilities
0.50
0.25
Digital
services Openness
Fiscal
and Integrity
financial
management
Regulation Human
Crisis resource
management management
Germany is ranked 24th overall Germany’s policy making score also requires
further analysis. Its thematic scores vary,
on the Index. It achieves on or whereas high ranking countries for this indicator
above average scores for all tend to do well across most themes. Specific
attribute indicators. themes to consider further include strategic
direction (this is based on an assessment of the
Germany is ranked 8th for integrity and 9th for extent to which strategic planning units influence
capabilities. The latter score suggests a civil government decision making), and policy
service with relatively good technical skills, monitoring.
alongside a high level of educational attainment It is worth noting that Germany has a federal
compared with other countries. government system. An issue for the future is
On core functions, Germany scores highly on the need to ensure the InCiSE framework can
regulation (ranked 7th overall) and does particularly
https://inciseradarvis.cloudapps.digital/
capture the strengths of2/6different systems.
well on the evaluation theme, ranking 4th and
5th for the relevant metrics. The main indicator
requiring attention is crisis/risk management,
although it should be noted that estimated data
was used for a number of countries.
The International Civil Service Effectiveness Index 35
4.12 Greece
5/26/2017 Radar chart
Tax
administration
Policy
making Inclusiveness
0.75
Social
security Capabilities
0.50
0.25
Digital
services Openness
Fiscal
and Integrity
financial
management
Regulation Human
Crisis resource
management management
Greece is ranked 29th overall Greece’s crisis/risk management score is around
the average, although it performs well on aspects
on the Index. The country’s of the integrated risk planning theme.
high inclusiveness score
The core functions where performance is
stands out (ranked 6th). relatively weaker are policy making, digital
Within the inclusiveness indicator, Greece does services, regulation, and fiscal and financial
particularly well on the metrics for the women’s management. On the latter, Greece’s
representation theme. Greece also scores above performance is only slightly below average for
average for the metric on ethnic and religious the medium-term budgeting theme. However,
groups representation. The remaining attributes its performance was weaker compared with
scores require further analysis to prioritise areas other countries for the performance budgeting,
for improvement. economic appraisal and evaluation themes.
https://inciseradarvis.cloudapps.digital/
Greece’s social security administration score On digital services, Greece’s
2/6
performance is
is above average as well, suggesting it has assessed to be relatively weak across all themes
reasonable administration costs relative to and there is scope to learn from the practices of
sector expenditure. the leading countries.
36 The International Civil Service Effectiveness Index
4.13 Hungary
5/26/2017 Radar chart
Tax
administration
Policy
making Inclusiveness
0.75
Social
security Capabilities
0.50
0.25
Digital
services Openness
Fiscal
and Integrity
financial
management
Regulation Human
Crisis resource
management management
Hungary is ranked 30th overall Some thematic scores are worth noting.
On policy making, Hungary comes top for
on the Index. The country’s two of the three metrics concerning policy
very high social security monitoring. The coordination of policy proposals
administration score is thematic score is also above average. Hungary
also achieves above average thematic scores
noteworthy. within the integrity indicator. On regulation,
Hungary has above average scores for the
Hungary is ranked 2nd overall for this indicator,
impact assessment theme.
suggesting it has a comparatively efficient social
security system with low administration costs The main core function indicators requiring
relative to sector expenditure. attention are HRM, plus digital services and fiscal
and financial management. All attribute indicators
https://inciseradarvis.cloudapps.digital/
merit further attention, initially
2/6
by considering the
scope to learn from the leading countries.
The International Civil Service Effectiveness Index 37
4.14 Ireland
5/26/2017 Radar chart
Tax
administration
Policy
making Inclusiveness
0.75
Social
security Capabilities
0.50
0.25
Digital
services Openness
Fiscal
and Integrity
financial
management
Regulation Human
Crisis resource
management management
Ireland is ranked 20th overall Ireland’s social security administration score
merits further attention as the sole metric used
on the Index and generally for this indicator suggests weaker systems
performs more strongly on efficiency compared with other countries. Ireland’s
core functions. regulation score warrants further investigation
as well as it performs better in some thematic
Ireland’s scores are high across all the HRM areas than others. Themes for potential analysis
metrics. This may reflect Ireland’s centrally include stakeholder engagement and evaluation.
administered systems, including at relatively
low grade levels, plus its strong regulation of The main attribute which would benefit from
civil service appointments and increased use further analysis is openness. However, estimated
of external recruitment. Ireland also scores well data was used for some metrics and this should
against the metrics concerning attracting and be borne in mind when analysing results.
retaining talent, notably the extent to which
https://inciseradarvis.cloudapps.digital/ 2/6
senior officials’ salaries are comparable with
similar jobs in the private sector.
On tax administration, Ireland is ranked 4th,
scoring consistently well across all themes,
and it is worth noting that the top 5 countries’
overall scores are all close for this indicator.
38 The International Civil Service Effectiveness Index
4.15 Italy
5/26/2017 Radar chart
Tax
administration
Policy
making Inclusiveness
0.75
Social
security Capabilities
0.50
0.25
Digital
services Openness
Fiscal
and Integrity
financial
management
Regulation Human
Crisis resource
management management
Italy is ranked 27th overall with the underlying data reflecting perceptions
of the composition and efficiency of public
on the Index. It generally spending.
achieves better scores for
Italy’s tax administration score is also relatively
core function indicators. low but with a wide variation in thematic scores.
It leads for the metrics on the extent and quality
Italy’s social security administration score is of digital provision theme but does less well in the
above average, suggesting it has relatively low other themes concerning the overall efficiency of
administration costs relative to sector expenditure. tax collection and the extent to which services are
Italy’s crisis/risk management scores are user focused.
above average as well. It achieves the joint All attributes scores require further analysis,
highest score for the post-disaster assessment although some positive metrics scores are worth
methodology theme (with 9 other countries).
https://inciseradarvis.cloudapps.digital/
noting. Italy scores well2/6relative to other countries
Italy is also joint top for one of the two metrics on the whistleblower protection metrics within
concerning risk monitoring. the integrity indicator, as well as the openness
The core function where Italy performs less well metrics on the degree and quality of consultation
compared with other countries is fiscal and with society. On inclusiveness, Italy scores well
financial management (FFM). Scores are below on two gender representation metrics relating to
average for the budgeting themes, and also for the proportion of women in senior positions in
the economic appraisal and evaluation themes, the civil service.
The International Civil Service Effectiveness Index 39
4.16 Japan
5/26/2017 Radar chart
Tax
administration
Policy
making Inclusiveness
0.75
Social
security Capabilities
0.50
0.25
Digital
services Openness
Fiscal
and Integrity
financial
management
Regulation Human
Crisis resource
management management
Japan is ranked 15th overall The core functions where Japan’s performance
is weaker compared with other countries are
on the Index. It is ranked top tax administration and regulation. The overall
for the capabilities attribute efficiency of tax collection and the extent to which
with a score well ahead of services are user focused are themes where future
improvements could be informed by the high
other countries. scoring countries.
The metrics used for the capabilities indicator
For regulation, Japan’s performance is below
suggest Japan has a civil service with very high
the average for each of the three themes
literacy, numeracy and problem solving skills,
concerning appraisal, evaluation and stakeholder
plus strong educational attainment levels.
engagement, and relatively low for the metrics
Japan is ranked a close 2nd for the HRM within the stakeholder engagement theme.
indicator as well. Japan’s high scores for the
https://inciseradarvis.cloudapps.digital/
On attributes, Japan’s inclusiveness
2/6
scores
meritocratic recruitment theme confirm its
are comparatively low. Learning lessons from
competitive civil service entry systems, including
the leading countries on increasing women’s
the use of formal examinations (ranked 2nd).
representation in the civil service could be an
Japan also scores well on attracting and
initial area of focus.
retaining talent.
40 The International Civil Service Effectiveness Index
4.17 Republic of Korea
5/26/2017 Radar chart
Tax
administration
Policy
making Inclusiveness
0.75
Social
security Capabilities
0.50
0.25
Digital
services Openness
Fiscal
and Integrity
financial
management
Regulation Human
Crisis resource
management management
The Republic of Korea is The Republic of Korea’s high HRM scores across
most metrics suggests its civil service has a
ranked 9th overall on the Index relatively meritocratic recruitment system. It does
and its position rises to 4th particularly well on the metric concerning the use
following adjustments for GDP of formal examinations systems to recruit staff.
The country’s scores suggest that it is also able
per capita. to attract and retain talent, although it is ranked
just below average for the metric on the extent
The Republic of Korea scores above average
to which senior staff salaries compare favourably
for all core functions, notably HRM (ranked 5th
with similar jobs in the private sector.
overall) and fiscal and financial management
(ranked 5th again). On the latter, one of the The Republic of Korea’s inclusiveness score
main data sources for this indicator is the OECD merits further analysis. Based on the available
performance budgeting index where the country data, the representation2/6of women in the civil
https://inciseradarvis.cloudapps.digital/
is ranked top. service is assessed to be weak relative to other
countries.
The International Civil Service Effectiveness Index 41
4.18 Mexico
5/26/2017 Radar chart
Tax
administration
Policy
making Inclusiveness
0.75
Social
security Capabilities
0.50
0.25
Digital
services Openness
Fiscal
and Integrity
financial
management
Regulation Human
Crisis resource
management management
Mexico is ranked 19th overall methodology and integrated risk planning) they
are less strong for others, for example the risk
on the Index and its position monitoring and international cooperation themes
rises to 2nd when scores are where there may be opportunities to learn from
adjusted for GDP per capita. best practice in future.
Mexico achieves the top position for regulation There is a similar pattern of thematic variation
which could reflect the country’s sustained for the HRM indicator. Mexico performs relatively
focus on regulatory policy reforms over the last strongly against the attracting and retaining
decade. It scores strongly across all themes talent theme where it is ranked top on the
and comes top for both metrics on stakeholder extent to which senior officials’ salaries compare
engagement. Mexico’s strong performance favourably with private sector counterparts.
against this indicator suggests they have some However, scores are relatively low for the
positive lessons to share with countries who wish
https://inciseradarvis.cloudapps.digital/
meritocracy of recruitment
2/6
theme and there may
to improve in this area. be lessons to learn from the leading countries.
When assessing relative performance, crisis/risk All attribute scores merit further analysis to
management is a core function for further prioritise areas for improvement. Estimated data
consideration as Mexico’s scores vary significantly was used for some metrics within the capabilities
across the themes measured. Although scores and inclusiveness indicators which should be
compare favourably for two themes (post disaster borne in mind when interpreting these results.
42 The International Civil Service Effectiveness Index
4.19 Netherlands
5/26/2017 Radar chart
Tax
administration
Policy
making Inclusiveness
0.75
Social
security Capabilities
0.50
0.25
Digital
services Openness
Fiscal
and Integrity
financial
management
Regulation Human
Crisis resource
management management
The Netherlands is ranked 14th Regulation is another core function where the
Netherlands’ performance is less strong relative
overall on the Index. It performs to other countries. There may be lessons to
very strongly on fiscal and learn from the leading countries concerning
financial management (ranked the stakeholder engagement and impact
assessment themes. For the policy making
2nd overall). indicator, there may be gains through focusing
on the policy monitoring theme as relatively
The Netherlands scores highly on tax
weak performance in this area reduced the
administration and crisis/risk management as
overall score.
well. Its openness score is strong too (ranked
6th), also capabilities (ranked 7th). Looking at the attribute indicators, the
Netherlands’ inclusiveness score merits
The Netherlands’ score for the social security
attention, although it does better on some
administration indicator is relatively weak,
https://inciseradarvis.cloudapps.digital/ 2/6
gender metrics than on others. Efforts to boost
suggesting there may be opportunities
women’s representation in senior management
for improvement in systems efficiency.
positions in the civil service, learning from the
practices of the high scoring countries, could be
an initial focus area.
The International Civil Service Effectiveness Index 43
4.20 New Zealand
5/26/2017 Radar chart
Tax
administration
Policy
making Inclusiveness
0.75
Social
security Capabilities
0.50
0.25
Digital
services Openness
Fiscal
and Integrity
financial
management
Regulation Human
Crisis resource
management management
New Zealand is ranked 2nd There are two core functions where New
Zealand’s performance is relatively weaker: crisis/
overall on the Index. Its risk management and tax administration. On the
high integrity and openness former, scores vary and the themes which merit
rankings are noteworthy. further analysis concern integrated risk planning
and risk monitoring.
New Zealand’s top integrity score is well
ahead of all other countries, reflecting a strong On tax administration, learning from the practices
all-round performance against most metrics. of the leading countries on some aspects of the
It does particularly well on the corruption extent and quality of digital provision could help
perceptions and work ethic themes, achieve a higher overall indicator score in future.
and comes top or joint top in 6 out of the
16 metrics for this indicator.
https://inciseradarvis.cloudapps.digital/ 2/6
New Zealand comes a close 2nd on
openness, achieving the top score for the
publishing laws theme and ranking 2nd on
the right to information.
44 The International Civil Service Effectiveness Index
4.21 Norway
5/26/2017 Radar chart
Tax
administration
Policy
making Inclusiveness
0.75
Social
security Capabilities
0.50
0.25
Digital
services Openness
Fiscal
and Integrity
financial
management
Regulation Human
Crisis resource
management management
Norway is ranked 8th overall Norway’s inclusiveness score is just below
average, although the thematic scores vary.
on the Index and its scores For example, while it scores relatively well against
are above average for most some metrics for the gender representation
indicators. theme its performance for the other theme is
below average. An initial focus on the latter,
Norway is currently one of two countries where learning from best practice in the leading
data is available for all 76 InCiSE metrics. countries, may help to achieve a stronger
overall indicator score in future.
Norway scores particularly well on tax
administration (ranked 2nd) and across all On core functions, Norway’s performance for
metrics for this indicator. Norway’s scores are the regulation indicator merits further attention
also very high for two attributes: integrity and as scores were below the average across the
openness (both ranked 3rd). On the latter,
https://inciseradarvis.cloudapps.digital/
themes, particularly impact
2/6
assessment work.
Norway does particularly well on the public
consultation theme.
The International Civil Service Effectiveness Index 45
4.22 Poland
5/26/2017 Radar chart
Tax
administration
Policy
making Inclusiveness
0.75
Social
security Capabilities
0.50
0.25
Digital
services Openness
Fiscal
and Integrity
financial
management
Regulation Human
Crisis resource
management management
Poland is ranked 21st overall Poland scores well on policy making (ranked 11th
overall) and is in the top 10 for metrics concerning
on the Index and its position the quality of policy advice theme. It also does well
rises to 12th when scores are on the policy monitoring theme. Poland’s above
adjusted for GDP per capita. average score for social security administration
suggests that it has relatively low administration
Poland is ranked top for the inclusiveness costs relative to sector expenditure.
indicator. This reflects very strong scores for
the metrics on the proportion of women in One core function where Poland’s performance
central government management and senior is weaker compared with other countries is
management positions, as well as levels of tax administration. Scores are below average
ethnic and religious minority representation. across the themes, particularly those covering
the overall efficiency of tax collection and the
For the capabilities indicator, while Poland’s extent and quality of digital provision. It is worth
https://inciseradarvis.cloudapps.digital/ 2/6
relative performance is above average for noting that Poland’s score for the digital services
the educational attainment theme, a focus indicator is below average as well and an initial
on the numeracy and problem solving skills focus on this area could have a positive effect on
metrics within the core capability theme, other core functions in future.
learning from the high scoring countries, may
help achieve a higher overall score in future.
46 The International Civil Service Effectiveness Index
4.23 Portugal
5/26/2017 Radar chart
Tax
administration
Policy
making Inclusiveness
0.75
Social
security Capabilities
0.50
0.25
Digital
services Openness
Fiscal
and Integrity
financial
management
Regulation Human
Crisis resource
management management
Portugal is ranked 26th overall For the HRM indicator, Portugal’s performance
relative to other countries is lower across most
on the Index and its position metrics within the meritocracy of recruitment
rises to 23rd when scores are theme, the main exception being the extent to
adjusted for GDP per capita. which formal examination systems are used
for recruitment purposes. Within the attracting
Portugal scores highly on social security and retaining talent theme, Portugal’s score is
administration (ranked 4th overall), suggesting relatively low on the extent to which salaries for
that administrative costs as a proportion of sector senior officials compare favourably with similar
expenditure are relatively low. It is ranked 6th for jobs in the private sector. Learning from the
digital services and is in the top 5 for three out leading countries may help to achieve improved
of the four themes within this indicator, including scores in future.
the extent to which services are user focused. On attributes, while Portugal’s inclusiveness
https://inciseradarvis.cloudapps.digital/ 2/6
The core functions where Portugal’s performance score is around the average its performance
is assessed to be weaker compared with other against the other indicators is relatively lower.
countries are regulation and HRM. On regulation, Further analysis would help prioritise areas for
the main themes which could be considered improvement.
for further attention relate to stakeholder
engagement and impact assessment.
The International Civil Service Effectiveness Index 47
4.24 Slovakia
5/26/2017 Radar chart
Tax
administration
Policy
making Inclusiveness
0.75
Social
security Capabilities
0.50
0.25
Digital
services Openness
Fiscal
and Integrity
financial
management
Regulation Human
Crisis resource
management management
Slovakia is ranked 31st overall The main core functions where Slovakia’s
performance is weaker relative to other countries
on the Index. It is ranked 14th are policy making, HRM and digital services. On
for the regulation indicator, policy making, it scores below the average for
with particularly strong all thematic metrics, particularly the coordination
of policy proposals. On HRM, learning from
scores for the stakeholder the leading countries might help achieve
engagement metrics. improvements across the themes. Further
Within the regulation indicator, Slovakia also analysis of Slovakia’s digital services indicator
scores well on the impact assessment theme. score would also help to identify areas for
improvement, as well as all attribute indicators.
Slovakia’s crisis/risk management score is
around the average mark. It is in the top 10
countries for both risk monitoring metrics,
https://inciseradarvis.cloudapps.digital/ 2/6
although its performance is less strong for
the integrated risk planning theme. An initial
focus on this theme, learning from best practice,
could help to improve Slovakia’s overall indicator
score in future.
Slovakia’s social security administration score is
around the average mark as well.
48 The International Civil Service Effectiveness Index
4.25 Slovenia
5/26/2017 Radar chart
Tax
administration
Policy
making Inclusiveness
0.75
Social
security Capabilities
0.50
0.25
Digital
services Openness
Fiscal
and Integrity
financial
management
Regulation Human
Crisis resource
management management
Slovenia is ranked 23rd 5th overall), suggesting that administrative
costs as a proportion of sector expenditure are
overall on the Index and its comparatively low.
position rises to 16th following
The main core function indicators where Slovenia
adjustments for GDP per capita. performs weakly relative to other countries are
policy making and HRM. On policy making, all
Slovenia’s very high inclusiveness score – ranked themes could be reviewed to establish priority
2nd overall – is noteworthy, reflecting a strong areas for improvement. Both themes within the
score for the metric assessing the proportion HRM indicator merit further analysis, although
of women in central government management Slovenia’s performance is above average for the
and senior management positions. It also tops metric concerning the use of formal examinations
the ethnic and religious minority representation systems for recruitment purposes.
metric for this indicator.
https://inciseradarvis.cloudapps.digital/
Slovenia’s performance2/6for all other attribute
Slovenia is ranked 3rd overall for crisis/risk indicators is below the average and they may
management and it achieves consistently good benefit from learning from the high scoring
scores across most metrics. Its scores are countries. However, a degree of estimated data
particularly high for the risk monitoring theme was used for the capabilities indicator which
(joint top on both metrics) and it also achieves should be borne in mind when analysing results.
strong scores for the post disaster assessment
methodology theme. Slovenia’s social security
administration scores are also high (ranked
The International Civil Service Effectiveness Index 49
4.26 Spain
5/26/2017 Radar chart
Tax
administration
Policy
making Inclusiveness
0.75
Social
security Capabilities
0.50
0.25
Digital
services Openness
Fiscal
and Integrity
financial
management
Regulation Human
Crisis resource
management management
Spain is ranked 18th overall on assessed to be below the average, particularly the
stakeholder engagement and evaluation themes.
the Index and its position rises
to 15th following adjustments On inclusiveness, whilst some scores for the
gender representation theme are above average,
for GDP per capita. a review of the leading countries’ scores would
Spain is ranked 7th overall for the tax help prioritise areas to focus on in future.
administration indicator, achieving a particularly
good score for the extent and quality of digital
provision. Spain is ranked 9th for the social
security administration indicator, suggesting
reasonable administrative costs as a proportion
of total sector expenditure.
https://inciseradarvis.cloudapps.digital/ 2/6
The main core functions where Spain’s
performance is weak relative to other
countries are regulation and fiscal and financial
management. On the latter, Spain achieves
a stronger performance for the medium term
budgeting theme and could learn from the high
scoring countries for the performance budgeting
theme. On regulation, Spain’s performance is
50 The International Civil Service Effectiveness Index
4.27 Sweden
5/26/2017 Radar chart
Tax
administration
Policy
making Inclusiveness
0.75
Social
security Capabilities
0.50
0.25
Digital
services Openness
Fiscal
and Integrity
financial
management
Regulation Human
Crisis resource
management management
Sweden is ranked 6th overall On inclusiveness, there is some variation in
scores by theme. For example, Sweden mostly
on the Index and achieves scores well compared with other countries on
above average scores in the gender representation theme. High scoring
most areas. countries for this indicator tend to perform well
across both themes and learning from their
Sweden’s integrity score is noteworthy (ranked practices may be beneficial.
2nd overall). It achieves especially strong scores For the HRM indicator, Sweden performs
for the work ethic theme and for the levels/ relatively well against most of the meritocracy of
perceptions of corruption theme. Sweden recruitment metrics. Lessons could be learned
is ranked 3rd overall for fiscal and financial from the leading countries for the attracting and
management, scoring particularly well against retaining talent theme, particularly the extent to
the medium term budgeting, performance which senior staff salaries compare favourably
appraisal and evaluation themes.
https://inciseradarvis.cloudapps.digital/ 2/6
with similar jobs in the private sector.
Indicator scores that are relatively weaker Sweden performs well relative to others in some
and which may merit further analysis include aspects of regulation, although its stakeholder
inclusiveness and HRM (where Sweden’s engagement theme score is below average and
performance is around the average). this could be an area for development in future.
The International Civil Service Effectiveness Index 51
4.28 Switzerland
5/26/2017 Radar chart
Tax
administration
Policy
making Inclusiveness
0.75
Social
security Capabilities
0.50
0.25
Digital
services Openness
Fiscal
and Integrity
financial
management
Regulation Human
Crisis resource
management management
Switzerland is ranked Switzerland scores well on regulation (ranked
5th overall), particularly the stakeholder
12th overall on the Index engagement and impact assessment themes.
and achieves the top Its crisis/risk management indicator scores are
position for fiscal and also above average.
financial management. Switzerland’s attributes scores are all on or
above average.
Switzerland performs well across all the fiscal
and financial management themes. It is worth Switzerland’s social security administration score
noting that Switzerland has topped the Global may warrant further investigation as the sole
Competitive Index, which is used for this metric used for this indicator suggests that there
indicator, for eight consecutive years. It also does could be opportunities to improve the efficiency
well in the OECD indexes used for this indicator. of the system.
Switzerland’s high performance may in part
https://inciseradarvis.cloudapps.digital/ 2/6
reflect the government’s decision over a decade Switzerland’s scores for the digital services and
ago to overhaul its accounting system because policy making indicators are also less strong
of growing financial management demands. compared with other countries. On the latter,
Switzerland could focus initially on improving its
scores for the quality of advice, strategic policy
direction and coordination themes, learning from
the practices of the high scoring countries.
52 The International Civil Service Effectiveness Index
4.29 Turkey
5/26/2017 Radar chart
Tax
administration
Policy
making Inclusiveness
0.75
Social
security Capabilities
0.50
0.25
Digital
services Openness
Fiscal
and Integrity
financial
management
Regulation Human
Crisis resource
management management
Turkey is ranked 25th overall The main core functions where Turkey’s
performance is relatively weaker are regulation
on the Index and its position and HRM. On the latter, a focus on the
rises to 10th when scores are meritocracy of recruitment theme could be
adjusted for GDP per capita. beneficial, although it is ranked 10th for the
metric on the use of formal examination systems
Turkey achieves the top position for the crisis/ to hire staff. Turkey’s regulation indicator scores
risk management indicator, scoring consistently show significant variation in relative performance.
well across all themes. It comes joint top for the It could focus initially on the impact assessment
integrated risk planning theme (i.e. the extent and evaluation thematic metrics, learning from
to which disaster risk is incorporated in national best practice in the high scoring countries.
policy planning). Turkey also scores well on Turkey’s performance against all the attribute
risk monitoring, demonstrating a consistent indicators merits further2/6analysis, especially the
and systematic approach to reporting. Turkey’s
https://inciseradarvis.cloudapps.digital/
openness indicator score which is low relative to
performance against this indicator reflects its well other countries.
recognised leadership and experience of disaster
risk planning and management.
Turkey’s social security administration score
is also high (ranked 6th) suggesting that
administrative costs as a proportion of sector
expenditure are relatively low.
The International Civil Service Effectiveness Index 53
4.30 United Kingdom
5/26/2017 Radar chart
Tax
administration
Policy
making Inclusiveness
0.75
Social
security Capabilities
0.50
0.25
Digital
services Openness
Fiscal
and Integrity
financial
management
Regulation Human
Crisis resource
management management
The UK is ranked 4th overall The UK scores highly on regulation as well
(ranked 2nd overall), coming top on the impact
on the Index. It is ranked assessment theme, and on tax administration
top for policy making, (ranked 5th overall), with consistently good
openness and social scores across all metrics.
security administration. The main core function where further attention
may be warranted is crisis/risk management
The UK is currently one of two countries where
where the UK’s performance is just above
data is available for all 76 InCiSE metrics.
average. On digital services, the UK scores
On policy making, the UK performs particularly relatively highly for the cross-border mobility
well on the quality of policy advice theme, of services theme (ranked 4th overall) but less
including whether it is evidence based. For the well against other themes considered. The UK’s
openness indicator, the UK scores highly on
https://inciseradarvis.cloudapps.digital/
scores for the integrity and
2/6
capabilities indicators
two themes – government data availability and may also benefit from further analysis, learning
accessibility, plus government data impact – from the leading countries.
coming top in two of the metrics used. It also
scores well on the public consultation theme.
The UK’s social security administration ranking
suggests that system costs as a proportion of
sector expenditure are comparatively low.
54 The International Civil Service Effectiveness Index
4.31 United States of America
5/26/2017 Radar chart
Tax
administration
Policy
making Inclusiveness
0.75
Social
security Capabilities
0.50
0.25
Digital
services Openness
Fiscal
and Integrity
financial
management
Regulation Human
Crisis resource
management management
The USA is ranked 10th overall The USA’s attributes scores are all above
average, although its performance for the
on the Index. The country’s capabilities indicator (ranked 12th overall)
strongest indicator score is could merit further analysis, particularly the
for policy making (ranked a core capabilities theme.
close 2nd).
Within the policy making indicator the USA
scores particularly well on the quality of advice
theme and metrics, including whether it is
evidence based, also on the coordination of
policy proposals.
The USA performs less well relative to other
https://inciseradarvis.cloudapps.digital/ 2/6
countries on fiscal and financial management.
A focus on the economic appraisal and
evaluation themes and metrics could be
beneficial, learning from the best practices of
the high scoring countries.
The International Civil Service Effectiveness Index 55
Chapter 5: Next steps
and conclusion
5.1 Next steps b) Refining the InCiSE framework
Framework tasks will include: exploring the
InCiSE is a long term project and the founding
potential to add new civil service functions (for
organisations have committed to supporting
example, security, foreign affairs, defence and
its development for a further four years. This
justice); ensuring each indicator in the framework
will include publishing an annual report and
is a unique concept, to avoid duplication or
developing an interactive website. An International
overlap between the themes being measured;
Advisory Panel has also been established to
and ensuring that framework does not favour
provide strategic advice to the project.
certain types of systems.
As described earlier, this initial Index is a pilot
One issue emerging from the InCiSE pilot is the
only which requires further work to refine the
need to look at additional ways of capturing the
methodology and make the data more robust.
strengths that can arise from federal government
The main issues that the project will focus on
systems and ensure that the framework
over the next 12 months are set out below.
can measure and compare the respective
a) Strengthening data collection core functions.
Tasks will include: determining how best to Changes to the InCiSE framework will be
collect data on the 5 omitted indicators (staff determined mainly through user feedback and
engagement, innovation, IT, procurement, discussions with data owners to learn lessons
finance); exploring how data collection for from their own experiences. The International
existing indicators can be strengthened, Advisory Panel will also be consulted.
including thematic gaps; expanding data sources
and looking at ways of avoiding over reliance c) Expanding country coverage
on a single survey; reducing instances of public InCiSE will explore the scope to expand the
sector performance being used as a proxy for Index’s current country coverage over time,
civil service performance; and cross-country including the potential to include non-OECD and
applicability issues. developing countries. Country coverage in future
will largely be determined by feedback from
Complementary work by other institutions may
countries on the usefulness of having their own
help to fill some data gaps over time. Additional
set of indicators, as well as the availability of data
data collection, for example through the
to produce results.
commissioning of specific country surveys, could
also help to provide valuable insights and the Several countries were excluded from the pilot
project will actively support this work. Index because they had fewer than 75% of
the metrics available. However, three countries
(Bulgaria, Romania and Croatia) had over 70% of
data available. A small increase in data collection
may help them meet the threshold for inclusion in
the next Index and this issue will be explored.
56 The International Civil Service Effectiveness Index
d) Increasing InCiSE partners and network 5.2 Conclusion
The founding institutions will actively encourage
The InCiSE pilot Index has already brought
more partners to join them and support the
together a rich volume of data and insights from
development of the InCiSE Index over time,
many international sources. Its launch provides
through advice, expertise and funding. The
an important opportunity to stimulate wide-
project’s new International Advisory Panel will
ranging discussions at country level, as well as
also play a key role in promoting InCiSE and
globally, about civil service effectiveness issues
encouraging more partners.
generally, as well as the relevance and usefulness
The Blavatnik School of Government will host of some data.
an international conference in September 2017
Feedback is vitally important to help improve
to discuss the results of the pilot Index with a
InCiSE and the project team would welcome
wide range of interested players, as well as the
responses from country governments, as well
future direction of the project. This event will
as policy and learning networks, think-tanks and
additionally provide a key opportunity to build
academia. InCiSE contact details can be found
support for InCiSE and increase its network.
at the front of this report.
The InCiSE project team will continue to
coordinate closely with other institutions who are
engaged in similar efforts to measure civil service
effectiveness. The team will also coordinate
closely with the many organisations who have
made a vital contribution to InCiSE so far by
permitting use of their research or survey data.
Continued collaboration will be essential to help
strengthen InCiSE in the coming years.
The International Civil Service Effectiveness Index 57
Appendices
Annex A: InCiSE indicator definitions
Annex B: InCiSE indicator data sources
Annex C: InCiSE country rankings data
Annex D: InCiSE indicator scores summary by country
Annex E: Summary graphs of country scores
and rankings for each indicator
Annex F: Reader’s guide
58 The International Civil Service Effectiveness Index
Annex A: InCiSE indicator
definitions
Core functions Tax administration: The efficiency and
effectiveness of tax collection (at the central/
Policy making: The quality of the policy making federal level).
process, including how policy is developed and
coordinated across government and monitored Social security administration: The efficiency
during implementation. and effectiveness of social security administration
(at the central/federal level).
Fiscal and financial management (FFM):
The quality of the budgeting process and the Digital services: The user-centricity,
extent to which spending decisions are informed transparency and cross-border mobility of
through economic appraisal and evaluation. digitally-provided public services and the
availability of ‘key enablers’.
Regulation: The extent and quality of regulatory
impact assessments and the degree of
Attributes
stakeholder engagement involved in them.
Integrity: The extent to which civil servants
Crisis/risk management: The effectiveness with
behave with integrity, make decisions impartially
which the government engages the whole of
and fairly, and strive to serve both citizens
society to better assess, prevent, respond to and
and ministers.
recover from the effects of extreme events.
Openness: The regular practice and degree
Procurement: The extent to which the
of consultation with citizens to help guide
procurement process is efficient, competitive,
decision-making and extent of transparency
fair, and pursues value for money.
in decision-making.
Human resource management (HRM): The
Capabilities: The extent to which the workforce
meritocracy of recruitment and extent to which
has the right mix of skills.
civil servants are effectively attracted, managed,
and developed. Inclusiveness: The extent to which the civil
service is representative of the citizens it serves.
Information technology: The extent to
which civil servants have the digital tools to Staff engagement: Staff levels of pride,
work efficiently. attachment and motivation to work for their
organisation.
Finance: The extent to which operations are
supported by well-managed, efficient finance Innovation: The degree to which new ideas,
systems, particularly on the alignment of finance policies, and ways of operating are able to
with the business strategy and the level of civil freely develop.
servant satisfaction with finance support.
The International Civil Service Effectiveness Index 59
Annex B: InCiSE indicator
data sources
Country coverage
Indicator Data source and frequency Data content
Policy making Bertelsmann Sustainable 41 countries. Updated The SGIs assess three
Governance Indicators annually. pillars of governance:
(SGI), 2016. policy; performance; and
democracy.
Fiscal and financial The World Economic 138 countries. Updated The GCI combines 114
management Forum (WEF) Global annually. indicators (grouped into
Competitiveness Index 12 pillars) that capture
(GCI), 2016. concepts that matter for
productivity and long-
term prosperity.
OECD medium-term 32 countries. Last The OECD indexes cover
budgeting index, (2012) survey 2012. Next date range of public financial
and performance unknown. management areas.
budgeting index, (2011).
Regulation OECD Indicators of 34 countries. Updated The OECD survey
Regulatory Policy and every 3-4 years. assesses 3 aspects of
Governance, 2014. the process towards
creating effective
regulation.
Crisis/risk UN Hyogo Framework Covers 168 countries, Country reports track
management for Action (HFA) 2005- rolled out gradually over progress towards
2015, (2015). 10 years from 2005. implementing UN
priorities for disaster risk
reduction action.
Human resource Quality of Government 159 countries. Last The QoG Survey gathers
management (QoG) Expert Survey survey 2015. data on structure and
Data, University of behaviour of public
Gothenburg, Sweden, administrations.
2015.
60 The International Civil Service Effectiveness Index
Country coverage
Indicator Data source and frequency Data content
Tax administration OECD Tax Administration 56 countries. Repeat The OECD survey
2015 survey, (2013 planned mid 2017. provides comprehensive
data). 190 countries. assessment of tax
The World Bank Doing administration systems.
Updated annually.
Business Index (DBI), The DBI measures
2016. regulations affecting
11 areas of the life of a
business.
Social security European Commission 33 European countries. Eurostat data includes
administration data (via Eurostat), 2014. Updated annually. administration costs
as a proportion of total
expenditure on social
security. This data is
used to capture overall
efficiency of social
security administration
systems.
Digital services European Commission 33 European countries. The E-Government
E-Government Updated annually. Benchmark assesses
Benchmark, 2014/15. quality of digital service
delivery across four key
areas.
Integrity Transparency GCB: 100 countries. GCB: corruption
International Global Updated annually. perceptions survey.
Corruption Barometer GCI: 138 countries. CGI: See above.
(GCB), 2013. Updated annually. QoG: See above.
World Economic QoG: 159 countries. GaaG: wide ranging
Forum (WEF) Global Last survey 2015. set of indicators on
Competitiveness Index
GaaG: 35 countries. governance and public
(GCI), 2016.
Updated every five years. sector performance.
Quality of Government
(QoG) Expert Survey
Data, University of
Gothenburg, Sweden,
2015.
OECD “Government at
a Glance” (GaaG) data,
2013 and 2014.
The International Civil Service Effectiveness Index 61
Country coverage
Indicator Data source and frequency Data content
Openness a) World Justice Project: a) 102 countries. Last a) The OGI measures
Open Government survey 2015. government openness
Index (OGI), 2015. b) 193 countries. based on citizen
b) UN E-participation Updated biennially perceptions and
Index (EPI), 2016. experiences.
c) 41 countries. Updated
c) Bertelsmann SGIs, annually b) The EPI measures
2016. the use of online
d) 92 countries. Updated
services to support
d) World Wide Web annually.
government
Foundation: Open e) 122 countries. information sharing
Data Barometer Updated annually. and engagement
(ODB), 2015.
f) 29 countries. Last with citizens.
e) The Open Knowledge report 2014. c) SGIs: see above.
Foundation: Open
Data Index (ODI), d) The ODB ranks
2015. government open
data policies,
f) OECD OURdata
planning, delivery and
index, 2014.
impact.
e) The ODI measures
the status of open
government data.
f) The OURdata Index
measures government
efforts to implement
the G8 Open Data
charter.
Capabilities OECD Programme 24 countries. Data PIACC survey measures
for the International collected 2011/12. adults’ proficiency in key
Assessment of Adult information-processing
Competencies (PIAAC) skills – literacy, numeracy
survey data, 2012. and problem solving;
and gathers data on how
adults use their skills at
home, work and in the
wider community.
Inclusiveness OECD Government at See above. See above.
a Glance (GaaG) and
Quality of Government
(QoG) data, 2015.
62 The International Civil Service Effectiveness Index
Annex C: InCiSE country
rankings data
The table below covers three sets of country rankings: an overall ranking based on
a composite score; rankings against each set of core functions and attributes; and then
a ranking against each of the indicators.
Table 2: Country rankings (without GDP per capita adjustments)
Country Fiscal and Human Social
code Policy financial Crisis/risk resource Tax security Digital
3-digit Composite Integrity Openness Capabilities Inclusiveness Attributes making man. Regulation man. man. admin. admin. services Functions
AUS 3 9 7 6 5 5 7 19 3 4 7 10 22 4 4
AUT 16 10 13 29 12 14 28 21 17 26 22 9 11 2 15
BEL 13 11 20 10 9 15 13 30 10 =8 4 18 25 18 13
CAN 1 4 9 3 3 2 5 6 4 =8 3 20 14 7 1
CHL 22 21 19 11 20 20 9 17 31 =8 24 23 24 20 23
CZE 28 27 24 25 26 25 27 11 12 20 28 31 23 28 28
DNK 11 5 4 17 23 8 3 10 19 28 10 3 26 3 11
EST 7 15 17 15 28 19 20 7 6 =8 18 1 3 1 3
FIN 5 6 5 2 4 3 4 8 21 2 14 14 19 5 8
FRA 17 12 12 22 27 18 18 13 18 27 12 19 28 16 20
DEU 24 8 15 9 15 13 26 18 7 31 13 24 27 19 26
GRC 29 28 29 23 6 26 29 28 29 21 25 27 13 29 30
HUN 30 25 30 30 31 31 24 31 20 24 30 25 2 30 29
IRL 20 16 26 20 13 22 16 20 28 =8 1 4 31 21 21
ITA 27 29 22 31 24 27 21 29 22 17 20 26 16 25 27
JPN 15 18 14 1 29 12 17 16 26 =8 2 29 20 13 17
KOR 9 20 11 16 25 17 12 5 11 =8 5 13 17 8 7
MEX 19 31 23 27 30 30 14 15 1 29 27 21 7 14 12
NLD 14 14 6 7 18 9 23 2 25 7 15 6 29 15 19
NZL 2 1 2 4 7 1 6 4 9 22 6 15 12 9 5
NOR 8 3 3 8 19 4 8 12 24 23 8 2 10 12 9
POL 21 19 21 19 1 16 11 22 16 30 19 30 15 26 24
PRT 26 24 27 24 17 24 22 26 30 25 29 22 4 6 25
SVK 31 30 28 28 21 29 30 25 14 = 18 31 28 18 31 31
SVN 23 23 25 21 2 21 31 23 13 3 23 16 5 22 22
ESP 18 22 18 26 22 23 15 27 23 = 18 21 7 9 17 16
SWE 6 2 10 13 16 6 10 3 15 5 16 8 8 11 6
CHE 12 7 16 5 11 11 25 1 5 6 9 11 30 27 14
TUR 25 26 31 18 14 28 19 9 27 1 26 12 6 24 18
GBR 4 17 1 14 10 7 1 14 2 16 11 5 1 23 2
USA 10 13 8 12 8 10 2 24 8 =8 17 17 21 10 10
The International Civil Service Effectiveness Index 63
Annex D: InCiSE indicator
scores summary by country
The table below covers three sets of country scores: an overall composite score; scores against each
set of core functions and attributes; and then individual scores for each of the indicators.
Table 3: Country indicator scores (without GDP per capita adjustments)
Country Fiscal and Human Social
code Policy financial Crisis/risk resource Tax security Digital
3-digit Composite Integrity Openness Capabilities Inclusiveness Attributes making man. Regulation man. man. admin. admin. services Functions
AUS 0.91 0.74 0.9 0.66 0.68 0.84 0.88 0.55 0.91 0.92 0.73 0.8 0.65 0.86 0.93
AUT 0.56 0.74 0.76 0.22 0.51 0.63 0.36 0.53 0.41 0.59 0.42 0.81 0.83 0.87 0.53
BEL 0.6 0.72 0.55 0.56 0.55 0.61 0.75 0.1 0.59 0.77 0.85 0.69 0.6 0.67 0.59
CAN 1 0.85 0.86 0.72 0.93 0.95 0.94 0.86 0.84 0.77 0.88 0.68 0.79 0.82 1
CHL 0.44 0.59 0.58 0.51 0.37 0.51 0.84 0.6 0 0.77 0.37 0.59 0.6 0.6 0.42
CZE 0.16 0.29 0.43 0.25 0.22 0.21 0.38 0.69 0.5 0.72 0.21 0 0.62 0.27 0.16
DNK 0.73 0.84 0.93 0.44 0.25 0.75 0.98 0.72 0.3 0.57 0.69 0.93 0.44 0.86 0.71
EST 0.81 0.66 0.65 0.45 0.16 0.52 0.57 0.81 0.69 0.77 0.54 1 0.99 1 0.95
FIN 0.87 0.83 0.91 0.73 0.8 0.93 0.97 0.78 0.26 0.98 0.58 0.76 0.69 0.85 0.82
FRA 0.5 0.67 0.77 0.28 0.16 0.54 0.6 0.66 0.34 0.58 0.68 0.68 0.38 0.69 0.48
DEU 0.4 0.74 0.68 0.59 0.45 0.67 0.42 0.56 0.65 0 0.67 0.52 0.41 0.62 0.29
GRC 0.04 0.19 0.24 0.27 0.65 0.18 0.29 0.27 0.07 0.71 0.31 0.32 0.79 0.05 0.01
HUN 0 0.38 0.02 0.21 0 0 0.49 0 0.29 0.63 0.1 0.48 0.99 0 0.05
IRL 0.46 0.66 0.42 0.34 0.5 0.47 0.66 0.54 0.1 0.77 1 0.93 0 0.59 0.47
ITA 0.21 0.18 0.47 0 0.25 0.13 0.53 0.2 0.25 0.74 0.49 0.42 0.77 0.46 0.28
JPN 0.57 0.65 0.74 1 0.16 0.68 0.61 0.6 0.14 0.77 0.99 0.26 0.68 0.72 0.52
KOR 0.78 0.6 0.77 0.44 0.22 0.55 0.8 0.87 0.55 0.77 0.82 0.77 0.71 0.82 0.88
MEX 0.47 0 0.45 0.24 0.14 0.07 0.71 0.64 1 0.54 0.22 0.68 0.85 0.7 0.68
NLD 0.58 0.67 0.9 0.62 0.44 0.74 0.5 0.94 0.18 0.84 0.58 0.86 0.11 0.7 0.51
NZL 0.95 1 0.98 0.7 0.65 1 0.93 0.88 0.63 0.69 0.77 0.71 0.81 0.8 0.91
NOR 0.81 0.87 0.96 0.6 0.41 0.85 0.84 0.66 0.23 0.68 0.73 0.98 0.83 0.78 0.78
POL 0.44 0.6 0.52 0.37 1 0.61 0.82 0.5 0.42 0.53 0.51 0.18 0.77 0.46 0.37
PRT 0.31 0.42 0.4 0.26 0.44 0.31 0.51 0.34 0.06 0.61 0.15 0.61 0.93 0.83 0.33
SVK 0 0.11 0.31 0.24 0.32 0.09 0.22 0.37 0.47 0.73 0 0.3 0.69 0 0
SVN 0.44 0.56 0.42 0.3 0.93 0.51 0 0.46 0.48 0.93 0.37 0.7 0.92 0.5 0.42
ESP 0.49 0.58 0.59 0.24 0.3 0.44 0.68 0.31 0.24 0.73 0.45 0.85 0.84 0.68 0.53
SWE 0.86 0.89 0.82 0.48 0.45 0.77 0.83 0.92 0.47 0.9 0.57 0.84 0.84 0.79 0.89
CHE 0.61 0.77 0.66 0.66 0.53 0.71 0.47 1 0.7 0.88 0.71 0.79 0.01 0.33 0.56
TUR 0.37 0.29 0 0.4 0.45 0.1 0.58 0.72 0.12 1 0.23 0.77 0.86 0.46 0.52
GBR 0.91 0.65 1 0.46 0.54 0.77 1 0.66 0.94 0.76 0.69 0.9 1 0.49 0.96
USA 0.74 0.67 0.86 0.48 0.62 0.73 0.98 0.44 0.65 0.77 0.55 0.7 0.68 0.79 0.73
64 The International Civil Service Effectiveness Index
Annex E: Summary graphs
of country scores and
rankings for each indicator
Core functions
a) Policy making
This indicator currently has four themes: the quality of policy advice; the role of civil servants in setting
strategic policy direction; policy proposal coordination across government; and monitoring policy
implementation. Some proxy metrics have been used for measuring the quality of policy advice. A fifth
theme, assessing the timeliness and accuracy of policy delivery, will be added when data becomes
available. All data for this indicator is drawn from the Bertelsmann Sustainable Governance Indicators (SGI).
The International Civil Service Effectiveness Index 65
b) Fiscal and financial management (FFM)
This indicator has four themes: economic appraisal; economic evaluation; medium-term budgeting
processes; and performance budgeting. Proxy measures have been used for some aspects.
Both the economic appraisal and evaluation themes are measured by the World Economic Forum
Global Competitiveness Index (GCI). The two budgeting themes are measured by two relevant
OECD indexes.
66 The International Civil Service Effectiveness Index
c) Regulation
This indicator has three themes relating to regulation policies and management: method, use and
quality of regulatory impact assessments; stakeholder engagement; and evaluation work. The sole
data source is the OECD’s Indicators of Regulatory Policy and Governance.
The International Civil Service Effectiveness Index 67
d) Crisis/risk management
This indicator has five themes regarding disaster risk reduction and management issues most relevant
for the civil service: integrated risk planning; risk monitoring; public information and awareness raising;
international cooperation and risk coordination; and post-disaster assessment methodology. A sixth
theme on preparedness for disaster response will be added when data becomes available. The data
source for all themes is the UN Hyogo Framework for Action.
68 The International Civil Service Effectiveness Index
e) Human resource management (HRM)
This indicator currently measures two themes: the extent to which civil service recruitment systems
are meritocratic; and attracting and retaining talent. InCiSE hopes to assess four additional themes
when data is available: talent deployment; performance management; quality of learning and
development; and level of civil servant satisfaction with HR services. For the two themes measured,
the data source is the Quality of Government survey.
The International Civil Service Effectiveness Index 69
f) Tax administration
This indicator currently measures three themes: the overall efficiency of tax collection; the extent to
which services are user focused; and the extent and the quality of digital provision. Two additional
themes will be assessed when data becomes available: the prevention of tax evasion; and the level of
tax gap measurement. The existing data sources are the OECD’s Tax Administration Survey and The
World Bank’s Doing Business Index (DBI).
70 The International Civil Service Effectiveness Index
g) Social security administration
This indicator is currently measured via one theme/metric only: assessing administration costs as
a proportion of total expenditure on social security, to capture the overall efficiency of the system.
Additional metrics are planned for future reports as more data becomes available, focused on
the extent to which services are user focused, the extent and quality of digital provision, and the
prevention of fraud. The existing data source is the European Commission (Eurostat). Estimated data
was used for most non-EU countries.
The International Civil Service Effectiveness Index 71
h) Digital Services
This indicator measures four themes: the extent to which services are user focused; the transparency
of services; cross-border mobility of services; and availability of key enablers (technical advances
made possible through, for example, infrastructure investment). It does not assess all the services
which governments typically provide digitally. The sole data source is the European Commission’s
E-Government Benchmark. Estimated data was used for most non-EU countries.
72 The International Civil Service Effectiveness Index
Attributes
a) Integrity
This indicator covers 6 themes: corruption level perceptions; adherence to rules and procedures;
work ethics; fairness and impartiality; striving to serve citizens and ministers; processes in place
to preserve integrity and prevent conflicts of interest. There are four data sources: Transparency
International’s Global Corruption Barometer (GCB); the World Economic Forum (WEF) Global
Competitiveness Index (GCI); the Quality of Government (QoG) Expert Survey Data; and the OECD’s
“Government at a Glance” (GaaG) data.
Fifteen countries have some missing data, partly because of the large number of metrics for this
indicator (16), and so estimated data has been used.
The International Civil Service Effectiveness Index 73
b) Openness
This indicator has six themes: the degree and quality of consultation with society; the existence and
quality of complaint mechanisms; government data availability and accessibility; government data
impact; the right to information; and the publication of laws. There are six data sources: the World
Justice Project’s Open Government Index (OGI); the UN E-participation Index (EPI); Bertelsmann
SGIs; the World Wide Web Foundation: Open Data Barometer (ODB); The Open Knowledge
Foundation: Open Data Index (GODI); and the OECD’s OURdata index.
74 The International Civil Service Effectiveness Index
c) Capabilities
This attribute currently measures only two themes: core capability (eg. problem solving, numeracy
and literacy skills); and the educational attainment of the workforce. InCiSE aims to broaden
the themes for this indicator once data becomes available to include leadership, commercial,
analytical and digital capabilities. The sole data source is the OECD Programme for the International
Assessment of Adult Competencies (PIAAC) survey. A high level of estimated data was used for
some countries.
The International Civil Service Effectiveness Index 75
d) Inclusiveness
This indicator currently has only two themes: the proportion of women represented in the civil service;
and ethnic/religious minority representation. InCiSE hopes to broaden the data coverage over time
once data becomes available on other key representation categories. There are two data sources: the
OECD’s Government at a Glance (GaaG) survey; and the Quality of Government (QoG) survey.
76 The International Civil Service Effectiveness Index
Annex F: Reader’s guide
F.1 Abbreviations
InCiSE International Civil Service Effectiveness
OECD Organisation for Economic Co-operation and Development
UN United Nations
VFM Value for money
GCI Global Competitiveness Index
WEF World Economic Forum
QoG Quality of Government
SGI Sustainable Governance Indicators
EPI E-participation index
ODI Open Data Index
ODB Open Data Barometer
PIAAC Programme for the International Assessment of Adult Competencies
GaaG Government at a Glance
SDG Sustainable Development Goals
OURdata Open, Useful, Reusable data
The International Civil Service Effectiveness Index 77
F.2 Country codes
The following ISO country codes are used in some tables of this report1:
ISO alpha- ISO alpha-
Country 3 digit country code Country 3 digit country code
Australia AUS Republic of Korea KOR
Austria AUT Mexico MEX
Belgium BEL Netherlands NLD
Canada CAN New Zealand NZL
Chile CHL Norway NOR
Czechia CZE Poland POL
Denmark DNK Portugal PRT
Estonia EST Slovakia SVK
Finland FIN Slovenia SVN
France FRA Spain ESP
Germany DEU Sweden SWE
Greece GRC Switzerland CHE
Hungary HUN Turkey TUR
Ireland IRL United Kingdom GBR
Italy ITA United States of America USA
Japan JPN
1
https://www.iso.org/obp/ui/#home
The International Civil Service Effectiveness
(InCiSE) Index project is a collaboration between
the Blavatnik School of Government and the
Institute for Government. The project has been
supported by the UK Civil Service and is funded
by the Open Society Foundations.
The International
Civil Service
Effectiveness
(InCiSE) Index
Technical Report
Users are free to copy, download and print InCiSE content and findings for their own use. Excerpts from
InCiSE reports and publications may also be used, but must be appropriately referenced. Please cite any
uses of this content as: “International Civil Service Effectiveness (InCiSE) Index, 2017”.
The Index is comprised of a variety of other data sources. Reuse is not applicable to data subject to
intellectual property rights of third parties. Please refer to other organisations’ corresponding websites
and data licensing restrictions to ensure compliance with their data limitations. Details of data sources can
be found in Chapter 4 of this technical report. Every effort has been made to adhere to third party data
regulations.
Please contact InCiSE@instituteforgovernment.org.uk for any queries.
Acknowledgements: Our thanks go to those who have given their time to shape this
inaugural pilot publication. This has included substantial contributions made by academics,
think-tanks, international organisations (particularly the OECD), and civil servants past
and present. Our thanks also to those organisations who have allowed use of their data in
the Index.
The International Civil Service Effectiveness Index | Technical Report 1
Contents
Chapter 1: Introduction 3
1.1 Why we need civil service effectiveness indicators 3
1.2 What InCiSE is aiming to do 3
1.3 Structure of this report 4
Chapter 2: Defining the civil service 5
2.1 Alternative definitions of the civil service 5
2.2 Scope of the InCiSE Index 11
Chapter 3: Defining the measurement framework 12
3.1 Purpose of the framework 12
3.2 Principles for the framework 12
3.3 Approach to assessing performance 13
3.4 The InCiSE framework 15
3.4.1 Definitions and justification of the attributes and functions 18
Chapter 4: Measuring against the framework 27
4.1 Data availability 27
4.2 Data underpinning the indicators 31
4.2.1 Functions 31
4.2.2 Attributes 44
Chapter 5: Index country coverage 53
5.1 Countries included in the Index 53
5.2: Imputation method 57
5.3: Imputation approach for indicators 59
5.3.1 Functions 59
5.3.2 Attributes 60
Chapter 6: Index results 62
6.1 Normalisation 62
6.2 Reporting of results 64
2 The International Civil Service Effectiveness Index | Technical Report
Chapter 7: Composite 68
7.1 Strengths and weaknesses of composites 68
7.2 The InCiSE composite 69
Chapter 8: Sensitivity Analysis 72
8.1 Introduction to sensitivity analysis 72
8.2 Sensitivity analysis 1: Adjusting for GDP per capita 72
8.3 Sensitivity analysis 2: Aggregation method 75
8.4 Sensitivity analysis 3: Exclusion of input data 78
8.4.1 Subjective assessment 78
8.4.2 Out of date data 81
8.4.3 Public sector proxy 83
8.5 Sensitivity analysis 4: Imputation method 85
Chapter 9: Next steps 87
9.1 Identified limitations of the Index 87
9.2 Priorities for the next 12 months 88
Appendices 90
Annex A: Data limitations 91
Annex B: Reference list 93
Annex C: Reader’s Guide 96
C.1 Abbreviations 9
7
C.2 Country codes 98
The International Civil Service Effectiveness Index | Technical Report 3
Chapter 1: Introduction
This chapter provides a brief background to the InCiSE Index, and sets out the structure of
the report.
1.1 Why we need civil service The creation of a new and concise set of civil
service effectiveness indicators would serve as:
effectiveness indicators
• An accountability tool: allowing citizens,
An effective civil service can play an important
government officials and politicians to
role in determining a country’s progress and
establish in a clear and concise way how
prosperity. But what constitutes an “effective
well their civil service is performing.
civil service” in the 21st Century? And once a
consensus has been reached on defining this, • A performance improvement tool: enabling
how do civil service leaders know whether their senior decision makers to see which
organisations are effective, and in which areas countries perform best in which areas,
are they performing more strongly than others? and learn from them.
The InCiSE Index seeks to help answer these
questions.
1.2 What InCiSE is aiming to do
A comprehensive set of international indicators
InCiSE aims to define “effectiveness” more
of civil service effectiveness does not currently
extensively than previously. It draws on a wide
exist. This subject area is also well recognised
range of existing international data sources
in academic, international and practitioner
and brings together a set of indicators each
communities as a highly complex area for
measuring a different dimension of civil service
analysis. This is partly because of data
effectiveness – and then produces a composite
limitations, different views on the definitions of
(an overall) score.
“civil service” and “effectiveness”, as well as
the need to take account of country context InCiSE has been developed following
factors when looking at performance issues. a literature review and in consultation
Nevertheless, there are many existing surveys with many experts, including academics
and data collections available globally that can from schools of government, think-tanks
be pulled together to provide a view on civil that monitor government effectiveness,
service effectiveness on an annual basis. international organisations, senior civil servants
(past and present) and subject experts.
4 The International Civil Service Effectiveness Index | Technical Report
InCiSE has also been the subject of an Including this introductory chapter, there are 9
independent, international peer review process. chapters of this report:
Three peer reviewers were selected: a senior
academic from a major European governance • Chapter 2 – Defining the civil service
research institute; a recently retired top civil considers the object of assessment, the
servant with practical experience of civil service “civil service” and establishes a definition
performance issues; and a senior governance of the unit of measurement that allows for
expert with a distinguished career in two major international comparison.
international institutions. Country results were • Chapter 3 – Defining the measurement
provided in an anonymous form for the review. framework sets out a common approach for
Between them, the peer reviewers were asked assessing the effectiveness of a civil service.
to examine the measurement framework of
‘effectiveness’, the methodology and approach • Chapter 4 – Measuring against the
used to produce the indicators, and the data framework describes the data included in the
being used. Index and the weighting of data.
The International Civil Service Effectiveness Index • Chapter 5 – Index country coverage lists the
(InCiSE) project is a collaboration between the countries included in the Index, and explains
Blavatnik School of Government and the Institute our approach to handling missing data.
for Government. The project has been supported
• Chapter 6 – Index results describes how
by the UK Civil Service, and is funded by the
country scores are produced and how results
Open Society Foundations.
are presented.
Despite some current limitations, the Index
• Chapter 7 – Composite discusses the pros
has already brought together a rich volume of
and cons of producing a composite and
data. The launch of this pilot Index will provide
explains how the InCiSE composite Index is
an opportunity to engage with a wide variety of
produced.
stakeholders on a range of issues, stimulating
discussions and feedback. • Chapter 8 – Sensitivity analysis describes
some of the uncertainties associated with
1.3 Structure of this report the modelling process and the subjective
choices, and the consequent impact on the
This Technical Report on the InCiSE Index is Index results.
intended to describe the methodology, data,
and limitations of the approach used. The results • Chapter 9 – Next steps highlights limitations
of the Index can be found in the accompanying of the pilot Index and areas for future
Main Report. consideration to develop the Index.
Given the 2017 publication is a pilot release, this
technical paper will be updated as the Index is
developed and as feedback is received. Should
you have comment on this paper please email
InCiSE@instituteforgovernment.org.uk.
The International Civil Service Effectiveness Index | Technical Report 5
Chapter 2: Defining
the civil service
This chapter considers the object of assessment, the “civil service”, and establishes a
definition of the unit of measurement that allows for international comparison.
2.1 Alternative definitions of the In the 2001 version of the International Monetary
Fund’s Government Financial Statistics Manual1
civil service
and in Eurostat’s European System of Accounts2,
The scope and responsibilities of the civil service the classification of functions of government
varies across countries. Moreover, given that civil is used as an internationally-agreed statistical
services across the world provide a wide-range grouping of government expenses.
of functions and are organised in a variety of
A second definition could take a national
ways, there is a need to establish exactly which
accounts perspective and again, taking a narrow
parts are being assessed, and how.
view, identify civil service entities as those which
In defining the civil service there are a number are owned by government and whose financial
of approaches that can be taken. First, the reporting places them within the System of
civil service can be defined by function. Taking National Accounts (SNA) category of General
a narrow view this could refer to the central, Government. The OECD have undertaken work
“upstream” agencies which set policy direction to establish a terminology and a new definition
and procedural regulation for the “downstream” of what has been called the “public domain”.3
agencies. This could include imposing regimes The new classification is now consistent with the
of transparency or accountability on them and system of national accounts (SNA).
marshalling budgetary and human resources for
them. Those upstream agencies also produce
policy outputs of their own for regulating
economic behaviour and through changing
tax and revenue, expenditure, state-owned-
enterprises and investment policies. A broader
functional perspective encompasses those
agencies which are responsible for service
delivery (although noting that services can be
commissioned or funded, as well as provided
by government).
1 IMF, 2001, Government Finance Statistics Manual (Washington: International Monetary Fund).
2 Eurostat, 2007, Manual on Sources and Methods for the Compilation of COFOG Statistics: Classification of the Functions of
Government (COFOG), Eurostat Methodologies and Working Papers (Luxembourg: Office for Official Publications of the European
Communities).
3 Pilichowski, E. and E. Turkisch (2008). Employment in Government in the Perspective of the Production Costs of Goods and Services in
the Public Domain. OECD, Paris; OECD (2009), Measuring Government Activity, Paris, OECD.
6 The International Civil Service Effectiveness Index | Technical Report
The darkest shaded areas in Table 1 delineate An even broader SNA perspective would
those institutions which might be regarded pull in state-owned enterprises selling goods
as part of the civil service, namely the or services at an economically significant
administrative units of central governments price, and financial and non-financial public
(and state governments, if a wider view of what corporations – to the extent that these are
constitutes the civil service is taken) and the controlled by government as set out in the SNA.
typically very small (in terms of employment, Although this category would not be included
not fiscal impact) social security funds at each as part of the civil service by practitioners or
level of government. Entities in government often researchers, some have argued that the core
lack a legal personality and are hence unable public sector, broader than the civil service
to own assets and incur liabilities which are as usually understood, should include market
reliant on central financial authorities. As Table producers and non-profits which are to all intents
1 shows, for the SNA, General Government and purposes entirely reliant on the public sector
agencies include Ministries and departments in as monopsony customer, and private enterprises
central government, along with state and local which have been granted a distinctive and
governments if taking a wider view. Since it is an statutorily privileged market position.
accounting and not a management perspective,
This is on the basis that while there may be little
project implementation unit arrangements are
direct public funding for these agencies there is
covered to the extent that they are included in often an assumption that government will meet
the budget (even if using consultants outside any implicit contingent liability that arises, with
of any civil service regime). A less often noted the probability that government would, in the
but occasionally significant group of agencies event of major operational failure, underwrite
within General Government encompasses the debts of these entities. They are thus
those organisations that are largely funded and underpinned by an implicit guarantee.
controlled by state or provincial government
but not owned by government. In some
countries, this includes schools and hospitals
which are fully funded by government and
where government determines the employment
and financial regimes, but where there is
an independent owner who is acting on the
government’s behalf, often a charity or other
not-for-profit.
The International Civil Service Effectiveness Index | Technical Report 7
Table 1: The public sector from a National Accounts perspective
Institutional domain How transactions are Examples
recorded in the national
accounts
The Public General Central Administrative units in Ministries and
“public sector government5 government central government departments in
domain”4 central government
All non-market non- Schools, hospitals,
profit institutions that are etc. that are
controlled and more than 50 largely funded
percent financed by central and controlled by
government units central government
but not owned by
government
State Administrative units in state Departments in
governments government states, provinces
All non-market non- Schools, hospitals,
profit institutions that are etc. that are
controlled and more than 50 largely funded
percent financed by state and controlled by
government units state or provincial
government but
not owned by
government
Local Administrative units in local Departments
governments government in counties,
municipalities
All non-market non- Schools, hospitals,
profit institutions that are etc. that are
controlled and more than largely funded
50 percent financed by local and controlled by
government units local government
but not owned by
government
Social All social security funds at Health fund,
security each level of government unemployment fund,
funds pension fund
Other public sector Market producers, controlled Publicly owned
by government, selling banks
goods or services at an Publicly owned
economically significant harbors, airports
price (“public enterprises”):
• Public financial (quasi-)
corporations
• Public non-financial
(quasi-) corporations
As defined by S.11 and S.12
in the SNA.
8 The International Civil Service Effectiveness Index | Technical Report
Table 1: The public sector from a National Accounts perspective
Private sector in the public domain Market producers, whose Profit or non-profit
indirect public funding private hospitals
comprises more than 50 accessible to
percent of total revenue: publicly insured
• Non-profit institutions clients
• Profit institutions
As defined by S.11, S.12 in
the SNA
Non-profit institutions Schools, hospitals,
serving households, etc. that are
financed more than 50 largely funded
percent by government, by government
but not controlled by but not owned
government: nor controlled by
• Non-profit institutions government
serving households
As defined by S.15 in the
SNA
Private enterprises with a Energy companies,
distinctive and statutorily local public transport
privileged market position: companies
• Private sector utilities National train
licensed to operate in company
very limited markets
(water, energy,
sewage, waste
disposal, post, but not
telecommunication)
Legal monopolies
As defined by S.11 in the
SNA
Source: Developed largely from Pilichowski and Turkisch6 with OECD7
Note: The darkest shaded area highlights those institutions which might be regarded as part of the civil service, namely
the administrative units of central governments. The medium shaded area covers those categories, in addition to the
darkest shaded area, considered part of General Government under the SNA. The lighter shaded area covers state-owned
enterprises selling goods or services at an economically significant price, and financial and non-financial public corporations
– to the extent that these are controlled by government as set out in the SNA.
4 This description is devised for this purpose and is not a recognized SNA term.
5 As defined by section S.13 in the System of National Accounts).
6 Pilichowski, E. and E. Turkisch (2008). Employment in Government in the Perspective of the Production Costs of Goods and Services in
the Public Domain. OECD, Paris.
7 OECD (2009), Measuring Government Activity, Paris, OECD.
The International Civil Service Effectiveness Index | Technical Report 9
The final approach explored was of a civil service This can present practical difficulties as, although
determined on the basis of employment regimes generally the civil service is understood to
(Figure 1). The most stringent definition limits the constitute a distinct body of staff within the
selected entities to those which are required to public sector, staff that are commonly referred
hire most employees under the civil service law, to as “civil servants” do not always have legally
excluding those using other legal employment distinct employment contracts. When it exists
regimes. formally, the essence of civil servant status is that
the legal basis for employment – the laws and
regulations that shape the nature of employment
contracts – is different from that found elsewhere
in the economy as defined by the general labour
law. It also is generally different from that found
elsewhere in the public sector, such as in the
health or education sectors or in state-owned
enterprises.
Figure 1: Typical legal employment regimes in general government
TOTAL PUBLIC EMPLOYMENT
SOE employees General Government
total civilian total subnational
Armed Forces central government government
total education
employees { Education Education
total health
employees { Health Health
total police { Police Police
Civilian Central Subnational
Government Government
excluding excluding
education, education,
health and health and
police police
often known as
“civil servants” { permanent
employees
permanent
employees
daily paid temporary temporary
employees employees employees
Source: World Bank 20078
8 World Bank (2007). The World Bank’s Administrative and Civil Service Reform Website. World Bank.
10 The International Civil Service Effectiveness Index | Technical Report
Historically, civil service employment was The three methods offered here for defining
not a formal agreement between two equal the centre of government based on functions,
parties, but rather a decision of the State, national accounts or employment regimes omit
with appointments made by an authorised other less useful criteria that could be applied,
public institution in accordance with the civil such as:
service law, with tight regulations that shape
the appointment process, and with many • A legal definition of the centre of government
constraints on dismissal. Today, civil service could comprise those entities that are
employment tends to share some features that created under the authority of the constitution
are typical of a voluntary arrangement between or by public law.12
an employer and employee in the private sector. • Alternatively, the centre of government could
There can be other arrangements, particularly be defined as the set of all entities that
in the health, education, military and police operate directly under the authority of the
sectors, that provide civil service-like protections political executive.
and responsibilities for public employees who
are not, in fact, civil servants. Subnational • Ownership is a further set of ideas for
government employment is considered a defining the institutional units that belong
separate, legally defined civil service in many to the public sector, drawing on accounting
countries.9,10 A number of organisations including standards for criteria concerning which body
the International Labour Organisations and the has the power to govern the financial and
OECD have considered the legal basis for civil operating policies of another entity.13
servant status.11
The problem with each of these alternative
conceptions is that they result in a large and
unwieldy group of agencies which, most
importantly, is rarely consistent with professional
or other working definitions.
9 Gow, J. I. and M. d. C. Pardo (1999), ‘Comparing Different Civil Services’, International Review of Administrative Sciences, 65 (4), 527-
550.
10 Cardona, F. (2000). Scope of Civil Services in European Countries: Trends and Developments. Seminar at the European Institute of
Public Administration. Maastricht, Sigma/OECD
11 SIGMA (1996a), Civil Service Legislation Contents Checklist – Sigma Paper No. 5, Paris, OECD.
12 Gill, D. (2002), ‘Signposting the Zoo – from Agencification to a More Principled Choice of Government Organisational Forms’, OECD
Journal on Budgeting, 2 (1), 27-80
13 Lienert, I. (2009). Where Does the Public Sector End and the Private Sector Begin? . IMF, Washington DC.
The International Civil Service Effectiveness Index | Technical Report 11
2.2 Definition in the InCiSE Index
The approach taken for the InCiSE Index is to define the scope of ‘civil services’ by outlining and
measuring performance on the core functions of civil services; the parts which can generally be
classified as civil service in every country. This approach leads to a focus on (i) functions which deliver
services or affect citizens directly and (ii) public management and policy functions carried out at the
centre of government.
Table 2 gives more detail about what is included and excluded in the InCiSE Index:
Table 2: Public sector areas included in the InCiSE framework
Parts/functions of the public sector Degree of inclusion in measurement framework
Civil service functions which deliver services A primary focus of the InCiSE Index with each
to citizens directly (e.g. tax and social security function assessed in an individual indicator.
administration at the central/federal level).
Central, public administration-type civil service A primary focus of the Index with each function
functions (e.g. fiscal management, policy making, assessed in an individual indicator.
regulation).
The ‘mission support’ functions which support these A primary focus of the Index.
core service delivery and central administration-type
functions.
The parts of civil services which direct and support Performance captured by indicators on the functions
the wider public sector on specific policy areas (e.g. above which cut across most of the policy areas
Health Ministry, Education Ministry, Environment governments typically deal with. For example
Ministry), but may not deliver public services to regulation, and policy making more generally, cut
citizens directly. across a number of policy areas including health,
education, environment, and so on. Performance
on specific policy areas not individually assessed
because:
-
Data is unlikely to exist which sufficiently isolates
the ‘oversight’ specific ministries provide over
different policy areas from the service delivery the
wider public sector provides in these areas; and -
Governments in all countries deal with a large
number of policy areas and there is a need to limit
the scope of the measurement framework.The wider public sector itself (e.g. workers in public Out of scope, although occasionally public sector
hospitals, schools and police departments). data is used to proxy for civil service performance.Local government. Out of scope. Whilst public servants working in
sub-national governments may in some countries be
technically classified as ‘civil servants’, the scope of
this Index is primarily concerned with the civil service
at central government level.
The unit of analysis of interest for the InCiSE Index is the civil service, not the public sector more
generally. Having said that, isolating civil service performance with currently available data is difficult,
particularly given the varying sizes and shapes of civil services internationally. This issue is discussed
further in Chapter 9 and Annex A which highlight some limitations of the Index.
12 The International Civil Service Effectiveness Index | Technical Report
Chapter 3: Defining the
measurement framework
Chapter 3 considers the purpose of designing a framework for the Index, the principles
specified, the framework derived and the justification for its components.
3.1 Purpose of the framework This framework, and the resultant Index, provides
civil services across the world with a powerful
The purpose of the framework is to define tool for (i) accountability, allowing citizens,
a common approach for assessing the commentators and ministers to establish how
effectiveness of a civil service, in a way which well their country’s civil service is performing and
could realistically enable international data to (ii) performance improvement, for example by
be collected to measure against it. Whilst there allowing senior decision makers to see which
are many alternative ways one could define civil countries perform best in which areas and learn
service effectiveness, the framework outlined from them. This learning potential would be
here, and the themes and dimensions therein, enhanced if the Index could connect with and
is informed by evidence such that if a civil service inform existing learning and research initiatives in
were to score highly against it, it is reasonable the field.
to conclude that the civil service would be
high-performing relative to its international 3.2 Principles for the framework
counterparts.
Prior to developing the framework we specified
Our approach to deriving a common framework a list of general principles for our indicators.
was to: These principles are shown in Table 3.
• Specify and adhere to a set of principles to
inform the development of the framework
(3.2);
• Draw on evidence (3.4.1) to identify key
features of a draft framework (chapter 4),
which was then extensively tested through
consultation;
• Build on existing indicators and data where
possible, while striving to develop a more
comprehensive framework capturing all
aspects of civil service effectiveness;
• Refine the framework through consultation
with a number of experts, including
academics, think-tanks, international
organisations, civil servants (past and
present) and subject experts.
The International Civil Service Effectiveness Index | Technical Report 13
Table 3: Principles for the InCiSE framework14
• Coherent framework of what are the key elements and drivers of an effective public administration.
• Comprehensive, meaning that it covers all relevant aspects and drivers of the performance of public
administration.
• Actionable, offering genuine policy insights into what drives excellent public administration performance
that can be implemented.
• Transparent in its methodology and assessment process to ensure credibility, robustness and
replicability.
• Feasible to collect from a large group of countries at reasonable cost.
• Replicable to allow for both time and cross country comparison.
The pilot Index provides actionable insights, 3.3 Approach to assessing
and is built on a coherent framework that is
performance
transparent, feasible and replicable:
The standard approach for assessing civil
• The indicator draws on literature to determine
service effectiveness would be to think in terms
the key elements of an effective civil service
of inputs, outputs and outcomes. This is the
and builds on the work of other indicators
approach taken by some recent studies15.
and data collection.
However, when looking specifically at civil
• All source data is specified and the services and the public administration-type
methodology to derive the Index set out functions they provide, this approach seems less
in this report, promoting transparency and attractive. While output and outcome measures
replicability. may seem to cut through the conceptual
uncertainty and simply ask what got done, in
• The Index is feasible to produce for a large practice they are likely to be problematic for
group of countries on a regular basis, three reasons:
largely due to its draw on existing data,
use of imputation for missing scores, and 1. They can be affected by exogenous factors,
normalisation of data to allow comparison. making it difficult to isolate the contribution of
the civil service.
Section 3.4 sets out the range of functions and
attributes which the Index aims to measure. 2. Measuring output is itself problematic
Work will be required post pilot publication to methodologically; difficulties include defining
develop the framework further. To this end we the units of output and obtaining information
encourage feedback as to how the indicator as to the value of this output, due to public
framework can be further strengthened. sector output being provided for free or for
an economically insignificant amount.
- Normative and procedural concerns
(i.e. how the output or outcome was achieved)
are relevant to effectiveness.16
14 A number of these principles were also used to develop World Bank ISMPS project http://www.worldbank.org/en/topic/governance/
brief/indicators-of-the-strength-of-public-management-systems
15 Goderis et al (2015), ‘Public Sector achievement in 36 countries: a comparative assessment of inputs, outputs and outcomes’,
The Netherlands Institute for Social Research.
16 Wieland J. (2014) Governance Ethics: Global value creation, economic organization and normativity, Springer International Publishing,
p205
14 The International Civil Service Effectiveness Index | Technical Report
Ascribing causality between public administration However, certain procedural measures remain
performance and outcomes is too difficult: “It is at the core of any measure of effectiveness,
reasonable to conclude that because causality in both where there is evidence to support the
achieving public policy outcomes is notoriously relationship between such procedures and
hard to assign, the quality of outputs is very hard positive outcomes, and because procedures
to determine.17” Without the certainty that there may have a benefit in themselves. For example,
is a causal relationship between a government meritocracy of recruitment procedures in the
output and the achievement of a stated policy civil service are important because there is a
objective, we cannot know the quality of the considerable evidence base to support the
output because quality, in this context, is a relationship between such procedures and
measure of how well the output is contributing to outcomes associated with an effective civil
the objective. service. However, the extent to which recruitment
processes reward merit is also important for the
Given this difficulty, the preferred approach principle of fairness which is valued in itself.
here is to focus on the effectiveness of the
procedures within the civil service which (often Aggregate inputs, such as the total human and
indirectly) affect those outcomes. The approach financial resource put in, are not measured at
deployed is therefore more process and output this stage. It may be important to compare
focused. Beyond just feasibility, one could argue performance on the framework against those
an advantage of process-based indicators aggregate inputs (for example this could be
is that they are more instructive for potential done in the form of a civil service efficiency ratio;
performance improvements as it is processes the ratio of the score for an indicator to the
which are ultimately changed to increase overall inputs). We have however examined the
effectiveness. We acknowledge the problem with sensitivity of the Index when taking into account
all procedural definitions of effectiveness that the the relative wealth of countries as measured
procedures, however defined, may not actually by GDP per capita (see Chapter 9 for more
correlate with the positive outcomes. information).
17 Holt J. and Manning N. (2014), ‘Fukuyama is right about measuring state quality: now what?’, Governance: An International Journal of
Policy, Administration, and Institutions, 27(4).
The International Civil Service Effectiveness Index | Technical Report 15
Figure 2 below shows what is captured in the every organisation, mission support functions
indicator set. support these core external functions (on both
sides). By looking across all 3 types of function,
The logic model highlights the focus on the the aim is to measure how well civil services
effectiveness of the procedures within the civil deliver the core elements of their roles.
service rather than outcomes, leading to the
more process and output focused approach Figure 3 shows the functions currently included
deployed. in the InCiSE framework.
Section 3.4.1 sets out the definition of each of
3.4 The InCiSE framework these functions, along with a justification for their
The InCiSE framework starts by defining the inclusion in the framework.
core characteristics of an effective civil service.
Attributes
To do this, it assesses effectiveness on the basis
of two inter-related dimensions, 1) the delivery Every civil service also has an underlying set of
of its core functions and 2) an underlying set attributes which are important drivers of how
of attributes which are important drivers of effectively they deliver core functions. These
effectiveness across all parts of the civil service. attributes should apply to all parts of the civil
service; they are not specific to particular parts
Functions or functions. The inclusion of attributes in the
Civil services across the world are of different framework is based on both a normative and
shapes and sizes, but there are certain core positive judgement: civil services should aim
functions which all of them deliver. On one side, to cultivate and demonstrate these attributes,
they deliver a set of central executive functions as they are commonly (but not necessarily
for ministers. These may help to formulate policy universally) understood as aspects of best
for the country (the effects of which are borne by practice, and the included attributes should
citizens). On the other side, civil services interact generally be determinants of performance across
more directly with citizens through the delivery of all functions. Figure 4 shows those attributes
services such as tax administration. As with currently included in the InCiSE framework.
Figure 2: Logic model of the civil service
Inputs Activities/ Outputs Outcomes Impact
procedures
Human and financial These inputs can be These activities These services These outcomes
resources are used to complete and procedures affect outcomes can lead to changes
needed to operate activities and help the civil service for citizens in communities
the civil service. procedures the civil deliver the services (e.g. educational and organisations
service undertakes it provides to outcomes), across the country.
ministers and ministers (e.g. trust
citizens. in government),
and the civil service
(e.g. trust in the
civil service).
Captured in parts Captured in parts Not captured in the
of the indicator of the indicator indicator set.
set but not the Primary focus of the indicator set. set but not the
primary focus. primary focus.
16 The International Civil Service Effectiveness Index | Technical Report
Section 3.4.1 sets out the definition of each of these attributes, along with a justification for their
inclusion in the framework.
Figure 3: InCiSE Core Functions
Central executive Mission support Direct service delivery
Policy making: The quality Procurement: The extent Tax administration: The
of the policy making to which the procurement efficiency and effectiveness
process, including how process is efficient, of tax collection (at the
policy is developed competitive, fair, and central/federal level).
and coordinated across pursues value for money.
government and monitored
during implementation.
Fiscal and financial HR management: The Social security
management: The quality meritocracy of recruitment administration: The
of the budgeting process and extent to which civil efficiency and effectiveness
and the extent to which servants are effectively of social security
spending decisions are attracted, managed and administration (at the
informed through economic developed. central/federal level).
appraisal and evaluation.
Regulation: The extent Information technology: Digital services: The user-
and quality of regulatory The extent to which civil centricity, transparency
impact assessments and servants have the digital and cross-border mobility
the degree of stakeholder tools to work efficiently. of digitally-provided public
engagement involved in services and the availability
them. of ‘key enablers’.
Crisis/risk management: Finance: The extent
The effectiveness with to which operations
which the government are supported by well-
engages the whole of managed, efficient finance
society to better assess, systems, particularly on the
prevent, respond to and alignment of finance with
recover from the effects of the business strategy and
extreme events. the level of civil servant
satisfaction with finance
support.
Figure 4: InCiSE Attributes
-
Integrity: The extent to which civil 4. Inclusiveness: The extent to which
servants behave with integrity, make the civil service is representative of the
decisions impartially and fairly, and citizens it serves.
strive to serve both citizens and
ministers. -
Openness: The regular practise and 5. S
taff engagement: Staff levels of pride,
degree of consultation with citizens attachment and motivation to work for
to help guide the decisions we make their organisation.
and extent of transparency in our
decision‑making. -
Capabilities: The extent to which the 6. Innovation: The degree to which new
workforce has the right mix of skills. ideas, policies, and ways of operating
are able to freely develop.
The International Civil Service Effectiveness Index | Technical Report 17
The attributes and functions identified are brought together to form the InCiSE framework shown
in Figure 5.
Figure 5: InCiSE Index framework
Overall Measurement framework Outcomes
inputs (process and output focussed) for citizens
- Total human Civil Service
resource - Total financial Central Mission Direct
resource
executive support service
functions functions delivery
Outputs functions Outputs
to ministers to citizens - Policy making 1. Procurement 1. Tax administration
- Fiscal and 2. HR (at the central/
financial management federal level)
management - IT 2. Social security
- Regulation administration
- Finance
Ministers 4. Crisis/Risk (at the central/ Citizens
management federal level) - Digital services
Attributes
-
Integrity 2. Openness 3. Capabilities 4. Inclusiveness
Policy effects 5. Staff engagement 6. Innovation Policy effects(‘Value for money’ and ‘use of evidence’ are other key attributes
identified. However, these concepts are already captured on the
functions side of the framework to the extent that their inclusion
as attribute indicators is considered unnecessary.)
This framework shows the attributes which drive and support the successful delivery of the civil
service across the three categories of functions. The ‘mission support’ functions (Procurement,
HR Management, IT and Finance) underpin both the ‘central executive functions’ providing outputs
to ministers and the ‘direct service delivery’ functions providing outputs to the public. For the reasons
described in section 3.3, citizen outcomes are not currently included directly in the framework,
although it is important to bear in mind that all procedures and outputs within the framework are
delivered with a view to influence them.
18 The International Civil Service Effectiveness Index | Technical Report
3.4.1 Definitions and justification of Policy Making
the attributes and functions Definition: The quality of the policy making
process, including how policy is developed and
In this section we set out the definition for coordinated across government and monitored
each attribute and function specified in Figures during implementation.
2 and 3, and the justification for its inclusion
in the framework from our literature review. Justification: Policy making remains a
A bibliography of references in this section central role of a civil service and the quality
can be found in Annex C. of evidence and appraisal are central to the
success of policy. Kaufmann et al. outline
three functions of good governance, including
‘the capacity of government to effectively
formulate and implement sound policies’
(1999). Policymakers need to ‘receive rigorous
analyses of comprehensive background
information and evidence, and of the options
for actions,’ according to the Office for Public
Management (OPM) and the Chartered Institute
of Public Finance and Accountability (CIPFA)
(2004). This paper also advises that ‘good
quality information and clear, objective advice
can significantly reduce the risk of taking
decisions that fail to achieve their objectives
or have serious unintended consequences’.
The necessity of sound evidence and reliability
in policymaking is echoed by Bovaird and Löffler:
‘The choice of a particular policy direction should
be informed by existing evidence on what has
been tried elsewhere and whether it has been
demonstrated to deliver the desired benefits,’
and ‘evidence can be used both to facilitate
accountability and to promote improvement in
policy-making, programme development and
service delivery’ (2003).
The International Civil Service Effectiveness Index | Technical Report 19
Fiscal and Financial Management Regulation
Definition: The quality of the budgeting process Definition: The extent and quality of regulatory
and the extent to which spending decisions impact assessments and the degree of
are informed through economic appraisal stakeholder engagement involved in them.
and evaluation.
Justification: The appropriate appraisal and
Justification: Fiscal and financial management evaluation of regulatory changes accompanied
is an important measure of every system of by sufficient stakeholder engagement is
public administration. The Indicators of the crucial to ensuring that any introductions are
Strength of Public Management Systems fully considered and fair, involving various
(ISPMS) from the World Bank state `Public stakeholders. This scrutiny is endorsed by
sector management arrangements must also many; the OECD, for instance, ‘recognis[es]
encourage fiscal and institutional sustainability that regulations are one of the key levers by
as less tangible but equally critical outcomes’ which governments act to promote economic
and ‘Reforms of budgetary and financial prosperity, enhance welfare and pursue
management systems… are often crucial for the public interest,’ and that ‘well-designed
development outcomes’ (2012). Holt and regulations can generate significant social and
Manning also consider that ‘public administration economic benefits which outweigh the costs of
practitioners break down the functioning of the regulation, and contribute to social well-being,’
central agencies into five management systems,’ (2012). The International Monetary Fund (IMF)
including fiscal and financial management which acknowledges the importance of regulatory
is made up of: ‘planning and budgeting; financial frameworks to successful governance: ‘From
management; and accounting, fiscal reporting the perspective of the IMF, countries with good
and audit.’ The OECD’s recommendation paper governance have strong legal and regulatory
on budgetary governance sets out ten principles frameworks in place,’ (2016). Additionally,
for good budgetary governance which include in promoting best practice, ‘[the] Regulatory
‘ensur[ing] that performance, evaluation and Impact Analysis (RIA) is a multiple stakeholder
value for money are integral to the budget assessment of the economic, environmental
process’ and ‘manag[ing] budgets within clear, and social impact of regulations. The OECD and
credible and predictable limits for fiscal policy,’ European Union have strongly promoted this
(OECD, 2015a). evidence-based approach towards legislation’
(Bovaird and Löffler, 2003).
20 The International Civil Service Effectiveness Index | Technical Report
Crisis/Risk Management Procurement
Definition: The effectiveness with which the Definition: The extent to which the procurement
government engages the whole of society to process is efficient, competitive, fair, and pursues
better assess, prevent, respond to and recover value for money.
from the effects of extreme events.
Justification: ‘Government procurement
Justification: The OECD Strategic Crisis accounts for an average of 15 per cent or more
Management report highlights crisis of a country’s GDP,’ (World Trade Organisation,
management as central to government’s role and 2015). As procurement makes up such a
a ‘fundamental element of good governance’ large proportion of countries’ GDP, it must be
(Baubion, 2013). New Zealand measures their managed appropriately. Effective procurement
public sector outputs against eight identified management can streamline contracts and
categories, of which two encompass contingent reduce outgoings, contributing to improved
and emergency capabilities: ‘so that an efficiencies in civil services. Phillips et al. also
adequate response will be available in time observe that: ‘effective procurement practices
to minimise loss, damage or injury,’ cited by provide governments with a means of bringing
Van Dooren et al. (2006). Studies have shown about social, environmental and economic
that credibility and trust in governments to reform’ (2007). On public procurement, the
deal with crises is vital both to reassure and World Bank states it ‘is a key variable in
encourage support from the private sector and determining development outcomes and,
general public, as outlined by Christensen et when carried out in an efficient and transparent
al. (2011). In the OECD’s recommendation of manner, it can play a strategic role in delivering
the council on the governance of critical risks, more effective public services. It can also act as
it is ‘recognising that effective risk governance a powerful tool for development with profoundly
is a means of maintaining or achieving national positive repercussions for both good governance
competitive advantage against a backdrop of and more rapid and inclusive growth’ (2016).
numerous geopolitical, environmental, societal
and economic uncertainties as it represents an
opportunity to invest in safer and better lives for
the future’ and ‘recognising that citizens and
businesses expect governments to be prepared
for a wide range of possible crises and global
shocks and to handle them effectively should
they arise,’ (OECD, 2014).
The International Civil Service Effectiveness Index | Technical Report 21
Human Resource Management Information Technology
Definition: The meritocracy of recruitment and Definition: The extent to which civil servants
extent to which civil servants are effectively have the digital tools to work efficiently.
attracted, managed and developed.
Justification: It is important for the public
Justification: ‘The public sector is very labour sector to keep up with IT developments in
intensive – around 70 per cent of the budgets order to maintain optimum efficiency. Advanced
of most public organisations are spent on IT can also allow for improvements in flexible
staff’ (Bovaird and Löffler, 2003), so good HR working patterns for civil servants. Madzova
management is key to the successful functioning et al. make the point that improved IT ‘can
of an exemplary civil service. Performance enhance the speed and efficiency of operations,
management can help to create incentives by streamlining processes, lowering costs,
for personal development in the civil service. improving research capacities and record
Fukuyama (2013) recognises that recruitment keeping,’ (2013), highlighting the cost saving
and reward ‘remain at the core of any measure which could be achieved. Magno and Serafica
of quality of governance… whether bureaucrats go further to identify three ways in which IT
are recruited and promoted on the basis of promotes good governance: ‘(1) by increasing
merit’. ‘If the HR policies are not right then transparency, information, and accountability;
public organisations will not attract the human (2) by facilitating accurate decision-making and
resources they need to perform the functions public participation; and (3) by enhancing the
of government and deliver the services that efficient delivery of public goods and services,’
government has promised the electorate’ (2001). However, skills must be in place to
(Bovaird and Löffler, 2003). harness this capability, and Fukuyama (2013)
acknowledges the technical expertise of civil
servants ‘remain[s] at the core of any measure of
quality of governance’.
22 The International Civil Service Effectiveness Index | Technical Report
Finance Tax Administration
Definition: The extent to which operations are Definition: The efficiency and effectiveness of
supported by well-managed, efficient finance tax collection (at the central/federal level).
systems, particularly on the alignment of finance
with the business strategy and the level of civil Justification: Effective tax systems can be
servant satisfaction with finance support. viewed as a critical building block for increased
domestic resource mobilisation which is
Justification: Managing government finance in essential for civil service effectiveness and good
the context of business operations contributes governance. ‘Successful tax extraction provides
towards value for money for the taxpayer to resources that enable the government to
the same extent as fiscal spending. ‘Taxpayers operate in other domains,’ Fukuyama highlights
are entitled to receive assurance that Public ‘it is a necessary function of all states, and
Administrations take due care in managing one for which considerable data exist’ (2013).
funds,’ and ‘the issue of adequate internal The role of tax administration as the basis of
control is at the heart of sound financial government operations in made clear by the
management of the national budget.’ (European OECD, ‘Strong tax administrations and sound
Commission, 2006). Bouchard and McCrae public financial management help maximise
point out that the ‘Control of money and the domestic resources that are necessary for
budgets is a core role of any finance function,’ government to function, to sustain social safety
and they find that ‘previous work on improving nets, to maintain long-term fiscal sustainability,
decision making has highlighted the importance and to free up fiscal space for pursuing socio-
of financial leadership and strengthened economics objectives,’ (n.d.). Although priorities
performance management at the top of and circumstances vary widely across countries,
government departments’ (2013). the drive to elevate the collective standard of
tax administration is of great importance. Holt
and Manning highlight the importance of tax
administration in measuring the effectiveness
of public administration and it is one of the
key functions highlighted by the World Bank
Indicators of the Strength of Public Management
Systems (2012).
The International Civil Service Effectiveness Index | Technical Report 23
Social Security Administration Digital Services
Definition: The efficiency and effectiveness Definition: The user-centricity, transparency and
of social security administration (at the central/ cross-border mobility of digitally-provided public
federal level). services and the availability of ‘key enablers’.
Justification: Social security administration Justification: A changing world and digital
plays an important role in civil services. environment provide the impetus for a civil
The OECD study Towards Better Measurement service to ensure modernity and remain user-
of Government, for example, highlights the centric for the public. In doing so, efficiencies
importance of social security administration. should be achieved to enable cost savings
Chalam says that ‘Social security is a human in processes while also allowing for further
right as well as a social and economic necessity. accessibility of services. The OECD has
All successful societies and economies have supported this view of potential benefits:
employed developmental strategies where ‘ICT is increasingly used to support broader
social security systems played an important public sector development objectives… by
role to alleviate poverty and provide economic changing service delivery approaches by
security that helps people to cope with life’s creating personalised, high quality services to
major risks or the need to quickly adapt to users, thereby increasing user satisfaction and
changing economic, political, demographic and effective service delivery; facilitating major work
societal circumstances’ (2014). McKinnon (2011) organisation and management changes creating
supports this view: ‘Several legal instruments back-office coherence and efficiency gains;
adopted by the United Nations recognize social increasing transparency of government activities,
security as a basic human right. The State and increasing citizen engagement.’ (Lonti and
has the responsibility to create the enabling Woods, 2008).
environment that would allow citizens to exercise
this right,’ as well as pointing out ‘the role of
social security as a requisite economic and social
stabilizer’. In summary, ‘Social security systems
play a central role in the efforts of every country
to promote and ensure the social and economic
well-being of its citizens’ (McKinnon, 2011).
24 The International Civil Service Effectiveness Index | Technical Report
Integrity Openness
Definition: The extent to which civil servants Definition: The regular practise and degree
behave with integrity, make decisions impartially of consultation with citizens to help guide the
and fairly, and strive to serve both citizens and decisions we make and extent of transparency in
ministers. our decision-making.
Justification: Integrity is one of the core values Justification: The need for transparency within
associated with a civil service. The International a civil service is imperative for the public to trust
Civil Service Commission highlights the and feel empowered to hold the government
importance of integrity to the work of United accountable for their actions, whilst reducing
Nations (UN) common systems staff: ‘The corruption. The World Bank ‘supports efforts to
concept of integrity… embraces all aspects encourage open and transparent government’
of behaviour of an international civil servant, in their advice for helping governments to
including… honesty, truthfulness, impartiality strengthen institutions against corruption (The
and incorruptibility. These qualities are as basic World Bank, n.d.). The United Nations outlines
as those of competence and efficiency.’ (Civil the need for transparency and accountability in
Service Commission, 2002). The World Bank governance; ‘[this] implies a proactive effort to
states: ‘A well-performing civil service resists make information accessible to citizens’ and it
petty corruption and provides the staff for is ‘one indicator of a government that is citizen-
many of the institutions that protect integrity in focussed and service-oriented’. (United Nations,
government’ (n.d.) while setting out the role of 1999). Graham et al. also make reference to
the World Bank in helping countries combat the United Nations Development Program’s
corruption. Indeed, the Charter of the United five principles of good governance, in which
Nations lists integrity as key to the qualities transparency is identified as a key characteristic.
employees must show: ‘The paramount The OECD summarises the importance of
consideration in the employment of the staff openness in that ‘access to information and
and in the determination of the conditions of proactive transparency help build citizens’ trust
service shall be the necessity of securing the in government.’ (n.d.).
highest standards of efficiency, competence,
and integrity.’ (United Nations, 1945). This paper
also outlines the need for impartiality as key to
independence of a good civil service as well as
Kaufmann et al. (1999) and Huther and Shah
(1999). Numerous studies aiming to establish
good governance have utilised similar metrics
in their analyses, for instance Muiithi et al. at the
LSE (2015). Therefore, the inclusion of integrity in
the InCiSE is deemed necessary and crucial for
the assessment of an effective civil service.
The International Civil Service Effectiveness Index | Technical Report 25
Capabilities Inclusiveness
Definition: The extent to which the workforce Definition: The extent to which the civil service is
has the right mix of skills. representative of the citizens it serves.
Justification: The need for a variety of certain Justification: A modern civil service should be
strong skills is vital for the successful operation representative of the public it stands to serve.
of any organisation, civil services included. In order to do so, institutions must be inclusive in
‘Public service organisations need people nature. ‘Governments are increasingly concerned
with the right skills to direct and control them about the importance of diversity in public
effectively’ (OPM and CIPFA, 2004). The institutions, to ensure that the needs, aspirations
standards for good governance set out by and experiences of a diverse range of citizens
the Office for Public Management (OPM) and are reflected in the decision-making process,’
the Chartered Institute of Public Finance and (OECD, 2015b). In their Post-2015 Millennium
Accountability (CIPFA) which outline seven Development Goal reflections, the OECD
principles for people in public life (known as the outlines the greater success felt by inclusive
Nolan principles), include leadership as a core public bodies: ‘Inclusive governments and an
skill. It goes on to list necessary skills as ‘the active civil society put forward more responsive,
ability to scrutinise and challenge information… equitable policies’ and that these ‘build trust in
including skills in financial management and the government and help create… public services
ability to recognise when outside expert advice that are better suited to diverse needs,’ (n.d.).
is needed,’ (2004). Fukuyama acknowledges The guiding principles to the international
the importance of educational attainment of civil civil service, set out by the International Civil
servants: ‘Another critical measure of capacity is Service Commission, support the claim
the level of education and professionalization of that civil servants must ‘respect the dignity,
government officials,’ along with the importance worth and equality of all people’ and have: ‘a
of digital capability: ‘what level of technical willingness to work without bias with persons
expertise they are required to possess’ (2013). of all nationalities, religions and cultures’ (2002).
The OECD Government at a Glance report
makes the point that opinion on the groups in
need of representation in public administration
has widened ‘and now includes a range of
dimensions such as women; racial, ethnic and
religious minorities; the poor; the elderly; the
disabled; and other minority groups such as
indigenous populations,’ and goes on to say that
‘a more representative public administration can
better access previously overlooked knowledge,
networks and perspectives for improved policy
development and implementation,’ (2015b).
A paper by the Office for Public Management
(OPM) and the Chartered Institute of Public
Finance and Accountability (CIPFA) highlights
the potential benefits of this view; ‘To enrich
governance deliberations by bringing together
a group of people with different backgrounds
– governing bodies need to recruit governors
from different parts of society. Public trust
and confidence in governance will increase if
governance… [is] done by a diverse group of
people who reflect the community.’ (2004).
27 The International Civil Service Effectiveness Index | Technical Report
Staff Engagement Innovation
Definition: Staff levels of pride, attachment and Definition: The degree to which new ideas,
motivation to work for their organisation. policies, and ways of operating are able to
freely develop.
Justification: Staff who feel that their roles are
making a real contribution, as well as a genuine Justification: The opportunities for government
interest and pride in their work will be more innovation are vast and numerous; ‘Governments
motivated and engaged. The OECD defines are operating in a new landscape. The public
engagement as: ‘Engaged employees are those sector faces economic, social and environmental
who are “committed to their organisation’s challenges; technology is revolutionizing how
goals and values, motivated to contribute to citizens interact with government; individuals
organisational success, and are able at the same and organisations across society are forming
time to enhance their own sense of wellbeing”’ new kinds of partnerships; and citizens are more
(2015d). The paper goes on to state that informed and connected than ever. Together
‘engaged employees are critical to successfully these factors create opportunities for new ways
manage change in the public administration, of thinking about government and how it works’
to enhance service orientation and to ensure (OECD, 2015c). The necessity for innovation has
customer satisfaction,’ (OECD, 2015d). A report been highlighted by Nesta: ‘For public sectors
for the UK Government by MacLeod and Clarke to become more adept at innovation they need
points out that ‘Employee engagement strategies to treat it with the same seriousness they deal
enable people to be the best they can at work, with handling risk, financial controls or regulatory
recognising that this can only happen if they enforcement’ (Mulgan, 2014).
feel respected, involved, heard, well led and
valued by those they work for and with’ (2009).
The paper also highlights the wider benefits
associated: ‘they are motivated and able to give
of their best to help it succeed – and from that
flows a series of tangible benefits for organisation
and individual alike.’ (MacLeod and Clarke, 2009).
Classification of value for money and use of evidence within the framework
There are concepts, such as the pursuit of value for money (VFM) and use of evidence, which could
be considered attributes but which are particularly relevant to some functions and are therefore
captured through this side of the framework.
By utilising the deep knowledge and experience of the founder organisations and their networks,
and by undertaking extensive consultation, we have identified a range of themes relevant to
each indicator, and metrics to measure each theme. These themes and metrics are described in
Chapter 4. There are a number of indicators and themes for which appropriate metrics could not be
identified and as such measurement of these indicators and themes has not been possible; they are
thus omitted from this edition. There are naturally a number of potential limiting factors relevant to the
metrics included in the Index and these are described in further detail in Annex A.
The International Civil Service Effectiveness Index | Technical Report 28
Chapter 4: Measuring
against the framework
Chapter 4 explores the data used to measure against the framework specified in Chapter 3,
and the weighting given to data to produce the score for each indicator.
4.1 Data availability Of the eleven functions proposed in the InCiSE
framework, eight are measured and included in
The InCiSE measurement framework, as the pilot InCiSE composite Index (explained in
outlined in Chapter 3, is such that if a civil Chapter 7). Of the six attributes in the InCiSE
service were to score highly against it, it is framework, four are measured and included in
reasonable to conclude that the civil service the composite.
would be high-performing relative to its
international counterparts. This chapter of the
report focuses on what is measured under
each indicator, and the data used to do so.
However, it should first be noted that existing
data does not enable measurement against
all of the InCiSE framework. Nevertheless,
the Index results provide valuable insights for
accountability and performance improvement
and we will strive to close gaps in data coverage
as it is developed further. The metrics included
in the Index represent a wide range of data
sources and are derived from a combination of
administrative data, survey findings and expert
assessments. Although every effort has been
made to ensure data is current and measures
the subject of civil service effectiveness closely,
due to limited data availability, some metrics may
measure wider public sector, rather than civil
service performance. While all data used is the
most recent available, not all of the data used
is updated every year. Several countries were
excluded from the pilot Index because they had
fewer than 75% of the metrics available.
29 The International Civil Service Effectiveness Index | Technical Report
Table 4: Indicators included in the InCiSE composite
Measured and included in the InCiSE composite. Not yet measured or included in the InCiSE
composite due to data availability.
Central executive functions Mission support functions
Policy making Procurement
Fiscal and Financial Management (FFM) Information Technology (IT)
Regulation Finance
Crisis/risk management
Attributes
Mission support functions Staff engagement
Human Resource Management (HRM) Innovation
Direct service delivery functions
Tax administration
Social security administration
Digital services
Attributes
Integrity
Openness
Capabilities
Inclusiveness
In some of the areas where data was not available we are aware of new data collection that may help
to fill gaps over time. In other areas, new data collection may need to be initiated over time to fulfill
the measurement of this indicator.
The International Civil Service Effectiveness Index | Technical Report 30
Figure 6: Functions and attributes included in the pilot InCiSE composite
Central executive functions Mission support functions Direct service
delivery functions
Policy making HR management Tax administration
(at the central/federal level)
Fiscal and financial management
Social security administration
Regulation
(at the central/federal level)
Crisis/Risk management
Digital services
Attributes
• Integrity • Openness • Capabilities • Inclusiveness
(‘Value for money’ and ‘use of evidence’ are other key attributes identified. However, these concepts are already captured on the
functions side of the framework to the extent that their inclusion as attribute sub-indicators is considered unnecessary.)
A wealth of data, from survey respondents, expert opinion and administrative data underlies the 12
indicators measured in the Index. In total, 76 metrics are spread across these indicators – although
some metrics are themselves indices meaning that the actual number of ‘total metrics’ is far higher.
34 metrics underlie the attributes, while 42 metrics underlie the core functions. The quality of this
data varies – in Figure 7 we set out an indicative assessment of the quality of data supporting
each indicator.
31 The International Civil Service Effectiveness Index | Technical Report
Figure 7: Indicative data quality for framework indicators
Rag rating summary
of data quality
Integrity
Openness
Attributes Capabilities
Weight: 1/3
Metrics: 34 Inclusiveness
Staff Engagement N/A
Innovation N/A
Policymaking
Fiscal and financial
Central Executive
Regulation
Risk/Crisis Management
Procurement N/A
Core functions
Weight: 2/3 HR
Metrics: 42 Mission Support
IT N/A
Finance N/A
Tax Administration
Direct Service Delivery Social Security Administration
Digital Services
The data quality was assessed according to the following criteria:
• Green – The indicator contains a large number of metrics, which seem to give a detailed, and
relatively (but not necessarily completely) comprehensive picture of performance on the indicator.
The metrics generally have few limitations. For example they get close to measuring civil service,
rather than wider public sector, performance, and they are gathered from a regularly updated data
source, so are up-to-date. Whilst metrics based on subjective expert assessments have potential
drawbacks (see Chapter 8 covering sensitivity analysis for further details), indicators which utilise
these where measurement by other means is particularly difficult are still given green ratings.
• Yellow – The indicator contains a number of metrics, which seem to give a fairly detailed picture
of performance on the indicator. However there are themes which should be measured under the
indicator, which currently are not. The metrics included generally have limitations, but these are
not prohibitive. Some may be based on subjective expert assessment. Some may measure wider
public sector, rather than civil service performance, and some may be out of date.
• Red – the indicator contains a small number of metrics, which do not generally give a detailed
or comprehensive picture of performance on the indicator, but do give some partial information
about the effectiveness of the civil service in this area.
The International Civil Service Effectiveness Index | Technical Report 32
• N/A denotes those indicators for which severe data limitations have prevented their inclusion in
the pilot Index.
4.2 Data underpinning the indicators
Many of the datasets from which the metrics are drawn are updated annually, enabling the Index to
be revised on a regular basis to reflect country developments. This iteration of the index includes data
up until January 2017.
In the following tables we set out the metrics underpinning each of the indicators and the weighting
given to each to make up the indicator score. Metrics were not identified for all the themes we would
like to measure – we identify where this is the case in the tables by use of ticks (✓), crosses (X) and
tildes (~).
✓ A tick indicates the theme is closely represented by one or more metrics utilised in the
framework
X A cross indicates that data has not yet been identified to represent this theme of an indicator
in the framework
~ A tilde indicates that the theme is only approximately represented by one or more metrics in
the framework
In each table the shorthand for the metrics, used in the Index interactive graphics, which are available
online, are shown in italics.
4.2.1 Functions
Central Executive
Policy Making
The score for Policy Making is calculated based on eight metrics from the Bertelsmann Sustainable
Governance Indicators which mainly assess the role civil servants play in setting strategic policy
direction, coordination of policy across government, and the monitoring of policy implementation.
The Bertelsmann Sustainable Governance Indicators are updated annually and cover 41 countries.
Data included is from 2016.
Bertelsmann Sustainable Governance Indicators:
“As a cross-national comparative survey designed to identify and foster successes in effective policymaking,
the SGI explores how governments target sustainable development. We advocate for more sustainable
governance, which is built on three pillars:
– Policy Performance
– Democracy
– Governance
Driven by evidence-based analyses, the SGI helps a variety of stakeholders throughout the OECD and EU
navigate the complexity of effective governance.18”
Further information about the Bertelsmann Sustainable Governance Indicators can be found at http://www.
sgi-network.org/2016/.
18 Sustainable Governance Indicators. Bertelsmann Stiftung. [Online]. Available at: http://www.sgi-network.org/2016/About. [Accessed on
27 April 2017].
33 The International Civil Service Effectiveness Index | Technical Report
Table 5 shows the metrics measured in the policy making indicator, the source of data, and the
weighting given within the indicator.
Table 5: Policy making indicator
Theme to be Currently Weighting Metrics to capture theme and the weighting each receives within
measured captured in in indicator the metric (shown in brackets).
indicator? (%)
The quality ~ 25 • Bertelsmann ‘Scholarly Advice’ sub-indicator is used as a
of policy proxy for the evidence base to policy decisions by assessing
advice the degree of civil servant to academic coordination: ‘How
influential are non-governmental academic experts for
government decision-making?’ (50%).
• Bertelsmann ‘Government Office Expertise’ sub-indicator
assesses: ‘Does the government office/prime minister’s office
(GO/PMO) have the expertise to evaluate ministerial draft bills
substantively?’ (50%).
Degree of ✓ 25 • Bertelsmann ‘Strategic Planning’ sub-indicator assesses:
strategic ‘How much influence do strategic planning units and bodies
policy have on government decision-making?’ (100%).
direction
Coordination ✓ 25 • Bertelsmann ‘Ministerial Bureaucracy’ sub-indicator
of policy assesses: ‘How effectively do ministry officials/civil servants
proposals coordinate policy proposals?’ (50%).
• Bertelsmann ‘Line Ministries’ sub-indicator assesses ‘To
what extent do line ministries involve the government office/
prime minister’s office in the preparation of policy proposals?’
(50%).
Timeliness X – No data identified
and
accuracy
of policy
delivery
Degree ✓ 25 • Bertelsmann ‘Monitoring Ministries’ sub-indicator assesses:
of policy ‘How effectively does the GO/PMO monitor line ministry
monitoring activities with regard to implementation?’ (33%).
• Bertelsmann sub-indicator ‘Monitoring Agencies’, assesses:
‘How effectively do federal and subnational ministries monitor
the activities of bureaucracies/executive agencies with regard
to implementation?’ (33%).
• Bertelsmann ‘National Standards’ sub-indicator assesses:
‘To what extent does central government ensure that
subnational self-governments realise national standards of
public services?’ (33%).
The weighting within the policy making indicator is equally split between the four themes where
metrics were identified. Within the themes, weighting was equally split between the metrics.
The International Civil Service Effectiveness Index | Technical Report 34
Fiscal and financial management
The score for Fiscal and Financial Management is calculated based on two sources:
• The World Economic Forum (WEF) Global Competitiveness Index (GCI)
• OECD medium-term and performance budgeting data
The WEF GCI covers 150 countries, and is updated annually. Survey data included is from 2016.
The OECD data covers 32 countries. The survey data included here is from 2011 (performance
budgeting) and 2012 (medium-term budgeting), and was published in the 2013 Government at
a Glance report.
Global Competitiveness Index:
“The GCI combines 114 indicators that capture concepts that matter for productivity and long-term
prosperity…. These indicators are grouped into 12 pillars...: institutions, infrastructure, macroeconomic
environment, health and primary education, higher education and training, goods market efficiency,
labor market efficiency, financial market development, technological readiness, market size, business
sophistication, and innovation. These pillars are in turn organised into three subindexes: basic requirements,
efficiency enhancers, and innovation and sophistication factors. The three subindexes are given different
weights in the calculation of the overall Index, depending on each economy’s stage of development,
as proxied by its GDP per capita and the share of exports represented by raw materials19.”
OECD medium-term budgeting:
“Data... draw upon country responses to questions from the 2012 OECD Survey on Budgeting Practises
and Procedures. Responses represent the countries’ own assessments of current practices and
procedures. The composite index… contains 10 variables that cover information on the existence of
medium-term perspective in the budget process, the number of years the estimate covers, the types
of expenditures included in the frameworks, the possibility of carry over unused funds from one year
to another and how they are monitored20.”
OECD performance budgeting:
“Data refer to 2011 and draw upon country responses to questions from the 2011 OECD Survey on
Performance Budgeting. Responses represent countries’ own assessments of current practices and
procedures. For EU member countries, results exclude any EU funding. The composite index… contains
11 variables that cover information on the availability and type of performance information developed,
processes for monitoring and reporting on results and whether (and how) performance information
is used on budget negotiations and decision making by the central budget authorities, line ministries
and politicians21.”
19 Global Competitiveness Index. World Economic Forum. [Online]. Available at: http://reports.weforum.org/global-competitiveness-
report-2015-2016/methodology/. [Accessed on 27 April 2017].
20 Government at a Glance, Medium-term expenditure frameworks. OECD. [Online]. Available at: http://www.oecd-ilibrary.org/docserver/
download/4213201ec027.pdf?expires=1493288249&id=id&accname=guest&checksum=F1942DF8725DA66F41F53532D47200D1.
[Accessed on 27 April 2017].
21 Government at a Glance 2013, Performance Budgeting. OECD. [Online]. Available at: http://www.oecd-ilibrary.org/docserver/
download/4213201ec029.pdf?expires=1493288108&id=id&accname=guest&checksum=C9DAFD442CFD772363B3F79047EE4F65.
[Accessed on 27 April 2017].
35 The International Civil Service Effectiveness Index | Technical Report
Table 6 shows the metrics measured in the fiscal and financial management indicator, the source of
data, and the weighting given within the indicator.
Table 6: Fiscal and financial management indicator
Themes to Currently Weighting Metrics to capture theme and the weighting each receives within
be measured captured in in indicator the metric (shown in brackets)
Indicator? (%)
Economic ~ 33 • ‘Public Spending’: WEF GCI business executive opinion:
appraisal ‘How would you rate the composition of public spending in
(use, quality, your country?’ (1) extremely wasteful; (7) highly efficient in
guidance) providing necessary goods and services (100%).
Economic ~
evaluation
(use, quality,
guidance)
Medium- ✓ 33 • OECD ‘Medium-Term Budgeting’ index (100%).
term
budgeting
Performance ✓ 33 • OECD ‘Performance Budgeting’ index (100%).
budgeting
The weighting within the Fiscal and Financial Management indicator gives a one third share to the
combined ‘economic appraisal’/’economic evaluation’ themes, as between these themes there is
one metric to approximately capture them, and one third each to ‘medium-term budgeting’ and
‘performance budgeting’.
Regulation
The score for Regulation is calculated based on six metrics from the OECD’s ‘Indicators of Regulatory
Policy and Governance’ to assess 3 parts of the process behind creating regulation. The OECD data
covers 34 countries and is updated every 3-4 years. Survey data included is from 2014.
Indicators of Regulatory Policy and Governance:
“Indicators of Regulatory Policy and Governance (iREG) present up-to-date evidence of OECD member
countries’ regulatory policy and governance practices advocated in the 2012 Recommendation of
the Council on Regulatory Policy and Governance. They cover in detail three principles of the 2012
Recommendation: stakeholder engagement, Regulatory Impact Assessment (RIA) and ex post evaluation,
and provide a baseline measurement to track countries’ progress over time and identify areas for reform.22”
22 Indicators of Regulatory Policy and Governance 2015. OECD. [Online]. Available at: http://www.oecd.org/gov/regulatory-policy/
indicators-regulatory-policy-and-governance.htm. [Accessed 27 April 2017].
The International Civil Service Effectiveness Index | Technical Report 36
Table 7 shows the metrics measured in the regulation indicator, the source of data, and the weighting
given within the indicator.
Table 7: Regulation indicator
Themes to be Currently Weighting Metrics to capture theme and the weighting each receives within
measured captured in in indicator the metric (shown in brackets)
Indicator? (%)
Ex ante ✓ 33 • OECD i-reg indicators – Regulatory ‘Impact Assessment –
appraisal: Primary Laws’ (50%);
application, • OECD i-reg indicators – Regulatory ‘Impact Assessment –
quality, Secondary Laws’ (50%).
sustainability,
Stakeholder engagement and ex post evaluation assessments
transparency,
are made on four areas of performance: methodology,
oversight.
systematic adoption, transparency and oversight/quality control.
Stakeholder ✓ 33 • OECD i-reg indicators – ‘Stakeholder Engagement –
engagement: Primary Laws’ (50%);
application, • OECD i-reg indicators – ‘Stakeholder Engagement –
quality, Secondary Laws’ (50%).
transparency,
oversight
Ex post ✓ 33 • OECD i-reg indicators – Ex-post ‘Evaluation – Primary Laws’
evaluation: (50%);
application, • OECD i-reg indicators – Ex-post ‘Evaluation – Secondary
quality, Laws’ (50%).
sustainability,
transparency,
oversight
The weighting within the Regulation indicator is equally split between the three themes. Within the
themes, the weighting was equally split between metrics.
For each of the 3 parts of the process, assessments are made for both primary and secondary laws,
giving 6 separate composite indicators in total. Data was collected through surveys of government
officials. Countries were asked to support responses with evidence.
37 The International Civil Service Effectiveness Index | Technical Report
Crisis/risk management
This indicator covers nine areas related to disaster risk management. All metrics are based on the
parts of the Hyogo Framework for Action (HFA) country progress reports, which track each country’s
progress towards implementing the UN priorities for disaster risk reduction action, most relevant to
the civil service. These progress reports contain scores for each country, but these are based on self-
assessment so are seen as less robust and are not included in our indicator. Scores on each of the
nine metrics here are instead calculated by ‘counting yeses’ to a number of binary questions which
are factual in nature (only examples are given for each area in the table but forty-three questions are
used in total). Survey data used here is from 2015.
Hyogo Framework for Action:
“The World Conference on Disaster Reduction was held from 18 to 22 January 2005 in Kobe, Hyogo,
Japan, and adopted the present Framework for Action 2005-2015: Building the Resilience of Nations and
Communities to Disasters (hereafter referred to as the “Framework for Action”). The Conference provided
a unique opportunity to promote a strategic and systematic approach to reducing vulnerabilities and
risks to hazards. It underscored the need for, and identified ways of, building the resilience of nations and
communities to disasters.23”
23 Hyogo Framework for Action 2005 - 2015. World Conference on Disaster Reduction. [Online]. Available at: http://www.unisdr.org/2005/
wcdr/intergover/official-doc/L-docs/Hyogo-framework-for-action-english.pdf. [Accessed 27 April 2017].
The International Civil Service Effectiveness Index | Technical Report 38
Table 8 shows the metrics measured in the crisis/risk management indicator, the source of data,
and the weighting given within the indicator.
Table 8: Crisis/risk management indicator
Themes to be Currently Weighting Metrics to capture theme and the weighting each receives within
measured captured in in indicator the metric (shown in brackets)
Indicator? (%)
Integrated ✓ 22 • ‘Risk Planning Extent’: The extent to which disaster risk
risk planning (2/9)24 is integrated into national policy planning is measured by
counting the types of policy planning into which risk is
integrated (out of 7, e.g. national development plans) (50%).
• ‘Disaster Spending Appraisal’: A score for economic
appraisal looks at whether the costs and benefits of
spending related to disaster spending are considered (50%).
Risk ✓ 22 • ‘Risk Assessment Quality’: The quality of multi-hazard risk
monitoring (2/9) assessment is measured by assessing the processes behind
risk assessments (out of 5, e.g. whether risk assessments
have agreed national standards) (50%);
• The ‘Degree of Risk Monitoring’ is measured by assessing the
extent and use of reports and databases of this type (50%).
Public ✓ 33 • ‘Early Warning Systems’: The quality of early warning
information (3/9) systems is assessed by looking at the processes in place
dissemination (out of 3, e.g. whether protocols are used and applied) (33%).
and public • ‘Public Information’ dissemination is scored by looking at the
awareness types of communications in place (out of 4, e.g. is a national
strategies disaster information system publicly available?) (33%).
• ‘Public Awareness Strategy’ looks at the relevant
workstreams in place (out of 5, e.g. whether public
education campaigns for risk awareness exist) (33%).
International ✓ 11 • ‘International Cooperation’ is measured by assessing the
cooperation (1/9) number of processes/activities in place for international
and risk cooperation and international risk management (out of 6,
coordination e.g. whether transboundary protocols are in place) (100%).
Preparedness X – No data identified
for disaster
response
Post disaster ✓ 11 • ‘Post Disaster Assessment’: A score is given for post
assessment (1/9) disaster damage and loss assessment by considering,
methodology for example, whether a specified methodology for this exists
(out of 3) (100%).
Note – percentages do not sum due to rounding
24 For this indicator, weightings between themes are rounded in the table to the nearest percentage point and are based on multiples of
1/9 fractions.
39 The International Civil Service Effectiveness Index | Technical Report
The weighting within the Crisis/risk management indicator relates to the availability of metrics
under each theme; there are nine metrics in total underlying this indicator across its six themes.
For the ‘integrated risk planning’ and ‘risk monitoring’ themes, both have 2/9 of the overall indicator
weighting, with weighting split equally across the two metrics for each theme. For ‘public information
dissemination and public awareness strategies’, this theme has 3/9 (1/3) of the weight, split equally
across its three metrics. Both ‘international cooperation and risk coordination’ and ‘post disaster
assessment methodology’ are measured by a single metric and are given 1/9 of the total weighting.
No suitable data to measure ‘preparedness for disaster response’ could be found so it is not
currently captured.
Mission support
Human resource management
The score for Human Resource Management is calculated based on Quality of Government (QoG)
expert assessments to determine two important parts of HR management: the meritocracy of
recruitment and attracting talent. Data included is from 2015.
Quality of Government:
The Quality of Government Institute at the University of Gothenburg have developed the QoG Expert
Survey Data.
“The QoG Survey is a unique data set with information on the structure and behavior of public
administration in a range of different countries. The data covers 159 countries and is based on a web
survey of 1294 experts.
The dataset covers different dimensions of Quality of Government, such as, politicisation, professionalisation,
openness, and impartiality.25”
25 Expert Survey Data. The Quality of Government Institute, University of Gothenburg. [Online]. Available at: http://qog.pol.gu.se/data/
datadownloads/qogexpertsurveydata. [Accessed 27 April 2017].
The International Civil Service Effectiveness Index | Technical Report 40
Table 9 shows the metrics measured in the human resource management indicator, the source of
data, and the weighting given within the indicator.
Table 9: Human resource management indicator
Themes to be Currently Weighting Metrics to capture theme and the weighting each receives within
measured captured in in indicator the metric (shown in brackets)
Indicator? (%)
Meritocracy ✓ 75 • ‘Applicant Skills’: QoG expert assessment of extent to which
of recruitment the skills and merits of the applicants decide who gets the
job when recruiting public sector employees (25%);
• ‘Connections Bias in Recruitment: Political’: QoG expert
assessment of extent to which political connections of the
applicants decide who gets the job (25%);
• ‘Connections Bias in Recruitment: Personal’: QoG expert
assessment of extent to which personal connections of the
applicants decide who gets the job (25%);
• ‘Recruitment via Formal Exam System’: QoG expert
assessment of extent to which public sector employees are
hired using a formal examination system (25%).
Attracting ✓ 25 • ‘Comparable Salaries’: QoG expert assessment of extent to
and retaining which senior officials have salaries that are comparable with
talent the salaries of private sector managers with roughly similar
training and responsibilities (100%).
Talent X – No data identified
deployment
(i.e.
minimising
skills gaps)
Performance X – No data identified
management
Quality of X – No data identified
learning and
development
Level of X – No data identified
customer (i.e.
civil servant)
satisfaction
The weighting within the Human Resource Management indicator awards 75% of the total weight
to the ‘meritocracy of recruitment’ theme. This is split equally among the four metrics which underlie
this theme. The ‘attracting and retaining talent’ theme has the remaining 25% with a single metric
capturing this theme. The remaining four themes are not currently captured in the indicator set
because of a lack of suitable data.
41 The International Civil Service Effectiveness Index | Technical Report
Direct service delivery
Tax Administration
The score for Tax Administration is calculated based on metrics mostly taken from ‘OECD Tax
Administration 2015’ which is a fairly comprehensive assessment of OECD tax administration
systems. Metrics included cover efficiency and the extent of digital use in tax collection made up
of 2013 data. In addition, some metrics are taken from the World Bank’s Doing Business Index
with data from 2016.
OECD Tax Administration 2015:
“Tax Administration 2015, produced under the auspices of the Forum on Tax Administration, is a unique
and comprehensive survey of tax administration systems, practices and performance across 56 advanced
and emerging economies (including all OECD, EU, and G20 members). Its starting point is the premise that
revenue bodies can be better informed and work more effectively together given a broad understanding
of the administrative context in which each operates. However, its information content is also likely to be
of interest to many external parties (e.g. academics, external audit agencies, regional tax bodies, and
international bodies providing technical assistance).
The series identifies some of the fundamental elements of national tax system administration and uses
data, analyses and country examples to identify key trends, comparative levels of performance, recent and
planned developments, and good practices.26”
Doing Business index:
“Doing Business presents quantitative indicators on business regulations and the protection of property
rights that can be compared across 190 economies – from Afghanistan to Zimbabwe – and over time.
Doing Business measures regulations affecting 11 areas of the life of a business. Ten of these areas are
included in this year’s ranking on the ease of doing business: starting a business, dealing with construction
permits, getting electricity, registering property, getting credit, protecting minority investors, paying taxes,
trading across borders, enforcing contracts and resolving insolvency.27”
26 Tax Administration 2015. OECD. [Online]. Available at: http://www.oecd.org/ctp/administration/tax-administration-23077727.htm.
[Accessed 27 April 2017].
27 Doing Business 2017. The World Bank. [Online]. Available at: http://www.doingbusiness.org/reports/global-reports/doing-
business-2017. [Accessed 27 April 2017].
The International Civil Service Effectiveness Index | Technical Report 42
Table 10 shows the metrics measured in the Tax Administration indicator, the source of data, and the
weighting given within the indicator.
Table 10: Tax administrator indicator
Themes to be Currently Weighting Metrics to capture theme and the weighting each receives within
measured captured in in indicator the metric (shown in brackets)
Indicator? (%)
Overall ✓ 33 • ‘Collection Cost’: Data from OECD Tax Administration on:
efficiency of cost of collection ratios (administrative costs/net revenue
collection collected) (50%);
• ‘Tax Debt’ ratios (total year end tax debt (excl. disputed)/net
revenue collected) (50%).
User ✓ 33 • ‘Time to Pay Taxes – Business’: Data from the World Bank’s
centricity of ‘Doing Business’ Index on the time it takes businesses to
services pay taxes (100%).
Extent and ✓ 33 • ‘E-Filed Tax Returns – Personal’: Data from OECD Tax
quality Administration on: the percentage of tax returns e-filed
of digital during the last fiscal year for personal taxes (33%);
provision • ‘E-Filed Tax Returns – Corporate’: Data from OECD Tax
Administration on: the percentage of tax returns e-filed
during the last fiscal year for corporate taxes (33%);
• ‘E-Filed Tax Returns – VAT’: Data from OECD Tax
Administration on: the percentage of tax returns e-filed
during the last fiscal year for VAT taxes (33%).
Prevention of X – No data identified
tax evasion
Level of X – No data identified
tax gap
measurement
The weighting within the Tax Administration indicator is currently equally split between the three
themes where relevant data was found. Within the themes, weighting is equally split between
the metrics.
43 The International Civil Service Effectiveness Index | Technical Report
Social Security Administration
The score for Social Security Administration is calculated based on metrics mostly taken from
data from the European Commission (via Eurostat) on administration costs as a proportion of total
expenditure on social security, capturing the overall efficiency of the social security administration
system. Total expenditure includes: social protection benefits, admin costs, sickness/health care
payments, disability payments, pensions, child benefits, unemployment benefits, housing benefits,
social exclusion benefits and other expenditure. The comparison of heterogeneous systems is difficult
so the conclusions which can be drawn from the indicator may need further investigation. Only a
single metric is included at this point.
Table 11 shows the metrics measured in the social security administration indicator, the source of
data, and the weighting given within the indicator.
Table 11: Social security administration indicator
Themes to be Currently Weighting Metrics to capture theme and the weighting each receives within
measured captured in in indicator the metric (shown in brackets)
Indicator? (%)
Overall ✓ 100 • Administration costs as a percentage of total expenditure on
efficiency of social security (100%).
distribution
User X – No data identified
centricity of
services
Extent and X – No data identified
quality
of digital
provision
Prevention of X – No data identified
fraud
The Social Security indicator currently wholly relies on one data source, given 100% of the indicator
weighting, which relates to the theme of overall efficiency of distribution.
The International Civil Service Effectiveness Index | Technical Report 44
Digital services
The score for digital services is calculated based on metrics taken from the European Commission’s
E-Government Benchmarks to assess the quality of digital services provided across four main
dimensions. A detailed assessment of the quality of services across seven life events, according to
the four dimensions, is made. The benchmark is usually published annually and covers 33 European
countries. Data is an average of Mystery Shopping exercises conducted in 2014 and 2015.
E-Government benchmarks:
“E-Government services were assessed in 34 participating countries, including all of the EU28. The
benchmark makes use of Mystery Shopping, where the quality and quantity of online public services
is measured by assessors acting as a user. The subject of the benchmark is a set of seven life events.
Together, these life events represent virtually all domains of government. Each life event is reviewed once
every two years.28”
Table 12 shows the metrics measured in the digital services indicator, the source of data, and the
weighting given within the indicator.
Table 12: Digital service indicator
Themes to be Currently Weighting Metrics to capture theme and the weighting each receives within
measured captured in in indicator the metric (shown in brackets)
Indicator? (%)
User ✓ 25 • European Commission E-Government benchmark: ‘User
centricity of Centricity’ (100%), indicates to what extent (information
services about) a service is provided online.
Transparency ✓ 25 • European Commission E-Government benchmark:
of service ‘Transparency’ (100%), indicates to what extent
governments are transparent as regards a) their own
responsibilities and performance, b) the process of service
delivery and c) personal data involved.
Cross-border ✓ 25 • European Commission E-Government benchmark: ‘Cross
mobility of Border Mobility’ (100%), indicates to what extent European
services users can use online services in another country;
The ✓ 25 • European Commission E-Government benchmark: ‘Key
availability of Enablers’ (100%), indicates the extent to which five technical
‘key enablers’ pre-conditions for eGovernment are used.
The weighting within the Digital Services indicator is equally split between the four themes. As only
one metric was used within each theme, this was given 100% of the theme weighting.
28 E-Government Benchmark 2016. European Commission. [Online]. Available at: https://www.egovernment.ch/index.php/download_file/
force/991/3343/. [Accessed on 27 April 2017].
45 The International Civil Service Effectiveness Index | Technical Report
4.2.2 Attributes
Integrity
The score for Integrity is calculated based on metrics covering six main themes related to the integrity
and other key values of civil servants. Metrics capturing perceptions of these values from country
experts, citizens and businesses make up 80% of the indicator. The remaining 20% assesses the
laws and procedures in place to preserve integrity and prevent conflicts of interest, such as the
degree of whistleblower protection. The indicator uses a range of data sources:
• Transparency International’s Global Corruption Barometer, which covers 100 countries.
Data from 2013.
• World Economic Forum’s (WEF) Global Competitiveness Index (GCI), which covers over 150
countries. Survey data from 2016.
• Quality of Government (QoG), which covers over 100 countries. Expert survey data from 2015.
• OECD private interest (2013), conflict of interest and whistleblower data (2014), which covers 36,
26 and 30 countries respectively. Survey data from 2013 and 2014.
Global Corruption Barometer:
“In the Global Corruption Barometer we ask people to tell us how corrupt they think different powerful groups
in their country are – that is whether they think “none”, “some”, “most” or “all” of them are corrupt. The nine
different groups that we ask about include high-level political actors (the president/prime minister’s office,
members of parliament and government officials); key public sector employees who interact with citizens (tax
officials, the police, judges/magistrates and local government councillors); and those who are not part of the
public sector, but are influential in political life (business executives and religious leaders).29”
Global Competitiveness Index:
See Policy Making above.
Quality of Government:
See Human Resource Management above.
OECD conflict of interest, private interest, and whistleblower data:
“Government at a Glance provides readers with a dashboard of key indicators assembled with the goal
of contributing to the analysis and international comparison of public sector performance. Indicators on
government revenues, expenditures, and employment are provided alongside key output and outcome data
in the sectors of education, health and justice.
Government at a Glance also includes indicators on key governance and public management issues,
such as transparency in governance, regulatory governance, public procurement and the implementation
of employment and remuneration reforms since 2009.30”
29 Global Corruption Barometer. Transparency International. [Online]. Available at: http://transparencia.org.es/wp-content/
uploads/2016/11/informe_barometro-2016-ingles.pdf. [Accessed 27 April 2017].
30 Government at a Glance 2015. OECD. [Online]. Available at: http://www.oecd-ilibrary.org/governance/government-at-a-glance-2015_
gov_glance-2015-en. [Accessed 27 April 2017].
The International Civil Service Effectiveness Index | Technical Report 46
Table 13 shows the metrics measured in the integrity indicator, the source of data, and the weighting
given within the indicator.
Table 13: Integrity indicator
Themes Currently Weighting Metrics to capture theme and the weighting each receives within
to be captured in the metric (shown in brackets)
measured in indicator
Indicator? (%)
Levels/ ✓ 30 • ‘Corruption Perceptions’: Global Corruption Barometer – % of
perceptions people viewing public officials/civil servants as corrupt (25%);
of corruption • ‘Public Officials Stealing’: QoG – expert opinion of how often
public sector officials steal or embezzle public funds (25%);
• ‘Public Officials Favours for Bribes’: QoG – expert opinion of
how often public sector officials grant favours for bribes (25%);
• ‘Government Favouritism of Business’: WEF GCI – business
executive opinion of extent government officials show
favouritism to well-connected firms (25%).
Fairness and ✓ 12.5 • ‘Fair Treatment by Public Officials’: QoG expert assessment of
impartiality extent to which public sector employees treat some groups in
society unfairly (50%);
• ‘Public Officials Act Impartially’: QoG expert assessment of
extent to which public sector employees act impartially when
deciding how to implement policy (50%).
Adherence ✓ 12.5 • ‘Public Officials Follow Rules’: QoG expert assessment of
to rules and extent to which public sector employees strive to follow rules
procedures (100%).
Striving ✓ 12.5 • ‘Public Officials Strive to Help Citizens’: QoG expert
to serve assessment of extent to which public sector employees strive
citizens and to help citizens (33%);
ministers • ‘Public Officials Strive to Implement Policies’ QoG expert
assessment of extent to which public sector employees strive
to implement policies decided by political leaders (33%);
• ‘Public Officials Strive to Fulfill Ideology’: QoG expert
assessment of extent to which public sector employees strive
to fulfil ideology of party in government (33%).
Work ethic ✓ 12.5 • ‘Employee Absences’: QoG expert assessment of extent to
which public sector employees are absent without permission
(50%);
• ‘Employee Efficiency’: QoG expert assessment of extent to
which public sector employees strive to be efficient (50%).
Processes ✓ 20 • ‘Post Employment Cooling Off’: OECD data on whether a
in place to post-employment cooling off period exists and whether it’s
preserve paid (25%);
integrity • ‘Lobbyist Protection’: OECD data on degree of protection
and prevent against lobbyists and other private interests influencing
conflicts of advisory groups (25%);
interest
• ‘Whistleblower Protection: Coverage’: OECD data on number
of groups who receive whistleblower protection (25%);
• ‘Whistleblower Protection: Degree’: QoG expert assessment of
degree of whistleblower protection (25%).
47 The International Civil Service Effectiveness Index | Technical Report
The weighting within the Integrity indicator is split between the six themes as follows: Levels/
perception of corruption – 30%; fairness and impartiality – 12.5%; adherence to rules and procedures
– 12.5%; striving to serve citizens and ministers – 12.5%; work ethic – 12.5%; and processes in
place to preserve integrity and prevent conflicts of interest – 20%. Levels/perceptions of corruption
and Processes in place to preserve integrity and prevent conflicts of interest themes contain metrics
from a wider variety of data sources and were therefore weighted more heavily.
Within each theme, the weight is further split between the metrics which measure that theme.
For all the Integrity themes, this weight is split equally among the metrics, although the number of
metrics under each theme varies: 1 metric (adherence to rules and procedures); 2 metrics (fairness
and impartiality and worth ethic); 3 metrics (striving to serve citizens and ministers); and 4 metrics
(levels/perceptions of corruption and processes in place to preserve integrity and prevent conflicts
of interest).
Openness
The score for Openness assumes there are two distinct themes embedded within the concept of
open government: societal consultation and transparency. Each get 50% weight within the indicator.
The data on both sides consists of existing composite indicators, for example composites designed
to measure open government as a whole and others looking more specifically at open government
data. There are four metrics on the consultation side (the top two themes in the table) and five
metrics on the transparency side (the bottom four themes in the table).
The indicator uses a range of data sources:
• The World Justice Project’s Open Government Index, which covers 102 countries. Survey data
from 2015.
• The UN’s E-participation Index, covering 193 countries and updated biennially. Survey data from
2016.
• Bertelsmann’s Sustainable Governance Indicators (SGIs), covering 41 countries and updated
annually, expert survey data from 2016.
• The World Wide Web Foundation’s Open Data Barometer, covering 92 countries and updated
annually. Survey data from 2015.
• The Open Knowledge Foundation’s Open Data Index, covering 122 countries and updated
annually. Survey data used from 2015.
• OECD’s OURdata index, covering 29 countries. Survey data from 2014.
The International Civil Service Effectiveness Index | Technical Report 48
Open Government Index:
“The WJP Open Government Index is the first effort to measure government openness based on the general
public’s experiences and perceptions in 102 countries. The Open Government Index is composed of four
dimensions: publicised laws and government data, right to information, civic participation, and complaint
mechanisms.31”
E-participation Index:
“The e-participation index (EPI) is derived as a supplementary index to the UN E-Government Survey.
It extends the dimension of the Survey by focusing on the use of online services to facilitate provision
of information by governments to citizens (“e-information sharing”), interaction with stakeholders
(“e-consultation”), and engagement in decision-making processes (“e-decision making”).
E-Participation Framework:
• E-information: Enabling participation by providing citizens with public information and access to
information without or upon demand
• E-consultation: Engaging citizens in contributions to and deliberation on public policies and services
• E-decision-making: Empowering citizens through co-design of policy option and co-production of service
components and delivery modalities.32”
Sustainable Governance Indicators:
“The SGI is a platform built on a cross-national survey of governance that identifies reform needs in 41 EU
and OECD countries.
The SGI brings together a broad network of experts and practitioners aiming to understand what works best
in sustainable governance.33”
Open Data Barometer:
“Covering 92 countries in the present edition, the Barometer ranks nations on:
• Readiness: How prepared are governments for open data initiatives? What policies are in place?
• Implementation: Are governments putting their commitments into practice?
• Impact: Is open government data being used in ways that bring practical benefit?”
Open Data Index:
“The Global Open Data Index is an annual effort to measure the state of open government data around
the world. The crowdsourced survey is designed to assess the openness of specific government datasets
according to the Open Definition.34”
OURdata index:
“The OECD OURdata Index measures government efforts to implement the G8 Open Data charter based on
the availability, accessibility and government support to promote the reuse of data, focusing on the central
OGD portal in each country.35”
31 Open Government Index. World Justice Project. [Online]. Available at: https://worldjusticeproject.org/our-work/wjp-rule-law-index/wjp-
open-government-index-2015. [Accessed 27 April 2017].
32 E-Participation Index. United Nations. [Online]. Available at: https://publicadministration.un.org/egovkb/en-us/About/Overview/E-
Participation. [Accessed on 27 April 2017].
33 Sustainable Governance Indicators. Bertelsmann Stiftung. [Online]. Available at: http://www.sgi-network.org/2016/. [Accessed
27 April 2017].
34 Open Data Index. Open Knowledge International. [Online]. Available at: http://index.okfn.org/about/. [Accessed 27 April 2017].
35 Open, Useful, Reusable Data Index. OECD. [Online]. Available at: http://www.oecd-ilibrary.org/docserver/download/4215081e.
pdf?expires=1493300444&id=id&accname=guest&checksum=665A4CCF1038B2E655ECB2D663FA9D34. [Accessed on
27 April 2017].
49 The International Civil Service Effectiveness Index | Technical Report
Table 14 shows the metrics measured in the openness indicator, the source of data, and the
weighting given within the indicator.
Table 14: Openness indicator
Themes to be Currently Weighting Metrics to capture theme and the weighting each receives within
measured captured in in indicator the metric (shown in brackets)
Indicator? (%)
The degree ✓ 37.5 • The ‘Civic Participation’ component of the Open
and quality Government Index (33%), which “measures the effectiveness
of societal of civic participation mechanisms, including the protection of
consultation the freedoms of opinion and expression, and assembly and
association, and the right to petition the government. It also
measures whether people can voice concerns to various
government officers and members of the legislature, and
whether government officials provide sufficient information
and notice about decisions affecting the community,
including opportunities for citizen feedback”;
• ‘E-Government Engagement’: The UN’s E-Participation
Index, which reviews the quality and usefulness of
e-government programs for the purpose of engaging people
in public policy-making and implementation (33%);
• ‘Negotiating Public Support’ Bertelsmann sub-indicator
(33%), which “assesses how successfully the government
consults with societal actors such as trade unions,
employers’ associations, leading business associations,
religious communities, and social and environmental interest
groups in preparing its policy.”
The existence ✓ 12.5 • The ‘Complaint Mechanisms’ component of the Open
and quality Government Index (100%), which “measures whether
of complaint people are able to bring specific complaints to the
mechanisms government about the provision of public services or the
performance of government officers in carrying out their legal
duties in practice, and how government officials respond
to such complaints. It also measures whether people can
challenge government decisions before another government
agency or a judge.”
Government ✓ 30 • ‘Open Data Practice and Impact’: The Open Data Barometer
data (ODB) measures the implementation of open data practice
availability and is the only index to also measure the impact of open
and data (e.g. how many use it) (33%).
accessibility • ‘Government Datasets Openness’: The Open Data Index
(ODI) measures whether publicly held data across 13
Government ✓ areas is defined as open, with results crowdsourced from
data impact volunteers reviewing websites (33%).
and support
• ‘Data Availability and Government Support’: The OURdata
for re-use
index also aims to capture the availability and accessibility
of data but uniquely it also attempts to measure the level of
pro-active support governments provide to foster innovative
re-use of the data (33%).
The International Civil Service Effectiveness Index | Technical Report 50
Table 14: Openness indicator
Right to ✓ 10 • The ‘Rights to Information’ component of the Open
information Government Index (100%) which “measures whether
(e.g. FOIs) requests for information held by a government agency
are granted. It also measures whether these requests are
granted within a reasonable time period, if the information
provided is pertinent and complete, and if requests for
information are granted at a reasonable cost and without
having to pay a bribe. This dimension also measures
whether people are aware of their right to information,
and whether relevant records – such as budget figures of
government officials, ombudsman reports, and information
relative to community projects – are accessible to the public
upon request.”
Publicised ✓ 10 • The ‘Publicised Laws’ component of the Open Government
laws Index (100%), which “measures whether basic laws and
information on legal rights are publicly available, presented
in plain language, and are made accessible in all languages
used by significant segments of the population. This
dimension also measures the quality and accessibility of
information published by the government in print or online
(i.e. active transparency), and whether administrative
regulations, drafts of legislation, administrative decisions,
and high court decisions are made accessible to the public
in a timely manner.”
The weighting within the Openness indicator is split between six themes as follows: Levels/perception
of corruption – 30%; fairness and impartiality – 12.5%; adherence to rules and procedures – 12.5%;
striving to serve citizens and ministers – 12.5%; work ethic – 12.5%; and processes in place to
preserve integrity and prevent conflicts of interest – 20%. This choice of weighting relates to the
number of metrics available under each theme.
Within each theme the weight is split by the number of metrics.
51 The International Civil Service Effectiveness Index | Technical Report
Capabilities
The score for Capabilities is based on OECD PIAAC data, which was collected between August
2011 and March 2012 in most participating countries. Around 166,000 adults, representing 724
million adults aged 16 to 65, were surveyed in 24 countries. Given that not all included countries are
covered, some countries have all data imputed for this indicator (see table 18).
Programme for the International Assessment of Adult Competencies:
“The Programme for the International Assessment of Adult Competencies (PIAAC) developed and conducts
the Survey of Adult Skills. The survey measures adults’ proficiency in key information-processing skills –
literacy, numeracy and problem solving in technology-rich environments – and gathers information and data
on how adults use their skills at home, at work and in the wider community.36”
Table 15 shows the metrics measured in the capabilities indicator, the source of data, and the
weighting given within the indicator.
Table 15: Capabilities indicator
Themes to be Currently Weighting Metrics to capture theme and the weighting each receives within
measured captured in in indicator the metric (shown in brackets)
Indicator? (%)
Leadership X – No data identified
capability
Commercial X – No data identified
capability
Analytical X – No data identified
capability
Digital X – No data identified
capability
Core ✓ 75 • ‘Literacy Skills’: Taken from the OECD’s PIAAC (Survey of
capability Adult Skills) data (analysis of the microdata): the proportion of
(e.g. the public sector getting level 4 or 5 for literacy skills (33%);
problem- • ‘Numeracy Skills’: The proportion of the public sector getting
solving, level 4 or 5 for numeracy skills (33%);
numeracy,
• ‘Problem Solving Skills’: The proportion of the public sector
literacy skills)
getting level 3 for problem skills (33%).
Educational ✓ 25 • ‘Educational Attainment’: The proportion of the public sector
attainment of with tertiary education, taken from the OECD’s PIAAC (Survey
the workforce of Adult Skills) data (analysis of the microdata) (100%).
The weighting within the Capabilities indicator is split between the core capability and educational
attainment of the workforce themes; the two areas for which usable data has been found. For the
core capability theme, the weighting is equally split between the three metrics. Thus, although the
core capability theme has three quarters of the overall weighting and the educational attainment
theme the remaining quarter, the four metrics underpinning the indicator are equally weighted.
36 Survey of Adult Skills. OECD, Programme for the Assessment of Adult Competencies. [Online]. Available at: http://www.oecd.org/skills/
piaac/. [Accessed 27 April 2017].
The International Civil Service Effectiveness Index | Technical Report 52
Inclusiveness
The score for Inclusiveness is based on the difference in demographics within central government
compared to the country’s workforce overall. A high score is given for having a similar demographic
profile. The indicator used survey data published in Government at a Glance 2013 (GaaG) and
collected in 2010, and Quality of Government survey data from 2015.
Government at a Glance
“Government at a Glance provides readers with a dashboard of key indicators assembled with the goal
of contributing to the analysis and international comparison of public sector performance. Indicators on
government revenues, expenditures, and employment are provided alongside key output and outcome
data in the sectors of education, health and justice. Government at a Glance also includes indicators on key
governance and public management issues, such as transparency in governance, regulatory governance,
public procurement and the implementation of employment and remuneration reforms since 2009. While
measuring government performance has long been recognised as playing an important role in increasing the
effectiveness and efficiency of the public administration, following the economic crisis and fiscal tightening in
many member countries, good indicators are needed more than ever to help governments make informed
decisions regarding tough choices and help restore confidence in government institutions.37”
37 Government at a Glance. OECD. [Online]. Available at: http://www.oecd-ilibrary.org/governance/government-at-a-glance-2015_gov_
glance-2015-en. [Accessed 27 April 2017].
53 The International Civil Service Effectiveness Index | Technical Report
Table 16 shows the metrics measured in the inclusiveness indicator, the source of data, and the
weighting given within the indicator.
Table 16: Inclusiveness indicator
Themes to be Currently Weighting • Metrics to capture theme and the weighting each receives
measured captured in in indicator within the metric (shown in brackets)
Indicator? (%)
Proportionate ✓ 50 • ‘Gender: Central Government Share’: OECD data – the
gender absolute difference between the share of total central
representation government employment filled by women and the share of
women in the labour force (25%);
• ‘Gender: Public Sector Share’: QoG data – the absolute
difference between proportion of women in public sector
and in the labour force (25%);
• ‘Gender: Management Share’: OECD data – the absolute
difference between the share of employment in top
management positions within central government filled by
women and the share of women in the labour force (25%);
• ‘Gender: Senior Management Share’: QoG data – the
absolute difference between proportion of women in central
government senior positions and in the labour force (25%)
Proportionate ✓ 50 • ‘Ethnic and Religious Group Representation’: QoG data
ethnic minority – expert opinion as to whether ‘Key ethnic and religious
representation groups in society are proportionally represented among
public sector employees (1=hardly ever, 7=almost always)’
(100%)
Proportionate X – No data identified
disability
representation
Proportionate X – No data identified
socio-
economic
representation
Proportionate X – No data identified
Lesbian Gay
Bi-sexual
Transgender
Other
sexuality
representation
The weighting within the inclusiveness indicator is equally split between the gender representation
and proportionate ethnic minority representation theme; the two areas for which usable data
has been found. For the gender representation themes, the weighting is equally split between the
four metrics.
For transparency, we set out limitations associated with the above data in Annex A. InCiSE is a long
term project and we expect to include further data and refine the data currently included in the Index
as we receive feedback about how the indicators could be further developed.
The International Civil Service Effectiveness Index | Technical Report 54
Chapter 5: Index
country coverage
This chapter examines which countries have been included and excluded from the
pilot Index, and explains our approach where a country is included but some of their
data is missing.
5.1 Countries included in the Index Covering the widest range of countries in the
Index is tempered by availability of data. There is
The ambition of the InCiSE Index is that considerable variation in country coverage for the
the indicators may serve two purposes: data used in this index (described in chapter 4).
firstly, to serve as an accountability tool, Expanding the range of countries would lead to
and secondly, to provide a performance a greater degree of data imputation, a reduction
improvement tool. To meet this ambition and in the scope of the framework or a greater
achieve the greatest impact the Index aims to reliance on proxies for the civil service. Table 17
cover a wide and growing range of countries shows the proportion of metrics available for
as it is developed further. each country. Only two countries, Norway and
the United Kingdom, had no missing metrics.
55 The International Civil Service Effectiveness Index | Technical Report
Table 17: Proportion of metrics available for each country
(Note: dark red area represents those countries not included in the pilot release of the Index)
Country Total number of metrics Proportion of metrics available (%)
Norway 76 100
United Kingdom 76 100
Austria 75 99
France 75 99
Italy 75 99
Netherlands 75 99
Finland 74 97
Germany 74 97
Denmark 73 96
Poland 73 96
Sweden 73 96
Czech Republic 72 95
Spain 72 95
Slovenia 70 92
Portugal 69 91
Slovakia 69 91
Hungary 68 89
Turkey 68 89
Belgium 67 88
Australia 66 87
New Zealand 66 87
Estonia 65 86
Mexico 65 86
Switzerland 65 86
Greece 64 84
Canada 62 82
Ireland 60 79
Japan 59 78
Korea, Republic of 59 78
Chile 58 76
United States of America 57 75
Bulgaria 57 75
Croatia 54 71
Romania 53 70
Iceland 51 67
Israel 49 64
Colombia 46 61
The International Civil Service Effectiveness Index | Technical Report 56
To balance the issue of country coverage and Across the metrics underlying each indicator
data availability, it was decided that any country there is some variation in the countries covered
with less than 75% of the data available would by the data. Table 18 gives an overview of where
be excluded at this stage from the Index. data was available for each indicator and where
Countries below this point have larger amounts data imputation was required.
of missing data and a greater proportion of the
total data would need to be imputed. Bulgaria Green indicates data was available for all metrics
is excluded to keep the Index limited to OECD within the indicator for that country; amber
countries, for simplicity at this stage. As a result, indicates that data was available for some
we have 31 countries currently included in the metrics within the indicator; and red indicates
pilot Index. Our hope is that further countries will that no data was available for the indicator such
be included as data is identified for use or new that all metrics were imputed.
data collection is initiated.
57 The International Civil Service Effectiveness Index | Technical Report
Table 18: Indicator data availability by country
Country Metric Attributes Direct Service Delivery functions Mission Central Executive functions
account (34 metrics) (26 metrics) Support (11 metrics)
(out of 76) functions
(5 metrics)
Integrity Openness Capabilities Inclusiveness Policy Fiscal & Regulation Risk Man. Human Tax Admin Social Digital
(16) (9) (4) (5) making Fin. Man. (6) (9) Resources (6) Security Services
(8) (3) (5) Admin (4)
(1)
NOR 76
GBR 76
AUT 75
FRA 75
ITA 75
NLD 75
FIN 74
DEU 74
DNK 73
POL 73
SWE 73
CZE 72
ESP 72
SVN 70
PRT 69
SVK 69
HUN 68
TUR 68
BEL 67
AUS 66
NZL 66
EST 65
MEX 65
CHE 65
GRC 64
CAN 62
IRL 60
JPN 59
KOR 59
CHL 58
USA 57
The International Civil Service Effectiveness Index | Technical Report 58
Policy making, fiscal and financial management such as efficiency and validity, when a correct
and human resources are the only indicators model is specified for the imputation39. An
where data is available for all metrics across imputation represents one set of plausible values
all countries currently included in the Index. for missing data, and so multiple imputations
However, this does not necessarily mean there represent multiple sets of plausible values.
are no limitations with the available data and For studies with roughly 10-60% missing values,
we aspire to continually improve data across past research suggests that multiple imputation
all indicators. There may be drawbacks to data is the method of choice.40
quality stemming, for example, from data which
is less current than is desirable, data measuring A number of methods of imputation (including MI)
concepts which are not a perfect fit with the use other variables to predict the missing values.
indicator in question, data which is based on MI requires assumptions to be made about
subjective assessment and data which uses the distribution of the variable and its predictor
the wider public sector as a proxy for the civil variables. Each missing value is imputed several
service. More detail on potential data limitations times, therefore generating several independent,
we have identified are set out in Annex A. completed data sets. Each completed data
set is analysed and then combined. This
For all other indicators, some degree of imputation method assumes that the data are
data imputation was required. A statistical missing at random, a hypothesis that cannot
methodology was adopted to estimate/ be verified, since we have no knowledge of
‘impute’ missing data. Section 5.2 describes the unobserved data.41 Nevertheless, the more
the approach to imputing missing data. predictors included in the imputation model,
the assumption that data is missing at random
5.2 Imputation method is more likely to hold because the uncertainty
associated with missingness is reduced.42
Researchers modeling data often encounter
the problem of missing data regarding one When a continuous variable contains missing
or more of the variables under investigation. values, a linear regression imputation
The most common approach is to remove those method can be used to fill in missing values.43
observations with missing values, allowing for a The method of multiple imputation utilised in
complete analysis of those for which full data is developing the Index fills in missing values in
available. However, this approach wastes data variables iteratively by using chained equations,
and reduces power, and also produces biased a sequence of univariate imputation methods.
estimates when the values are not missing This requires that a predictive model of the
completely at random38. One alternative is to use variables with missing values can be specified;
one of the many methods available for imputing the predictive model can include other variables
the missing values. Of the available methods, with missing values, while taking into account the
multiple imputation (MI) is attractive with problems associated with predictors that have a
theoretical and simulation studies showing that it high degree of missing data. Variables correlated
yields estimates with good statistical properties, to the variable with missing data need to be
utilised in the predictive model.
38 Little R. and Rubin D. (1987), Statistical analysis with missing data.
39 Little R. and Rubin D. (1987), Statistical analysis with missing data.
40 Barzi F. and Woodward M. (2004), Imputations of Missing Values in Practice: Results from Imputations of Serum Cholesterol in
28 Cohort Studies, American Journal of Epidemiology 160 (1): 34-45.
41 Ibid.
42 Shafer J.L. (1997), Analysis of incomplete multivariate data.
43 Rubin D. B. (1987), Multiple Imputation for Nonresponse in Surveys; Schenker, N., and Taylor J.M.G. (1996) Partially parametric
techniques for multiple imputation. Computational Statistics & Data Analysis 22: 425–446
59 The International Civil Service Effectiveness Index | Technical Report
The technique used for undertaking multiple For variables where strong correlations with other
imputation in the Index is multivariate imputation datasets are not observed, median imputation
using chained equations (MICE). MICE is very is used, with values imputed independently
popular in practice. Its popularity stems from of any predictor. With unconditional median
the flexibility it offers for imputing the different imputation, the median value of the variable
types of data in observational studies. The in question for the countries where data is
variable-by-variable specification of MICE allows available is substituted for each missing value.
practitioners to simultaneously impute variables Since all imputations are the same, this method
of different types by choosing from different will underestimate the variance for the variable.
univariate imputation methods appropriate for For those metrics where median imputation has
each variable. Being able to specify a separate been used, and for the functions and attributes
model for each variable provides flexibility in these metrics feed into, caution should be taken
incorporating certain characteristics specific to when interpreting the results and less weight
each variable.44,45 should be allotted to the relative position of
countries. However for interpreting the overall
Data from a wider range of countries than those results of the Index we are confident the use
included in the pilot edition of the Index has of median imputation has not led to significant
been utilised for multiple imputation. While the bias, due to the small proportion of missing data
overall availability of data for some countries led overall (10%) and the small proportion of missing
to our judgement to exclude them from the first data imputed using this technique (4%).
release of the Index, for particular datasets which
covered these countries, this data was utilised A mixture of multiple and median imputation is
for imputing values for the 31 countries included used in the estimation of missing data for the
in the Index. InCiSE Index. Further details and discussion
on the implementation of MI are available from
a number of sources46 and Section 5.3 sets
out the method of imputation used to estimate
missing data for each metric that feeds into the
Index, documenting the correlations observed
between variables and explaining the predictive
models used for multiple imputation.
44 For more information about multivariate imputation using chained equations, see van Buuren, Boshuizen, and Knook (1999);
Raghunathan et al. (2001); van Buuren et al. (2006); van Buuren (2007); White, Royston, and Wood (2011); and Royston (2004, 2005a,
2005b, 2007, 2009), among others.
45 The number of iterations necessary for MICE to converge depends on, among other things, the fractions of missing information
and initial values. The higher the fractions of missing information and the farther the initial values are from the mode of the posterior
predictive distribution of missing data, the slower the convergence, and thus the larger the number of iterations required. Current
literature suggests that in many practical applications a low number of burn-in iterations, somewhere between 5 and 20 iterations,
is usually sufficient for convergence. In any case, examination of the data and missing-data patterns is highly recommended when
investigating convergence of MICE.
46 StataCorp LP, Stata Multiple-Imputation Reference Manual, Release 13, 2013. For more information about multivariate imputation using
chained equations, see van Buuren, Boshuizen, and Knook (1999); Raghunathan et al. (2001); van Buuren et al. (2006); van Buuren
(2007); White, Royston, and Wood (2011); and Royston (2004, 2005a, 2005b, 2007, 2009), among others. For more information about
the compatibility of conditional specifications, see Arnold, Castillo, and Sarabia (2001); van Buuren (2007); and Arnold, Castillo, and
Sarabia (1999) and references therein.
The International Civil Service Effectiveness Index | Technical Report 60
5.3 Imputation approach For this indicator, multiple imputation was used.
Our analysis found that a number of Bertelsmann
for indicators
Sustainable Governance Indicators metrics are
As discussed in section 5.2, the dataset utilised suitable predictors for OECD iREG metrics.
for generating imputed data contains data for Our imputation model utilised these metrics
countries which are not included in the Index. as complete predictors in addition to the iREG
Data was imputed for all countries in this larger imputation variables.
dataset, and as such the approach includes
Crisis/risk management
imputation for some metrics where no imputed
data is present in the Index, because this covers For the crisis/risk management indicator,
a more restricted set of countries. Separate all metrics are missing for 8 countries in the
imputation models were developed for each Index and have been imputed. No obvious
indicator and for some indicators a number of predictor variables were discovered, so simple
imputation models were devised for the different median imputation was used. This assigns the
themes. In a small number of cases, metrics median value for countries where data was
that are not featured directly in the indicator available as the estimate of data for countries
framework are used as predictors in imputation where it is missing.
models for metrics that do feature in the Index,
Human resources management
due to their correlation with these metrics; this is
highlighted where relevant. For the human resources management indicator,
for all the metrics which contribute to the score,
data was available for every country in the Index
5.3.1 Functions
and no imputation was required.
Policymaking
Tax administration
For the policymaking indicator, for all the metrics
For tax administration, data is missing for one
which contribute to the score, data was available
or more metrics for 11 countries in the Index,
for every country in the Index and no imputation
although there are no countries where all data
was required.
is missing.
Fiscal and financial management
We found significant correlations between the
For the fiscal and financial management cost of collection ratios metric and the World
indicator, for all the metrics which contribute to Bank’s ‘Doing Business’ Index on the time it
the score, data was available for every country takes businesses to pay taxes. We also found
in the Index and no imputation was required. significant correlation between the three digital
Regulation metrics on e-filing of tax returns and the cost
of collection ratios. Correlation was weaker
Data was available for the metrics included in between the digital metrics and the time it takes
this indicator across every country in the Index to pay business taxes metric.
but one. The score for Regulation is calculated
based on six metrics from the OECD’s. The tax debt ratio appears to be correlated
‘Indicators of Regulatory Policy and Governance’ with the time it takes businesses to pay taxes
(iREG). Data was missing for two metrics; but the correlation is not statistically significant.
Regulatory Impact Assessment – Primary laws With these observed relationships we proceeded
and Stakeholder Engagement – Primary Laws. with a multiple imputation approach with all
metrics included as predictors, except for the tax
debt ratio. The tax debt ratio metric was imputed
using simple median imputation.
61 The International Civil Service Effectiveness Index | Technical Report
Social security administration 5.3.2 Attributes
For social security administration only one
Integrity
metric is currently included in the indicator.
Data is missing for 8 countries in the Index and For integrity sixteen metrics are included in
data has been imputed for these countries the indicator and data is missing for four of
using multiple imputation. We considered and the metrics. The Global Corruption Barometer
examined a wide range of metrics outside of metric measuring the percentage of people
this indicator to obtain a suitable predictor viewing public officials/civil servants as corrupt is
and observed a correlation with one of the missing data for 5 countries in the Index. OECD
dimensions of the Quality of Government Expert data on whether a post-employment cooling off
Survey. We acknowledge that the observed period exists and whether it’s paid, data on the
correlation does not identify or imply a causal degree of protection against lobbyists and other
relationship between these metrics and we wish private interests influencing advisory groups,
to identify and develop data to strengthen this and data on the number of groups who receive
indicator as a priority. whistleblower protection are also missing for
1 country for the first metric and 7 countries for
Digital services the latter two metrics.
The score for Digital Services is calculated
The imputation approach taken considered these
based on metrics taken from the European
metrics in two groups. Excluding the three OECD
Commission’s E-Government Benchmarks.
metrics with missing data, we found all integrity
All metrics are missing for 8 countries
metrics are highly correlated, noting that the QoG
in the Index and have been imputed by
expert assessment of the extent to which public
multiple imputation. We found that the Online
sector employees strive to fulfil ideology of party
Service Index metric of the UN e-Government
in government is negatively correlated. On this
Development Index47 was generally correlated
basis we proceeded with multiple imputation for
with all the E-Government Benchmarks
missing data in this group of metrics with all as
metrics and so this metric was utilised as
predictors for each other.
a complete predictor.
For the second group, our examination of the
data showed that among the three OECD
metrics in this group with missing data, data for
each metric is fairly clustered around one point
and simple median imputation is used.
47 UN E-Government Development Index https://publicadministration.un.org/egovkb/en-us/Data-Center
The International Civil Service Effectiveness Index | Technical Report 62
Openness Inclusiveness
The approach to imputation for openness For capabilities, five metrics are included in the
treated metrics concerning societal consultation indicator and data is missing for two of these;
separately from those concerning transparency. the OECD data covering the share of total central
There are four metrics on the consultation side government employment filled by women and
with five metrics on the transparency side. the share of employment in top management
positions within central government filled by
For societal consultation, we observed women as absolute differences in comparison
correlation between the metrics with missing to each of those equivalent shares of women
data (the ‘complaint mechanisms’ and in the labour force as a whole. For these two
‘civic participation’ components of the Open metrics data is missing for 10 and 12 countries
Government Index) and the metrics with non- in the Index respectively. We proceeded with
missing data (UN’s E-Participation Index and multiple imputation for missing data with all
‘Negotiating public support’ Bertelsmann metric). metrics utilised as predictors for each other.
Multiple imputation was undertaken with these
metrics utilised as complete predictors.
For the transparency group of metrics, we found
all metrics are highly correlated. On this basis we
proceeded with multiple imputation for missing
data in this group, with all metrics utilised as
predictors for each other.
Capabilities
For capabilities, four metrics are included in the
indicator and data is missing for all of these.
For 3 of the metrics 10 countries were missing
data and for one of the metrics 13 countries
were missing.
We examined the relationship of these metrics
to others outside of the capabilities indicator.
The most suitable predictors observed were the
QoG expert assessment of the extent to which
the skills and merits of the applicants decide
who gets the job when recruiting public sector
employees and EU membership; both metrics
are positively associated with skills variables.
Our imputation model utilised these metrics as
complete predictors.
63 The International Civil Service Effectiveness Index | Technical Report
Chapter 6:
Index results
In this chapter we describe the approach to the normalisation and reporting of Index
results and advise on their interpretation.
6.1 Normalisation Worked Example
The normalisation method is illustrated below
The data sets which feed into the Index capture
for a metric underpinning the Integrity indicator;
a range of aspects of the civil service through
the Quality of Government Institute expert
survey respondents, expert opinion and
assessment of the extent to which public sector
administrative data. Additionally, the datasets
employees strive to help citizens.
utilise a variety of different measurement scales.
To enable comparison of scores and for the The first column of the table presents the country
purposes of aggregating the data to provide scores prior to normalisation.
scores for the different functions and attributes,
and for the composite Index, we transform the For each country score, the minimum score
data into comparable units with the same scale – observed for the metric (2.32) is subtracted
a process known as normalisation. from this score and the result is then divided
by the difference between the maximum (6.29)
The normalisation method we employ in our pilot and minimum scores observed. This provides
InCiSE Index is the Min-Max method. The Min- the normalised score for a country. For example
Max48 process of normalisation preserves the Australia’s score before normalisation is 5.48.
distribution of the data and scales all numeric Subtracting the minimum score of 2.32 from
variables in the range [0,1]. Scoring 1 means that this score gives 3.16. The difference between
country has the highest score on that metric, 0 the maximum and the minimum scores is
the lowest. An example of normalisation using 3.97. Dividing 3.16 by 3.97 gives Australia’s
the Min-Max methodology is as follows: normalised score of 0.80 to 2 decimal places.
48 Handbook on Constructing Composite Indicators, OECD
The International Civil Service Effectiveness Index | Technical Report 64
Table 19: Normalisation example
Country QoG Institute expert assessment Normalised score
of the extent to which public
sector employees strive to
help citizens
AUS 5.48 0.80
AUT 5.43 0.78
BEL 5.43 0.78
CAN 6.24 0.99
CHL 5.40 0.78
CZE 4.00 0.42
DNK 5.53 0.81
EST 4.63 0.58
FIN 5.33 0.76
FRA 5.09 0.70
DEU 5.03 0.68
GRC 3.60 0.32
HUN 3.73 0.36
IRL 5.00 0.68
ITA 3.70 0.35
JPN 5.75 0.87
KOR 4.74 0.61
MEX 2.32 0.00
NLD 5.21 0.73
NZL 6.29 1.00
NOR 5.57 0.82
POL 5.33 0.76
PRT 4.53 0.56
SVK 3.89 0.40
SVN 4.75 0.61
ESP 4.91 0.65
SWE 5.92 0.91
CHE 5.60 0.83
TUR 3.92 0.40
GBR 5.18 0.72
USA 5.25 0.74
65 The International Civil Service Effectiveness Index | Technical Report
In mathematical terms, each data point for a 6.2 Reporting of results
country and a time is transformed according
to the formula given below: For the pilot edition of the Index, results are
reported for each metric online, normalised
between 0 and 1 according to the method
set out in Section 6.1, following imputation
where required. Results for each indicator,
the 8 functions and 4 attributes, are also
reported online and within the Main Report.
where and are the These indicator results are aggregated from
minimum maximum value of across all the contributing metrics according to the
countries at time .49 weightings described in Section 4.2.
One issue with standardising the range but not The scores for the composite (the overall) Index
the variance can be that, if you have outliers in are built from the indicator scores according to
your data set, normalising your data will scale the the weighting approach set out and described
‘normal’ data to a very small interval. Variables in more detail in Section 7.2. These are provided
with higher variation will have more effect on the both online and within the Main Report.
final outcome than variables with lower variance.
However, analysis of our data has shown the The normalised results are presented by way of
standard deviations of the normalised variables radar graphs, an example is shown in Figure 8.
are all between 0.18 and 0.35 (the maximum
value for this standardised range would be 0.5).
Looking ahead to future editions of the Index,
another variant of the Min-Max method is,
where the minimum and maximum for each
variable are calculated across countries and time,
in order to take into account the evolution of the
indicators.50 In future editions of the index the
distinction between developments in absolute
and relative performance of countries may be a
pertinent analytical question. For example, it is
possible for absolute and relative performance to
move in opposite directions over time. Adopting
such a method may help to address the issue of
comparability between years.
This transformation is not stable when data for a
new time point becomes available. This implies
an adjustment of the time period analysed,
which may affect the minimum and maximum
for some variables and hence the values of .
Maintaining comparability between the existing
and new data would require the Index for the
previous years to be recalculated.51
49 Ibid.
50 Ibid.
51 Ibid.
The International Civil Service Effectiveness Index | Technical Report 66
5/26/2017 Radar chart
Figure 8 Radar Graph Example – Poland
Tax
administration
Policy
making Inclusiveness
0.75
Social
security Capabilities
0.50
0.25
Digital
services Openness
Fiscal
and Integrity
financial
management
Key
Regulation Human Poland
Crisis resource Average (means) scores
management management
The normalised data, and radar diagrams drawn from this, allow for an assessment of relative
performance of the group of countries included in the Index. So for example, it may allow statements
such as ‘Country X is assessed to perform more strongly on the Inclusiveness indicator than country
Y’ or ‘Country X is assessed to perform more strongly than the average for all countries in the Index’.
The indicator and composite Index results should be interpreted in the context of the quality and
depth of the underlying metrics, and the degree and method of imputation utilised to estimate
missing data; these are all described within this Technical report.
A low score does not necessarily mean that a country is performing badly, or that such improvements
are more pressing, than for an indicator which has a higher score. The nature of a normalised
comparative index means that it is showing a comparative assessment against other countries,
not an absolute assessment. As such, relatively weaker scores may aid identification of areas for
improvement or learning from more strongly performing countries, but will not indicate priorities
for reform and attention on their own. In Figure 8, while the relative score for Crisis Management
may be higher than that for Regulation, this does not imply that a country will rank more highly
https://inciseradarvis.cloudapps.digital/ 2/6
for this indicator. The average relative performance (score) for Crisis Management is higher than
for Regulation, such that Poland’s score for Crisis Management is somewhat below the average,
whereas its score for Regulation is much closer to the average. The higher average score for Crisis
Management reflects the grouping of country results which are closer to the leading country than for
Regulation.
The distribution of results within the Index, indicators and underlying metrics shows the degree of
closeness of results in various parts of the ranking, which may indicate whether a particular country
is the clear leader (or follower) in performance or whether there is a wider group of relatively strong
(weak) countries with similar scores. Information covering the distribution of results is provided in the
Main Report, with further detail available in figures 9 and 10, which show the ranking and distribution
of indicator scores.
67 The International Civil Service Effectiveness Index | Technical Report
Figure 9 Bar Chart Examples
The bar chart examples demonstrate the variation observed in the distribution of results across the
function and attribute indicators. For example, the 10 strongest performing countries for Fiscal and
Financial Management are more tightly grouped in their relative performance than the strongest
performing countries for Regulation. At the other end of the distribution, scores among the weakest
performing countries for Regulation are in fact more tightly grouped than the weakest performing
group for Fiscal and Financial Management; this can be observed from the range of scores occupied
by the bottom 10 countries for each indicator. The composite Index weights the scores for indicators
rather than the ranks.
The International Civil Service Effectiveness Index | Technical Report 68
Figure 10 Distribution of Indicator scores
Figures 10 provides an alternative, summary view, of the distribution of indicator scores. Clustering of
scores can be seen for some indicators, while others are more equally spread.
Integrity Mexico New Zealand
Openness Turkey United Kingdom
Capabilities Italy Japan
Inclusiveness Hungary Poland
Policymaking Slovenia United Kingdom
Fiscal and Financial Management Hungary Switzerland
Regulation Chile Mexico
Crisis/Risk Management Germany Turkey
Human Resource Management Slovakia Ireland
Tax Administration Czech Republic Estonia
Social Security Administration Ireland United Kingdon
Digital Services Slovakia Estonia
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
Lowest score Highest score Median
68 The International Civil Service Effectiveness Index | Technical Report
Chapter 7: Composite
In chapter 7 we describe the advantages and disadvantages associated with the use of
composite measures. We then describe the construction and weighting of the InCiSE
composite and how our approach mitigates against some of the issues with composite
indicators that have been identified.
7.1 Strengths and weaknesses of composites
A composite index is the result of compiling individual indicators into a single index in accordance
with an underlying framework. The composite can measure multi-dimensional concepts which cannot
be captured by a single indicator.
Composite indicators which compare country performance are increasingly recognised as a useful
tool in policy analysis and public communication. However, composite indicators can send misleading
policy messages if they are poorly constructed or misinterpreted.
The main strengths and weaknesses of using composite indicators are included in Table 2052.
Table 20: Strengths and weaknesses of composite indicators.
Pros: Cons:
• Can summarise complex, multi-dimensional • May send misleading policy messages if poorly
realities with a view to supporting decision makers. constructed or misinterpreted.
• Are easier to interpret than a battery of many • May reduce complex things to simple
separate indicators. rankings which could lead to overly simplistic
• Can assess progress of countries over time. interpretations.
• Reduce the visible size of a set of indicators • May be misused, e.g. to support a desired
without dropping the underlying information base policy, if the construction process is not
transparent and/or lacks sound statistical or
• Thus make it possible to include more
conceptual principles.
information within the existing size limit; the
number of underpinning measures can be much • The selection of indicators and weights could be
higher than the number of composite measures. the subject of political dispute.
• By providing summary measures and visualisations, • May disguise serious failings in some dimensions
can place issues of country performance and and increase the difficulty of identifying proper
progress at the centre of the policy arena. remedial action, if the construction process is
not transparent.
• Facilitate communication with general public (i.e.
citizens, media, etc.) and promote accountability. • May lead to inappropriate policies if dimensions
of performance that are difficult to measure
• Help to construct/underpin narratives for lay and
are ignored.
literate audiences.
• Enable users to compare complex dimensions
effectively
52 Saisana M. and Tarantola S. (2002), State-of-the-art report on current methodologies and practices for
composite indicator development, EUR 20408 EN, European Commission-JRC: Italy
The International Civil Service Effectiveness Index | Technical Report 69
The table sets out some of the most important, To account for these limitations and to promote
and connected, benefits and drawbacks of transparency in deploying a composite index,
developing and utilising composite indicators. we have taken the following steps:
Our reasoning for utilising a composite indicator
can be summarised as follows: i) We describe our favoured approach
to constructing a composite Index in
“The key strength of aggregate indicators is their Section 7.2.
ability to convey information on many parameters
succinctly (Booysen, 2002; Hahn, 2008; Zhou & ii) We have tested and consulted on the chosen
Ang, 2009; Balica, 2012b). Therefore, composite approach widely, refining and adjusting the
indices are powerful and communicative tools method over time; seeking the views of a
because they present clear and concise results range of multilateral organisations, academic
to non-technical audiences such as scores institutions and other experts in the field of
or rankings (Kenney et al., 2012). That helps public governance, including those most
to promote a multi-stakeholder dialogue familiar with the usage and drawbacks of
in establishing common understanding of composite measures.
supranational concerns and overcoming socio- iii) To promote a fuller understanding of the
political barriers of decision making (Preston et ramifications of different approaches to
al., 2011: 183). The two main advantages of weighting of indicators, we have also
aggregate measures are: tested the impact of a range of alternative
-
Variables that cannot be directly observed weightings on the Index results. More
may be inferred by integrating multiple information about the sensitivity analysis
indicators as part of a composite indicator. is provided in Chapter 8. -
Composite indices usage helps to overcome v) Alongside the composite Index scores
the problems of precision, reliability and and rankings, we present ranks across all
accuracy by reducing the influence of indicators in our Main Report. This provides
measurement error as the number of a fuller picture of how the Index is built up
observations from multiple sources increase from the indicators.
(Kaufmann & Kraay, 2007; Maggino &
Zumbo, 2012).53” 7.2 The InCiSE composite
Bearing these potential benefits in mind, The InCiSE Index brings together the indicators,
we remain aware that even the most carefully each measuring a different dimension of
constructed of these indices can lack effectiveness into an overall composite indicator
transparency and comparability over time, of civil service effectiveness.
suffer from selection bias and be of limited use
in helping countries to identify how effectively
to improve the quality of the civil service.54
53 Muriithi et al (2015), Quantifying Governance: An indicator-based approach, Report for DFID by MPA students at the LSE
54 Arndt C. and Oman C. (2006), Uses and Abuses of Governance Indicators, OECD Development Centre Studies
70 The International Civil Service Effectiveness Index | Technical Report
Weighting Our preference would be for each attribute
The measurement framework outlined in and function indicator to be given equal weight
Section 3 gives a comprehensive overview of within their respective shares of the Index
civil service effectiveness. However, existing data weighting. Due to poorer data quality on two of
does not enable one to measure against all of it. the attributes, Capabilities and Inclusiveness,
Of the 6 attributes proposed for measurement, their weighting is reduced. A similar adjustment
so far 4 are included with an individual indicator. could be made for Tax and Social Security
Of the 11 core functions proposed, 8 are Administration, but given their overwhelming
included with an individual indicator. There are importance as civil service functions it was
12 indicators in total. A number of data sources, judged that reducing their weight would not
most with multiple metrics contained within be appropriate. Figure 11 sets outs the Index
them, underlie the 12 indicators in the Index. weighting of scores. The Index is more heavily
In total, 76 variables are spread across these weighted towards functions as a greater number
indicators. of metrics were available, and these were
generally judged to be of better data quality
A weighted average of scores on each indicator for the purpose of our Index. As a result, core
then gives the Index; an overall composite score. functions were given two-thirds of the overall
Functions make up two-thirds of the overall composite weight, and attributes one-third.
weight and attributes one-third.
The International Civil Service Effectiveness Index | Technical Report 71
Figure 11: Data weightings in the InCiSE Index
A B C D
Integrity /
1 18 16 /
1 12 /
19
Openness /
1 18 9 /
1 12 /
19
Attributes Capabilities /
1 18 4 /
1 12 /
19
Weight: 1/3
Metrics: 34 Inclusiveness /
1 18 5 /
1 12 /
19
Staff Engagement /
1 18 – – –
Innovation /
1 18 – – –
Policymaking /
2 33 8 /
1 12 /
1 12
Central Fiscal and financial /
2 33 3 /
1 12 /
1 12
Executive Regulation /
2 33 6 /
1 12 /
1 12
Risk/Crisis Management /
2 33 9 /
1 12 /
1 12
Procurement /
2 33 – – –
Core functions
Mission HR /
2 33 5 /
1 12 /
1 12
Weight: 2/3
Metrics: 42 Support IT /
2 33 – – –
Finance /
2 33 – – –
Tax Administration /
2 33 6 /
1 12 /
1 12
Direct
Social Security /
2 33 1 /
1 12 /
1 12
Service
Administration
Delivery
Digital Services /
2 33 4 /
1 12 /
1 12
A Preferred weight if data on all-indicators was available (equal weight within attributes and functions)
B Number of data metrics found and included
C Preferred weight given missing data for some indicators (equal weight within attributes and functions)
D Data quality-adjusted weight (used in this Index)
At a further level of disaggregation, Section 4 describes how the individual metrics are weighted in
each of the indicators they support.
72 The International Civil Service Effectiveness Index | Technical Report
Chapter 8:
Sensitivity Analysis
In chapter 8 we describe the sensitivity analysis undertaken, and the resultant effect on the
pilot Index results.
8.1 Introduction to sensitivity In the sensitivity analysis undertaken for
the InCiSE Index we focus on four main
analysis
assumptions: adjusting for GDP per capita,
The development of the Index involves stages the method of aggregation, exclusion of one
where subjective judgements have to be limiting dataset at a time and imputation of
made: the selection of individual data sets, missing data.
the treatment of missing values, the choice
of aggregation model and the weights of the 8.2 Sensitivity analysis 1:
indicators, for example. Adjusting for GDP per Capita
The quality of the Index depends on the Despite its acknowledged shortcomings,
soundness of its assumptions. Good modelling GDP per capita is still the most commonly used
practice requires that we develop an evaluation summary indicator of living standards. Much
of the confidence in the model, assessing the of the policy advice provided by international
uncertainties associated with the modelling organisations is framed on cross-country
process and the subjective choices taken. This comparisons of per capita GDP.55 The framework
role is fulfilled by sensitivity analysis, scrutinising established by the InCiSE Index sets out to
aspects of the relationships between inputs into measure and compare effectiveness of civil
the Index and its final output. services through an approach that is comparable
The approach taken to sensitivity analysis shows across countries. However, the economic
how variation in the output can be apportioned, resources available to develop and maintain
qualitatively or quantitatively, to different sources the effective institutions and practices of civil
of variation in the assumptions, and of how the services, and governments’ ability to raise
Index depends upon the information fed into revenues through taxation, are dependent on
it. Sensitivity analysis can help to gauge the the level of economic development and income
robustness of the composite indicator ranking levels of the economy, as measured by GDP
and increases transparency, to identify which per capita.
countries are assessed more or less favourably
under certain assumptions and to help develop
a debate around the Index.
55 World Economic Outlook, April 2003 – Chapter 3: Growth and Institutions
The International Civil Service Effectiveness Index | Technical Report 73
Research on growth and institutions has • The method used here is as follows: the
sought to identify the deep structural country’s Index score (prior to normalisation)
determinants of countries’ level of development. is divided by its GDP per capita, scores
The various measures of institutional quality are then normalised between 0 and 1.
reflect the generally high correlations among For example; if two countries had obtained
themselves and measures of economic identical scores in the Index but one country
performance (Table 21). All appear closely (A) had a GDP per capita twice that of the
related to cross-country differences in GDP other (B), then after adjusting for GDP, B’s
per capita, as well as to other measures of score (and position) would be higher than A’s.
economic performance such as growth rates
and the volatility of growth. Building on the Secondly, investigating this sensitivity allows
close correlation between institutional quality for comparison of performance across smaller
and development, recent analyses attempt groups of countries with similar income/GDP
to address the possibility of reverse causality per capita characteristics. In future, this may
from development to institutions, and the enhance the possibility of the Index serving
relative significance of institutions compared as a tool to drive performance improvement,
with other influences on development, for example by allowing decision makers to see
such as trade openness, geographical factors, which countries perform best at a similar income
and economic policies.56 level, this may support adopting arrangements
and improvements of practices that are more
Those countries with higher GDP per capita transferable between countries of comparable
may also be historically more stable, with income levels.
greater social capital and more established
institutions. While these are recognised as Thirdly, this adjustment allows for what might be
aspects which may support a highly effective considered a ‘fairer’ approach to assessment;
civil service, in considering the reverse causality assessing effectiveness in light of varying income
from development to institutions, and the impact levels and resources among countries, and
of other influences – it is helpful to test the estimating whether countries are stronger or
sensitivity of adjusting the Index for GDP per weaker performers considering their relative
capita for several reasons. resource constraints. This adjustment may
provide insights into capacity for effectiveness
Firstly, it allows us to measure and estimate the or potential for improved effectiveness, and may
importance of income levels for Index scores by enhance the impact and take up of the Index
assessing how Index scores are affected when among those countries with lower incomes and
the following adjustments are made: perhaps lower scores in the core Index.
• Scores and relative position are revised
to reflect income levels, the extent of the
revision depends on the difference between
a country’s GDP per capita and the average,
and also its original score in the Index.
56 Kaufmann, Kraay, and Zoido-Lobatón (1999b); Heritage Foundation (2003); Gurr and Marshall (2000); and World Development
Indicators, World Bank (2002)
74 The International Civil Service Effectiveness Index | Technical Report
Table 21: Index rankings comparison after adjustment for GDP per capita
Country Index Score Rank GDP Adjusted Rank after
Index Score adjustment
Canada 1.00 1 0.76 5
New Zealand 0.95 2 0.87 3
Australia 0.91 3 0.67 9
United Kingdom 0.91 4 0.74 6
Finland 0.87 5 0.71 7
Sweden 0.86 6 0.62 11
Estonia 0.81 7 1.00 1
Norway 0.81 8 0.43 18
Korea, Republic of 0.78 9 0.76 4
United States of America 0.74 10 0.43 19
Denmark 0.73 11 0.52 13
Switzerland 0.61 12 0.32 24
Belgium 0.60 13 0.45 17
Netherlands 0.58 14 0.40 21
Japan 0.57 15 0.52 14
Austria 0.56 16 0.38 22
France 0.50 17 0.42 20
Spain 0.49 18 0.48 15
Mexico 0.47 19 0.95 2
Ireland 0.46 20 0.22 26
Poland 0.44 21 0.58 12
Chile 0.44 22 0.68 8
Slovenia 0.44 23 0.48 16
Germany 0.40 24 0.27 25
Turkey 0.37 25 0.66 10
Portugal 0.31 26 0.36 23
Italy 0.21 27 0.19 27
Czech Republic 0.16 28 0.16 28
Greece 0.04 29 0.05 29
Hungary 0.00 30 0.01 30
Slovakia 0.00 31 0.00 31
The International Civil Service Effectiveness Index | Technical Report 75
8.3 Sensitivity analysis 2: While our approach to aggregation is reasonable
and justified, we acknowledge there are a
Aggregation Method
range of possible approaches and weightings
The InCiSE index brings together the underlying which could be followed in building up the
data sets into ‘indicators’; attributes and composite from these component indicators,
functions, each measuring a different dimension with corresponding arguments in favour. Indeed,
of effectiveness. These attributes and functions there are also a range of possibilities for building
are then drawn into an overall composite up the indicators from their constituent datasets
indicator of civil service effectiveness. Attributes (Chapter 4 provides further detail as how the
contribute one third of the composite weighting indicators are weighted across the contributing
and functions make-up the remaining two thirds datasets). As part of the sensitivity analysis
(see Chapter 7 for further detail and justification undertaken, we investigated the impact of
for the aggregation approach adopted). alternative aggregation approaches to test the
impact on the composite results obtained from
the particular approach adopted.
We first looked at the impact of adjusting the
weighting between attributes and functions
such that each contributes half of the composite
weighting. Results are shown in Table 22.
76 The International Civil Service Effectiveness Index | Technical Report
Table 22: Comparison of scores and ranking under adjusted aggregation method. Attributes and
Functions each provide half the weighting.
Country Index score Rank Index score Rank
adjusted for
equal weighting
Canada 1.00 1 1.00 1
New Zealand 0.95 2 0.98 2
Australia 0.91 3 0.90 3
United Kingdom 0.91 4 0.89 5
Finland 0.87 5 0.89 4
Sweden 0.86 6 0.85 6
Estonia 0.81 7 0.75 8
Norway 0.81 8 0.83 7
Korea, Republic of 0.78 9 0.73 11
United States of America 0.74 10 0.75 9
Denmark 0.73 11 0.74 10
Switzerland 0.61 12 0.64 12
Belgium 0.60 13 0.61 15
Netherlands 0.58 14 0.63 13
Japan 0.57 15 0.61 14
Austria 0.56 16 0.59 16
France 0.50 17 0.51 17
Spain 0.49 18 0.48 19
Mexico 0.47 19 0.37 24
Ireland 0.46 20 0.47 21
Poland 0.44 21 0.49 18
Chile 0.44 22 0.46 22
Slovenia 0.44 23 0.46 23
Germany 0.40 24 0.48 20
Turkey 0.37 25 0.30 26
Portugal 0.31 26 0.31 25
Italy 0.21 27 0.19 27
Czech Republic 0.16 28 0.17 28
Greece 0.04 29 0.08 29
Hungary 0.00 30 0.00 31
Slovakia 0.00 31 0.02 30
Also considered is an alternative weighting where attributes contribute two thirds of the composite,
with functions making up the remaining third. Results for this sensitivity are shown in Table 23.
The International Civil Service Effectiveness Index | Technical Report 77
Table 23: Comparison of scores and ranking under adjusted aggregation method. Attributes two
thirds weighting and Functions one third.
Country Index score Rank Index score Rank
adjusted for two
thirds attributes
weighting
Canada 1.00 1 0.99 2
New Zealand 0.95 2 1.00 1
Australia 0.91 3 0.89 4
United Kingdom 0.91 4 0.86 5
Finland 0.87 5 0.92 3
Sweden 0.86 6 0.84 7
Estonia 0.81 7 0.68 11
Norway 0.81 8 0.85 6
Korea, Republic of 0.78 9 0.68 12
United States of America 0.74 10 0.75 9
Denmark 0.73 11 0.76 8
Switzerland 0.61 12 0.67 13
Belgium 0.60 13 0.62 15
Netherlands 0.58 14 0.68 10
Japan 0.57 15 0.64 14
Austria 0.56 16 0.61 16
France 0.50 17 0.53 19
Spain 0.49 18 0.47 23
Mexico 0.47 19 0.27 25
Ireland 0.46 20 0.47 22
Poland 0.44 21 0.54 18
Chile 0.44 22 0.49 20
Slovenia 0.44 23 0.48 21
Germany 0.40 24 0.55 17
Turkey 0.37 25 0.24 26
Portugal 0.31 26 0.31 24
Italy 0.21 27 0.17 28
Czech Republic 0.16 28 0.19 27
Greece 0.04 29 0.11 29
Hungary 0.00 30 0.00 31
Slovakia 0.00 31 0.05 30
78 The International Civil Service Effectiveness Index | Technical Report
From the results of this sensitivity analysis 8.4.1 Subjective assessment
it appears while there is an impact on the
Index scores and rankings obtained under As noted earlier, the InCiSE Index is based on
the two alternative aggregation methods, this a mixture of administrative data and subjective
impact is small in relation to the overall Index; or perceptions‐based measures, taken from
most countries obtain very similar scores and surveys of households and firms as well as
rankings under the alternative aggregation expert assessments produced by various
approaches when compared to the approach organisations. Our view is that perceptions data
followed in the Index. This is reassuring, and have particular value in the measurement of
is indicative of the high degree of correlation governance. First, perceptions matter because
between the indicators. However, naturally if agents base their actions on their perceptions.
the weighting possibilities were pushed further If citizens believe that the courts are inefficient
towards their limits or extended to the level of or the police are corrupt, they are unlikely to
the data sets underlying the indicators, more avail themselves of their services. Similarly,
significant impacts on the scores and rankings enterprises base their investment decisions
might be observed; although the merits of ‐ and citizens their voting decisions ‐ on their
such an alternative weighting would likely be in perceived view of the investment climate and
question. Our favoured approach to aggregation the government’s performance. Secondly, in
appropriately takes into account a broad many areas of the Index framework, there are
range of attributes and functions to allow for a few alternatives to relying on perceptions data.
reasonable assessment of overall civil service For instance, this has been particularly the case
effectiveness. for Integrity, an attribute in the framework, which
almost by definition leaves few records that
8.4 Sensitivity analysis 3: Exclusion can be captured by purely objective measures.
Additionally even when objective or fact‐based
of Input Data data are available, this may capture the notion of
In Chapter 4 we set out the data underpinning laws and arrangements “in theory”, which often
each of the indicators and the weighting differs substantially from the reality that exists
given to each to make up the indicator score. “in practice”. For example, in every one of the 70
Where we have identified limitations in the countries covered in the 2007 and 2008 waves
data used, these are specified. It is important of the Global Integrity Index, it is formally illegal
to examine the results of the Index and for a public official to accept a bribe. Yet, despite
investigate whether utilising data which has them being identical when measured in theory,
limitations has any skewed effect on the results there are large differences across these countries
obtained. We therefore examine the exclusion in perceptions of the frequency with which bribes
of three categories of data that have general are in fact accepted by public officials.
limiting qualities; data resulting from subjective
Despite these advantages, one might
assessment, data deemed to be out of date,
nevertheless reasonably be concerned about
and data where the wider public sector is used
various potential problems in the interpretation
to proxy the civil service.
of the subjective data we rely upon in the Index.
Broadly such concerns question the extent to
which perceptions data adequately capture the
relevant reality. A first basic issue is simply that
perceptions data on civil service effectiveness
are imprecise. This by itself is not surprising
– as we have argued above, all measures of
effectiveness are necessarily imprecise proxies
for the outcomes they are intended to measure,
but imprecision alone does not disqualify the use
of perceptions‐based data.
The International Civil Service Effectiveness Index | Technical Report 79
A potentially more serious concern is that there Yet another potential source of bias comes
are various systematic biases in perceptions from the possibility that different providers
data on civil service effectiveness. One possibility of governance perceptions data rely on
is that different types of respondents differ each other’s assessments, and as a result
systematically in their perceptions of the same make correlated perceptions errors. This
underlying reality. For example, it could be would undermine the information content in
the case that business people, represented such indicators. Assessing this concern is
by owners of the businesses covered in a difficult because the high correlation between
survey, or the expert assessments provided governance perceptions rankings from different
by commercial business information providers, sources could be due either to perception errors,
have different views of what constitutes an or due to the fact that these sources are in fact
effective civil service than other types of accurately measuring cross‐country corruption
respondents, such as households or public differences and so necessarily agree with
sector agencies. each other.
Another possibility is that biases are introduced Concern about these potential sources of bias
by the ideological orientation of the organisation will be moderated to the extent that excluding
providing the subjective assessments of underlying data based on subjective assessment
governance. An additional type of bias might affects the resulting Index scores and rankings.
be the possibility that subjective assessments The results of this aspect of the sensitivity
of governance are driven by factors other analysis are demonstrated in Table 24.
than governance itself, such as the level of
development or recent economic performance
of a country.
80 The International Civil Service Effectiveness Index | Technical Report
Table 24: Sensitivity analysis – Removal of subjective data
Country Index score Rank Adjusted Index Adjusted Rank
Score
Canada 1.00 1 0.94 4
New Zealand 0.95 2 0.95 3
Australia 0.91 3 0.93 5
United Kingdom 0.91 4 0.92 6
Finland 0.87 5 0.87 7
Sweden 0.86 6 0.98 2
Estonia 0.81 7 1.00 1
Norway 0.81 8 0.83 8
Korea, Republic of 0.78 9 0.76 9
United States of America 0.74 10 0.66 11
Denmark 0.73 11 0.70 10
Switzerland 0.61 12 0.62 13
Belgium 0.60 13 0.46 20
Netherlands 0.58 14 0.61 14
Japan 0.57 15 0.48 18
Austria 0.56 16 0.63 12
France 0.50 17 0.48 19
Spain 0.49 18 0.51 17
Mexico 0.47 19 0.57 15
Ireland 0.46 20 0.36 23
Poland 0.44 21 0.24 26
Chile 0.44 22 0.32 25
Slovenia 0.44 23 0.54 16
Germany 0.40 24 0.33 24
Turkey 0.37 25 0.44 21
Portugal 0.31 26 0.37 22
Italy 0.21 27 0.14 28
Czech Republic 0.16 28 0.21 27
Greece 0.04 29 0.02 30
Hungary 0.00 30 0.00 31
Slovakia 0.00 31 0.05 29
The International Civil Service Effectiveness Index | Technical Report 81
8.4.2 Out of date data The project aspires to utilise the best available
data and where possible, this data should be
The project aims to define a framework recent. Where data is not released annually and
comprising the necessary aspects constituting is less recent, the inclusion of data sets has
an effective civil service, in a way that can been based on a judgement of its relevance to
realistically enable international data to be the framework despite this drawback. However,
collected to measure against it. The framework inclusion of these datasets may reduce the
developed takes into account a range of extent to which the composite reflects recent
functions and attributes of civil services and developments towards greater or reduced
utilises data from a range of sources. Many of civil service effectiveness among the countries
these data sources are released and updated included in the Index. A country that has made
annually, such that on release of the Index, rapid progress along a number of fronts may
we can be confident that the data utilised be dragged back by the inclusion of a measure
presents a recent and highly relevant assessment which is somewhat out of date and drawn from
of the aspect of effectiveness this data a period before the effects of such initiatives
underpins. However, some of the data sources had been felt. Table 25 shows the impact on
are updated less frequently, were released the rankings of excluding data released prior to
with the scheduled update process still to be 2013, the year judged as a cut-off point in this
confirmed, or were published with no scheduled sensitivity analysis.
updates envisioned.
82 The International Civil Service Effectiveness Index | Technical Report
Table 25: Sensitivity analysis – Removal of data pre-2013
Country Index Score Rank Adjusted Index Adjusted Rank
Score
Canada 1.00 1 1.00 1
New Zealand 0.95 2 0.99 2
Australia 0.91 3 0.95 3
United Kingdom 0.91 4 0.92 5
Finland 0.87 5 0.94 4
Sweden 0.86 6 0.88 6
Estonia 0.81 7 0.80 9
Norway 0.81 8 0.88 7
Korea, Republic of 0.78 9 0.75 12
United States of America 0.74 10 0.80 10
Denmark 0.73 11 0.81 8
Switzerland 0.61 12 0.78 11
Belgium 0.60 13 0.73 13
Netherlands 0.58 14 0.71 14
Japan 0.57 15 0.63 16
Austria 0.56 16 0.61 17
France 0.50 17 0.57 18
Spain 0.49 18 0.53 20
Mexico 0.47 19 0.40 23
Ireland 0.46 20 0.69 15
Poland 0.44 21 0.48 22
Chile 0.44 22 0.56 19
Slovenia 0.44 23 0.38 24
Germany 0.40 24 0.52 21
Turkey 0.37 25 0.35 25
Portugal 0.31 26 0.32 26
Italy 0.21 27 0.24 28
Czech Republic 0.16 28 0.26 27
Greece 0.04 29 0.05 29
Hungary 0.00 30 0.04 30
Slovakia 0.00 31 0.00 31
The International Civil Service Effectiveness Index | Technical Report 83
8.4.3 Public sector proxy
The Index is intended to cover the effectiveness
of the civil service as defined in Chapter 2,
rather than the wider public sector. Occasionally,
however, due to availability of data, data covering
the wider public sector data is used to proxy
for the civil service where this is deemed to be
a reasonable proxy. To ensure that the use of
such proxies does not significantly distort the
Index results, we developed a version of the
Index where these proxy measures are excluded.
Table 26 compares the results and rankings for
this sensitivity.
84 The International Civil Service Effectiveness Index | Technical Report
Table 26: Sensitivity analysis – Removal of public sector proxy data
Country Index Score Rank Adjusted Index Adjusted Rank
Score
Canada 1.00 1 1.00 1
New Zealand 0.95 2 0.87 5
Australia 0.91 3 0.98 2
United Kingdom 0.91 4 0.92 3
Finland 0.87 5 0.84 7
Sweden 0.86 6 0.86 6
Estonia 0.81 7 0.83 8
Norway 0.81 8 0.78 9
Korea, Republic of 0.78 9 0.92 4
United States of America 0.74 10 0.70 11
Denmark 0.73 11 0.72 10
Switzerland 0.61 12 0.55 15
Belgium 0.60 13 0.37 19
Netherlands 0.58 14 0.34 21
Japan 0.57 15 0.25 24
Austria 0.56 16 0.50 16
France 0.50 17 0.60 13
Spain 0.49 18 0.63 12
Mexico 0.47 19 0.59 14
Ireland 0.46 20 0.33 22
Poland 0.44 21 0.22 25
Chile 0.44 22 0.31 23
Slovenia 0.44 23 0.48 17
Germany 0.40 24 0.18 27
Turkey 0.37 25 0.37 20
Portugal 0.31 26 0.46 18
Italy 0.21 27 0.18 26
Czech Republic 0.16 28 0.01 30
Greece 0.04 29 0.06 29
Hungary 0.00 30 0.12 28
Slovakia 0.00 31 0.00 31
The International Civil Service Effectiveness Index | Technical Report 85
8.5 Sensitivity analysis 4: A range of alternative approaches to imputation
are possible, and of course one response to
Imputation Method
missing data is list-deletion of those countries
Chapter 5 describes the statistical methodology with missing data, although this would not meet
adopted to estimate or ‘impute’ missing data. the objective of our chosen imputation approach
With imputed data included, a complete data set by allowing the Index to represent a range of
is estimated and the Index can be determined. countries despite missing data. To examine the
While the approach to imputation adopted is impact of the favoured imputation approach we
sensible, based on demonstrated relationships compared the Index calculated to an approach
between variables and the proportion of missing using simple mean imputation to estimate
data, it is important to consider the impacts of missing data. Mean imputation is a method in
alternative imputation methods. which the missing value on a certain variable is
replaced by the mean of the available cases.
This method maintains the sample size and
is easy to use, but the variability in the data is
reduced, so the standard deviations and the
variance estimates tend to be underestimated.
Table 27 documents the comparison of the Index
generated under the two imputation methods.
86 The International Civil Service Effectiveness Index | Technical Report
Table 27: Sensitivity analysis – Differing imputation methods
Country Multiple Multiple Mean Imputation Mean Imputation
Imputation Index Imputation Rank Score Rank
Score
Canada 1.00 1 1.00 1
New Zealand 0.95 2 0.94 4
Australia 0.91 3 0.91 6
United Kingdom 0.91 4 0.98 2
Finland 0.87 5 0.97 3
Sweden 0.86 6 0.91 5
Estonia 0.81 7 0.87 8
Norway 0.81 8 0.88 7
Korea, Republic of 0.78 9 0.75 10
United States of America 0.74 10 0.69 11
Denmark 0.73 11 0.79 9
Switzerland 0.61 12 0.65 12
Belgium 0.60 13 0.61 16
Netherlands 0.58 14 0.63 13
Japan 0.57 15 0.62 15
Austria 0.56 16 0.63 14
France 0.50 17 0.53 18
Spain 0.49 18 0.55 17
Mexico 0.47 19 0.42 22
Ireland 0.46 20 0.50 19
Poland 0.44 21 0.46 21
Chile 0.44 22 0.42 23
Slovenia 0.44 23 0.49 20
Germany 0.40 24 0.42 24
Turkey 0.37 25 0.37 25
Portugal 0.31 26 0.34 26
Italy 0.21 27 0.20 27
Czech Republic 0.16 28 0.18 28
Greece 0.04 29 0.07 29
Hungary 0.00 30 0.02 30
Slovakia 0.00 31 0.00 31
The International Civil Service Effectiveness Index | Technical Report 87
Chapter 9:
Next steps
In chapter 9 we note some limitations of the pilot Index, and planned next steps.
9.1 Identified limitations of the Index Data Updates
Whilst every effort has been made to make use
This paper has offered a summary of key
of the most up-to-date data, each iteration of
aspects of the methodology to construct the
InCiSE will, by necessity, have a cut-off for data
pilot Index. When interpreting the results of the
being included in that years Index. InCiSE reports
Index the limitations listed, which are common to
will specify the date up to which data released
many other international indicators, and planned
has been included. For instance, the inaugural
next steps should be noted. The development of
pilot report includes data released up to January
the pilot Index is the culmination of the first stage
2017. The InCiSE Index will be updated annually
of an ambitious and innovative effort to develop
with the most recent data available.
and measure against a framework of civil service
effectiveness. Wide-ranging and sustained Varying depth of data
engagement from inception to publication has
With some data metrics only representing,
offered challenge, creativity and expertise to
for example, one survey question, but others
support improvement of the Index up to the
representing scores on an entire index relevant
point of the pilot publication and beyond. As
to the indicator (e.g. Open Data Barometer
this Index develops, we hope to overcome or
score measuring open data), the depth of data in
reduce the impact of these limitations as well as
indicators varies.
strengthening the indicator in other areas.
Public sector performance being used as
The general limitations of the Index are set out
a proxy for civil service performance
below. Limitations associated with particular
datasets included in the Index are identified in Some of the metrics used in the Index explicitly
Annex A. try to measure performance of the public
sector, not just civil service bodies. Therefore
Missing data until similar data collections are completed for
Only two countries have non-missing values the civil service, these measures can only be
across all 76 metrics. This is because different considered proxies. The OECD handbook on
included data sets cover different sets of constructing composite indicators notes that:
countries. However, a number of countries have “Proxy measures can be used when the desired
close to the full complement of data, and missing data are unavailable or when cross-country
data for these countries has been imputed. The comparability is limited.”
imputation methods utilised are well supported
for the degree of missing data observed and
are informed by observed relationships between
metrics where possible.
88 The International Civil Service Effectiveness Index | Technical Report
Mixture of quantitative and qualitative/ a) Strengthening data collection
subjective data Tasks will include: determining how best
The Index mixes quantitative and qualitative to collect data on the 5 omitted indicators
data. The OECD handbook on constructing (staff engagement, innovation, IT, procurement,
composite indicators notes that “Given finance); exploring how data collection for
a scarcity of internationally comparable existing indicators can be strengthened,
quantitative (hard) data, composite indicators including thematic gaps; expanding data sources
often include qualitative (soft) data from surveys and looking at ways of avoiding over reliance
or policy reviews.” on a single survey; reducing instances of public
sector performance being used as a proxy for
Possible ‘spill over’ across indicators civil service performance; and cross-country
The types of things measured under some of applicability issues.
the indicators will occasionally be relevant to
others. Attempts have been made to minimise Complementary work, particularly by the OECD,
occurrences of this by adapting the framework may help to fill some data gaps over time.
to ensure each indicator is a unique concept. Additional data collection, for example through
the commissioning of specific country surveys,
Proxy measures of effectiveness may also help to provide valuable insights.
The nature of the true level of civil service b) Refining the InCiSE framework
effectiveness in a country is inherently
unobservable, and therefore any observed Framework tasks will include: exploring the
empirical measure of the aptitude or capacity potential to add new civil service functions
will be a proxy for the broader dimensions of (for example, security, foreign affairs, defence
effectiveness it reflects. One consequence of and justice); and ensuring each indicator in
this is that our estimates of effectiveness are the framework is a unique concept, to avoid
subject to uncertainty. Users should not over- duplication or overlap between the themes
interpret small differences in performance (across being measured.
countries or, in future editions, over time) in the An issue that has emerged through the pilot
aggregate Index and the underlying indicators. is the need to consider how best to adapt
The presence of uncertainty does not imply that the InCiSE framework to enable it to measure
the Index cannot be used to make meaningful and compare core functions within federal
comparisons of effectiveness across countries government systems where some aspects of
or over time. this work are often carried out by a lower tier.
9.2 Priorities for the next 12 months Changes to the InCiSE framework will mainly
be determined through user feedback and
InCiSE is a long term project and the discussions with data owners to learn lessons
founding organisations have committed to from their own experiences, as well as through
supporting its development for a further four guidance from an International Advisory Panel.
years. This will include publishing an annual
report and developing an interactive website. c) Expanding country coverage
An International Advisory Panel has also been InCiSE will explore the scope to expand the
established to guide this work. Index’s current country coverage over time,
including the potential to include some non-
As described earlier, this first edition of the Index OECD countries. Country coverage in future
is a pilot- further work is required to refine the will largely be determined by feedback from
methodology and make the data more robust. countries on the usefulness of having their own
The main issues that the project will focus on set of indicators, and the availability of data to
over the next 12 months include: produce results.
The International Civil Service Effectiveness Index | Technical Report 89
Several countries were excluded from the pilot Feedback is vitally important to help improve
Index because they had fewer than 75% of the InCiSE Index and the project team would
the metrics available. However, three countries welcome responses from country governments,
(Bulgaria, Romania and Croatia) had over 70% of as well as policy and learning networks, think
data available. A small increase in data collection tanks and academia.
may help them meet the threshold for inclusion in
the next Index. The InCiSE project team will continue to
coordinate closely with other institutions who
d) Increasing InCiSE partners and network are engaged in similar efforts to measure civil
The founding institutions will actively encourage service effectiveness, as well as with the many
more partners to join them and support organisations who have made a vital contribution
the development of the InCiSE Index over to InCiSE so far by permitting use of their
time, through advice, expertise and funding. research or survey data. Continued collaboration
The project’s International Advisory Panel will will be essential to help strengthen InCiSE in the
also play a key role in promoting InCiSE and coming years.
encouraging more partners.
The Blavatnik School of Government will host
an international conference in Autumn 2017
to discuss the pilot Index with a wide range of
interested players. This event will additionally
provide a key opportunity to build support for
InCiSe and increase its network.
The InCiSE pilot Index has already brought
together a rich volume of data and insights.
Its launch provides an important opportunity to
stimulate wide-ranging discussions at country
level, as well as globally, about civil service
effectiveness issues generally, as well as the
relevance and usefulness of some data.
90 The International Civil Service Effectiveness Index | Technical Report
Appendices
a) Annex A: Data limitations
b) Annex B: Reference list
c) Annex C: Reader’s Guide – Abbreviations and Country Codes
The International Civil Service Effectiveness Index | Technical Report 91
Annex A: Data limitations
Table 28 sets out the limitations of the data used within the Index. The data is described in chapter 4
of this report.
Table 28: Data limitations within the Index
Indicator Limitations
Policy making • The data may be capturing the wider public sector rather than the civil service,
as the indices are designed to measure government performance. For example,
‘Strategic planning’ may include the performance of policy-related ministerial
decisions and/or academics. The ‘Monitoring agencies, bureaucracies’ metric
includes subnational governments, so is only a proxy for the civil service.
• A large number of the metrics included are subjective. However, the
standardisation process embedded in the Bertelsmann methodology helps
to regulate scores across countries.
• The survey questions give only a partial picture of the quality of policy advice.
For example, they do not capture the quality of written and oral briefing,
the quality of draft legislation, or the extent to which policy advice is based
on evidence.
Fiscal and financial • Some of the data used in this indicator is from 2012.
management • More detailed data on the use and quality of economic appraisal and evaluation
would improve the robustness of this indicator. The current metrics only partially
measure what we would like to include in this indicator.
Regulation • The metrics are mainly based on information provided to the OECD by countries
directly. Whilst countries are asked to support the information they provide with
evidence, it may be possible that countries exaggerate claims about the quality
of their processes.
Crisis/risk • The Hyogo framework gives an overview of best practice in crisis/risk
management management, but many of the countries in this indicator set appear to already
be adhering to large parts of it. This leads to a lack of variation in scores.
• Ideally this indicator would also be able to measure the quality of response from
civil servants after risks have occurred. All metrics included so far only contain
details on what risk management strategies and activities exist, with less detail
on their quality.
Human Resources • Data on meritocracy is based on expert opinion, which may not necessarily
Management accurately reflect reality. Data on meritocracy also refers to ‘public sector
employees’.
• Attracting talent is measured using expert assessments, but hard quantitative
data would be preferable for assessing financial incentives provided.
• Important themes of Human Resources Management do not have data
available, for example the quality of learning and development opportunities.
Financial incentives do not give a full picture of how ‘attractive’ an organisation
is to work for.
Tax administration • Cost of collection ratios are problematic because amount of revenue collected
depends to a large extent on tax rates. Adjusting revenue collected for tax rates
may be a potential solution, but how feasible this might be is unknown.
• Further investigation may be needed into the validity of comparing across
different countries’ tax systems (e.g. into how it’s been decided which taxes
should be included in each country).
92 The International Civil Service Effectiveness Index | Technical Report
Table 28: Data limitations within the Index
Social security • Whilst the included metric may give a good overall picture of efficiency,
only having one metric does not give a comprehensive picture of system
performance.
• Further investigation may be needed into the validity of comparing across
different countries’ welfare systems.
Digital services • The data only covers mystery-shopper assessments of seven life events so
does not assess all of the services which governments typically provide digitally.
Integrity • QoG data on the core values and principles civil servants adhere to is for public
sector, not civil servants specifically. OECD data on whistleblower protection
also covers the public sector.
• A large amount of the data included is subjective. A lot of it relies on expert
opinion, which is subject to the choice of experts. Citizen and business
perceptions data have the usual limitations which surround perception-based
measurement.
Openness • The indices are designed to measure open government in general, so may
capture elements of ministerial and wider public sector performance.
Capabilities • Leadership capabilities within the organisation are important, but not currently
fully captured. At present this indicator does not cover more specific measures
of skills, for example capabilities in specific areas such as economics and
statistics. Measures of whether these skills are deployed in the right areas may
also be desirable but difficult to obtain.
• There is currently no data on the educational attainment of civil services across
the world. OECD PIAAC data gives an estimate of public sector educational
attainment. Industry codes can be used to get closer to civil service but sample
sizes at that level were too small to be reliable.
Inclusiveness • Data is not available on most demographics of interest.
• Most OECD data is from 2010. However for a small number of countries data is
for 2011 and in one case is for 2009.
• The QoG gender data asks experts to estimate gender representation (%),
which may not always be accurate. For ethnicity metrics they are asked
to score ethnic/religious inclusiveness on a scale of 1 to 7; if management
information data were available it would give a more accurate assessment.
• This indicator penalises countries for having over-representation of
‘disadvantaged’ groups. An alternative approach would be to give the same,
‘perfect’ score if a disadvantaged group is overrepresented.
The International Civil Service Effectiveness Index | Technical Report 93
Annex B: Reference list
Baubion, C. (2013). OECD Risk Management: Strategic Crisis Management. OECD. [Online].
Available from: http://www.mmc.com/content/dam/mmc-web/Files/Strategic-Crisis-Management-
paper-July-2013.pdf. [Accessed on 21 February 2017].
Bouchard, P. and McCrae, J. (2013). Financial Leadership for Government. [Online]. Institute for
Government and The Chartered Institute of Management Accountants (CIMA). Available from:
https://www.instituteforgovernment.org.uk/sites/default/files/publications/2739%20IFG%20-%20
Financial%20Leadership%20v10%20(online-version).pdf. [Accessed 10 March 2017].
Bovaird, T. and Löffler, E. ed. (2003). Public Management and Governance. 2nd edn. Routledge.
Centre for the Study of Governance Innovation (GovInn). (2014).What is Governance Innovation?
[Online]. Available from: http://governanceinnovation.org/what-is-governance/. [Accessed 7 March
2017].
Chalam, K.S. ed. (2014). Governance in South Asia: State of the Civil Services. Pp. 59-72. SAGE
Publications Ltd.
Christensen, T., Fimreite, A.L. and Lægreid, P. (2011). Crisis Management: The Perceptions of
Citizens and Civil Servants in Norway. Administration & Society, 43(5), pp.561-594.
European Commission. (2006). Welcome to the world of Public Internal Financial Control. [Online].
Available from: http://ec.europa.eu/budget/library/biblio/documents/control/brochure_pifc_en.pdf.
[Accessed 10 March 2017].
Fukuyama, F. (2013). What Is Governance? Governance: An International Journal of Policy,
Administration, and Institutions, 26(3), pp. 347-368.
Graham, J., Amos, B. and Plumptre, T. (2003). Principles for Good Governance in the 21st Century.
[Online]. Institute on Governance, Policy Brief No.15. Available from: http://unpan1.un.org/intradoc/
groups/public/documents/UNPAN/UNPAN011842.pdf. [Accessed 17 March 2017].
Herbert, N. and Healey, J. (2015). Time to Govern Up. [Online]. Govern Up. Available from: http://
www.governup.org/latest/. [Accessed 22 February 2017].
Holt, J. and Manning, N. (2014). Fukuyama is Right about Measuring State Quality: Now what?
Governance: An International Journal of Policy, Administration and Institution, 27(4), pp.717-728.
Huther, J. and Shah, A. (1999). Applying a Simple Measure of Good Governance to the Debate on
Fiscal Decentralization. [Online]. World Bank. Available from: http://siteresources.worldbank.org/
INTWBIGOVANTCOR/Resources/wps1894.pdf. [Accessed 1 March 2017].
International Civil Service Commission. (2002). Standards of conduct for the international civil
service. [Online]. Available from: http://icsc.un.org/resources/pdfs/general/standards.pdf. [Accessed
15 February 2017].
International Monetary Fund (IMF). (2016). Consultation on the IMF Guideline on Governance Issues.
[Online]. Available from: https://www.imf.org/external/np/exr/consult/2016/governance/.
[Accessed 9 March 2017].
94 The International Civil Service Effectiveness Index | Technical Report
Kaufmann, D., Kraay, A. and Zoido-Lobatón, P. (1999). Governance Matters. The
World Bank Development Research Group Macroeconomics and Growth and World
Bank Institute, Governance, Regulation and Finance. Policy Research Working
Paper 2196. [Online]. Available from: https://poseidon01.ssrn.com/delivery.php?
ID=492099070121007123122084103071079102064038020065064007072124071027074112085
110025086006103020004074093027019087120122087004002065107067088079116012120070
113003011075024126024116123112090078&EXT=pdf. [Accessed 20 February 2017].
Lonti, Z. and M. Woods (2008), Towards Government at a Glance: Identification of Core Data and
Issues related to Public Sector Efficiency, OECD Working Papers on Public Governance, No. 7,
OECD Publishing. doi:10.1787/245570167540
MacLeod, D. and Clarke, N. (2009). Engaging for Success: enhancing performance through
employee engagement. A report to Government. [Online]. Available from: http://dera.ioe.
ac.uk/1810/1/file52215.pdf. [Accessed 8 March 2017].
Madzova, V., Sajnoski, K. and Davcev, L. (2013). E-Government as an Efficient Tool towards Good
Governance (Trends and Comparative Analysis throughout Worldwide Regions and within West
Balkan Countries). Balkan Social Science Review, (1), pp.157-174.
Magno, F. and Serafica, R. (2001). Information Technology for Good Governance. De La Salle
University. [Online]. Available from: http://unpan1.un.org/intradoc/groups/public/documents/apcity/
unpan002708.pdf. [Accessed on 3 March 2017].
McKinnon, R. (2011). Good Governance in Social Security Administration. International Social
Security Review, 64(4), pp.3-8.
Mulgan, G. (2014). Innovation in the Public Sector – How can public organisations better create,
improve and adapt? [Online]. Nesta, version 1. Available from: http://www.nesta.org.uk/sites/default/
files/innovation_in_the_public_sector-how_can_public_organisations_better_create_improve_and
adapt.pdf. [Accessed 7 March 2017].
Muriithi, K., Jimenez, M., Jannin, N., Sajid, N., Singh, S. and Sharma, S. (2015). Quantifying
Governance: An indicator-based approach. The London School of Economics and Department for
International Development Capstone Team. [Online]. Available from: https://assets.publishing.service.
gov.uk/media/57a08971ed915d622c000209/61506_Capstone_Report_DFID_FINAL_Quantifying_
Governance__Indicators.pdf. [Accessed 21 February 2017].
OECD. (2007). Towards Better Measurement of Government. [Online]. OECD Working Papers on
Public Governance, 2007(1). OECD Publishing. Available from: http://www.oecd.org/governance/
digital-government/38134037.pdf. [Accessed 10 March 2017].
OECD. (2012). Recommendation of the Council on Regulatory Policy and Governance. The OECD
Regulatory Policy Committee. [Online]. Available from: http://www.oecd.org/governance/regulatory-
policy/49990817.pdf. [Accessed 9 March 2017].
OECD. (2014). Recommendations of the Council on the Governance of Critical Risks. Meeting of The
OECD Council at Ministerial Level. Paris 6-7 May 2014. [Online]. Available from: http://www.oecd.org/
gov/risk/Critical-Risks-Recommendation.pdf. [Accessed on 2 March 2017].
OECD. (2015a). Recommendation of the council on Budgetary Governance. Public Governance &
Territorial Development Directorate. [Online]. Available from: http://www.oecd.org/gov/budgeting/
Recommendation-of-the-Council-on-Budgetary-Governance.pdf. [Accessed on 3 March 2017].
The International Civil Service Effectiveness Index | Technical Report 95
OECD. (2015b). Inclusive government for a more inclusive society. Government at a Glance 2015,
OECD Publishing, Paris. [Online]. Available from: http://dx.doi.org/10.1787/gov_glance-2015-6-en.
[Accessed on 21 February 2017].
OECD. (2015c). The Innovation Imperative in the Public Sector: Setting an Agenda for Action, OECD
Publishing, Paris. [Online]. Available from: http://dx.doi.org/10.1787/9789264236561-en.
[Accessed 7 March 2017].
OECD. (2015d). Public Sector Leadership for Improved Employee Engagement and Organisational
Success. Lead-Engage Perform. OECD expert meeting, 21-22 January 2015. [Online]. Available
from: https://www.oecd.org/gov/pem/agenda.pdf. [Accessed on 3 March 2017].
OECD. (n.d.). Building more effective, accountable, and inclusive institutions for all. [Online].
Post 2015 Millennium Development Goals: Element 6, Paper 1. Available from: https://www.
oecd.org/dac/_POST-2015%20effective%20and%20accountable%20institutions.pdf. [Accessed
1 March 2017].
Office for Public Management Ltd (OPM) and The Chartered Institute of Public Finance and
Accountability (CIPFA). (2004). The Good Governance Standard for Public Services. The Independent
Commission on Good Governance in Public Services. Hackney Press Ltd.
Phillips, W., Caldwell, N. and Callender, G. (2007). Public Procurement – a pillar of good
governance? [Online]. Available from: https://www.researchgate.net/publication/272487779_Public_
procurement_A_pillar_of_good_governance. [Accessed on 22 February 2017].
The Committee on Standards in Public Life. (1996). Summary of the Nolan Committee’s First Report
on Standards in Public Life. [Online]. The National Archives. Available from: http://webarchive.
nationalarchives.gov.uk/20140131031506/http:/www.archive.official-documents.co.uk/document/
parlment/nolan/nolan.htm. [Accessed 17 March 2017].
The World Bank. (2012). Indicators of the Strength of Public Management Systems: A key
part of the Public Sector Management results story. PRMPS Discussion Paper: Draft. [Online].
Available from:http://siteresources.worldbank.org/PUBLICSECTORANDGOVERNANCE/
Resources/285741-1354024300711/ISPMS.pdf. [Accessed 21 February 2017].
The World Bank. (2016). Benchmarking Public Procurement 2016. [Online]. Available from:
http://bpp.worldbank.org/~/media/WBG/BPP/Documents/Reports/Benchmarking-Public-
Procurement-2016.pdf. [Accessed 22 February 2017].
The World Bank (n.d.). [Online]. Available from: http://www1.worldbank.org/publicsector/anticorrupt/
corruptn/cor02.htm. [Accessed 1 March 2017].
United Nations. (1945.) Charter of the United Nations. Article 101. [Online]. Available from: http://
www.un.org/en/sections/un-charter/un-charter-full-text/. [Accessed 15th February 2017].
United Nations Public Administration. (1999). Transparency in Government. Presentation for ILEA
Seminar. [Online]. Available from: http://unpan1.un.org/intradoc/groups/public/documents/un/
unpan012062.pdf. [Accessed 20 February 2017].
Van Dooren, W., Manning, N., Malinska, J., Kraan, D. J., Sterck, M. and Bouckaert, G. (2006). OECD
project on Management in Government: Comparative Country Data. Issues in Output Measurement
for “Government at a Glance”. OECD, Technical paper 2 (Second draft).
World Trade Organisation. (2015). Government Procurement Agreement: opening markets and
promoting good governance. [Online]. Available from: https://www.wto.org/english/thewto_e/20y_e/
gpa_brochure2015_e.pdf. [Accessed 1 March 2017].
96 The International Civil Service Effectiveness Index | Technical Report
Annex C – Reader’s Guide
C.1 Abbreviations
InCiSE International Civil Service Effectiveness
SNA System of National accounts
OPM Office for Public Management
CIPFA Chartered Institute of Public Finance and Accountability
ISPMS Indicators of the Strength of Public Management Systems
RIA Regulatory Impact Assessment
OECD Organisation for Economic Co-operation and Development
IMF International Monetary Fund
UN United Nations
VFM Value for money
GCI Global Competitiveness Index
WEF World Economic Forum
iREG Indicators of Regulatory Policy and Governance
QoG Quality of Government
SGI Sustainable Governance Indicators
EPI E-participation index
ODI Open Data Index
ODB Open Data Barometer
PIAAC Programme for the International Assessment of Adult Competencies
GaaG Government at a Glance
MI Multiple imputation
MICE Multivariate imputation using chained equations
OURdata Open, Useful, Reusable data
The International Civil Service Effectiveness Index | Technical Report 97
C.2 Country codes
The following ISO country codes are used in some tables of this report57:
Country ISO alpha – 3 digit Country ISO alpha – 3 digit
country code country code
Australia AUS Korea, Republic of KOR
Austria AUT Mexico MEX
Belgium BEL Netherlands NLD
Canada CAN New Zealand NZL
Chile CHL Norway NOR
Czech Republic CZE Poland POL
Denmark DNK Portugal PRT
Estonia EST Slovakia SVK
Finland FIN Slovenia SVN
France FRA Spain ESP
Germany DEU Sweden SWE
Greece GRC Switzerland CHE
Hungary HUN Turkey TUR
Ireland IRL United Kingdom GBR
Italy ITA United States of America USA
Japan JPN
57 https://www.iso.org/obp/ui/#home
98 The International Civil Service Effectiveness Index | Technical Report