This submission relates to the scope of the review, points 2 (delivery high quality advice) and 6 (acquiring and maintaining skills), with particular reference to the objective 'understand and deploy technology and data to drive improvement'.
There is an push in many sectors at the moment for organisations to be 'data-driven'. In fact, at summits I have attended recently, not being data-driven is seen as not understanding your business or your customers (the APS’s customers being both the Australian people and the government of the day). Being data-driven is taken to mean 'we make decisions based on available evidence, and we create the bulk of that evidence by analysing data that we collect, even if the data isn’t collected for the purpose of making decisions'. In practice, being data-drive is very much built around business intelligence tools and following correlations down wending paths, even when spurious.
The panel should be very cautious regarding recommending that the APS be a 'data driven' organisation. A more fruitful path would be to recommend that the APS has access to (perhaps as a corporate function within departments, or through partnerships with universities), highly skilled and experienced people who can work with policy and strategy experts to truly understand issues, patterns and causality. That is, APS staff need to work with experts who can help them contextualise data on both sides of the science-policy nexus.
Data-driven decisions are predominantly made by people who don't understand the intricacies of post-hoc data analysis. Understanding the ways data can be collected, the biases that arise and the importance of not torturing data (squeeze data hard enough and it will tell you whatever you want to know!) is critical to truly evidence-based policy. The single biggest risk of pushing a data-driven agenda on the APS is that it becomes an organisation that uses p-hacking to justify its decisions. P-hacking––the continual refinement of algorithms and data queries until a non-significant result appears to become significant––is a noted issue in science at present, given the drive for 'significant' results; see Head et al. 2013, http://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1002106. The APS must avoid this if it is to remain an impartial and trusted adviser.
Public servants, in general, do not have the training, skills or experience to use data directly to make recommendations or decisions, and I argue that they should not need to. The APS needs experts in policy design, communication, administration, strategic thinking and many other fields. It can take many years of statistical training and practice to understand the nuances of data analysis, and the APS should be looking to lean on expertise rather than expecting staff to be 'data scientists'. The skills the APS in general must therefore acquire and maintain to provide high quality advice are not data science skills, but critical thinking and understanding the nature of evidence.
Thank you for considering my submission. I would be happy to discuss any part in more detail.