A few of my thoughts, from an ICT background:
The public service shouldn't start works on policy that hasn't yet passed as legislation, or at least passed the lower house. It's a waste of money and resources.
I have heard the term 'data rich but information poor' a lot lately, and i think instead the problem of insights lies with being 'question poor'. We lack questioning and inquisitive executives. Data analysts would love to explore questions and problems in data the public service has. We need better executives who are more interested in their department and the public, not just experts at general management, PR and policy delivery.
The public service needs to fix it's data policy to share data between departments. We need more specific and simple rules about which data cannot be joined. There have been cases of rejecting the sharing of code, metadata, and system data between departments. These things are not customer data, and code is just code, its a bit over the top.
The public service cant be all-out project based and lose sight of its responsibilities what programmes are trying to achieve. There are concerning requests and demands thrown between and within departments that fall on deaf ears because the request is not part of a funded project or is too loosely related to a programme's responsibilities.
There is so much knowledge required about government policy and programmes in order to build new services or make changes, which takes time to learn. It doesn't make sense to have so many contractors who will need months to learn complex programmes before doing a good job, especially the low level knowledge required from programmers and analysts.
Similarly, SES staff should in no way be allowed to move so frequently, they only get knowledgeable enough for their roles after months. APS staff retention is poorer when project work is poorly negotiated and staff are receiving terrible direction from their superiors (in my opinion, perhaps a HR question for a data analyst). Why do SES only stay in a role for 1 year? They never get a chance to give good direction and have a positive impact on their branch / division. SES staff should at least be people who have been in the department a while. The departments aren't like a different retail companies you can just jump one to the other, they are entirely different industries and require time to build knowledge.
I think the public service needs people with an ICT background in business teams, project leads and executive roles. Digital systems and services can't keep being directed by non-technical people - seeing as government services and the way we work is all digital. Our systems and services would be fine if they were built with better, low level requirements that analysts and programmers would be more capable of giving.
The public service should absolutely be utilising the probation period for permanent staff, since its the only tool and chance to remove someone from a position they shouldn't be in. Rarely does someone fail their probation period, move them on to a role that will suit them better.
We need more knowledgeable recruiters and more specific recruitment process for ICT and technical staff. Too many times I have seen job descriptions and selection criteria be a mix of an impractical range of skills such as 'statistics, web developer, data modeller' all in one. It seems the team or recruiter are not sure what they really need, and end up with the wrong person.