As an APS member for roughly 10 years, preceded by over 20 years in the Royal Australian Navy, and more than 10 years working for organisations doing business with various Commonwealth departments, I have 1st hand experience from a number of perspectives, with different bodies, in different eras.
Therefore I will limit myself to what I perceive as key areas, with broad Service-wide applicability.
Whilst containing the size and cost of the Public Service are pretty well essential aims, current practice does one at the expense of the other (no pun intended). Extreme strict limits on staff numbers mean that many tasks are out-sourced at far greater expense. In some instances it also becomes necessary to pay out-sourced project managers, adding another layer of unnecessary cost. The end result is far greater overall cost to the Government than would be the case were the activities carried out by (additional) staff (either full-time or non-ongoing contract staff).
Much to-do is often made of reducing red-tape. However we live in an increasingly complex technological world, which necessitates regulation. This is somewhat exacerbated by public expectations (somewhat unrealistic in my opinion) that Government will solve all their problems.
The requirements and expectations of such matters as Privacy legislation and WH&S, whilst of themselves entirely laudable, merely add to the requirements for bureaucrats and procedures, to be complied with by both public and private organisations – hence more red tape. The challenge is a balancing act.