#007/2010 – A joint inspectorate response to the Evaluation of Comprehensive Area Assessment (CAA) Year One

The independent evaluations undertaken by the consortium led by Shared Intelligence and OPM provide a valuable insight into the perceptions, effectiveness and impact of CAA in its first year.

We welcome the feedback and learning that the reports provide and will use this new evidence to shape the development of CAA. The main messages echo the feedback already provided by the Local Government Association (LGA) and in our other discussions with representatives of local public services.

We are encouraged that there remains widespread support for the principles of the new approach. The evaluation also suggests that the new CAA Lead role has been well received, and that inspectorates have made progress in focusing on outcomes and local priorities. Most importantly, we welcome the early signs that the new approach has prompted better working across some local public services.

We acknowledge that the new approach represents a significant change for both inspectorates and inspected organisations and partnerships, and that there were implementation difficulties in year one. We accept we have more to do to demonstrate a fully joined-up approach between inspectorates, and to address the view of inspected organisations that the cost and effort required to administer the process was not reduced in the way they had expected. In particular, during year two and beyond, we will:

Improve our joint working across inspectorates;
Use the baseline of the first year assessments to ensure future work is risk-based and proportionate;
Reduce the administrative impact of CAA and, in particular, carry out a significant review of our approach to use of resources assessments;
Ensure much greater engagement of Councillors in CAA;
Improve the transparency of our joint judgements and emerging findings; and
Promote a greater focus on learning and improvement to ensure Oneplace adds value for the public and that the assessments add value for local public services.

Actions for all inspectorates together

Improving joint inspectorate working

The first year of CAA has delivered significant changes in the assessment of local public services and the way the inspectorates work together, responding positively to feedback in recent years. The independent evaluation recognises this, but also illustrates that the inspectorates need to further improve their ability to join up, and clearly demonstrate this.

We accept that our joint working in the first year was held back by the time taken to get locally focused staff in place across inspectorates. We are fully committed to develop and build on the new relationships and ways of working established in year 1 to ensure that CAA 2010 is an increasingly joint assessment.

The extent to which we can fully join up all assessment activity is, however, limited by some factors beyond the control of the inspectorates. In particular, we are restricted by the structure, approach and priorities reflected in different performance frameworks and their sponsoring government departments. Statutory requirements for separate assessments of children and young people and adult social care remain in place and prevent the area assessment being the clear main focus for local assessment. The timing of the release of key performance information is another factor.

We are taking the following steps to further align our work in 2010:

We will publish the result of assessments of children’s services and adult social care alongside CAA in December 2010;
We will further enhance the skills, knowledge and capacity of inspectorate staff through a continuing programme of joint inspectorate training and events; and
The National Inspection Planning Group is now fully established to better co-ordinate and timetable inspections of local services.

We will continue to better align our work within these constraints and welcome feedback on progress, but government action is needed to address the wider issues.

Engaging elected councillors as peers

We now have robust arrangements for engaging elected councillors as peers in supporting CAA judgements. This will allow us to make more effective use of this valuable resource than was possible in the first year of the new framework. CAA Leads all now have peer member contacts assigned. Peers will also play a valuable role in Expert Advisory Panels providing a unique insight into the quality of our assessments.

A proportionate approach

The first year of CAA has provided a substantial baseline assessment. In 2010 our work will be more proportionate and risk based, reflecting our findings in year We will build on what we already know to ensure that 2010 is not a re-run of 2009, but focuses on what’s new, what has changed and where there are important gaps or issues we said we would follow up to identify the risks to outcomes in the future. We will prioritise this work to ensure we only address the most pressing issues for local people.

Reducing the burden

The perception of inspected organisations was that the cost and effort required to administer the process was not reduced in the way they had expected. In particular, the use of resources assessment was felt to constitute a significant part of the continuing burden. The independent research by OPM indicates that councils incurred widely differing costs in administering both the former CPA framework and CAA. The research identifies an increase in the cost of administering use of resources, reflecting the change in the assessments overall scope. However, the overall picture shows a reduction in cost for single tier and county councils, while total costs for district council remained relatively unaltered. This suggests that some have found more efficient ways of providing information and working with the inspectorates than others.

CAA constitutes only one part of the overall performance framework. The OPM cost evaluation report focuses on CAA and does not include for example, government requirements such performance information collection and submission, the Local Area Agreement process or Government Office engagement or monitoring. There is still further to go to reduce both the financial impact and perception of burden on councils.

Some costs identified may be related to CAA, as a new framework, becoming established and attracting greater attention, particularly among the most senior staff. However, the costs identified illustrate a wide degree of choice exercised by councils participating in CAA. We will seek to identify those that engaged in CAA most cost and time efficiently and help spread their learning to other areas and organisations.

Enhancing impact

We are encouraged that the evaluation indicates that CAA is already influencing councils and partnerships in many places to provide a stronger focus on improvement and joint working. This supports our own experience from working with local partnerships.
A clear message from the evaluation is that we need to engage local elected members in forming the assessments and sharing the results. We will encourage CAA Leads to present their findings to Overview and Scrutiny Committees. We are also discussing with the LGA how to most efficiently engage councillors especially in county and district areas.
The evaluation also identifies that some district councils did not feel adequately engaged in CAA in year 1. Our experience is that we made better progress in reflecting district issues in some county areas than others and we are sharing the more successful approaches with all CAA Leads.
However, we are aware that capacity of district councils tends to be more limited and the timeline for year 1 made an effective contribution all the more challenging. OPM’s evaluation showed a limited change in the burden of assessment that we will seek to address. We will stagger our work with district councils in year 2 to help ensure that they can more effectively engage in the process. This will also enable the early reporting of district council organisational assessments. We believe that this will provide a better opportunity to contribute to the focus on district level issues within the area assessment while balancing the need to reduce the burden on district councils.
The Oneplace website has been significantly better used than the equivalent reports on the former CPA framework. It has attracted positive feedback for its accessibility and the plain language writing style. We want to build on this in 2010 to further enhance the use of Oneplace for independent information and to encourage people to engage with their local public services.

Improving transparency

In 2010 we will address the feedback that we need to be more transparent in the judgements we make and in the evidence we use.

CAA year 2 will provide a full year of engagement with councils and partnerships, rather than the compressed timeline of year 1. Sharing evidence and emerging findings earlier will help to support greater transparency in our judgements and reporting. We will make clearer, for example, the focus on future prospects in red flags and the evidence on which they are based.

We will provide a clearer explanation of how our national consistency arrangements will operate. Specifically, we will explain how the Expert Advisory Panels and National Overview Panels will provide challenge and ensure consistency in our assessments (including flags).

We will also use our quality assurance approach, together with co-ordinated support for CAA Leads and joint inspectorate teams, to provide a stronger focus on people whose circumstances make them vulnerable and on value for money, both of which were highlighted in the evaluation.

Promoting learning and improvement

Some people perceived that our new style of public reporting, whilst accessible, did not contain new insights for those responsible for running local public services. We think it is important to retain the focus of our reports as a resource for the public but we will support CAA Leads in sharing our local and national learning with LSPs and local organisations.

Through CAA Leads and our communications teams, we will encourage greater analysis and use of Oneplace by local public services as a resource to support improvement and learning. The first stage of this process was to publish a National Overview Report of CAA at the end of February 2010. We will continue to focus on how green flags can be used to support learning and improvement most effectively.

Actions for individual inspectorates

Audit Commission

Auditors appointed by the Audit Commission carry out the use of resources assessments as part of CAA. The Audit Commission accepts that this element of CAA was perceived as burdensome by local bodies. For 2010, auditors have been guided to update the use of resources assessments on a risk basis.

Where audited bodies achieved key lines of enquiry (KLOE) scores of level 3 (performing well) or level 4 (performing excellently) in year 1, auditors should undertake only sufficient work to satisfy themselves that there has not been any deterioration in performance, and that the arrangements are still operating effectively. This should be a light touch exercise to refresh existing evidence.

In planning and carrying out use of resources work, auditors and CAA Leads will liaise closely to ensure they share relevant evidence and make effective links between the elements of the CAA framework to reduce the burden on audited bodies.

We will also carry out a more fundamental review of our approach to use of resources and organisational assessments for implementing in 2011. In future years, we will also seek to make more of the annual audit letter as the prime accountability report for organisations.

Care Quality Commission

The CQC welcomes the evaluation of CAA and the key learning points which it provides us with as an organisation. We are already working with the other inspectorates on actions which will address some of the concerns highlighted in the evaluation. These include investigating a better alignment between reporting dates for our assessments of health and social care and those for CAA; as well as building on the relationships developed in the first year of CAA. We are reorganising our front line field force to create a team in each region that will focus on performance assessment, including our contribution to CAA.

As an organisation we have done a lot of work to make sure our assessments which feed into CAA are proportionate, focussing on key areas for health and adult social care in an area in order to reduce the administrative impact of inspection. This work will continue to develop as we look at our assessment of health and adult social care into the future. The evaluations provide a useful touchstone in moving forward with this work.

Ofsted

Ofsted recognises that there are aspects of the process overall that need further development and improvement, in particular issues around communication and information sharing with councils and between individual inspectorates. We will have more direct dialogue with local authorities as part of the CAA process in 2010 to improve this. We will also work with our partner inspectorates to develop a clearer process to streamline the contribution all inspectorates make to the area assessment.

Her Majesty’s Inspectorate of Constabulary

Her Majesty’s Inspectorate of Constabulary (HMIC) acknowledges the evaluation of CAA, which recognises the considerable investment made by participating inspectorates in the first year. HMIC remains committed to supporting the overall aims of CAA in particular we recognise the value in the pooling of information and through Oneplace providing the public with the collective views of the Inspectorates on the effectiveness of local partnerships in delivering local services.

HMIC will continue to build upon the excellent relationships developed between the partner inspectorates. We will also ensure that, where possible, CAA complements the public assurance we continue to provide through our own inspection processes, accessible through our recently launched Police Report Card.

HMI Prisons

HMI Prisons acknowledges the achievements of the first year of CAA and its value to the public. HMI Prisons’ primary focus is on the inspection of nationally managed services and this means that our contribution to CAA has necessarily been limited, but we will continue to support partner inspectorates in a proportionate way by sharing relevant findings from our inspections of places of detention and by contributing to joint criminal justice inspectorate thematic inspections.

HMI Probation

HMI Probation acknowledges the achievements of the first year of CAA and its value to the public. HMI Probation independently inspects probation and youth offending work and will continue to share the results of youth offending inspections to inform the CAA area assessments as in 2009. HMIP will also continue to contribute to joint criminal justice inspectorate thematic inspections.