Scrutiny – is the devil in the detail?

Last updated:31 March 2010

Linda Phipps
Health Scrutiny Advocate, CfPS & Non-Executive Director
Leeds Partnerships NHS Foundation Trust

As a scrutineer – a NHS NED, a Governor or a Councillor on an Overview and Scrutiny Committee – what are the questions keeping us awake at night?

Here’s one that has exercised me in these roles – which I’m sure will resonate with many others.  If Assurance is a key part of the Board’s agenda (“Assurance: the Board agenda” DH, 2002), how do we do obtain this?  What kinds of assurance do we need (e.g. assurance on finance, clinical practice, quality, user experience).  And what kind of scrutiny do we do?  The crowded agendas of Board and Committee meetings often include items requiring the scrutiny of lengthy and/or complex documents.  These might include the Assurance Framework, analysis/mitigation of key risks, detailed procedural documents for agreement or ratification, a report of an investigation, or performance reports. 

Often detailed scrutiny will be delegated to sub-committees, as part of a chain of assurance and so as to de-clutter Board etc agendas.  There will be a requirement that the Sub-Committee provides assurance back to the Board on the effectiveness of this scrutiny and that there are/are not causes for concern.  Such documents for scrutiny might be scheduled for 10-30 minutes discussion.  Sufficient time to discuss salient points/issues of concern.  But not sufficient say for a full detailed page-turn type scrutiny. Therefore the question is, HOW should we scrutinise such documents in a brief space of time in a way that adds value and provides assurance?

In any case, Boards and their sub-Committees seek to be strategic in focus – and to delegate detailed scrutiny to their subcommittees.  So what is the balance to be struck?  How do we gain assurance on such documentary evidence – or pass assurance upwards to the Board or our parent committee that a document has been scrutinised in sufficient detail, and is fit for purpose, and that there are no major causes for concern?

At recent meetings of my NHS Foundation Trust’s Risk Management and Governance Committee, the issue of how Committees gains assurance on behalf of the Board and the whole organisation, has been extensively discussed.  In the spirit, not of identifying that there is a problem, but of learning and improving, using a targeted approach to the best use of resources.
 
It was agreed that as well as issues around the processes of how the Committee gains assurance, there is an important factor around behaviours such as how committee members approach their contribution to the work of the Committee and how they gain assurance or prepare to raise any concerns around governance processes or specific documents. 
 
Within this consideration of the effectiveness of governance processes, it was proposed that a piece of work be carried out by Internal Audit which would help to give assurance on the effectiveness of the governance processes sitting beneath the Committee.  It was agreed that this would take one area of work and look at the development, implementation and use of a policy or procedure, through a “cradle to grave” piece of work to be undertaken through Internal Audit. 
 
The process proposed was to select a policy/procedure (about 6-12 months old) that has had time to be implemented.  Internal Audit would then look at each stage of its development to test how people were involved.  They would find out how effective the consultation had been; how well the policy/procedure conforms to guidance on procedural documents; and to what extent staff have been informed about the procedure; and what assessment of compliance with it in practice, and of the effectiveness of its impact on outcomes, has been carried out by managers.
 
The essential concept was to learn about the process through following the policy or procedure’s “journey” through the organisation.   The outcome would be to consider independent observations and to give assurance that the Committee - and the organization overall - can place reliance on the systems and processes that lie beneath its ratification role in respect of policies and procedures.  To identify where (within which areas of the organisation) there may be good or better practice in rapid scrutiny of complex documents, from which we can learn.  There was interest in learning from the “best” Trusts – and from those which have identified gaps between their theory (policies) and practice.

It was anticipated that we would identify some options for changing practice and processes to enhance the value and relevance of assurance processes (e.g. sampling; or delegation to other or special groups.  Another part of a risk-based approach to assurance could be that where there is a lower risk or good track record, perhaps only one member of the group reads the documentation and provides assurance back to the rest of the Committee/group.)  Some of these ideas have recently been brought into a rapid improvement event designed to look at the efficiency and effectiveness of processes.

However, the other aspect of this debate – about how behaviours contribute to effective assurance – is probably a harder area to assess.  Does this lend itself to an audit-based approach?   Or is an observational process needed?  What aspects could this cover – for example:

Inputs

• How papers and data are prepared, the quality of summary sheets and whether they simply summarise the contents of papers, or whether they raise the key issues to be aware of in relation to assurance issues
• The amount of time committee members commit to reading papers, raising queries in advance, or preparing questions to ask around assurance
• Whether documents are scrutinized by page turning, or by chair’s request for points to raise, or by exception?
• Self-review at an individual level e.g. what questions did I ask to challenge the degree to which we are assured and the risks to assurance and to controls and assurances on controls?  What proportion were strategic?  

 
Outputs

• At meetings, observation of the nature and quality of the debate on assurance and the amount of contribution from all committee members
• Whether committee members operate at a strategic level or whether they reinforce their assurance by occasional “deep dives’ into the detail
• Self –review at Committee level around effective allocation of time and sufficient time to scrutinize the most important documents or aspects.

Outcomes

• What impact on assurance does the activity of a Committee/group have? 
• What is its added value? 
• Is the organization safer – or more/sufficiently assured on risks?

The Audit Commission has confirmed that this is a crucial and fruitful area to consider – though they are not able to commit resources to speculative i.e. unfunded work in this area at this time.  Perhaps other Boards and Committees have some similar ideas – or practical suggestions for how to scrutinise complex areas briefly – in a way that provides real and effective assurance? 

31 March 2010