Achieving quality and consistent outcomes

Download Full Handbook Download This Section

1. Why are quality and consistency important?

1.1 In order to comply with equal pay legislation, it is important that organisations are assured of the quality and consistency of their job evaluation work. Consistency is vital to ensure equal pay for work of equal value and to reassure staff that their outcomes have been achieved fairly.

This chapter outlines good practice in ensuring quality and in undertaking checking to ensure consistency of outcomes both internally, against other local matching and evaluations in order to avoid local grading anomalies and consequent review requests, and also where possible externally, with outcomes from other organisations, in order to avoid locally matched or evaluated jobs getting ‘out of line’ with similar jobs elsewhere.

1.2 The first measure to ensure quality and consistency of matching and evaluation is to follow the agreed procedures outlined above and to take such additional steps to help ensure that panels are able to work effectively. This includes ensuring that: 

  • All panel members have been fully trained and updated in using the NHS JE scheme; in matching or local evaluation, as appropriate, and in the avoidance of bias.
  • Panels are conducted in partnership and constituted so as to reflect the diversity of the workforce as far as is possible (e.g. differing occupational backgrounds, gender, ethnicity etc).
  • Obvious sources of bias and inconsistency have been eliminated e.g. exclusion by agreement of panel members known to have strong views for or against jobs to be evaluated and those from the job group being matched or evaluated.
  • Where possible, there is a mix of experienced and newer panel practitioners.

1.3 The most common source of poor quality and inconsistency in local matching and evaluation is inadequate or inaccurate job information, whether in the form of a job description and any additional input for matching, or a completed and analysed JAQ for local evaluation. Possible steps to minimise problems arising from such job information include: 

  • In advance of the post going to panel, joint quality assurance (by job evaluation leads or their nominees) of the written job information to identify obvious omissions or inaccuracies.
  • When the panel meets to consider the post, ensuring that panel members can seek additional information from jobholders and/or line managers, where it is agreed that this is necessary.

1.4 Quality and consistency of matching/evaluation panel outcomes are improved by: 

a. Matching or evaluating jobs in family or equivalent groups (e.g. all finance jobs, all unique specialist jobs from an occupational group) as this allows for ongoing comparisons and provides some immediate internal consistency checks. 

b. Prior to matching or evaluation, panel members should read the most relevant national profiles (e.g. finance profiles for finance jobs, specialist and highly specialist healthcare professional jobs for unique specialist healthcare jobs), noting features which are similar to those of jobs to be matched or evaluated locally. 

c. Avoiding being influenced by anticipated pay levels. Job information should not state salary information; if the outcome is out of line with current or anticipated salary levels, this will be dealt with later. 

d. Cross-checking individual factor level outcome against national profiles with similar features during the process (not necessarily similar jobs e.g. the physical skills demands of an IT job requiring keyboard skills could be checked against clerical and secretarial jobs on this factor) to ensure the appropriate national profile has been selected.

1.5 Once a matching or evaluating panel have agreed an outcome, the panel members should carry out a preliminary check to ensure they have followed the correct procedure, considered all available job information and made accurate, comprehensive and coherent notes to record their findings. 


2 Consistency checking

2.1 The quality and consistency of all panel decisions is confirmed by a process of consistency checking, which also undertakes monitoring of outcomes across the organisation.

2.2 A full consistency check should be undertaken by a designated partnership pair. (e.g. comprised of management and staff side job evaluation leads who are experienced job evaluation practitioners and trained in consistency checking).

2.3 The consistency checking process is as follows:

  • Completed matching forms and evaluation reports should be checked for quality to ensure that all boxes have been filled in and reasons given in relation to the job in question for all the factor levels awarded.
  • The outcomes (for each factor as well as the job as a whole) should be checked for consistency against:
    • Other matches completed by the same and other matching panels.
    • Other local matches within the same occupational group* and job family*.
    • Other local matches within the same pay band.
    • National profiles for the same occupational group* and pay band.
    • Check total weighted score and rank order of jobs for the organisation.

2.4 Any apparent inconsistencies in matching should be referred back to the matching panel with any queries and/or comments. The consistency checkers should NOT substitute their own decision. The original panel should then review the match or evaluation in question and answer any queries or make amendments to the original match, as appropriate.

2.5 It is recommend that, especially in the case of evaluations, outcomes are compared with all relevant national profiles e.g. all those which are in the same job group and pay band. An evaluation may have been required as the post requirements do not conform to the normal tasks and responsibilities for a role. Consistency checking should confirm these differences are justified with the evidence when compared against the national profile.

3 Further advice on consistency checking

3.1 Consistency checking is largely a matter of taking an overview of a batch of results and applying common-sense, but there are some useful questions to ask, for example:

a. Do manager and supervisor jobs match or evaluate higher than the jobs they manage or supervise on those factors where this is to be expected for example responsibility for policy and service development, responsibility for human resources, freedom to act? If not, is there a good reason for this?

b. Do specialist jobs match or evaluate higher than the relevant practitioner jobs on those factors where this is to be expected for example, knowledge, analytical and judgemental skills, responsibility for human resources (if teaching others in the specialism is relevant)? If not, is there a good reason for this?

c. Do practical manual jobs match or evaluate higher than managerial or other jobs where hands-on activity is limited on those factors where this is to be expected e.g. physical skills, physical effort, working conditions? If not, is there a good reason for this?

3.2 Consistency checking is made easier when records are stored on a computerised system. Such a system can flag up inconsistencies, missing data or where correlations between certain factors are not as expected, for example, KTE level 7 with FtA level 1.

4. Advice on avoiding bias in relation to perceived job status

4.1 NHS Staff Council is aware that there are sometimes problems with over-evaluation and under-evaluation of jobs at the upper and lower ranges of the salary scale.

4.2 Organisations are strongly advised to use their partnership arrangements on an ongoing basis to check particularly carefully their outcomes for bands 1 - 3 and bands 8 - 9 to ensure that these are safe and that there is sufficient robust evidence to justify the outcome. If it is discovered that an outcome is unsafe, then this should be rectified in order to maintain the integrity of the JE scheme in your organisation, either through referral back to a panel in order to obtain a robust outcome or under a joint quality locally agreed assurance/governance process. Any disagreement with the outcome should be dealt with through the process detailed in Chapter 15.

Over-evaluation of jobs
JEG has encountered examples of inflation of various factors in respect of band 8c/d/9 outcomes, for example, jobs with titles such as deputy director of finance or head of capital investment, where panels may have made assumptions about factor levels based on little evidence. This may be because there is a belief that a job deserves high factor levels on the basis of perceived status, job title, level of job in the organisation and perceived previous salary levels. The danger in this approach is that it may lead to some jobs being banded higher than the evidence suggests, in other words an unsafe outcome (the ‘halo’ effect).

Under-evaluation of jobs 
There is evidence of this happening particularly with jobs deemed to be in band 1. Lower factor levels appear to have been awarded on the basis of assumptions being made about the processes undertaken or the level of knowledge or skill needed to carry out those processes. Job rationales, particularly in the case of band 1 jobs, had been frequently underscored and had little differentiation from the rationales in band 2 jobs.

4.3 All parties will need to satisfy themselves that the chosen process is consistent with the NHS JE Scheme matching/local evaluation and review process. It is important that all ground rules should be jointly agreed in advance of embarking on the exercise, for example ensuring up-to-date/accurate and jointly agreed job descriptions/person specifications; whether or not matching to national profiles is possible; what the outcome possibilities are and, once these are identified, what rules on protection etc will be put into place. This will all need to be done in partnership and the responsibility for any misapplication should also be shouldered in partnership

4.4 Normally, any anomalies should have been discovered during the consistency checking stage. During this process, a careful assessment should be made across the individual bands to ensure that the outcomes are similar in terms of demand. This will help to avoid the risk of challenge under equal pay legislation

5 Concerns about local consistency

5.1 Staff or managers who have any outstanding concerns about local consistency should first raise them with the Job Evaluation leads so that they can be investigated. JE Leads may wish to check their outcomes with a neighbouring trust or organisation for a bench marking comparison.

5.2 If concerns cannot be resolved locally they can be referred, by either party to the Country JE leads or the JEG secretariat (JEG chairs and NHS Employers job evaluation lead) for advice. See Chapter 15 for details of how concerns and disputes can be addressed. 

Latest Tweets

Latest Blog Post

Tapping into the lived experience of BME staff

15 / 5 / 2019 7.30am

Dr Habib Naqvi reflects on the links between the Workorce Race Equality Standard, workforce representation, staff engagement and patient satisfaction.

Why Register?

Sounds great, what next?

Not now, I will register later

Log In