AlphaPlus - Supporting awarding organisations

Training services for awarding organisation teams

Reliability of qualifications

Level:  Introductory

Duration:  4 hours 

Description: The Regulator has placed increased focus on reliability in recent years, and not only for general qualifications. But how do you measure the reliability of your assessments?

And how do you interpret the results from those measurements in order to decide whether action is required? Would an assessment candidate get the same results if they took the assessment for a second time?  Or if they had a different marker?  What if they faced slightly different tasks/questions?  How much error is in the scores from a particular assessment and how does this affect the likelihood that a candidate is given the wrong outcome?  All these questions and others are within the domain of reliability.  

This session will introduce some of the key aspects of this topic, seemingly complex but so key to good assessment, and give attendees key messages to take away for their professional work.

Rather than focusing on complex statistics, we start from practical considerations and look at qualitative interpretation before moving to consider numerical approaches.  The programme also looks at practical implementations of reliability measurement in areas where collecting data can be difficult (for example in evidence-based vocational qualifications).

This unit is often delivered alongside the validity unit.

Topics covered:

  • rating the reliability of assessment types
  • reliability Indices and their practical application
  • dealing with key reliability concepts – internal reliability, inter-form reliability and intermarker reliability
  • concepts of reliability and validity taken together
  • how practical reliability measures can be used to introduce transparency and increase confidence in assessment performance.

Typical Task - Working with own assessments to gauge and use appropriate measures of reliability.

Contact

For more information about our AO training service, please contact Giilian Whitehouse:

Email Gillian

Phone: 07881 361718

Sample training materials

We recognise that many of our courses deal with complex topics which can take time to understand. All our delegates receive a complete training pack comprising the materials covered, a handbook covering the subject, copies of the exercises, and a targeted reading list for further study.

We've uploaded some of our training materials for you to view. Please click on the link below to access a zip file of these PDFs:

  • Vocational qualification development guidance
  • Vocational qualification development presentation
  • Vocational qualification development reading list

Download sample training materials