Turning education and skills policy into practice

Impact studies

AlphaPlus delivers robust impact studies to government and its departments and intermediary agencies.  We have expertise and experience in evaluating the impact of policy and its implementation, for example, the impact of changes to the curriculum and its assessment on teaching and learning. Our team bring subject and sector experience and expertise as well as an in-depth understanding of educational research methodology.

We use a theory-of-change approach to evaluation. During the evaluation design stage, AlphaPlus works with the client to develop a theory of change/logic model with the programme or policy team, which covers:

  • what the programme/policy hopes to achieve within the defined timelines, i.e. outputs, outcomes, impact – and how this nests within other developments
  • the social and behavioural assumptions that underlie the policy/development of the programme; processes; vision and strategy; implementation and delivery; and outcomes
  • how constituent parts of an intervention’s outputs and outcomes can be mapped to the expected impact; and define causal links
  • how this fits with other parallel interventions/programme outcomes and negative or positive implications
  • identifying differential inputs and impacts
  • the context for the programme, e.g. wider policy implications

This helps to develop a set of agreed perspectives and allows the team to be explicit about the assumptions that have been made. The model maps the linked processes underpinning the work and how positive outcomes, leading to the desired impact, are expected to be achieved. This allows the identification of expected impact (impact study) and defines the inputs (process evaluation). If there is a relatively tight timeframe for the policy/intervention and its evaluation, the theory of change approach also enables the identification and evidencing of outcomes (short- to medium-term) as progress towards impact (long-term). This work supports the identification of the key evaluation questions and sets the parameters for data collection and analysis.

We ensure our evaluation design foregrounds enquiry and research rigour, and uses methodologies that are open and transparent to show a clear relationship between research findings, conclusions, interpretation and recommendations. The team has a strong track record of delivering high quality evaluation and consultation reports, which not only present complex information and outcomes in an accessible way, but also translate the key findings to actionable recommendations.

Our approach does not limit our data collection and analysis to any particular research paradigm, e.g. purely quantitative/realist or qualitative/constructionist methods. In practical terms this means we offer expertise in the use of a range of evaluation strategies, including case studies, surveys, meta-analysis of secondary data, and literature reviews. We have experience in using interviews, focus groups, questionnaires and observations for the collection of data.

We also offer training for policy makers and programme managers to support the evaluation of policy and its implementation. The outlines are examples of work with project managers and policy makers (Course outlines). Our approach is to liaise with the client and adjust the content, mode of delivery and learning outcomes to meet the needs of the specific target audience.