Before you buy.

Before you buy any e-Learning resources you should ensure that proposed purchases are part of your overall commissioning plans and are aligned to your strategic and operational framework for e-Learning.

If you do this you will be able to demonstrate value for money and reduce waste through duplication.

However, value for money is not the only criterion against which you should evaluate the purchase of potential resources.

You need to create your own set of criteria linked to your own strategy which is fit for your specific purpose, but here are some ideas for what you might include to ensure return on investment.

• Does the resource have employer relevance?
• How will it improve the learner experience?
• What is the type, size, and distribution of the learner audience?
• What is the life span of programme? Will it need updating?
• What is the expected time to complete the programme (either face to face or via web portal)?

What are the delivery costs including – location, number of training sessions required, and learner support mechanisms including requirements for tutor support, administration, and any potential backfill arrangements.

There are also some useful pointers about the purchase of NHS e-Learning resources in: Modernising Healthcare Training which include: The recommendation that the development of e-Learning initiatives should have a measurable impact on performance and productivity and that e-Learning should be an investment rather than a cost, highlighting its potential for delivering wider NHS targets and agendas. The types of evidence suggested in the report could be used to assess impact including:

• evidence of upskilling, a more confident workforce, numbers of people upskilled
• recruitment and retention
• progression to other roles
• improvements in patient safety
• other benefits to patients - e.g. patient education and contribution to learning
• patient feedback
• compliance with mainline metrics and regulatory requirements
• productivity
• evidence of improvements and changes in practice
• money saved through quality improvements
• money saved through reducing opportunity cost of training
• reduction in time to impact, rapid embedding of new knowledge
• reduction in accidents
• reduction in complaints
• reduction in Clinical Negligence Scheme for Trusts (CNST) premiums.

The report suggests two promising methodologies to help with this process:

• The Return On Investment Institute (ROI) methodology.
• Kirkpatrick's four level evaluation model.

These are briefly described below.

The Return On Investment Institute (http://www.roiinstitute.net/) is a US based organisation whose approach includes techniques to isolate the effects of a development initiative from other organisational issues. Its methodology is used for the measurement and evaluation of all kinds of performance improvement programmes and activities, including training and development, coaching, consulting, and project management. The methodology has been used successfully in corporate environments, public sector settings, and organisational contexts and is being used in the UK by the Skills for Business Network.

The beauty of the ROI model is that it iteratively informs the development process. For example, it can be used at the design stage to maximise the ROI of e-Learning materials. In addition, there is an emerging body of evidence related to its utility in the UK which could usefully be explored.

Kirkpatrick's four level evaluation model

According to Kirkpatrick's model, evaluation should always begin with level one, and then, as time and budget allows, should move sequentially through levels two, three, and four. Information from each prior level serves as a baseline for the next level's evaluation. Thus, each successive level represents a more precise measure of the effectiveness of the resource, but at the same time requires a more rigorous and time-consuming analysis.

Related Resources

•  
  Paving the Way
  

  Document providing guidance on pedagogy, accessibility both technical and non-technical, standards and quality assurance in relation to NLN materials.

  Download file (1.95 MB pdf)

  
  

  http://www.nln.ac.uk/?p=Using
•  
  Modernising healthcare training: e-Learning in healthcare services
  

  A report Commissioned by SHAs and DOH - April 2006, also known as the 'Road Map for e-Learning'. A thorough and useful study.

  Download file (649.8 KB pdf)

  
  

  http://www.nesc.nhs.uk/pdf/national_e-learning_report_screen_version_final.pdf
•  
  Popular Learning Models
  

  Some of the more popular learning theories and models used by education practitioners and those with an interest in developing people. The models and theories inform and underpin many modern learning approaches and come out of learner centred and constructivist theories.

  view_page.php?id=73
•  
  Kirkpatrick model of evaluation in the Toolkit
  

  A summary of the four levels of the Kirkpatrick model of evaluation in the e-Learning Readiness Toolkit.

  page_82.html
•  
  Commissioning E-learning in the NHS, Some Key Principles
  

  A paper that sets out the principles that might be adopted by the National Management Group and other commissioners of e-Learning within the NHS to promote confidence, quality, transparency and value for money.

  Download file (154 KB doc)