The measurement of effectiveness of online training is a hot topic right now. In this article, I outline how you can use the Kirkpatrick’s model of training evaluation to measure training effectiveness, its impact, and the ROI of eLearning.
How To Determine The ROI Of eLearning
The measurement of ROI of eLearning needs an integrated approach that should begin during the Training Needs Analysis or TNA phase and should successively build up right up to the determination of its impact on business.
How Do You Begin The Exercise To Measure The Effectiveness Of Online Training?
At EI Design, we use the approach shown here. Essentially, we focus on each stage from TNA to ROI calculation as the right action at each stage will impact the ROI of eLearning positively.
- Training Needs Analysis (TNA)
This is the baseline and will be eventually used to measure the desired gain for the learners (acquisition of new skill or fixing a gap) and if this gain resulted in the required impact the business had sought. Capturing the needs of the learners and having clarity on how you will measure their progress (that is, the expected gain) is a key component of the exercise.
- Determining the right training format (online, blended, or ILT)
The next step is identifying the right format of training that aligns best with the TNA (Sometimes, training may not be the answer, and you need to identify supporting measures like coaching or mentoring). The selection of the right format is crucial in encouraging the learners to pursue it and also in ensuring that they connect with it, complete it and apply the learning on the job.
- Identifying the learning strategy
The selection of the right learning strategy format is vital in engaging the learners. As we know, only an effective learning strategy can create a sticky learning experience. However, it is equally important to note that having formal training alone may not be enough. We also need to provide room for application of knowledge and practice or nudges to mastery. Hence, the learning strategy must have a combination of formal training and performance support intervention to successfully manage your mandate.
- Determining the gain by the learners
An effective assessment strategy is the main approach to determine this. However, it is important to ensure that the measurement of this gain also factors for the validation of the application of the acquired learning and is not limited to the validation of knowledge acquisition.
- Assessing the gain for business
We can look back at the parameters identified during the TNA stage and assess the required gain that has occurred and if the business saw the required impact.
- ROI determination is the final outcome of this exercise
Once we monetize the gain and compare this with your costs, we will arrive at the ROI of your eLearning. Often, this outcome may need you to go back to the TNA and reassess or tweak the way forward.
To show how these aspects are interconnected and are part of a cycle is captured in the diagram here.
ROI determination methodology: One of the popular models used for ROI determination is the Kirkpatrick’s model of training evaluation. I will outline what it entails and how can it be used to determine training effectiveness and the ROI of eLearning.
What Is Kirkpatrick’s Model Of Training Evaluation?
Kirkpatrick’s model of training evaluation is one of the popular models used to evaluate the effectiveness of training. It was created in 1959 and has undergone revisions in 1975 and 1994. The fact that it has survived for over half a century affirms the value it continues to offer.
It features the following four levels:
Level 1: Reaction
Level 2: Learning
Level 3: Behavior
Level 4: Impact
Initially, the model was viewed as a pyramid with each level building up from the previous one.
Increasingly, the same 4 levels are viewed as a “chain of evidence,” and I feel this reflects a more relevant connection between the levels.
How Can You Practically Use The Kirkpatrick Model Of Training Evaluation To Determine The ROI?
To use the Kirkpatrick model of training evaluation, we need to identify two aspects at each level namely:
- What are we measuring?
- How will we use this outcome (to improve training effectiveness and increase its impact)?
Level 1: Reaction
Objective: At this level, the focus is to determine the learner’s reaction to the training. Today, we have wide-ranging options through Learner Analytics to identify if the learners liked the training if they found it useful and if they would be able to apply the learning.
From an evaluation perspective, this feedback enables L&D teams to assess if they are on track or if any further changes are required.
Level 2: Learning
Objective: At this level, the focus is to determine what was learned or gained (this should be attributable directly to the training).
From an evaluation perspective, this feedback enables L&D teams to assess if they met their mandate (captured during TNA) that could include:
- Knowledge gain.
- Acquisition of a new skill.
- Further proficiency gain on an existing skill.
- Behavioral change.
The pointers from this stage of evaluation would point to:
- The need for further training.
- The need to supplement formal training with other measures that could include performance support intervention or mentoring/coaching.
Level 3: Behavior
Objective: At this level, the focus is to determine if the learner behavior changed (again, this should be attributable directly to the training).
From an evaluation perspective, this feedback enables L&D teams to assess if there was a demonstrable change in the learner’s behavior.
Often, this is can be tricky. Although, learners had successfully cleared the assessment, yet there is no demonstrable change.
This may need re-assessment to determine why this is not happening.
- Sometimes, it could be because learners have no opportunity to demonstrate what they learned, and often, it may point to the need for reinforcement.
- There may be a need to have refresher programs to be offered over an extended period of time till the required gain is observed.
Level 4: Impact
Objective: At this level, the focus is to determine if the business saw the gain and if the required impact was created on account of the training.
From an evaluation perspective, this feedback enables L&D teams to review if the expected impact identified during the TNA phase indeed happened.
How Can You Use Kirkpatrick’s Model Of Training Evaluation To Measure ROI?
ROI determination (or Level 5): This is an add-on to the initial model (that has 4 levels) and is referred to as the Kirkpatrick-Phillips Evaluation Model of training.
In simple terms, you will have a positive ROI of eLearning if the demonstrable gain from the training exceeds the cost you incurred to create and deliver the training.
I hope this article gives you practical insights into how you can enhance the impact of each stage from TNA to the evaluation of its impact and see a positive ROI of eLearning.
The post Determining The ROI Of eLearning – Using Kirkpatrick’s Model Of Training Evaluation appeared first on eLearning.
The post Webinar on Strategies To Create Compelling And Immersive Compliance Training – Featuring 8 Examples appeared first on eLearning.
eLearning Trends In 2018: Which One Suits Your Needs Best?
During my review of eLearning Trends for 2018, I relooked at the trends by validating their adoption from the following perspectives:
- Data from our customers on the adoption of these trends.
- My research on what is happening in the wider global landscape.
Overview Of My Mid-Year Review (CY 2018) On eLearning Trends For 2018
During this review, I used the following 4 parameters and then mapped my initial eLearning Trends and Predictions for 2018 to each of them.
- Parameter 1: What trends reflect the approaches that align better with the way we learn?
- Parameter 2: What trends are able to influence and improve employee performance?
- Parameter 3: What approaches can help us measure performance gain?
- Parameter 4: What approaches can help with ROI determination?
- Of the total 18 trends that I had outlined in the first edition of my eBook on eLearning Trends and Predictions for 2018, I saw a clear resonance with 15 of them based on the feedback I received from our customers and my research.
- Additionally, the following two trends that were part of the “Emerging or Maturing—Watch out for” show the following adoption:
AR/VR for immersive learning: limited adoption.
Artificial Intelligence or AI in learning: still futuristic.
- The last trend in rapid eLearning tools was not part of my mid-year review focus. However, this adoption (notably Articulate Rise and iSpring) has caught eLearning developers’ attention in early 2018. My assessment is that it will continue to do so in the balance year, too.
Here is the original list that was featured in my eBook eLearning Trends And Predictions For 2018. Download it to gain insights on each of these trends.
Part 1-What will continue to offer value
- Digitization of ILT (to Blended or Fully Online)
- Performance Support Tools
- Social Learning
- Mobile Apps or Apps for Learning
- Video-Based Learning (Videos and Interactive Videos)
- Next Gen-LMS/LCMS
- Rapid eLearning Tools
- Informal Learning
Part 2-Coming of age—Increase in adoption
- Learning Portals
Part 3-Maturing and emerging—Watch out for
- Big Data—Reporting and Analytics, and its usage in the determination of ROI
- Learner Analytics to enhance learning experiences
- Artificial Intelligence (AI) in learning
- AR/VR for immersive learning
Here is a summary of how these eLearning trends map to my current research and findings.
Parameter 1: What Trends Reflect The Approaches That Align Better To The Way We Learn?
The following 5 eLearning trends map to this, and I have added some reasons why you should adopt them.
- Mobile learning: This trend will continue to see increased adoption for corporate training, including formal training and performance support, as well as to support ILT delivery.
- Digitization of ILT (to blended or fully online): This trend is poised for rapid acceleration, as more and more organizations opt for this flexible approach to reach a wider audience in a shorter time and with lesser budgets.
- Microlearning: This trend has caught the attention of L&D teams and learners. It will continue to accelerate and be used as a series of nuggets for formal training or just-in-time learning aids, as part of the performance support intervention.
- Personalization: This trend is gaining momentum, as it addresses the interest and aligns better with the proficiency of the learners.
- Curation and user-generated content: While the Internet provides information readily, we often need to wade through reams of data to find the apt one. The combined usage of curation in conjunction with the flexibility to have learners contribute to it (through user-generated content) will see traction.
These are also reflected in the wider adoption of approaches that resonate better with learners leading to a wider adoption of the following two trends:
- Mobile apps for learning
- Video-based learning (videos and interactive videos)
Parameter 2: What Trends Are Able To Influence And Improve Employee Performance?
The following 4 eLearning trends map to this, and I have added some reasons why you should adopt them.
- Performance Support Tools (PSTs): These just-in-time learning aids are available to learners during their workflow exactly at their moment of need. Accelerated by the wider usage of mobile learning and microlearning, their usage will grow further to facilitate the application of the acquired learning on the job.
- Gamification: This trend has seen a wider adoption this year and will continue to be used for serious learning in corporate training. I also see an increase in the usage of gamification in training areas that have not experimented much with it (for instance, compliance training and application simulation training).
- Social learning: Although I still see limited focus by organizations on this trend, it is one trend worth investing in. I believe it aligns naturally with the way we learn, and the usage of specific portals offering collaborative learning (as well as curation and user-generated content) or standard social learning platforms will see a steady increase.
- Informal learning: Like social learning, this too is recognized as another medium to support learning. Although the onus of informal learning largely rests with the learners, yet organizations can do a lot to promote it in the workplace. It would be interesting to see how this adoption shapes up.
Parameter 3: What Approaches Can Help Us Measure Performance Gain?
The following 2 eLearning trends map to this, and I have added some reasons why you should adopt them.
- Next gen-LMS: Learning platforms and LMSs that enable learners to “pull” information of their choice, rather than the traditional approach of “push”, are the future. They support other aspects like learning journeys, personalization, collaborative learning, and offer recommendations based on the learner’s consumption. Their analytics offers valuable insights on how learners are interacting with the eLearning courses and how they are consuming content, while they can also provide insights on learning effectiveness. This trend will see significant acceleration.
- Learning portals: Designed to meet certain key initiatives, they feature the aspects highlighted in the next gen-LMS, along with learning analytics. They will continue to be used where legacy LMS platforms can’t meet these mandates.
Parameter 4: What Are The Approaches That Can Help With ROI Determination?
The following 2 eLearning trends map to this, and I have added some reasons why you should adopt them.
- Learner analytics: Using learner analytics is vital to validate the assumptions you had as you created the initial learning designs. More specifically, it provides insights on what more should be done. For instance, Tin Can API can provide useful insights on how learners are interacting with the course, and this analysis can be used to tweak the learning design and obtain better results.
- Big Data reporting and analytics: Big Data in eLearning is the huge data collected from multiple sources, including LMS/LCMS and Learning portals. Increasingly, a lot of learner insights are also collected through surveys/polling. Because of the sheer magnitude of the data and non-availability of a single tool to process this, reporting and analytics on Big Data is a tall order. This trend will continue to mature, as a number of techniques enable the processing of this data to improve learning experiences and assess the impact of training.
I hope this mid-year review of eLearning Trends in 2018 will help you as you look at tweaking or enhancing your learning strategies in the balance part of the year.
The post Updated eLearning Trends In 2018—Featuring 4 Parameters To Help You Select The Right One! appeared first on eLearning.
Learning technology is constantly changing. Sometimes it’s hard to differentiate one type of platform from another. The three types of learning technology I’m most frequently asked about are learning management systems (LMS), learning content management systems (LCMS), and learning record stores (LRS).
In this video, I’ve done my best to give you the “traditional” definitions of these platform types, but please know that some platforms don’t perfectly fall under any of these labels. For instance, some learning management systems (LMS) utilize xAPI to a degree, even though xAPI is more frequently associated with learning record stores (LRS). Some learning management systems have features or add-ons that would traditionally be classified as functions of a learning content management system (LCMS) or content management system (CMS).
The bottom line is, if you want to know whether a learning technology platform can serve a specific function, ask the vendor. Learning technology is getting more robust as vendors incorporate features that go beyond traditional classifications.
The post Learning Technology Defined: The Difference Between an LMS, LCMS & LRS appeared first on eLearning.
Why You Should Use Performance Support Tools
According to the 70:20:10 model for Learning and Development, we learn:
- 70% from on the job experiences (experiential learning).
- 20% from interactions with others (social learning).
- 10% from formal (structured learning).
As we note from this model, organizations need to focus more on the informal and social learning aspects to effectively achieve their learning mandates.
Let’s first understand what exactly Performance Support Tools (PSTs) are and why you should invest on them.
What Are Performance Support Tools (PSTs) And How Can They Be Used?
Performance Support Tools or PSTs are learning aids meant to help employees with on-the-job support at the precise moment of their need. They are designed to support and guide employees when at work.
They are easily available and usually embedded directly into the learners’ workflow helping organizations provide the required task support and up the productivity quotient of their employees.
- They are a great fit for informal learning.
- They can be also be used to support formal training to improve the stickiness of learning and its eventual application on the job.
- PSTs accelerate learning as they are part of the learner’s work-flow and are available precisely at the time of their need. They are designed to be easy to assimilate and easy to apply.
- You can integrate microlearning-based strategies to create them and see the application levels zoom.
- Couple them with platforms for social learning and you will see the impact of your training multiply.
Why Does It Make Business Sense For Organizations To Invest On Performance Support Tools?
If I were to ask you what you had for breakfast today, your reply would be an instantaneous one. It isn’t quite going to be the case if I ask what you had for dinner a month ago. We’re creatures who keep forgetting things!
As a matter of fact, we forget 80% of what we learned in 30 days as shown in this diagram featuring the “Forgetting Curve” by Herman Ebbinghaus, a German Psychologist.
Formal learning, as you may be aware, takes place for a few days in a year. Now view this from the prism of the Forgetting Curve and you’re bound to have second thoughts about your investments on using only formal training to make the kind of impact you’d like on your learner performance.
So What’s The Way Forward?
A great way to address this challenge is to have a blend of both formal and informal training and offer a solution that treats “Learning as a continuum”.
Here’s what you can do:
- Ascertain the portions from your formal training program that you want your learners to remember, check, refer or apply, and use Performance Support Tools (PSTs) to reinforce those messages.
- Design your PST nuggets to be stand-alone assets aligned to specific learning outcomes.
- Weave your training nuggets into a “learning path” and keep the knowledge thread relevant and in appropriate order over the course of your identified training period.
- Combine the power of reinforcement with new assets for higher recall, retention, and application of the knowledge gained on-the-job.
The result? You’ll be able to see a dip in your formal training costs and an increase in learner productivity and performance.
How Do Performance Support Tools Help Learners?
Performance Support Tools (PSTs) are informal in nature and much to the learners’ liking, they help them learn and work simultaneously. Performance Support Tools (PSTs) are a learners’ delight as they:
- Help them perform their tasks without having to seek external help.
- Cut down on time that would otherwise have been wasted by experienced employees helping the lesser experienced employees.
- Help new employees complete their tasks in quick time and with the desired quality, especially when formal training is not available.
- Can be used to help new employees understand complex tasks and processes in a simplified way.
- Are a great way to learn about new systems, upgrades, or when something is being introduced afresh in the workflow/process.
What Are The Possible Ways To Offer Performance Support Tools?
There is no dearth of options when it comes to offering Performance Support Solutions. At EI Design, we offer Performance Support Tools in various formats, including:
1. Mobile Apps.
Mobile Apps and just-in-time information go hand in hand. They are an ideal way to offer Performance Support at the time of learners’ need – anytime, anywhere.
2. Videos And Interactive Videos.
If you had to unpack and start operating your brand new camera and you’re not sure how to go about it, what would you choose – leafing through the manual or a three-minute YouTube video? Most modern day learners opt for the latter. All the more reason for you to leverage on videos and interactive videos to offer Performance Support when your learners are stuck and need help!
3. Decision Making Scenarios.
Learners keep encountering complex situations at work. You can make their life easier through short branching scenarios helping them practice their way out of tough situations and take decisions in a risk-free environment to bring about application of learning on-the-job.
4. Whiteboard Animations.
Animations and illustrations help you say things you would otherwise find difficult to convey. Their innate ability to incite the lighter side in your learners goes a long way in conveying important messages and long-term retention of that information.
5. Kinetic Text/Animations.
True, a picture speaks more than a thousand words. But you don’t need a thousand words every time to say what you want to say. Sometimes, just a few words do the job – especially when they’re set in motion and animated. If text is all you’ve got and the concept you want to convey is not that easy to visualize, Kinetic Text/Animation is what you should be looking at. You can club it with background music or a voice over and the impact will be there for you to see.
6. Interactive PDFs.
Got information locked in lengthy documents? You can convert them into interactive PDFs, provide a user-friendly navigation, and add some visual appeal to please the eye. They’re a great format to help learners go through factual and knowledge-based information quickly and easily. Using this format, you can offer your learners tips, fact sheets, checklists, practical forms, participant guides, processes, and procedure documents.
You can convert your documents into HTML5 compatible eBooks and make them multi-device compatible. They can also be converted as SCORM packages. You can package key information in your eBooks in the form of key takeaways from a training program, participant guides, processes, and procedure documents.
8. Expert Videos, Webinars/Recorded Webinars.
Experts carry that aura around them, don’t they? Irrespective of whether it’s a classroom tutor, eLearning guide, or workplace mentor, learners treat them with respect and value their insights. You can offer Performance Support to your learners in the form of expert videos and videos of webinar recordings. You can also track your learners’ performance through assessments with SCORM packaging.
These are short episodes of audio or video that learners can download on their device and watch/listen to when traveling, waiting for an appointment, or any other occasion to kill time. They find a lot of takers as they are brief and entertaining. You can use these formats to offer Performance Support at the time of learners’ need as they can be watched/listened to on-the-go.
You can also take a look at this video on the different formats that you can use to offer Performance Support:
What More Is Possible?
As we saw, the organizational learning strategy needs to focus on ways and means to enhance on-the-job learning. Performance Support Tools (PSTs) help you achieve this mandate.
PSTs can be aligned easily to your other key learning strategies like:
- mLearning or mobile learning (by making PSTs in mobile-friendly formats).
- Designing learning for the millennial generation (innovative formats of PSTs can be aligned to the way Millennials learn).
- Micro portals.
I hope this article provides you with the required insights on Performance Support Tools (PSTs) and why you should use them. Do contact me if you need any specific support to integrate Performance Support in your learning strategy.
The post What Is Performance Support And Why Should You Use Performance Support Tools? appeared first on eLearning.
Wikipedia defines Gamification as “the use of game thinking and game mechanics in non-game contexts to engage users in solving problems”. While Gamification has been applied in several domains, our focus has been on its application in learning. Research certainly confirms the advantages of learning through Gamification. Let me share a case study that compares the gains in learning achieved through serious game concept vs a traditional eLearning approach.
Gamification In Learning
Wikipedia defines Gamification as “the use of game thinking and game mechanics in non-game contexts to engage users in solving problems”. As per Wikipedia, “Gamification techniques strive to leverage people’s natural desires for socializing, learning, mastery, competition, achievement, status, self-expression, altruism or closure. Gamification strategies include use of rewards for players who accomplish desired tasks or competition to engage players. Types of rewards include points, achievement badges or levels, the filling of a progress bar or providing the user with virtual currency. Making the rewards for accomplishing tasks visible to other players or providing leader boards are further ways of encouraging players to compete”.
While Gamification has been applied in several domains, our focus has been on its application in learning. The games that we design are therefore geared to meet definite learning outcomes and our Gamification strategies broadly map to:
- Tasks or concepts that are overlaid on the learning content but are not related to the content
- Contextual tasks or concepts that are overlaid on the learning content
- Partial gamification (notably in inline checks and end of course assessments)
Research certainly confirms the advantages of learning through Gamification. Let me share a case study that compares the gains in learning achieved through serious game concept vs a traditional eLearning approach.
We had to design a quick online training for Project Managers who were being groomed to handle Account Management practices. The course on Account Management Fundamentals outlined the basics of account management and more specifically, what they must do to move their organization up the value chain.
The learners needed to understand two aspects:
- Their organization’s view that was to scale the newly acquired/existing customer to an Account and then to a Key Account
- The need to work on changing the customer view and scale their organization from a service vendor to a strategic partner (This journey was mapped to four levels/quadrants and it represented the challenges that the learners must face to accomplish the required mandate.)
First approach (traditional eLearning based)
The first solution was in a traditional eLearning format. To get the online training quickly off the ground, we used the Articulate Presenter based approach to present the basics of Account Management. This is what the first version of the course looked like.
Gains and challenges
The initial mandate was to quickly train the Project Managers on the essentials of Account Management and this was met.
However, the feedback from the learners was that they need specific cues on how to accomplish each step to move up the value chain.
Second approach (serious game based)
We reworked with the revised learning mandate and now chose a Gamification approach.
- The key objective for the learner was to understand how to meet the organizational mandate to graduate from Level 1 association with customers (as a service vendor) and move up progressively to Level 2 (as a service provider), Level 3 (as a value provider) and finally to Level 4 (as a strategic partner).
- These four levels were mapped to four challenges/quadrants as shown in the first screenshot.
- The entire learning was driven through interactive scenario based questions (based on real life situations) that the learners had to attempt at each level/quadrant in the game. Based on their responses that enable them to clear a given quadrant, they would be provided access to move to the next one.
- The approach was designed for scalability wherein the questions could be easily updated. We also provided a randomized pooling of questions to ensure that on every return, the learner encountered some new challenges.
- The second screenshot presents the rules of the game.Learners had the option to refer to the learning content at anytime in the game as a lifeline.
- On successful completion of all four quadrants, the learners had gone through typical situations they are likely to face in real life, practice them in a safe environment and realize their goal of becoming a true Account Manager.
This approach met the mandate and the serious game concept enabled the learners to handle the real life challenges in an engaging and immersive format. The ongoing updation of the question bank and their randomization kept them intrigued and also enabled them to continuously hone their skills.
The post Gamification In Learning: Featuring Gains Through A Serious Game Concept appeared first on eLearning.