The post Change a Course Cover Image in Adobe Captivate Prime appeared first on eLearning.
While it is obvious that Learnability or learning effectiveness of online courses has an impact on ROI, its measurement is rather a tall order. This article outlines how you can use our framework to predict, measure, and validate the learnability or learning effectiveness of your online courses.
What Is Learnability Or Learning Effectiveness Of Online Courses?
Learnability or learning effectiveness of online courses can be defined as the ease and speed with which the learners can acquire the required information/knowledge/skill. There is an associated aspect that you need to be mindful of, that is, the scope of learnability. Once this measured, we can assess the progress from knowledge acquisition to application reflected as performance gain.
The scope of learnability evaluation can be looked at in two ways:
- Initial Learnability: Initial Learnability applies to the performance over a single, short term usage period.
- Extended Learnability: Extended Learnability applies to the performance change over time.
What Is The Significance Of Learnability Or Learning Effectiveness Of Online Courses?
The significance of learnability is evident. Only well crafted courses with the right learnability can help learners perform better and help organizations meet their goals.
Studies have revealed that learners lose up to 40% of their time due to “frustrating experiences”with courses. The common causes being missing, hard to find, unusable, and irrelevant features of the course.
Similarly, a study carried out by Federico Borges found that learners get frustrated due to a variety of reasons including the learning strategy used, lack of prompt feedback, the company culture, the learning material that was designed, and so on.
Does Learnability Impact The Computation Of ROI On Training? Is This Impact Direct Or Indirect?
Most of us are familiar with Kirkpatrick’s model of evaluation as shown here:
- Level 1 – Learner Reaction – Was the course relevant, useful, and worth my time?
- Level 2 – Learning – Did the course lead to increase in skill/knowledge? Did the course meet the required cognition level?
- Level 3 – Application of learning – Was there a performance improvement or change in behavior based on the learning?
- Level 4 – Business impact – Was there a tangible and demonstrable value after the training?
This model can be extended to measurement of ROI on training by mapping monetary value to the final gain (business impact).
Impact of Learnability on ROI on training: Learnability or learning effectiveness impacts each level of Kirkpatrick’s model, eventually helping you maximize ROI on training.
- Learnability impacts level 1 and 2 directly and enables application of learning in level 3. Once these three are in place, the business impact is likely to be met. Subsequently, the ROI on training can be calculated (mapping the business impact to a monetary gain).
How does this really work to impact the ROI?
- A better and meaningful user experience increases the stickiness of learning.
- It also increases the learners’ motivation to take the course leading to higher completion rates.
- Once these are in line, you can expect to see an increased application of the learning and this in turn will lead to the desired impact on the learners and business.
- By evaluating your online courses for learnability, you can predict its impact as well as identify measures to increase it. This will go a long way in achieving a positive ROI on your training spend.
NOTE: Validation of the learning experience is important when it comes to determining ROI on training. As suggested by Keri Bennington of UNC Kenan-Flager Business School in a report, “ROI measures should be related to performance after the Learning and Development experience and, according to some, tied to a dollar figure. For example, time saved or increased output (or both) as a result of improved performance following participation in a development program can then be compared to a dollar figure.”
Can Learnability Be Measured?
Yes, learnability or learning effectiveness of online courses can be measured but there are several challenges associated with this exercise:
- There is a maze of parameters that influence it.
- Adding to the volume of parameters that can influence learnability is the second challenge that not all parameters are measurable.
- The significance of each parameter changes with each training type.
- Finally, the feedback can be objective (easy to implement) as well as subjective (often difficult to fix).
How Do We At EI Design Measure Learnability?
In spite of the fact that measurement of learnability is a tall order, at EI Design, we offer a unique framework that you can use to predict, measure, and validate the Learnability of online courses (for new or legacy courses).
Our Approach: We began our exercise by looking at various building blocks of a course and how learners view them or interact with them. Using these cues, we created 6 metrics which map to each of these to help us evaluate the learnability or learning effectiveness as shown here:
- Metrics 1: Interface Design.
- Metrics 2: Course Information and Instructions (navigation).
- Metrics 3: Content Structuring (to meet the required level of cognition).
- Metrics 4: Task Performance (to interact and learn).
- Metrics 5: Usability (overall experience).
- Metrics 6: Feedback on Design Elements.
Measurement: Our framework also allows you to dynamically assign the criticality based on the nature of the course, content, or learners. Then the aggregated Learnability Index is generated.
Take a look at an example of the Learnability Index of an existing course and how we upped it by enhancing it through our Learnability guidelines.
Validation: This can be validated by the learners and the feedback can be used to enhance the learning design further, if required.
Take a look at some reports that show how the course fared with learners (validation of the predictive learnability).
How Can You Use Our Framework To Maximize ROI On Training?
You can use our framework to predict the learnability of the courses you are developing. You can also use it to evaluate the Learnability of your legacy courses and then uplift it. Through our learnability framework, you will be able to:
- Get access to a predictive methodology to build learnability in your new courses.
- Use it to evaluate the learnability of your existing courses: You can diagnose issues that reduce the learning effectiveness and hamper performance and step up the Learnability Index to the required levels.
- Validate your assumptions with learners: You can check if your courses pass the litmus test by validating the impact of the courses from the folks who matter – the learners!
- Gain crucial insights for future learning interventions that can enhance the effectiveness of the training curriculum and performance for the learners.
- Improve learning retention and meet the prescribed cognition level.
With these, you will be able to have greater clarity on demonstrable gains for learners as well as the business and maximize your ROI on training.
How Can Our Learnability Framework Help Learning And Development Teams?
Learning and Development teams can use our learnability framework to:
- Predict, measure, and continuously enhance the learning effectiveness of trainings.
- Have a robust framework in place to constantly improve learning and performance.
- Have a data driven approach to address user experience related issues in learning (actionable learner feedback).
- Demonstrate a positive impact on ROI on training.
You can take a look at this video of our Webinar on How To Predict, Measure, And Validate The Learnability Of Online Courses for more in-depth insights:
I hope this article provides cues on how you can use our framework to measure and enhance the learnability or learning effectiveness of your online courses, how you can validate the predictive Learnability with your learners, and see if you are on target or what further measures will help you hit the bull’s-eye. If you have any queries on this, do contact me at email@example.com.
The post How Can You Measure The Learning Effectiveness Of Online Courses And Create A Positive ROI? appeared first on eLearning.
Recent themes to my work has been the nature of how, and what, we share. I wanted to reflect a little on my own ‘sharing’ here, and try and split the sharing from social media, if possible.
There are obvious easy ways to write about my sharing (per platform) but also I want to think about the why? So, why? I can’t deny one major factor is to reach a wider audience than just those I immediately work with on a day-to-day basis. By sharing my ideas or thoughts or projects or interests I’m obviously creating and managing my brand (me) but I also hope to be of some influence to others working in the same sphere as me.
Obviously, there’s this blog. Back when I started writing here I used to write about the day-to-day tasks and tools I used. The last few years has seen me change direction, mainly due to possible conflict of interest with where I’ve worked and the need to keep some commercially sensitive things private. I’ve developed it more recently to be about the why I do things and how I develop myself or my work, my attitude to learning and technology and how use them both. I write here to share experiences and ideas, books I’ve read and reviewed, books I’ve written and curated, etc. I write to have a brain-dump, drop ideas or stress, I write to see what you all think … What do you think?
I share my blog posts on Twitter so I can reach more people, and engage the wider field of learning technology. It reaches more people this way and I can engage in conversations beyond my own understanding, therefore helping me widen my appreciation and knowledge for my work. My Twitter activity involved my blog but also other aspects of my work, and sometimes home life too, but mainly my work. I save tweets to my ‘like’ (although I still don’t use it as “ooh, I like this tweet” but rather as a save feature to go back and read or reply to something after the fact) and add people to my lists. Twitter is my go-to place all day and pretty much everyday. My network or followers and those I follow grows and changes all the time, therefore my exposure to new ideas or tools does too.
I’ve been and gone on LinkedIn before and, at the moment, am back and engaging here again. The audience is different to Twitter, less chat and more ‘sharing’. Perhaps it’s because it’s viewed as more of my online CV, or perhaps because there’s different mechanisms for comments, etc. I don’t know, but LinkedIn is an acquired taste. Currently I like it, but I take it each day at a time with all my sharing.
I use both these platforms more for searching and reading different themes, less so for my own sharing, but I appreciate the work others are putting into their sharing activities here. For some these are important channels for sharing their work or ideas, and that’s fine.
This is, after all, about what works for you or me. There is no rule that will work for everyone, we are each individual and have different perspectives and needs and likes, and this is what we each bring to the wider community. THIS is what makes our personal (learning) networks so vibrant and interesting. This is why I love to share .. I take so much from the community on all these platforms, I want to add something back in the hope (need?) that it makes a difference to someone like something I’ve just taken. Isn’t sharing great!
Sharing: why and how. It may be a tweet, a blog post, an idea, a photo. This is sharing. For me.
Click To Tweet
I suppose I ought to add this here too although I’m still thinking of dumping my account here. Facebook has only ever been about family and friends. I dabbled in having a work-type account but realised the audience was the same, but smaller, than my twitter audience so decided it wasn’t worth the extra time to manage and curate it.
Above all I try and keep my sharing professional. I have interests that creep into my sharing every now and then, mainly on Twitter. Yes, I have two kittens, I drink tea not coffee, I love Lego. But it’s still shared with a view to what my audience may be interested in. I don’t follow celebrities, for the most part, as I’m just not that interested in what they’re doing. Unless they are the kind of people I think are celebrities like Steve Wheeler, Stephen Heppell, Sue Beckingham, Amy Burvall, Maren Deepwell, et al (see the people I’ve been lucky to work with on my books, these are the celebrities in my world!). Then, of course, I’m a groupie and will follow them anywhere I can.
What about you? What is your strategy (if you have one) for sharing?
When you buy a new car or buy a new TV you go to a showroom who deals in the car or TV, either it’s an official retailer for the item or it has a reputation you trust. Well, we used to at any rate. It seems these days, and I’m equally guilty of this, we go online and find the cheapest version. There, that’ll do. Even if we use a ‘reputable’ website we may find ourselves buying the £5 USB cable made by a company we’ve never heard of instead of the £15 cable from one we have. “It’s OK, it’s from Amazon, it’ll be OK.” (Other online retailers exist, try them out too sometime!)
Is it the same with our learning? When choosing a college or university we look at a lot of things about it, not only the details of the course and individual topics within it but things like accommodation, proximity to the town or shops, on-campus events, clubs, sports facilities, reviews from previous students, etc. I don’t remember even thinking about who would teach me my degree, I looked into everything BUT the teaching staff. Is this wrong?
It seems different when looking at the different MOOCs on offer, I find myself looking at the course team as much as the course syllabus itself. A MOOC on Shakespeare? Why, yes please … but who wrote it and who’s delivering it? Ahh, a ‘renowned Shakespearean academic’ in Professor Bate and it’s been developed by the University of Warwick (ranked consistently in the UKs’ top 10 universities). That kind of makes up my mind .. even though the course page doesn’t say much about the course contents, other than the promo video.
I’ve worked on a number of MOOCs and online courses as well as blended and campus/classroom based courses. There are many differences in what I/we do depending on the audience and delivery method, but the courses that have an element of face-to-face contact doesn’t really need the teacher introduced as part of the designed materials. This is, or should be, done in person. Often the first lecture or contact point with the students will be an introduction made by the teacher on who they are, what their background is and why they are the one who should give the course. Often courses are taught by a team, sometimes led by the senior academic and supported by either junior academics or PhD students. Are they also included in the list of authors or facilitators? They have equal right to be there, especially if the learners have more contact with them (in person or online) than the ‘lead’. This is content given to the students and often not part of the slides they can download for each lecture. There may be some info on the VLE, but is it really enough to showcase the breadth of knowledge behind the course and it’s creation?
Authenticity and credibility in online learning
Click To Tweet
For these courses with contact time it seems it doesn’t really matter that this stuff isn’t written in to the course itself. For online courses of any nature or audience it is imperative this information is front and centre. If you can highlight prior to the course (especially for MOOCs) the credibility of the authoring and teaching team it will enhance the authenticity of the course itself.
This is often overlooked in some online courses and is why I insist on having this information front and centre in the courses I work on. This gives the course and the whole course team the credibility to be the ones to deliver and facilitate the course, and it gives the content and materials the authenticity needed to demonstrate to the learners that this team has experience and background to be the best team to lead it.
There are so many options and ways to learn online, sometimes the number of courses on a similar subject exist. So, which one will you choose? The one that looks nice? The cheapest one, or the one that has been developed and delivered by the best team possible, therefore giving you the best possible learning experience?
Yeah, me too.
Irrespective of the assessment criteria or type of assessment used at the end of a course, we champion the achievement and base ‘learning’ on the final grade. For right or wrong, this is the state of schools, colleges, universities and MOOCs .. a pass grade equals success, not necessarily a quality learning experience.
When a course or programme goes through review, either for changes or it’s new, the conversation will always turn to the assessment. Is the assessment indicative of the course and the course aims? is the assessment type appropriate to the delivery method? Is it a straight forward 100% exam or mix of coursework and exam? If coursework is included in the final grade can the documentation be deliberately vague to allow flexibility in how and what the coursework is (project, group, video, report, tests, etc.)?
All well and good. Well, not really good but you know what I mean. But which is more important … the learning and knowledge acquisition or the assessment grade? Most of us would say the former, the learning and being able to retain an apply the knowledge. But education requires a certificate that shows more than just attendance. It requires to show the standard to which the holder has worked and can work. Without a score or grade (80% or 2:1) there is no meaning to the achievement for an employer to gauge the ability of the certificate holder.
Is there an answer? Could the achievement be recreated and reassessed to accommodate more meaningful information pertaining to the individual and how they ‘work’ and ‘learn’, and what kind of person they are? This is usually a reference on an application, but wouldn’t it be good if this had more emphasis on an application than a grade? Making something that can’t be gamed would be the hard part, anyone can find someone to write a glowing report and review, just like you can find online examples of buying the academic paper or script.
What comes first when planning your course? The learning, or the achievement?
Click To Tweet
You could argue we’ve already got an achievement for learning that goes beyond the assessment with Open Badges. If so, why haven’t we seen them used more widely? What is holding us, or rather the employers, so tight to the grade result and not the achievement? A few years ago there was lots of talk about the scope and strength of Open Badges. Surely that hasn’t gone away. I hope it hasn’t gone away.
Following on from my two previous posts which cover my thought process about familiarity in learning design and how distractions can affect both our work and learn environments, I wanted to write about instruction. Instructions we give as well as those we receive.
When you start something new, at work or at home, do you read or follow the instructions? If it’s a new cabinet (Ikea anyone?) or piece of furniture, you’ll probably follow the instructions quite clearly. I know I do. Same with Lego? Yes, me too, although I do like to mess with Lego and see what weird-yet-satisfyingly-symmetrical construction me and my boys can come up with.
Even with new technologies I usually like to read a little of the instructions to get me started, at least to the point where I know how to charge it and when it’s ready to use. These days most modern companies provide some excellent get-you-started instructions with their products; enough for the likes of me who just want to get started, more detailed versions online for those who want to delve deeper.
When we have a new person join our team we often find ourselves working through an induction programme, introducing them to key people they need to know (IT, HR, estates, etc.) and then spend time showing the ropes in the VLE, LMS, online HR system, file server, phones, etc. See, we take care of our own and make sure they have enough to get started, then step back and give them room to find their feet, all the time being a careful parent ready to step in and answer any questions.
When was the last time you read the induction materials, and I mean read not just link-and-fact checked them?
Click To Tweet
But what of our online learners? We have probably developed a full-on induction or on-boarding process for them. We’ve probably not revisited it for a couple of years as well, but have we done too much or too little for our distant and online clients (yes clients … there is a payment transaction going on, even in MOOCs these days). When was the last time you read the induction materials, and I mean read not just link-and-fact checked them? What about getting them tested by someone new to the programme?
In my time I’ve seen old induction instructions that are out of date, yet still valid because links work and the platform hasn’t changed. That shouldn’t mean we can let them be. In the last few years I’ve seen major changes in how different learning platforms are used. What they do are still mostly the same, but how we use them is constantly, or should be constantly changing. Therefore the induction programme should also be changed to reflect that too. Again, it’s not just about the click-this and click-that instructions, but the information around why we are asking students to do something that needs checking.
What do you do then? Do you keep referring back to those initial instructions throughout the courses, reminding the learner about the tools or help available, or do you rely on them remembering it and, hopefully, reviewing the induction programme? When you use a different or new tool with the learners do you write some guide for them, on both the how it works and why you’re using it? I bet you do, but do you go and add it to the induction programme for the next cohort of learners? You should.
For me the process of inducting learners to your organisation or platform never ends, or rather ends when they complete the course or programme and ‘graduate’. If they’re studying a three year degree it’s an easy bet that the tools and how you use them will change (again, SHOULD change!) over the lifespan of their studies. If your learners are only with you for a short while, a matter of weeks, then there’s still no reason to not keep them informed with either email communications or VLE announcements when they’re going to encounter something new as part of their learning. If they’re used to MCQs week on week, then you start using discussion boards, then a reminder about what they are, what you expect from the learning in the discussion, and how to use them is a good way to introduce the activity.
My last post was all about the ways in which familiarity can bring a sense of consistency to not only delivering online learning content, but also designing and developing it. This time I want to look at the way in which distraction can prevent the well designed and impeccably delivered learning materials.
Whether you work in an office or, like me at the moment, at home, we all look to manage our working environment. Working in an office, small or large, will often mean managing how people interact with you when you’re trying to concentrate, preventing the creep of office chatter or ‘work’ noise.
Personally, if I want to focus I use music and, when in an environment with others, headphones. Depending on the quality of your headphones you may find you provide more distraction for co-workers as your headphones leak sound. A while ago I bought a set of AKG Y50BT headphones … not noise-cancelling, I can’t afford a decent set of those, but these on-ear ‘phones are really good at reducing noise ingress. The only downside of these is that my ears get hot.
Working from home means I don’t have to wear the ‘phones unless I have to. I can have music playing in my study or, if I work in the kitchen or conservatory, from the laptop or Amazon Echo (yes, I got one). Obviously working from home is great, but that’s once the kids have gone to school. Come mid-late afternoon, they’re back. I don’t want to impact my home life so back I go to the ‘study’ (smallest bedroom, until I get the garden office built!), close the door and try and let the house carry on as normal.
You can’t really do that in a shared office either, can you? Some people I’ve spoken with while writing this have a dedicated ‘quiet’ room where one or more can go to work in ‘silence’, or rather without interruption. Now that’s a good idea!
Making sure those around us know when we’re available to chat or when we want to focus and not be disturbed. I’ve worked with people who’ve had different techniques for this; one had a sign they’d hang on the back of their chair when they’re not to be disturbed. One manager used to wear a hat when he wanted to focus and be left alone. Another used to put his headphones on.
How do you work or learn in a distracted environment?
Click To Tweet
All this is great for me and how I set my working environment up, but what about for those of us who are also learners? What about for learners who are not as experienced or comfortable in sitting down at a computer or computing device to ‘learn’? What does distraction mean to them?
Firstly we, the learning design/development community, have to recognise that no matter how hard we try we will never be the most important thing in the learner’s life – family, friends, work, fitness, health, etc. will always exert a pull on their time and commitment. We have to respect that and enable the learning to fit around their existing lives. Even those learners who are fully committed to the course(s) and spend as long as possible, or even longer than we recommend, need to be able to learn when it suits them. Learners will often be doing it after a full day of work, family, etc. No everyone is at their mental best at this time either, so we need to make the learning as ‘easy’ to access as possible (see my post ‘familiarity‘ for more on this).
I’ve often seen, in online courses, a timer at the beginning of a section – “Time to complete: 0.5hrs”. There are often timescales ‘imposed’ on the course itself, most MOOCs will say something like “5 hrs per week“. This has often raised questions about whether we ought to be this prescriptive about how long the ‘learning’ should take, after all people read or learn at different rates.
When you create your online course, do you help the learner by explaining how they can set up their time and environment to prevent distraction? Would the (novice) learner benefit from our experience if we told them “find somewhere quiet, turn the TV off, close down social media tabs, don’t look at your phone notifications, switch the phone to silent, etc.”. But what about those people who focus more when there is noise (not distraction, but noise .. music, family, TV, cafe, etc.). I know it works for me. Sometimes.
Distraction doesn’t mean isolation or quiet. Something distracting to me might be essential for you to focus and relax. The thing here, for me, is that we have had the luxury to find out these things for ourselves. For our learners, what can we do to help them find their ideal ‘learning environment’? If your course has an on-boarding process or initiation stage then use it to highlight what is expected of them, how much time (and how often) they ought to spend on the course and it’s readings.
Provide as much information for the learners, without overloading them, to make the decisions for themselves. And try it out.
Over the years and role changes I’ve used a variety of different VLEs. From Blackboard to FutureLearn, and from custom in-house developed VLE to customised large-scale MOOC platform. So, how important is familiarity when working, designing and developing on these platforms?
Firstly, are we talking about the familiarity I need to navigate the multitude of features and processes to get the course built and delivered? Or do we mean the familiarity the learner needs in order to have a smooth and tangible learning experience, whether they sit down and structure their learning or dip in as and when they can? Let’s try and deal with both.
- For me: If you’re new to the platform it’s good to write notes to yourself as you do something new, work out how a feature works, etc. This is also a great resource for you or the rest of the team to open discussion around the how and why of particular approach to presenting a learning resource. Keep ideas, plans, design/colour schemes, times, asset library, etc. all in one place for easy reference.
- For the learner: Accept that the learner may not have read your carefully scripted course page or expensive course promo video and repeat it at the beginning of the course. The odds are that you put a lot of effort into that content so make sure it’s of use at the start of the course. It will need to be modified, you don’t need the marketing/promotional terminology here, so make sure it reads like the rest of the course (the ‘voice’ of the learning). Carry this approach to the whole course, not just the start: explain why you’ve included a video to watch and what the learner should think about while they watch it. Explain the structure of the course and what it means for their journey, and how the journey ends. And what happens after that.
Structure and navigation
- For me: A new platform will mean a lot of different, well, everything! Who hosts, manages or supports the platform? Who are they, where are they, when are they available? Make them your new BFF and ask for help as well as providing a fresh pair of eyes and offer feedback from your own experience on other platforms to see if you can provide efficiencies or development to improve. Always ask questions and always explain why, as well as showing them your results.
- For the learner: A consistent structure and navigation to the course will help the learner feel more comfortable and relaxed, therefore are more likely to retain the knowledge you’re presenting them with. As with the previous item, explain how the structure works, explain how to use the navigation, and above all keep the consistency of design that you’ve worked hard to develop. If you use colour of font size as a code of activity or resource identification, use it every time (you’ d be surprised how often I’ve seen inconsistencies, usually across courses rather than within the same course).
Example: FutureLearn navigation, Warwick’s ‘Leadership for healthcare improvement and innovation’.
- For me: Personally I hate templates or a forced way of working, but the method and structure they offer are hard to ignore. There’s a reason why templates work and that, as I mentioned previously, provides a consistency across courses, programmes, and team members. if you’re working in isolation, then the template probably doesn’t make sense to you as you already know what you’re doing. If you working a part of a larger team then the template provides the working structure you all need to adhere to to get that consistency I talked about.
- For the learner: The template should not be something the learner ever really notices. The template is there to provide a consistent learning experience for the learner. If it works they wont notice it. If it fails they’ll complain of not understanding what they should be doing, or when, or how, or why. The template will provide familiarity and structure.
Text and images
- For me: Nothing bores me more than a course full of pages and pages of text, no visual cue at all as to what’s happening. If nothing else a well placed image showing the general theme or topic helps bring the page to life. While some subjects are clearly more visual than others, there’s no excuse for not using some Creative Commons or licensed images, a YouTube video also explaining the subject, concept, interview with an expert, educator, practitioner, etc. While we try and accommodate as many styles of presenting learning materials, and those materials often reach us from the educator in text form, we would not be doing our job if we didn’t try and find a visual solution to break the text blocks up, even if it’s only a different way of presenting the text.
- For the learner: if the learner wanted to read a textbook to gain the knowledge and qualification from the course, they’d that. Often what one learner likes is not what another likes. While one person can read book after book and retain the knowledge easily many cant, me being one of them. The inclusion of different sorts of activities helps, but so do different approaches to presenting the learning materials: image, charts, photos, infographics, video interviews, to-camera teaching presentations, video case studies, high-profile documentaries (check the ownership and originality if you’re using these from YouTube), etc. There’s always a way to bring something visual to the course.
Example: Documentary – DHL International Supply Chain, loaded to YouTube by DHL.
… now make an activity out of it, introduce some questions that the video can help with but requires the learner to go further afield to find answers and more resources for. Make the image or video part of the learning, not the learning itself.
- For me: If the whole team uses the term ‘page’ or ‘step’ to indicate a different element of a learning package, then be sure you all use that term. By using a variety of different terms to mean the same thing you will forever be translating instructions from one source to another for different things. Something will always get lost in the translation, mistakes will be made no matter how hard you try, and there will be more work down the line when you have to unravel the mess. Be sure the terms you use within the team are consistent (that word again) and appropriate. If you work with a new educator who’s used to different terms and ways of working then open the dialogue and work out what’s best – do they change to accommodate you and your team, or do you change your processes to accommodate them? Decide early on and stick to it!
- For the learner: No one wants to read a course that is heavy in jargon, acronyms, complicated academic terminology or badly presented materials. No one. Even if you’re writing for advanced Masters level students you should still use appropriate language, explain an acronym, and avoid jargon. You obviously don’t want to dumb the language down so it sounds like you’re being condescending to them, but there is a level that is acceptable. Find it, stick to it, and test it!
Familiarity in learning has always been about consistency – consistency in the approach to design and present the materials, consistency in language appropriate to the level of the course and the intended audience, consistency in quality of photos or images or videos, consistency in length of pages or steps. By being consistent in what you do and how you do it your course will also offer a consistency the learner will become accustomed to, which will bread familiarity and comfort with. From here it will be easier to follow the learning and complete the course.
Today, most organizations use eLearning as a significant part of their training delivery. As traditional eLearning moves towards mobile learning or mLearning and provides learners the flexibility to learn on the device of their choice (notably tablets and smartphones), the eLearning adoption is gaining further momentum. eLearning and mobile learning provide several benefits to organizations. However, the focus is now shifting to determining its impact and the Return On Investment or ROI of online training.
Measuring The ROI Of Online Training
In this article, I will begin with a quick summary of the benefits that eLearning offers, what ROI is, and how you can measure it. I will also outline the ROI methodology we use.
What Are The Advantages Of eLearning?
I am quoting extensively from my earlier article Return Of Investment (ROI): Are You In?. This article had originally appeared in CrossKnowledge’s Learning Wire Blog. It also outlines the measures to maximize the ROI.
Over the last two decades, most organizations have made investments in eLearning primarily for the following benefits:
- Anytime, anywhere access (on demand availability).
- Self-paced, interactive, and more engaging learning (learner perspective).
- Less disruptive delivery (in contrast to ILT).
- Cost-effective (particularly when reaching out to a large audience).
- Consistency of message and easy updating of content.
- Easy tracking of learner progress and completion (business perspective).
While the eLearning advantages are well accepted, increasingly organizations are seeking ways and means to determine its impact on learners as well as on business. Let’s see what Return Of Investment (ROI) is and how you can assess if your eLearning or online training initiatives are generating the required ROI.
What Is ROI?
ROI is the return on investment that an organization makes (ROI = Gain or Return/Cost). It can be determined through two factors namely the Investment made (or cost incurred) and Value/Gain accrued (or return).
A successful eLearning initiative should be able to demonstrate gains that are more than the investment.
How To Determine Costs And Assess Returns?
Costs are fairly easy to define and would normally include the cost of eLearning course development as well as associated costs of team members (including teams that are associated with the development process and Subject Matter Experts).
Typically, there would also be associated costs of the supporting delivery (Learning Management System, Administrative cost of managing the initiative, and other related infrastructure required for delivery).
Determining the “value” or “gain” is far more tricky. We nail this by looking at the gains for the organization as well as for the learners.
- Organizational perspective.
Let’s begin by re-looking at the gains most organizations seek when they adopt eLearning and see how many of these translate to reduction in costs and hence improvement in returns.
- Less disruptive delivery.
This translates to man-days available now to the organization that would have been allocated to travel and training in the ILT mode.
- Reduced travel costs.
These can be determined easily.
- No associated costs for trainers.
These can be identified easily.
- Less disruptive delivery.
- Learners’ perspective.
Next, let’s take a look at the gains that accrue on account of effective eLearning course designs:
- Immersive and engaging learning translates to better assimilation. This in turn leads to proficiency gain and a tangible increase in productivity.
- More learners across the organization can be trained in lesser time (while they get the flexibility to learn at their own pace).
- Coupled with tracking, the eLearning initiatives can be scheduled and completed faster as compared to ILT sessions.
What ROI Methodology Can Be Used?
Most of us are familiar with Kirkpatrick’s model of evaluation. In today’s context, adding Phillips’ ROI calculation as the fifth level makes this framework even more useful and relevant. By using Level IV evaluation data, we can convert the results into monetary value. Then we can easily compare them against the cost of the eLearning program and determine the ROI.
To give you a sense of how it can be practically used, let me summarize the approaches we typically adopt:
- Level 1:Reaction is measured by taking feedback from learners. We have used online surveys in the past but now we add features of “Like the course” and “Recommend the course” options within our eLearning course framework.
- Level 2: Learning can be easily measured through scoring patterns in the end of course assessments.
- Level 3: Behavioral changes are certainly more difficult to assess. We use a combination of techniques to assess how much of the newly acquired learning is being applied on the job. This could be measured through improvements in efficiency or doing the same task with a new approach.
- Level 4: Business impact is generally measured through productivity gain, impact on quality measures through reduction in re-works, getting higher number of work assets first time right, and so on.
- Level 5: ROI is normally calculated by converting the business impact gains (as shown in level 4) to a monetary value.
I hope this article was useful in understanding the ROI definition and more significantly, what ROI methodology will enable you to measure the ROI of online training. At EI Design, we do workshops that can enable you to adapt the standard ROI methodology to your organization. Do reach out to me if you need further details.