Agile Is Important For eLearning Development Today, Isn’t It?

Agile has been here in this industry for quite some time but it's fairly new in eLearning development. In this article, we will see how the Agile methodology is better in all aspects as compared to the traditional ADDIE methodology. This post was first published on eLearning Industry.

Agile production – online learning needs to get its skates on

Der Artikel ist „hingeworfen“: Ja, die Entwicklung von Online-Lernprogrammen, soweit sie dem klassischen Projektmanagement folgt (ADDIE), ist oft aufwändig und teuer. Und, ja, es sollte schneller gehen. Einige Stichworte, wie das aussehen könnte, erwähnt Donald Clark: „1. Assess and edit content, 2. Avoid SME iterations, 3. AI generated online learning, 4. Review actual content (online).“

Doch die Argumente werden überdeckt durch einfache Rechnungen, die „Cost Savings“ ausweisen, was mich an alte „E-Learning ist preiswerter als Präsenztraining“-Diskussionen erinnert. Und Donald Clark reduziert E-Learning auf die Inhaltsentwicklung und -vermittlung. Aber das war auch schon bei ADDIE das Problem.

„Business managers are often surprised when their request for online training will take many months not days, at 15-25k per hour of learning – oh and you’ll not be able to evaluate much, as the data is unlikely to tell you much (we have a thing called SCORM).“
Donald Clark, Donald Clark Plan B, 20. August 2018

Bildquelle: Goh Rhy Yan (Unsplash)

Evaluating the development and impact of an eLearning platform: the case of the Switzerland Travel Academy

Eine E-Learning-Fallstudie aus der Tourismus-Branche, eingebettet in aktuellen Veränderungen („eTourism“) und in daraus resultierende Notwendigkeit der Travel Agents, auf dem Laufenden zu bleiben. Vor diesem Hintergrund haben die Autoren die Einführung und Nutzung eines Kursangebots untersucht. Und zwar mit Hilfe eines neuen Evaluationskonzepts, eine Verbindung aus ADDIE und Kirkpatrick, das sie selbst entworfen haben und das aus ihrer Sicht den dynamischen, zirkulären Entwicklungsschritten eines E-Learning-Projekts besser entspricht. Es bleiben Fragen, zum Beispiel, wie man das Konzept in Projekten unterschiedlicher Größenordnung umsetzt.

„A new evaluation framework was proposed combining and adapting the ADDIE model with Kirkpatrick’s model, according to which evaluation of an eLearning platform should occur on two levels: internally and externally.“
Elide Garbani-Nerini, Nadzeya Kalbaska und Lorenzo Cantoni, in: In Stangl, B., Pesonen, J. (Eds.). Information and Communication Technologies in Tourism. Wien, New York: Springer 2018, S. 450-462 (via ResearchGate)

Bildquelle: Tommaso Pecchioli (Unsplash)

Designing Instruction For Authentic Learning

An applied approach to Instructional Design where the Backwards Design model is integrated into the design phase of ADDIE to create Authentic Learning. Increase the transfer of knowledge effectiveness rate by allowing students to become active learners instead of passive receivers of information.

This post was first published on eLearning Industry.

A Practical Check-In With 4 Of The Most Popular Instructional Design Models

When choosing which Instructional Design model to apply in a learning course, it's often helpful to determine which methodologies are most applicable to the needs of your learners. The following article highlights 4 of the most popular Instructional Design models and how best to utilize them in training.

This post was first published on eLearning Industry.

ADDIE Vs. Backward Design: Which One, When, And Why?

Opinion is divided between which of the two popular Instructional Design models is ideal for developing an online course. Both have their pros and cons, and it is quite likely that one of the two is better than the other in a specific situation. This articles explores the ADDIE vs. Backward Design dilemma.

This post was first published on eLearning Industry.

Getting To Know ADDIE: Part 5 – Evaluation

Getting To Know ADDIE: Evaluation 

We started our journey by studying the target audience, formulating the learning goals, and performing technical analysis. We then proceeded to choosing the format of the course and developing the educational strategy. The next step was creating a prototype and getting busy developing the course itself. In the previous installment we spoke about preparing the teachers, learners, and the environment.

Let us take a look at the individual steps comprising the final stage of the ADDIE framework, Evaluation.

Formative Evaluation

Formative evaluation runs parallel to the learning process and is meant to evaluate the quality of the learning materials and their reception by the students. Formative evaluation can be separated into the following categories:

  1. One-to-One Evaluation.
  2. Small Group Evaluation.
  3. Field Trial.

1. One-to-One Evaluation. 

Imagine that you are conducting a training teaching medical students to use an X-ray machine. You play a video explaining the basics of operating the device. One-to-one evaluation involves you gauging the effectiveness of the video taking into account the age and skillset of the target audience. It is necessary to evaluate the following aspects of the video:

  • Clarity.
    Was the main idea of the video well understood?
  • Usefulness.
    Did the video help in achieving the goals that were set?
  • Relevancy.
    Can the video be used to good practical effect in regard to the place it takes in the curriculum and the material being studied in parallel?

It is important to keep evaluation questions clear, concise, and to the point.

2. Small Group Evaluation. 

This type of evaluation is meant to understand how well do the activities included in the course work in a group setting. Form a small group, preferably consisting of representatives of the various subgroups that make up the student body that is the course’s target audience.

When doing the small group evaluation, you should ask the following questions:

  • Was learning fun and engaging?
  • Do you understand the goal of the course?
  • Do you feel that the teaching materials were relevant to the course’s goals?
  • Was there enough practical exercises?
  • Do you feel that the tests checked the knowledge that is relevant to the course’s goals?
  • Did you receive enough feedback?

3. Field Trial. 

Once the small group evaluation is complete, it is recommended to do one more trial, this time under conditions as similar as possible to the actual environments that will be used in the learning process. This “field trial” will help you evaluate the efficacy of learning in a specific environment and under specific conditions.

Summative Evaluation

The main goal of summative evaluation is to prove, once the course is finished, that the performed training had a positive effect. For that, we use the Donald Kirkpatrick training evaluation model, which has long ago become the standard for evaluating the effectiveness of training.

Summative evaluation helps us find answers to the following questions:

  • Is continuing the learning program worthwhile?
  • How can the learning program be improved?
  • How can the effectiveness of training be improved?
  • How to make sure that the training corresponds to the learning strategy?
  • How can the value of the training be demonstrated?

Donald Kirkpatrick divided his model into 4 levels:

  • Level 1: Reaction.
  • Level 2: Learning.
  • Level 3: Behavior.
  • Level 4: Results.

Let us examine them in more detail.

Level 1: Reaction. 

First thing to be analyzed once the training is complete is how the students reacted to the course and the instructor (if applicable). Usually, the data is obtained with the help of a questionnaire containing a number of statements about the course that students need to rate from one to five, depending on whether they agree or disagree with a particular statement. These questionnaires are usually called “Smile sheets”.

Level 2: Learning. 

On this level we test the knowledge and skills acquired during the training. This evaluation can take place right after the training is concluded, or after some time has passed. Tests and surveys are used to evaluate the training results and to assign to them a measurable value. Another option is to have the learners who have completed the training to train other employees, conduct a presentation for colleagues from different branches, or help in adapting and training new hires. Besides helping internalize the acquired knowledge, this has the additional benefit of speeding up the knowledge transfer process within the company.

Level 3: Behavior. 

According to Donald Kirkpatrick, this evaluation level is the hardest to implement. It involves analyzing the changes in the learners’ behavior as a result of participating in training, and also understanding how well and how often the acquired knowledge and skills are being employed in the workplace. In most cases, the latter reflects the relevancy of the knowledge delivered via the training, as well as the motivation to use the newly acquired knowledge the training may have imparted. For this level, the best evaluation tools are observing the learners’ behavior in the workplace and focus group testing.

Level 4: Results. 

Finally, the fourth level deals with analyzing the financial results of the conducted training. Namely, whether the delivered results matched up to the goals that had been set, whether the company’s financial indicators (sales volume, decrease in expenses, total profit, etc.) improved as the result of the conducted training, and so on. Other factors that can be taken into account include increase in productivity, improvements in quality, decrease in workplace accidents, increase in the number of sales, and decrease in turnover.

For this reason it is important to determine the factors that will be taken into account to determine the effectiveness of the training beforehand, and to measure them before and after the training is conducted.

Evaluation on this level is both difficult and expensive. To obtain results that are as accurate as possible, it is recommended to use one of the following methods:

  • Using a control group (consisting of employees that have not participated in the training).
  • Performing the evaluation after some time has passed since the completion of the training, so that the results would be more pronounced.
  • Performing the evaluation both before and after conducting the training.
  • Conducting the evaluation a number of times during the course of the training.

Is It All Worth It?

Carrying out evaluation following the Kirkpatrick model is time-consuming and not always cheap, but it provides valuable insight into whether it is worthwhile to continue a training program and whether it will deliver the expected results and earn back the money spent on it, so that you can make the correct choice. In addition, this model helps gauge the effectiveness of the training department, and its alignment with the organization’s goals. Some companies neglect to perform third and fourth level evaluation, contenting themselves with analysis on the basic reaction level. However, this denies them the benefits of a clear understanding of the effectiveness and usefulness of the conducted training. Summative evaluation helps in getting on the right track, even if the conducted training is found to have been of substandard quality. It enables you to correct past mistakes and improve the training, so that it may better benefit the next group of students.

Evaluation As The Final ADDIE Stage 

Despite the fact that evaluation is the final stage of the ADDIE methodology, it should be considered not as a conclusion of a long process, but as a starting point for the next iteration of the ADDIE cycle. Diligent evaluation will enable you to review and improve the educational program. Instructional Design is an iterative process, and evaluation should be carried out on a regular basis. Besides, keep in mind that to achieve best results, it is recommended to keep an eye on the quality of the course under construction throughout the development process according to the ADDIE framework, and not only at its conclusion.

Have fun building, and best of luck to you!

This post was first published on eLearning Industry.

Getting To Know ADDIE: Part 4 – Implementation

Getting To Know ADDIE: Implementation

Once the Development stage (the one we spoke about in the previous installment) is finished, it is time to proceed to the next one - namely, Implementation. During this stage, the materials created during development are introduced to the target audience and the learning process starts.

The application of materials can take different forms:

  1. Learners acquire knowledge autonomously by taking an electronic course and trying to understand the material with no outside help.
  2. Knowledge transfer is facilitated by an instructor or a group of instructors using the developed materials as a basis for teaching. They deliver the information to learners and make sure that the main concepts of the course are well understood.
  3. Learners study a part of the course autonomously, while the rest is explained by an instructor, who also controls the acquisition and retention of knowledge from the parts of the course the learners studied with no assistance.

The data gathered during the Analysis stage can help you decide what form would work best in your circumstances. Take into account the type of knowledge the course imparts (theoretical knowledge or practical skills) and the characteristics of the target audience (computer proficiency, occupation, motivation level, discipline, and age), as well as the existing infrastructure of the company where training is being carried out.

Depending on the chosen format, the Implementation stage will likely include the following main steps to a greater or lesser degree:

  1. Training the instructors.
  2. Preparing the learners.
  3. Preparing the environment.

Let us take a more in-depth look at all three steps.

Training the instructors

It is not unusual for a person who was engaged with the development of an electronic course to be involved in its implementation and the delivery of knowledge to learners. However, it is by no means rare for the instructors to be uninvolved in the creation of the course he or she has to teach. In this event, it is necessary to adequately train the instructor and make sure that he or she has all the necessary information about the course before the learning process begins.

In general, the instructor has to have good understanding of the following:

  1. The goal or goals of the course.
  2. The course’s activities.
  3. The course’s media content.
  4. The course’s tests.

One implication of this is that all necessary materials must be prepared before the first lesson starts, so that the instructor will have the time to review them, absorb the mets-information, and obtain an adequate mastery of the information presented in the course. Sometimes, it becomes necessary to teach dozens of instructors at a time, who will then go on to teach the course in different branches or organizations. For this reason, the amount of time necessary to adequately teach an instructor or instructors can vary greatly depending on the circumstances.

Preparing the learners

The next step of the implementation process is to prepare the learners for the upcoming education process. First and foremost that means making sure that they are familiar with the tools and have the knowledge required for completing the course. Are the learners proficient in the use of programs they will use during the course? Are they aware of the course’s goals, and its schedule?

Sometimes it may be necessary to carry out preliminary education to create the necessary foundation for the teaching of the course’s materials. You may also need to explain how to use the media player software used in the delivery of the electronic course, or how many points will be awarded for the completion of the course’s tests and what is the required passing grade.

Preparing the environment

During this step it is necessary to ensure that the technical and organizational requirements of the course, formulated during the Development stage, are met, and to prepare the environment where the teaching will be conducted. Depending on the chosen format, the preparation may include the following:

  • Setting up a projector and a screen of adequate size.
  • Setting up the audio in the room/auditorium where the learning will take place.
  • Making sure that the computers that will be used for teaching have sound cards installed, connecting and testing the speakers/headphones.
  • Downloading and installing the necessary software and/or plugins (Java Virtual Machine, Flash, etc).
  • Supplying a whiteboard and marker pens.
  • Preparing the necessary printouts that will be distributed to the learners.

An adequately prepared environment helps both the learners and the instructors to concentrate on the learning process with a minimum of distractions.

In conclusion

Once the instructors have been taught, and the learners and the environment prepared, the learning process can begin. Keep in mind that implementation is a key stage of the ADDIE process, because it is during this stage the information contained in the course you created is transferred to the target audience. This makes it imperative that you pay attention to the feedback received from the learners taking the course, and address it.

It is a shame to make a single blunder during the implementation stage that can negatively impact knowledge acquisition, as well as the learners’ reaction towards the course. For example, failing to install a plugin necessary to view the course’s videos, or putting the learners in a noisy room can greatly affect the final result in a negative way. If you are responsible for the learning process in your company, pay extra attention during the Implementation stage. It would not hurt to accompany the instructors and learners before the learning process begins. This way, the quality of the learning process is least likely to be impacted.

This post was first published on eLearning Industry.

Getting To Know ADDIE: Part 3 – Development

Getting To Know ADDIE: Development  

Having scoped out the target audience, settled on what knowledge the course aims to impart, and composed a plan during the Design stage, we are prepared to move on to Development; a key stage of the ADDIE process, though not the last one. If during the previous stages we were chiefly concerned with analyzing the requirements and planning the education process, now we are getting down to business and beginning to work on the course proper.

The Development stage can be divided into three main phases:

  1. Creating a prototype.
  2. Developing the course.
    • Development.
    • Quality assurance.
  3. Conducting a test run.

1. Creating a prototype.

Usually, a prototype is created to demonstrate the general concept of the course to the higher-ups or clients and getting their approval for moving forward with the course development. A prototype need not be large; a couple of pages usually suffices for demonstration purposes. Ideally, every page in the prototype should have a different structure. For example, one page can contain an illustration and some text, another an interactive task, and the third one contain nothing but text. This way you can cover most use cases and demonstrate what the majority of the pages in the course will look like.

In addition to the prototype, it is customary to provide a short summary of your course plan. The summary should be short (one, two pages at most), but having read it, a person in charge of approving the plan should have an adequate understanding of how the finished course will look like. Based on the prototype and the summary, they should be able to make the decision to either give you a go-ahead to proceed with the development of the course, or request that changes be made to the plan first. It would not hurt to supply the document containing the education strategy developed during the Design stage as well. Should the person in charge have questions or concerns after reading the plan summary and reviewing the prototype, this document may give them the answers they seek. Once the higher-ups are satisfied with the direction you have chosen, you can greenlight the Instructional Designers and proceed to the creation of the course.

2a. Developing the course.

During the Design stage, while creating the education strategy, you should have settled on what types of material you will use. Having taken the feedback from the higher-ups or clients into account, it is time to start developing the course.

Here are a few considerations you should keep in mind during development:

  • No one is thrilled by having to read pages upon pages of dry text.
    Spice your course up with media content. Anything from illustrations to videos to graphs and tables will make your course look better and help the learners to acquire information on the visual level. Interactive tasks will make the course more engaging and give the learners some hands-on practice related to the topics being learned.
  • Make an effort to present the information in a logical order.
    Introduce new topics only after the learners have had a chance to grasp the basics and understand all underlying concepts.
  • Regardless of how far you proceed into the development stage, always keep in mind the main educational goals the course aims to achieve.
    Do not disregard the data collected during the Analysis stage; it is there to help you achieve those goals.

2b. Quality assurance.

You will do well to make a habit of constantly testing the course as it is being developed. Quality assurance professionals look at the produced content through the eyes of the end user, spotting both typos and technical errors. A fresh look is always beneficial, and it helps to reveal issues that may not be apparent to the Instructional Designers.

To make sure that quality assurance does not hinder the development of the course, it is customary to divide the course into sections or modules, so that once a section is completed, it is passed to the QA or directly to the customer for evaluation, while the development team begins working on the next one. By the time the second section is finished, you should have already received the feedback about the first one, so that you can fix any shortcomings in the first section while the second is being reviewed. Thus, working on individual separate modules helps you to both develop rapidly and keep the course quality high.

3. Conducting a test run.

Another way to ensure that your course meets the required standards, in addition to the creation of a prototype, is to have a number of learners complete the finished sections of the course. Their feedback is recorded and changes are made to the course based on it, and then the corrected sections are returned to the learners for another pass. The iterations continue until all the kinks are worked out.

It is also important to measure the time it takes the learners to complete the course and see how it measures up to the goals set before the course. If the average completion time is significantly longer than planned, consider revising individual pages and/or sections, or even removing pages containing non-essential information outright.

Having finished developing the course, it is a good idea to once again submit it for review to higher-ups/clients for additional feedback.

In Conclusion 

If you did not slack during the Analysis and Design stages, Development would be a great deal easier. To reduce the number of iterations, listen carefully to all feedback and use it to improve the course and resolve all discovered issues before sending the course sections for another review. It is important to keep the higher-ups happy, so that they would leave you to your work, and you would not have to spend the valuable development time arguing. In the next installment we will talk about Implementation, and, in parting, let me wish you patience, which is essential during development. Until next time!

This post was first published on eLearning Industry.