Familiarity

Over the years and role changes I’ve used a variety of different VLEs. From Blackboard to FutureLearn, and from custom in-house developed VLE to customised large-scale MOOC platform. So, how important is familiarity when working, designing and developing on these platforms?

Firstly, are we talking about the familiarity I need to navigate the multitude of features and processes to get the course built and delivered? Or do we mean the familiarity the learner needs in order to have a smooth and tangible learning experience, whether they sit down and structure their learning or dip in as and when they can? Let’s try and deal with both.

Explain everything

  • For me: If you’re new to the platform it’s good to write notes to yourself as you do something new, work out how a feature works, etc. This is also a great resource for you or the rest of the team to open discussion around the how and why of particular approach to presenting a learning resource. Keep ideas, plans, design/colour schemes, times, asset library, etc. all in one place for easy reference. 
  • For the learner: Accept that the learner may not have read your carefully scripted course page or expensive course promo video and repeat it at the beginning of the course. The odds are that you put a lot of effort into that content so make sure it’s of use at the start of the course. It will need to be modified, you don’t need the marketing/promotional terminology here, so make sure it reads like the rest of the course (the ‘voice’ of the learning). Carry this approach to the whole course, not just the start: explain why you’ve included a video to watch and what the learner should think about while they watch it. Explain the structure of the course and what it means for their journey, and how the journey ends. And what happens after that. 

Structure and navigation

  • For me: A new platform will mean a lot of different, well, everything! Who hosts, manages or supports the platform? Who are they, where are they, when are they available? Make them your new BFF and ask for help as well as providing a fresh pair of eyes and offer feedback from your own experience on other platforms to see if you can provide efficiencies or development to improve. Always ask questions and always explain why, as well as showing them your results. 
  • For the learner: A consistent structure and navigation to the course will help the learner feel more comfortable and relaxed, therefore are more likely to retain the knowledge you’re presenting them with. As with the previous item, explain how the structure works, explain how to use the navigation, and above all keep the consistency of design that you’ve worked hard to develop. If you use colour of font size as a code of activity or resource identification, use it every time (you’ d be surprised how often I’ve seen inconsistencies, usually across courses rather than within the same course).

Example: FutureLearn navigation, Warwick’s ‘Leadership for healthcare improvement and innovation’.

Template

  • For me: Personally I hate templates or a forced way of working, but the method and structure they offer are hard to ignore. There’s a reason why templates work and that, as I mentioned previously, provides a consistency across courses, programmes, and team members. if you’re working in isolation, then the template probably doesn’t make sense to you as you already know what you’re doing. If you working a part of a larger team then the template provides the working structure you all need to adhere to to get that consistency I talked about.
  • For the learner: The template should not be something the learner ever really notices. The template is there to provide a consistent learning experience for the learner. If it works they wont notice it. If it fails they’ll complain of not understanding what they should be doing, or when, or how, or why. The template will provide familiarity and structure.

Text and images

  • For me: Nothing bores me more than a course full of pages and pages of text, no visual cue at all as to what’s happening. If nothing else a well placed image showing the general theme or topic helps bring the page to life. While some subjects are clearly more visual than others, there’s no excuse for not using some Creative Commons or licensed images, a YouTube video also explaining the subject, concept, interview with an expert, educator, practitioner, etc. While we try and accommodate as many styles of presenting learning materials, and those materials often reach us from the educator in text form, we would not be doing our job if we didn’t try and find a visual solution to break the text blocks up, even if it’s only a different way of presenting the text.
  • For the learner: if the learner wanted to read a textbook to gain the knowledge and qualification from the course, they’d that. Often what one learner likes is not what another likes. While one person can read book after book and retain the knowledge easily many cant, me being one of them. The inclusion of different sorts of activities helps, but so do different approaches to presenting the learning materials: image, charts, photos, infographics, video interviews, to-camera teaching presentations, video case studies, high-profile documentaries (check the ownership and originality if you’re using these from YouTube), etc. There’s always a way to bring something visual to the course.

Example: Documentary – DHL International Supply Chain, loaded to YouTube by DHL.

… now make an activity out of it, introduce some questions that the video can help with but requires the learner to go further afield to find answers and more resources for. Make the image or video part of the learning, not the learning itself.

Langauge

  • For me: If the whole team uses the term ‘page’ or ‘step’ to indicate a different element of a learning package, then be sure you all use that term. By using a variety of different terms to mean the same thing you will forever be translating instructions from one source to another for different things. Something will always get lost in the translation, mistakes will be made no matter how hard you try, and there will be more work down the line when you have to unravel the mess. Be sure the terms you use within the team are consistent (that word again) and appropriate. If you work with a new educator who’s used to different terms and ways of working then open the dialogue and work out what’s best – do they change to accommodate you and your team, or do you change your processes to accommodate them? Decide early on and stick to it! 
  • For the learner: No one wants to read a course that is heavy in jargon, acronyms, complicated academic terminology or badly presented materials. No one. Even if you’re writing for advanced Masters level students you should still use appropriate language, explain an acronym, and avoid jargon. You obviously don’t want to dumb the language down so it sounds like you’re being condescending to them, but there is a level that is acceptable. Find it, stick to it, and test it!

Familiarity in learning has always been about consistency – consistency in the approach to design and present the materials, consistency in language appropriate to the level of the course and the intended audience, consistency in quality of photos or images or videos, consistency in length of pages or steps. By being consistent in what you do and how you do it your course will also offer a consistency the learner will become accustomed to, which will bread familiarity and comfort with. From here it will be easier to follow the learning and complete the course.

Image source: Pete Birkinshaw (CC BY 2.0)

Articulate Storyline 360 – Publish as video – Review

In an eLearning course, the content is presented in chucks with page navigations and interactions. If you already have eLearning courses and you want to upload them to online course sites then it can be a challenge. There are many online course sites like Lynda, Skillshare, CreativeLive, Udemy, etc., that accepts video based content. To…

How To Measure The ROI Of Online Training?

Today, most organizations use eLearning as a significant part of their training delivery. As traditional eLearning moves towards mobile learning or mLearning and provides learners the flexibility to learn on the device of their choice (notably tablets and smartphones), the eLearning adoption is gaining further momentum. eLearning and mobile learning provide several benefits to organizations. However, the focus is now shifting to determining its impact and the Return On Investment or ROI of online training.

Measuring The ROI Of Online Training

In this article, I will begin with a quick summary of the benefits that eLearning offers, what ROI is, and how you can measure it. I will also outline the ROI methodology we use.

What Are The Advantages Of eLearning?

I am quoting extensively from my earlier article Return Of Investment (ROI): Are You In?. This article had originally appeared in CrossKnowledge’s Learning Wire Blog. It also outlines the measures to maximize the ROI.

Over the last two decades, most organizations have made investments in eLearning primarily for the following benefits:

    • Anytime, anywhere access (on demand availability).
    • Self-paced, interactive, and more engaging learning (learner perspective).
    • Less disruptive delivery (in contrast to ILT).
    • Cost-effective (particularly when reaching out to a large audience).
    • Consistency of message and easy updating of content.
    • Easy tracking of learner progress and completion (business perspective).

While the eLearning advantages are well accepted, increasingly organizations are seeking ways and means to determine its impact on learners as well as on business. Let’s see what Return Of Investment (ROI) is and how you can assess if your eLearning or online training initiatives are generating the required ROI.

What Is ROI?

ROI is the return on investment that an organization makes (ROI = Gain or Return/Cost). It can be determined through two factors namely the Investment made (or cost incurred) and Value/Gain accrued (or return).

A successful eLearning initiative should be able to demonstrate gains that are more than the investment.

How To Determine Costs And Assess Returns?

Costs are fairly easy to define and would normally include the cost of eLearning course development as well as associated costs of team members (including teams that are associated with the development process and Subject Matter Experts).

Typically, there would also be associated costs of the supporting delivery (Learning Management System, Administrative cost of managing the initiative, and other related infrastructure required for delivery).

Determining the “value” or “gain” is far more tricky. We nail this by looking at the gains for the organization as well as for the learners.

  • Organizational perspective.
    Let’s begin by re-looking at the gains most organizations seek when they adopt eLearning and see how many of these translate to reduction in costs and hence improvement in returns.

    • Less disruptive delivery.
      This translates to man-days available now to the organization that would have been allocated to travel and training in the ILT mode.
    • Reduced travel costs.
      These can be determined easily.
    • No associated costs for trainers.
      These can be identified easily.
  • Learners’ perspective.
    Next, let’s take a look at the gains that accrue on account of effective eLearning course designs:

    • Immersive and engaging learning translates to better assimilation. This in turn leads to proficiency gain and a tangible increase in productivity.
    • More learners across the organization can be trained in lesser time (while they get the flexibility to learn at their own pace).
    • Coupled with tracking, the eLearning initiatives can be scheduled and completed faster as compared to ILT sessions.

What ROI Methodology Can Be Used?

Most of us are familiar with Kirkpatrick’s model of evaluation. In today’s context, adding Phillips’ ROI calculation as the fifth level makes this framework even more useful and relevant. By using Level IV evaluation data, we can convert the results into monetary value. Then we can easily compare them against the cost of the eLearning program and determine the ROI.

EI Design Kirkpatricks Model

To give you a sense of how it can be practically used, let me summarize the approaches we typically adopt:

    • Level 1:Reaction is measured by taking feedback from learners. We have used online surveys in the past but now we add features of “Like the course” and “Recommend the course” options within our eLearning course framework.
    • Level 2: Learning can be easily measured through scoring patterns in the end of course assessments.
    • Level 3: Behavioral changes are certainly more difficult to assess. We use a combination of techniques to assess how much of the newly acquired learning is being applied on the job. This could be measured through improvements in efficiency or doing the same task with a new approach.
    • Level 4: Business impact is generally measured through productivity gain, impact on quality measures through reduction in re-works, getting higher number of work assets first time right, and so on.
    • Level 5: ROI is normally calculated by converting the business impact gains (as shown in level 4) to a monetary value.

I hope this article was useful in understanding the ROI definition and more significantly, what ROI methodology will enable you to measure the ROI of online training. At EI Design, we do workshops that can enable you to adapt the standard ROI methodology to your organization. Do reach out to me if you need further details.

Source: https://www.eidesign.net/how-to-measure-the-roi-of-online-training/

The University of tomorrow is …?

I’ve just read this article and wanted to share a couple of thoughts I had while I was reading it: “It’s the end of the university as we know it”

The title is clearly clickbait, testing your resolve to read beyond the tweeted headline, knowing full well ‘the end of the university’ will get people interested (or enraged that this kind of talk is still going on … MOOCs anyone?). That the URL is not the same as the title implies they might change the title at a later stage … “/the-future-of-the-university-is-in-the-air-and-in-the-cloud/”?

Here are some soundbites from the article:

“Shocking as it might seem, there is one catch-all answer that could be the remedy to many of these concerns: Cut the campus loose. Axe the physical constraints. The library? Classrooms? Professors? Take it all away. The future of the university is up in the air.”

Another, when looking at the history of how and why universities are set up like they are:

“It is untenable for universities to continue existing as sanctums for a small group of elite students, taught by top scholars.Technology isn’t only refashioning the ways in which we live and work, but also changing what we need to learn for these new schemes of existence: It’s returning us to a need for specialized learning, for individualized education that is custom-tailored to one’s needs. A world in which most of our learning is more self-directed and practical is, in many ways, a return to an apprenticeship model that existed before industrialization.”

Predictions on the future of learning, at universities at any rate:

Online “cloud” teaching is cheaper; universities can offer such online-based (or majority-online) degrees at the lowest rate—making for a cheap(ish) degree, available to everyone with access to the internet, and taking place completely digitally. Meanwhile, other students will pay a premium to interact with professors and have more of a traditional campus experience. At the highest end, the richest or most elite students may get the full Oxford tutorial experience, brushing elbows with the best of scholars; they’ll just have to pay through the nose for it”

Read the article, let me know what you think – agree or disagree with the tenet of the article, that this is the end of the university?

Image source: Dave Herholz (CC BY-SA 2.0)

What’s next for MOOCs?

I’m not going to get in to the detail of whether MOOCs have been the disruptive element for learning as many opined four or five years ago, many have written much more eloquently on this than I ever could. For more just search for related terms or read this and this and this and this.

I will, however, pass a few words and a little judgement on one aspect of some of the developments I’ve been following for ‘online learning’ – accreditation. 

Firstly, has anyone else noticed that the original MOOC platforms don’t refer to the courses that are offered through them as MOOCs anymore? Even the platform that pushed ‘free online learning’ at every opportunity has dropped the ‘free’ from nearly all pages and courses. Obviously the ‘free’ business model was never going last long once the platforms realised that they had massive overheads to cover (staff, hosting, support, development, etc.), and that doesn’t cover the costs incurred by partners to develop the courses either.

For me online learning, whether it’s an degree awarded from an established College or University or a ‘free’ MOOC-esque course, has always been about the value the course is able to offer the student taking it. That value is both about the actual content and subject as well as the value the new knowledge has to the individual who has taken (and presumably passed) the course.

This value could be the

  1. personal satisfaction in gaining new or further knowledge,
  2. learning about a new skill or subject that has semi-professional interest (a subject at the periphery of the individual’s profession, but is not essential to it) or
  3. something that is specifically relevant to the individual’s immediate role or career progression.

Learning is but one side of the reason someone will invest time and effort into learning. The learning needs a purpose – undertake a course on Shakespeare because you’ve always like his plays and want to know more about the plays and playwright. A family member is diagnosed with dementia and you want to know and understand more about the condition, etc. This is all well and good, but people who take courses for these reasons are unlikely to buy any kind of certificate or further learning opportunity from it. They are likely to go on and take other related courses, again to further their understanding.

People who take online courses who are doing it for a professional purpose (changing job or role, career progression, professional interest, etc.) are more likely to purchase some form of certificate, but it’s still not guaranteed. I’ve taken (well, started!) a few MOOCs related to my job and interests, and finished one (the #EDCMOOC)! 

For me, the future of this kind of learning is what the course can really offer those people who complete it. A certificate is not enough – being able to show I completed 75% or 95% or another arbitrary number of the steps and all test questions means next to nothing. The certificate does not give any indication to whoever I show it to about what I had to do to get those steps completed or whether the test were 5 questions or 50. Did the course have an active educator or was it facilitated by an academic (not the course creator) or student from the partner institution? Was it facilitated at all, or just a click-next learning journey with a few tests or discussion points?. No, for me, if I’m going to pay for the course ‘certificate’ it needs to show something much much more. It needs to show how valuable it is to the industry I work in. Sometimes even the institution that created course isn’t enough pull for the certificate to mean anything.

A medical MOOC certificate would mean so much more if it was accredited by the International Council of Nurses, a marketing course accredited by the Chartered Institute of Marketing, etc. Not only could / should the course offer the opportunity to earn valuable CPD points but the accredited course outcome should be something a current or future employer would look at and immediately see the value to them; that this candidate is coming to work here with a good resumé, has shown initiative by taking further learning opportunities and is showing the skills to find and evaluate the courses that will offer them the best opportunity to further themselves.

I don’t think the way forward for MOOCs is for degree-credits either but it’s a popular route, probably as it’s easier to sell to the University partners than anything else. Only time will tell. 

BOOK REVIEW: “Design for How People Learn” by Julie Dirksen

“Design for How People Learn” by Julie Dirksen

© Ugur Akinci

“Design for How People Learn” by Julie Dirksen is a heavy book, both literally (top-quality glossy paper) and figuratively. It’s an important reference work that I think all trainers, instructors, and e-learning designers should read.

We are lucky to have today e-learning tools like Adobe Captivate and others. Setting up slides, quizzes, links, buttons, voiceovers, inserting images, videos etc. is a breeze.

However, when it comes to designing a learning experience that would actually help the students learn, and make a difference in their lives by changing the way they ACT, I believe there is no one-click app for that.

That complex skill, which requires us to put ourselves in the shoes of our prospective audience, needs to be deconstructed first, its individual components pulled apart and laid bare, and then reconstructed for a training package that really works and changes lives.

Dirksen’s easy-to-read and well-illustrated book accomplishes that goal in 300 full-color gorgeous pages.

The author starts from the A-B-Cs of the subject like “How do we remember?” and “How do you get their attention?” and ratchets up the discussion to higher orbits by diving into different design styles and goals: designing for knowledge, for skills, for motivation, and designing for habits.

One of my favorite chapters in this lovely guide is the last chapter devoted to “Designing Evaluation” since it asks the same questions that I ask myself all the time: does it work? Are the students learning anything? Do they remember anything? Do the learners actually start DOING the right things once the training is over?

Another favorite section is “How learners are different from you?” since for me the greatest trap is to assume that my readers are more or less like me, which they rarely are. Just to realize the ways in which our readers are different from us and understand what we should do to close that perception and cognitive gap is a major design accomplishment, I believe.

The book is rich in laying out the general principles and the research that supports them. But it’s also jam-packed with examples and illustrations to drive home the message.

Chapters are divided into sections, each with its easy-on-the-eye subheader, making it a pleasure both to peruse through the volume and to stop and dive deeper by concentrating on various characters playing different roles during the design process.

For example, in the subsection “Remember, Change Is Hard,” Dirksen presents the photos of four characters, each with a “plausible” excuse not to change and keep doing everything the same old way. Such presentations make the material immediately accessible since it becomes so easy to identify with the characters. We end up saying “yeah, I do that too…” after which we have a renewed and stronger commitment to the material in front of us.

Here are some suggestive headlines from sections that might change the way you design your training materials in the future:

  • “How can you know what your learners are thinking?” (p.51)
  • “What’s pace layering?” (p.74)
  • “Storytelling & Conditioned Memory” (pp. 110, 111)
  • “Tell them less, not more” (p.185)
  • “Create friction” (p.166)
  • “Have learners consider what they already know” (p.162)
  • “How do you give directions?” (p.177)
  • “The anatomy of a habit” (p. 231)
  • “Social and informal learning” (p. 243)
  • “What are we trying to measure?” (p. 272)
  • “Are the learners actually doing the right things?” (p. 283)

Get your copy today. Highly recommended.

Reading: Lurkers as invisible learners

I’ve always been annoyed at being called a ‘lurker’, it’s a term that has a different meaning for me when talking about the engagement, or not, of students in an online class – read my post ‘Listener or Lurker?’ from 2013. In this instance the paper ‘Learners on the Periphery: Lurkers as Invisible Learners‘, by Sarah Honeychurch and colleagues, defines as a ‘lurker’ or ‘silent learner’ or ‘legitimate peripheral participant’ as.

“… hard to track in a course because of their near invisibility. We decided to address this issue and to examine the perceptions that lurkers have of their behaviour by looking at one specific online learning course: CLMOOC. In order to do this, we used a mixed methods approach and collected our data via social network analysis, online questionnaires, and observations, including definitions from the lurkers of what they thought lurking was … [our] research findings revealed that lurking is a complex behaviour, or set of behaviours, and there isn’t one sole reason why lurkers act the ways that they do in their respective communities. We concluded that for a more participatory community the more active, experienced or visible community members could develop strategies to encourage lurkers to become more active and to make the journey from the periphery to the core of the community.”

I’m far more comfortable with the terms used here, and reasons why students don’t engage perhaps how we’d like them to, or indeed in the way we design the course. We need to accept and address that not everyone taking online learning, whether it’s a free MOOC, paid-for CPD course or fully online degree, wants to be social, vocal, or indeed visible in the online environment. We can provide the base materials and ask the students to go off and read around the subjects, we can offer opportunities to engage and ‘test’ themselves on different types of course activities. The only way we know the students are engaging in the subject and materials is usually if we assign marks or grades to the activities, especially if those marks carry weight on the course’s final grade.

Reference

Honeychurch, S., Bozkurt, A., Singh, L, and Koutropoulos, A. (2017). Learners on the Periphery: Lurkers as Invisible Learners. European Journal of Open, Distance and E-Learning. [online] Available at: http://www.eurodl.org/?p=current&sp=full&article=752 [Accessed 21 Jun. 2017].

Making a MOOC ‘successful’

Designing a ‘successful’ MOOC is one thing. Making a MOOC ‘successful’ is something completely different.

Much has been written by far better and more eloquent people than me (here and here and here and here and here) on what makes a successful MOOC – all about interactions, journeys, optimum length, appropriate materials, platform, etc.. But what about making a MOOC successful? To me, there is a difference.

This isn’t about making / building / designing a MOOC, it’s about making / encouraging / promoting / informing people about the MOOC.

The argument about MOOC success, learner retention, completion numbers, registrations, etc., is one that will rage on and on, everyone has an opinion, everyone looking at something different, all very valid, and all very important questions. There isn’t a definitive answer – each MOOC is different, for a different audience, for a different demographic (maybe), and designed in a way that different learner ‘profiles’ can get something different out at the end. If indeed they reach the end, which of course they don’t have.

No, making a successful MOOC requires more than a lead academic(s) subject knowledge, learning technology, instructional/education design, assessments, an appropriate learning goal/journey, working platform, etc. You need all the other stuff as well.

The other stuff you need? Well, try: 

Support
Support for the academic to develop the course in the first place. This could be time allocated to their schedule to relieve them from teaching, research, meetings, etc. It could be time and resources to create the materials and resources needed for the MOOC. We could be talking extra training in how to design for online audiences (not everyone has designed for an online audience before), or media training, or training in how to facilitate or moderate the course once it’s running. This support is required from more areas than just the team helping to put the course together in the first place.

Support is also required for all other aspects of the course progression. If there are facilitators or moderators (use whichever term you prefer – I don’t like ‘moderators’) do they require training in netiquette or how to handle the volume of comments as well as the quality (or lack of) in the comments?

If you are running the course more than once and, let’s face it you should if you’ve put all this effort into planning and designing it, then who will continue to be involved? If the facilitators change, the training starts from scratch.

Coverage
This is a biggie, and depends solely on each course and the intended audience. If you’re aiming at business professionals for specific business process, and the outcome is they get a certificate that can be used for CPD or other such professional recognition, then you need to contact the appropriate professional bodies (trade, commercial, publishing, etc.) and see firstly about accreditation or, at least, whether they will endorse the course or send a notice to their members/readers informing them it’s running.

If your course is a teaching course in something specific, aimed at giving the online student a taste or perspective of online learning from your Institution, perhaps at a degree level education, then make sure you’ve something to give them or sell to them afterwards? Can the certificate be used on an application form as evidence of XYZ? Is completion (by whatever metric ‘completion’ is measured in the course or on the platform) enough to warrant further study with you, and can it be used as an incentive?

Has a course, with a specialist audience (perhaps not ‘massive’ in the sense of tens of thousands of active users, but with a well defined but small community that covers a very specialist industry, theory, concept, etc.) got what it takes to attract the right people without fuller involvement of their trade body or publication?

Marketing
Nothing can happen without someone telling someone else about the course. Yes, most of us have Twitter and LinkedIn and Facebook accounts. I’ll put my hand up and admit I’ve been very active pushing my/Warwick’s latest MOOC (Literature and Mental Health) as much as I dare on my various channels. But, here’s the rub – yes, I have +8,500 followers on Twitter, so the reach of my tweets are quite high, but these followers are looking for tweets about learning design, learning technology, reflections on conferences, that kind of stuff. I am not the right person, or rather my social accounts are not the right fit, to be marketing this course.

I’ll use examples here from the Literature MOOC, to help explain what I mean.

Without the full cooperation of the individual department or Institution marketing, your course will only get limited coverage. Yes, I can tweet about my courses. So can the lead educator(s). So can anyone else interested in the course. What you need, to make a course successful is maximise the audience that is out there. You need to get the big players in the field of influence the course covers involved, get them tweeting about it, get them to blog about it, write to their subscriber list about it – and not just a mention, you need a full on (positive) review and endorsement. That will mean something to their readers, not just another advert that we all ignore.

The marketing team(s) need to target more than just the list of local businesses, or the alumni network, or a generic email list. While these may have their place in this and other uses, MOOCs need more. If you have the contacts, then get national press involved too!

A MOOC about psychology? What would really help get the word out would be positive interaction and authentic ‘reviews’ from the likes of Psychology Today (Facebook: 6.6m followers) or the Psychology Magazine (Twitter: @psychmag 48k followers).

A MOOC about automotive engineering? Having international organisations like Automotive News (Twitter: @Automotive_News 128k followers) interested, or get it added to the Engineering Management Science Manufacturing LinkedIn group (288k members) would help.

For me (putting my old hat on I used to wear as a freelance/self-employed web designer) the marketing is NOT about sending a press release and waiting for the news to hit. It’s not about the phone call where you ask something. It’s about building those relationships (if you don’t already have them) with an organisation who can benefit, either themselves or their members, from the course. Let them see the course, talk to them about the purpose and requirements, offer some advice or editorial, accept advice or criticism in return (you never know, it may make the course better, or at least more applicable to their members).

When I work on an online course or MOOC I want as many (appropriate) people to be able to get on it. Big Data isn’t for everyone. Not everyone wants to know about Shakespeare. Reading isn’t to everyone’s taste. But in order to make the courses successful we need to reach out to individuals to whom these courses could be important, or certainly interesting. If we can’t get to them directly, through whatever channels we have, we need to reach out to the places they congregate; the trade publications, the fan sites, the online communities, the conferences, etc.

After all, if you put the effort in to designing the best online course and best online experience for your thousands of students/learners, you want them to be able to find it in the first place. Yes?

Image source: Marjon Kruik (CC BY-NC-ND 2.0)

MOOCs – question on purpose, quality, student retention, feedback, etc.

Ahh, questions around the purpose, quality, value, etc. in and around MOOCs have started again, and justly so.

  • Disclaimer: Like many I have opinions, but not answers.

The recently raised questions, started by Fred Riley on the ALT mailing list, have produced a good set of resources for those of us who are starting to ask these questions, needing a more comprehensive or value-added answer.

Fred’s original query was:


Does anyone on this list know of any recent research and/or articles on the teaching quality of MOOCs? I’m thinking of things such as:

  • student retention, with MOOC drop-out rates being notoriously high (I plead guilty to that myself :( )
  • student surveys and qualitative feedback
  • how many students in a MOOC platform (eg FutureLearn) go on to take further courses in that platform

I’m sure that there are many other indicators of quality – those are just off the top of my head. I’m not in the MOOC game myself as yet, other than as a punter, but I’m looking to get into the development side of things.


In some instances, especially around the data of students/learners taking further courses (across MOOC platform providers as well as within) is difficult, but I hope we can get to a stage where this kind of data is available and open to interrogation (if only for the individual partner to  query their own courses).

Here are some of the resources shared, in response to Fred’s original query:

If you have any further links or resources that would help Fred and the ALT mailing list, please reply to the thread on the mailing list. If you don’t have access then please leave the link or your comment below for everyone to have the opportunity to read.

Yes, OK. Fred’s question also raises the question around the ‘quality’ of a MOOC, the validity in the data of learner retention or ‘steps completed’ as triggers for saying a MOOC is of a certain quality, or the student was ‘successful’ on the course, but these are for another post. Fred answered this quite clearly on the ALT mailing list that, for him “retention is IMO and indicator of quality as perceived by the student – the better retention, the more students are engaged with the course and its materials. If they don’t like a course, they’ll drop out.”

NB: I’ve helped run several runs of the Warwick/FutureLearn ‘Shakespeare and his World’ MOOC and use this as an example I use where the statistics provided for the 10 week course don’t necessarily match the actual experience. Case in point is the number of learners who complete the course, in that they take all the tests and mark at least one step as complete in each of the 10 weeks. We know from the learners themselves, from their comments, feedback, tweets, etc., that they take what they want from the course – one learner may only like Shakespeare’s comedy’s, another likes on his tragedy’s, so they will omit the plays/weeks they don’t like. They should still be viewed as a successful learner, and I’;m sure they think that of themselves, as in their own mind (and in ours!) they got what they wanted from the course, yet did not actually ‘complete’ it.

If there is one question for 2016 and MOOCs, it’s whether there is any way to really truly, honestly, understand the ‘value’ of a MOOC?

Image source: State Library Victoria (CC BY-NC 2.0)

The Interview Process #altc

From this year’s ALT conference I enjoyed (finally) meeting Wayne Barry, EdTechBook contributor, and chatting about his ALTC presentation.

Wayne’s presentation looked at a different way of interviewing candidates for Learning Technologist positions using standard questions and short presentations, but also the inclusion of a short role-play exercise. Each candidate is given advance notice that they will engage with an ‘academic’ who is interested in introducing elements of distance learning to their module. During the short exercise (many people took issue with the use of the term ‘role-play’) candidates will exhibit both knowledge of their discipline as well as the ability to listen, engage, problem solve, and debate with a member of the team taking the role of an academic.

So, how do you find out if someone will fit in to your office and team environment? Can you do this by just questions? Do competency based questions offer enough space for someone to fudge their way through the process, or rather offer the interviewers enough insight to see the tRuth behind the candidate?

This reminds me of this video, from Heineken: Job Interview. Slightly over the top, but you get the idea – by changing the process you find out many different things (hopefully good) about the candidates. Enjoy!

YouTube: Job interview at Heineken