Testing for Instructional Designers — A Common Mistake

Somebody sent me a link to a YouTube video today -- a video created to explain to laypeople what instructional design is. Most of it was reasonable, until it gave the following example, narrated as follows:

"... and testing is created to clear up confusion and make sure learners got it right."

image from https://s3.amazonaws.com/feather-client-files-aviary-prod-us-east-1/2016-11-03/d044b9e8-731e-42ff-a2ef-a9b5df14f533.png

Something is obviously wrong here -- something an instructional designer ought to know. What is it?

Scroll down for the answer...

Before you scroll down, come up with your own answer...

.

.

.

.

.

.

.

.

.

.

.

.

Answer: 

The test question is devoid of real-world context. Instead of asking a text-based question, we could provide an image and ask them to point to the access panel.

Better yet, we could have them work on a simulated real-world task and follow steps that would enable them to complete the simulated task only if they used the access panel as part of their task completion.

Better yet, we could have them work on an actual real-world task... et cetera...

Better yet, we might first ask ourselves whether anybody really needs to "LEARN" where the access panel is -- or would they just find it on their own without being trained or tested on it?

Better yet, we might first ask ourselves whether we really need a course in the first place. Maybe we'd be better off to create a performance-support tool that would take them through troubleshooting steps -- with zero or very little training required.

Better yet, we might first ask ourselves whether we could design our equipment so that technicians don't need training or performance support.

.

.

.

Or we could ask ourselves existential questions about the meaning and potency of instructional design, about whether a career devoted to helping people learn work skills is worthy to be our life's work...

Or we could just get back to work and crank out that test...

SMILE...

 

 

Improving Our Performance in Watching the Presidential Debates

I work in the learning-and-performance field. I watched the first two presidential debates. They were problematic to say the least. So, I thought to myself, "Hey Will, you're a learning-and-performance professional. What might be done to improve them?" Actually, it was more like, "Damn, somebody's got to make these debates better. They're not creating the best outcomes. They're not really educating us on the important issues for the presidential election. Our debates are not helping the democratic process. Indeed, they're probably creating harm."

24564574914_0cdd268f92_zPicture above by Donkey Hotey 2016 Creative Commons on Flickr

In looking at the debate process, there are several leverage points, including the following:

  1. The debate format.
  2. The questions asked.
  3. The candidates' responses.
  4. The citizenry's cognitive processing of the debate.
  5. The news media's and social media's messaging about the debate.
  6. The citizenry's cognitive processing of the media messaging.

For most of these, we -- the citizenry -- have very little direct leverage to influence the process. Together, we can influence the social-media messages, but as individuals, we have very little influence there. Tears in the ocean.

But certainly the media and the candidates are acting with intentionality to influence the citizenry. Candidates act bold and confident because they know that people, in general, are suckers for those who display confidence. Candidates use certain words to influence -- even if those words are too general to be meaningful (words like freedom, equality, strength, diversity). Candidates avoid talking about complicated issues, because they think we can't understand. Candidates speak in terms of black and white, good and bad, either this or that -- though the world is shaded in sepia tones -- because they think we need certainty.

They act because they think we as citizens will react in predictable ways. But what if we changed and improved our responses to the debates? What if we as citizens improved our cognitive processing and instead of having knee-jerk reactions, we improved our thinking? What if instead of being persuaded by irrelevancies, we were persuaded by a clear understanding of the issues? Is there anything we can do to improve and deepen our thinking about the debates?

Yes! We can do a better job! Not a perfect job, not a brilliant job, but we can improve and deepen our thinking if we take some time to think about what we care about and what matters.

I'm sure my efforts here will be inadequate, but I offer a one-page checklist to spur your thinking. Take a look, try it out at Wednesday's final Presidential debate and let me know how you'd improve it. Tell me what works and what doesn't. Better yet, let me know what else we can do to improve the results of the debates.

 

Try the Presidential Debates Insight Checklist

 

Read My Article on LinkedIn

 

 

WHAT ELSE? WHAT DO YOU THINK?

 Okay, I really feel like this effort is too inconsequential and too unlikely to make a difference, so I'd love to hear your ideas:

  • What's wrong with this approach?
  • What could make it better?
  • What else could be done to improve the outcome of the debates -- that is, creating a better informed citizenry.
  • Are there other tools in our learning-and-performance toolbox we might use?
  • What could make our debates great again? (sorry, couldn't resist)

Effects of Burnout on the Brain and on Cognitive Performance

Great Article: Burnout and the Brain by Alexandra Michel, writing in The Observer, a publication of The Association for Psychological Science.

Article link is here.

Major Findings:

  • Stress may cause changes in the brain.
  • Stress may cause problems with:
    • attention
    • memory
    • creativity
    • problem-solving
    • working-memory problems in general

Will's Caveats:

  • Studies were mostly correlational, so not clear whether there is cause-and-effect relationship.

Defining Stress:

  • Stress is NOT caused just by working long hours. As the article says:

"a comprehensive report on psychosocial stress in the workplace published by the World Health Organization identified consistent evidence that 'high job demands, low control, and effort–reward imbalance are risk factors for mental and physical health problems.' Ultimately, burnout results when the balance of deadlines, demands, working hours, and other stressors outstrips rewards, recognition, and relaxation."

Learning-and-Performance Ramifications

  • If we want our organization's employees to work at their best, we can't put them under long-periods of stress.
  • We need to give them more control of their work, reward them appropriately especially with recognition and status (not necessarily with money), promote periods of rest and relaxation, and give employees input into their job environment.

Gary Klein leads Conference on Naturalistic Decision Making

Gary Klein, a winner of the Neon Elephant Award, is hosting a conference on naturalistic decision making.

Click here to learn more...

Klein is a powerhouse in utilizing research to support people at work. For those who are deeply interested in supporting on-the-job learning and performance, this is the kind of conference that will give you real-world science that you can use in your work. Forget the wild ideas of the informal-learning industrial complex, and get yourself grounded in the real world.

Triggered Action Planning

This article was originally published in Will's Insight News, my monthly newsletter.

It has been updated and improved to include new information.

Click here if you want to sign up for my newsletter...

 

Radically Improved Action Planning

Using Cognitive Triggers to Support On-the-Job Performance

 

Most of us who have been trainers have tried one or more methods of action planning--hoping to get our learners to apply what they've learned back on the job. The most common form of action planning goes something like this (at the end of a training program):

"Okay, take a look at this action-planning handout. Think of 3 things from the course you'd like to take away and apply back on the job. This is critically important. If you feel you've learned something you'd like to use, you won't get the results you want if you forget what your goals are. On the handout, you'll see space to write down your 3 action-planning goals. I'm going to give you 20 minutes to do this because it's so important!"

Unfortunately, that method is likely to get less than half the follow-through that another--research based--method may get you!

 

When we as trainers do action planning, we are recognizing that learning is not enough. We want to make sure that all of our passionate, exhaustive efforts at training are not wasted. If we're honest with ourselves, we know that if our learners forget everything they've learned, then we really haven't been effective. This goes for e-learning as well. There's a lot of effort that goes into creating an e-learning course--and, if we can maximize the benefits through effective action planning, then we ought to do it.

 

 

Before sharing with you my radically improved action-planning method, it's critical that I motivate it. Look at the above diagram. It shows that the human mind is subject to both conscious and sub-conscious messages. It also shows that the sub-conscious channel is using a broader bandwidth--and when humans process messages consciously, they often filter the messages in ways that limit the effectiveness of those messages.

 

One of the most important findings from psychological research in the past 10 years--I hate to call it "brain science" because that's an inaccurate tease--is that much of what controls human thinking comes from or is influenced by sub-conscious primes. Speed limit signs (conscious messages to slow down) are not as effective as narrowing streets, planting trees near streets, and other sub-conscious influencers. Committing to a diet may not be as effective as using smaller dishes, removing snacks from eyesight, and shopping at farmer's markets instead of in the processed-food isles of grocery stores.

 

We workplace professionals tend to use the conscious communication channel almost exclusively--we think it's our job to compile content, make the best arguments for it's usefulness, and share information so that our learners acknowledge its value and plan to use it. But, if a large part of human cognition is sub-conscious, shouldn't we use that too? Don't we have a professional responsibility to be as effective as we can?

 

My action-planning method does just that. It sets triggers that later create spontaneous sub-conscious prompts to action. I'm calling this "Triggered Action Planning"--a reminder that we are TAP-ping into our learners' sub-conscious processing to help them remember what they've learned. SMILE.

 

The basic concept is this:  We want learners, when they are back on the job, to be reminded of what they've learned. We should do this by aligning context--one of the Decisive Dozen research-based learning factors--in our training designs. We can do this by using more hands-on exercises, more real work, more simulations--but we can extend this to action planning as well.

 

The key is to set SITUATION-ACTION triggers. We want contextual situations to trigger certain actions. So for example, if we teach supervisors to bring their direct reports into decision-making, we want them to think about this when they are having team meetings, when they are discussing a decision with one of their direct reports, etc. The SITUATION could be a team meeting. The ACTION could be delegating a decision, asking for input, etc., as appropriate.

 

In action planning, it's even simpler. Instead of just asking our learners what their goals are for implementing what they've learned, we also ask them to select situations when they will begin to carry out those goals. So for example:

  • GOAL: I will work with my team to identify a change initiative.
  • SITUATION-ACTION: At our first staff meeting in October,
    I will work with my team to identify a change initiative.

Remarkably, this kind of intervention--what researchers call "implementation intentions"--has been found to create incredibly significant effects, often doubling compliance of actual performance!!!!!!!!!!!!!

 

I think this research finding is so important to workplace learning that I've devoted a whole section of my unpublished tome to considering how to use it. Instead of using the term "implementation intentions"--it's such a mouthful--I just call this trigger-setting.

 

The bottom line here is that we may be able to double the likelihood that our learners actually apply what they've learned simply by having our learners link situations and actions in their action planning.
  

 

New Job Aid for Triggered Action Planning

 

You can easily create your own triggered-action planning worksheets or e-learning interactions, but I've got one ready to go that you can use as is--FREE OF CHARGE BECAUSE I LOVE TO SHARE--or you can just use it as a starting point for your own triggered-action-planning exercises.

 

Click here to download the triggered-action-planning job aid (as a PDF)

 

Click here for a Word version (so you can modify)

 

 

Research:

 

Gollwitzer, P. M., & Sheeran, P. (2006). Implementation intentions and goal achievement: A meta-analysis of effects and processes. Advances in Experimental Social Psychology, 38, 69-119.

 

Bjork, R. A., & Richardson-Klavehn, A. (1989). On the puzzling relationship between environmental context and human memory. In C. Izawa (Ed.) Current Issues in Cognitive Processes: The Tulane Floweree Symposium on Cognition (pp. 313-344). Hillsdale, NJ: Erlbaum.

 

Roediger, H. L., III, & Guynn, M. J. (1996). Retrieval processes. In E. L. Bjork & R. A. Bjork (Eds.), Memory (pp. 197-236). San Diego, CA: Academic Press.

 

Smith, S. M., & Vela, E. (2001). Environmental context-dependent memory: A review and meta-analysis. Psychonomic Bulletin & Review, 8, 203-220.

 

Thalheimer, W. (2013). The decisive dozen: Research review abridged. Available at www.work-learning.com/catalog.html.

Safety, Learning Design, and Organizational Culture

As a learning consultant, I've been called into workplaces to do work-learning audits specifically focused on safety. Unfortunately, what I've seen too often are poor safety-learning practices. People often talk a good game of safety, but their practices are just not effective. Let me give you one example. I was at a manufacturing plant and was told that all team meetings talked about safety. However, what I saw at actual team meetings was a perfunctory exhalation about safety that was likely to have zero effect on actual safety outcomes. Seriously, many team leaders would say something pithy like "10 fingers, 10 toes" and that would be it!!

To be truly effective, safety messages have to follow the principles of all good learning design. Specifically, safety messages have to be context-based. They have to refer to actual workplace situations, and get employees to visualize and anticipate safety-critical situations and the actions that are needed in those situations. Safety messages also have to prompt employees to retrieve these situation-action links and do that in a manner that is repeated in various ways over time.

Recently, while teaching a workshop, one of the participants told a great story about how General Electric has built a set of cultural expectations that propel safety. The author--who wants to remain anonymous--wrote up the following overview of what he/she observed at GE.

I have had the pleasure to conduct training for the field service organization at GE.  One key aspect of the field service organization is safety.  A seemly simple task of lifting a heavy object with a crane can easily result in fatality by a shift in the chain causing the object to swing out of control.  During my work I was impressed with the relentless focus on safety, which was not just in words, but in action.  I thought it would be useful to share an example of how safety is built into their culture.

Each day of a training session, or any meeting for that matter, always started with a safety moment.  This discussion focused on the potential safety issues that could come up, and precautions that need to be followed.  I would start the training by having the hotel facility manager come in and cover the emergency procedures.  If I failed to start any training session in this manner, a participant would, without exception, come to me during the first break indicating that we forgot the safety briefing.  Unlike other organization where I would be asked to show a safety video, and people would count sheep until it ended, this safety briefing was seen as important to all the participants. 

At the start of each training day, and after lunch, a participant would be assigned to share a safety moment in their work that enabled someone to avoid a potential injury.  There was never a problem getting participants to accept responsibility for conducting one of these safety moments.  In fact, after sharing their experience, there was always a round of applause from the other participants.  This consistent practice, and positive reception by individuals of all levels helps to foster a strong safety culture within the organization.

In talking with the author of this observation, I was amazed at how deeply ingrained a culture of safety was in this GE environment. From this example, here are lessons learned--many of which will be relevant even to those who are not dealing with safety, but who are focused on performance-improvement in general.

  1. They focused on specific safety issues and situations.
  2. They focused on safety ubiquitiuosly, not just in training and not just when it was "safety time."
  3. People bought into the importance of safety--they didn't just go through the motions.
  4. There were expecations that safety discussions were scheduled into everything.
  5. Many people wanted to volunteer to lead safety discussions--not just people designated as safety officers.
  6. People really appreciated the safety discussions--and they showed their appreciation.
  7. Management was not the only driver of safety.
  8. Safety messages were repeated, and spaced over time.

Special thanks to the anonymous author and to GE for demonstrating that safety can be inculcated into workplace practice.