The Total Learner Experience

Create Losing Designs

We spend a great deal of time planning for learner success. Do we ever create the ability for learners to lose? Should we?

Our society puts a premium on “winning” -- nowadays it seems like every team gets a trophy -- not just the winner. Americans are culturally bound to the ideal of winning. However, in the real world people lose all of the time: athletes lose games and tournaments, politicians lose elections. In high stakes situations like these, losing has a real consequence. How does this affect learning?

Low stakes losing occurs every day. When we lose, we tend to either give up, or continue to practice to overcome the failure. Any casual gamer is familiar with losing and attempting to overcome the loss by quickly replaying the game. Losing, in this context, can be a big motivator, driving the will to practice, which can lead to increased skill. Learning to lose effectively is actually a skill in itself. Resiliency in the face of losing and using strategies to improve is an important life lesson and professional skill. If losing always equates to complete failure, then learners stop striving, stop attempting creative solutions, and see themselves as incapable.

In learning design, a primary goal should be to create an atmosphere where losing is acceptable and intrinsically motivates the learner to try again. The potential learning value from this type of experience is measured not by how often the player loses, but by how much they improve through repeated practice.

In your training design include opportunities for low stakes losing before learners are thrust into their high stakes, real-world situations -- be it a qualifying exam or on the job performance.

Recommendations for a “losing” design:

• Make content available on demand so that learners can review material as needed
  • A common mistake is being overly controlling with content. If the goal is for learners to gain knowledge and skills, shouldn’t they have free access to the tools that can make that happen?
• Communicate how the learner is performing relative to passing scores or via peer-to-peer review as appropriate
  • By communicating how comparatively well they are doing, learners can better sense where they can improve. This also gets their competitive juices flowing.
• Allow plenty of practice opportunities before the final assessment occurs
  • If the only time that learners can demonstrate knowledge is a final graded test, their opportunities to “lose” are reduced to one high stakes moment. Build in low stakes assessment opportunities that prepare learners for the one that counts.

IBM CityOne goes live

I was a bit excited this morning to see that IBM's CityOne had finally launched. I'm always a fan of innovative games with real life applications and I am currently searching for activities to use with my high school students. Billed as a SimCity-like experience, but with an educational side, I was expecting to have some fun building a city to my specifications and doing some high level problem-solving.

That build-up would indicate some disappointment-- and indeed there was. The glossy cityscape I viewed was inviting and attractive. A small icon indicated my first problem to solve. I clicked, exposing the specific city-planning issue related to water. Three solutions were presented with various pricetags. I choose the most expensive, most comprehensive solution and was presented with some boilerplate response. Mousing and clicking over the screen to find my next challenge I determined (perhaps incorrectly) that I had no more challenges. I ended this first of ten rounds thinking that I couldn't make much progress if I only had ten challenges to solve.

I made it to turn 7, barely. By this round I had begun to have challenges from all 4 areas (water, energy, retail and banking), but they all had the same basic format. There were few of the identified game attributes-- little challenge, no suspense, I had no idea what I was competing for or against and I didn't know how or why I could fail-- and I didn't really care.

I was attracted to the game because of this line from Gizmodo: "The idea here, presumably, is that it's always a good thing to educate the populace about the these sorts of problems, and, hey, who knows, someone might unwittingly stumble on a solution we can actually use." This is hard to see how that could happen  because all of the activities are call and response. There is no room or opportunity for players to create their own innovative solutions.

This seems like a marketing/training tool go awry. To launch the game I had to enter detailed personal info-- not surprising-- but the limited options for industry or occupation indicated to me that IBM had a  target audience in mind. One of the follow up questions asked you to indicate if you were a software purchase "decider". Many of the game solutions included using software (presumably developed and marketed by IBM) to clean up a variety of perceived major city issues.

It's disappointing to me that this game is so lame. I have no problem with companies reaching out through games-- I just want them to be fun. There was no fun to be had.

Building Motivation in Online Learning

When creating online learning, the primary goal is to create a course or program that fosters "a continuing motivation to learn," which is defined by Maehr as "voluntary engagement in continuing to learn more about a given topic."

The complexity of human motivation makes it hard to find a one-solution-fits-all model for learners. However, the ARCS model establishes basic criteria an instructional designer can leverage to increase the potential for establishing learner motivation while reducing learner malaise, frustration, and/or dropout. 

The ARCS model consists of these four components:

• Attention
• Relevance
• Confidence
• Satisfaction

Consider the following tactics for each ARCS component. However, there is no substitute for proper analysis to uncover the motivational elements specific to your audience. Without that analysis, you are likely to include too many, too few, or wrong motivational elements.

Attention:

Gain learner attention by integrating:

• High-resolution graphics
• Interactivity and animation
• Conflict and failure
• Problem-solving and inquiry

Relevance:

Provide relevance to the learner's situation:

• Describe clear goals that are consistent with learner expectations
• Connect content to the learner's prior experience and/or knowledge
• Include guided instruction and navigational systems (while allowing self-determination in the consumption of the content)
• Incorporate "authentic" activities based on learner's job or tasks

Confidence:

Make the learner feel confident and aware of learning outcomes:

• Communicate clear expectations for successful completion
• Teach to the learner's "Zone of Proximal Development" -- avoid tasks and/or activities that are too easy or too hard
• Include activities that require critical thinking and minimize guessing
• Provide positive and contextual feedback that emphasizes learner achievement

Satisfaction:

Design for happy learners by:

• Recognizing learner achievement through reward mechanisms and in-context encouragement
• Providing opportunities for the learner to apply what they're learning
• Designing the right amount of work for the subject material
• Creating consistency between the content and assessment mechanisms throughout the material

Also, consider these additional design strategies:

• Reduce the word count on the screen compared with printed or narrated text
• Improve the quality of quizzes (consider quizzes as a motivational tool)
• Incorporate more interactive elements
• Create an automatic messaging system triggered by learner progress that can send positive email providing encouragement, reminders, empathy, and advice. 

References:

• 1986. Maehr, M.L. The motivation factor: A theory of personal investment. Lexington Books.
• 2004. Keller, J.M. and Suzuki, K. Learner motivation and E-learning design: a multinationally validated process. Florida State University, USA, Iwate Prefectural University, Japan.

Games4Change Festival-- Quick Thoughts

I'm attending the Game4Change Festival in NYC for the first time this year. I might do a bit more formal commenting in later posts, but these are a few of the thoughts that keep coming up:

1) STEM-- everybody kept talking about STEM content (science, technology, engineering and math) but I always wonder if STEM content in games is accurate and/or complex enough. I don't know what the answer is to the balance of engagement and science content, but the brief glances I took at some games were encouraging.

2) What is the role of facilitators and teachers for these games? I am not sure that higher level learning outcomes happen for the majority of learners/players without some guided instruction. I know that some choose to become engrossed in this worlds, but the vast majority do not. Also, who will be pushing kids to these worlds. It seems like you will need lots of engaged ambassadors helping get the word out and getting the kids initially turned onto these games.

4) "Everyone"-- People kept saying that everyone plays games. There was a good bit of homogeneity to the crowd. Mostly young, male, and white, with a few Indians and women. This was a laptop-carrying crowd, and while access was mentioned during the education days, it wasn't so much yesterday during the main session. While I spoke with some practitioners who seemed to get it, a lot of the comments I heard tended to assume technological prowess.

By far, the getting kids involved with social games-- as both creators and players-- was the best takeaway I had from this conference. It reaffirmed and rekindled my belief in the power of kids doing.

What the Apple/Adobe Flash Debate Means for Learning Developers

I've been following the "Apple vs. Adobe" debate about Flash for several weeks now, and have recently come to the conclusion that it's not really about Apple forcing its will on Adobe as many in the media seem to be reporting. A milestone occurred yesterday when Adobe's Flash Platform product manager, Dan Chambers blogged about Adobe giving up on creating further iPhone app development tools.

As a designer and developer of corporate training materials, I obviously have a vested interest in this debate. Why, you may ask? Well, if you scan the landscape of eLearning development you can be rest assured that a large majority of online educational material relies on Adobe's Flash platform for its delivery.

I go back all the way to the pre-Internet Authorware days... yes, Authorware 1.0 when it was Mac-only authoring software. I loved Authorware. It was designed specifically for development of computer-based training programs. As the Internet era dawned and Macromedia put its energy behind Flash, I saw the writing on the wall for Authorware. I mean, at first, it was great... Macromedia kept rolling out new versions, and even put a lot of effort into the Authorware web player. The dark days eventually came, and I knew when I talked to some friends at Macromedia what was happening when they lamented that no new Authorware engineers were being hired... and there was only one left at that time!

As Flash emerged, the learning developer community embraced it and began retrofitting it for educational interactivity. I never completely transitioned from the flowline to the timeline. I still remember Robert Milton's explanation of the flowline in Authorware training, and how dang logical it was. For me, the timeline was anything BUT logical.

When Flash became the de facto standard for learning development, chaos ensued:

• Constant player updates adversely affected courses designed in earlier versions of Flash
• Just TRY to get Flash to communicate with any LMS without having a programming wizard (if you could find one). I used to beg Andrew Chemey (a Macromedia engineer) to help, and he could always make it work, but he could never explain how.
• Say goodbye to Authorware's text handling capabilities and welcome Flash's completely horrible rendering of fonts.
• Throw out the idea of modular development, which was Authorware's coup de grace.
• Expect a new or heavily modified scripting language with major updates
• And the list can go on

All this for a "light web player" -- people complained about the 3-6mb Authorware web player (this was pre-broadband). Granted, Flash has made important progress as it relates to learning development, but this whole "Apple/Adobe" conversation got me to thinking about the state of tools in eLearning design and development.

Flash the parent, and all its unruly siblings such as Articulate, Captivate, Camtasia... all the tools that produce .swf files, are essential for various tasks in eLearning development. And Adobe seems to be whining that Apple is being a big bully about the Flash platform. I think it's actually the opposite. I think Adobe has been the bully over the last several years.

Apple is basically saying, "we don't want to have to adopt or promote a company's proprietary technology on our platform when it's not consistent with the technologies that serve as the foundation for our platform." Flash is anything but open.. it's locked technology that requires software available only from the vendor. It's not open source, or even open for that matter.

I remember the days before the Flash player (remember DHTML?). I remember coding HTML to work in various versions of browsers, and it wasn't pretty. Flash made life a whole lot easier on several fronts because it created its own "walled garden" that was guaranteed to operate consistently across multiple operating systems and browsers. I fear HTML5 will bring back the whole issue of compatibility and interoperability.

I also know from first-hand experience that creating a software product with the Flash platform is fraught with potential issues because any potential change in player versions could cripple features/functionality. And try getting updates from Adobe on changes, or roadmaps on functionality.

I tend to side more with Apple on this matter, only because they're really not bullying Adobe -- they're just saying, look we have our "walled garden" and we've decided what the best technologies are to support it. You have your proprietary technology that may be a standard with a specific group of developers, but it's not an open standard for the Internet.

What does this mean for learning developers? I think just as we all were forced to move away from Authorware several years ago, we need to have our eye on where the industry is going. I tend to think more open technologies will always win at the end of the day. But that must be countered with how we get our jobs done as well. Tools like Flash do allow developers to get work done in an efficient manner. Change is guaranteed, and learning new tools and languages will always be a task we must perform.

Designing for the Auto-Compulsion Loop (ACL)

Instructional Designers can take a lesson from the popularity of social games such as Farmville and Mafia Wars. These social games are played primarily through Facebook and allow players to play asynchronously in competition and cooperation with their friends. A recent study conducted by Popcap resulted in some interesting statistics (via Raph Koster's website):

• There are around 100 million social game players
  
• 24% of US and UK Internet users play social games at least once a week
  
• Slightly more women than men
  
• Average age of 43
  
• Only 6% of them are 21 and younger
  
• They’re not all housewives: 41% of them work full time, and only 11% are homemakers.
  
• 2/3 of them also play other sorts of games — casual or hardcore.
  
• And if they, do they spend more time on the social games than on the casual or hardcore games
  
• And they’re stuck: more than half have been playing for a year.
  
• A third play more than once a day
  
• 2/3 of them play over an hour a week — 12% report over 10 hours.
  
• And they report that their social game playtime is increasing
  
• For half of them, it’s the MAIN reason they use Facebook.
  
• And also, half of them say it’s very unlikely they will spend real money

The underlying phenomenon behind these statistics is in the factors that motivate players to repeatedly engage -- this audience isn't the stereotypical hardcore 20 hour a week WoW addict -- the motivating factors seem to be the granular achievements earned while playing combined with the inclusion of the player's real friends. 

Language Communities on the Web

Having spent many years in a language classroom, I was excited to discover the abundance of free language instruction on the web. The three I found the most impressive were Livemocha, Babbel, and Busuu. All three offer interactive language lessons for a variety of European languages while Livemocha includes a number of less traditional languages (such as Persian, Esperanto, and Urdu).  Busuu is based upon the European system of language levels (A1, A2, B1, B2…) and was selected as an official UNESCO project while Livemocha follows the American system of classes (101,102, 201…)

Here is a rundown of each:

Busuu tracks your language progress via a “language garden” where your plant has the opportunity to grow into a tree as your progress through the lessons. Busuu’s strength is vocabulary building since each lesson begins with a picture, pronunciation, English translation, and example sentence. It also attempts to have you learn the language via short passages and related questions.  Like Livemocha, there is also a writing section where other users rate your responses. This feature is the main advantage of a true online community where the website can take advantage of a speaker’s native fluency. Thus, the user becomes both the student and the teacher.

Babbel gives the user more freedom to adjust the language to their level and has a stronger focus on common phrases.  My favorite aspects of Babbel are its forum and the chatrooms. In the easy to use forum, you can ask specific grammar questions not covered in the lessons. For instance, what really is the difference between nouvel and neuf? Also, the chatroom is a quick and easy way to put your language skills to the test. It should be noted that although all three sites offer a great array of services with a paid membership, Babbel is the most limited for users not willing to pay.

Livemocha tries to incentivize its users to progress with a points system that also hands out awards at different levels. While it is rather basic, I can attest that I have continued to practice lessons simply to attain the next shiny medal available. The lessons are very structured. Each includes a section to learn vocabulary/phrases, review them, complete a writing submission, and record a spoken section. The last two sections are then rated by native speakers in the community. In a sense, this tool provides you with free teachers who correct your grammar and pronunciation. One area I would like to see them add would be grammar. This fall before visiting Turkey, I wanted to use Livemocha to learn a few phrases and sentences. However, I found it difficult to extract even basic grammar only from the vocabulary and sentences the site provides.

Creating Effective Training Evaluation

Dolly: Brandon ranted about ineffective evaluations while I was holed up under the 2 feet of snow here. The snow is still here, but my Internet is finally back. His comments about evaluation made me wonder what I consider effective evaluation to look like. Cumulative evaluation should be as authentic as possible. If you want someone to be an effective salesperson, you'd better identify what competencies are required and then have them replicate and practice as best they can in the classroom environment.

Brandon: Good point. I concur. However, more and more organizations are hit with the high-cost of travel and the inefficiency of removing people away from their jobs for instructor-led training. We need to provide effective strategies for authentic evaluation in an online format.

So I can think of two ways to recreate these situations in the digital world: virtual simulation and/or role-playing. Learners will have different profiles. Does someone who is technically minded need the same practice as someone who is naturally a people person? Both these individuals need to come out with the same skill sets at the end, but perhaps they need different practice. Formative self assessments combined with flexible course sequencing can allow individuals to focus on their areas of real need.

That just describes a sales training course I completed using Thiagi's 4-Door Model. It allowed the learner to "self-adjust" the content based on their own self-leveling of knowledge. It does beg the question though: how do you "branch" an evaluation based on different learner profiles? Can you have an effective evaluation instrument via a dynamic system that presents a contextual series of assessment items based on the learner's individual profiles? 

I think that you can have a self-assessment that gives the learner guidance in their own strengths and weaknesses.

OK, so you're talking about self-assessment. What about an actual skills certification? One that can affect a learner's job status, salary, or in the case of compliance or regulatory situations, a learner's knowledge that could have life or death consequences? Can a dynamic evaluation instrument provide the appropriate assessment of knowledge?

I don't know what you mean by "dynamic evaluation instrument".

"Dynamic pooling" is when the system displays content based on a learner's input at the time of input.

What you are talking about now is less about the assessment/evaluation used and more the consequences related to that assessment or evaluation. The SATs are pretty weighty and they've been using a dynamic response for years. A student gets progressively harder questions until incorrect responses are entered. Then a cycle commences where easier and harder questions are given to the student until the logarithm determines what the student's level of mastery is. I'd say that stakes are high, but the College Board feels comfortable using this dynamic response.

Right. So I'm saying we need a similar system in corporate learning where appropriate assessment techniques seem to be a missing factor. In eLearning, there is a cycle of ineffective self-check systems instead of situational problem-based assessments. For example, a colleague was discussing with me the usage of "misconception" problems in assessment. They can present more authentic situations such as posing the problem in a scenario and requiring the learner to identify the parts of the scenario that are wrong or inaccurate. Even using open-ended question types where the system evaluates based on a keyword or series of keywords can be quite effective. How are we truly able to measure whether knowledge or skills are transferred if we're not willing to properly evaluate?

I like that because some people are just natural or trained to be good test takers. That kind of exercise actually accurately tests knowledge that they have, rather than their ability to suss out a poorly written stem or item. I know you are all about performance. Shouldn't people's training be based upon actual deficits in their performance and knowledge? And if they are high performers or can demonstrate mastery of course materials, shouldn't they be rewarded by being allowed to choose their own training regimen, anyhow?

Designing A Degrading Learning Experience

Since Dolly is snowbound in Virginia without internet access right now, I decided to post this week without her... and she is sorely missed. 

Recently I was asked to perform some basic instructional design on an already created eLearning storyboard. One of the duties assigned to me was to edit existing content, and ensure the content was optimized for the engine the developers would use to build the course. As I went through the content, I decided to add a knowledge check interaction at the end of each module. Although these would not be mandatory interactions, they would provide an opportunity for the learner to self-check their learning, and to receive contextual feedback. Before I continue this story, let me digress just a bit about instructional design:

Earlier this year I was presenting to a group of technical writers and instructional designers on what instructional design is. This was a request from a group of technical writers who really felt like they were also performing instructional design duties when they created "tutorials" and/or "labs". They wanted clarification on what my idea of instructional design is.

I noted three primary responsibilities of a typical instructional designer:

1. Applies instructional strategies to aid in the acquisition, creation, retention and retrieval of knowledge.
2. Matches content type to strategy. Creates a learning environment by providing examples and non-examples for teaching concepts and facts.
3. Applies systematic approaches to instructional messaging to aid in learning (pedagogy).

There are a few key components that an instructional designer brings to a learning intervention:

• Designs the information to assist the learner in building on prior knowledge
• Scaffolds the information to enable new knowledge to be acquired
• Provides relevant, contextual feedback in the learning environment

OK, back to my main story... the course design assistance I was providing... Just remember the items I listed above, cause they come into play in this story.

So I was informed that new game templates had been created by the developer that I could leverage to make the knowledge checks more engaging. So, as a game designer, I thought great! Can't wait to get my hands on them. This was a basic "learning by listening" course presented in a PowerPoint-like template to begin with, so any form of interaction would help (I thought) -- until I lifted the hood on the "game" template and peered inside its functionality.

The game template provided was based on the standard Wheel of Fortune format. As you can see in the screenshot below (click the graphic to enlarge), it contains four primary components:

1. Question selection mechanism in the form of a wheel with a spinning arrow
2. A fixed region for the item types. Two types of items are available: multiple choice with single or multiple correct response options, and a true or false type
3. A timer
4. A score indicator

 

The template affords eight game questions. The learner clicks the "SPIN" icon, the arrow spins and randomly lands on a question number. The question is presented in the fixed question region and the timer begins to count down from five minutes awaiting the learner's response which requires selecting a choice, and then clicking a Submit button.

Your first thought may be similar to mine when I came upon this game-like interaction: a nice graphical interface and higher levels of interactivity affording the learner the chance to actually do something. Great!

But as I began to develop the test items, I dug a bit deeper into the functionality, and quickly realized this interaction was not just instructionally unsound, but detrimental to the learning opportunity at hand. A templatized interaction should first strive to be instructionally sound, and then be visually engaging (if appropriate to the learning context). This template has the following problems:

• A game should heighten emotion. It does not meet the basic criteria I establish for a game. A game has to contain an element of play. There are many definitions of play, but one component I look for is heightened emotion. To heighten emotion there must be a compelling desire for the learner to want to engage. Think of the standard Wheel of Fortune game: the choices on the wheel contain different types of elements, such as values (5000 points, 100 points, 300 points) that denote item difficulty. There are also spoilers, such as "Bankruptcy" (forcing the player to lose all their points). Those elements heighten player emotion after spinning the wheel.. "do I want the 5000 point question... oh gosh, I hope it doesn't land on bankruptcy..." This game simply has question numbers as wheel choices. It would not be difficult to change these choices to make them more game-like.
• A game should contain an element of unpredictability. This template displays the same questions in order each time it's played. It does not randomize items from a larger pool. So every time the learner plays, question 1 is question 1. This corrupts the basic tenet of the game: it should easily be able to provide a "new" experience each time it's played. This is not hard to program into the Flash engine. I've done it. This feature would encourage the player to "play again" which potentially increases their ability to learn.
• Visuals should always support the learning content. I wrote feedback for each item before discussing any limitations with the developer (I never thought my feedback would be limited). After the developer received my items with feedback I was told that I must limit the feedback to 108 characters (not counting spaces). By limiting the feedback space, I am forced to NOT display all the feedback I may need for the learner to gain a deeper understanding (which really is the point of the whole interaction, right?). By simply reducing the size of the wheel graphic, I could gain maybe another 100 characters for feedback. The wheel is only the conduit to display the question in this representation anyway, so having it take up MORE screen real estate than the question box itself is just bad design. Reduce the graphic, and give me more room for content. 
• Adult learners need to know there's a reason for what they're doing, otherwise, they will become distracted, bored, and/or question the authenticity of the intervention. To attract the learner to a richer level of engagement, an element of competition is desirable. With multiple rounds, for instance, the player can achieve a higher game level or at least compete against the game for points. This can also provide a scaffolding opportunity. In this template, the score is meaningless -- there is no leaderboard, no saving of the score, and no demonstrable representation of the value of the scoring algorithm. The learner needs to easily understand the "value" of their score. What does a score of "30,000" mean anyway?
• As an instructional designer, I have a better idea of the time element required for the item other than an arbitrary number created by a programmer. When I wanted to control the timing options for the game, I was told that the timer amount was fixed at five minutes. I may have question items in a True or False format that require just a bit of time, and not the whole five minutes. Sure the player can answer quick, click Submit and move on... but with 5 minutes on the timer, where is the challenge?
• Every item type should support learning outcomes. True or False item types simply do not. I can't defend them. This item type is just not instructionally sound. What data does it give the learner? It does provide them a 50% chance of gaining more points. These item types should never be used.

Games designed for learning can be powerful interactions to help attract and engage learners, and have shown to be powerful agents in building problem-solving skills. I have nothing against game "shells" or templates (I worked with Thiagi to design game shells years ago that are still in use today), however, if divorced from sound instructional design, they can be demotivating. If you decide to use templatized game engines be sure to validate the instructional integrity before you adopt and use them. The best is to create them from scratch and design them to your unique requirements. Templates can provide a solid foundation for reusable, rapid development, but they should never take precedence over the ability to create an instructionally sound learning intervention.

Now is the Time to Do Less of More

This week we struck up a conversation about the general state of eLearning design in the corporate world. We ruminate over our belief that companies should consider doing less of more:

BRANDON: So I thought this week we would discuss the overwhelming number of requests for "training" that come from business units and stakeholders. It seems like some people think "training solves all the problems."

I've been on the inside and the outside of several corporate learning organizations over the years, and one trend that I've seen explode recently is a "factory mentality" designed to "templatize" training. I've seen these factories operate almost 'round the clock packaging "rapid eLearning" courses with little regard for formative or summative analysis, or interaction that motivates a learner to participate. It's really a "page turner" world out there in many instances. Some of this is pure economics -- many learning organizations are funded by business units. If a business unit allocates a specific amount of dollars for "training" they expect those dollars to actually be spent, regardless of the necessity or quality of the training.

Why have training if it's not good? I've counseled the organizations I work for/with to do "less of more". I firmly believe if most training organizations just stopped producing about 40% of what they are doing today -- just stopped cold turkey -- no one would even notice.

DOLLY: I think one reason that training can seem so irrelevant is because it is so divorced from the day to day activities that the jobs actually require. So, these people who are in sales for instance, talking and corresponding with clients everyday, come to training and spend 4 days listening to someone speak at them. These week-long explorations of Powerpoint slides don't engage the learner. They just become data dump sessions.

  Yeah, I agree. I personally think sales training should be high-touch, situational, and as contextual as possible. It seems to me that there are two avenues to drive down when producing sales training: basic "transfer of information" about products, services, etc., and scenario-based/role-play simulations that place the participant in authentic situations. At Sun, we leveraged the community for the transfer of information component by providing a user-generated content platform. Sales people ate it up. They could get small chunks of product information or sales techniques from experts in the field and download it to their mobile device. We then had "Sales University" for the mandatory accredited training courses.

And the thing is, if people are just going to be sitting in darkened rooms, why bother going to the expense of shipping them across the country? If it's just memorization of information, there's plenty of fairly easy and cheap ways to get that across in an online setting. However, both face to face and online activities can be so much more engaging and rich.  

Agreed. One thing that seems to be missing is an evaluation of actual sales skills done in a formal manner. If we're training these folks to sell, then we need to really look at each individual and evaluate their readiness to sell. It's one thing to lecture them, have them role-play, provide feedback, and then send them on their way. Where do we assess their readiness to do their job? Is that for their manager to determine outside of the training? If so, is training's role just to "provide the foundation"?  

And  evaluation doesn't have to be a test. It can be a demonstration, a portfolio or series of smaller activities built into the curriculum.  

Yes, one vendor I worked with provided role-playing scenarios in an online format using computer webcams and FlipCams. Participants would video themselves doing their pitch, and then upload it for the cohort and the facilitator to critique. 

In my eyes, that's a perfectly valid evaluation, as long as the learning objectives were to improve their sales pitch, not to learn the capitals of African countries. The whole point is to have evaluation that supports your objectives and curriculum. Excellent evaluation can be painless and seamless.