April 2016 / 41 posts found

  • 29. April 2016

    The art of being a subject matter expert on a project that uses worked examples

    Sprout Labs is embarking on building a new program that is based around an instructional design approach called worked example. This blog is partly for the subject matter experts (SMEs) that are working that project but I thought other people might find it useful as well.

    worked example 01

    A worked example is a step-by-step demonstration of how to perform a task or solve a problem. Worked examples are used extensively in the teaching of mathematics and are eminently suited to training in highly-technical areas as well as in teaching and learning for the high-level skills that professionals require. Jeroen J. G. van Merriënboer and Paul A. Kirschner talk extensively about worked examples in their book Ten Steps to Complex Learning. The worked example method is similar to the case study approach. The difference is that worked examples focus on the thinking and decision-making processes that an expert needs to apply. In other blog posts I hope to go into more details about what they are and how the can be used.

    Often in the highly complex technical areas that Sprout Labs works, the traditional approach has been to show the model or talk about the legalisations first. Just like in a university program, it’s about “teaching” an abstract model that can be applied later. Where this approach falls down is when the learner needs to make complex decisions that require a great level of tacit knowledge. What worked examples do is focus on the practise of applying the models 1st. At a later date the abstract model can be shown or even better have the learner mindmap, draw or write up the model that they saw in action.

    In past projects we have found that preparing the content for worked examples is now often not easier for subject matter experts. As an expert you need to slow down your decision making and think about the decisions that have often become automatic to you. Often you have become more used to talking about the models or legalisation that you are required to apply than talking about how you apply those models and legislation.

    A great way to think about how to prepare the content for a worked example is “telling the story about how you do a task”..Some statements that might help you are

    • The 1st thing I do is ..
    • At this stage I'm thinking about ..
    • With this stage I need to watch out for ...
    • Here I need to apply ...
    • Here I need to think about ...
    • This bit is easy because ...
    • One of the more difficult bits is ..
    • You might need to need to think about ...
    • Here I apply ...
    • When I see something I like ... I need to think about .... and check ...

    Later you might remove the “I” statements, but to start with they are great way to move away from abstract thinking and to start to thinking about “telling the story of process”

    In later posts I’ve given some examples of how we have used worked examples in healthcare learning and legal training.

  • 29. April 2016

    Focus on getting results – measure performance not learning

    Focus on getting results blog

     

    Introduction of the 70:20:10 model means many changes for learning and development teams, one of which is in evaluation. A common question asked in our webinars is, “How do you measure learning in a 70:20:10 program?” In this blog post I'm going to explore that question.

    Why do we measure learning?

    There are two broad reasons why learning and development teams measure learning programs:

    • as a means to continuously improve the program

    • to gauge the business outcomes that the program has generated.

    The problem with focusing on continuous improvement

    An end-of-program survey is normally the main method used to gain data for continuous improvement of a training program. The reality is, however, that these surveys tend to seek only information about components like the quality of the catering and the performance of the trainer. If it’s an elearning module the usability of the interface is often the subject.

    Some also include some type of self assessment of learning and behaviour change, for example was the course relevant, and what are you going to change in the workplace after doing the course? The data is collected and used to drive decisions about the next time the course is run. Getting learner feedback is important, but this is only one dimension of what learning evaluation can be.

    Collecting data based on reactions works okay in an event-driven approach to learning, but it doesn't work in a 70:20:10 learning model where learning is continuous and embedded into the flow of work. It's not a mature approach to learning measurement. What is amiss here is that this approach focuses on the measurement of the process of learning, not on the outcome. Learning should result in a change of behaviour and an on-the-job performance improvement. One way to think about learning is as an input for performance improvement.

    Measuring business outcomes

    The other way to think about learning measurement and evaluation is to focus on measuring the performance outcome from a program at an individual and organisational level. This is not always easy to do. I often talk about learning not being a straight line, by which I mean simply designing and implementing a program doesn't always mean you will get good results. Organisations are complex, and there are numerous factors that affect the success of a program. One learning and development manager I worked with used to run a coaching development company and was opposed to outcome-focused measures because he felt that as an external provider he couldn't influence all the factors.

    What I've found is that focusing on business outcomes means that in the early stage of designing a program the conversations quickly transform. Often there is a realisation that a training solution is only part of the whole, which quickly leads to an approach that is more focused on performance improvement. Focusing on outcomes is a great approach to gain support for a more integrated, holistic 70:20:10 approach. It's more likely to lead to conversations about the integration of learning into the flow of work, performance support, and how managers are involved in the implement and rollout of programs.

    The first challenge in a performance-focused evaluation approach is deciding what metrics are to be measured and how that data will be collected. Learning and development teams are used to collecting data from surveys that they control, or data from the learning management system, but are normally not involved in the collection of wider business performance data. A few approaches I've seen work well in the past are:

    • using business metrics that are already being collected and reported on

    • benchmarking approaches – where data is collected before, during and after the program has run.

    Focusing on performance outcomes means that the outcomes are clear to everyone including the participants, managers, and anyone involved in the implementation of the program.

    Measuring how often employees access a learning portal is measuring an input, not an outcome. While this type of data can still be useful for continuous improvement, a performance-focused approach would instead correlate the data about what employees are doing (the inputs) with what the performance outcomes are. This means you might look at a sample of employees who have improved their performance as part of the program, and then, if the program is focused on performance supports, you would look at what resources they have access to. A further interview about the factors that contributed to their performance improvement would be useful too.

    The 70:20:10 model means changes in the evaluation methods for learning and development professionals. It means moving beyond just collecting data about learning input, and it provides the opportunity to focus evaluation and measurement on the business outcomes. This can also lead to the wider business understanding of learning’s effect on business metrics.

  • 28. April 2016

    What can content curators learn from art curators?

    content and art curators

    In an earlier stage of my work life I was a full-time creative (now it's just a part-time activity). One of my roles during this time was as a freelance contemporary art curator. In this post I thought it could be interesting to think about an art curator and what a content curator might learn from them when it comes to learning design.

    In an earlier stage of my work life I was a full-time creative (now it's just a part-time activity). One of my roles during this time was as a freelance contemporary art curator. In this post I thought it could be interesting to think about an art curator and what a content curator might learn from them when it comes to learning design.

    What does a curator do?

    A curator creates new ideas by associating art works together in new ways. Generally speaking, you might think of art curating as comprising five activities.

    1. Choosing works
    The core of a curator’s job is choosing works for an exhibition. They work from a theme that might be inspired by an idea or an existing work. This process of selecting works is not about the quality of the work itself but about how it fits together with the other pieces in the show.

    2. Placement
    Most curation activities result in an exhibition in space, and a key skill in associating works effectively is in designing how the area within the gallery is arranged. Design and placement means making sure the individual works are set up in a way that maximises their impact. The parallel process in learning design is the sequencing of content and resources.

    3. Providing a context
    Most exhibitions include a catalogue – a list of the works in the show – usually with one or more essays about the show’s themes or ideas. The catalogue acts as a record of the event but additionally the essays provide a context and background for the ideas.

    4. Logistics
    The day-to-day work of curating is "making it happening", which means things like arranging contracts, organising shipping and managing budgets. For a content curator these logistics are the nuts-and-bolts activities like adding resources to a database.

    5. Expanding activities
    Even with most traditional exhibitions in a gallery it’s not just about collecting items for display. The exhibition can be expanded in several different ways.

    • Events – these range from a talk by the artist to hosting a full conference.
    • Education kits – customised information for teachers and students at different levels. Another way of thinking about education kits is re-packaging the ideas into a different format for a new audience.

    Often when people get started in content curation the focus is on just choosing content, but lessons from the art curation world teach us the following things.

    • Think more about how resources are presented and how the presentation process can transform them into a distinct learning sequence.
    • Provide more of a context for the collection of ideas, e.g. write simple guides for reflection on why the resources have been chosen.
    • Think about how the collection could be expanded to include an event or two.
    • Re-package the collection for different audiences.

    Types of art curators

    Collectors

    When we think about curators we generally think about them as being collectors. They come across a work and find that it is interesting, or relates in some way to another work they’re interested in. The equivalent process in content curatorship is that of finding a resource and tagging it.

    In art curating there are generally two approaches.

    1. Artist driven. In this approach the curator is really a servant tasked with displaying the artist’s work in the best possible light.
    2. Idea driven. This is centred more around the curator and what they want to communicate. These types of curators are often known as “hunters”.

    Hunters

    Hunters are similar to collectors but are more focused and targeted. They have a strong idea they want to communicate and they look for a certain type of artist to help them showcase it. The way they work is less random. They might also be what we call a “commissioner”.

    Commissioner

    A commissioner will approach an artist with an idea to gauge their response or see if they are interested in making new works for the show. This approach is highly centered on what the curator is interested in.

    Archivist

    The archivist is more likely to work for an institution. Their interest generally lies in making sure creative works are not lost. The challenge for the archivist is in choosing which works need to be collected and preserved for the future.

    I think most content curators for learning fall into the collector or hunter category. For content curation to be valuable to an organisation it takes the focused mindset of the hunter.

    In another blog post in this series I’ve talked about the act of content curation offering a powerful learning experience for learners themselves. For example, as part of a development plan a learner could research a collection of resources that might be useful for the rest of the team.

    The commissioning of resources isn’t a common approach in content curation, however it does offer potential, for example in building a resource portal or repository. The mindset of an archivist could be useful for knowledge sharing and knowledge capture.

    Thank you to Sam Johnstone, one of the Sprouts, for his input into this blog post.

    Related posts

  • 28. April 2016

    6 Tips For Pricing eLearning Services: A Practical Guide For eLearning Professionals

    Pricing your own eLearning services can be one of the most challenging aspects of becoming an eLearning contractor, freelancer, or vendor. Too high and you cut yourself out of the eLearning market, too low and you won't cover all of your costs. In this...
  • 27. April 2016

    5 Ways To Test Online Training On A Tight Budget

    Your online training program needs to run smoothly right from the start, but how can you ensure that it will achieve the desired results BEFORE deployment? There's only one way to find out, and that's by putting it through the pre-launch paces. In this...
Load more posts
s2Member®