Planning and evaluating social learning

We talked about the new version of our framework in the last blog post.

Here is a video where we provide an overview this new version, along with an example of its applicability. Going through each cycle of the framework, we describe how it can be used for developing social learning and its ability to transform practice. We are writing this up, but this video gives you a preview of what’s to come.

Let us know how you think you can use this.


Below is the text of the video…

Transcript

Urgency
We’re living in a time when things are moving fast. The rules of the game are changing. Science is changing. Technology is changing. Geo-politics is changing.

Learning fast is the only mode of survival. But here’s the crazy thing: our models of learning have not kept up.

Old learning models
For many people learning starts with something that’s known. It’s then transmitted to someone who doesn’t know it.

But for the projects we’re involved in this simple view doesn’t work. In the real world things are too dynamic. And complex.

Need for new model
Knowledge doesn’t sit still. You’ve got to be on your toes – you’ve got to improvise, solve problems, strategize, jump on opportunities. And bring others along.

Learning that matters today is social, in real time, and inventive.

More often than not, what we need to learn is not yet known. And that’s why we need a new learning model.

Importance of learning models
Most people don’t think about the learning models they use. But it matters a lot.

Reward me for being good, punish me for being bad – you’re using a behaviorist model. You think that learning is molding behaviour.

Give your students a long explanation – you’re using a cognitivist model. You want this explanation to become a cognitive structure in their head.

Help your kid work out a question for herself – you’re using a constructivist model. You think it’s best people construct their own knowledge.

These are just a few examples. And most of us use multiple models. But they are important because they guide how you think you learn, how you foster learning, and how you evaluate it.

Framework
Let’s look at our framework which we use to address complex twenty-first century learning. It’s driven by data about what’s creating value for different stakeholders – and it’s responsive to a changing and unpredictable environment.

For us learning starts with a joint activity – conversation, design, problem solving, benchmarking – you name it.

This framework focuses on the value produced by social learning. It distinguishes between different types of value and models learning as the dynamic flow among them.

For us learning starts with a join activity. A conversation, design, problem-solving, benchmarking… you name it.

Immediate value
You meet others who understand you, talk shop, think together, have fun, get to know each other, feel inspired.

You get value from just participating. We call this immediate value.

Potential value
All going well, this activity gives you confidence, new insights, good ideas, new perspectives, unexpected solutions, a new contact …

You might even produce a document – like the people on this photo did.

We say that these things represent potential value, because they may – or may not – end up being helpful to you.

For many people learning ends there. Not for us.

Applied value
Let’s imagine now that you try one of those good ideas when you get back. You change your practice. Collaborate with someone you met.

In a traditional learning model this should be unproblematic since you already have the knowledge.

But putting something into practice is very creative. It involves a lot of learning and generates new knowledge.

We call this applied value.

Realized value
As a result of all this chain of events, you would hope to see some improvement in performance. Your own or your organization’s.

This we call realized value.

Loops
Of course, it doesn’t always work. It might lead to nothing, or it might lead to a disappointment. Whether it’s a success or a failure you need to feed that back because it’s an important piece of information that will lead to further learning.

This feedback creates loops that are a key dimension of the model. Learning has to go all the way into practice and then back. And then into practice again.

It’s these learning loops that make the learning relevant, adaptive and dynamic.

Story
We’ll take a pause here to walk through these cycles with an example from a recent project where this model was used.

This is Honorable Zitto,the Chairperson of the Public Accounts Committee in Tanzania.

The role of a public accounts committee is to hold the government to account for its use of public funds.

The project I was supporting brought together the Chairs of these committees from seventeen countries across Eastern and Southern Africa to learn together in different ways.

Hon Zitto was at one of the project workshops where he had some interesting conversations with other members about good practices in procurement.

We would say that enjoying these conversations with his counterparts from other countries gave him immediate value.

At the workshop he picked up something about the role of the National Audit Office that they didn’t do in Tanzania. It was that the Audit Office needs to scrutinize the procurement process before it’s completed – and not just afterwards.

Picking up an idea like that is a good example of potential value.

Back in Tanzania he heard something that now raised a red flag. The National Electoral Commission had procured election equipment – but the company that had been awarded the tender was the same company who had prepared the tender documentation.

A chance to apply what he learned.

Armed with examples of good practice from other countries in the region, he convinced his Auditor General to carry out a pre-procurement audit.

He also shared the story with the media — another good practice from the network. This added pressure on the Auditor General’s Office to take action.

See how creative this is. It involves combining multiple insights and seizing the opportunity. This is why it makes sense to call it applied value.

Within a month, as a result of the The Auditor General’s action, the tender, which was valued at 126 million US dollars, was cancelled and re-advertised.

OK, so the story reached realized value. Some money was probably saved, but more important, a transparent process was put in place.

Was this fed back into the community?

We collected many stories like this as part of the ongoing project evaluation – and shared them with network members.

A bigger picture
This story brings another important point about social learning: it doesn’t happen in isolation. A story like this is embedded in a broader context — a context that involves many different stakeholders.

This project’s been going on for several years. It includes networks of Public Accounts Committees in Eastern and Southern Africa, their clerks, the World Bank, the German Development Agency and various local organizations.

A key factor in the learning potential of the project was the quality of the conversations among stakeholders to fit their activities in a bigger picture.

Strategic value
The quality of these strategic conversations is an integral part of social learning and one that’s often neglected. We call it strategic value.

There’s another one that’s often taken for granted. It’s the learning of the project support team and community leaders.

Enabling value
Yes, it couldn’t happen without a lot of support and leadership – logistical, coaching, facilitation, technology, agenda design…a lot of learning there.

It’s a key aspect of the learning process. We call it enabling value.

Transformative value
Learning is not limited to an improvement in performance. It can also generate new perspectives or new definitions of success.

It can even trigger broader cultural and institutional transformations.

We call this transformative value. It doesn’t always happen. But there is always the potential that it might.

And when it does, it’s often the most dramatic aspect of learning….

and – potentially – the most contentious.

Story
Another story from the same project but from a different member shows this transformative value and its potential to disturb the status quo.

If you had been in Zambia last year and happened to turn on the T.V. you might have seen government ministers being questioned about the use and mis-use of public finances.

What you might not know is that Hon Mwale, the Chair of the Public Accounts Committee in Zambia had been inspired by some other countries in the network to have these hearings televised. It became a very popular series.

Here we see learning even transforming the broader political culture.

Yes, BUT they became a political hot potato. Suddenly the broadcasts were stopped and the director who had set it up was in danger of losing his job.

Yes, transformative value often upsets existing power structures.

While there is still controversy about how and whether these broadcasts should be continued, another unexpected outcome has been a sharp increase in the number of whistle-blowers contacting the Public Accounts Committee.

That’s a very interesting story – transformative in many ways and very useful to other network members when it gets fed back.

Multiple pieces and a dynamic flow between them
For us all of these pieces need to be in place and with a dynamic flow among them if learning is to make a difference in today’s world.

Uses of the framework
People are using this framework in many ways.

The value-creation cycles show you where to focus your attention.

And value-creation stories like the one we just told you explain how your project is making a difference.

If you are planning a project you can use the framework to structure conversations with various stakeholders – set aspirations and decide what conditions need to be in place.

If you are running a project, you can use the cycles to design activities and stories to create ongoing feedback loops.

If you are evaluating a project, you can use the framework to structure your data collection and analysis. You can follow indicators at each cycle and use stories to attribute outcomes to project activities.

Closing
We find many people – across sectors – struggling to integrate learning in all phases of their project – from planning to implementation to evaluation.

Unfortunately, learning remains a side-show in many projects – a bit of training, knowledge-sharing, a piece of research, an evaluation report. But today embedding social learning has to be a strategic part of any innovative project.

Our framework does just that. It acts as a shared language for negotiating aspirations, for framing the design, and for driving the learning through ongoing feedback loops and data collection.

Many of our clients are placing it at the centre of their initiative. This gives us hope. It shows we can change the discourse on learning to one that is going to address the challenges of today.

25 thoughts on “Planning and evaluating social learning”

    1. I’m going to start a debate about these issues on mirandalink. I’ll report back…Very well explained – and vital in developing effective learning processes in a changing world.

    1. Thanks, Doris. BTW – we believe we still have your hat somewhere in the attic! If you find yourself in California, we’ll look for it and dust it off for you 🙂

  1. Chris Blackmore

    I enjoyed watching this video and hearing about your framework. I like the way it can be used to bring the different elements together, as your examples showed. Useful for us to think about this in the context of water governance in Europe where we have been focusing on increasing transformation potential of key stkeholders, aiming (but not always succeeding) to take a systemic approach. I will circulate it to my colleagues. Thanks! Chris

  2. Thanks for the video and explanation. I looked at your framework a few years ago for some research I did with IISD on designing networks. The video is a really useful addition to bringing the words to life. AND I’ll use it with researchers I work with to also show how one can communicate their research in different ways. Great work!

  3. Hi Michelle. We initially came up with the framework for our work with communities of practice and networks, but we have been suprised how many people find it useful for other applications including research. Thanks for dropping by!

  4. Bernadette Mercieca

    I like the idea of your value added framework and will be using it in my research on communities of practice for middle years Maths teachers in Melbourne.

  5. Loved the metal music! Kudo to Kenan!
    Obviously loved the content too!
    You clarified what happened in my CoP in the last 10 years.
    Participants saw the immediate value during the face-to-cafe meeting and potential value in their organization.
    Unfortunately the applied value was rarely seen. Lack of courage from their parts? Dysfunctional work environment?
    But at least one participant made it and went from Applied to Realized and saved 175 000$ in the first 9 months of implementation.
    Your model is wonderful!!

    1. Sometimes we find that applied and realized value does not appear much because people are not really aware of it or of the need to report it back. So if your community becomes more familiar with the idea of feeding back value-creation stories to create learning loops, you may starting seeing more of it.

  6. I am currently applying for a position at a small college in Oregon and have been asked to deliver a short presentation on a “tool” that the college might find useful. I am intending to deliver something on Communities of Practice and have been quite interested in your most recent material and framework.

    The story from Tanzania was impressive but it seems that the current framework seems as if this was simply pure luck that one member of the audit team discovered new information about audit practices which were then put into place and this becomes an investment in “faith-based” learning. I do not mean to demean the framework, only to better understand the practice.

    If this is a formulation for a new chemical compound we have some basic principles that will assist us in determining what chemicals we start with and what the end compound will consist of; the initial step of this framework seems to say just throw a bunch of folks together and pray that something comes out of the social interaction. It seems to me that using some of the “old fashioned” systematic approach to training (SAT) in which you do some level of analysis with respect to understanding the problem and at least starting with what would seem to be an appropriate mix of knowledge experts in a particular area might allow for one to prime the pump in order to ensure that there is a greater opportunity to move successfully through the latter steps.

    I realize that often times, in the current world scenario, that perhaps we don’t know what we don’t know and therefore don’t know what we need to know. Nevertheless, it seems to me that there needs some focus on a framework that assists in driving one towards a solution that provides value.

    Thanks in advance for your thoughts.

    pax,
    -rr

    1. Great questions….

      Are you suggesting that it would be good to propose some more specific learning objectives? Or are you proposing that there could be some guiding principles to help set the conditions? 

      In the framework we shy away from strict objectives. Rather, each value cycle has aspirations (end compound) and conditions i.e. what needs to be in place that would help achieve those aspirations. The activities might – or might not – include some content experts (each case is different). There might also be other types of experts in enabling and strategic value.

      The systematic design of learning activities is not really what the framework is about. We have another discipline for that. What the framework is about is that once a learning activity takes place, whether random or highly systematic do we have a way to account for the value it creates. Where the framework becomes systematic is in creating tight causal attribution and in suggesting loops that bring the value back into the learning process. 

      The systematic nature of the framework is in extending the design beyond immediate and potential value (which is where most training stops) and into applied, realized and the whole cycle.

  7. Thanks so much for your reply and for giving additional information regarding the value creation framework. The definition of aspirations and conditions adds significant understanding for me to the process.

    As I come from a background where we are somewhat rigid with the development of task analysis and learning objectives connecting those dots to learning outcomes is that whichI am most comfortable.

    In the initial example from Tanzania, I would assume that your client had a basic business background. Perhaps that was not true; at any rate while it would be possible to come up a solution by attending an art symposium or an animal husbandry seminar (and I simply extrapolating to the enth degree to stress the model) it would seem much more likely that a viable solution would come from attendance of a meeting around proper business practice, where again you are meeting with folks who speak your language, deal with the same issues that you do, and are trying to solve problems in the same general venue (per the definition of a community of practice).

    Thus, based on what you have said, it seems as if it is important to have a focus (in writing) of aspirations and conditions required to meet those aspirations. This to me is at least a basic road map, albeit written in a number two pencil.

    Thanks again for assisting in my developing my understand.

    pax,
    -rr

  8. ~BOOM!! You introduced this on Thursday when I was completely brain dead. It felt so familiar and I couldn’t put my finger on it. The key pieces of the framework (immediate value, potential value, applied value, and realized value) map well to Kirkpatrick’s 4-Level Evaluation model (http://www.nwlink.com/~donclark/hrd/isd/kirkpatrick.html). Kirkpatrick is designed to work with the type of training that RR refers to whereas your model (as you say) was developed for communities of practice and extends to landscapes of practice. However, for me, your model would be easier to implement in either location.

  9. Hi Barb, yes you are right. As we say in the 2011 publication (footnote, page 19) “The first four cycles in this framework are an adaptation of the four-level model of Donald Kirkpatrick (1976, 1994), which has become a standard in the training and program evaluation literature. In Kirkpatrick’s work, these four levels are called Reaction, Learning, Behaviors, and Results. Even though these terms do not apply very well to community/network evaluation, the categories can be adapted to address issues of value creation in communities and networks. This is what we have done here (see also Wenger et al., 2002). The fifth cycle is an addition we have made specifically for the work of communities and networks.”

    Not sure who RR refers to, but yes people working in many different contexts have said that it’s useful.

  10. Dear Bev and Etienne
    Can you believe it, your newsletter has been sitting in my inbox all these years – I had switched off notifications! (Always a little behind, me :-(. The 2015 Landscapes of practice framework has been my bible for research in the last few years. So wish I had picked this up earlier.
    Just wanted to say I remember your ‘Bev and Etienne-treat'(BEtreat) in the Stellenbosch mountains near Cape Town in 2013 with great fondness. (Did I win the race to the top of the mountain Bev?) The experience was a highlight in my own education, and the tools I learned continue to provide a beacon for understanding my students’learning. Anyone who can attend a BEtreat will find it just that, a treat:-)
    Best wishes, Claudia

    1. Hi Claudia

      Great to hear from you. And to know that our BEtreat has been useful, even a beacon!

      Hmmm, my memory is that I won the race to the top 😉

      Bev

    2. Hi Claudia

      Great to hear from you. And good to know that the BEtreat was useful to you, even a beacon.

      Hmmm… my memory is that I won the race to the top 😉

      Bev

  11. A colleague and I are about to submit an article about professional and identity development in a writing course that is embedded in the real time scenario of computer science in education during the 2016 protests in higher education institutions in SA. UCT had to close down campus for the last 7 weeks of lectures and turn to a distance mode (although they called it blended). Too much violence and damage to property…
    We’re also designing a course on team management and communication, seeking guidance from the value-creation framework…

  12. Pingback: UDL Applications to Practice: Secondary Transition, Career & Technical Education, and Career Assessment | Universal Design for Learning

  13. Pingback: Designing for change – | Wenger-Trayner

Leave a Reply to Anonymous Cancel Reply

Your email address will not be published. Required fields are marked *