Promoting and Assessing Value Creation in Networks
* Article: Etienne Wenger, Bev Trayner et al. Promoting and Assessing Value Creation in Networks: A conceptual framework.
By Julia Storberg-Walker:
'In 2011 the father of communities of practice (CoPs), Etienne Wenger, along with colleagues Bev Trayner and Martin de Laat, jumped in the fray to weigh in on the issue and published "Promoting and Assessing Value Creation in Networks: A conceptual framework." This article translated Kirkpatrick's evaluative taxonomy to the online community environment, and offered a framework for connecting community outcomes (for example, meeting a new contact) to positive returns (for example, a new sale is made as a result of the new contact) for an organization. Wenger et al's ideas are still in the percolating stage, but I believe they are on the right track. And, I know they are guided by the demand from practitioners to help them "prove" that communities do indeed contribute to the organizational bottom line." (http://elearnmag.acm.org/archive.cfm?aid=2222516)
Understanding the Five-Level Framework
Julia Storberg-Walker on Etienne Wenger's Framework for Assessing Value in Communities of Practice (CoPs)
"Let's first look at how the framework is designed. It is built from the premise that there are three pieces of a CoP:
domain (WHAT is the CoP about); community (WHO is sharing the domain); and practice (what the community DOES together).
These three pieces combine to define a CoP, and the evaluation framework is able to assess the value of each piece-if you do it right. Like all evaluation models, it is recommended that when you initially start to design your CoP you want to take into consideration how to evaluate it. How often have you been asked to design and deliver a program, and tagged evaluation at the end in a rush because of time or resource constraints? You know you don't get the maximum benefit…likewise, if you don't consider how to evaluate the ROI of a CoP when you design the community, you won't have the maximum benefit that the framework can provide.
The framework itself has five levels-four that were adapted from Kirkpatrick's model, and one that was added on top of pile (see Figure 1). The first level—related to the satisfaction level—is called "immediate value" and it assesses what just happened, for example, in a webinar. Did community members like it? Learn from it? Enjoy the facilitator? The second level is called "potential value," and I like to think of this as the new knowledge or understanding that is lying latent but ready to be put to use in the future. This level looks at change: Did the webinar change participants? Did it change how they related to each other? Did it provide them with new resources? Questions like these point to potential use, but do not measure actual use. The third level does this, and it is called "applied value" and this is where the model starts to become interesting to CEOs and others who sponsor CoPs. The applied value level captures evidence about how CoP members actually used the new tool, or collaborated with a new colleague, or applied the new knowledge/skill in service of the job.
You're saying yes, but, I'm still not getting hard metrics like reduced development time, improved efficiencies, or financial returns. The fourth level in the framework provides this, and the level is called "realized value." This is like Kirkpatrick's fourth level—it measures the difference made by the CoP in the work of the organization. Did the use of the tool (as measured in level three) produce an increase in efficiency? New sales? Reduction in cost? This level will be the most helpful because it produces evidence that the CoP is (or is not) producing a benefit—or value—to the organization.
The beauty of the first four levels is that they illuminate the path from attending a webinar, to learning a new skill, to applying the skill, and to measuring the change as a result of the skill. Each step along the way can be captured and measured by using the framework, and each step can be easily understood and described to decision makers. There is a huge amount of face validity to the framework, which helps decision makers to diminish their natural resistance to evaluation schemes that rely on inference and conjecture. In this framework, it is easy to see and accept the value creation stream that a CoP can create.
The fifth and final level may or may not be of interest to decision makers, but the level will be important to CoP members who care about the community. The fifth level is where the community changes as a result of the activity occurring in the first four levels. At this highest level, the framework examines changes in the community—norms, standards, practices, and thought leadership—that has occurred as a result of activity within the community. While there may not be an immediate financial reward connected to this level, the changes in the community will likely impact all other levels and change what members learn, do, and practice.
Combined, the five-level framework is a huge step forward in our attempts to evaluate, measure, and understand the impact that CoPs can have on people and organizations. Go visit Etienne and Bev's website, read the article, and watch Nancy's short video for more information. I believe you will be able to take away great ideas for how to measure your own community of practice." (http://elearnmag.acm.org/archive.cfm?aid=2222516)