From: Stephen Bounds a href="mailto:km@bounds.net.au" target="_blank">km@bounds.net.au>
To: KM4Dev a href="mailto:km4dev-l@dgroups.org" target="_blank">km4dev-l@dgroups.org> 
Cc: Arthur Shelley a href="mailto:Arthur@organizationalzoo.com" target="_blank">Arthur@organizationalzoo.com> 
Sent: Thursday, 2 May 2013, 4:52
Subject: [km4dev-l] RE: Measuring knowledge management: survey report for the KM4Dev innovation fund - #OEKM-KM

Hi Walter & Arthur,

Firstly, I wanted to congratulate Walter and the KM4Dev Innovation Fund 
on conducting the survey and the workshop and publishing the results. 
More of this needs to happen!

However, I wanted to make a couple of point about your list of 100 KM 
indicators.  My concern with these is that they are, at best, a proxy 
for an assumed business benefit.

I say 'assumed' because even if, for example, metric #26 "% of enquiries 
answered within X days" scored well, this wouldn't necessarily justify 
the expense of obtaining that number.  There is still the next step of 
turning that into a concrete benefit to the organisation.

Now _of the whole list_, only 3 (#90, 91, 92) directly demonstrate a 
business benefit. #94 squeaks in *if* you can robustly measure the 
savings (which is very difficult).  While they may be useful *internal* 
indicators of a knowledge program's success or otherwise, they are not 
strong external measures of success, because they start from an implicit 
belief that the activities being measured are a good thing.

I think it's important that we be cognisant of the difference and think 
about how we can develop more robust external measures of success.

Arthur: I'm curious about this quote --

> Many of the immediately observable benefits of good KM are intangible,
> such as collaborative development of ideas, leveraging one's network
> or COP to mitigate a risk BEFORE it becomes an issue, building trust,
> loyalty and belonging etc. these are not EASILY measured, BUT they
> CAN BE, as can the symptoms of their absence.

I really like your idea of tying intangible measures to tangibles, but 
are you aiming to prove a correlation between two measures or something 
stronger?

Cheers,
-- Stephen.

====================================
Stephen Bounds
Director & Principal Consultant
knowquestion Pty Ltd
E: sb@knowquestion.com.au
M: 0401 829 096
====================================

On 2/05/2013 10:24 AM, Arthur Shelley wrote:
> Nancy, Steve, Eric, Walter & Phillip,
>
> This report & discussion completely aligns with my experiences in KM in
> the corporate and education sectors as well. There is generally a
> passionate KM "tribe" who actively live the principles of KM and "the
> others". Members of the tribe often find philanthropic forums such as
> KM4Dev, actKM, SIKM Leaders ... to engage with others of their kind
> (because they are a minority group in their own organisation)
>
> In a good organisation, the tribe has sufficient members to impact the
> wider culture and "KM is practiced" (but sadly often not measured or
> actively discussed to highlight the true impact of what it does). Many
> of the immediately observable benefits of good KM are intangible, such
> as collaborative development of ideas, leveraging one's network or COP
> to mitigate a risk BEFORE it becomes an issue, building trust, loyalty
> and belonging etc. these are not EASILY measured, BUT they CAN BE, as
> can the symptoms of their absence.
>
> The best way to highlight the benefits are to build a culture of
> storytelling around the successes. It is not bragging, it is sharing to
> generate motivation and awareness amongst others. It is important to
> give the credit to those who are involved. That is, "look at what these
> people achieved through collaboration", not "look at how the KM team got
> them to be better". See http://organizationalzoo.com/blog/?p=82
>
> All intangibles can be made more visible through linking to plausible
> tangibles. For example, collaboration builds trust which increases
> loyalty and therefore lower staff turnover (which has significant cost
> benefits in lower training  & recruitment costs and the inefficiencies
> of new workers getting up to speed). Putting numbers on this that are
> reasonable estimates does get attention and has been successful in
> organisations I have been involved in. Just start looking at how the
> value can be created through KM and work back to find the success
> stories.  Then share the stories with the decision makers (not just
> amongst the "converted tribe"). I have examples of such success stories
> that link intangibles to tangibles if you want them.
>
> Arthur
> www.organizationalzoo.com a href="http://www.organizationalzoo.com/" target="_blank">http://www.organizationalzoo.com>
> Tweeting as Metaphorage
>
> On 01/05/2013, at 20:08, "Eric Mullerbeck" a href="mailto:emullerbeck@unicef.org" target="_blank">emullerbeck@unicef.org
> <mailto:emullerbeck@unicef.org>> wrote:
>
>> Hi Nancy,
>>
>> Regardless of how well or poorly development organizations do in
>> supporting country-level results, they will not get out of that
>> business, as it is the core of what they do (purely research-based
>> organizations excepted). But development organizations could easily
>> decide to get out of the business of (formal support for) KM, as it is
>> not perceived as being a core activity – even though we on this list
>> might believe that it is.
>>
>> Therefore I think Steve’s concerns are very much to the point.
>>
>> Best
>>
>> Eric
>>
>> *From:*nancyw@fullcirc.com <mailto:nancyw@fullcirc.com>
>> [mailto:nancyw@fullcirc.com]
>> *Sent:* 25 April, 2013 10:53 AM
>> *To:* KM4Dev
>> *Subject:* [km4dev-l] RE: Measuring knowledge management: survey
>> report for the KM4Dev innovation fund - #OEKM-KM
>>
>> Could it be that these results are more an overall indicator of the
>> state of development organizations than of KM? I think if you asked
>> these kinds of measurability questions about things other than KM you
>> would find similar challenges. Look at how little large development
>> organizations successfully work to support country level results in a
>> tangible, practical and measurable way. How many so called "knowledge
>> products" produced by development organizations even have clear
>> distribution strategies, let alone indicators of use and application?
>>
>> What about the issue of KM being something akin to "blood" flowing
>> through an organization vs programme?
>>
>> I think this may be a system problem, not a "KM" problem per se. That
>> said, I also think we are accountable and responsible to always moving
>> our own part of the practice forward. But I'd caution us to consider
>> the wider context beyond KM.
>>
>> The survey does what a good suvey should do. Provoke more questions!
>>
>> Nancy
>>
>> At 06:28 AM 4/25/2013, you wrote:
>>
>> Thank you Walter for this interesting and eye-opening report.
>>
>> I can only express my concern about the findings, which in my view
>> provide further fuel to those who question the merits of KM.
>>
>> We need to be able to show objectively measurable benefits which
>> result from KM work, as difficult as this is.
>>
>> I believe our community needs to be far more active in this area or we
>> may be swept away by the wind.
>>
>> Steve Katz
>> Chief, Knowledge Management and Library Services
>> FAO - Rome
>>
>> *From:* w.mansfield@lboro.ac.uk <mailto:w.mansfield@lboro.ac.uk>
>> [mailto:w.mansfield@lboro.ac.uk <mailto:w.mansfield@lboro.ac.uk>]
>> *Sent:* 25 April 2013 14:00
>> *To:* KM4Dev
>> *Subject:* [BULK] [km4dev-l] Measuring knowledge management: survey
>> report for the KM4Dev innovation fund
>>
>> Dear all,
>>
>> Earlier this year, with support from the KM4Dev Innovation Fund, we
>> circulated a survey on *The use of indicators for the monitoring and
>> evaluation of knowledge management and knowledge brokering in
>> international development
>> (see project proposal
>> a href="http://wiki.km4dev.org/The_use_of_Indicators_for_the_Monitoring_and_Evaluation_of_KM_in_International_Development" target="_blank">http://wiki.km4dev.org/The_use_of_Indicators_for_the_Monitoring_and...>
>> at
>> *http://wiki.km4dev.org/The_use_of_Indicators_for_the_Monitoring_and...
>> *)
>> *
>>
>>
>> Thank you to all those who took the time to respond to this survey and
>> to send us materials.
>>
>>
>>
>> We are now presenting the survey report which explores the challenges
>> and current practices in the monitoring and evaluation of knowledge
>> management and knowledge brokering.  It collates and presents the
>> responses of 68 knowledge and evaluation practitioners. Please find
>> below a link to this survey report:
>> KM4dev%20indicator%20survey%20report.pdf
>> a href="http://storage.ning.com/topology/rest/1.0/file/get/1560700717?profile=original" target="_blank">http://storage.ning.com/topology/rest/1.0/file/get/1560700717?profi...>
>> (http://tinyurl.com/cxbruvx)
>>
>>
>>
>> Warm regards,
>>
>>
>> Walter Mansfield and Philipp Grunewald (Loughborough University)
>>
>>
>>
>> Walter Mansfield