(meanwhile, on the KM4Dev mailing list) M&E of KM - a fascinating discussion to follow

Hereby some of the messages exchanged until now. Follow the rest on: KM4Dev



From: mhetzel@asocam.org [mailto:mhetzel@asocam.org
Sent: 03 May 2013 20:42
To: KM4Dev
Subject: [km4dev-l] RE: Measuring knowledge management: survey report for the KM4Dev innovation fund - #OEKM-KM


Dear everyone,

I just looked at the 100 indicators and I am a bit disappointed. They do not help me very much in my work.


I am interested in KM for development and for development cooperation. Not in KM in generally.


I had mentioned in a previous post, the various levels that exist worldwide for the development cooperation (see above). For me a set of indicators should help to measure the "architecture"/the “design” of the KM stategy and KM activities in this system. For me, the key question is whether it has made a KM stategy/activities to different levels (which ones?), are there mechanisms established "to force" aid workers to deal better with information, knowledge and experiences through appropriate methods?, so that people who need support (often called "the poors") better get this support (more effectively)?


For me the question is not if KM, but rather how in the development system.


Once I read an interesting article (from 2003) that said that KM strategies in development cooperation shouldn´t be designed within organizational limits. I totally agree (see: Knowledge Sharing in Development Agencies: Knowledge Fortress or Knowledge Pool? Geoff Barnard, Paper prepared for the EADI/IMWG Conference, Dublin, September 2003)


To measure how many participants are in an online forum… realmente no sé dónde estamos.


Slightly confused greets Tina :-)



Levels of aid system and possible starting points of WM measures:

- Donor organization in the North

- Organization in the North

- Donor organization in the South

- Organization in the South

- Program level

- Project level

- Level Results

- Focus topic (who is participating? Which level?)

- Focus “in the field” (rather than office)

- Focus territory (for example province)

- Focus partner organization

- Focus target group





Martina Hetzel

Oficial de Proyecto


ASOCAM, Servicio de Gestión de Conocimiento

Intercooperation América Latina & HELVETAS Swiss Intercooperation


Email: mhetzel@asocam.org

Telf: (593) 2447292    Fax: (593) 2438545.  Dirección: Hernando Sarmiento N39-77 y Hugo Moncayo. Quito Ecuador


Visite nuestra página web: www.asocam.org

Apoyando el intercambio sobre Cambio Climático: http://www.interclima2012.info



De: p.grunewald@lboro.ac.uk [mailto:p.grunewald@lboro.ac.uk
Enviado el: jueves, 02 de mayo de 2013 9:47
Para: KM4Dev
Asunto: [km4dev-l] RE: Measuring knowledge management: survey report for the KM4Dev innovation fund - #OEKM-KM


Hi everyone,


Good to see that our report raises these discussions and I hope that everyone takes away from the reports and discussions something that they find useful.

I agree very much with Charles’s comment.


To me the tangible and intangible distinction has a lot to do with tacit and explicit knowledge (and information). The term knowledge management suggests that we are in the business of managing knowledge. Knowledge however is a very individual and highly contextualised thing (mostly tacit – difficult to externalise, write down, etc.). For management to occur there need to be objects (thatcan be managed). These objects can be either explicit knowledge or information (what we often call knowledge products) or processes (processes of learning, knowledge sharing, etc.).

Both of these are tangible because people meet and share knowledge and knowledge products are downloaded, printed, etc. This are the things we do not struggle measuring.


However, what we really want to know are how people’s (tacit) knowledge changes and how this influences what they do (policy, practice). These things are situated in the realm of the tacit and sometimes not even the person her-/himself can tell what influence a certain conversation, blogpost, etc. had on how they think and act.


Thus, I think that we have to be realistic about what we can achieve through M&E (numbers and stories alike) even if this means seeing (and selling) KM as a supportive function that tries to facilitate and improve processes that would be going on anyways.


I know that this is quite a controversial thing to say but I thank that there is two sides to the coin: 1. Being as good at KM (and M&E) as we can possibly be and 2. Managing our own and other’s expectations.


Best wishes,









From: nadejda_loumbeva@yahoo.co.uk [mailto:nadejda_loumbeva@yahoo.co.uk
Sent: 02 May 2013 14:06
To: KM4Dev
Subject: [km4dev-l] RE: Measuring knowledge management: survey report for the KM4Dev innovation fund - #OEKM-KM


Hi everyone.


Excellent discussion.


Just a point to throw in in the pot: As we monitor, evaluate and take stock of the benefits of our KM interventions, we should not just look at how KM has enhanced our organisations from the point of view of learning, collaboration, decreased turnover, etc (all very important things). We should also show how KM interventions (be those CoPs, workshops, trainings) have enhanced the delivery of our organisations on their strategic goals and objectives. In other words, we should show how KM contributes in a direct way to our organisations fulfilling their mandates.


I think this discussion is really important to our community and the future of KM.






From: dhewac@yahoo.co.uk [mailto:dhewac@yahoo.co.uk
Sent: 02 May 2013 14:06
To: KM4Dev
Subject: [km4dev-l] RE: Measuring knowledge management: survey report for the KM4Dev innovation fund - #OEKM-KM


Our collective headache is trying to achieve intangible outcomes using tangible things.  Investing X amount of dollars to improve people's lives; spending X number of days and dollars to persuade poor people to change their habits.



From: Stephen Bounds a href="mailto:km@bounds.net.au" target="_blank">km@bounds.net.au>
To: KM4Dev a href="mailto:km4dev-l@dgroups.org" target="_blank">km4dev-l@dgroups.org> 
Cc: Arthur Shelley a href="mailto:Arthur@organizationalzoo.com" target="_blank">Arthur@organizationalzoo.com> 
Sent: Thursday, 2 May 2013, 4:52
Subject: [km4dev-l] RE: Measuring knowledge management: survey report for the KM4Dev innovation fund - #OEKM-KM

Hi Walter & Arthur,

Firstly, I wanted to congratulate Walter and the KM4Dev Innovation Fund 
on conducting the survey and the workshop and publishing the results. 
More of this needs to happen!

However, I wanted to make a couple of point about your list of 100 KM 
indicators.  My concern with these is that they are, at best, a proxy 
for an assumed business benefit.

I say 'assumed' because even if, for example, metric #26 "% of enquiries 
answered within X days" scored well, this wouldn't necessarily justify 
the expense of obtaining that number.  There is still the next step of 
turning that into a concrete benefit to the organisation.

Now _of the whole list_, only 3 (#90, 91, 92) directly demonstrate a 
business benefit. #94 squeaks in *if* you can robustly measure the 
savings (which is very difficult).  While they may be useful *internal* 
indicators of a knowledge program's success or otherwise, they are not 
strong external measures of success, because they start from an implicit 
belief that the activities being measured are a good thing.

I think it's important that we be cognisant of the difference and think 
about how we can develop more robust external measures of success.

Arthur: I'm curious about this quote --

> Many of the immediately observable benefits of good KM are intangible,
> such as collaborative development of ideas, leveraging one's network
> or COP to mitigate a risk BEFORE it becomes an issue, building trust,
> loyalty and belonging etc. these are not EASILY measured, BUT they
> CAN BE, as can the symptoms of their absence.

I really like your idea of tying intangible measures to tangibles, but 
are you aiming to prove a correlation between two measures or something 

-- Stephen.

Stephen Bounds
Director & Principal Consultant
knowquestion Pty Ltd
E: sb@knowquestion.com.au
M: 0401 829 096

On 2/05/2013 10:24 AM, Arthur Shelley wrote:
> Nancy, Steve, Eric, Walter & Phillip,
> This report & discussion completely aligns with my experiences in KM in
> the corporate and education sectors as well. There is generally a
> passionate KM "tribe" who actively live the principles of KM and "the
> others". Members of the tribe often find philanthropic forums such as
> KM4Dev, actKM, SIKM Leaders ... to engage with others of their kind
> (because they are a minority group in their own organisation)
> In a good organisation, the tribe has sufficient members to impact the
> wider culture and "KM is practiced" (but sadly often not measured or
> actively discussed to highlight the true impact of what it does). Many
> of the immediately observable benefits of good KM are intangible, such
> as collaborative development of ideas, leveraging one's network or COP
> to mitigate a risk BEFORE it becomes an issue, building trust, loyalty
> and belonging etc. these are not EASILY measured, BUT they CAN BE, as
> can the symptoms of their absence.
> The best way to highlight the benefits are to build a culture of
> storytelling around the successes. It is not bragging, it is sharing to
> generate motivation and awareness amongst others. It is important to
> give the credit to those who are involved. That is, "look at what these
> people achieved through collaboration", not "look at how the KM team got
> them to be better". See http://organizationalzoo.com/blog/?p=82
> All intangibles can be made more visible through linking to plausible
> tangibles. For example, collaboration builds trust which increases
> loyalty and therefore lower staff turnover (which has significant cost
> benefits in lower training  & recruitment costs and the inefficiencies
> of new workers getting up to speed). Putting numbers on this that are
> reasonable estimates does get attention and has been successful in
> organisations I have been involved in. Just start looking at how the
> value can be created through KM and work back to find the success
> stories.  Then share the stories with the decision makers (not just
> amongst the "converted tribe"). I have examples of such success stories
> that link intangibles to tangibles if you want them.
> Arthur
www.organizationalzoo.com a href="http://www.organizationalzoo.com/" target="_blank">http://www.organizationalzoo.com>
> Tweeting as Metaphorage
> On 01/05/2013, at 20:08, "Eric Mullerbeck" a href="mailto:emullerbeck@unicef.org" target="_blank">emullerbeck@unicef.org
> <mailto:emullerbeck@unicef.org>> wrote:
>> Hi Nancy,
>> Regardless of how well or poorly development organizations do in
>> supporting country-level results, they will not get out of that
>> business, as it is the core of what they do (purely research-based
>> organizations excepted). But development organizations could easily
>> decide to get out of the business of (formal support for) KM, as it is
>> not perceived as being a core activity – even though we on this list
>> might believe that it is.
>> Therefore I think Steve’s concerns are very much to the point.
>> Best
>> Eric
>> *From:*nancyw@fullcirc.com <mailto:nancyw@fullcirc.com>
>> [mailto:nancyw@fullcirc.com]
>> *Sent:* 25 April, 2013 10:53 AM
>> *To:* KM4Dev
>> *Subject:* [km4dev-l] RE: Measuring knowledge management: survey
>> report for the KM4Dev innovation fund - #OEKM-KM
>> Could it be that these results are more an overall indicator of the
>> state of development organizations than of KM? I think if you asked
>> these kinds of measurability questions about things other than KM you
>> would find similar challenges. Look at how little large development
>> organizations successfully work to support country level results in a
>> tangible, practical and measurable way. How many so called "knowledge
>> products" produced by development organizations even have clear
>> distribution strategies, let alone indicators of use and application?
>> What about the issue of KM being something akin to "blood" flowing
>> through an organization vs programme?
>> I think this may be a system problem, not a "KM" problem per se. That
>> said, I also think we are accountable and responsible to always moving
>> our own part of the practice forward. But I'd caution us to consider
>> the wider context beyond KM.
>> The survey does what a good suvey should do. Provoke more questions!
>> Nancy
>> At 06:28 AM 4/25/2013, you wrote:
>> Thank you Walter for this interesting and eye-opening report.
>> I can only express my concern about the findings, which in my view
>> provide further fuel to those who question the merits of KM.
>> We need to be able to show objectively measurable benefits which
>> result from KM work, as difficult as this is.
>> I believe our community needs to be far more active in this area or we
>> may be swept away by the wind.
>> Steve Katz
>> Chief, Knowledge Management and Library Services
>> FAO - Rome
>> *From:* w.mansfield@lboro.ac.uk <mailto:w.mansfield@lboro.ac.uk>
>> [mailto:w.mansfield@lboro.ac.uk <mailto:w.mansfield@lboro.ac.uk>]
>> *Sent:* 25 April 2013 14:00
>> *To:* KM4Dev
>> *Subject:* [BULK] [km4dev-l] Measuring knowledge management: survey
>> report for the KM4Dev innovation fund
>> Dear all,
>> Earlier this year, with support from the KM4Dev Innovation Fund, we
>> circulated a survey on *The use of indicators for the monitoring and
>> evaluation of knowledge management and knowledge brokering in
>> international development
>> (see project proposal
>> a href="http://wiki.km4dev.org/The_use_of_Indicators_for_the_Monitoring_and_Evaluation_of_KM_in_International_Development" target="_blank">http://wiki.km4dev.org/The_use_of_Indicators_for_the_Monitoring_and...>
>> at
>> *http://wiki.km4dev.org/The_use_of_Indicators_for_the_Monitoring_and...
>> *)
>> *
>> Thank you to all those who took the time to respond to this survey and
>> to send us materials.
>> We are now presenting the survey report which explores the challenges
>> and current practices in the monitoring and evaluation of knowledge
>> management and knowledge brokering.  It collates and presents the
>> responses of 68 knowledge and evaluation practitioners. Please find
>> below a link to this survey report:
>> KM4dev%20indicator%20survey%20report.pdf
>> a href="http://storage.ning.com/topology/rest/1.0/file/get/1560700717?profile=original" target="_blank">http://storage.ning.com/topology/rest/1.0/file/get/1560700717?profi...>
>> (http://tinyurl.com/cxbruvx)
>> Warm regards,
>> Walter Mansfield and Philipp Grunewald (Loughborough University)
>> Walter Mansfield

Views: 93


You need to be a member of Knowledge Management for Development to add comments!

Join Knowledge Management for Development

Donate !

We all get a great deal out of our engagement in KM4Dev. Maybe you would be happy to express appreciation through a regular voluntary contribution. Crowd funding works! 

background and info



note if the donate link above does not work for you, click here on donate! and at the bottom of that page click on the donate logo



© 2020       Powered by

Badges  |  Report an Issue  |  Terms of Service