Measuring without changing
Jul. 29th, 2007 12:31 pm![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
andrewduckerProblem: Some people are crap at their job.
Solution: Find out which ones and either get them to improve or get rid of them.
Problem: Identify which ones are crap at their job.
Solution 1: Measure people's results and compare them to expected "good" results.
Solution 2: Have senior/experienced people look at the way people work and tell them when/where they're going wrong.
Problem with solution 1: Measurement is an overhead, causing inefficiency by taking time away from real work, and annoying people who just want to get on with their jobs.
Solution: Keep measurement to a minimum for people that are doing a good job, concentrate measurement and oversight on people who are less experienced or who have had recent problems.
Other problem with solution 1: Imposing metrics causes people to produce results that match those metrics - "Teaching to the test". (Measure coders by the lines of code they produce and they will produce ridiculous amounts of it.) As most jobs are too complex to be reduced to a series of metrics, introducing strong ones either completely distorts the work produced or causes large amount of extra work as people both fulfill the metrics _and_ get the work done.
Solution: Not at all sure. Reducing measurement for people that are doing a good job would help, but having metrics at all will distort things in unwanted ways. I'm not at all sure this is solvable.
Problem with solution 2: Experienced people are frequently wedded to their own methods, corrupt or otherwise biased.
Solution: Sufficient openness might mediate against this - and this is something that I'd like to see tried. Getting responses from all the people that deal with a particular person would hopefully compensate for any one person's bias. And anonymous feedback methods can be very helpful (although frequently hated by the people being reported on - sites that allow students to give anonymous ratings of their teachers have caused a massive fuss recently).
Overall I can't think of anything bulletproof. Anyone got any thoughts?
Solution: Find out which ones and either get them to improve or get rid of them.
Problem: Identify which ones are crap at their job.
Solution 1: Measure people's results and compare them to expected "good" results.
Solution 2: Have senior/experienced people look at the way people work and tell them when/where they're going wrong.
Problem with solution 1: Measurement is an overhead, causing inefficiency by taking time away from real work, and annoying people who just want to get on with their jobs.
Solution: Keep measurement to a minimum for people that are doing a good job, concentrate measurement and oversight on people who are less experienced or who have had recent problems.
Other problem with solution 1: Imposing metrics causes people to produce results that match those metrics - "Teaching to the test". (Measure coders by the lines of code they produce and they will produce ridiculous amounts of it.) As most jobs are too complex to be reduced to a series of metrics, introducing strong ones either completely distorts the work produced or causes large amount of extra work as people both fulfill the metrics _and_ get the work done.
Solution: Not at all sure. Reducing measurement for people that are doing a good job would help, but having metrics at all will distort things in unwanted ways. I'm not at all sure this is solvable.
Problem with solution 2: Experienced people are frequently wedded to their own methods, corrupt or otherwise biased.
Solution: Sufficient openness might mediate against this - and this is something that I'd like to see tried. Getting responses from all the people that deal with a particular person would hopefully compensate for any one person's bias. And anonymous feedback methods can be very helpful (although frequently hated by the people being reported on - sites that allow students to give anonymous ratings of their teachers have caused a massive fuss recently).
Overall I can't think of anything bulletproof. Anyone got any thoughts?
![[identity profile]](https://www.dreamwidth.org/img/silk/identity/openid.png){kind=small}
no subject
Date: 2007-07-29 11:58 am (UTC)no subject
Date: 2007-07-29 12:03 pm (UTC)I know professions differ a lot so it's hard to generalise, but for computing say, I don't think these things are a major problem. In software engineering if you're running your team / group / company well you will have statistics being generated at various levels that can be interpreted by senior people alongside an understanding of what tasks are being done and how people are doing them. Together it's not that hard to work out if people are crap or not.
As long as the manager in question has good domain knowledge and an awareness of what their team members are like, and are doing, it shouldn't be too hard to work out if someone is 'crap'. Working out who is crap, who is average, and who is good is fairly easy IME. Trying to be more specific than that is rather hard though.
no subject
Date: 2007-07-29 12:09 pm (UTC)You have to measure to identify people who are highly competent at their job (hence, reward them accordingly) against people who need support to achieve their potential (hence, offer them extra training or an easy way out). There is no other way of fairly determining what everyone is up to.
The best way of achieving measurement is through a "balanced scorecard", where you mix a selection of detailed, specific measures with performance indicators. For example, in the case of an IT function, you would combine pure financial measures (costs, overtime) with less specific measures of activity (problems resolved, time spent on projects, customer satisfaction). You report this with comparisons against prior periods, adding commentary to describe why certain targets where not achieved or what will be done to ensure acceptable levels will be met.
It'll take a while to determine what is an acceptable level of performance: the financial measures, for instance, would generally be taken against budget, but you would need to monitor activity over time to set agreed levels for the softer measures. However, the benefits are obvious - you can set targets, support under-achievers, reward above-and-beyond efforts, assist people at appraisal and reassure management that deviations are due to external circumstances, or are being addressed.
Make sure that those who are to be measured are aware of and involved in this process, so that "performance measurement" does not appear to be a tool with which to punish them...but, at the same time, they must show competence as an individual and as a team to gain reward. The end result should be something transparent enough to allow people to understand what they need to do and that you can't pad one statistic without impacting another (your example: you could produce many lines of code but the program will take longer to deliver; much better to produce tighter code which resolves a particular problem quickly).
The key element is having realistic targets - once those are in place, measurement becomes a routine activity with no further overload on those undertaking it.
Hope this helps - any number of management wankery books will contain schemes to measure performance. It's just a case of taking the best elements of several of them to show that you can performance both as an economic unit AND an individual.
no subject
Date: 2007-07-29 08:26 pm (UTC)no subject
Date: 2007-07-29 09:22 pm (UTC)And that's without even adding in the fact that managers, employers, and whatever other bosses you want to count, have their own ideas of how to do a job well that may or may not be in touch with things like, well, reality. Admittedly, making sure they do their jobs well is an ideal solution, but people who's job involves measuring others' work rarely like to turn a critical eye on themselves.