Can management ever be truly evidence-based?

Evidence-based management has been making headway in recent years. I first wrote about this three years ago after some conversations with my former tutor Rob Briner, who is an enthusiastic advocate.

If more managers are looking for evidence before introducing something new to their organisations, this is an improvement. For too long, executives involved in the management and development of people tended to seize on every fad and quack remedy that came onto the market. Provided it had a set of impressive tools, lots of easy-to-follow lists, simple categories into which you could slot people, a flashy slide presentation and an aeroplane-read book to back it up, managers bought it. Scientific evidence rarely came into the mix. In fact, a bit of new-age mumbo-jumbo was more likely to impress in certain quarters than mere science. If we are moving away from that and starting to look at some hard facts before jumping onto the latest band-waggon, that can only be a Good Thing. 

But can management ever really be evidence-based in the way that science based disciplines such as medicine can?

Scientific research is governed by a rigorous set of requirements; for example new research must be testable and repeatable. The problem for any research on management is that the conditions in a workplace are constantly changing, so repeating an experiment even with the same people on a different day might yield different results because other organisational factors could be contaminating the environment. It is therefore almost impossible to repeat any experiment in organisational behaviour in the same way that an experiment could be repeated in a laboratory. As time moves on, so do people. They might be the same people but they will have been influenced by any number of events between one experiment and the next.

The same applies to organisations. Because each organisation is made up of different actors, an intervention in one might yield different results from a similar intervention in another. This is why management research is more difficult to generalise. For example, a bonus scheme that motivates people in one organisation might demotivate them in another.

And, of course, there is the good old Hawthorne Effect. Sometimes, people just change their behaviour because they are being watched. In such cases, a researcher can’t be sure whether it was his intervention that caused the change or just the fact that he was watching. As Adrian Furnham says, “to observe is to disturb.” A researcher changes an organisation just by being there. As soon as he walks through the door, the organisation is no longer the same place. Contamination of the research environment is almost inevitable in any behavioural experiment.

Rigorous scientific experiments also require control groups. This presents a further problem for management research. To assess the effectiveness of an intervention, an organisation would have to use it for some staff while excluding others and do so repeatedly until the behavioural outcomes could be conclusively proved to come from the intervention. All fine in theory but the excluded groups would almost certainly complain about being excluded.

Furthermore, live management research suffers from restriction of range because firms only take on people they think are good. For example, to evaluate recruitment tools to scientific standards, managers would have to hire people who failed the process and measure their performance too. Only then could it be conclusively demonstrated that good scores predict good performance and poor scores predict poor performance. Again, few organisations are going to take that risk for the sake of scientific experimentation.

For this reason, a lot of research in organisational behaviour uses groups of students. This study on incentive payments quoted recently by Aditya Chakrabortty is a good example. It tells us how students from Chicago and MIT and a group of villagers in rural India responded to incentive payments. All very interesting and it sheds some light on how people behave when the pressure of high rewards is piled on. Does it tell us how people in Bank X or Government Department Y will respond to an incentive scheme? Not really.

Research in management, then, can never reach the same levels of scientific rigour as, say, medical and pharmaceutical research. The clean research environments,  control groups and repeatable and testable results that would be demanded from scientific research are simply unachievable in the field of organisational behaviour.

Management research is not an exact science. Unlike a doctor, who can predict with near-certainty how a human body will respond to a certain treatment, a manager or occupational psychologist can never be entirely sure how people will respond to a particular organisational intervention. Much as managers would love to be able to pull lever X and be sure of result Y, people and organisations don’t work like that. If ever I create a management tool with repeatable and near-certain results, I will let you know – from my tax-haven on the Cayman Islands.

None of this is to say that managers should not look for evidence before they develop new practices. The criticism that much of what is claimed to be leading-edge management has no evidence base at all is well founded. In fairness, advocates of EBMgt like Rob Briner don’t suggest that managers should look for conclusive evidence just that they should pay a lot more attention to research before they introduce new techniques into their organisations. 

Perhaps we should think of the difference between evidence-based management and science-based disciplines such as medicine in similar terms to the difference between civil and criminal law. Medicine requires ‘beyond all reasonable doubt’, for evidence-based management, the balance of probabilities will do.

If we don’t adopt such a view, evidence-based management could become yet another excuse for not taking action. The public sector, especially, suffers from analysis paralysis and the postponement of decisions until more data can be gathered. Anything that reinforces such tendencies should be resisted. The futile search for conclusive proof could be used by some people to put off important decisions indefinitely. Combine the requirement for evidence with the legalistic internal processes that bedevil some parts of the public sector and inertia would rule for ever.

A ‘What If’ piece in yesterday’s XpertHR from Paul Kearns shows what could happen if managers were to become too purist about EBMgt. His elegant ‘reductio ad absurdam’ describes a dystopia in which executives decisions and membership of professional management bodies are subject to the same disciplines as those which apply to medical profession. In this world, the evidence behind all management techniques would have to “satisfy a similar standard to the process of new drug trials in the pharmaceutical industry”. Of course, no management techniques would ever pass such a test, therefore nothing would ever get done and no-one would ever get accredited to Paul’s General HRM Council. Which is, I think, the point he is trying to make. Take something to an absurd conclusion and you get absurd results.

Management is driven by fashions and fads. More emphasis on gathering evidence before rushing to implement the latest technique would save organisations a lot of unnecessary expense. That said, certainty and predictability in management research, or indeed any research into human behaviour, are elusive. The best managers can hope for is to make educated and informed decisions on the balance of probability. Even though research into people management is becoming ever more sophisticated it will never be an exact science. Management will, as it always has, require a combination of clever analysis, good relationships, sound judgement and a lot of good luck.

This entry was posted in Uncategorized. Bookmark the permalink.

8 Responses to Can management ever be truly evidence-based?

  1. Pingback: Tweets that mention Can managment ever be truly evidence-based? | Flip Chart Fairy Tales -- Topsy.com

  2. Pingback: Can managment ever be truly evidence-based? - Rick - Member Blogs - HR Blogs - HR Space from Personnel Today and Xpert HR

  3. This is an interesting article and my thoughts are that getting to a point where we know fully how people are going to behave, is like looking for the elusive Higgs particle. However, whilst looking for proof that the standard model in physics is the right thing to do I do think we’re overcomplicating the ‘Art of Management’. Managers, and middle managers especially, have too many artificial constraints placed on them, which makes them ineffective.

    Managers are required within an organisation simply to help their colleagues get the job done by providing leadership and guidance. That’s really all there is to it but in the real world they are deflected from the task in hand by high levels of noise that really don’t help them succeed.

    One of my favourite examples is performance management. To me this is simply to categorise how well a colleague is doing their job and should be un-related to pay & bonus. Any HR folks out there will probably start going for their keyboards right now but please bear with me on this. The ideal for me is to have three categories which are

    1) Outperforming – Great this person should be earmarked for a promotion
    2) Doing a good job – Fantastic and thank you
    3) Under performing – Need some help or may be in the wrong job

    There is no end of year review, as the manager lets people know regularly how they’re doing so there are also no surprises. In the real world though performance is often linked to pay and bonus and with that comes a whole load of processes to follow which get more complex year on year. A good example of this is an IT company of around 4,000 people who run reviews twice yearly with one of them being linked to pay and bonus. There’s also a stipulation that managers hold monthly 1-2-1’s which are checked to make sure they happen. If you talk to your colleagues on a regular basis and don’t need a 1-2-1 you would still have to hold one to tick that particular box. The main point here though is how much time it takes colleagues to go through the process. After a bit of research it was found that each colleague spent around 5 days writing up their performance documents and creating personal & team targets. So for 4,000 employees this takes 20,000 man days out of the entire company which is huge chuck of time especially when the business is providing services to customers and most colleagues are on the front line. As a rough guide this is like being short of 90 full time employees over a year but in reality there will be a bulge upwards every six months for a period of two weeks. Worst case is during these periods it could reduce productivity by 25%.

    I have used this example to re-inforce the points made in this article and for me the research that was carried out into performance management in this organisation was too narrow and didn’t take into account the behaviours of their colleagues. Management for me is a combination of clever analysis, good relationships, sound judgement, a lot of good luck and I would also add a graduation certificate from the school of hard knocks.

  4. Pingback: Rick - Member Blogs - HR Blogs - HR Space from Personnel Today and Xpert HR

  5. Stephen Timmons says:

    The book by Pfeffer and Sutton “Hard Facts, Dangerous Half-Truths, and Total Nonsense: Profiting from Evidence-based Management” argues quite persuasively for evidence-based management, and, as you say, it’s got to be better than most current practice.

    However, the work of John Ioannidis shows that even evidence based medicine is not as scientific as most people think

  6. Pingback: The one where we’re divided by a common language | mastersorbust

  7. ptkwp2010 says:

    Evidence based people management is not nearly as elusive as the ‘Higgs Boson’ and can be found at a tiny fraction of the cost
    http://www.amazon.co.uk/Professional-HR-Evidence–Management-Development/dp/0415632323/ref=sr_1_1?s=books&ie=UTF8&qid=1361964641&sr=1-1

  8. Pingback: Cavalier or Zen? « West : People Matters

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s