Tuesday, August 14, 2007

Research and a three-year-old

The incident that triggered the thought process behind this post happened when my son was about 3 years old. This was the time when he was trying to figure out 'cause and effect' relationships. So he used to say things like "If I shout in the class, my teacher will scold me", "I ran very fast in the park. That is why I fell down" etc. Those days we used to have an evening ritual. I would put my son on my shoulders and go for a walk. This 'sitting on the shoulders' arrangement made conversations easy even when there was a lot of noise all around. So this led to a lot of interesting discussions. There is nothing quite like a conversation with a curious, confident and talkative three-year-old to force one to be aware of and to question one's assumptions!

These walks would take us near a manned railway crossing/gate. Since he likes to see trains, we would stand there for a long time. After a few days he told me about a 'discovery' he has made "The gate has closed. That is why the train is coming"! Now, we all know that the 'causation' (if any) is the other way around. But purely based on his observations this was not so. He sees one thing happening (gate closes). After that something else always happens (train comes). Based on his 'life experience so far' (or his understanding of the 'system'/'universe') it was reasonable for him to think that if something happens and something else always happens after that the first thing might be causing the second thing (this principle had worked for him in the two examples mentioned above - running in the park and shouting in the class).

So, how would I convince him that his conclusion was wrong? The only way that I could think of was to tell him about the larger system (the railway system in this case - that makes the trains run and the gates close). This solution 'worked' only because there was someone around who knew about the larger system. He could not have come to the 'correct conclusion' purely based on his observations and his life experience thus far (i.e. based on his understanding of the 'system' at that point) .

Now, if we look at the research in behavioral science (or may be research in general), often we don't have the luxury of fully knowing the larger system in which the phenomena that we are observing are happening. Also there might not be anyone who has an adequate understanding of the system to 'enlighten' us. Actually, such understanding might not even exist! (as all the 'possible' events/system behaviors might not have been observed or even taken place so far - e.g. unusual/rare events/system behaviors like those that could result from malfunctioning of railway signals, human error, train breakdowns, accidents etc. or events like 'two trains passing through the railway gate at the same time on parallel tracks' that could arise from from a peculiar/uncommon combination of factors - if we stick with our original example). Often, there is no way we can study the 'entire system' (actually it would be very difficult even to determine the exact boundaries of the relevant 'system' in a particular study). We might not be in a position to look at all the data. So have to decide what data we would study and what data we would leave out. This could bring in biases (e.g. selection bias, survivorship bias etc.) and limitations. Thus, there is a significant risk that we might make the wrong inference (since we are limited by our observations and our current level of understanding of the system).

In addition to this, there are the standard problems with spurious correlations, mistaking correlation for causation, determining the direction of causation ('A causes B' or 'B causes A' or 'C causes both A and B' etc.) and assumptions regarding the homogeneity/uniformity of the system (assuming that findings that are valid in one part of the system are equally valid in other parts of the system). Of course, there are ways of expanding both our 'current level of understanding' and our data set/observations (e.g. study of the existing 'research' in the domain- if relevant and available). But, if we examine most of the 'research' that happens within organizations (for diagnosis and decision making - to solve the immediate problems in particular organization contexts), the pressures of time and resources might dilute the efforts to expand the 'understanding and data set'. Again, it is possible that the 'system' might have changed (in subtle but significant ways - without us noticing it) from what it was at the time we studied it/derived inferences on system behavior. Considering the nature and pace of change in many of the human systems that we are taking about, this could pose a big challenge for making available 'valid actionable inferences' to guide our decision making. Keeping all this in mind, can we expect to do always better than what my three-year-old had managed to do?

Note: I am not saying that useful behavioral research can't be conducted in organizations. My point is just that it requires a convergence of 'realistic expectations', 'will' and 'resources' - which, unfortunately, is not very common in most 'real world' organization contexts. If the 'research problem' can be defined narrowly, I would not even rule out the possibility of 'experiments' (though 'experiments' might not be a 'politically correct' term in organization contexts ; 'pilot studies' might be more appropriate). If such experiments can be conducted in the filed of medicine (where - literally - 'life and death' issues are involved), why can't we try them in business organizations (with proper precautions)? Of course, the problems like the ones that I have mentioned above (e.g. too many variables, difficulty in conducting 'controlled experiments', insufficient understanding of the system, biases in selection of data, assumptions about homogeneity and stability of the population/system etc.) still apply. But we might still get some useful information and/or insights.

Any comments/thoughts/ideas?

See somewhat related posts here, here and here.

6 comments:

bombay dosti said...

Wow!!Chanced upon your blog from the HR Collective blog.
That was a beautiful post.
Its amazing the way you have linked your son's questions to reasearch methodology.
Thanks for making me think!!

Prasad Kurian said...

Thank you.

Kunal Gulati said...

This is quite an interesting piece...one can see a lot of parallels in our lives -

1) Superstitions: Cat crosses the street. Later that day, you fall ill. Does that mean there is a causal link between the two incidents?
2) Placebo: If you get well after taking a particular drug, does that mean it was the drug that cured you?
3) Lightning & Thunder: Does the former cause the latter as per the sequence of their occurrence?

More often than not, the expectation of one thing following another does not lie in the things themselves, but in our mind!

Prasad Kurian said...

Thanks Kunal.

If we stretch this line of reasoning (at the risk of 'mixing' matter and spirit) it can be argued that reality itself is a mental construct (or even a social construct). But I think that it is worthwhile for us to think about these things (even if they are illusions) while we are on this side of the grave as they are on the same plane of illusion as we are!

For a somewhat related post see

http://prasadokurian.blogspot.com/2007/01/of-everlasting-and-timeless.html

Kunal Gulati said...

On similar lines, this is what Descartes had to say -

1) When we dream, we feel we are experiencing reality
2) What separates our waking feelings from our dream feelings?
3) When I consider this carefully, I find not a single property which with certainty separates the waking state from the dream
4) How can you be certain that your whole life is not a dream

The line of demarcation between illusion/dream and reality is often blurred! In fact, when you dream in your sleep, you might as well conclude that your waking life state was a dream!

Prasad Kurian said...

Thanks Kunal.