It’s a convoluted path from objective questionnaire instruments that only make sense in the subjective reality of respondents. Why not just enquire directly into the subjective reality of managers and staff in organisations, and bypass the "objective" instruments?
I am delighted that Tom Gibbons has joined the discussion and debate on this blog about the place of instruments in organisations (here) as well as other topics such as self organisation. Tom is Managing Director at TMS Americas, which is the organisation that represents the well-known Margerison and McCann Team Management Profile and associated instruments, so Tom is an expert on profiling. TMS is also well represented in NZ by TMS Ltd and I certainly like the profile well enough to have become accredited in administering it.
In a comment on my previous post on reification (what’s that?) Tom explained how he uses the profile as a vehicle for starting conversations that would not otherwise be possible. I think this is an admirable use for a profile, because I think that it is important in organisational change to foster new conversations. After all, people gain insights from filling in a questionnaire and then receiving feedback from someone on how they stack up in terms of the criteria of the instrument.
My comments in this post are not related only to the Team Management Profile, but to all psychological profiles, and I have experience of many. It is commonly understood that the participants in the instrument do not know what the criteria are at the time they fill in the instrument, and this is seen as an enhancement to the objectivity of the instrument. Many instruments are designed to obscure the criteria through, amongst other things, the format of the questions and through asking the same question in a number of different ways, for example through forced choice between two criteria. So, participants are told that they can’t fool the computer programme. For some instruments, the delivery of the feedback via computer programme is also seen as making the feedback more objective.
On two occasions, as part of (successful) applications for jobs (in different consulting firms!), I completed the Myers Briggs profile and the feedback I gained from the recruiters was that the test was just to make sure that "I was not an axe murderer." As if the Myers Briggs profile could tell a recruiter whether or not I would kill someone, and also whether or not I would choose an axe as my modus operandi. Perhaps it is no coincidence that neither of those firms are still operational. (Or perhaps the fact I joined them is why they are no longer operational!).
Perhaps you can dismiss my experience as as poor practice from those administering the profiles. Nevertheless, the same feedback from both about being an axe murderer? Why couldn’t one recruiter at least have chosen a different means of perpetration?
Getting back to my point, it’s interesting to me that the feedback from the "objective" computer (which after all was programmed by a person) is seen as more useful, the closer it is to the experience of the respondent. In other words, the more I can recognise my own personal (subjective) experience in the feedback from the instrument, the more "real" (or objective) the feedback seems to be.
In workshops, respondents are commonly asked a question such as "What is there in the profile that you recognise as you?" I know because I’ve done this myself in the past.
In other words, the result of these instruments is that those implementing them ask their respondents to consider their own subjective reality, to then turn it into an abstraction from their experienced reality by answering generic questions interpreted by a computer. This abstraction (i.e. you are an ABCD, or a swinger-expressive, or a chimp, or a triangle) is then fed back to the person and they are asked to identify with this abstraction. In order for the person to feel this diagnosis (your "type" or "preference") which is an abstraction is valid, it needs to be as close to the experience of the person as possible. Areas that don’t fit are conveniently explained away, or perhaps peer pressure plays its part in having everyone agree to the diagnosis.
What I have described is the following process of completing these instruments:
- Taking subjective experience of the participant (what the participant thinks their experience is).
- Asking questions that interpret the subjective experience of the participant in terms of the framework of the researcher (a person) who designed the questionnaire. This is what I mean by abstracting from this subjective experience.
- Feeding back the abstraction of the researcher to the participant in the form of the diagnosis (of which type / profile you are). Often the feedback in written form is the output of a computer programme (originally programmed by the researcher).
- The participant makes sense of the diagnosis (which is an abstraction) in terms of the experience of the participant.
- The diagnosis (abstaction) is given value by the participant depending on the degree to which is matches the participant’s subjective experience.
I can’t help but wonder if we’d be better to enquire into experience directly, with direct questions about the subjective experience of the participants. Which is what I find myself doing in my own consulting practice.
When we use questionnaires and suchlike instruments we are going to the abstraction of questions designed into an instrument, which by definition have to be designed so as to cover as many situations as possible, then we only believe the results if they are congruent with our own experience. Why not just bypass the abstract instruments, and explore together the reality of what is going on around us?
Do we need such instruments in order to start conversations that otherwise would not take place?
Hello Stephen,
At the risk of this response being a defense of these types of instruments simply because my company sells and uses them, I’ll go ahead anyway…
I would make a couple of points here. First, your position questions the usefulness of these types of instruments. I think it could be argued that your position is an abstraction on your part since it has to be passed through the filters of your experience before it can be expressed. This would mean any sense making of experience is to some degree an abstraction of the actual subjective experience itself. However, if enough people with similar experiences came to the same conclusion, you could say that a reliable pattern of sense making was in place and you could attempt to measure that pattern. The risk of course is that you eliminate the possibility of any novelty occuring with this measurement if you assume it has to be accurate all the time.
I think this is what you are describing when you talk of how these instruments often get used. However if you brand the use of these instruments globally as not useful then you may be doing the exact same thing, eliminating the possibility of novelty in their use.
Second, if people do not have a language to adequately make sense of their experience it is very difficult to have any other perspective than what they have now. These instruments can provide a different language to assist in the sense making of experience and if used this way I think they can be of value.
I do think that these instruments are not useful and even dangerous if used as labels or as a fact, which treats identity as a thing, and my sense is that this is the point of your inquiry above.
But to cast what can be helpful data aside because lots of people don’t use it well is not helful either. Perhaps it’s better to help them us it more effectively. Readers of your blog might be interested in our position on this, captured somewhat in our own blog entry http://tmsamericas.wordpress.com/2008/09/30/preference-fetishes-when-preference-data-becomes-more-inportant-than-whats-actually-going-on/
Comment by Tom Gibbons — September 16, 2009 @ 5:42 am