09 March 2006

On online surveys

This has little to do with teaching and learning, but a great deal to do with the credibility of on-line surveys (and hence with research methods).

I recently signed up with a survey company in order to manage a questionnaire to former students (which proved to be more problematic than I had anticipated, but that's another story). The signing up process included asking me whether I would like to earn money by responding to on-line surveys. I was a little intrigued, so I said yes.

Today I received the first request to participate. It proved to be a market "research" survey about chocolate and other snacks. I answered honestly, only to get to the end to get a message (sorry, I should have copied and pasted the wording) which said in effect that I was not the kind of person they wanted answers from! (So I would not get any credits for answering.)

From a methodological point of view, sampling from that small subset of consumers who sign up to respond to such surveys is very dubious in the first place (given that they in turn are a small subset of net-savvy people, who are in turn a small subset of the population --however defined--at large). But effectively throwing away answers which do not suit is the unforgivable methodological sin.

OK—they were only interested in regular consumers of chocolate bars and crisps and their preferences, andbut I buy them vary rarely. (I always buy a multi-pack of crisps at Christmas for some reason and then throw most of them away the next Christmas because I've just bought another pack... why?) But how legitimate is sampling from net-savvy, money-motivated geeks? Only if it can be demonstrated that those "qualities" are independently correlated with product consumption. Has that test ever been done? Who knows? I do know that I read the results of some very strange polls in .net magazine; I don't believe a word (or statistic) of them, but some credulous marketing executive might.

So what? That's their problem. I want valid and reliable research; trust me, I'm an academic!

05 March 2006

On a familiar assignment

A correspondent today asked about a familiar assignment on teacher education courses. He was asked to take a recent teaching session and to relate it to the learning theories he had been learning about (in this case within a thousand words). It's a sufficiently common assignment, although one which I have come to believe unhelpful, to pass on my response (with additions);

I replied:
  • "I've set exercises and assignments like this in the past (although not in 1k words!) From the tutors' point of view it seems like a good idea, encouraging reflection and application and the like. Unfortunately, the results are almost invariably disappointing, and that is not the fault of the students. It is because the assignment is based on a fundamental misunderstanding about the relationship between theory and practice.
  • Practice is a "blooming, buzzing confusion". It's multi-dimensional; it's protean; it's not repeatable; it's fragile--it always teeters on a knife-edge between success and disaster... And theory is neat, and almost static, and enshrined in supposingly authoritative books--and it focuses on just one aspect of everything which is going on in the class.
  • In other words, theories have a "range of convenience"; they are good at explaining some things, but have nothing useful to say about others. You cannot teach by applying theories. Gagne is the theorist who was most explicit about attempting to set up a template for lessons, with Ausubel coming up behind; but try to use their ideas to account for a whole session, and all you end up doing is explaining why you didn't follow their prescriptions. Just for fun, you might try applying my "mayonnaise" model (http://www.doceo.co.uk/tools/mayonnaise.htm) and seeing whether that works.
  • In terms of the three groups, it's important to note that they are not mutually exclusive. Following the "range of convenience" argument above, most self-respecting teaching will have elements of all three, but probably at different time-scales. Behaviourism is important second-to-second; cognitive theory informs session planning; and humanistic theory informs overall strategy (although of course it has implications down at the second-to second level).OK--now you are even more confused! Console yourself that you are confused at a higher level than before!
But there is more to it than that. The theory is supposed to be descriptive about effective teaching and learning. But we (teacher educators [or even "trainers"]) have made it prescriptive. In a broadly "scientific" framework, theory follows practice or experiment or observation; it does not dictate it. That's dogmatism.

No wonder so many teachers are dismissive of much of their training.

David Hume pointed out that you can't derive an "ought" from an "is" (http://www.doceo.co.uk/heterodoxy/) Theory in teaching is simply there to direct your attention to things which may not be obvious as you face a class, and to help you to move on from your initial natural preoccupation with your own practice to pay attention to whatever the students are learning.

In the final analysis, I could not care less whether the graduates of our courses can recite the theory, but I do want them to have used it to inform their practice--assuming, of course, that it helps to do that. Sometimes it doesn't. Much recent hoo-ha about "learning styles", for example, has really been counter-productive; see http://www.doceo.co.uk/heterodoxy/styles.htm but particularly the references at the bottom of the page.

Teaching is a craft, and just as your specification of behavioural learning objectives can never really aspire to create/generate/programme/whatever proficient practitioners in your own discipline, so it is with teacher education. Theory has its place, and it is an important place, but it is the servant of practice and experience.