For those on H809, H800 and H817 which are all current, as well as anyone else on the MAODE but treading water between modules, the following paper from Linda Price and Adrian Kirkwood will be of interest.

Using technology for teaching and learning in higher education: a critical review of the role of evidence in informing practice. (2013) Price and Kirkwood. From our very own Institute of Educational Technology

What forms of evidence (if any) have influenced teacher’s practices?

As a pragmatist I’ve always wanted to believe that decisions are always made on the best possible evidence; humans aren’t like that though. The e-learning industry was as much a part of the 2001 dot.com bubble as anyone else creating content and putting it online. Clients wanted even if they had no evidence that it worked or not and even once you have pages of content online for a while they wouldn’t listen if you said ‘this is all going to end in tears’ – or rather, questioning their motives, trying to understand where the value would come from. It came in time. Thought FT Knowledge pulled out and another site I was working on, Ragdoll, turned from an information portal to a sales platform for its TV shows … then an online TV Channel.

Now in week 4 of H809 we are preparing the first TMA. My approach has been to read as many papers as possible until a pattern starts to form. I could be reading short stories, or listening to rock ballads – the goal is the same, to see and understand the shape of good research.

This is a mixed-method study that includes:

• A Literature review – using a framework – 96 papers/reports reviewed
• A Questionnaire – analysed using content analysis – SurveyMonkey completed by 58
• Interviews – using inductive thematic analysis – 8 interviews conducted

I had thought these 96 papers would be given as references or in an appendix. I guess only those that are cited appear. I would have liked to see the SurveyMonkey questionnaire too. But would this mean there would be no need for the paper – just release all the research and data and let readers draw their own conclusions?

If that happened I wonder how many diverse views we would get from 10 or even 100 responses. However objective we try to be it surprises me how different reports can be, sometimes to the extent that I wonder if people have been looking at the same event. The human mind is a wonderful and contrary thing.

Re-enactments of traditional activities in different media formats.

In the medical professions research favours positivist experimental methods. From large-scale controlled quantitative experimental studies such as clinical field trials.

Mixed method = a pragmatist paradigm.
Methodological triangulationn = research from more than one perspective

Increasingly, though my interests are diverse, I do find the research done on the use of e-learning in medecine of particular interest. There is a greater clarity and objectivity where you have 1000 medical students put through a randomized controlled trial over several months and the outcomes on their knowledge, or recall of facts, can be tested in a formal examin. There are no ifs or buts about naming organs, muscles groups or bones in the human body. It becomes less certain if you are testing changes in knowledge in say sociology as a result of using student forums or blogs. As Dianna Laurillard says when people push for answers, ‘it depends’. The variables are many and complex.

‘If conclusions from each of the methods are the same, then validity is established’. (Price & Kirkwood, p3. 2013)

This is a pattern that I could see myself applying:

Sequential mixed-method design (Cohen et al., 2011, p. 25)

1. Literature review – informed who might be suitable for interview
2. A short practitioner questionnaire
3. Interviews with practitioners
4. Analytic coding (Cohen et al., 2011)

My wife does medical market research and has to take or create transcripts, as well as do the interviews sometimes, with medical specialists. Some 35 hour long interviews must then be analsyed using a system, currenly manual, where phrases and terms are categorised and clustered. From this an attempt is made to write a comprehensive and object report. I always thought she was having to write a paper or sometimes the equivalent of a thesis every few months.

Clients of course want the ‘heads up’ or the ‘abstract’ and the report reduced to a series of slides. They must read the report but they far prefer to have it presented to them. By the time you are ready to stand up and talk someone through the findings you ought to feel fairly confident about keeping it succinct.

QQ When contemplating using technology for teaching and learning, what do practitioners considers as evidence of enhancement?

After three years of the Masters in Open and Distance Education I am delighted to say that as a student I have got my head around all of these

E-learning artefacts that could be studied as such:

• Blended learning/e-learning/hybrid courses
• Audio/podcasts
• Video resources/lectures/games
• Multimedia tools
• Virtual laboratories/fieldwork
• Blogs
• Collaborative tools/wikis
• Online discussion boards/conferences/forums
• E-Portfolios
• Online courses resources
• Electronic voting/personal response systems
• Assistive technologies

I should go back and put these into a table to indicate where across H807, H800, H808, H810 and H809 I have done these. Some expansion could be given to forums. I got my blended learning not through the MAODE which is entirely online, but from B822 Creativity, Innovation and Change. There isn’t much use of video either – though these days through TED lectures and a few OU inaugural lectures you get a taste. For video and interactivity I did parts of a video-based Social Media course. I’m familiar with virtual labs from OU Stories in the press. I first used electronic voting in 1997 during a live, broadcast event at Unipart Group of Companies … and then during a day long workshop on Creative Commons at the Open University. I have seen assistive technologies in the IET Labs, but also on visits to special needs schools and of course, studied assistive technologies as part of H810.

There are Micro, Meso and Macro scales

• Accounts of innovation
• Lessons learned
• Changes in practice

Respondents were more likely to be influenced by direct contact with colleagues and by experience of engaging with relevant work or personal activities. (Price and Kirkwood p. 10. 2013)

• Institution’s Centre for Academic Development 40%
• Academic Colleagues 25%
• Departmental advice for e-learning 12%
• Inductive thematic analysis (Braun & Clarke, 2006)

This is revealing of human nature and human desires. Despite all the technology that might keep us at our desks there is still a desire to seek and take advice from another person. This is so much more apparent in the commercial world where sales people or project managers take clients through what e-learning technology can do, the strengths and weaknesses. Clients are then sold packages, platforms and tools. They want to hear from the experts, they don’t want to read the papers – or to go on a course (though a few do some or all of the MAODE).

Four themes were identified by Price and Kirkwood.

1. Nature of evidence and its collection
2. Use of evidence
3. Generating and sharing own evidence
4. Changes in practice

Teachers are more concerned about ‘what works’ while researchers are more concerned about ‘why it works’ (Hargreaves, 1997, p.410).

We are all guilty of having our own agendas and perspectives.

Practitioners preferred to consult an academic developer or colleagues for guidance, rather than reading journal articles. (Price and Kirkwood p. 14. 2013)

CONCLUSION

Educator may think they are ‘improving’ learning in that learners retain more, achieve higher grades and get it down smartly and for less cost – they key driver and outcome is for a more flexible offering that that offered previously.

The academic developer’s role appears to be key in mediating evidence for practitioners. (Just as, I would suggest, the commercial developer’s role is key in mediating evidence for learning and development managers in business). i.e. we won’t review the evidence, that’s your job. Sell us something that works, that we can afford.

‘A dissonance has been observed’ by Norton, Richardson, Hartley, Newstead & Mayes (2005)

– subjectivity in categorisation and sampling methods countered by pragmatist paradigm adopted in this mixed-methods approach.  (Price and Kirkwood p. 14. 2013)

REFERENCE

Braun, V.,  & Clark, V. (2006) Using thematic analysis in psychology. Qualitative Research in Psychology, 3 (2), 77-101

Cohen, L., Manion, L., & Morrison, K. (2011) Research methods in education (7th edition). abingdon, Oxon. Routledge.

Hargreaves, D.H (1997) Educational research and evidence-based practice. London. Sage.

Norton, L., Richardson, T.E., Hartley, J.,  Newstead, S.,  & Mayes, J. (2005) Teachers’ beliefs and intentions concerning teaching in higher education. Higher Education, 50 (4), 537-571

Price, L., & Kirkwood, A.T. (2013) (in press). Using technology for teaching and learning in higher education: a critical review of the role of evidence in informing practice. Higher Education Research & Development.