Home » E-Learning » Evaluation
Category Archives: Evaluation
9 ways to create the perfect online course
Fig.1 Mosaic by featured in the University of Cape Town FutureLearn course ‘Medicine and the Arts’
Don’t call MOOCs MOOCs, they are ‘courses.’
Don’t even call them online courses. I suppose therefore, don’t call it e-learning either or even online learning … it is simply ‘learning’. I am on my eighth or ninth course with FutureLearn. I may have three or four open at any one time and complete two of these at least. I love ‘Medicine and the Arts’ from the University of Cape Town while I am both maddened and intrigued by ‘The Mind if Flat’ from Nick Chater. I’m certain that online courses longer than a couple of weeks should not be treated like books or TV programmes. What works best, as the University of Cape Town shows, is to get the entire team involved. They have a lead host and presenter who each week introduces several colleagues, something like four to six each week. It is stimulating and necessary to hear from so many different voices.
1. The Platform Provider
Brand and technical aspects. Think of this as the channel. It has both technical and brand qualities. Is it smart? Is it current? Does it all work faultlessly? Is it intuitive? Is it simple? I’ve done many FutureLearn courses but struggle every time with Coursera and EdX. Feedback on Udacity is dire from both suppliers and users.
2. Funding/Cost or Cost Benefit
You can’t make a movie in $125,000 dollars. If a 30 point 16 week distance learning course from the OU costs £1.5m to produce should a 3 week MOOC cost up to £300k? It’s a poor comparison is the cash cost may be a fraction of this: a university team’s job is to plan a programme of teaching anyhow. What matters is how a budget is spent. The learning designer for an online course is like the scriptwriter for a movie: they provide the blueprint. Is the investment worth it?
3. The Subject Matter
Are you true to your subject? Don’t try to be something you are not. Is it ‘made’ for an online course, rather than shoe-horned from a regular, traditional ‘classroom’ lesson plan? Would it be better served on a different platform in a different way? Can you teach sports coaches or movie directors online? Or rather, what can you, and what can you not teach them? Are you fully exploiting the affordances of the platform and easily linked to alternatives on the Internet.
Who do you attract and is this the same as who you get? Who do you attract by level of education, age, gender, culture and location. Are you getting the audience you want as participants? The contribution participants make is crucial. Are there enough active voices to sustain this? Be aware of the extreme differences in digital literacy skills and competences. Do you know your audience? How do you relate to those who start the course? Do you try to appeal to multiple ‘personas’ – a dozen student character types, as the Open University does, or do you appeal to one person only, as an author would do?
One advocate over more than a couple of weeks will tire. It will feel like an ego trip any way. How good is the mix of contributors? Both in what they have to saw and show, and their levels of and variety of experience. An online course is not necessarily akin to a TV documentary that can be carried by a single presenter. Is it a one man show or a team effort?
What are the hidden and implicit goals? To attract students, to build reputation, for the good of mankind? To make money? To massage an ego? What do results say in terms of those completing a course? Doing assignments and getting to the end then singing the praises of the team? Another guide can be whether as a production fulfils the initial Creative Brief. Both qualitative and quantitative research is required to provide answers.
7. Brand and creation/production values
Is is possible to stay true to your own brand, even have a distinct image, when on someone else’s platform? Are the values of the design, creation and delivery consistent with the standards and image of your institution? If not publishing, and not TV what is it? It cannot be a lecture series with a reading list and essay put online. It has to pick the strengths from individual media platforms to succeed in this multimedia setting.
These must never be taken lightly. There are examples of trite, ill-thought through multiple-choice quizzes: these are a learning opportunity. A good quiz makes you think, challenges your knowledge, and provides feedback whether you get it right or wrong. Bravely ‘Medicine and the Arts’ has both quizzes and a regular written assignments. These are not onerous yet some participants are scared by a 300 to 500 word piece of writing. They oblige you to read back through the week’s activities before replying.
How ‘sticky’ is the content? Has it got people talking to each other, not simply replying to the headline. Are people connecting as ‘friends?’ Are they continuing this relationship beyond the ‘walled garden’ of the ‘open’ learning site? Does interest in the subject, in the presenters and the institution ‘have legs’ – does it last for the years before a person may make the time, and raise the funds, to take a formal course?
How to take notes for effective studying and assessment
Fig.1. The muddy sides of the River Ouse, Piddinghoe. At low tide.
We are very good at forgetting: it’s vital.
We see, feel, sense far too much in our daily lives (which includes when asleep). Come to think of it what on earth was I doing on a student exchange to North America last night where I am twenty years older than my hosts … (probably sums up how I feel about the workplace).
See. Some memories are made for us, or by us whether or not we want them.
Learning though requires us to gather, create and retain stuff. Some of this stuff is forgettable; it doesn’t resonate, or is poorly taught or expressed. Or we simply don’t get it the way it is expressed, or the first time around.
Make it a memory
At an OU Residential School the session on revision was packed. The tips made us laugh: sucking a choice of Polo Fruit sweets by subject theme – when you come to the exam repeat and each sweet will link you to that period of revision. Odd. But it worked often enough for me to convince me of its value.
Fig.4. Ebbinghaus and his ‘Forgetting Curve’
The science from the likes of Hermann Ebbinghaus and his ‘Forgetting Curve’ simply indicates how something fades, unless you go back to it a few times over several days over which period you make it stick. It doesn’t say anything about the ‘stickiness’ of the memory in the first place. Sometimes this stickiness is made for you. There is drama, there is an explosion. Most likely, by chance, the learning is anchored by some unrelated event like the fire alarm going off – that won’t work for 50 different things though.
Fig. 5 Multiple ways of making ‘it’ stick: read (book and e-book), highlight, tag and take notes.
If the module, or your tutor isn’t doing it for you then the next step is to dig around for a book, video or image that does it for you.
Most likely, and of far greater value, is for you to turn that lesson into a memory of your own creation. There is always value in taking notes, so never listen to the presenter who says ‘no need to take notes I’ll give you the slides afterwards’. Never trust the quality of the slides. What the person said will be of more value then the slides. You, and your handwriting, and your doodles are how it starts to become a memory. Then when you write up or rewrite those notes you do it again. You make it into something.
Fig.6 The River Ouse at low tide.
I’m fixating on the horror of drowning in a shell-hole in the First World War.
Ever since I was a boy those images of cowboys and Arabian princes sinking into quicksand has horrified me. What must it have been like? Walking the dog by the River Ouse at low tide just as it turned the gurgling of water backing up and filtering into the muddy bank gave me the shivers. That sound was ominous. It made a memory of the walk and the thought. It’s also what is sustaining me as I work at a short story.
Fig. 7 A family memory of a wedding in California. Will it stick?
We’ve talked about ‘memory making’ in the family.
It is the event, and the sharing of the event. My late mother-in-law was horrified that her daughter couldn’t remember a road-trip they did across the US when she was 13. I concluded that she hadn’t remembered much, or couldn’t remember much when it was mentioned out of the blue, as the trip was never shared. Conversations are and were always about current and future events. This is why it helps to get the old photo albums out from time to time. But there’s a loss. Do we make them anymore? Visiting a mislabelled album online is never the same.
Fig. 8. My late grandfather John Arthur Wilson MM with the author Lyn Macdonald at the spot north of Poelcappelle, Belgium where he buried two of his mates – 75 years after the event. He recalled it ‘like yesterday’.
Recalling the First World War
Some veterans would talk, others remained silent. Those who did not want to remember could and did forget. My late grandfather was a talker; it drove my mother mad. I came to love his recollections. Clearly, there were events that would have burned themselves into the memories of these men, but unless they talked about it, in a veteran’s association or with family and friends it was not going to stick. No wonder veterans would seek each other out over the decades. Nudged by histories and movies their memories could be changed though; sometimes they came to say what was expected of them ‘the rats were huge, the generals useless, the German bunkers impenetrable, the mud up to your waist, the sound of the individual shells … ‘
Whatever activities and devices are built into your module, you are responsible and can only be responsible for making something of it. Take the hint. Engagement takes time so make the time for it. These days it is made easier through the Internet. You can keep a blog to share or as a learning journal; you can talk it over with fellow students either asynchronously in a forum (or blog), or synchronously in a webinar. You can ‘mash it up’ with images, grabs, doodles and annotations. You can make it your own. It’ll stick if you want it to but superglue requires effort. Someone else ‘sticks it’ for you and it won’t happen.
Towards an assessment of my teaching
I often share a post I am writing as I do so. In this case having identified the story to tell : running a workshop to solve a ‘messy’ business problem I am pulling together or creating supporting images, in the above case a grab and mashup from Martin Weller’s book ‘The Digital Scholar’ – my goal is to be recognised as one. In a forum post as an Open University ‘Master of arts in Open and Distance Education’ (I graduated in early 2013) I suggested this could be achieved in four years – John Seely Brown thinks that eLearning speeds things up, while Weller reckons on ten years.
Reflecting on a presentation on learning to students at Oxford Brookes University
Fig.1. Rescue having failed a 4 tonne whale is dragged from Stinson Beach.
As a student on Oxford Brookes University’s online course ‘First Steps into Teaching and Learning 2014’ here in week 4 we have been challenged to consider an experience from teaching or being taught and in a five minute presentation reflect on this.
My interest is teaching postgraduates and/or ‘in the workplace’.
I should be feeling I’ve stumbled into the right time and place with this one having just given a ten minute presentation online as part of the Open University Masters in Open and Distance Education module H818: The Networked Practitioner, however with that one, despite every expectation to exploit my love of and experience with linear and interactive media I resorted to a Powerpoint. I needed to improve the script up to the line and this offered the flexibility I could not have had with a Prezi or video. There were too many cumbersome technical barriers and trips that I wasn’t happy to pursue or risk.
What I’m doing here is thinking through a presentation I need to prepare. Sharing this, if and where feedback can be garnered, then informs the decisions I take.
My immediate idea, often my best, is to do a selfie-video talking to camera while hurtling around a roller-coaster at Thorp Park. It would sum up the terror, thrill, highs and lows of taking a day long workshop with a class of some 40 year 9s (12/13 year olds) in a secondary school that had/has a checkered history.
The second idea, to change the setting radically, would be a workshop with nine on creative problem solving – the objective was to come up with answers to a messy problem, though the motivation to be present for most was to experience a variety of creative problem solving activities that I had lined up. This nine in an organisation, included MBAs, prospective MBAs, a senior lecture, junior and senior managers and officers: colleagues and invited guests from different departments. This example is probably the most appropriate.
A third might be something I attended as a student – apt because doing this in 2009/2010 in part stimulated me to take an interest in learning: I wanted to know what was going wrong. Here we had prospective club swimming coaches doing everything that was unnatural to them – working from a hefty tome of paper, sitting through a lecture/seminar and expecting assessment to be achieved by filling in the blanks on course sheet handouts. This from people with few exceptions who left school with few or no qualifications – often troubled by Dyslexia. They were swimming coaches to dodge this very kind of experience. It was, you could tell, hell for some. The misalignment could not have been greater. Here the immediate visual image, apt given the subject matter, would be to watch a fish out of water drown – or nearly drown and be rescued. What really grated for me in this course was the rubbish that was taught – too many gross simplifications and spurious science.
Based on the above I should challenge myself to do the video as I need to crack loading and editing. The fish out of water, whale actually, I can illustrate from photographs and the experience this summer of being present as a 4 tonne whale beached and drowned on Stinson Beach, California (See Fig.1. above).
We remember thanks to repetition
|From E-Learning III|
Repetition or re-visiting is vital. We cannot help but change our perspective as we gain more experience, insights and knowledge. We need repetition in order to get ‘stuff’ into the deeper recesses of our brains where wonders are worked. Therefore, far better to exposure to brilliance often, rather than giving them something less than brilliant simply because it is new, or an alternative. If nothing else Web 2.0 ought to be giving students the chance to find and limit themselves to the best.
What is digital ‘academic’ scholarship? Should 19th and 20th century definitions even apply?
Martin Weller published ‘The Digital Scholar’ in 2011 on a Creative Commons Licence. You can download it for free, or purchase the book or eBook, and then do as you will with it. When I read it I share short excerpts on Twitter. I’ve blogged it from end to end and am now having fun with a simple tool for ‘mashing up’ designs called ‘Studio’. It’s a photo editing tool that allows you to add multiple layers of stuff. I rather see it as a revision tool – it makes you spend more time with the excerpts you pick out.
You cannot be so open that you become an empty vessel … you have to create stuff, get your thoughts out there in one way or another so that others can knock ’em down and make more of them. Ideas need legs. In all this ‘play’ though have I burried my head in its contents and with effort read it deeply? Do we invoke shallow learning and distraction with openness? If we each read the book and met for a tutorial is that not, educationally, a more focused and constructive form of ‘oppenness’?
In relation to scholarship shoulf the old rules, the ‘measures’ of academic prowess count? In the connected world of the 21st century ‘scholarship’ is able to emerge in unconventional ways, freed of the school-to-university conveyor belt.
Weller, M (2011) The Digital scholar
University Of California Approves Major Open Access Policy To Make Research Free
Over 125 million people want a university education – current global provision struggles to cater for 5 milllion – making the source content free doesn’t create an educational package – it lack the scaffolding, assessment and accreditation, but it is food for the hungry.