Home » E-Learning » e-Assessment (Page 2)

Category Archives: e-Assessment

‘If you’re not lost and confused in a MOOC you are probably doing something wrong’


Photo credit: Robin Good

 

‘MOOCs indicate that we are seeing a complexification of wishes and needs’ – so we need a multispectrum view of what universities do in society. George Siemens, (18:51 25th March 2013).

 

A terrific webinar hosted by Martin Weller with George Siemens speaking. Link to the recorded event and my notes to follow.

I took away some key reasons why OER has a future:

 

  1. Hype between terrifying and absurd.
  2. State reduction in funding will see a private sector rise.
  3. Increase in rest of world’s desire for HE OER
  4. Certificates growing.
  5. The Gap
  6. Accelerating time to completion
  7. Credit and recognition for students who go to the trouble to gain the competencies.
  8. Granular learning competencies and the gradual learning and badging to stitch together competencies.

 

And a final thought from the host:

‘If you’re not lost and confused in a MOOC  you are probably doing something wrong’.  Martin Weller (18:45 25th March 2013)

Which rather means I may be doing something wrong!

I posted to Linkedin, I am neither confused, nor lost. Indeed I have a great sense of where I am and what is going on, have met old online friends and am making new contacts and enjoy using two of my favourite platforms: Google+ and WordPress.  (All the fun’s at H817open)

 

A selection of papers are proving enlightening too:

 

1) John Hopkins Bloomberg School of Public Health OpenCourseWare (2009) Kanchanaraksa, Gooding, Klass and Yager.

 

2) The role of CSCL pedagogical patterns as mediating artefacts for repurposing Open Educational Resources (2010) Conole, McAndrew & Dimitriadis

 

3) A review of the open educational resources (OER) movement: Achievements, challenges, and new opportunities. Report to The William and Flora Hewlett Foundation.

 

I’ll post a 500 word review of the above shortly as per H817open Activity 7.

The value is both expanding the reasons for OER as well as having a handful of objections, negatives and concerns. Like all things regarding e-learning, they is no panacea for putting in the time and effort.

And a couple of others that look interesting:

 

Disruptive Pedagogies and Technologies in Universities (2012)  Anderson and McGreal

 

Open education resources: education for the world? (2012) Richter and McPherson

 

H817 Open MOOC Activity 4: Identifying priorities for research

English: Postage stamp depicting Martin Luther...

English: Postage stamp depicting Martin Luther, the initiator of the Protestant Reformation (Photo credit: Wikipedia)

Imagine you are advising a funding organisation that wishes to promote activity and research in the area of open education.

  • Set out the three main priorities they should address, explaining each one and providing a justification for your list. Share this in the Week 1 forum  and compare with priorities of others.

In this activity you are just expected to start thinking about these issues, and to use your own experience and intuition; you are not expected to research them in depth. You will build on this work during next week, and also for the assignment.

1. Get noticed

A clarion call to those who matter to you online to take notice, take an interest, give it some thought and give it a go. Get the message right. Know why you are doing it. Express it with conviction, professionalism and enthusiasm. Seek out like-minded people.

 

2. Use the medium

All Martin Luther could do was nail an edict to a church door, but it started the Reformation. There too many doors, and a plethora of nails when you go online. Brief  a team of web experts and web savvy to design a platform and structure that is robust and provides ‘scaffolding’ for what comes next. A river is free and open, a flood or tsunami cannot be controlled. Put in a slide and provide kicker floats and water polo balls. Gather ‘big data’. Watch everything and keep a record.

 

3. Be a great host

In France you’d be called an ‘animateur’ or ‘realisateur’ in North America you might be ‘the host with the most’ and in Britain ‘a jolly good chap or chappess’. To go online successfully is to transfuse the live component of the institution you represent into the World Wide Web – think of it as an organism that requires nurturing. Be there for them. Have plenty of eager, online net residents to welcome the naïve and the expert, the time starved and those with time on their hands.

Why skiing is my metphor for life and learning

Fig.1.   Mont Turia from the summit of Aiguille Rouge, Les Arcs at 3250m

On the last day, on the last run of my first week’s skiing I broke my leg rather badly. I was 13. I was in hospital for a week. In a wheelchair for two months and had the leg re-broken as it wasn’t setting properly. I spent six months at home. Idiot. But most 13 year old boys are.

I missed the next season.

For the following 20 years skiing mattered – a gap year working in the Alps (Val D’Isere in the Sofitel Hotel working 13 hours a day 7 days a week), a decade later researching a TV documentary and book  (Oxford Scientific Films, Skieasy Ski Guides), falling in love with a fellow skiing enthusiast (we’ve been married 20 years), a honeymoon on the slopes and ten years later, on the slopes with a 4 and 6 year old, then again when they were 10 and 12. 

I miss it.

(See above – the last week of the season, Tignes. The only people on the slopes are the ‘seasoniers’ who have worked since December. It is like being on the beach. A stream that flows above Val Claret melts and various ponds form. We ski it.)

Early in the afternoon I’d asked my girlfriend if she’d marry me. I was feeling cock-a-hoop.

We’ve been back twice in the last decade. There have been other priorities. I’ll be taking my 14 year old son out later this month or in April. Is that wise? At this age teenagers really are prone to take risks and can lack the physique.

Reasons to celebrate and look forwards

37 months to the day after starting the Masters in Open & Distance Education (MAODE) I got the final result, for H810: Accessibility in Open Learning – supporting students with disabilities, today. 84.

It has been so worth it and such a better, engaging, effective, experience than my undergraduate degree in a traditional university some decades ago. I feel as if I have earned it for a start. I have survived disasters rather than succumbed to them.

I am a reading, thinking, writing machine.

I feel like someone who has come to skiing late in life and has caught the bug. My mother started skinning in her mid 40s … and in her 50th year (unencumbered by her husband who was with wife three by then) sold the house and did a belated ‘gap year’ working a season in the Alps. The equivalent for me has to be the intellectual challenge of doctoral research.

More reading, thinking and writing – with research and teaching too I hope.

Onwards.

Tutor Marked Assignment One  (TMA01) for H809 (Practice-based research in educational technology) is due on Monday.

Why more?

‘Practice-based research in educational technology’, to use skiing as a metaphor, is like learning to ski ‘off-piste’. Apt, as the tracks I make are ones I have planned, rather than keeping to the groomed, signed and patrolled ‘safety’ of the regular runs.

And my reward?

Fig. 2. Mont Blanc – From the Ski Resort of La Plagne,  Above Montchavin. Les Arc on the right . The road to Val d’Isere clinging to the mountain in the middle distance Bourg St. Maurice in the bottom of the Valley

Skiing en famille.

We’ve not been out for five years so it should be a treat. It has to be on a shoestring, so short of hitching to Bulgaria can anyone recommend ways to keep the cost down?!!

 

The Educational Impact of Weekly E-Mailed Fast Facts and Concepts

In this study, the authors assessed the educational impact of weekly Fast Facts and Concepts (FFAC) e-mails on residents’ knowledge of palliative care topics, self-reported preparedness in palliative care skills, and satisfaction with palliative care education.

The more papers I read, like learning a foreign language, the thinner the blur between mystery and comprehension in terms of judging a paper and its contents. My goal is to be able to conduct such research and write such papers. I understandably feel that a first degree in medicine and a second masters degree in education is required at this level. At best I might be able to take on psychology or neuroscience. My preference and hope would be to become part of a team of experts.

Purpose: Educational interventions such as electives, didactics, and Web-based teaching have been shown to improve residents’ knowledge, attitudes, and skills. However, integrating curricular innovations into residency training is difficult due to limited time, faculty, and cost.

What – A clear problem:

Integrating palliative care into residency training can be limited by the number of trained faculty, financial constraints, and the difficulty of adding educational content with limited resident duty hours. (Claxton et al. p. 475 2011)

Who – Participants

Beginning internal medicine interns

Why – Time- and cost-efficient strategies for creating knowledge transfer are increasingly important. Academic detailing, an educational practice based on behavioral theory, uses concise materials to highlight and repeat essential messages. Soumerai  (1990)

How – We designed this study to assess the educational impact of weekly e-mailed FFAC on
internal medicine interns in three domains: knowledge of palliative care topics, satisfaction with palliative care education, and self-reported preparedness in palliative care skills.

Methods

This randomized, controlled study of an educational intervention included components of informed consent, pretest, intervention, and posttest.

Fast Facts and Concepts

FFAC are 1-page, practical, peer-reviewed, evidence-based summaries of key palliative care topics first developed by Eric Warm, M.D., at the University of Cincinnati Internal Medicine Residency Program in 2000.6

Intervention

One e-mail containing two FFAC was delivered weekly for 32 weeks to interns in the intervention group.

Pre-test

All participants completed a pretest that assessed knowledge of palliative care topics, self-rated preparedness to perform palliative care skills, and satisfaction with palliative care education.

Method: Internal medicine interns at the University of Pittsburgh and Medical College of Wisconsin were randomized to control and intervention groups in July 2009. Pretests and post-tests assessed medical knowledge through 24 multiple choice questions, preparedness on 14 skills via a 4-point Likert scale and satisfaction based on ranking of education quality.

The intervention group received 32 weekly e-mails.

Control Group
No e-mails were sent to the control group.

Post-test
Respondents completed a post-test 1 to 8 weeks after the
intervention

Educational equipoise
All study participants were informed of the content and the online availability of FFAC during recruitment. At the conclusion of the study, both control and intervention groups were given a booklet that contained all the e-mailed FFAC.

Statistical analysis

Descriptive statistics and t tests were used to compare the demographic data between the control and intervention groups. Medical knowledge, preparedness, and satisfaction were compared pretest and post-test within groups by Wilcoxon tests and between groups via Mann-Whitney U tests. The data did not meet assumptions for multivariate analysis due to the small sample size. Only univariate analysis was performed.

Although traditional academic detailing techniques include educational outreach visits and distribution of printed graphic materials, e-learning techniques such as e-mail delivery of educational content, listservs and Web-based tutorials can also be considered rooted in this behavioral theory given their focus on repeated, concise content.

Pain assessment and management, breaking bad news, communicating about care goals, and providing appropriate medical care for a dying patient are necessary skills for surgery, family medicine, pediatric, obstetrics and gynaecology, physical medicine and rehabilitation, emergency medicine, neurology, radiation oncology, anesthesiologist, and psychiatry residents.

Studies that focus on e-mail education interventions have shown that weekly e-mails change the behavior of e-mail recipients, improve learner retention of educational content and that retention improvements increase with the duration over which e-mails were received. (Kerfoot et al. 2007 ) (Matzie et al. 2009)

Results: The study group included 82 interns with a pretest response rate of 100% and post-test response rate of 70%. The intervention group showed greater improvement in knowledge than the control (18% increase compared to 8% in the control group, p = 0.005).

Preparedness in symptom management skills (converting between opioids, differentiating types of pain, treating nausea) improved in the intervention group more than the control group ( p = 0.04, 0.01, and 0.02, respectively).

There were no differences in preparedness in communication skills or satisfaction between the control and intervention groups.

Conclusions: E-mailed FFAC are an educational intervention that increases intern medical knowledge and self-reported preparedness in symptom management skills but not preparedness in communication skills or satisfaction with palliative care education.

REFERENCE

Claxton, R, Marks, S, Buranosky, R, Rosielle, D, & Arnold, R 2011, ‘The Educational Impact of Weekly E-Mailed Fast Facts and Concepts’, Journal Of Palliative Medicine, 14, 4, pp. 475-481, Academic Search Complete, EBSCOhost, viewed 25 February 2013.

Matzie KA, Price Kerfoot B, Hafler JP, Breen EM: (2009) Spaced education improved the feedback that surgical residents given to medical students: A randomized trial. Am J Surg 2009;197:252–257.

Price Kerfoot B, DeWolf WC, Masser BA, Church PA, Federman DD: (2007) Spaced educational improves the retention of clinical knowledge by medical students: A randomized controlled trial. Med Educ 2007;41:23–31.

Soumerai SB, Avorn J: (1990) Principles of educational outreach (‘academic detailing’) to improve clinical decision making. JAMA 1990;263:549–556.

 

Struggling to mark assignments? Get the students to do it

I just watched Daphne Koller’s TED lecture on the necessity and value of students marking their own work. (for the fifth time!)

Whilst there will always be one or two who cheat or those who are plagiarists, the results from ‘Big Data’ on open learning courses indicate that it can be a highly effective way forward on many counts.

1) it permits grading where you have 1,000 or 10,000 students that would otherwise be very expensive, cumbersome and time consuming

2) as a student you learn from the assessment process – of your work and that of others

3) student assessment of other’s work is close to that of tutors though it tends to be a little more harsh

4) student assessment of their own work is even closer to the grade their tutor would have given with exceptions at opposite ends of the scale – poor students give themselves too high a grade and top students mark themselves down.

Conclusions

a) it works

b) it’s necessary if learning reach is to be vastly extended

c) isn’t human nature a wonderful thing?! It makes me smile. There’s an expression, is it Cockney? Where one person says to another ‘what are you like?’

Fascinating.

‘What are we like?’ indeed!

 

Self and Peer Grading on Student Learning – Dr. Daphne Koller

Fig. 1. Slide from Dr Daphne Koller‘s recent TED lecture (Sadler and Goodie, 2006)

I just watched Daphne Koller’s TED lecture on the necessity and value of students marking their own work. (for the fifth time!)

Whilst there will always be one or two who cheat or those who are plagiarists, the results from ‘Big Data’ on open learning courses indicate that it can be a highly effective way forward on many counts.

  1. it permits grading where you have 1,000 or 10,000 students that would otherwise be very expensive, cumbersome and time consuming
  2. as a student you learn from the assessment process – of your work and that of others
  3. student assessment of other’s work is close to that of tutors though it tends to be a little more harsh
  4. student assessment of their own work is even closer to the grade their tutor would have given with exceptions at opposite ends of the scale – poor students give themselves too high a grade and top students mark themselves down.

Conclusions

  •  it works
  •  it’s necessary if learning reach is to be vastly extended
  • isn’t human nature a wonderful thing?! It makes me smile. There’s an expression, is it Cockney? Where one person says to another ‘what are you like?’

Fascinating.

‘What are we like?’ indeed!

REFERENCE

Philip M. Sadler & Eddie Good (2006): The Impact of Self- and Peer-Grading on Student Learning, Educational Assessment, 11:1, 1-31

 

%d bloggers like this: