In this study, the authors assessed the educational impact of weekly Fast Facts and Concepts (FFAC) e-mails on residents’ knowledge of palliative care topics, self-reported preparedness in palliative care skills, and satisfaction with palliative care education.
The more papers I read, like learning a foreign language, the thinner the blur between mystery and comprehension in terms of judging a paper and its contents. My goal is to be able to conduct such research and write such papers. I understandably feel that a first degree in medicine and a second masters degree in education is required at this level. At best I might be able to take on psychology or neuroscience. My preference and hope would be to become part of a team of experts.
Purpose: Educational interventions such as electives, didactics, and Web-based teaching have been shown to improve residents’ knowledge, attitudes, and skills. However, integrating curricular innovations into residency training is difficult due to limited time, faculty, and cost.
What – A clear problem:
Integrating palliative care into residency training can be limited by the number of trained faculty, financial constraints, and the difficulty of adding educational content with limited resident duty hours. (Claxton et al. p. 475 2011)
Who – Participants
Beginning internal medicine interns
Why – Time- and cost-efficient strategies for creating knowledge transfer are increasingly important. Academic detailing, an educational practice based on behavioral theory, uses concise materials to highlight and repeat essential messages. Soumerai (1990)
How – We designed this study to assess the educational impact of weekly e-mailed FFAC on
internal medicine interns in three domains: knowledge of palliative care topics, satisfaction with palliative care education, and self-reported preparedness in palliative care skills.
This randomized, controlled study of an educational intervention included components of informed consent, pretest, intervention, and posttest.
Fast Facts and Concepts
FFAC are 1-page, practical, peer-reviewed, evidence-based summaries of key palliative care topics first developed by Eric Warm, M.D., at the University of Cincinnati Internal Medicine Residency Program in 2000.6
One e-mail containing two FFAC was delivered weekly for 32 weeks to interns in the intervention group.
All participants completed a pretest that assessed knowledge of palliative care topics, self-rated preparedness to perform palliative care skills, and satisfaction with palliative care education.
Method: Internal medicine interns at the University of Pittsburgh and Medical College of Wisconsin were randomized to control and intervention groups in July 2009. Pretests and post-tests assessed medical knowledge through 24 multiple choice questions, preparedness on 14 skills via a 4-point Likert scale and satisfaction based on ranking of education quality.
The intervention group received 32 weekly e-mails.
No e-mails were sent to the control group.
Respondents completed a post-test 1 to 8 weeks after the
All study participants were informed of the content and the online availability of FFAC during recruitment. At the conclusion of the study, both control and intervention groups were given a booklet that contained all the e-mailed FFAC.
Descriptive statistics and t tests were used to compare the demographic data between the control and intervention groups. Medical knowledge, preparedness, and satisfaction were compared pretest and post-test within groups by Wilcoxon tests and between groups via Mann-Whitney U tests. The data did not meet assumptions for multivariate analysis due to the small sample size. Only univariate analysis was performed.
Although traditional academic detailing techniques include educational outreach visits and distribution of printed graphic materials, e-learning techniques such as e-mail delivery of educational content, listservs and Web-based tutorials can also be considered rooted in this behavioral theory given their focus on repeated, concise content.
Pain assessment and management, breaking bad news, communicating about care goals, and providing appropriate medical care for a dying patient are necessary skills for surgery, family medicine, pediatric, obstetrics and gynaecology, physical medicine and rehabilitation, emergency medicine, neurology, radiation oncology, anesthesiologist, and psychiatry residents.
Studies that focus on e-mail education interventions have shown that weekly e-mails change the behavior of e-mail recipients, improve learner retention of educational content and that retention improvements increase with the duration over which e-mails were received. (Kerfoot et al. 2007 ) (Matzie et al. 2009)
Results: The study group included 82 interns with a pretest response rate of 100% and post-test response rate of 70%. The intervention group showed greater improvement in knowledge than the control (18% increase compared to 8% in the control group, p = 0.005).
Preparedness in symptom management skills (converting between opioids, differentiating types of pain, treating nausea) improved in the intervention group more than the control group ( p = 0.04, 0.01, and 0.02, respectively).
There were no differences in preparedness in communication skills or satisfaction between the control and intervention groups.
Conclusions: E-mailed FFAC are an educational intervention that increases intern medical knowledge and self-reported preparedness in symptom management skills but not preparedness in communication skills or satisfaction with palliative care education.
Claxton, R, Marks, S, Buranosky, R, Rosielle, D, & Arnold, R 2011, ‘The Educational Impact of Weekly E-Mailed Fast Facts and Concepts’, Journal Of Palliative Medicine, 14, 4, pp. 475-481, Academic Search Complete, EBSCOhost, viewed 25 February 2013.
Matzie KA, Price Kerfoot B, Hafler JP, Breen EM: (2009) Spaced education improved the feedback that surgical residents given to medical students: A randomized trial. Am J Surg 2009;197:252–257.
Price Kerfoot B, DeWolf WC, Masser BA, Church PA, Federman DD: (2007) Spaced educational improves the retention of clinical knowledge by medical students: A randomized controlled trial. Med Educ 2007;41:23–31.
Soumerai SB, Avorn J: (1990) Principles of educational outreach (‘academic detailing’) to improve clinical decision making. JAMA 1990;263:549–556.
- Palliative care: knowing when not to act (oup.com)
Even a well-designed quasi-experimental study is inferior to a well-designed randomised controlled trial.
In favour of randomized control trials
Torgerson and Torgerson (2001)
The dominant paradigm in educational research is based on qualitative methodologies (interpretive paradigm). Torgerson and Torgerson (2001. p. 317)
Despite fairly widespread use of quantitative methods the most rigorous of these, the randomised controlled trial (RCT), is rarely used in British educational research. Torgerson and Torgerson (2001. p. 317)
Even a well-designed quasi-experimental study is inferior to a well-designed randomised controlled trial. Torgerson and Torgerson (2001. p. 318)
Something I’ll need to get my head around – again!
The first problem is the statistical phenomenon of regression to the mean (Cook and Campbell, 1979; Torgerson, C.J., 2000) All groups studied need to have the same regression to the mean chances. Torgerson and Torgerson (2001. p. 318)
Non-randomised quantitative methods are nearly always inferior to the randomised trial. Torgerson and Torgerson (2001. p. 319)
Nearly 40 years ago Schwartz and Lellouch described two types of randomised trial: the ‘explanatory trial’ and the ‘pragmatic trial’ (Schwartz and Lellouch, 1967).
The explanatory trial design is probably the one with which most people are familiar.
This type of study is tightly controlled and, where possible, placebo interventions are used.
Thus, one may take a large group of children all from a similar socio-economic background and attainment and randomly allocate them into two groups. One group receives the intervention under investigation whilst the other receives a dummy or sham intervention. Torgerson and Torgerson (2001. p. 320)
The pragmatic trial : the environment in which the trial is conducted is kept as close to normal educational practice as possible.
The children, or schools, are allocated the new intervention at random. A disadvantage of the pragmatic approach is that the trials usually have to be much larger than the explanatory approach but the pragmatic trial approach is probably the most feasible and useful trial design for educational research. Because the trial mimics normal educational practice, there is a greater variation that can make it harder to detect a small effect. To cope with this the sample size needs to be increased accordingly. Torgerson and Torgerson (2001. p. 320)
The underlying idea of a randomised trial is exceedingly simple.
Two or more groups of children, identical in all respects, are assembled. Clearly, the individual children are different but when groups of children are assembled by randomisation, and with a large enough sample size, they will be sufficiently similar at the group level in order to make meaningful comparisons. In other words, the differences are spread equally across both groups, making them essentially the same. Torgerson and Torgerson (2001. p. 321)
The analysis of a randomised trial is actually simpler than other forms of quantitative research because we know the two groups are similar at baseline. Torgerson and Torgerson (2001. p. 321)
Randomisation creates groups with the same proportion of girls, with the same proportion of pupils from various socio-economic and ethnic groups, with the same distribution of ages, heights, weights etc. – this is the simple elegance of randomisation! Torgerson and Torgerson (2001. p. 323)
To avoid observer bias blinded outcome assessment must be undertaken. Torgerson and Torgerson (2001. p. 324)
Qualitative methodologies are well suited to investigating what happens with individuals; RCTs are appropriate for looking at the larger units relevant to policy makers. Torgerson and Torgerson (2001. p. 3264)
SCHWARTZ, D. and LELLOUCH, D. (1967) Explanatory and pragmatic attitudes
in therapeutic trials. Journal of Chronic Diseases 20, 637–648.
Torgerson, C, & Torgerson, D 2001, ‘The Need for Randomised Controlled Trials in Educational Research’, British Journal Of Educational Studies, 3, p. 316, JSTOR Arts & Sciences IV, EBSCOhost, viewed 13 February 201
- Effects of the Endpoint Adjudication Process on the Results of a Randomised Controlled Trial: The ADVANCE Trial (plosone.org)
- RCTs, skeptics, and evidence-based policy (errorstatistics.com)
- Smith and Pell: Parachute Use and Randomised Controlled Trials (delong.typepad.com)
(These notes have been prepared and considered as part of the Open University postgraduate module on e-learning H810: Accessible Online Learning – supporting disabled students as past of their MA in Open and Distance Education)
Accessible examinations and assessments
- Are the particular assessments or examinations are core to the course?
- What adjustments are permissible within particular assessments or examinations without compromise to academic, or other prescribed, standards, such as competences required by professional bodies?
- Is the successful achievement of the highest grades and awards, based on performance in examinations and other assessments, equally attainable by disabled students?
These three questions are universal.
Offered in a table, as part of a questionnaire they should be answered by a variety of people up and down the chain of command – from assessors and tutors, through courseware designers to subject matters and the dean of faculty.
I gave this some thought in four different contexts:
- teaching swimming coaches as the Amateur Swimming Association’s Level 1, Level 2 and Level 3 Teaching Aquatics levels;
- admissions policy at a collegiate university such as Oxford
- for corporate Learning & Development departments via their e-learning provider
- for a learning institutions in the creative industries and media.
I’d ask various further questions:
- What kind of adjustments should we reasonably be expected to make to accommodate students with disabilities?
- What does our research say about the attraction of certain courses or the problems that have been encountered. What have we addressed and fixed? Where do the current problems lie?
- Do we meet the criteria of the DDA? If so, is this enough? Is the bar too low? How much do we wish to achieve?
- Are we communicating policy at all stages from advertising courses, faculties and colleges – to the assessment process and examinations, the results experted and how these will be achieved?
- (Oxford) Are we or should we proactively seek to attract students with disabilities ?
- Should we seek to attain a certain % in the student population of students with representative disabilities?
- Are there colleges or faculties or individual champions which have a vision that is either more for or against such accommodation?
- What student bodies exist, university funded or student organised, around support for disabled students?
Putting content online, or simply the universal digitization of resources, immediately creates opportunities for students with disabilities who can then apply technology to enhance or adjust the may the content is communicated, shared or discussed in keeping with their needs, expectations, experiences and ambitions. Improving opportunities has to be a two way process, working with the students who have or have not found ways around or through barriers whilst seeking to offer, suggest and provide services, whether software, assistive technologies or other interventions (assistants, scribes, parking, physical access to building, facilities).
In relation to disabled students who needs to know what? After an audit of awareness who then needs to improve their knowledge and awareness?
If there is a blank requirement for basic awareness, at what levels and to what degree can further training be provided to that understanding is fully integrated horizontally and vertically.
- Do we need ethical and moral guidance?
- Should the university vision or mission statement be adjusted?
- Where are we taking legal advice from? Is the legal position fully understood by those who need to know?
A student using Dragon speech software in an exam for the first time was revealing:
- He had been using it for two months previous to the exam (like learning the piano, I would have got far further along the learning curve so that as I suer I felt I could make the technology sing – for some the right pen or pencil is crucial to exam preparation.)
- Allowing for the idiosyncracies of the programme
- Started the exam much the same as anyone else – with fear and trepidation
‘My exam results proved that although I may not be able to express myself with the technical aptitude of most people my age and intelligence using this equipment I was at least able to demonstrate that I had been working and learning’.
Making adjustments to or accommodating students for different exams will be tricky. Various questions have to be answered:
- Is the assessment or examination is core to the course?
- What adjustments are permissible without compromise to academic, or other prescribed standards or competencies?
- Are the highest grades and awards, based on performance in examinations and other assessments, equally attainable by disabled students?
It strikes me that greater transparency and collaboration is necessary. This would be beneficial, but is also apt in the modern paradigm of e-learning. See it as an opportunity to review course content in a different way.
Excellent, broad, clear communication is required. Faculties need to go out of their way to be sure students have understood when they apply for the course and as the course plays out.
Assessment strategies should be:
- properly designed and kept under review
- rigorous, consistent and at an appropriate level
- effective measures of student attainment
- able to guarantee the validity, equity and reliability of assessments.
On the basis the QAA standards have already been followed, further improvements to satisfy a range or disabilities ought to be possible. Broadening the reach would be recommended across visual, hearing, mobility and cognitive impairments.
Assessment is a pedagogical tool
These milestones, these hurdles are key to embedding learning and to starting the process of making it stick. Repetition of testing on the same subject improves the chances of it being remembered – a test in the real world, applied to a problem or task that is repeated is similar to an assessment, and as in the real world, repeated assessments, like a challenge, need to be different each time in scope, scale, context and outcomes.
The examination is one thing, marking is another. It isn’t simply guidelines on how the exam will be set and assessed, but the desires of the student, their goals and the expectations and ambitions of their peer group, college, profession and family.
Learning from the way in which the Paralympics take place – an ulimate test, I wonder if students with disabilities might also achieve a classification related to their disability. In this way, if, for example, awarded a 2.2 a letter or code might be affixed to it.
Fairness is debatable and it would have to be transparent, which in turn expects comment and criticism so therefore requires both the mechanisms and people to respond and to take action. This is therefore an additional cost, technically to provide the means of transparency, feedback and communication, but also in recruiting, retaining and supervising full-time or part-time personnel to undertake these tasks.
It strikes me that an institution or department, through an individual champion and with some credible followers, need to embrace accessibility and then follow through on a case by case basis, writing up and sharing experiences so that knowledge can be shared and where found barriers removed, reduced or circumvented. As a swimming club, proximity of special needs schools, practitioners and a flexible pool operator means that we have many disabled swimmers with a variety of needs. The club has become a regional hub for best practice. The lessons learnt, the expertise and the development of helpers, assistant teachers and coaches all contribute to the knowledge pool. Similar hubs are required within organisations – educational institutions or companies, if the diversity of possibilities for people with disabilities are to be met.
If only it were as simple as considering dietary needs in a restaurant, whether vegetarian or vegan, or allergies to certain foods.
Pointless second guessing the detail if once in a blue moon someone appears on a course or module with one of these disabilities.
Again from sport, and following on from insights from the Paralympics:
I wonder if there would be value in a log book and portfolio, more than just a blog and e-portfolio, but a detailed transferable record, including medical record, exam attainment, accommodations made in the past (successful and failures) … meeting certain criteria so at various times a students progress can be monitored and where assessments take place a judgement taken accordingly. In swimming athletes keep a log-book of training and test sets, as well as galas and land training … as times achieved over various distances swimming different strokes are constantly monitored it matters that they include periods of ill health and absence, even physical growth or weight gain.
- What, in their words, are the advantages of e-learning to the student with a disability? (mymindbursts.com)
- Access to learning for students with disabilities (mymindbursts.com)
- Ofqual: schools abusing ‘extra time’ rules in GCSEs (telegraph.co.uk)
- How about designing a user interface based on the teachings of Maria Montessori? (mymindbursts.com)
- Alums with disabilities cite campus pros and cons (futurity.org)
- Tough exams and learning by rote are the keys to success, says Michael Gove (schoolsimprovement.net)
Fig.1. Building Construction W B McKay 1943
Are you the learning architect or the learning builder?
It is flattering to the group from Learning & Development that they can be likened to architects. Whilst many will have a degree, some don’t – whilst some may have a post graduate qualification, very few do. None I’m sure will have spent six or seven years in formal study that has lead to recognition by the Royal College of E-Learning Designers – there is no such professional qualification, nor is there any period of formal study, a mix of studio work and academic research, that leads to a qualification of this calibre.
The exceptions are those with first degrees and MBAs and at the pinnacle of this discussion, Christopher Alexander who has first and second degrees from Cambridge and a PhD in architecture from Harvard.
Many in academia have the second degree and PhD – but they generally lack the experience designing learning outside undergraduate and postgraduate tertiary education, which is quite a different beast to the short courses and continual professional development desired in the workplace.
If I were to take the building trade by way of an analogy I would say that the learning and development manager is the client – while the architect is an agent or agency that you hire in for their design expertise and knowledge of foremen and project managers, builders and electricians – the project leaders, programmers and art directors of e-learning creation.
The L&D manager may be a subject matter expert but is far more likely to draw upon expertise from within their organisation.
Which of the following made the biggest contribution to your learning when you first set out in your current career asked Clive Shepherd?
Fig.2. What has contributed most to your learning?
This depends of course on when a person knows they are set on a career path.
How many people come into Learning & Development (L&D) having decided on this path as an undergraduate?
As a graduate trainee I expected a mix of on the job and formal training – this mix turned out to be around 95% to 5% while contemporaries elsewhere were getting 50/50 of none at all. This is the formal way of graduate training and can last two or three years. Think of lawyers (barristers and trainee solicitors), accounts, bankers and teachers … doctors, dentists, vets and architects.
Away from the presentation I like to click around as for me to understand a concept it helps to perceive its inception.
In turn, if you check the references for Jay Cross’s 2006 ‘Informal Learning: Rediscovering the Natural Pathways that Inspire Innovation and Performance’ you’ll find where his ideas may have came from – Robert A Heinlein (1961) ‘Strangers in a Strange Land’ and R Nelson Bolles (2005) ‘What Color’s Your Parachute’ are there along with John Seely Brown (2005) ‘The Only Sustainable Edge’.
There are some inspirational ideas and link here:
Workflow learning ties learning into the workflow within an organisation. According to Jay Cross it takes us to support and on-demand services that are designed to exist within the real tasks we do in our everyday work.Out of this work on workflow learning came an even wider, and what he regards as more important set of reflections.
Fig.3. Zoom.It History of Corporate Education.
This timelines the history of corporate and executive training. It is like a touch-screen and zoom control all in one. The Bayeux Tapestry in digital form (now there’s an idea over 900 years old). I spotted a typo – you’ll find it says something about ‘Toyota: Clean Production’ rather than Lean Production. We should consider the content in other ways – I know a PLC that set up an internal ‘university’ in the mid 1970s – or maybe they called in a training centre. Same difference?
If Clive Shepherd got his idea of the learning architect from Jay Cross I imagine Jay Cross in turn got the idea from Professor Christopher Alexander.
Christopher Alexander’s Notes on the Synthesis of Form was required reading for researchers in computer science throughout the 1960s. It had an influence in the 1960s and 1970s on programming language design, modular programming, object-oriented programming, software engineering and other design methodologies. He is cited through-out the Open University’s Masters in Open and Distance Education (MAODE) as an originator of design practice that was applied to computer design and therefore could be applied to e-learning design.
Here’s the education of someone who can rightfully call themselves an architect and do so in the context of learning, even of e-learning.
In 1954, Christopher Alexander was awarded the top open scholarship to Trinity College, Cambridge University in chemistry and physics, and went on to read mathematics. He earned a Bachelor’s degree in Architecture and a Master’s degree in Mathematics. He took his doctorate at Harvard (the first Ph.D. in Architecture ever awarded at Harvard University), and was elected fellow at Harvard. During the same period he worked at MIT in transportation theory and in computer science, and worked at Harvard in cognition and cognitive studies.
Fig.4. The Timeless Way of Building
‘The Timeless Way of Building’ proposes a new theory of architecture (and design in general) that relies on the understanding and configuration of design patterns.It is these design patterns that came to the attention of creators of e-learning modules in the 21st century, the idea that designs for subjects or cohorts might be replicated and shared across the online learning community so that you might say a) fits an undergraduate arts course, while b) is the model for a health & safety module in industry, c) gives you language learning in primary school while d) offers an elective in urology to 4th year medical students.
To become an architect requires a considerable commitment.
Take the three year undergraduate course in architecture at the University of Cambridge
Entry Requirements: A* AA : Likely to include Maths and Art or History of Art.
Students may stay on at Cambridge to complete an MPhil at RSA exams to qualify in six years (this includes a year in a placement)
‘The three year BA(Hons) course is unusual in the University in combining both arts and sciences. As such it provides a unique range of skills which lead to a wide range of careers, not just architecture’.
Throughout the BA tripos studio work carries 60% of the marks.
The remaining 40% is made up from exams and other forms of coursework (dissertations, etc). Studiowork in all years is handed in for marking at the end of the year. Studiowork is time-consuming and probably requires more hours per week than any other course in the University. Students are also expected to work during the Christmas and Easter vacations.
I labour this point because as someone who has gone from corporate communications and video based training to computer based training and e-learning I would never liken myself to a cardiologist, even a qualified lawyer or certified accountant, let alone an architect. An educator perhaps, but I don’t have a formal teaching qualificaiton, only sports coaching and the MAODE when I graduate early next year.
Fig. 5. BRICKS – Building Construction W B McKay 1943
Several other analogies have been used in the e-learning literature, some that still have a building or architecture theme to them.
What we get here is learning design broken down to brick-sized components, some call them ‘interactivities’ (a term I often here working in a design agency). I find the idea of atoms in a chemical reaction (Wiley, 2001) too small, even if we are dealing with binary code it isn’t something that we see anymore. Gilly Salmon (2002) would have liked ‘e-tivities’ to catch on – she puts these in a logical sequence, building blocks towards a module. At the Open University they tend to be called ‘Learning Objects’. Chris Pegler (2004) finds this idea too unresponsive preferring if we go with the Lego Technics. Littlejohn et al (2008) describe these components as:
Digital assets – a single item, image, video or podcast or an nformation objects: a structured aggregation of digital assets designed purely to present information.
Learning activities -tasks involving interactions with information to attain a specific learning outcome.
Learning design – structured sequences of information and learning activities to promote learning.
Fig. 5. BRICKS – Building Construction W B McKay 1943
For pure inspiration I like the digital architect as a goal for an undergraduate setting out on a long course of formal and applied study. L&D directors and managers approach an e-learning agency as they would a firm of architects and together they write a brief. This is proposed, scheduled and costed then a scheme of work begins.
The delivery, depending on the scale of it, might be akin to anything from a brick arcade (health and safety induction to leisure staff) to a bungalow to a housing estate (induction of trainee solicitors in an national firm of solicitors), an office block or a factory (long term management development for an international engineering business).
Alexander, C (1970) The Timeless Way of Building
Cross, J (2006) The Informal Learner
Downes, S (2000) Learning Objects. Available from http://www.newstrolls.com/news/dev/downes/col;umn000523_1.htm
Littlejohn, Falconer, Mcgill (2008) Characterising effective eLearning (sic) resources
Pegler, C and Littlejohn, A (2004) Preparing for Blended e-Learning, Routledge.
Salmon, G (2002) E-tivities
Shepherd, C (2011) The New Learning Architext
Wiley, D.A. (2000) Connecting Learning Objects to instructional design theory: a definition, a metaphor, and a taxonomy. In D.A. Wiley (ed), The instructional use of Learning Objects. Available from http://reusability.org/read/chapters/wiley.doc
- Clive Shepherd – the book, in person, ideas on learning and development in the World Wide Web 2.0 (mymindbursts.com)
- What happens if you ask an author to sign your copy of their eBook? (mymindbursts.com)
- Harvard Referencing made simple (mymindbursts.com)
- Over 1,000 posts on e-learning and creativity (mymindbursts.com)