Home » Posts tagged 'sensecam'

Tag Archives: sensecam

H809 Tutor Marked Assignment QQs on scope, sense cam and blogging

Blogging

The idea of looking behind blogging came from the reading … but did I reference it???

i.e that innovations never occur in isolation, there was always something else beforehand.

The mistake we all make is to assume that innovations land on a pristine landscape and we react with typically human surprise at this new marvel that will either revolutionise or destroy everything. I need to remember where I read that!

Something on innovations … eeek.

It does matter though, with blogs there is clearly a history of

a) keeping a diary

b) citizen journalism in the form of leaflets and ‘letters to the editor’

c) authors keeping a writer’s journal and

b) scientists and explorers keeping a formal ‘log’.

That and human nature to write stuff down – well, at least 1% of the population do, which gives the other 99% something to read.

Life Logging

Sense Cam came out of the efforts of Gordon Bell, now 81, and for the last 10 years head of research at Microsoft.

He got it into his head to digitise everything and then wear a gadget around his neck to capture even more. This seems moronic and his own writing isn’t academic, more a memoir, but others, Microsoft and University of Southampton, have pressed on. The Sense Cam is a fag-packet sized device you hang around your neck – a camera with a light and sound sensor, then triggers the taking of a picture as you go about your daily business (could be awkward). At the end of the day these pictures are downloaded and software filters the stuff.

Southampton (WebSciences) have examples of this.

You can now buy a SenseCam made by Microsoft and various Microsoft Research Labs are trying them out. The hope is that in time such a device will help support those with dementia or any kind of memory fade … the evidence from Southampton illustrate Ebbinghaus’s ‘Forgetting Curve’ – how we forget stuff pretty fast over days/weeks against use of various methods, including a Sense Cam. It does appear, naturally, that looking back regularly at a set of carefully selected pictures (I think there has be human intervention for obvious reasons) the patient/student subject is far better able to recall, retain, and therefore I presume to restore and ‘fix’ memories better.

I am starting to wonder if a person is indicating for Alzheimer’s or some such that they might use such a device ?

Or the Google Glass device to do the same thing. If I were a first year medical student doing my disection I’d like to use a sense cam to personalise a record of the activity, for example.

If I go down the blogging route ‘is blogging a valid activity for student assessment’ is far too broad while ‘Can blogging by students of journalism writing in English in Hong Kong be used as a formal part of assessment’ might be doable. Off the top of my head here, but let’s say there are 4 to 6 colleges where such a course is offered in Hong Kong …

So what about a geographically defined study?

China might be problematic due to restrictions on use of the Internet (and its vast size). Perhaps Poland!? Somewhere where the numbers aren’t huge. Then again, doesn’t it depend on the methods and tools you use? I am struck by this stuff they call ‘Big Data’ where a cohort of 10,000 on an Open Course (this at Stanford using Coursera) can reveal the nuances of ‘poor teaching’ – where in the past 1 or 2 students made the same mistake it goes unnoticed, but when 2000 students make the very same mistake then there’s clearly something wrong with the course.

To use Diana Laurillard’s apt phrase ‘it depends’. (don’t ask me where or when she said it, if you know, please tell me so that I can reference it correctly).

 

Why lifelogging as total capture has less value that selective capture and recall.

Beyond Total Capture. Sellen and Whittaker (2010)

Abigail Sellen and Steve Whittaker

Abigail J. Sellen (asellen@microsoft.com) is a principal researcher at Microsoft Research in Cambridge, U.K., and Special Professor of Interaction at the University of Nottingham, U.K.

Steve Whittaker (whittaker@almaden.ibm.com) is a research scientist at IBM Almaden Research Center, San Jose, CA.

may 2010 | vol. 53 | no. 5 | communications of the acm 77

You read about people looking at ways to capture everything – to automatically lifelog the lot. They constantly look for new ways to gather and store this data. What’s the point? Because they can? Who are they? Is it a life worth remembering? And do we need it all?!

Far better to understand who and what we are and use the technology either to ameliorate our shortcomings, aid those with memory related challenges and tailored to specific needs captured moments of worth.

There’s ample literature on the subject. I’m not overly excited about revolutionary change. It is as valuable, possibly more so, to forget, rather than to remember – and when you do remember to have those memories coloured by perspective and context.

My interest is in supporting people with dementia and cognitive difficulties – perhaps those recovering from a stroke for whom a guided path of stimulation into their past could help ‘awaken’ damaged memories. If lives are to be stored, then perhaps grab moments as a surgeon operates for training purposes, recreate a 3:1 tutorial system with AI managed avatars so that such learning practices can be offered to hundreds of thousands rather than just the privilege, elite or lucky few. Use the device on cars, not people, to monitor and manage poor driving – indeed wear one of these devices and see your insurance premiums plummet.

What does it mean to support human memory, rather than capturing everything.

Five Rs

  1. Recollect
  2. Reminisce
  3. Retrieve
  4. Reflect
  5. Remember

Despite the device SenseCam stimulate memories are as quickly forgotten as any other.
Whittacker 2010 – Family archives of photos are rarely accessed

Of less value than the considerable effort to produce these archives justifies.

  • Can’t capture everything
  • Need to prioritise – selectivity (as SwimTag and swim data)
  • Visual
  • Memory taxonomies
  • Quick and easy to use better than accuracy – good enough
  • Cues not capture – the data cues memories that are different to the image captured.
  • Memory is complex

Play to the strengths of human memory, help overcome weaknesses.

Incorporating the psychology of memory into the design of novel mnemonic devices opens up exciting possibilities of ways to augment and support human endeavours.

REFERENCE

Sellen, A. J., & Whittaker, S. (2010). Beyond total capture: a constructive critique of lifelogging. Communications of the ACM, 53(5), 70-77.

Whittaker, S., Bergman, O., and Clough, P. Easy on that trigger dad: A study of long-term family photo retrieval. Personal and Ubiquitous Computing 14, 1 (Jan. 2010), 31–43.

 

Automatically Augmenting Lifelog Events Using Pervasively Generated Content from Millions of People

English: This depicts the evolution of wearabl...

English: This depicts the evolution of wearable computers. See http://wearcam.org/steve5.htm for the original JPEG file. (Photo credit: Wikipedia)

 

Automatically Augmenting Lifelog Events Using Pervasively Generated Content from Millions of People

In the pursuit of pervasive user-generated content (ugc) based on senors, by augmenting visual lifelogs with ‘Web 2.0’ content collected by millions of other individuals.

We present a system that realises the aim of using visual content and sensor readings passively captured by an individual and then augmenting that with web content collected by other individuals. Doherty and Smeaton (2009)

  • Lifelogging, like keeping a diary, is a private and exclusive form of reverse surveillance. Doherty and Smeaton (2009)
  • Using SenseCam from Microsoft. Zacks (2006)
  • human memory operates by associating linked items together. Baddley (2004)
  • supportive of those patients suffering from early stage memory impairment. Berry e al (2009)
  • enhancing SenseCam gathered images by data mining from ugc sites such as Flickr and YouTube. Doherty and Smeaton (2009)
  • See also MyLifeBits. Bell and Gemmel (2007)
  • A commercial lifelogging product ViconRevue. OMG
  • Flickr has over 95 million geo-tagged images. (2010)
  • YouTube has 100 million video views per day (2010) YouTube fact_sheet

It has a camera and a range of other sensors for monitoring the wearer’s environment by detecting movement, temperature, light intensity, and the possible presence of other people in front of the device via body heat.

(I’d like the sensecam to be smaller still and include a microchip in a swimmer’s cap, or goggle or swimsuit to monitor various other factors, including heart rate, blood sugar levels and carbon dioxide). 

How the mind disects, stores and correlates the information if gathers is somewhat different to the linear recording or cataloguing of current systems though. 

After her first stroke a patient found engagement when otherwise unable to communicate by looking at family photographs on an iPad. After a second stroke the same patient, deemed incapable of comprehension or communication, responded to hundreds of images of paintings she had known in her lifetime – in particular responding to the question posed when looking at one painting. Where is it? Ans; Louvre. What is it? Mona Lisa. (Vernon, 2012)

As sensing technologies become more ubiquitous and wearable a new trend of lifelogging and passive image capture is starting to take place and early clinical studies have shown much promise in aiding human memory. Doherty and Smeaton (2009)

Fig.1. A game of pairs – our minds are far more interprative, chaotic and illogical when it comes to visual associations based on what we see around us.

However, it is presumptious, prescritpive and even manipulative to assume that a person recalls ‘more of the same’ when visualing sensing or surveying a place. The foibles of the human mind and system is that noises and smell, the temperature and weather, and the time of day have a part to play. I visit Trafalgar Square and smell pigeons even though they are long gone. I visit Buckingham Palace and recall finding a woman dead on the pavement one late evening. I see snow and think of the broken leg I got from skiing in my teens – not snowmen. I see any icecream van and think specifically of Beadnell Bay, Northumberland.

The mind is far, far more complex than a fancy game of ‘pairs’. I have perhaps 30,000 of my own images online, so why support, replace or supplement these with those taken by others? What if during my lifetime I tag, link and assocaite these images? How might these be linked to another personal log – a diary of some 2.5 million written over a 30 year period?

There are research challenges involved in further improving the quality of the lifelog augmentation process, especially with regard to “event-specific” lifelog events, e.g., football matches, rock concerts, etc. Other research challenges include investigations into selecting initial seed images based on adaptive radii, more sophisticated tag selection techniques, and also considering how interface design and varying methods of visualisation affect users’ acceptance of augmented data.

REFERENCE

Baddeley, A., Ed. Your Memory: A User’s Guide; Carlton Books: New York, NY, USA, 2004.

Bell, G.; Gemmell, J. A digital life. Scientific American Magazine, March 2007.

Berry, E.; Hampshire, A.; Rowe, J.; Hodges, S.; Kapur, N.; Watson, P.; Smyth, G.B.G.; Wood, K.;
Owen, A.M. The neural basis of effective memory therapy in a patient with limbic encephalitis. J.
Neurol. Neurosurg. Psychiatry 2009, 80, 582–601.

Doherty,.R. and Smeaton.A.F. (2009) Automatically Augmenting Lifelog Events Using Pervasively Generated Content from Millions of People

Vernon, J.F. (2012) Use of hundreds of image grabs of contempary artists, Leonardo da Vinci and Van Gogh to communicate with an elderly patient after a series of catastrophic and ultimately fatal strokes.

Zacks, J.M.; Speer, N.K.; Vettel, J.M.; Jacoby, L.L. Event understanding and memory in healthy
aging and dementia of the alzheimer type. Psychol. Aging 2006, 21, 466–482.

Automatically assisting human memory:

English: Microsoft SenseCam

English: Microsoft SenseCam (Photo credit: Wikipedia)

 

SenseCam browser (Microsoft). A wearable device that takes a picture every 22 seconds. Hodges et al (2006)

 

  • Tools for lifelogging
  • Hundreds of thousands of images grabbed and presented to aid memory … and memory rehabilitation.
  • Automatic content analysis techniques

 

(There is a reason why we forget. The quote from James on the need to spend as much time recalling the record if everything is remembered is like that of Lewis Carroll and a map the size of the real world – neither had the advantage of limitless digital storage capacity and the ability to zoom in and out or back and forth – to expand time, not simply record it.). 

 

  • A visual record of your day. Berry et al (2007)
  • 2000 to 5000 images a day

 

Best practice

 

  • Only activate the device for significant events

 

Methods of review

 

  • Clustered time view
  • Geographical map (required GPS)
  • Interactive story authoring
  • Motion sensors identify events – typically 20-30 in a day.

 

PROBLEMS

 

  • Cognitive overload
  • Keyframe image selection a human endeavour
  • An entusiastic lifelogger might expect to gather 100,000 images a month.

 

OF NOTE

 

  • Key frame selection only of note if it picked a poor image.

 

REFERENCE

 

Berry, E., Kapur, N., Williams, L., Hodges, S., Watson, P., Smyth, G., et al. (2007). The use of a wearable camera, SenseCam, as a pictorial diary to improve autobiographical memory in a patient with limbic encephalitis. Neuropsychological Rehabilitation, 17(4), 582601.

 

Hodges, S., Williams, L., Berry, E., Izadi, S., Srinivasan, J., Butler, A. et al. (2006). SenseCam: A retrospective memory aid. In UbiComp: 8th International Conference on Ubiquitous Computing, Vol. 4602 LNCS (pp. 177193). New York, NY: Springer.

 

 

 

Making swim coaching a tad easier with SwimTag

Fig. 1. A plethora of session plans – what a year of elite swimming training looks like

Swimtag today, skitag tomorrow

Serendipity had me click on Swimtag and I’m hooked – as a swimmer and coach, but for the purposes of this note as a prospective PhD student looking for a research project for the next three years.

Fig. 2 . Swimtag

My interest is in e-learning, sport and virtual assistants / augmented learning.

Armed with a set of swimtags I’d like to research their use with a range of swimmers: masters, elite athletes, learn to swim and swimmers with disabilities. We have all of these in our 1000+ member swimming club Mid Sussex Marlins SC. Early days – I have only just completed a Masters in Open & Distance Education and am tentatively speaking to potential supervisors at the Open University, Oxford Internet Institute and Web Sciences at Southampton University with a view to submitting a doctoral research project in the next couple of months.

My vision is how swimtag becomes as commonplace as swim goggles, then translates into other sports and other fields, including business, but also as a potential prosthesis for people suffering from dementia or memory loss so potentially tied into other data capture devices.

I am seriously looking at funded PhD research for the next 3/4 years.

I am interested in e-learning, so Learning & Development particularly for v. large organisations. There is a groundswell of interest in devices/software that enhance or support memory and learning. There is a fertile crossover between health – providing support say to those who would benefit from cognitive support, what we call ‘lifelogging’ – so gathering pertinent data about the world around you, then using this in an artificially edited form (using Artificial Intelligence algorithms) to supplement memory loss or to enhance learning potential as a virtual companion. Those recovering from a stroke or with dementia included.

It may sound like science fiction but people have been working on these ideas for a decade or more.

I appreciate that simply tagging vulnerable people who may wander off and not know how to find their way home is one way to support but I’m thinking about quality of life and facilitating memory and communication too.

 

%d bloggers like this: