“USE THEM OR DO SOMETHING ELSE”
Details about the conference and presentations.
Website with proceedings and information.
Published paper: Message Bag: Can Assistive Technology Combat Forgetfulness? on ACM, so membership may be required.
This is a look at Augmented Human conference. I had a fantastic experience going to this conference, the talks were a great way to bring life to the papers, and to really understand the motivations for the work and where it was heading. There was also a great diversity of work presented and many left me inspired and excited to move my own work forward. I have made notes as I listen to the talks, and these are just my personal thoughts as I was watching, so they may not make that much sense, but mostly for me to refer to later. Enjoy the photos.
Day 1
Finger Draw
Capture colours and textures from real objects, takes the world as its template, so facing the ‘world’ takes a picture, then the user can apply colours or textures on top of that ‘sketch’, can use colours and textures from the places, user can point at the sky and it will use that colour.. can store those colours and textures, so you don’t always need to carry it with you, so you can carry it in the nature walk, and it will store them for later. Implementation, hardware and software – eye ring, camera, Bluetooth, video demo of the objects working.
Smart Finger: related to the previous presentation:
Magic finger, slurp, pick and drop, digital camera, these connect objects and turn them digital… wanted to capture the digital information from the object, capturing images of physical objects, pasting physical data. Also, transfer between two smart fingers.
Finger mounted webcam and push button on the finger, also recognises QR codes, patterns, smart identification, stickers on apples, QR on tee shirts, face recognition,
Augmented Forearm: Simon Oldberding
Wristwatch, digital information on the user’s wrist, cover the entire space of the forearm, showed other touch inputs interactive bracelet, moving it to interact, wrist movements. Clothing, can this be used for interactions, forearm space, using the physiological properties of the forearm, for a personal and public display (showed display system with Arduino)
Mostly displayed to the public, but area near the elbow is more private… the display could show information relevant to everyone, such a as a tube map – but it could change to tell him where his stop is. Sleeve interactions, sleeve zoom – moves up the sleeve, shows the agenda for example, or can be used to do peek and see private information.. Sleeve –store, as a magician, use for digital items, item at hand (digital) user can swipe item in or could shake it out. Taking into account clothing and forearm. Tiled display was used. Issues could be with ‘twist’ if the display was larger… (small full-colour displays)
EyeRing: related to earlier 2 talks, pointing interaction for visually impaired people potentially, social acceptability is a concern, interactive, portable, now separated into 4 categories, pointing, tapping, gesturing, pressing / clicking. Form factor, several iterations, showed the implementation, UART to serial jpeg camera to microcontroller, digital input pushbutton switch, UART Bluetooth modules, can be considered an immediate interface, hands-free operation as you are wearing it. – shopping assistant, to identify the colours and prices for clothes, (video shown for use for partially sighted / visually impaired people) smart copy-paste also demo’d.
Study : within subjects, identify currency notes, 12 participants, when the number of notes are smaller, then iring was better, when there is 5+ notes it is quicker. (several articles are written so research into it possible)
Whole hand modelling:
Studies and further work, measurements for how to better track the motions, pinch etc
Tangential force sensing system on forearm
Previous tactile sensors, human-object: eg touch pad like iPad
Object-object, robot hand as an example, embed a tactile sensor…
Human-human
Sensors on fingers, acceleration (ando 2009) strain gauge on finger, vibration through finger, Makino 2010 with these the human can touch anything with their index finger. Only single finger,
Their system, two infrared sensors on the forearm, which becomes active sensor, so the area between can be touched, you can see the skin deformation, force, to use for evaluation massage effects, usability of cosmetic products. Human machine interface… forearm modelled with points a and b and where the force is applied, simulation – colour showing upward / or across..
Played pong on the arm as a demo, other uses for skin surface for input device, so far these do not detect the force, whereas this one does. Limitations is one dimension, could use multiple sensors, the posture of the hand affects the measurement. So arm has to be in a fixed location to avoid this.
Manipulation of Emotional Experience
Concept creating your emotions virtually.- emotions from behaviour, changes to facial expressions affect emotional state.
Examples, false heart rate experiment study, culture independent, universal features of emotions, not actual changes, but recognised as if changing, although you are not physically smiling, you recognise the change of expression as your own. – no wearable device required
Generating deformed faces to display face emotions, smile etc Facial feedback can manipulate emotional state, facial feedback can manipulate preference
Roller type itch relief device
Methods of itch relief, medication, pain, temporary relief, scratching, pain, but damages the skin. Introduce pain or burning sensations, by the simultaneous hot and cold, painful illusion, alternating hot and cold, roll the device over the area, – itch stimuli to patients cheek, lactic acid was placed then scale 0-100 0 non, 100 very itchy. – several conditions, off, vibrate, temperature, or vibrate and temp
From tests showed hot and cold temperatures were most effective
Paired Vibratory Stimulation for Haptic Feedback
Touch panel devices, mobiles, games, tablets, cameras etc, many have lack of haptic feedback, so which are we touching, what is the input? How can we know which we are touching? Tactile feedback, it is only at the point of contact – but what about slide for example, can use the vibrator in the phone, so you feel the vibration in both your holding hand and your finger – so it doesn’t happen only at contact point. Study done where there is a small vibrator on the finger. Nail mounted tactile display, 2002 Ando
A paired vibratory Stimulation using two vibrators, one attached to device one to fingernail. Shown frequency at 200kHz most effective- but over 1000 we don’t feel. We can feel a high frequency if there are two close high frequency,
Do we really feel vibration with this setting? 1.2kHz and 1.4kHz
3 types of stimulation, both on, only device on, only fingernail on, then asked if they feel vibration or not, device and fingernail, is overwhelmingly better, almost 100% device only is less than 10% then asked where the vibration happens, looked at iphone, paired, or fingernail 200hz vibration, with this system, it is the contact area that feels it the most, other possible applications for this system? Human skin surfaces, hard to give haptic feedback, possibly, use 2 vibrators again, for skin surface detection on contact area.
Human augmentics, create something that can sense you invisibly, lots of things are invisible and extremely deadly, example, radiation, comic heros can detect them, spiderman has spidersense, devices use data visualisations as displays, sensor receptors on body, braille alphabet, why not use the receptors to convey information, tactile displays, related work, gemperle et al (tactile displays)
ActiveBelt, eight vibrators, Tsukada et al, distances conveyed through pulses
Human senses blind spots – someone approaches from behind or follows, system to create an awareness of the environment… actually feeling of the environment around you, controller box and sensor modules, servo motor, pressure arm, ultrasonic motor, as you are closer you feel more pressure, as you are further you feel less pressure (Arduino mega) 10-pin connectors to plug in or out sensor modules,
Pulse; reflection; send reading; process adta; send it back; then next sensor module…
For as many sensors as you have… can’t operate 2 or more at a time, interference at the moment… servo rotation also creates interference, need to wait for the rotation to finish, all the sensor modules, takes 52milliseconds, 1.3 seconds as a maximum… several experiments, senses someone behind, then given a task to navigate through an area with shelves etc… Then testing outside, then open environments, then ninja stars to be thrown to them as a final test
Possible applications, similar devices to be used to compensate dysfunctional or missing sense.. maybe exterior of a car, visual impairment,
Feedback for firefighters
Device free interaction – what is the sensor at the end, for interaction method…
SEMarbeta: sketch gesture video
Urination splattering…
Using music / notes that they hit…
AR systems
Medical, architectural, navigation,
Room sense, using active sound probing
No standard for in room positioning, possibly use dedicated infrastructure, or existing infrastructure; this is an indoor position system using no infrastructure, based on sound fingerprint, for pre trained indoor positions, sound reflections, it is unique for the room and even the positions within the room, as a fingerprint, passive and active fingerprinting, use input sound, background sound, active – needs loudspeakers, can use a smart phone, room impulse measurement, microphone and loudspeaker,
A sensing architecture:
CEEDs the collective experience of empathetic (data?)
Example, to find a travel destination, we present pictures, measure the gazing behaviour, then also heartbeat measure, sync could be a potential issue, make sure sync between images and readings from sensors are accurate, also to be sure no large delay.
[symple_testimonial by=””]
Thad Starner
USE THEM OR DO SOMETHING ELSE (this was one of the key things said by Albrecht Schmidt, conference organiser, which really hit a note, how if we are making devices, we need to be using them… and it was especially appropriate before Thad’s talk, as he has been wearing prototypes for 20 years)
If you build something, no point in just looking at it, use it in everyday life, constant use to understand what it is good for, how to use it, to see the social interactions, use it or do something else.
2 second rule
[/symple_testimonial]
DAY 2
Device free and device bound, radio waves,
device free approach, infers the context of the object through how it is changing, moving,
Improving Activity recognition
Without sensors, with time use surveys, filling in questionnaires, about the days, looked at german and American, activity rhythms, repeating activities, adding sensors to the body, looking at long term, sleeping, showering, train, work… Rhythm, Modelling.. possibly use for elderly, possibly predict future activity, time use surveys, usa every year, Germany every 10 years, german one has 10 minute time slots, showing you need to use your time use survey from your own area, as there are differences, TUS shows promising results, maybe wear a sensor on your wrist, use prior knowledge to boost activity recognition. Idea is to use this to then enhance things like fitbit etc… when you are sleeping,
Activity Recognition
Weights and exercise, improving technique… how to asses the quality of the technique you are using…
Engineers meet clinicians: Parkinson’s
Freezing gate, Parkinson’s sufferers, device to sense or predict when this will happen, a cue of ‘tick’ sound, so the sound needs to happen just before this will happen…
Ball POV
My presentation which was on day 2, after the first break.
A tool for mental workload, evaluation and adaption
Sensors put into the skull skin, metal workload, adjust the speed of text presentation in real time using mental workload, portable and dry electrode device: EMOTIV higher IQ higher mental workload? Do you get better at handling it if you consistently have a higher mental workload.
Investigation, fNIRS brain sensing,
Carlton 2009 Bailey 2001 Bawden 2001 stress annoyance anxiety, relationships, Brain sensing,
Pupil dilation correlated with workload, functional Near InfraRed Spectroscopy
Emotion + Reason, Prefrontal Cortex, ratings over time, ratings improved, the more movies people saw the more they did like, unifying information from multiple sources, can we use sensors to optimally deliver information using brain and body sensors.
Who are you? Camera recognises faces, could put by the door to recognise people about to come in , maybe your next appointment etc..
Design of artifacts
Not to replace, but to assist, not to enslave but empower, Design ethics, Steve Mann, cyborg.. Englebart, augmenting human intellect, problem-solving, 1962 Englebart, memory, motivation, decision-making, mood, Fear Factor, Engle, humans don’t exist singularly, exist in the system with language, artefacts… with all these associated units tied together closely, H-LAM/T
Augmentation via processes, you can augment a task by making it easier to do a process within the task. Lowest energy state, to make a process easier makes a person more likely to do it verticallearning.org
What would you like to be the desired state, then make the technology to help the user get to the desired state, design the things to enable that, (ask yourself, how would I like to change to achieve my desired state…)
Consider the desired state after augmentation, identify the involved proceses for the task, identify how artifacts can change a process or the process hierarchy
Desired state, sync short term, actions with long term goals – I have long terms goals, I have long term vision for myself, wearcam.org always consider ethics as you design to augment humans.
NeuroPlace, Environmental Psychology,
Parks and libraries perceived as relaxing, shops can make us stressed… (jewellery shop?) EEG research, games, learning, car driver fatigue, out door environments. Classifying the data into the type of place it is based on emotions felt into that area.
Monitoring device
Assistive technology, used to increase, maintain or improve functional capabilities of individuals with disabilities, AT that do not stigmatise people, but that are defined by the persons own life goals and are perceived as optically attractive
RFID tags as reference for phone location and orientation in daily life Florian Wahl
Where do people carry phones… men 60% trouser women 61% handbag… from this paper, User diaries, effort intensive for people and researcher, error report, often phone location is changed subconsciously, using RFID tags to locate the location of the phone, NFC, tag shirt pocket / trousers etc, with stick on patches, the phone logged with times etc, location, only read tags with one side so put two tags in, CRNTC+ ID recorded instances, evaluated against the user diary, user diaries drop drastically over periods of time, 4 misses of tag readings, user generates own config file, predefined categories, scan tag, tag description, etc… 83% accuracy, throughout the whole time, diary, you have high accuracy in the first few days, then drops. Tag configurator application, larger evaluation in progress, fine tuning of tag placement. Could set to unknown if the phone is opened, as that means it is being used. Desk with tags in places – then place items on that tag. Tag entry / exit points…
