As part of the Being Human Festival, I was involved in the technical support for the Sonic Bodies installation. This project from Maria Kapsali was bringing together Sonolope and some new explorations. These explorations included bringing sculpture to visually impaired people. Works were prepared ‘Pan’, and ‘Kiss’, based on artworks that were present in the gallery. These pieces were then sonified, and created into experiences through movement.
A project I am involved in a technologist capability is HOUSE, where I will be using Sonolope to add interactivity to the performances happening over the week of Tuesday 30 January 2018, to 3rd February.
Details of the project:
HOUSE begins with the story of Robert Arthington, a rich man locally known as the Headingley Miser. He built a large house for his bride; but the bride never came. So he lived alone in one room, on half a crown a week, and received his visitors in the dark.
Around him 19th Century Headingley was growing from a country village separated from Leeds by fields and farms to a vibrant suburb, where industrialsts and imperialists alike built themselves splendid houses. The miser’s millions meanwhile were supporting missionary projects around the world. Arthington, Liberia, bears his name to this day.
Created by A Quiet Word in collaboration with members of the local community, HOUSE is a site-specific performance that explores how property and power connect Headingley and the wider world. You are invited behind closed doors for a conversation in the dark, and to where the present overlays the past.
Tuesday 30 January to Saturday 3 February, 6pm and 8pm
Booking is essential. The performance lasts approximately two hours and involves some walking outdoors between sites and upstairs. Due to the domestic nature of the sites, access is limited. Meet at 57 Headingley Lane, LS6 1AA.
Booking fee of £5 per person. To book, visit eventbrite.com and search for ‘House’, or visit aquietword.co.uk/house
Proceeds will go to Shelter and St George’s Crypt.
This is a bit of a retro post for me. I wanted to capture some of the Apps and Hats show that myself and lovely Hattie did for a while back when the iPhone was launched and the App Store was just getting going. Continue reading “Apps and Hats”
Write up of some of the details from the Crumpler Prototype.
Details: iteration 1.0 This change was the first proposal after working through the proof of concept prototype there were issues to address and further investigate. Initial changes from the testing prototype bag was removing the LCD screen, because I was trying to scale down and wanted the simplest of communication with the user. The following table is the items that were used to build this bag. There was a period of testing different circuit boards for their compatibility, programmability and aesthetics. After researching the available components at this time the LilyPad series of items seemed the most relevant as it offered a way to use components that could be attached to the outer surface of the bag, therefore leaving the bulk of the bag mostly unaltered. No compromise on space or weight for the original bag.
USB LiPoly Charger to charge 3.7V LiPo cells at a rate of 500mA or 100mA per hour. It is designed to charge single-cell Li-Ion or Li-Polymer batteries. The board incorporates a charging circuit, status LED, selectable solder jumper for 500mA or 100mA charging current, external LED footprint, USB input, mounting holes. There is also a ‘SYS OUT’ which allows you to connect the charging circuit directly to your project so you don’t need to disconnect the charger to use.This is a very slow way to charge the batteries, and this is an issue that will need to be addressed.
LilyPad Buzzer This is a small buzzer, use 2 I/O pins on the LilyPad main board and create different noises based on the different frequency of I/O toggling. Loud enough to hear inside a pocket but not obtrusively loud. 20mm outer diameter Thin 0.8mm PCB
SLO18 RFID reader 5V Supports Mifare 1K, Mifare 4K and Ultralight. It does auto Real-time detecting tag which moves into or out of detective range and reports through one output pin’s logic level. In addition, it integrates all necessary components and antenna in one PCB. Frequency : 13.56MHz Protocol : ISO14443A Tag supported : Ultralight, Mifare Mini, Mifare 1K, Mifare 4K, FM11RF08 Interface : I2C Supply voltage : 4.4 – 7.0VDC Dimension : 65 × 45 mmUsing this RFID reader would mean finding a board suitable to work with the 5V requirement.
Polymer Lithium Ion Battery – 6AhrEach cells outputs a nominal 3.7V at 2000mAh – 3 cell pack (6Ahr) is terminated with a standard 2-pin JST-PH connector – 2mm spacing between pins.The 3.7V may be an issue that needs looking into – to get enough power to the board to light the LEDs/
Lilypad 328 Main Board has more I/O ports to access and use for tags / LEDs and operated at 5v so the reader choice had to meet with the same voltage.It has an ATMega328 and Arduino bootloader. It supports automatic reset 55mm outer diameter 0.8mm PCB
Slide Switch Simple slide switch to power ON/ OFF the bag. It can be used to switch other things, but that is it’s purpose for this bag. 7.75×18.1mm Thin 0.8mm PCB
FTDI basic Breakout 5V This is only needed for programming the board, and can be used with all the 5V boards to program. USB to serial IC. The pinout of this board matches the FTDI cable to work with official Arduino and cloned 5V Arduino boards. It can also be used for general serial applications. The major difference with this board is that it brings out the DTR pin as opposed to the RTS pin of the FTDI cable. The DTR pin allows an Arduino target to auto-reset when a new Sketch is downloaded. This is a really nice feature to have and allows a sketch to be downloaded without having to hit the reset button. This board will auto reset any Arduino board that has the reset pin brought out to a 6-pin connector.This board has TX and RX LEDs.
Details of Testing Programme
This prototype is a departure from the first proof of concept bag. It was carried around on all my daily journeys to test the bag initially for working issues, sociability and general fixes that would need to be done before testing it with a user.
Results and Conclusions for next Iteration
This bag I used on a daily basis from March 2013 until May 2013, and then less frequently due to moderations / user testing from June –July 2013. The bag is used daily highlighted a lot of issues. Comments from this time period: The importance of the placement of the on / off switch – it was hit accidentally a few times, and I didn’t realise some times – so moving it to better placement / alternative, and maybe adding a vibe board to alert the user that the bag had switched on, so if it was knocked then they would be made aware. With this first iteration, due to the much larger amount of LEDs (the first system had 5 this one has 13), I would have to place a small card in the bag to remind the user of the items and the corresponding colours as there were a lot of LEDs. This doesn’t seem as practical and I did have people asking me how I would / or how someone would remember the LEDs and what they were for. This needs addressing. The battery weight and charging would need to be addressed, there isn’t a clear way currently to be aware if the bag is low on battery power or how long it needs to charge. After using the bag, I wanted notifications to be more intense the nearer to the time the event / leaving etc, happened… this is scope for future work and implementation because I’m not sure how this could work. There was an occasion where I had forgotten my student card and I needed it so as I was leaving it would have been good to have gotten a more urgent notification of some sort. Perhaps indicating the items importance/use or urgency. I was trying to maximize the number of tags that could be read, but I think this would overload a user. Lastly, it does work in the rain and the battery casing is inside the bag so there is no issue from that aspect too.
This prototype (Message Bag 1.1) builds on the initial message bag, and has the basic features of it enhanced. It serves as an initial testing prototype to work through initial flaws and design decisions as well as features and how they would work, or what purpose they would serve. The initial message bag that was created has had the components removed and they are built from the ground up, being modified or altered to better and more accurately serve the needs of this prototype.
Message Bag, Nov. 2011
Message Bag has an integrated LCD for messages as well as an RFID that activates LEDs so you know what items you have packed. Using Arduino.
for V1.1, these components are available from most sites that sell Arduino items
Initial thoughts were to get an LCD screen to display messages, so I did some research into how they worked, what different kinds there were, this information from Arduino was very close to what I was looking for. My LCD had a back light so there were 2 more wires to add, an additional power and ground to power the screen.
I also incorrectly assumed the 16 x 2 size screen or larger would be too much for a first project so I went with a 16 x 1. This caused me a lot of challenges that I wasn’t aware of at the time. This post from Tronixstuff, had a good rundown of the different types of LCD screens you could use.
The issue with the 16 x 1 display was that it displayed 2 rows of 8 characters side by side. This firstly took me a log time to realise and work into the code as well as it didn’t really have the exact effect I was after. Due to time constraints, I went with the one that I had ordered, I would like to replace this or do a different screen if I make another similar item. I would go for 16 x 2 as a minimum.
You must include the LiquidCrystal library, and here was the code for that as well as setting the pins.
You must then add to void setup() the initialization of the LCD by telling it the number of rows and columns.
lcd.begin(8,2); // start the LCD, a 16 x 1 has 2 rows of 8
Then within loop() where you want your display to print something you need the following code, so mine was within sensor values and if statements dependant on the values of the sensor readings:
lcd.print("You are too far.");
The distance sensor was one of the first sensors I implemented to test it and get the code working initially to get readings.
The idea here was that it would change messages based on the distance of the bag to people. The issues with this is it didn’t track huge distances, and also it could literally detect anything so it wouldn’t be changing according to people, so it was pretty inaccurate for my purpose. Other issues became that it was reading values pretty quickly, even with a slight delay, so the messages were recycling too quickly. When I slowed it down, it seemed to users that the change wasn’t registered. After watching people use it and getting feedback this would be something that I would need to alter.
This sensor was pretty straight forward although I had noticed I was getting erratic readings when testing it – so I swapped the resistor and it seems that the one I had been using was broken. Switching the resistor fixed it and it worked well as a way to light the bag in the dark. A future addition would be to be able to turn off these lights if the wearer didn’t want them on. Code from Arduino is pretty straightforward as well as the schematic.
Method for Arduino (Teensy) Board 2 RFID Reader :
The RFID code was on a separate board and part of the bag, so essentially I had two separate code files, one for each Arduino. The reader I was putting on a breadboard with a Teensy so that the whole unit could fit into the inside pocket of the bag. The teensy needed a power source as it doesn’t have that as part of the board so I added power from a battery as shown at pjrc.com.
For me the RFID part of the implementation really felt like it was an idea with potential. I loved getting this to finally work and it seemed a good addition to a bag and a feature I would like as well. Here is a short clip showing the LED lighting up when the item is placed in the bag.
I had an issue with the Reader and it seemed to read the values initially then stopped reading and displaying the values. I tried changing the components, altering the wires, changing the Arduino board and when I was still getting sporadic results, it was suggested that actually my soldering had not made full contact and this could be the issue. I checked the board and it was the problem. Because the board was soldered first into a breakout board it was a really difficult issue to fix so I initially used a screwdriver to bang it closer to make contact with the wire. I have since re-soldered this board and it has fixed the error. As it was my first time soldering I hadn’t realised about this being an issue so although it was very frustrating it was a good lesson to learn. As far as the code goes, initially I put code on to read the tags so I knew which numbers I was reading. Then when the numbers are recognised, some action happens.
In the loop() we are reading the serial numbers. When the code is uploaded you have to remove the wire going to the RX, and then put this wire back so it can send the values we are looking for.
The tags are then checked to see if they match the values we declared already – if not we are given the value on the serial monitor.
This was pretty straight forward to add, I put them on a little circuit board to space them evenly for display on the bag. I had an issue again with my soldering and two of the LEDs were not working. One I had to remove and res-older and the other light was broken so had to be replaced.
This was added so that there was feedback for the user when they scan an item. It just emits a noise so they know it was successful. I also included a small LED so they can also see that it was a success in case it’s too loud where they are.
Implementation & Observation
The code I implemented in stages to check what was working and what needed to be tweaked. For this new implementation, I removed the distance sensor from the bag, and in V1 the LEDs responded to lighting in the room, when it was dark they would produce a pattern. In V1.1 this will be a more used feature, and the LEDs will now light to acknowledge that a tag has been scanned, providing visual feedback for the user. In addition to that there is auditory feedback through the piezo – this was implemented in V1, however this time, there is a slight change in that if the tag is recognised, it is a short pulse, slightly higher pitch, when it is not recognised, the tone is a little lower, and lasts for a longer amount of time. With this change in sound, it should make it quicker to notice when an unrecorded tag has passed near the reader.
Additionally, the LCD was used to display messages based on proximity, mostly for a fun factor in the bag for V1, in our second version, we now alter this so that the LCD will display the name of the item that is scanned by the user. Again providing additional cues for them to note what has been scanned.
There are still LEDs to help them to see what is in the bag currently, but this still isn’t done in an ideal way, the LEDs light up on the outside which is helpful, but the tags will have all had to have been pre-scanned and added before hand, so this is something that I would look to modify in a future version. Additionally, just having the items written on the outside is a little clumsy with hand writing on the outside to match the LED, this really needs a rethink.
Additionally, just having the items written on the outside is a little clumsy with hand writing on the outside to match the LED, this really needs a rethink.
The current RFID reader, the ID12 has limitations in this capacity too. Shown, is the breakout board that needs to be used to mount it a little more easily, by lining up the pins into a breadboard. You don’t need to connect an external antenna which is a really great thing about this particular reader, and there are only a few pins needed. One thing to note is that you will be connecting the RX pin, which is also used when you are uploading your sketches so you must just remove this wire when you upload the sketch, and then replace it for it to work. I have seen an implementation where someone didn’t use this pin and did say it still worked, but it isn’t something I have implemented or tried, and as I will be using a different reader next time it won’t really affect the prototype that will be used for a controlled study.
For a future iteration, I will implement a different reader, that can work more ‘real time’, making note of when an item has been removed from near the sensor, currently this isn’t the case. Also, things like the size of the components will need to be taken into account, and using a teensy in place of the Uno would be another change to implement for message bag 2.0.
This was created after attending the Augmented Human ’13 conference. I wanted to create a second working prototype that was more robust than the first iteration. The first one created was a proof of concept style bag, so that I could visualise how these components could potentially work together, but it also provided valuable insight into how the bag may or may not be used and general problems that will surface when you build a physical object. (note this post does not cover the working description of what it does, it only charts the creation of the bag and observations having used it daily).
There are many challenges when working with wearables, there is a balance with innovative, possibly unusual interfaces, power requirements (which I have been dealing with), possible network resources and privacy concerns. (Starner, 2001) There is also a need to establish what social boundaries there may be which may limit the success of the bag.
This second iteration has been built and deployed, it is still a prototype, and I am using on a daily basis. The goal of this is to determine the suitability of the bag and RFID readers for use in our everyday lives, using it every day will bring to my attention things that are not practical, can be improved or just do not work. Some of the issues that were brought up with the first bag are addressed and modified or removed to test what differences there will be within the functionality of the bag. The other main objective of this prototype is to develop a comprehensive understanding of how Message Bag v2 can be used, and use this as a basis for further research into forgetfulness, possible triggers and methods that work best for retrieving a ‘message’ of what you have forgotten and how ubiquitous computing can be further used for alleviating those symptoms of stress associated when an individual does forget.
In Message Bag v2 there is no longer an LCD display and there is more emphasis on the LEDs for memory aids for the user. The initial reason for removing the screen was the pull on battery life, and also the ability to read the screen at a distance – say across a room, is a lot more difficult than a bright LED that can easily get the users attention from across the room.
If the LED is flashing then it is virtually impossible to ignore and will capture the user’s attention, that is a physiological reaction that has been used in stores to attract consumers attention before so we know this does work. Similar to the first prototype, we still use RFID technology with passive tags to record the items that are in the bag. Passive tags use the incident energy to both sense and communicate – they transmit their series of letters and numbers, and can be a little less accurate than active tags. A major advantage though is that they have a much lower cost, are small and they do not need a battery.
The first bag used an ID-12 reader, but as it is a little bulky and harder to secure in a wearable environment this was changed to an SLO18. This also has an integral antenna, but it is flat and has holes at the corners making it a lot easier to secure in a wearable item.
Micro cable used to connect the board to the computer to program the board – also needs an FTDI Basic Breakout 5V to connect it, that’s only needed to program the Lilypad, once you have your code on it, you remove the breakout and you can use it for other projects, so you only need one.
I used this USB LiIon/LiPoly charger for this project but have since found one also from SkPang to ease the order by going to one site for most of the components. (since then I also found a cheaper alternative at Hobbytronics)
There is also various consumables, such as the thread used to sew the components to the bag and I also used multi core wire to wire in the RFID reader to the Lilypad, mostly because there was a possibility of a few wires overlapping which I didn’t want, I wanted a strong connection and I went multi over single core so that it would be more flexible in the material. Initially, when I was sewing the bag, the wire I used was really hard to sew, and frayed a lot and had a lot of resistance to the material making it time consuming to sew. I then found a 3 ply medium conductive thread which seems a little stiffer and slides through a lot easier. You also need a variety of tags to use with the bag.
The components are small & discreet enough to go unnoticed in an obvious way when the bag is turned off. The bag was taken out in rain and the components did get wet, the surface of the components was covered in droplets but it had no effect. Additionally, because the sewing is done on the inside of the bag, the circuits themselves did not get exposed to water. The bag does have water resistant properties so the interior was always protected. No damage sustained and the components are described as able to be in water i.e. you can wash them if needs be.
The objects that I was aiming to pack and remember, are packed without issue, but on one occasion, I wondered if I forgot my train tickets, and actually had forgotten my railcard and oyster – so these would need to be tagged as well. Part of the issue will be deciding which items to tag, and be sure that the items I find essential I have pre tagged. This will also be an issue when I come to prototype the next iteration of bags that will be used by testers, they will most likely at this stage have to tell me the objects they want to be programmed and it will have to be done before I issue them the bag. This isn’t ideal and will most likely need to be addressed in further bags.
Additionally, because I had forgotten my card, it would be handy to have a more persistent reminder as it nears the time that I need to leave, maybe the lights flash brighter if I have notified the bag that I have an appointment at 4pm and it knows I have still not packed my bag by 3.45 or similar. In keeping with that observation – can the bag notify me as I set my time of leaving? Do I type into the bag daily or weekly when I leave? Is there a more intuitive way to accomplish this? Can I somehow tell the bag what time I am leaving in the morning, maybe preprogrammed in some way, to say as it gets nearer the time and maybe I am more frantic with trying to get ready, that it has some kind of audible noise? Or maybe I record a voice message saying the items, and it plays this back closer to the time – and louder? Unless I dismiss the notification? Or each item has an audible reminder, and if the RFID is there that particular reminder isn’t played? So I would record my voice for each item? i.e. scan a tag – record my voice, and then each one is labelled alongside that item? What about turning it on in public to repack it – say at work? Can you lower the volume? or disable voice? Maybe the lights flash faster the more important? Is this a common cue? the flashing meaning ‘hurry up’?
There isn’t a low battery warning for a user, how will they know that the battery needs charging or is it a case that they plug it in every night?
They need to turn the bag on or off, will they remember to turn it on? Or to turn it off?
Also, a few times the switch for the single LED switched to on – this is not good as it may have potential embarrassment if it goes off in an environment that the user does not want it on, especially if it has the audio working and it is notifying you that items are packed.
From using the bag on a daily basis, both during the week and weekends, at all variety of places and times (late evening shopping for groceries, going to teach an evening class, walking in the daytime in town, travelling on the train, the bus, the underground) it is difficult for people to remember all the lights connected to which items, this is very important as the hope is to reduce the cognitive load not increase it.
The next bag iteration will feature an LCD screen again, and a reduced amount of LEDs. I had comments about the bag in a few environments including a college where I was working a supply day and the students commented on the bag, ranging from thinking it was cool, the idea was cool, to some saying it’s cool but still looks a bit homemade… so there seems to be general social acceptance but some hesitation due to the fact it is actually ‘sewn’ on. As the prototypes get developed this is also getting more honed in and precise.
Something that came up in previous similar studies is that people noted that they would like some sort of weather notification so they also knew the context with which to pack their bag. This could be another future implementation to create a truly smart bag.
Websites for Components
Websites I have purchased these components from or the components are available:
This post examines The interface described will be coded using Xcode, using the interface builder (fig. 20) section of Xcode as well as coding the elements that will not be immediately visible on screen (fig. 21).
Yo! Create (iOS App), Oct. 2011
An app to help creativity, through the use of exercises and activities to try. Coded in Xcode.The app needs work and a lot of additions still but was an initial idea to try to incorporate ideas about creativity, and idea generation exercises to help people who are trying to think projects through etc. It is something I want to continue work on and I think it would work really well as an iPad app, with more space visually and possibly more varied techniques for them to try.
Almost a year ago (2010) I had an app idea for an app that I would like to use with/for my students. I got very excited at the idea, and set out to map it out and around six months ago, after brainstorming with a Twitter mate the app name of ”Yo! Create” came up. It was a brilliant idea and I had to go with it, so I booked it on the app store.
Meanwhile, (back at the bat cave and all that), I set to design it for iPad, and decided to use the shiny new beta of Xcode, which I knew would probably mean apps would be able to be submitted for October time. ‘No problem’, I thought, because I wanted to get this first version right and lay down the foundations for updating it too. I tried a few alternative interfaces and finally settled on one particular design.
I started building it with the new StoryBoards feature in Xcode and then by the time it was looking alright and like I was making progress I got an email saying that I was going to lose the name for ever if I didn’t submit my binary within two weeks. There had been other notices but it was going to an email address that hadn’t been working so I didn’t realise this was about to happen. By the time I realised, I had over a week to submit my unfinished app.
The problem was- I’d done it in the Xcode that wasn’t able to be used to submit an app with yet, and it had Storyboards, which wasn’t backwards compatible *pulls out hair*. So I had to go into an old Xcode and start fresh, just make a quick iPhone version and get it submitted so I didn’t lose that name.
Long story short… I managed to get a working – and not to shabby version of Yo! for iPhone submitted and approved. I have not made it live on the store just yet as I am now in a dilemma – do I resurrect the original iPad version and build upon that, resubmit and go from my initial plans? Or do I work with what was approved as it’s actually a good base to the app anyway and add the iPad version as was intended?
I am bordering on just releasing the initial app as it is for free to get some feedback and make decisions from there… but just to send a little reminder out there- plan in the timeframe for your reserved app names as you may find you lose it forever.