Additional Notes for Martin Maguire's Presentation
Slide 1: Title page
The title of this presentation is “Trends in User Interfaces and the Implications for People with Disabilities”.
It is presented by Martin Maguire from Loughborough University.
Email address: firstname.lastname@example.org
Slide 2: The Loughborough Design School
I am from the new Loughborough Design School at Loughborough University.
Originally I worked at HUSAT founded by Professor Brian Shackel for the Ergonomics and Safety Research Institute was formed led by Magdalen Galley in 2002. In 2010 we merged again with colleagues from the Department of Human Sciences and the Department of Design and Technology to form the Design School. The School has about 700 students and about 90 members of academic, research and administrative staff.
The web address for the School is: www.lboro.ac.uk/lds
Slide 3: Introduction
There are a number of developments and trends in user interfaces for consumer products and public systems.
This talk will review some of these trends and discuss whether there are benefits or barriers for people with disabilities including people with age related disabilities which may be less severe.
I feel it is also important to include people inexperienced with technology and less confident in using it. This may include both people with disabilities (although of course many are also confident) and the mainstream population.
- Standards and mobile keyboards
- Trend towards touch
- Larger and more intelligent screens
- Bar codes and the fridge
- Gesturing to products
- Talking to products
- Ambient interfaces
Slide 4: Standard sized and mobile keyboards
ISO standard keyboard guidelines are well established for standard computer keyboards but do not really fit mobile devices which have smaller keys that are much closer together with less travel and smaller labelling.
Specific guidelines for stationery detachable keyboards are as follows:
- Key strike surface 12-15mm wide by 15mm high.
- Distances between adjacent keys 18-20mm.
- 3.2mm spacing between keys.
- Pushing force <= 2 Newtons.
- Displacement or travel 2 to 4mm.
The Tiresias Mobile phone keypad recommendations for disabled users (from developed by John Gill are useful to apply to mobile phones or small devices as they reflect good ergonomic practice:
- Good visual contrast between the keys and body of device.
- Key tops convex or flat with raised edge.
- Keys as large as possible.
- High contrast, clear and large visual markings on the key tops.
- Ideally internally illuminated.
- Keys raised above phone body (5mm).
- Tactual indication on the '5' key.
- Pushing force 0.5 to 1 Newton.
- Auditory and tactual feedback.
- Function/numeric keys tactually discernable.
Can we offer some numerical baseline values for keypads on mobile devices?
Slide 5: The trend towards touch – mobile phones
A mobile phone was recently advertised as a “Big Button Mobile Phone For Elderly OAP Persons, New Unlocked” – a functional phone but with advertising unlikely to attract older or visually impaired users.
One of the strengths of the mobile phone market is that there is such a wide range of phones available and the opportunity to play with handset models in a phone shop that customers can often choose a phone that meets their ergonomic needs in terms of size and feel of buttons, clarity of labels on buttons, clarity of text on screen, etc.
Many people with less severe disabilities are thus satisfied with choosing a mass market phone to meet their needs. There are of course phones designed specifically to meet the needs of older or less confident users. While they meet a certain market need, they are not attractive to many people with visual impairment.
An ex-student and colleague at LU, who is blind, much prefers to use current mainstream technology if he can and so has always chosen a standard mobile phone. This meant he could get upgrades when each phone contract ended and so that his friends could borrow and use his phone. He used to look for an ordinary phone but with buttons that were distinctive and which he could operate. Now he has moved on to using an iPhone.
Slide 6: The trend towards touch – mobile phones
There have been many developments which have helped make mobile phones more accessible including screen reader software such as ‘Nuance Talks and Zoom’ (Screen Reader & Screen Magnifier) for symbian phones and ‘Mobile Speak’ (another screen reader for a range of phones). Now on the iPhone and iPad, accessibility features have been built into the device and appears effective.
The iPhone low vision accessibility options include:
- White on black
Thus users with visual impairment can use the latest mobile technology.
Devices like the iPAD are also seen as helpful to engage people with mental disabilities.
See for example the following video which shows how an iPAD has helped an autistic child to speak:
See also a touch to speak example: http://www.youtube.com/watch?v=vomkNSluWW4
Should the iPhone and iPad accessibility features be standardised on?
Will Android offer the same accessibility built in to its operating system?
Is there anything that either Apple or Android phone do better to provide accessibility that standards could be built on?
Slide 7: The trend towards touch – good ergonomics?
While smart phones with touch screen have now become so popular there is still a question mark over how usable they are especially for people who do not consider themselves experts or enthusiasts with technology.
They have some important benefits such as the space for a larger keyboard than many phones with physical keys and allow techniques such as predictive text and SWYPE to improve typing. They also allow images and text to be zoomed and generally offer a flexible attractive way of interacting.
However touch screen phones without physical buttons give no tactile feedback (although some phones try to simulate it), there is no physical indicator on the 5 key of a keypad, the user needs to be quite dexterous to operate it.
Using my own phone it is easy while using the camera to press the ‘back’ button shown with the backward arrow and to go out of camera mode.
(Mobile device covers can also cause problems such as a particular Kindle cover which it was found covered the page turning keys on the edge of the device.)
There are also a number of gestures on phones like the iPhone that need learning: single tap, double tap, three finger tap, double tap and swipe etc. This adds complexity to the user interface although once learned gives the user access to a wide range of features.
Question: Can these gesture actions be standardised?
Slide 8: The trend towards touch – surfaces
Many other devices are now moving towards touch screen interfaces including public information kiosks, heating control panels, larger touch surfaces (e.g. in museums) and Microsoft's touch table which potentially could be used in the home.
These kinds of interface tend to be more engaging and in the case of a large touch surface will be more accessible to people with mobility impairment or cognitive impairment.
Slide 9: Larger and more intelligent screens
Screen technology has advanced drastically over the past few years leading to CRT displays being replaced with flat screen LCD and plasma displays.
Many people now have much larger flat screen TV which can be beneficial for people with vision problems in viewing the screen image and screen text (e.g. Teletext or electronic programme guide) and for people with hearing impairments to view subtitles.
The next development is the introduction of Smart TV which provides internet access via the TV. The Samsung Smart TV, for instance, includes access to the Samsung App Store, streaming video from your online video subscription services, and surfing the web. This way of interacting may make interacting with the web, using email, making video calls and downloading ‘Apps’ less daunting for those who don’t feel they want to interact with a computer. This is relevant when it is considered that according to Reuters 25% of people in the EU have never used the web (15 December 2011) and according to the Daily telegraph 8.4 million Britons have never used it (16 November 2011).
Getting the SMART TV connected:
Some new TV style smart devices such as the Samsung Window act as touch screen to access applications although the need for this type of device has yet to be proved.
SMART TV review demonstration videos:
Similarly 3D TV is a new development in TV domestic watching. Further research is needed to see how this can be of benefit for people with disabilities.
Slide 10: Bar codes and the fridge
Another user interface development is the use of bar codes to register products as they are loaded into the fridge.
The user manages food shopping by scanning the barcodes of items, or a shopping receipt, with a scanner on the fridge door, or by describing the item via voice recognition technology. The fridge's computer can monitor its contents and automatically add food to a user's online shopping account when stocks are running low. It can even suggest recipes based on the ingredients you've got left and once a recipe is chosen, the fridge can switch on the oven to the correct temperature and set a timer via a wireless connection. The screen on the door then tells the cook exactly what to do.
Although a technologically advanced idea a critical article was written on this concept. See: http://www.guardian.co.uk/lifeandstyle/2012/jan/11/homes-fooddrinks
Currently this level of intelligence does not seem to be desired by consumers and in a project survey that we ran for an NDA project on kitchen’s, only 38% of a group of 40 older people wanted a fridge where the user could “Press button on fridge to read out food items present or run out of.”
However if the idea does take off it could have benefits for people with mobility disabilities in managing fridge contents and efficiency of shopping.
See: also demonstrating a smart fridge within internet access features :
and a smart washing machine with a touch screen and remote control:
Slide 11: Gesturing to products
Thanks to devices such as the Nintendo Wii and the Sony Playstation Move, people are now able to play games that involve gestures. These devices need the user to hold a type of wand which can track their movements in 3D space and translate into movements on screen in the game.
With the XBOX Kinect which operates with the Microsoft XBOX, the user does not need to hold anything and they can interact with a game by movement and gesture alone. A sensor records a person's full body movements in real time and interprets those movements in commands to the XBOX. One colleague was invited to a Kinect party when her Zumba session was not on over the holidays. Such a device can thus promote an active lifestyle for older people and raise awareness of the capabilities of the technology.
The KinMINDS project in Spain is using the Microsoft Kinect to open up new opportunities for people with mental disabilities. They use this technology to invite them to interact with and enjoy game play which can encourage them to be more out going and to interact with others. This may address some limitations and problems of conventional occupational therapy, such as patients' low motivation, lack of time to develop or apply adapted exercises for individuals with specials needs in heterogeneous groups and therapist skill.
Some TVs are now coming on to the market which have gesture recognition for changing channels – particularly useful if the remote gets lost down the sofa (and of particular benefit to users with mobility problems). Head gesture interfaces have been around for some time for wheelchair control by users with upper limb impairments to prevent use of joystick or armrest button controls.
Additional notes on using the Kinect: As far as non gaming actions go, the Kinect doesn't have wide spectrum of ways of interacting with the interface, however what Microsoft have done is impressive and shows the potential of gesture related software. When you switch on the XBOX you are presented with the home screen/dashboard, which has pages of tiles similar to those of the windows phones. Using the Kinect you can point you hand out in front of you and a cursor appears on screen which moves wherever you are pointing. You use this cursor to select different options in the interface. To move between pages you simply swipe your hand from one side to the other. Kinect also features voice controlled navigation, for example you can say 'Settings' and the screen will show the settings menu, or while watching a DVD, you can say pause and the film will pause. One student owner reports that it is very simple and intuitive. He has let people use his Kinect who have never used or been told how to use it, and they found themselves navigating the interface easily within a few seconds of exploring the controls. As for people with disabilities the possible applications for the technology are endless. An example he found in a literature review was a gesture based model that was designed to recognise 18 T’ai Chi gestures, and was used to provide a virtual environment for the relaxation of cancer patients.
Slide 12: Talking to products
Voice recognition by products and systems has advanced a long way and now quite reliable systems exist that allow people to issue commands over the phone to read out code numbers to check a credit card balance or to specify names to book a hotel.
Voice recognition in fact came about to help people with disabilities by providing a means to replace the keyboard.
Phones are also enhanced with voice recognition for speech dialling either handheld or hands-free in a car. This innovation is likely to be very beneficial for people with severe motor impairments allowing them to use a mobile phone more easily.
New TVs also have voice recognition for changing volume, channels etc. The design requires the TV to have a name as a prefix to commends so that it does not receive commands in error as a result of people talking.
Slide 13: Ambient interfaces – in the kitchen
The kitchen provides a platform for explaining and exploring the application of pervasive computing technology in a domestic setting. Newcastle University are using pervasive computing to support older people using appliances that are instrumented to allow wireless collection of activity data and the display of helpful information. The Ambient Kitchen project test bed set up Professor Patrick Oliver at Newcastle University is one example of a facility that is testing technological innovations in the kitchen particularly for those with cognitive impairment...
The recipe recommender reacts to the items the user places on the work surface, selecting recipes which most closely match the items chosen. The Ambient Kitchen can sense which ingredients the user has available, presents the nutritional value of the recipe they have chose, as well as step-by-step preparation instructions.
Medication reminder: People with dementia often struggle to remember to take the right medication at the right time. The medication reminder displays reminders when a medication due. The Ambient Kitchen tracks the medical container and resets the reminder only when the item has been removed from a cupboard and replaced again.
Situated media: Using RFID technology household items such as food products can trigger media. In the the Ambient Kitchen, media has been ‘attached’ to a number of objects, e.g. adverts for food items and trailers for DVDs.
Sensor display: Software tests the output of each of the devices embedded in the kitchen including views from each of the cameras, the RFID readers output, and the sensor network values. This offers people the opportunity to interact with the devices, and understand the complexity of the underlying system
For further information, see:
Slide 14: Ambient interfaces – near field communication
NFC allows a device, usually a mobile phone, to collect data from another device or NFC tag at close range. In many ways, it’s like a contactless payment card that is integrated into a phone. In other ways, it’s similar to Bluetooth, except that instead of programming two devices to work together, they can simply touch to establish a connection.
Potential benefits for people of NFC:
- Contactless Payment
- Public transport payment
- Health care smart card (providing and collecting information)
- Ease of use (touch device to connect)
- Smart objects providing information when passing e.g. at a tourist site at a bus stop.
For further information see:
Slide 15: Conclusions
The question is how these kinds of developments will fit into standards.
The kinds of issues that need to be considered are:
- Accuracy / reliability of the new technology
- Simplicity: keeping the number of gestures or voice commends limited
- Avoiding unwanted side effects such as preventing people from using the abilities that they do have.
Such new technologies should try to provide good person-system match allowing them not only to be assisted but also to continue doing what they can do and are good at.
Technology researchers and developers should consider some of these ergonomic requirements when reporting their work which can be helpful for ergonomics and design standards in the future.
See also “The future of user interfaces”: