Ben Fletcher

I thought I’d have some fun tonight!

Inspired by Charlie Swinbourne I had a go at being an amatuer (not that Charlie is these days!) film maker. Here’s the 3 hours worth of effort, which is also available on YouTube

In the evening yesterday over at Portcullis House, Westminister, we were lobbying for better provision for deaf people to communicate with hearing people over the phone and Internet. The BBC picked this up here - incidentally today BBC also interviewed me with 13 questions here. It was great to see SignVideo and Teletec demonstrating high quality captioning and video interpreting technologies.  The services need urgent funding to become real and as widespread as in, amongst others, America and Sweden. I got to meet Ofcom people Peter Bourton, Commerical Policy, and Katie Hanson, Consumer Policy, whom I knew from Sense. We discussed the US’s model of varied tax schemes versus UK’s across the board. In the US,  federal and state governments add specific taxes to bills such as “tax for relay services” on phone bills, whereas the UK has just the VAT and smoking, petrol, etc., duties. We compared the situation in Sweden, with high taxation providing services, with the one which exists in the UK: low taxation with businesses expected to provide what is termed as reasonable - often inadequate or poor  - access. I met the lobby sponsor Malcolm Bruce MP who emphasised the need for more awareness of the urgency of the issues in political circles (”talk to your MP too,” he said).

Speakers all said that in general the 1 pound a minute it costs deaf people for communication over the phone and Internet compared to the 1 pence a minute it costs hearing people is a gap that needs narrowing. Tish Kerfoot who had the good fortune to live in America and now has been living in London for over a year explained, in BSL, how the video service really opened up the world for her - and me when I had the opportunity - where everyday conversations went from unclear and prolonged to very amicable and productive.

Meanwhile, Jeff and I wondered about deaf people who are also blind, people who weren’t mentioned in the speeches, naturally tactile gloves and augmented video streaming came up but today a colleague Twittered about something Jeff and I didn’t see coming:

Airbourne Ultrasound Tactile Display

Looking at the interview with BBC again, I was described as “hearing and visual impaired” which is absolutely fine but I think a more appropriate description for some deaf people in the UK, which has more urgency, would be “language impaired” - we need full language access!!

Looking back to the blog entry about train station as a partially blind citizen, what can augmented reality technology do?

Everything is of greyness.  To the untrained sight, it is not immediately very clear where the taxi rank may be.

Augmented Reality

The same except that the view is now augmented with useful assistance.  A pink path that suggests where the taxi rank may be.  A yellow circle that suggests where the first taxi’s passenger door may be.

The same thing, but from a different standpoint.

Visor

Photo Credits: Leonard Low

This visor is not just a visor, but augmentation is projected onto the back of the visor, providing an augmented view for the wearer. The augmentation of the pink and the yellow objects can possibly one day be projected onto the visor. The intriguing technical challenges may include determining the visor’s idea of location and orientation to accurately augment the computed objects to non-computed objects such as the taxi rank and the taxi passenger door.

This picture shows very clearly where to not walk to, to avoid falling off the platform! What could be more useful is that no longer does the citizen need to listen out for announcement or keep an eye on the public display, it’s all written out on the platform.

Thanks

Ian Hughes for this entry’s inspiration.

I imagine myself arriving at a train station somewhere random in Sydney.  I am a citizen who is:

• deaf without speech (hearing impaired); and
• partially blind (visually impaired).

What to do? Here is a demonstration of a small Proof of Concept.

The top two-thirds of the screen shows the map of the station. The bottom one-third contains the text saying welcome to the station.

Text to Speech Technology

The text is also spoken using the phone’s built in technology which translates text into audio for the blind citizen or James Bond.

Navigation

I press the big button, and a menu is brought up.

This list shows just the Go to Ticket Office option, but of course it would contain options for other places such as Toilet, Barrier Line, Taxi Rank, and so on.  I scroll to the Go to Ticket Office option and select it.

I can navigate to the ticket office using a choice of the map with the route, the text, or the voice guidance.

Location Aware Technology

I can tell the phone where I am going manually or automatically: manually by using the phone’s joystick as I walk along; automatically by using location aware technology.

Location Aware Messaging Technology

As soon as I arrive at the ticket office, the phone, knowing that I’m at ticket office, informs me that the office is closed and that the barrier line is open. This is because of the location aware messaging technology, which uses my location information to provide me with more relevant information.

The phone then takes me to the barrier line.  Before I catch the train, I would like to go to the toilets.

Profile

The phone knows I want to go to the gentlemen, rather than ladies, because my profile is provided with the phone.  This profile includes my gender; it can also contain my disabilities and alert the staff of my needs as I enter the station.

Let’s say that I am actually female and I ended up in the ladies.  I want to take a beeline to the next train for London.

The phone picks up my new location as being in the ladies, rather than the gentlemen, and takes me to Platform 3 which has the next train to London.  This is calculated through messaging technology which should be linked to the railway’s information backends.

Rendezvous

The phone provides me with up to date information regarding the train delay and how to make a connection with the conductor if I need his assistance. This eliminates the need for a good eyesight to read the public display and/or hearing to be alerted through the speaker.

Development

This is a standalone application which runs on the Nokia N95 using Python (for Series 60) and MQTT (a lightweight messaging protocol).  I hope to develop the same application that is web-based and therefore can be used by a compatible mobile phone with no need for an application installation.

Contacts

I was inspired to write this when I heard through Andy Stanford-Clark that Glenn Wightwick, another inspiring colleague from Australia, produced something similar, and with thanks to Bharat Bedi and Dave Conway-Jones for some more inspiration behind this work.  Lastly, but by no means least,  the LAMA team who worked on very similar scenarios.

See Also

Location Aware Messaging for Accessibility (LAMA) - an Extreme Blue project.

MQTT - the lightweight messaging protocol used in this PoC.

Really Small Messaging Broker (RSMB) - the lightweight messaging broker used in this PoC which can also be downloaded.