The open-source software Stephen Hawking says changed his life

The famous physicist talks about how he spent three years working with Intel to devise new software that makes it simpler for him to communicate with the world.  

Hawking and his carer Pete read the paper.

Image: A.Pattenden 

 

Professor Stephen Hawking suffers from a neurological disease that has left him almost entirely paralysed.
 
For the past 20 years the world-renowned physicist has relied on a computer to communicate with the world - controlling the machine using a muscle in his cheek. His limited movement meant everyday tasks were a painstaking process and even talking via his speech synthesiser took longer than one minute per word.

Worse, Hawking found he was becoming fatigued, that his cheek muscle could only control the computer for so long each day, limiting his ability to work and speak to his friends and family.
 
Hawking needed a more efficient way to control his computer and turned to Intel founder Gordon Moore for help. Moore set up a group within Intel Labs to help Hawking, and over the next three years that group worked with Hawking to develop a new user interface and predictive text system. After years of tweaking the system to suit Hawking's needs, the team devised software capable of doubling his rate of speech and that let Hawking control everyday software such as web browsers ten times faster than before.
 
"My old system is more than 20 years old and I was finding it very difficult to communicate effectively and do the things I love to do," he said at an event to unveil the Assistive Context Aware Toolkit (ACAT) system in London today.
 
"With the improvements made I can now write much faster and it means I can continue to give lecture, write papers and books, and speak with family and friends more easily."
 
Much of that software that drives Hawking's new system will be released to the world under an open source licence in January next year, allowing other disabled people to benefit from and build on top of the system.

Letting Professor Hawking do more

The finished system was the product of hours of observation of Professor Hawking's life at work and at home, and constant feedback from Hawking on what he did and didn't like.

Intel quickly realised that Hawking didn't want a shiny new interface - he wanted the system he had spent 20 years learning how to use, he just wanted it to be easier.
 
"We realised he wasn't looking for a new and radical interface. He wanted something that looked and felt like his current interface but much more effective and faster to operate," said Lama Nachman, principal engineer with Intel.
 
The key behind the improvements that ACAT makes possible is letting Hawking achieve more each day, said Pete Denman, user experience and interaction designer with Intel Labs. While they couldn't prevent Hawking's cheek muscle from getting fatigued, they could reduce how often he needed to use that muscle to control his computer.
 
"If we reduce the numbers of steps to get to that point that's more work he can do," he said.
 
The ACAT system can be split into three parts. The input from the infrared sensor that sits on Hawking's glasses and detects movement of his cheek muscles when Hawking wants to select an item on screen. The interface that selects letters to form words and lets Hawking speak, as well as interacting with web browsers and other software. The software that predicts what Hawking is typing as he selects the letters, similar to autocomplete on modern smartphones.
 

Being able to compose text is vital for Hawking, both to speak via his synthesiser and also to write his lectures and books. Hawking's system pieces together words one character at a time, selecting each letter of the alphabet in turn and waiting for Hawking to select the one he wants.
 
Hawking's priority was reducing the number of characters he had to select to form each word. To achieve this Intel worked with SwiftKey, the British company whose technology underpins predictive text keyboards on smartphones.

By analysing a corpus of Hawking's past work, and the text he inputs each day, SwiftKey devised a probabilistic model of how Hawking uses language. The model is able to more accurately predict which word he is typing so that he only has to type between 15 and 20 percent of the characters - roughly doubling Hawking's rate of speech.

Making it easier to use Windows

Intel also demonstrated how cumbersome it could be for Hawking to use a Windows PC. To click and open a file in the Windows operating system, the computer would step vertically down the screen line by line, until Hawking indicated it was vertically aligned with the position of the file in the window. The system would then repeat the process, stepping horizontally across the screen until the pointer was over the file, allowing Hawking to click and open it. The whole process - typically something that takes seconds - would usually take upwards of three minutes.

To simplify the process Intel came up with a file manager that could step through each file in a folder - allowing Hawking to access documents and other files far more rapidly. They also devised application-specific menus whose options change depending on the software Hawking is using, which made moving back and forward within web browsers and other common tasks far simpler.
 
One thing that will stay the same is Hawking's voice. The synthesiser he's been using for years may be more robotic than modern alternatives but Hawking says he will not swap it, insisting that it's his trademark.

Helping others to communicate

Hawking has a motor neuron disease (MND) related to amyotrophic lateral sclerosis (ALS), a condition that has progressed over the years.

He is not alone in the affliction, quadriplegia and MND affect more than three million people worldwide. MND impacts voluntary muscle activities, like speaking, walking, swallowing and general movement of the body. Progressive in nature, it causes increasing disability and eventually death.
 
By open sourcing ACAT, Intel hopes developers will build on the software to help other MND sufferers, as well as anyone who needs to control a computer using non-standard inputs such as blinks or eyebrow movements. The toolkit will be available under an open-source licence from January next year, with the SwiftKey prediction engine replaced by an open-source alternative.
 
Just as the ACAT has made Hawking's life easier, so he hopes it will allow other disabled people to achieve more each day.
 
"As technology has got smarter it has opened up possibilities I didn't even predict. The technology that is now being developed to support the disabled is leading the way in breaking down the communication barriers which once stood in the way," he said.
 
"This new system is life changing for me and I hope it will serve me well for the next 20 years."

~ Nick Heath

Related Posts