Novartis and Microsoft team up to develop intelligent camera system for MS

Novartis' Vas Narasimhan

Novartis ($NVS) has been riding a med tech wave lately, launching new projects to create digital and patient monitoring devices. In its latest move, the company is working with tech giant Microsoft ($MSFT) to develop an intelligent camera system for multiple sclerosis (MS).

The system, dubbed Assess MS, uses Microsoft's Kinect motion camera and machine-learning software to track patients' movements and see whether the disease is getting worse. Microsoft, along with three of the top MS clinics in Europe, set up a system that could offer advantages over current screening methods for the condition.

"Novartis is leveraging digital technologies to transform patient care and drug development," Vas Narasimhan, Novartis' global head of development, said in a statement. "We are excited about our collaboration with Microsoft Research to develop Assess MS, a more consistent way to measure motor dysfunctions caused by multiple sclerosis, which could lead to the development of better therapies and care for patients."

MS is often difficult to monitor because some individuals' symptoms progress quickly, while others' symptoms appear and disappear over time. Doctors can track MS through tests that ask patients to carry out simple motions. Physicians watch the individual and use a rating scale to rank the severity of the symptoms. But the tests can be subjective, so there was a need for a tool that could monitor the condition more consistently.

That's where Assess MS comes in. Researchers used a Kinect depth-sensing camera to identify a patient's gestures and recognize a person's body parts as they carry out the tests. The scientists then created algorithms from the recordings, so a computer could pick up on the differences between a healthy individual and a patient with progressing MS.

"What you don't want to do with these systems is replace the expert," Abigail Sellen, a principal researcher in the Human Experience and Design group at Microsoft's Cambridge, U.K.-based lab, said in a statement. "You want to bolster the expert. What we're doing is giving (doctors) a set of data that they can then weave into their judgment."

Scientists also wanted to create a system that would stand up in real-world settings, not just the lab. The team designed a second screen that moves behind the camera that allows doctors to guide the patient without getting in the way. Researchers also created a simple animation that demonstrates movements for individuals, making it easier for the camera to pick up on motion.

Cecily Morrison, a researcher in Microsoft's Cambridge, U.K.-based lab--Courtesy of Microsoft

"It needs to be a collaboration between the patient, the clinician and the technology," Cecily Morrison, a researcher in Microsoft's Cambridge lab, said in a statement. "What we try to do is strike a balance."

Microsoft and Novartis still need to prove that its camera can pick up small motions like tremors in the human eye, Paul Matthews, head of the brain sciences division at London's Imperial College, told Bloomberg. And the companies need to collect more data to refine the system's algorithms.

But Microsoft and Novartis seem confident in their recent progress. The software has already analyzed 150 to 300 videos for each movement, Bloomberg points out, completing phase one of the project. Now, the companies are planning to expand to 5 new clinics and hospitals to get more patient videos to improve the algorithms. Eventually, the team could create a system that operates in at-home settings, making it easier for patients to track symptoms.

"We are approaching parity with humans on accuracy, but perhaps better than humans with consistency," Microsoft researcher Antonio Criminisi told Bloomberg. "But before feeling confident that this really works all the time, we need to be able to look at many more patients."

- get more from Microsoft
- here's the Bloomberg story