AdHawk Microsystems launches cameraless eye-tracking sensors for AR/VR

0
171

Attend Transform 2021 to learn the key topics in Enterprise AI & Data. Learn more.

Eye tracking has proven to be a critical part of augmented reality and virtual reality systems, but is expensive with cameras. Therefore, AdHawk Microsystems is introducing eye tracking, which uses sensors that do not use cameras.

AdHawk’s MindLink technology uses microelectromechanical systems (MEMS), tiny chips that perform mechanical functions. AdHawks MEMS will lead to easily wearable eye-tracking technology that can be integrated into AR glasses or VR headsets, CEO Neil Sarkar said in an exclusive interview with GamesBeat. The company built the sensors into its own prototype glasses that researchers will soon be able to use.

The prototype glasses connected to Android or PC are now available for pre-sale. The AdHawk system comes in the form of traditional glasses that can be worn all day. It is faster, lighter and more energy efficient than competing products and delivers high-precision data in medical quality. The idea is to open the connection between the eyes, the brain and the world around us.

This is important in that the highest quality data enables researchers to study conditions such as anxiety, cognitive stress, reading problems, ADHD, concussions, epilepsy, as well as early indications and progression of Alzheimer’s and Parkinson’s diseases.

Origins

Above: Prototype of the AdHawk AR glasses.

Image credit: Adhawk

Sarkar launched AdHawk Microsystems in 2017 in Kitchener, Ontario, Canada. He had a background as a chip engineer and studied MEMS in graduate school. The sensors he studied have actuators that can operate with an accuracy of less than a nanometer, and they scan a beam of light over your eye. He spun the company out of the University of Waterloo with funds from Intel Capital.

“We had this solution for eye tracking that didn’t require cameras or high-performance processing,” said Sarkar. “You can actually work at very high speeds with low latencies and very little computational effort. So [it] doesn’t need a lot of electricity. That made it very suitable for AR and VR products. And that was really the bread and butter for the company. “

The company raised additional funds in 2019 from Samsung Venture Investment Corp., HP, Essilor International, Sony Innovation Fund, Intel Capital, Brightspark Ventures, and Silicon Valley Bank.

AdHawk enables immersive experiences in AR and VR by enabling faster gaming, more effective training and effortless user interfaces. The company is now expanding its eye tracking technology to clinical research, where traditional practices rely on qualitative measurements of eye movement as indicators of neurological health.

“Today we really see ourselves as a full-stack eye-tracking solution provider, with around half of our business having software that covers everything from the embedded system to applications that use eye-tracking,” said Sarkar .

The company has 35 employees with a fabless semiconductor model, which means it designs its components and has contractors manufacture them.

“We have integrated our eye-tracking solution into products such as VR and AR headsets,” said Sarkar. “But due to the schedules, we don’t want to be at the mercy of a for some of the products that ship [manufacturer] so that our technology actually sees the light of day. That’s why we decided to only bring our own product to market. This is exactly what the MindLink is – lightweight glasses with eye tracking. “

How it works

Above: AdHawk Microsystems uses MEMS to measure your eye movements.

Photo credit: Adhawk Microsystems

AdHawk is replacing cameras with ultra-compact MEMS that eliminate power-hungry image processing and thus improve speed and energy efficiency by more than double that of its closest competitor, the company said. As soon as the patient wears the AdHawk MindLink glasses, he starts the app and taps on “Start tracking”. The system is so fast that it can predict exactly where a user will look next – up to 20 milliseconds before their eyes fixate – and the gaze is captured 500 times per second with an accuracy of more than one degree, according to AdHawk .

“Every time the beam hits a part of the eye that reflects it back to a photodetector, we get a pulse,” said Sarkar. “Sometimes we get impulses from corneal glimmers that are like reflections from the surface of your cornea. But sometimes we get pulses when the ray crosses the threshold between your iris and your pupil. We can measure both your cornea and your pupil thousands of times per second with the sensor. “

MindLink not only offers greater speed and precision, it also offers flexibility as it is not restricted by bulky equipment that can only be used in a laboratory setting. This means doctors and researchers can now perform eye-tracking exams in more settings than ever before, including a doctor’s office, on-site, or even on a patient’s home.

As you can see in the video, a micromirror scans a beam of light in two axes under a microscope. “That’s exactly what we did to replace cameras,” said Sarkar. “And so we don’t have to take 100 pictures every second.”

Applications

Eye tracking is important for games as it shows what the player is seeing. There is no need to render images that the user cannot see, saving processing power. And rivals like Tobii have done a decent business showing where professional gamers look on a screen so amateur gamers can learn how to play action games better, quick.

“You could also start using applications where you aim with your eyes instead of a mouse or controller,” Sarkar said.

In terms of applications, Sarkar believes the technology will enable new research linking eye movements to neurological and eye health. For this reason, the company’s kits are aimed at researchers in healthcare and academia. AdHawk supports researchers at the University of Waterloo and the University Health Network.

“There are many correlations between the signals we measure and what’s going on in your head,” said Sarkar. “That’s probably the most exciting thing.”

AdHawk’s eye tracking glasses can be comfortably worn to accurately measure eye movement, pupil size and head movement using an integrated IMU (Inertial Measurement Unit) sensor. As a result, researchers can collect data that will help them study anxiety, ADHD, concussions, and epilepsy while improving their assessments of cognitive load and reading problems. The data will also be useful in researching the progression of neurodegenerative diseases such as Alzheimer’s and Parkinson’s.

The device could also be used to keep track of whether a driver is paying attention in traffic.

“The sampling frequency of an eye tracking system relates to how often per second the position of the eyes is registered by the eye tracker. A higher sampling frequency improves a researcher’s ability to estimate the eye’s true path as it moves, “Elizabeth Irving, professor in the University of Waterloo’s School of Optometry and Vision Science, said in a statement. “In the past, a higher sampling frequency was associated with higher costs because more expensive cameras and more power were required. This is what makes the AdHawk system so special – they found a way to offer mobility, quality, and speed – all without sacrificing affordability. “

The kits are expected to be shipped within six to eight weeks of the pre-sale closing date. The device is not approved by the Food and Drug Administration.

GamesBeat

GamesBeat’s credo for the gaming industry is “where passion meets business”. What does that mean? We would like to tell you how important the news is to you – not only as a decision maker in a game studio, but also as a game fan. Whether you’re reading our articles, listening to our podcasts, or watching our videos, GamesBeat is here to help you learn, and enjoy, the industry.

How are you going to do that Membership includes access to:

  • Newsletters like DeanBeat
  • The wonderful, instructive and funny speakers at our events
  • Networking opportunities
  • Special member-only interviews, chats, and “Open Office” events with GamesBeat staff
  • Chat with community members, GamesBeat staff and other guests on our Discord
  • And maybe even a fun prize or two
  • Introductions to like-minded people

become a member