In-cabin driver monitoring is increasingly being included within new car models, increasing safety of the occupants and alerting drivers to dangerous drowsiness levels which may warrant them unsafe to continue driving.
Driver monitoring company Cipia has recently announced that Chinese automaker Chery will be using its AI-powered Driver Monitoring System in six car models that will be shipped globally.
The system can detect distracted driving, drowsiness, seatbelt use, smoking, phone use and much more to alert drivers of unsafe driving and prevent accidents.
To learn more about this technology and the future development we spoke to Tal Krzypow, vice president of product.
Frankie Youd (FY): Could you tell me a little bit of company history and what your role involves?
Tal Krzypow (TK): We are a computer vision company. Following the Euro NCAP roadmap that was released in 2017 (it called for driver monitoring in vehicles for maximum safety) we realised the potential in the market and we pivoted the company to focus on automotive. Today we are 100% focused on in cabin sensing and specifically driver monitoring systems.
Since then, we’re very proud of market uptake and today we work with five OEMs [vehicle manufacturers] on 28 different vehicle models, both in the US and in China. We’re already in production with the three of the OEMs.
I’ve been with the company for nine years, I manage different aspects in my current role I manage the product roadmap, engagements with key customers, customer support – lots of areas.
Could you explain how the driver monitoring system works?
We developed advanced computer vision algorithms that belong to the role of the AI. What these algorithms do is they analyse video. We work with infrared video, because we want to be able to deal with any kind of lighting and environmental conditions. Whether it’s broad daylight, or sideways sun coming in from the sunset, whether it’s pitch black because it’s in the middle of the night or in a parking garage where there is no external light source – we’re working with infrared cameras which have infrared emitters, and they provide the lighting for the image.
The nice thing about infrared is that we human beings can’t see it, so there’s nothing to interfere in any way with our operation of the vehicle.
Expanding on this, with TV infrared remote controllers when you press them you can see light coming off the end of the remote; that’s because there is no proper filtration and it’s in the 850-nanometre bandwidth. We are operating about that in the 940 so it’s not visible and it’s fully safe.
The computer vision algorithms analyse the facial features of the driver, they detected the eyelids, the pupils, the iris, the position of the pupil versus the eye, the gaze direction, the head pose – it is all analysed. Everything is extracted with a very high level of accuracy; the specific angles of the head pose or the openness of each eye over time will be fully analysed.
Once we establish all this information, we employ a second algorithm that belongs to the realm of human factors. These algorithms use logic to translate the visual cues into the physiological state of the driver. This analyses how a series of blinks for example, is translated to drowsiness, or a series of gaze patterns is translated to distraction.
We can detect if the driver is wearing a seatbelt or not – when they aren’t a warning comes up. We even detect if the driver is wearing glasses or not. We can detect phone use, cigarette smoking and roaming eyes promote the ‘distraction’ alert.
Beyond that, we also focus on different areas of interest in the vehicle. If you’re looking at the sideview mirror or the infotainment console, we know exactly whether the driver is engaged in the driving task, or if they are distracted.
Our algorithms are also smart enough to factor in the different areas of interest. For example, the rear-view mirror is something where you can see in your peripheral vision the road in front of you pretty well even when looking in it, so we can be more lenient. However, if I’m looking downward or at a phone for example, then it merits an alert or even a rapid intervention.
I am assuming that the facial data isn’t stored by the company?
It is important to note that when we are integrated in a vehicle’s production as part of the safety systems. We’re just another sensor in the vehicle. No video ever goes out from the car for analysis. From our perspective, we do everything locally in real time.
We translate the image to match the data and the image can then be discarded. There is no need to keep the video even locally. Even when we run facial recognition for a convenience purposes to know who the driver among the household members is – to adjust the convenience functions like the seat position, temperature level, volume level, things like that – we take the face, and we convert the face into a string of numbers.
The string of numbers is what we store and when we see a new face, we convert it to a stream of numbers and we compare numbers to numbers. We don’t compare images. We can’t go back from the numbers to an image, so the privacy is fully maintained. It’s fully secure, it’s a sensor and nothing more.
How did the partnership with Chery come about and what does this involve?
We recognise the importance of the Chinese automotive market. The Chinese market has already crossed the point where it manufactures more vehicles than Europe and the US. What we love and appreciate about the Chinese market is that there is willingness to move very quickly. Whereas typically in production programmes in Europe and US we see that it takes two to three years – things move very quickly in China.
Within a year or a year and a half we are able to have the vehicle on the road; there is a lot of investment in technology, infrastructure, and the ability to push a product into the market.
We were already well established with Chery, when we started the engagement; the first one was in the beginning of the year and several months later we had the second, and now we have announced the third. They have gained experience with us; it builds their confidence in the partnership.
The recent announcement is for three additional models, which doubles the number of models we have right now. What we’re especially pleased and happy about is that several of the models are going to Europe and to other regions; especially Europe because of the regulatory requirements and the safety standards.
The technology is also offered to fleets, what are the benefits?
For a fleet operator this technology brings tangible benefits in avoiding unforeseen costs. A vehicle involved in accident and repair is an asset that now has downtime. The entire fleet schedule is a mess, the service level agreement that the company has with a customer may be hurt with a need to pay compensation. It may be that cargo was hit, and there are insurance issues there, also.
The second aspect is that fleets spend a lot more time on the road. Vehicles are generally used more intensively, on the road for more time and are also in conditions that are often more challenging. If I’m a driver and I need to get from one city to another I need to drive now for five, six hours, it’s a lot more tiring than a local shopping run. I’m therefore more susceptible to drowsiness and I’m more likely to be involved in accidents.
Assisting fleets with this technology promotes the safety on the roads as a whole in general.
What do you hope the future holds for this technology?
I hope that in a few years’ time we will have additional, advanced capabilities to detect dangerous situations such as driving under the influence of alcohol – that is still a major problem. It’s something that we lack a solution for today in private vehicles; we believe that this is a great field to push the technology into and see what kind of benefits we can bring there.
Beyond that, I feel that OEMs today are still very much constrained or driven by the safety regulations and standards. They are not leveraging sufficiently the technology or the comfort and improved experience features that they could have. There’s so much more that you can do when you understand the gaze of the driver or a passenger and where they’re looking because it’s the most powerful indicator for human intent.
When we look at something we want to interact with it, whether to consume information or to cause an action. With safety, there is still much to do; better integration with the systems for example, and the intervention of the vehicle at the moment the driver is impaired. I hope that requirements for intervention will help push the market in this direction.
There are a lot of interactions that can happen with the smart assistant, the personal assistant in the vehicle that goes beyond voice and combines both modalities, the voice and the vision to understand I am looking at it, or looking elsewhere in the vehicle. I certainly hope that that we’ll be able to discuss these kinds of applications.
