A smarter car: How your vehicle might soon be able to tell if you're in a bad mood

Cars that recognise when you are becoming prone to road rage could potentially quell annoying bells and chimes and play some mellow jazz instead

By the end of the year, Ford plans to add Amazon’s Alexa personal assistant to some of its cars (What Car?)

When you shop for cat food online, Amazon knows who you are. When you look for the best airfares for summer vacation, Google knows who you are. Soon, when you get behind the wheel, your car may recognise you, too.

Using cameras with facial recognition software and other biometric indicators, automakers are looking to personalise the driving experience with cars that stare back at you, quietly adjusting seats and driving modes.

They may even anticipate your wants and desires by playing your favourite music based on your mood. And it’s not only about convenience, but also about the possibility of improving safety and security.

“It’s not just about personalisation,” said Zachary Bolton, a systems and technology engineer at Continental Automotive Group, explaining the sophistication of such systems.

“We can use the gleam, the twinkle in your eye to determine precisely where you’re looking.” Engineers can then dynamically adjust the human machine interface, putting critical information, say, about a stalled car up ahead, or the fact that you are about to exceed the speed limit, directly in a driver’s line of sight on the dashboard or in a display on the windshield. Conversely, by tracking downward eye movements, the car could “see” that a driver was distracted and sound a warning.

Continental has already demonstrated in-car systems that allow drivers to register their faces using something as simple as a driver’s licence picture.

An interior infrared camera is used to overcome potential obstacles like sunglasses, which would stymie a conventional video camera. Putting the camera in the centre instrument cluster also helps to pinpoint the driver’s eyes, even if she’s wearing a hat. The biggest technical challenge – glare caused by sunlight – can be filtered out using machine learning, Mr Bolton said.

Once the car knows who you are, systems in vehicles like the Chrysler Portal concept car would automatically adjust the seat for maximum comfort, select a driving mode – for example, one driver likes to let the car do most of the work; another likes taking control in sport mode – and suggest a destination based on the owner’s past behaviour.

Watching a driver’s face can also give a car important clues about that person’s state of mind. For several years, carmakers like Ford and tech companies like Intel have been interested in determining whether a driver is happy or sad. Depending on your mood, a car could change its tune, playing a bouncy Beach Boys song and changing the interior lighting to improve your attitude.

Honda’s NeuV concept car, for example, has a large customisable LCD dashboard and a cloud-connected, onboard computer that uses artificial intelligence to interact with drivers.

Honda’s NeuV uses artificial intelligence to interact with drivers (Honda)

NeuV employs what the company coyly refers to as an “emotion engine” to grease the wheels of the conversation, and its automated personal assistant can read “facial skin vibrations” to help it isolate the driver’s voice and better understand spoken commands.

There are practical reasons as well, designers say, for detecting a driver’s emotional state: a calm driver is a safer driver. So cars that recognise when you are becoming angry and thus prone to road rage could potentially quell annoying bells and chimes in the car and play some mellow jazz to soothe you.

By replacing keys and remote control fobs, biometrics like facial recognition could also make cars more difficult to steal. In its prototype FF 91 sport utility vehicle, the electric car startup Faraday Future uses an external camera mounted in the door frame to detect the car’s owner and automatically unlock the vehicle. Such techniques can create new security challenges, however.

In this digital age, our faces are everywhere: in online company profiles, on Twitter accounts, even tagged in friends’ Facebook accounts. Finding an image to print out and foil a car’s facial recognition system would not be very difficult.

Fortunately, engineers have devised high-tech countermeasures.

Stereoscopic video cameras, for example, can tell the difference between a flat image and a three-dimensional object. Continental’s cameras measure the distance of reflected light off various parts of a person’s face, ensuring that it is a real object rather than a high-resolution shot of the owner’s visage.

“Some systems have added blink detection and aliveness detection,” said Steve Grobman, chief technology officer for security at Intel.

But he acknowledged that it was still possible to thwart such technology. “We had 3-D masks printed that we ordered and were able to trick some systems,” he said. But he said most thieves would be unlikely to go to such extreme lengths to steal a car.

“It all depends on the level of accuracy you need,” said Yoni Heilbronn, chief marketing officer of Argus Cyber Security, which works with automakers to short-circuit hacking threats. “Retina scans are even better than facial recognition” as a potential solution, he said, “but by adding another level of authentication you lose some of the convenience”.

On the other hand, high-tech personalisation could be used not only to create amenities for single owners, but also to instantly adapt a vehicle to suit a variety of drivers.

Valets, for example, could be automatically prevented from accessing personal information in a navigation system or driving faster than, say, 30 mph. In a ride-sharing situation, such systems could also be used to quickly tailor a car’s interior to the physical characteristics of different drivers and passengers.

Rental cars would be easier to operate and safer, Mr Bolton said. “If I know where your head is and where your eyes are,” he said, “I can adjust the position of the steering wheel and the mirrors so you don’t have to fumble looking for the right buttons.”

The FF 91 uses a camera mounted in the door frame to detect the car’s owner and automatically unlock the vehicle (Getty) (Getty Images)

Even air bags could be fine-tuned, reducing the intensity of their deployment depending on the size and position of a driver or young passenger.

Some elements of the personalised driving experience are already coming to cars. By the end of the year, Ford plans to add Amazon’s Alexa personal assistant to some of its cars, said Dave Hatton, manager of Ford’s mobile applications for connected vehicles. It will not only allow personalised music stations to play with a voice command, but also enable drivers to juggle chores like adding items to an existing grocery list with just a few words.

Such convenience may come with some trade-offs on privacy.

“It’s a huge concern,” said John Simpson, privacy project director at Consumer Watchdog, a nonprofit advocacy group.

“All that data is in some database without your consent or knowledge about how it’s going to be used,” he said, adding that there was little if any current government regulation to curtail such use.

Using traffic-tracking programmes like Waze, some consumers have already signalled their preference for convenience in return for giving up some information like their location. Programmers also point out that such services are optional: you don’t have to let the company track you – but then you may get stuck in traffic for 45 minutes.

Today, basic biometric technologies like facial recognition software are used for everything from signing into Windows laptops to thwarting toilet paper thieves in Beijing. Fingerprint readers are commonly used to unlock smartphones. As consumers become more accustomed to such systems, the introduction of the technology in vehicles may seem like a natural evolution rather than a creepy intrusion.

And it could be fun. Consider the entertainment and social media repercussions in the vein of James Corden’s “Carpool Karaoke” segments.

A built-in camera could record and broadcast your sing-a-long on Facebook or Twitter – assuming the car was in autonomous driving mode. Of course, drivers could grab quick selfies on the road, too.

“It’s a novel ideal,” Mr Bolton said, “but remember that an infrared camera makes your face look a little like a skeleton, so it’s not that flattering.”

© The New York Times