DETECT
FACIAL ACTION UNITS
NVISO’s unique holistic platform to recognize facial action units enables many applications such as emotion analysis, facial behavior analysis, and affective computing. Industry use cases included digital avatars, pain detection, and driver monitoring for drowiness and fatigue.
DETECT
FACIAL ACTION UNITS
NVISO’s unique holistic platform to recognize facial action units enables many applications such as emotion analysis, facial behavior analysis, and affective computing. Industry use cases included digital avatars, pain detection, and driver monitoring for drowiness and fatigue.
FACIAL ACTION UNIT
RECOGNITION SOFTWARE
The NVISO Neuro SDK provides facial action unit recognition capabilities for researchers, developers, and manufacturers of human machine interfaces (HMI). The SDK can automatically analyze facial movements by coding them into the Facial Action Units (FACS) taxonomy as a part of a real-time system. The SDK uses selected components of computer vision, machine learning and state-of-the-art graph based algorithms that work on an embedded system or can run on a standard PC.
Facial action units (FAUs) are the basic building blocks of facial expressions. FAUs include eyebrow raiser, eyebrow lowerer, eye opener, lip pressor, lip corner depressor etc. Each FAU is associated with a specific muscle in the face. FAUs are essential for our ability to communicate non-verbally. For example, if you want to ask someone what they think about something you can raise your eyebrows and lower your eyelids and that tells them exactly what you mean without having to say anything at all.
Facial action units are key indicators of emotions that play an important role in nonverbal communication. They can detect drowsiness and fatigue in drivers (and other people), which is critical for safety applications such as automotive, healthcare and security industries. Another use for this technology is for digital avatars in video games and virtual reality applications (VR). These characters need to look as realistic as possible so that players feel like they’re interacting with another human.
STATE-OF-THE-ART
FACIAL ACTION UNIT DETECTION TECHNOLOGY
SCIENCE BASED - FACIAL ACTION CODING SYSTEM (FACS)
Facial Action Coding System (FACS) is a detailed way of describing the facial expressions. It was developed by Paul Ekman and Wallace Friesen in 1978. FACS is used to describe the facial expressions, which are very important in emotion recognition. Facial action units (FAU) are the basic building blocks of facial expressions. They are readily identifiable movements of the muscles in the face and they can be used to express a range of emotions. For example, if you smile or frown, you are using the muscles around your mouth to achieve that effect. FACS provides a list of 68 different FAUs, which NVISO groups these into three categories: 1) Upper face movements : Eye and brow 2) Lower face movements : Lip, mouth, nose, chin, and jaw 3) Asymmetrical movements : Left and right
IMPROVED ACCURACY - GRAPH CONVOLUTIONAL NEURAL NETWORKS
Graph convolutional neural networks (GCNs), a type of neural network that uses graphs as its representation rather than the standard feedforward networks used in most previous work. While it was previously thought that these models were unsuitable for the task of facial action unit detection (for example, due to their large number of parameters), work by researchers at Google Brain has shown that GCNs can outperform traditional neural networks for this task. NVISO's GCN based approach is now considered the state-of-the-art as it can be used to model the relationships between AUs by learning the shared spatial structure. This approach is able to learn a low-dimensional embedding for each AU that captures the spatial relationship between them. This greatly outperforms the previous approach of using only convolutional neural networks (CNNs) to classify an image only which suffered from high false negative rates due to the complexity of AU recognition.
REDUCED BIAS - LARGE SCALE SYNTHETIC FACE DATABASES
In recent years, there has been a surge of interest in the use of deep learning methods to classify heads. While these techniques have shown great promise, they are often limited by their inability to adapt to new subjects and contexts. This problem is particularly pronounced when the classifier is applied to new age groups or ethnicities. The problem is further complicated by the fact that head appearance can be significantly altered by specific medical conditions causing asymmetrical behavior of the face. NVISO overcomes this these problems, by using very large scale databases of head images that have been annotated with a large number of key points, head angles, and segmentaton. These databases allow us to train models that are robust across different ages, genders, and ethnicities, while still maintaining high accuracy on a single individual.
USE CASES
FACIAL ACTION UNIT RECOGNITION APPLICATIONS
DIGITAL AVATARS
Digital avatars are computer-generated images or animations that represent a real person, such as an actor or celebrity. Eye tracking can be used in digital avatars to make them more human by mimicking the way people interact with each other and the environment. When an avatar looks at something, we expect it to focus on that object and not wander off too quickly. When their eyes roam around the room, they should track back and forth, not stop abruptly. Eye tracking is important for creating realistic digital avatars because eye tracking methods can be used to interpret the user’s intention and emotional state: gazing, blinking and eye openness. NVISO eye tracking on software can be used to precisely detect the eye movements of a user and then animate their face accordingly using artificial intelligence algorithms. This allows digital avatars to be more lifelike than ever before!
PAIN ASSESSMENT
Facial expressions can also be used to detect pain in patients who cannot communicate verbally due to illness or injury. This type of technology is especially useful when it comes to elderly patients who may have cognitive problems preventing them from telling doctors how they feel physically or emotionally.
Pain is one of the most common symptoms experienced by elderly people, and it can be difficult for doctors and nurses to accurately assess this type of pain. This is because they may not be able to get close enough to their patients, or they may not have adequate training in how to interpret facial expressions. Facial action unit detection software has been found to be able to make this process easier, by identifying specific facial movements associated with pain and providing an output which shows what kind of pain is being experienced.
DROWSINESS AND FATIGUE
The need for automated eye tracking software is growing because human drivers are getting more and more drowsy and fatigued. According to the U.S. Department of Transportation, there were 5,987 fatal accidents in 2018 that involved drowsiness or fatigue as a factor in the accident. The practical implementation of a Driver Monitoring system cannot make use of only eye tracking. 3D headpose is a back-up signal acting as a critical back-up to gaze in various situation (as for example when wearing sunglasses or extending the range of attention measure at large head orientation angles). Face, head, and eye landmarks can also be used to track the blink rate, blink duration, and perclos used in deriving drowsiness and fatigue indicators.
NVISO NEURO MODELS™
ULTRA-EFFICIENT DEEP LEARNING AT THE EDGE
NVISO Neuro Models™ are purpose built for a new class of ultra-efficient machine learning processors designed for ultra-low power edge devices. Supporting a wide range of heterogenous computing platforms ranging from CPU, GPU, DSP, NPU, and neuromorphic computing they reduce the high barriers-to-entry into the AI space through cost-effective standardized AI Apps which work optimally at the extreme edge (low power, on-device, without requiring an internet connection). NVISO uses low and mixed precision activations and weights data types (1 to 8-bit) combined with state-of-the-art unstructured sparsity to reduce memory bandwidth and power consumption. Proprietary compact network architectures can be fully sequential suitable for ultra-low power mixed signal inference engines and fully interoperable with both GPUs and neuromorphic processors
PROPRIETARY DATA
NVISO Neuro Models™ use proprietary datasets and modern machine learning to learn from billions of examples resulting in an extraordinary capacity to learn highly complex behaviors and thousands of categories. Thanks to high quality datasets and low-cost access to powerful computing resources, NVISO can train powerful and highly accurate deep learning models.
RUN FASTER
NVISO Neuro Models™ store their knowledge in a single network, making them easy to deploy in any environment and can adapt to the available hardware resources. There is no need to store any additional data when new data is analysed. This means that NVISO Human Behaviour AI can run on inexpensive devices with no internet connectivity providing response times in milliseconds not seconds.
RUN ANYWHERE
NVISO Neuro Models™ are scalable across heterogeneous AI hardware processors being interoperable and optimised for CPUs, GPUs, DSPs, NPUs, and the latest neuromorphic processors using in-memory computing, analog processing, and spiking neural networks. NVISO Neuro Models™ maximise hardware performance while providing seamless cross-platform support on any device.
HUMAN CENTRIC
AI SOLUTIONS
CONSUMER ROBOTS
Human–robot interaction plays a crucial role in the burgeoning market for intelligent personal-service and entertainment robots.
AUTOMOTIVE INTERIOR SENSING
Next generation mobility requires AI, from self-driving cars to new ways to engage customers. Build and deploy robust AI-powered interior monitoring systems.
GAMING AND AVATARS
The gaming industry (computer, console or mobile) is about to make extensive use of the camera input to deliver entertainment value.