Overview

Our collaboration with a global leader in human behaviour analysis using face and voice sensing artificial intelligence. Their goal is to assist clinicians, patients, and their friends and families in assessing, treating, and monitoring health, mood, and mental state. In this case study, we will explore their facial video recording project, the challenges faced and the solution implemented.

‍

Problem Statement

Our client aimed to develop a video analysis system capable of understanding behavioral patterns based on face and voice technology. They needed to collect a dataset of facial recordings from individuals with various facial shapes and sizes, skin tones, and ages. The project required participants to record 2-3 minute videos using the front-facing camera of their Android 6 or higher or iOS 14 or newer mobile devices. English-speaking individuals were preferred, and there were no initial geographic restrictions.

‍

Solution

To address the problem statement, we devised a comprehensive solution with the following components:

‍

Dataset Size and Video Length:

Twine AI aimed to collect a dataset comprising 1,000+ participants. Each participant was requested to record a 2-3 minute video, with a maximum limit of 5 minutes. The videos were captured using the front-facing camera on their smartphones.

‍

Participant Variety:

To ensure diversity in the dataset, we targeted individuals with a variety of whole facial shapes and sizes, facial feature shapes and sizes (such as large eyes, small eyes, large lips, small lips, etc.), and different skin tones. The age range of the participants was not limited, as long as they were 18 years or older. Gender balance was also considered during participant selection.

‍

Video Quality and Platform:

The videos were recorded using the front-facing cameras of participants' smartphones, ensuring compatibility with Android 6 or higher and iOS 14 or newer. Blueskeye emphasised smartphone video quality captures accurate behavioral information effectively.

‍

Implementation:

We adopted a phased approach to data collection, which involved sourcing 100 participants per month on a rolling monthly basis. The data collection process began in November 2022 and continued until the dataset reached 1,000+ participants.

‍

Conclusion

Our client successfully developed a facial video recording project for behavioral understanding using face and voice technology. By utilising machine learning algorithms, they objectively analyzed face and voice data, providing valuable insights to clinicians, patients, and their friends and families. The project faced challenges in collecting a diverse dataset of facial recordings, but Twine AI implemented an effective solution by engaging participants through their smartphones. The client expressed satisfaction with the results achieved and showed a keen interest in continuing the project with additional time for improvements.

Partner with Twine for your AI requirements, and allow us to assist you in maximizing your algorithms' capabilities with tailor-made datasets crafted with precision. Get in touch with us now to explore how we can cater to your project requirements.