Foot Dataset Generation

Business Challenge

For a recent augmented reality application, Laan Labs needed to estimate the position & orientation of human feet in order to display realistic footwear on a user. To solve this computer vision problem of foot pose estimation in real-time, Laan Labs used state-of-the-art neural networks- versatile machine learning models that have the unfortunate characteristic of requiring a lot of training examples.

Image data can be very costly to produce- requiring elaborate photography sessions, hiring actors, and renting obscure props. Image data annotation is labor-intensive and can be error-prone.

Laan Labs needed to gather 10,000+ annotated images that are representative of the real-world conditions a smartphone user might encounter: various camera angles, lighting conditions, footwear, and background scenery.

“with [synthetic data] ... countless variations of images and annotations can be procedurally rendered”

Approach

Laan Labs’ approach was to leverage Synthetic Data, by developing software that generates unique photorealistic images with precise ground truth labels.

A combination of artist-created and reconstructed 3D foot models were used as the seed of a 3D environment. Image annotations were computed directly from the 3D scene. Various aspects of the scene were parameterized, such as: camera angle, lighting, background imagery, post-processing effects, foot pose, and footwear. With the parameterized 3D scene, countless variations of images and annotations can be procedurally rendered using cloud computing.