Car Dataset Generation

Business Challenge

For a recent mobile app project, Laan Labs needed to create an augmented reality experience around the exterior of an automobile. In order to overlay realistic graphics onto a car in AR, one must accurately estimate the position & orientation of the vehicle in real-time.

Image training data can be very costly to acquire and may require labor-intensive, error-prone manual annotation. Cars, in particular, are large and shiny- and can look very different depending on weather conditions and surrounding scenery due to their high reflectivity.

Laan Labs needed to gather 10,000+ images that are representative of the real-world conditions a user might encounter when photographing a car with a smartphone.

Synthetic data provides unique photorealistic images with precise ground truth labels

Approach

Laan Labs' approach was to leverage Synthetic Data, by developing software that generates unique photorealistic images with precise ground truth labels. A selection of artist-created 3D car models, background scenes, and lighting models were chosen as the seed of a 3D environment. Image annotations were computed directly from the 3D scene. Various aspects of the scene were parameterized, such as: camera angle, lighting, and background. Vehicle-specific attributes, such as: vehicle pose, paint color & matting, window tinting, wheel selection, were customizable. With the parameterized 3D scene, countless variations of annotated car images can be procedurally rendered using cloud computing.

Business Value

By generating synthetic data, Laan Labs produced the 10,000+ labeled images the project needed without photographing and hand-annotating thousands of real cars. Labels were computed directly from the 3D scene, which removed manual annotation and the errors that come with it, and new variations in lighting, paint and background could be added by changing parameters rather than organizing another shoot. The dataset was used to train the car pose estimation model behind the app's augmented reality experience.

Technologies Utilized

Computer Vision, Neural Networks, Augmented Reality, Python, iOS, TensorFlow, CoreML