series:

The Rise of Synthetic Image Data for Deep Learning

The modern revolution in machine learning and robotics have been largely enabled by access to massive repositories of labeled image data. Yet obtaining real-world data can be expensive, time-consuming, and inconvenient. In response, simulation tools have grown in popularity, and have become fundamental in the development of algorithms, particularly in the fields of Robotics and Deep Reinforcement Learning. In this three-part series, I discuss the co-evolution of simulation tools, particularly those that generate photorealistic images, and the data-hungry machine learning algorithms they have fueled.

• In Part 1: The Importance of Simulation in the Age of Deep Learning, I discuss the historical significance of simulation tools in the field of robotics and the promise and limitations of photorealistic simulation tools.
• Part 2: Bridging the Realism Gap for Synthetic Image Data is a survey of different approaches to making simulated data more useful for real-world applications.
• Finally, in Part 3: Simulation Environments for Robotics and Deep Reinforcement Learning, I list a number of simulation environments for different applications and some trends I think reveal where the community is headed.