World Labs' AI can create interactive 3D scenes from a single photograph.

World Labs, the startup co-founded by renowned artificial intelligence pioneer Fei-Fei Li, has unveiled its first project: an AI system capable of generating 3D scenes, similar to those in video games, from a single image. While various artificial intelligence technologies can transform photos into 3D models and environments, the scenes created by World Labs stand out for being interactive and modifiable. According to the company, its technology allows users to "enter any image and explore it in 3D," with everything not present in the input image being generative.

The scenes generated by the AI, which are available for users to explore in a demonstration on the World Labs website, feature an impressive, albeit somewhat cartoonish, appearance. They are rendered in real-time in the browser and include a controllable camera and adjustable simulated depth of field. As the depth of field effect is increased, background objects become blurrier.

World Labs' system falls into a new category of AI known as "world models." Many of these models attempt to simulate games and 3D environments but often exhibit issues with artifacts and consistency. For instance, the world model from the startup Decart, called Oasis, has low resolution and quickly forgets level layouts. In contrast, World Labs' approach ensures that the scenes remain consistent after being generated and comply with basic laws of physics, providing a sense of solidity and depth.

Additionally, its system can apply interactive effects and animations to the scenes, such as changing object colors and dynamically adjusting background lighting. However, there is room for improvement, as World Labs' scenes are not fully explorable; user movements are limited to a small area, and rendering errors, such as objects blending in unnatural ways, can occur. World Labs clarifies that this is just a "first look" and that they are working on improving the size and fidelity of their generated worlds.

Founded earlier this year, World Labs has raised $230 million in venture capital from various investors, including Andreessen Horowitz (a16z), Ashton Kutcher, Intel Capital, AMD Ventures, and Eric Schmidt. With a valuation exceeding $1 billion, the company plans to launch its first product in 2025. In addition to interactive scenes, World Labs aims to develop tools that are useful for professionals such as artists, designers, developers, filmmakers, and engineers, focusing on a broad spectrum of clients ranging from video game developers to film studios. According to Justin Johnson, the company's co-founder, while it is already possible to create virtual and interactive worlds, this traditionally requires millions of dollars and significant development time, and their goal is to make the creation of vibrant and interactive 3D worlds more accessible.