Sigil Wen presents DiffusionCraft | Demo Day, Fall 2022

Detailed Summary

Introduction (0:00 - 0:29)

Sigil Wen introduces DiffusionCraft, a Minecraft server he created for fun, emphasizing that he is a startup founder who hasn't yet attended college. He explains that the server allows users to generate structures within Minecraft using AI, showcasing an initial super flat world that quickly reveals complex AI-generated objects.

Demonstrating AI-Generated Structures (0:29 - 1:23)

The demonstration begins by zooming out to reveal an animated version of Elon Musk, entirely constructed from Minecraft blocks, specifically ice blocks. Sigil highlights that DiffusionCraft is a public server accessible at diffusioncraft.com, and he notes that the generated content is dynamic due to public contributions. He then shows another creation, a basketball player, and expresses surprise at the AI's ability to generate such detailed and specific figures.

The Technology Behind DiffusionCraft (1:23 - 2:00)

Sigil explains his fascination with how a local AI model, without internet access, can encode and generate such complex structures from natural language. He mentions that he worked on DiffusionCraft with another Stanford student, Brandon, during a hackathon. He shares his personal connection to Minecraft, having dreamed of being a Minecraft YouTuber, and hopes this project might help achieve that goal.

Collaborative Vision and Server Mechanics (2:00 - 2:54)

Sigil invites the audience to join the public server, mentioning his plan to wipe the current world and restart it, while also taking snapshots of the entire world. He draws a parallel to r/place, envisioning DiffusionCraft as a platform where anyone, from professors to young children, can log on and contribute to building the world. He states that he will report a timeout for the project.

Technical Details and Performance (3:33 - 3:47)

Sigil elaborates on the technical setup, explaining that a GPU instance runs Stable Diffusion, the AI model responsible for generating the Minecraft structures. He states that the inference time for these generations is typically less than 10 seconds, showcasing the efficiency of the system.

Notable Quotes

"I'm Sigil, I'm not normally his my first friends and family event... I haven't gone to college yet so just because they did gatherers building but this is something I made for fun it's called the fusion craft and it's a Minecraft server." — Sigil Wen

"The cool part is is that it's made out of all Minecraft blocks right so if I go down here fly down okay these are ice blocks." — Sigil Wen

"This is a public service so you can access it if you go to diffusioncraft.com." — Sigil Wen

"I just find it very interesting how like a a local model with no access to the internet had can encode all of this within its weight or natural language." — Sigil Wen

"I'm trying to do something like this where anyone from you know uh you your professor to your like seven-year-old cousin and pop can log on to this the server and just add different things to it and then I'm going to report a timeout." — Sigil Wen

"Basically what's happening right now the promising to um this GPU instance running stable diffusion uh and I think inference time should be um less than 10 seconds." — Sigil Wen