01 / Why a town
Why a town
It’s a hobby project and still very much under development. Each character represents an agent, and the world gives me a way to see their activity instead of following everything through logs and text.
People have been DMing me asking how I built my little agent town, so here’s a breakdown 😊 It might look complex, but the underlying system of agents was already built and running without a visual UI. The town is essentially a visual skin over that system. I just gave the agents cute little faces and somewhere to live 😂 I wanted an easier, more enjoyable way to see who was working on what across my projects.
It’s a hobby project and still very much under development. Each character represents an agent, and the world gives me a way to see their activity instead of following everything through logs and text.
Visually, it started as a basic 2D town, then became a pixel art city. From there, I made concept art for the world and its 32 characters, incorporating places I love, like Geisel Library, and spaces I wanted in the town, like a cozy jazz and record café.
The world runs in the browser using JavaScript and Three.js. I tested Tencent’s Hunyuan3D 2.1 and Microsoft TRELLIS to turn my concept images into 3D models on my GPU. Hunyuan worked better for my characters, but the buildings needed a different approach. You can see why in the bottom row: soft edges, wobbly details, and architecture that looked like clay.
I switched to Blender scripts that assembled the buildings from individual parts. That gave me more control over the shapes, followed by several rounds of adding siding, porches, plants, stone, and vines.
The characters needed cleanup too: colours, rigging, moving ears and tails, and fixes like restoring the moth’s wing patterns. BiRefNet handled background removal, and Blender handled rigging, animation, and cleanup. I compared the results against the concept art throughout the process.
Behind the town, a Python server and job queue connect the characters’ activity to the agents’ actual work. The agents can use Claude or Qwen3 32B running locally through vLLM or Ollama.
My setup is an RTX 5090 with 32 GB of VRAM, a Ryzen 9 9950X, and 64 GB of RAM. The image-to-3D generation step uses about 17 GB of GPU memory in my setup.
The image shows the progression:
Still plenty to build and fix, but apparently giving my agents faces also meant giving them a café, beach cottages, and a much nicer work environment than mine.