Welcome to part 2 of my Indie Open World series! Last time I wrote about how I put together a Blender workflow for authoring a large world terrain that can be non destructively split into tiles using Geometry Nodes, and prepared for export.
This time, I'm going to describe how I'm importing these tiles into Unity, the beginnings of my world editor, and my lightweight system for updating/rendering a large world with persistent entities.
Exporting the world from Blender gives me a model prefab containing a bunch of tile meshes.
There's some other random stuff too but let's agree to ignore those
It seemed to me like the first few things I'd need to figure out were:
To know the location of the tile, I'd already decided that at least for now I'd just store this information in the name of the mesh asset, which was already done. I don't want to have to manually place all the tiles into some kind of world prefab, instead I'd like a tool that does this for quick iteration.
I also don't want to have to create prefabs for each tile manually, so the tool should probably do that too.
My go to solution for storing data in Unity is the ScriptableObject, so the idea of storing multiple maps immediately sounded like a good match for this. Plus, the WorldMap asset could have methods to automatically parse the map tile names to get their location and generate the necessary prefab assets!
First I sketched out the data the world map asset would hold, the primary thing being the list of tile meshes.
Other than the meshes, it also has config values for the base size of a tile, and the scale factor the world as a whole will have. The base size is necessary so that I know how to position the tiles and translate from tile coordinates (-1, 2) to world position (-100, 200). I could have attempted to calculate this base size from the bounds of the mesh, but I didn't want to deal with any small inaccuracies due to tile vertices not being quite aligned to the grid. Additionally, I could have just scaled up the meshes in Blender, but I felt like being able to easily scale up the world would be a handy iteration tool.
Now I need to generate prefabs using these meshes for each tile. In addition to the terrain mesh, I intend each tile prefab to contain objects that aren't considered game world "entities", like vegetation, rocks, and other decorations / props. Those will be something we'll edit in later, but for now I just need a good way to create the initial prefabs.
I won't go too deep into the editor code to create and edit prefabs, but the basic idea is to temporarily create a GameObject in the scene, make modifications to it, then use UnityEditor.PrefabUtility.SaveAsPrefabAsset to save that as a prefab in the project. I name the tile prefab based on the tile coordinates, so the cool thing is I can detect if this tile has been created before, load it, make any necessary changes, and save it again, preserving any edits I might have made to the prefab previously.
I also created a small component TerrainTile to store some data about each tile prefab, especially its tile coordinates so I wouldn't have to keep parsing the name to get the coordinates after the first time.
With that, I had my basic World Map asset which could generate prefabs for each tile mesh using an inspector button, and set up all the relevant coordinate data for the tiles by parsing the tile names.
If you want to have inspector buttons for [ContextMenu] attributes check out my gist!
Look at all them prefabs
Here is what it looks like to edit a single terrain tile prefab, which I've already populated with some extra decorations like vegetation and spawn regions for fish.
it's all cyan in the scene view not because it's one of my favorite colors but because i've been too lazy to fix my custom fog effect for the editor view
Ok, so far we can import our terrain tile meshes into Unity, create a World Map asset to describe the world they are a part of, and easily create prefabs for each of those tiles which know their position on the map. This is actually enough that we can start putting together the tiles at runtime into a navigable world!
My basic plan for this is some kind of WorldMapManager which looks at the camera position and spawns tiles that are within some distance threshold. Once the distance goes beyond that threshold, the tile can be hidden. If necessary I could also destroy and recreate tiles as they go out of range, but unless I run out of memory I think I'd prefer to keep them around to avoid constantly instantiating things and also to help preserve the state of tiles where possible during a single play session.
The first thing we're going to need to be able to do is easily convert from world space to tile space and vice versa. This is quite simple, world space is simply scaled up from tile space based on the final world size of a tile. So as mentioned previously, given a tile coordinate like (-1, 2), we can convert to world space by multiplying by tileBaseSize * tileScale. To go back, we just divide by the same amount. I made a bunch of helper methods to convert to and from these spaces, which would allow me to figure out which tiles the camera is close to.
As usual, I'm starting with more or less the simplest way of doing this I can think of. This is to iterate over every tile, see if it should be visible based on the distance to the camera, and then apply that state. Perhaps in the distant future there might be too many tiles to comfortably iterate over every frame. If that happens, I'll probably just time-slice it and update N tiles per frame.
This is the extremely simple implementation, where SetTileVisible will either first spawn the tile game object if it doesn't exist yet, otherwise it will set it active or inactive.
i couldn't think of anything interesting or funny to say about this image
I decided to compare the distances in tile space, as eventually world space might be big and this will help avoid any float precision issues that come up. Also there's more tiles than cameras, so it felt like it was only fair that the camera should convert its position rather than every single tile.
renderDistance is defined in tile space, and is kind of just a magic number that I tweaked until I couldn't see the edge of the map. It could probably be calculated to just be the camera view distance in tile space. One day I may also want to update this to consider a tile visible if any part of the tile is within the renderDistance rather than just the center point. For now, I just include add a small buffer to the distance value to account for this.
I store the tile game object instances in an array indexed by their tile coordinate, with the usual 2d array to 1d array conversion of int tileIndex = tileCoord.x + tileCoord.y * sizeX where sizeX is the number of tiles wide the world is. There's also a bit of extra math to offset all the tile coordinates to start at 0 so there are no negative indices. This lets me directly find the relevant tile instance given a tile prefab reference, and if the tile at that index is null I know to create it for the first time.
things like setting the name should probably be disabled in a shipping build to avoid extra allocations, but also we're already instantiating a whole chunk of terrain so what's a few strings?
Just with this simple setup so far, I can now drag the camera around the scene and see the tiles get created/hidden/shown as it moves around! Note that in Unity, meshes are automatically culled from rendering when they are outside of the camera frustum anyway, so hiding the terrain tiles when offscreen isn't saving much in terms of rendering. Really the main point of this system is to avoid spawning every single tile at once, and instead spawn them lazily as the player explores. It's also an easy way to avoid updating things like vegetation or other random props in the tile prefabs that have logic on them.
Oh, and later, I may want to generate low poly LODs of the terrain tiles to use when they are far way. This system will be easy to modify to have tiles use those lower detail meshes as they get farther from the camera, which will have a useful performance impact assuming I ever hit my vertex count budget.
Ignore the monkey, there are no Blender monkeys in this game
And for good measure, here's a quick clip of flying around the world in first person, to demonstrate how you can't see the tiles loading in.
So far, we can create our world terrain, split it up, and fly around the world seamlessly without noticing tiles loading in around us. We can also edit each world tile prefab to add extra props like vegetation and other mostly-static stuff.
However, there's currently a big thing missing, which is things that move around like NPCs and other larger landmarks. The tricky thing with NPCs is that since they move around, what happens if they move outside of the tile they came from? If we do nothing, that could result in an NPC popping out of existence right in front of the player when the NPC's parent tile is unloaded. You could maybe unparent them from the tile to let them move around, but then you'd have to track them separately somehow so you can unload them when they get far away enough.
For some types of moving things, the above solution might work well enough, and indeed it is what I do for things like fish, sea creatures, and other types of objects that don't need any kind of permanence. I place fish spawning zones in the tile prefabs, those spawn zones spawn their fish objects outside of the tile's hierarchy, and a separate LOD system is used to remove the fish as they get far enough away from the camera.
For things like NPC submarines that the player will encounter in the world, a more complex system is needed to achieve what I want.
World Entity System Goals
Why is it important that entities receive updates and exist even when not close to the player? Mainly because I want to try to simulate a "living" world as much as possible. This means NPC submarines traveling to and from points of interest, which means they need to know which points of interest exist far away. This will allow the player to organically encounter a traveling NPC in a way that wouldn't be possible just by randomly spawning encounters as they explore, and I think it will create a lot of fun design possibilities for emergent gameplay.
I built something similar to this in my previous game Sail Forth and it worked quite well, so I decided to basically remake that system with some improvements.
The main idea behind the system is that the world entity itself is the low-detail simulation of the object, and at some configurable distance from the camera, the entity will spawn the high-detail GameObject version of itself into the scene, and despawn itself when it's suitably far away. It will be up to each entity implementation to figure out how to store any necessary state needed to recreate the high detail version.
Entities exist outside of the scene hierarchy, not using GameObjects. I decided to use ScriptableObject for this for several workflow ergonomics reasons which we'll see later, but really these could even just be a plain struct and be even cheaper to create.
Each 'entity' in the world is represented by an instance of a WorldEntityBase, which is a ScriptableObject. One of the workflow ergonomics previously mentioned is that since these are Unity Objects, they can be Instantiate()ed, so I can use the assets as essentially entity prefabs. At runtime, I'll Instantiate instances of each entity from the relevant asset to represent each entity instance in the world.
Here is an example of one such entity prefab, this is for a trader submarine NPC.
The runtime stuff is just a style convention I've been using instead of using properties or HideInInspector, I simply like the ease of debugging this affords without having to enable Debug on the inspector and it lets me know that I don't need to set any of those fields
I chose to make a base class for all entities and derive different types of entities for each unique type of entity I needed. It's too bad that ScriptableObjects can't have components, and honestly there isn't a huge reason to not just use GameObjects for this. I don't think the cost of having even a few hundred game objects sitting in the scene (with no Update callback, critically) would be much more than the same number of ScriptableObjects, and having the component workflow might be a big improvement over this.
That said, this is working so far so unless I run into big issues where components are necessary I'll probably just stick with this approach.
Entities are managed by a WorldEntityManager, which is responsible for managing their lifecycle and updating them, in addition to telling them when to spawn and despawn their high detail counterparts. Since the majority of entities are offscreen, and there may potentially be hundreds of them, I chose to time-slice their updates. This means that I only update N entities per frame, where N is some tweakable number that I can adjust. I track the total elapsed time between each entity's last update, so the dt value they get represents the actual time since their last update, allowing them to simulate offscreen accurately.
Most entity updates are very simple: for submarines they are mostly just moving in a direction. Since there is no actual submarine game object with physics spawned unless they are in view, this movement is just a pure translation with zero physics or collision checks being done. In fact, I don't even worry about whether their position is intersecting with the world, I leave that to the spawn logic to position the physical submarine in a sensible place given the entity's position.
While the high detail physical submarine is spawned, the entity just updates its own position to follow the real submarine object, which has its own logic. Sometimes the entity manipulates the submarine's AI or settings based on what goals the entity had, like setting the destination for the submarine path finding to head towards.
An example of how simple the update logic is for a submarine entity
This also shows an example of how entities can query for other entities in the world, which allows them to interact with each other regardless of where the player is.
Here's a quick video of the entity system in action, you can see the "low-detail" entity, represented by a sphere in the editor, traveling along a path until it suddenly gets close enough to the player to warrant spawning the real submarine, which takes over the entity logic from there.
my favorite genre of game to make is apparently the kind that requires 1 million debug draw lines
Since each derived entity class also implements its own Spawn() and Despawn() methods, its very flexible as far as what it means for an entity to be spawned. In the simplest case they just spawn a prefab at their position, whereas submarines are a bit more complicated and require some extra setup after spawning the prefab. An entity technically doesn't have to spawn anything in Spawn(), it could simply use the callback to know to begin doing more complex update logic.
It's worth noting that in the future, when there are hundreds of entities, I may end up wanting to further optimize the update method such that nearby entities receive more frequent updates than distant ones. I'm leaving this until I see how things behave at that scale, but if I do end up needing this I plan to somehow spatially partition the entities and simply update the nearby ones more. In Sail Forth I had around 300 entities max and never ended up needing to do this.
I don't want to totally blow up the size of this article by also describing the entire save system, so for this aspect I'll simply say that the abstract WorldEntityBase class requires a LoadSave() and WriteSave() method to be implemented by all its derived classes. These methods get called as the game is saved or loaded and work in terms of another base class, SaveDataWorldEntity, which can be derived from to store any special data about that entity.
ezpz save data
This actually already covers most of what I need entities to do, the last remaining bit is that I need an easy way to place them in the world.
So far, I've just been editing the world by editing the individual tile prefabs. At first I was tempted to just stick some kind of entity spawner object in the tile prefabs, but the issue is I need these entities to exist even if the player has never visited that part of the map. If they've never visited that area, those tiles have never been spawned, thus any entities tied to those tiles wouldn't be either.
So, we need some system outside of the tiles to know where all the entities are.
I decided the simplest and most convenient way to place entities in the world was to create some kind of custom Unity editor window which could store data in the World Map asset. In order to edit properties about an entity in the scene, I made a WorldMapEditorEntity component which is only ever used at edit time by the custom editor window. It places these game objects in the scene, and then when the scene is saved it copies all their data into the World Map asset. This way I can utilize the normal Unity translate gizmos and general game object editing workflow like duplication, etc, and the entity game objects are just thrown away once I'm done editing.
The white spheres are entities, obviously
Using the WorldMapEditorEntity, I can choose what type of entity it is by selecting the appropriate entity prefab asset and move it around in the scene normally. The WorldMapEditor window handles updating the appropriate data for the entity, hooking into the scene save callback to store the data in the world asset, and assigning/maintaining persistent IDs for every entity so that they can find themselves in the save data.
At the end of the day, the World Map asset from earlier stores the list of entity data, and the World Entity Manager uses that data to spawn entities when the game is started.
FYI: put a string field at the top of a class or struct to give array elements a name
Another bonus of this editor window is that it uses the World Map Manager to spawn all the tile prefabs and lay them out when I use the Load Editor World button. This way I can edit the world tiles in context of the whole world instead of individually, without which would probably be pretty hard to have confidence that I'm editing the right part of the world.
I think that just about covers the whole system! In summary:
This is a super early version of the system. There's still a bunch of game to build, and I'm certain I'll end up making a lot of changes to this whole system as I discover more things I need. This is also a somewhat specific system that is tailored to my game's exact needs, so your mileage with such a system will vary! I try to put this kind of disclaimer wherever I can, just remember that I'm in no way making any kind of claim that this is the best way to do anything. I'm simply documenting what I'm doing for my game, in the hopes it might give ideas or be helpful in some way to others! 😊
One thing that's definitely on the horizon as far as improvements go is having entities display themselves a little more informatively than plain white spheres. I think it shouldn't be too hard for them to just spawn a preview version of themselves while the editor is active.
Thanks for reading! Next I hope to start more of a devlog series introducing what this game is about and summarizing the progress I make every few weeks or so.