I was thinking exactly the same thing that you.
Recast is a great solution to explore the walkable mesh of the world, but maybe I didn’t read carefully to understand the complexity of the problem and probably I am missing something :)

Anyway is always cool read about different approaches to solve problems, specially when the approach doesn’t give the expected results, it is great to experiment, thanks for the post :)

Based on my experience with random walk on a console title with jumping, no, the plot is still readable.

Braid forever changed my taste for games.
I’m expecting The Witness to be even more interesting.
I hope you keep raising the game quality bar, so other will follow. Who knows, maybe some day. :)

I’m hoping it’s going that direction, adding in a shooter style rather than P&C.

What I am saying is that your proposed solution does not seem to address any problem that we have in any way that we cannot already address… yet it introduces new problems via the sampling step.

This is characteristic of many tech-driven ideas: they claim to solve problems, but really don’t (or else they solve the small problems but leave the big ones). I used to be mainly a programmer who had tech-driven ideas. After a while I learned to see the way in which they so often go awry. It can be subtle if you are not used to seeing it.

This has nothing to do with “thinking outside the box” or whatever. I do plenty of that, believe it.

I totally understand what you’re saying. I was jumping too early to propose a silver bullet I have used myself :) But really I’d like to challenge you to think outside the box where the norm is that that physics system is _the_ way to implement player movement.

Any explicit navigation space is kind of a precomputed dual representation of your dynamic model. Being able to visualize it, is very powerful design tool and implied in the original post. Just like your physics collider approximates the player movement, so does the choice of “sampling” approximate your navigable space. There are no right solutions on either system, you only get a good enough compromise.

The way I have implemented the voxelization is that: first I use conservative voxelisation to detect solid space, then for each potentially walkable voxel I check if there is enough space for the character the stand there, and mark it as walkable. Finally, the walkable area is dilated by the walker radius. The end result is conservative estimation of space where the character can stand without colliding any geometry or falling off from cliffs.

Automatic walkable area detection (using voxels or otherwise) does not mean that you need to use the visible geometry to generate the walkable areas, you can use the same tweaks and tricks you’d do for physics colliders. I have found (RLE) voxels to be very flexible data format to do all kinds of tricks which are more costly otherwise (i.e. minkowski sums, morphological opening, closing, lowpass filtering, painting, surface types, etc). You can easily use volumes to paint the surface for special purposes (i.e. to disable walking, or mark water areas).

Sounds like a party to me. Get a real job! :P

I’m going to let Jon comment on this, because I’m not sure what he wants to say publicly about the interface and so on. I was simply saying that from a _systems_ perspective, we need to be able to handle continuous movement and continuous look-around; that’s a core requirement. The interface design isn’t my purview so I can’t comment on that.

– Casey

IF you’re thinking about possibly taking over the whole movement and collision thing for this, then you could also look at other kinds of automatic ground mesh generation.

e.g. at PathEngine we have a mesh generation pipeline based on BSP combination of source geometry, which is much more precise than voxels, and which then sounds like it could be quite a good fit for this.

You can choose exactly what geometry to pass into the automatic generation process, of course, so you’d pass in those bevelled / smoothed collision volumes instead of the world geometry components they are designed to replace, and the ground mesh coming out of the BSP combination would then reflect your bevelled or smoothed geometry appropriately.

I agree that choosing which steep areas should be impassable purely based on ground slope is not at all ideal, and in this case I’d recommend turning off maximum slope checking and instead flagging walkable areas explicitly in the source geometry (e.g. by painting flags into terrain data, or marking objects by type, or by instance, or keying this partly off of the applied texture, or whatever).

The BSP processing tends to be a lot more computationally expensive than voxel processing, but can be parallelised.

The post was already a bit long as it is, so some things got left out :) The truth is that you cannot tell that information from the _screenshot_, you can only tell that information when you actually watch it run. When it flood fills down the first side, it will not penetrate through, but when it finds its way around to the other side via another path, it then reconnects back up.

This is not actually how I will do this detection in the future, it was just an accident that happened during testing :) In the future I plan to actually draw edges that were only traversed in one direction, but I haven’t gotten to that yet.

– Casey

I think the overhead views you demonstrated would be especially useful in identifying problem areas, if I understand the visualization.
Problems marked in yellow: http://img571.imageshack.us/img571/3064/thewitness.png

You could perhaps improve the visuals by drawing transparent filled cyan polygons between walkable points, and transparent bright-red polygons between wall points. Though that’d assume that everything between walkable points don’t have collision boxes that are so tiny they fit between your points. Perhaps you should also mark in bright yellow every collision box that is so small it’s likely to be missed by your sampling of the terrain?

I also don’t see in the last picture how you realized that you can walk one direction but not the other. I’m not noticing the visual signs that indicate that.

Yeah, that makes sense, but I don’t think it would necessarily have to come across as a restriction (that just depends on the implementation!). A click-to-move system might be a nice homage to the prerendered times and it seems like there’s some room for really usable innovation there.

Obviously, many PC gamers will only accept WASD/mouselook, but others might actually prefer a simplified movement scheme to match the methodical, pensive nature of the game. It would certainly at least be appropriate for touchscreen platforms!

Anyway – I’m rambling now. As always, you guys are doing fantastic work; keep it up!

Hmm…. thats weird. :)

Then, verify that this connectivity is actually walkable, by traversing the tree and checking each “portal” (only ones with at least one walkable surface attached to it). Most of them should be trivial-accept (two large empty spaces connected by a large portal). For smaller portals you could do “proper” swept intersection type stuff, or just throw a bunch of random physics probes at it like you’re doing now, but at least you’re only doing it for the edge cases.

It is worth pointing out, also, that the appealingness of this kind of voxel approach is often inversely proportional to your standards of gameplay quality.

We want to make walking around feel good, while also ensuring that puzzles aren’t broken / etc. This often involves making collision volumes that don’t actually match visible geometry: say, we might bevel them in a certain way so that you bounce off them smoothly, or we might intentionally make them smaller than visible geometry in some cases and bigger in others.

The appeal of doing a voxelization of visible geometry is that everything happens for you magically and you can walk around. But the more stuff you actually want to hand-tweak, the less magical the automated thing is.

In other words, the idea of making a game where you can walk up any terrain unless the slope is X is fundamentally not what we want to do here. It is pretty far from what we are doing. What we are doing is more labor-intensive, but the belief is that it will yield better results for this kind of game.

(But yeah, as Casey implies, there is also an interpretation layer that happens via the code that affects the definition of what “walkable” is. We do a lot of hand-placement of collision volumes to set up this definition, but the code needs to extrapolate all the details, and there are lots of ways to do that.)

Thanks for the kind words! Regarding the suggestions:

1) One of the problems with doing what you propose is that I don’t currently clean up edges that cross each other. That’s a problem just in general, and not a fault of your suggestion :) But it’s what would prevent that suggestion from looking clean without me first doing a pass to fix intersecting edges. But in the places where edges aren’t crossing too badly, then I suspect it would produce a reasonable mesh, yes – kind of the dual of the edges, if you will.

4a) Actually I do kind of like the idea of determining the length of time it takes to traverse parts of the island. I don’t know how Jon feels about it, but it does seem like a good idea to have some measurements like that just to understand how long it will take people to go between certain places they might want to.

4b) I’m not sure exactly what you mean by “bottlenecks”. Are you saying to create sort of a “chances that I will be in this location given random traversals” value for each vertex, and then you look at that to see where players tend to be, all other things being equal? That does seem like a useful idea, although as you suspected, I don’t know if Jon would be interested in this kind of data because at some point it would definitely cross into “metrics-driven game design” territory :)

– Casey

I’m not sure I understand how voxels solve the same problem. I suppose if you are saying a priori that you don’t care what the _original_ definition of walkable was in the game, then sure. But if you want to know what the _game_ considers walkable, voxels don’t help you, because how do you know whether or not you can reach that voxel, or what the shape of the actual walkable area is inside the voxel?

But yes, I agree that if/when I start replacing the collision system itself, then something like rebuilding the walkable regions with voxels is on the table, since there I’m technically in control of what the definition of “walkable” is in the first place, so I do not have to use the definition induced by the game’s collision system. But this is a very different problem statement than determining what the walkable area is _for a given game system_.

Know what I mean?

– Casey

A few brief suggestions for your open questions:
#1: For post-processing and data visualization, what about selecting a point, picking a colour depending on the proportion of white to red edges going from that point, then painting a polygon on the ground at that point. That polygon would always end at 1/2 of each edge.
So in a rough sense, you could have something like “green = trivial to traverse” to “red = cannot be traversed” or something.
I have no good idea what to do in border areas where edges cross each other. Perhaps my suggestion is just impractical.
#4: You might want to find out how long it takes to go from point of interest A (perhaps there’d be one at each of the several hundred puzzles) to interest point B? Although that sounds like the kind of optimization Jon Blow might dislike for being to Valve-y. I don’t know. At the very least, these numbers might give some additional ideas for optimal movement speed or world size.
Also, perhaps you could use your existing data to find bottlenecks? (As I understand it, The Witness is not about navigational puzzles whatsoever, so bottlenecks may be mostly undesirable.) I.e. if you go from point A along a random path to point B and come along point C, that’s a bottleneck to that one path, which is okay. But if most paths in an area go from point X to point Y and come along that same point C, that might be a problem – or an indication to make point C more interesting.

In any case, thanks for writing this.

You should definitely take a look at the Recast https://code.google.com/p/recastnavigation/
Test the demo (if using windows, I recommend to build it first), and take a look at the different debug options to see how the navmesh gets build. I made it so take my opinion with a grain of salt :)

While I was working on Recast I went though a bunch of sampling schemes and eventually ended up using voxelization as basis for walkable area detection. It was several orders of magnitude faster than anything else, and so robust that you can throw pretty much anything at it.

When using tiles the build process is embarrassingly parallel (each tile can be processed independently). That also allows you to do dirty rectangle type of incremental updates and what not. Since the process is quick, you can often afford to make a quite dense grid. Usually folks are using 20 or 30cm cell size, but in some cases where the navmesh is used for player movement people are going as dense as 5cm cell size.

I’ve often ran into debates on whether the navmes or physics should drive player movement. Physics and AI navigation are always just approximations, and there is usually really bad coupling between the two. The worst thing you can do is to make one of them to drive the other. I’m biased, but I would pick navmesh first and use physics to “plant” the character into the world (i.e. feet, walk height, etc).

I’m not going to argue that my algorithm is better; it’s not. You find exact edges whereas mine does not. Yours handles polygon soup where mine assumes that you can’t walk up a steep slope and I did skip over handling the overlapping areas. Yours is more computationally efficient, mine throws CPU cycles at the problem. But I still think mine is simpler from the perspective that it doesn’t require as many IQ points from the programmer, which can make the difference between doing something and getting overwhelmed doing nothing. (It’s the same reason why oct-trees were more popular than BSP until common tools were released) I apologize for not being clearer.

None the less, my recommendation of testing with a smaller collision body will help on any algorithm that uses floating point. If you have a hole that’s exactly big enough you may or may not find it due to precision issues.

Keep up the good work.

Yes, the edges get tested in both directions. This is actually a natural consequence of the algorithm if you don’t take steps to prevent it; because each point attempts to connect with those around it, so long as you don’t intentionally prevent it from checking points to which it already has an edge, it will naturally traverse the reverse direction of edges incident on it.

Right now I do allow some edges _not_ to be tested in both directions, namely those that occur in oversampling regions. Right now the walk mapping time is dominated by the cost of calling the player motion code, so if/when I rewrite and optimize that, if it is significantly faster than it is now I will probably force reverse-direction testing on all edges.

– Casey

Are you testing each edge to ensure you can walk both directions?

When you eventually do write your system for overnight testing via user input, may I suggest looking to Roomba for inspiration? Namely the wall following behavior.

I had what was labeled “triangles of doom”. They were navmesh triangles that were broken in some way. Couldn’t walk into, could walk into but not out, etc. There was a small variety and I hated them all. By the end of development the fixes were all data/level design side. When a new triangle of doom was found I would debug, learn how to detect, color code triangle in navmesh debug render mode, and automatically output all debug triangles.

You have two “bad” cases already – slopes and one way pathing. Both should be possible to programmatically detect and spew to console / debug output. They may get fixed by physics system bugs but eventually you’ll reach a point where all pathing bugs are due to bad data. Automatic detection and reporting of that is necessary.

I really like the Bubble Meshing algorithm described here: http://www.research.ibm.com/trl/projects/meshing/bubble/bubbleE.htm

Unfortunately, it might be patented.

It depends how dimensional a space you want to have. At some point, RERT’s start being a win again, if you’re talking about systems where there’s many different configurations for the player (an example might be a game where you have multiple configurable limbs, ie., a giant robot game or something like that, where you had to consider the placement and orientation of the limbs as to whether or not the robot could move through various openings). So the flood-filling approaches in general will only scale up to some small number of dimensions, I think. But simple running and jumping probably aren’t very difficult to shoehorn in.

As for drawing things, well, that’s another story. I agree it might be tought to visualize the results of such a system even if it did work well. I haven’t thought much about that problem since we don’t have to solve it for this game :)

– Casey

I have not looked at Recast specifically, but now that I am starting to work on actually revising the collision system, I have been thinking about the best ways to change it into something workable. Some of that thinking is about whether or not preprocessing is a win, and what sort of preprocessing I might do. At the very least, I suspect I will do some sort of geometry processing to produce an analytic walk map so that I can compare it to the Walk Monster walk map and ensure that they coincide. Whether or not I actually do preprocessing to produce a new and different collision representation… it’s still too early to tell.

– Casey

First, although the system is drawing the graph on the ground, it’s actually doing the full testing process to see whether or not the player could have walked that path. So off-ground things that would have prevented player motion are implicitly accounted for.

Second, I had the same thought for finding missing collision volumes, but unfortunately the way The Witness world works is not amenable to this. There are lots of actual objects that litter the ground which are not “collidable” objects, because it is considered to annoying for the player to have to navigate around them, so we’d rather just let you walk over them, etc. So the system can’t really automatically detect what should and should not have a collision volume, unfortunately.

There may be some metric I can employ that’s like “if the thing is a certain size” or something like that, and maybe I’ll try that at some point just because it might catch a few things, but unfortunately there’s no way to definitively make something that can detect all cases of a forgotten volume (at least not that I’ve thought of yet).

– Casey

The movement in The Witness, at least on the PC, is all continuous, so you can walk wherever you want and look wherever you want, just like any other mondern first-person game. It does not restrict the player to clicking on edges of the screen to turn fixed amounts, or moving forward in specific amounts, etc., like older pre-rendered games did.

– Casey

The collision system internally already does (at least try to) do continuous collision detection, so in theory there should not be errors of the form that would be correct by successive refinement like you describe. However, I have not actually looked into this case yet, so I can’t say what the actual problem is or whether it would be solved by various fixes.

How much do you think the complexity of this solution would be multiplied by having a larger player state space, eg jumping, running vs walking vs crouch-walking, and other means of propulsion? The algo visualizes very nicely when it’s just a player walking, I’m curious as to if a jumping player would make the plot unreadable.

First you rasterize your polygon soup into voxels (keeping track of the steepest slope at each voxel). Next, you use the slope and connectivity information to flag walkable voxels, and finally, you convert the contiguous walkable voxels into a navigation mesh. Now instead of dealing with moving a capsule through polygon soup, you are just moving a point on the surface of a connected triangle mesh!

In addition to finding orphaned collision volumes, you could use this to find objects that are lacking collision volumes. There are a couple of different ways you could do this but the simplest would probably be to see if there are any walkable points contained within a collidable object’s ground-level footprint.

Overall a very interesting article and certainly an enjoyable read. I look forward to your next post.

Also thank you for putting such a great amount of time and effort into making sure the movement for The Witness is perfect. Movement bugs are probably the most annoying and distracting to be found in games, cannot wait to play :D

This is not a good algorithm for a number of reasons. First, “cover the world with squares” is glossing over the main hard part of these types of algorithms. How do you propose to cover an arbitrary polygon-soup world with squares, exactly? Second, checking the world on a regular grid means that you will never find bugs in the collision system that don’t align to that grid. Third, subdividing and rewalking rectilinear edges doesn’t properly probe boundaries. You could have all sorts of thin-hole bugs along a boundary and never find them because you’re only ever stepping rectilinearly or diagonally to the grid on power-of-two boundaries.

Etc., etc.

The LDS algorithm I propose is actually much simpler to code than what you’re suggesting (it’s very few lines of code), doesn’t have these problems and handles the world covering problem implicitly, without needing to have a grid imposed for it. It also takes far fewer unnecessary samples since it does not need to use subdivided rectangles to probe what are inherently non-rectangular boundaries.

1. An interesting take on the specific problem.
2. It’s great to see your overall thought process tackling a fairly open-ended problem
3. It’s especially exciting to see the level of detail and care going into the game because I’m really excited to play it :)

Keep ’em coming!

As for parameters, make sure your starting grid size is smaller than the smallest collideable area. Also, run this test with the player being half the size you intend to use in the final game. The minimum subdivision threshold should be half of your test object size (so 1/4 player size).

Yes, definitely. Right now there hasn’t been any need for such things because there are blatant problems showing up that are trivial to reproduce. As we improve the collision system, and we get down to more subtler bugs, I suspect that we will need more assistance with repros like you say.

– Casey

There are only 2 things hard in programming: cache invalidation, naming things and off by one errors. https://twitter.com/Jonathan_Blow/status/267836389956149248

Thank you! Controls are so amazingly important these days! I’m glad to see you’re taking them seriously. :)