Well, I think we will post a little more frequently, because we'll have increasing amounts to say about the game as we get closer to release. In the meantime, here's:

(It is longer than Long Screenshot #1!)

Voiceover

This week we just finished recording the majority of the final voiceover. We're going to do a little more in early January, but most of it is in the can. After New Year's we'll make some postings where we announce who the voice actors are (you may know some of them from other recent games!)

Physical Box Release

We've had some people asking us whether there will be a physical retail release of The Witness. It's seeming likely, as we have been in talks with a couple of parties for a while about that possibility. *However*, it looks like if there is a retail release, it won't happen until a little bit after the digital release. The reason is just that we need all the time we can get up until the release date in order to get all the finishing touches into the game; but because of the lead-time involved in physical manufacturing and distribution, we would have to freeze the game right about now in order to have it on store shelves for the release date. *Or* we could delay the digital release date in order to wait for the physical. But I don't see a good reason to do either of those things. I would rather just focus our energy on making the game as good as we can make it, and a retail version can lag behind by a little bit.

What Else is Going On

We've gotten the bulk of the translations into the game for all the languages mentioned in the previous blog posting. There is a small percentage of text that will come in a little later, but for the most part those languages are in. Also, we will be adding Hungarian to the list of languages suppoorted at release.

Here are some screenshots of Arabic and Russian:

Of all the languages, supporting Arabic took by far the most programming work, and as you can see from some gaps between the letters, and a couple of cases where letters are too close to each other, it's not quite done yet. I will be working on that as soon as I finish with this blog posting! But I think even with these issues it is starting to look nice.

We're doing *lots* of little gameplay fixes all around the island, like, what does collision feel like in this area when you bump into things, what happens if you are standing in this doorway and the door tries to close on top of you, all that kind of thing.

Visuals are mostly locked down at the moment; we are no longer changing big things about how the game looks, we're more making spot-fixes here and there every time we see something we don't totally like.

Music and Sound Effects

Once in a while I get a question, from someone who really liked the music in Braid, what the soundtrack in The Witness is like. The answer to this question is that there isn't one. There is (almost) no music in the game. This is not an arbitrary decision, but is in fact very important to the coherence of the thing we are making.

The Witness is a game about being perceptive: noticing subtleties in the puzzles you find, noticing details in the world around you. If we slather on a layer of music that is just arbitrarily playing, and not really coming from the world, then we're adding a layer of stuff that works against the game. It'd be like a layer of insulation that you have to hear through in order to be more present in the world.

Instead, we put a great deal of care into the sounds of the world around you, in a way that maximizes immersion in the game. This is made trickier by the game's setting: you are alone on this island, and there are not even any other animals. There are no birds in the trees! In everyday life if we imagine the sound of nature, we'll think of some elements that have no place on the island: a forest naturally has the sounds of birds, plains with smaller shrubbery will have crickets, a marsh will have the sounds of many insects. There's none of that in this game because in this game you are really alone, and it has forced us to be very creative with the audio in order to ensure things have depth and texture to them. This work is being done by Wabi Sabi Sound, who did the sound for the very atmospheric Dead Space series, and more recently some smaller, artier games like Ori and the Blind Forest.

We thought it would be fun to release a few more small videos to give you more of a feel of what the world of the game is like, so here's:

Story and Voicework

Lately I have been doing a lot of work on story and voice acting stuff. When wandering around the island, you may find voice recordings that were placed by ... well, at the outset of the game, you don't know who; and as the game goes along, there is an interesting mystery to unfold about who these people are or were. In the game currently we have what is known as "temp voice", which is me reading all the parts under very noisy conditions and just getting them into the game so we can make sure the data works properly with the engine, to get all the subtitles into the game and roughly synced with the audio (which we need to do sooner than later so we can give the translators those subtitles, which they will translate into many different languages).

We have cast the real actors and starting tomorrow we'll do our first test readings with those actors. We'll use this to get a new layer of temp voice in the game, which will get it closer to final; this will allow me to see how the game really feels with this stuff in there, possibly to adjust the nuances of the way the recordings appear in the game and the way they are played back. When everything feels good we'll do a set of final recording sessions. There's not a whole lot of time until release, and we have several holidays coming up, but we'll get it done!

We are working with Warner Brothers Game Audio in Burbank, California to do the voice stuff; they are providing the facilities and voice direction and production assistance. After we do a few recording sessions I will have more details to share in this department.

Localization

Earlier, I mentioned subtitles. As many games do, we will be localizing our game so that people who speak many different languages will be able to understand the words. We are not recording voice acting in different languages, mainly because it is pretty hard to make sure quality is high when you do that. It is hard enough making sure acting is good in just *one* language. We generally seem to feel the same way about movies: a foreign-language dub of a movie is generally considered to be inferior to a subtitled version, because the acting is so important.

Our localization efforts will involve menu text and subtitles, just as for a good movie.

This is not 100% final, but here's the list of languages we are planning to have translations for, at launch:

• English
• Italian
• French
• German
• Spanish (Spain)
• Spanish (Latin America)
• Portuguese (Portugal)
• Portuguese (Brazil)
• Arabic
• Russian
• Chinese (Simplified)
• Chinese (Traditional)
• Japanese
• Korean
• Polish
• Indonesian

We may add more languages later but man this stuff is expensive, so there is a limit to how many languages we can do at once.

We will have more news for you before too long!

Here is an example, a set of stairs added to a roof structure:

It looks ok, but looking close up you can see how flimsy the roof attachment is:
Or the way the railings are assembled together:

Deanna showed us that it would make more sense to extend the main roof support to accommodate for the stairs:

An in the case of the railings, to do a better job showing the assembled pieces, removing the awkward intersections:

Another example is this stone wall door frame. It lacks any assembly detail for its size, making it feel it was made from one single gigantic piece of stone:
And here is the revised version:

Another type of issue is how two different surfaces meet. The more detailed the island gets, the more these stand out as incomplete. As an example, the way this wood structure connects with the floor:

And after the revision:

Some of this work might seem minute but it really makes the whole environment feel more grounded, adding a lot to the experience!

It does *not* mean that the game is done. We still have a lot to do! But it *does* mean that the nature of the work changes and becomes simpler, because we don't have to be making high-level creative decisions any more. It is now much more about turning the finish-the-game crank (making sure stuff plays well and polishing it up) for anything related to game design, modeling and texturing.

There are a few categories that still need creative decisions (story stuff, the menu system, etc), but for those of you who are awaiting the game's release, this is pretty good news.

There are currently 677 puzzles in the game. That number might be slightly different when it ships (we might cut a few, for example!)

We've been revising the modeling and texturing for areas near the end of the game. There are some nice treats in store!

We've been working on some of the "special effects" in the game, most notably particle effects. Our particle system was old and slow, so we are speeding this up, and then we will make some modifications to help make particle authoring an easier process.

We've been making some baseline functionality smoother. For example, so that you don't need to think about saving your game, The Witness autosaves your progress every minute or so. On a high-end PC, you don't notice the autosave, but on a low-end PC or on a console (because consoles tend to have different memory access properties than PCs), scanning the world for stuff that needs to be saved can be pretty slow, like, it could take 1/10 of a second, which would result in a hitch in the frame rate happening every minute. So we are hammering on this to make it faster, which should also result in saved game files being smaller, since we'll be excluding a lot of things that don't need to be saved.

Ignacio has posted the conclusion of his series on irradiance caching -- at long last! -- and released the source code.

This code is a significant chunk of what makes lighting in The Witness as nice as it is.

Here is a screenshot from another angle:

Most of the key detail has been added to the walls and I feel, if we had to send this build to E3, it wouldn't be terrible.
We managed to get in some really nice framing details that I won't spoil here but I'm sure players will appreciate them!
Here is a shot showing the ruined walls work.

Tomorrow I'll make sure I walk around a lot to hunt down any major issues, get the overall color vibe (mostly get more variation on the walls) and any extra details.

Orsi's Vegetation Update:

To give a little variation to the post today, I thought I would include some things I have been working on as well. While Luis had been working on the structures around the entry yard, I have been placing the vegetation and cables, and re-working some puzzle related things. Laying out the vegetation early on was important to give the place a sense of scale and to get the overall vibe of the place. I am very happy we got an other chance to re-do all of it, because I think it turned out much more magical this time around:

Even though it's a much smaller area than it used to be before, I used a different combination of plants in different areas, so people would be less likely to get lost:

I still have polish to do, to match everything to the walls Luis finalizes tomorrow, and then we should be ready for E3!

Here are the new modular pieces that were done for detailing:

We just finished the day by meeting with the architects and discussing the interior layout of the space so tomorrow I'll start on the interior building and ruin walls.
Here is how it currently looks:

Last week, as you could see from previous screenshot, we got  a rough layout of the area in the engine and tried to find answers to any remaining issues.

Today the goal was to sort out the design  and navigation of the main entrance and the small towers.
This is one of my first mockups that didn't look quite right.

The architects took a look at it and provided some 3d models of how castle stairs are usually assembled as well as sketches of how it should look:

After, I tried to figure out the details from references and what "pieces" I would need to build the structure.
Two years ago I went for vacations in Portugal and tried to see some castles. One of them in Guimaraes struck me as very similar to the entry yard. I made sure I got some pictures and now they are proving to be useful with angles that would be hard to get from online research.

Here is the current result using different modular stairs/floor/battlement pieces:

And the current overall look by the end of today:

Tomorrow I'll be tackling  the rest of the exterior walls and the 3 other towers.

Here's the blog post.

Here's the video:

Update: And here's another post, complete with a video showing in detail how the grass planting works in the actual game.

Working on The Witness, Part 5

Working on The Witness, Part 6

In this one he talks about how to distribute grass. Legally. As well as the different colors of noise one might want to be familiar with in order to do this well.

If you are interested you can take a look at the slides PDF here:
GDC2014_LuisAntonio_TheArtOfTheWitness

Unfortunately the PDF merges animated slides so there is some content that you won't be able to see and you are missing the actual talk but it should be available in a few weeks at the GDC Vault!

*UPDATE - The talk is now available at the GDC Vault, you can  check it here:
http://www.gdcvault.com/play/1020552/The-Art-of-The

Last year I was pessimistic about VR systems in the near future, since the ones I had tried didn't seem to offer much (certainly not enough to justify wearing a bulky headset all the time and fighting back feelings of nausea).

But I was fortunate enough to get a sneak peek at Valve's virtual reality system. It is so much better than anything else I had used that I was instantly very excited by it. Right away I could see games you might design for this system that had been impossible before; so it isn't just a matter of the system being very immersive (which it is) but that it can take video games in general to a new place.

We worked with Valve to get support for the system in The Witness, and that's where those screenshots come from (but when I took the shots, an HMD was not plugged in, and as a result what you see is a dummy calibration that does not correspond to any useful image-warping; the screenshot was really more about just being happy about the stereo display capability.) In late November I spent a few days at Valve's offices working with Atman Binstock and Doug Church to adapt the HMD's input system more thoroughly for The Witness and to build a few scenes that are playable on the device.

(In theory the whole game is playable on the device, but when you are rendering for VR the performance demands become much higher, because you are rendering two scenes at higher resolution than you might previously have been rendering one scene. So the full game world with all content currently does not run fast enough for this device, but that is the kind of problem that gets solved over time).

We had to keep this under wraps, though, because Valve's system was not announced. But this month at Steam Dev Days they revealed details of the system.

For more details on the Valve system, Lee Vermeulen wrote up a nice post of his impressions:

http://www.3delement.com/?p=332

So, we will be supporting this device (and any similar devices) with The Witness. But I am really looking forward to see what gets designed specifically for devices like this.

Luis has been doing a lot of work lately on the mountain and especially the stuff at the top.

Other folks have been doing a bunch of work on things that are not as obvious in this picture!

Current puzzle count: 595

Normally I would post an Island Snapshot, but the viewpoint we have been using for Island Snapshots is a little bit unfortunate lately, since the main thing taking up the view is the mountain, which is an area we haven't worked on modeling-wise lately. (It is a big area so we are going to do a concentrated push on it in a little bit, when some other things get locked down.)

So, here are some screenshots of other stuff we have been working on lately. Both areas are in progress and will look better than this in the final game!

A lake that's central to the island...

A swampy area with light buildings and walkways. The vegetation you see here is temporary! But I am happy we are finally starting to work on this area; if you look at it in the trailer, for example, it's obviously very much more temporary.

There are some basic hacks going on; for example, right now, we are not drawing the real sky; it's just solid blue. Also, right now we are just loading this starting area, but it should not be long before we can load the whole game (Salvador wrote the streaming system a while back and we have been testing it extensively on the PC, so for the iPad it should Just Work. Famous last words.) Also, as you can see, the panel-tracing controls could use some work.

When we can wander around the island generally, I'll post another video update!

I didn’t think Witness was going to be an easy project by any stretch, but I certainly didn’t expect it to be one of our biggest creative challenges. I’ve worked on nearly 80 projects spanning feature film, games, ads and other sorts, and my job as sound designer had always been to broaden the experience in every possible way. The Witness inverts this notion.

A tree of any significance in a film or game will have a bird perched on its branches chirping away. Cicadas will buzz. Wind will rustle its leaves. Branches will creak, stress and morph right before your ears. All this happening without you actually seeing it…or really expecting to, interestingly.

In The Witness every element has purpose. A key game mechanic, in fact, relies on the player’s perception as they explore. So, we’re not just talking aesthetics…noise (literally stated) ruins the game’s playability. Noticing, as a buddhist would call it, is key to discovering solutions and even finding the puzzles themselves. Nothing in the Witness is superfluous.

All this said, the island is a tangible living breathing organic place following roughly the same laws of our universe. The trees exist. They have mass and energy. My eyes receive bands of light reflecting off them, and in fact they are quite beautiful. So then what do my ears hear? Silence is not only a bad artistic choice, it's untrue.

At this point, you’ve probably asked, “What about music?” It’s no spoiler that there isn’t a gigantic speaker system in the sky fed by a mic’d orchestra. So….

None the less, the island is a fascinating place, with lots to see, hear and do. Geoff and I ‘the audio team’ have had an interesting time exploring 1000′s of layers of sonic subtly, and finding ways to engage players to listen ever deeper. This is a unique project for game sound, and we though it would be of interest to share our challenges and discoveries since starting with Jon in 2010 (and of course the challenges yet to meet).

Here's the interview.

This hiatus was the primary motivation behind writing my previous entry as a cliffhanger: although I had fixed the bug, I didn’t have time to adequately document the bug via debugger inspection, so it would have been hard to write anything meaningful showing how the bug manifested itself in practice. I wanted to wait until I had a chance to go through every last little bit of it, so I could be as specific as possible. In that sense, it was meant to be a bit of a cliffhanger for me as well.

But as it turned out, it was much more of a cliffhanger than I’d originally anticipated.

Good News, Bad News

The good news: congratulations to those of you who posted “floating point precision” as a suggestion for the cause of the bug. You aren’t any dumber than I am.

The bad news: we’re all dumber than we should be.

For those of you who don’t already know all about floating point precision problems, I’ll explain the problem we all thought we were solving. The original code, verbatim, looked like this:

bool ray_vs_plane(Vector3 *xpoint, Vector3 const &p0, Vector3 const &dir,
                  Plane3 const &plane) {
    Vector3 n = plane.get_normal();

    float denom = dot_product(n, dir);
    float epsilon = 0.0001f;
    if (fabs(denom) < epsilon) return false;

    float t = - ((plane.d + dot_product(n, p0)) / denom);
    if (t < 0) return false;

    *xpoint = p0 + dir * t;
    return true;
}

To fix the bug, I wrote a modified version to use for the transform manipulator:

bool ray_vs_plane(Vector3 *xpoint, Vector3 const &p0, Vector3 const &dir,
                  Vector3 const &plane_point, Vector3 const &plane_n)
{
    Vector3 local_p0 = p0 - plane_point;

    float denom = dot_product(plane_n, dir);
    float epsilon = 0.0001f;
    if (fabs(denom) < epsilon) return false;

    float t = - (dot_product(plane_n, local_p0) / denom);

    *xpoint = p0 + dir * t;
    return true;
}

I’ve done exactly the same operation, but instead of using the d value of the plane equation to solve for t, I just move the ray itself to be centered around a point on the plane, so that the d value is implicitly zero. My reasoning was very simple: if I subtract two things that I know are close together (the camera location and the picked point on the manipulator), I will have less cancellation error than if I subtract two things that may be wildly different in scale (the d value and the dot product of the camera location and the plane normal).

This is pretty weak sauce as far as numerical precision improvements go. I am far from an expert on the subject (read some Forman Acton, for example, if you want to see how the pros roll). But, when I did this, it fixed the bug perfectly, so I assumed, at least for the moment, that it must be a reasonable fix.

But, honestly, I didn’t really have a good feeling about it.

Christmas Stew

Sometimes, when there’s a lot to do, you’ll live with a bug fix because it works, rather than because it makes sense. But invariably, at least for me, I end up going back to look more closely at something if I don’t feel like I fully understood it. I am sure this is mostly just an obsessive personality trait, but regardless, I rarely find the time to be wasted, because I almost always learn something from the process.

In this specific case, I didn’t have time to look into the problem in depth before I left. I planned to do so for the eventual writeup, but of course, I couldn’t do much without being able to run the code, and while I was on Christmas break, that wasn’t an option. Unfortunately, this meant that I had the entire Christmas break to stew about the bug that I wasn’t sure I’d really fixed properly.

From the beginning, the thing that didn’t sit right with me about the fix was that when Jon originally reported the problem to me, he didn’t mention anything about it happening more on the outskirts of the island than on the interior. Jon’s a very experienced programmer, so if something like that were happening, he would probably notice and include that in the report, because he’d know it would help me find the bug faster.

Instead, it seemed pretty clear that the bug was related more to the camera angle than the position in the world. While obviously the camera angle affects the input values, and could have a number of effects on the numerical precision of the program, it’s not intuitively the kind of thing that I would expect. If these sorts of algorithms were that sensitive in this way, I would have expected to encounter similar problems with camera-relative plane intersections many times before — but I never had.

The more I thought about that, the more it gnawed at me that I couldn’t go see exactly what was happening and prove to myself why the fix worked. I did have the game on my laptop, because I’d done some initial Linux porting work, so I realized that maybe, if I was lucky, I could reconstruct the problem without actually being able to load and run the editor.

That is when I became very thankful that I had been writing these blog entries, because it meant I had stopped to take this screenshot before I left, specifically so I could do the writeup over break:

Do you see what I see? That lovely entity number up in the corner is pure gold for step one of reconstructing the problem. Because I could see the entity number, I was able to go into the repository and get the data file that says where that entity is placed in the world. That let me know the numbers of the center point of the manipulator: -304.114, 193.4923, 4.7582.

If you’re an experienced 3D programmer, then you probably are thinking the same thing that I was thinking when I saw those: how the hell could I be having numerical precision problems with such low numbers? The Witness’s scale is one unit per meter, which means that, at the scale of the dragging we’re talking about, there should be absolutely no problems with solving a single plane equation, regardless of how sloppily you did it.

But if that was the case, then how did I fix the bug? This lead me to step two of reproducing the problem: writing a test program. I wrote a little C program that would take numbers around that point and, for a large set of programmatically generated planes and rays, perform the intersection using both the old ray intersection routine and my “improved” one.

Perhaps unsurprisingly, these results reinforced my suspicions that there wasn’t enough of a difference between the two routines to account for the original bug. In all the tests, my “improved” routine was no more than one decimal digit more precise; so if the original routine produced intersection points that were actually, say, 0.0001 units away from the plane, the new routine might produce ones that were 0.00001 away.

Clearly, the fix just plain shouldn’t have worked. But it did.

Why?

The Wrong Part of the Post

I would love to be able to say that I guessed the real reason before stepping through the code, but I absolutely didn’t.

When I returned from Christmas break, I eagerly booted up the Witness and stepped through the code very carefully, keeping accurate track of all the numbers as I went, waiting to find that super-elusive numerical thing that I had overlooked. To my complete surprise, it turned out to have absolutely nothing to do with precision.

In fact, somehow, although I didn’t know it at the time, I actually did write about the root cause of the problem in my original post. It just wasn’t in the part that I thought. Indeed, rather than Always Look Behind You foretelling the eventual solution, it was Foreign Languages that contained the hint: due to my naivete with The Witness codebase, I misunderstood what it was doing with planes. It turns out that this one line from the original routine was the entire culprit:

    Vector3 n = plane.get_normal();

You will notice that it only occurs in the buggy version of the ray intersection code. In my version, I couldn’t use the plane structure, because I didn’t want an “a, b, c, d” representation of the plane. I wanted “a, b, c, p”, if you will, where p is an actual 3D point on the plane. So I never called get_normal().

Why does this matter? Well, upon stepping through all the code, I realized that get_normal(), being true to its name, takes the unusual step of normalizing the a, b, c values of the plane before returning them to you as a vector. This means that the a, b, c values of the plane that you pass in when you create the plane aren’t the ones you get out.

That wouldn’t be a problem if it weren’t for the fact that, when you create one of these planes, it does not normalize the values you pass in:

    Vector3 plane_n = cross_product(camera_back, translation_axis);
    plane_n = cross_product(plane_n, translation_axis);
    grabbing_plane = plane_from_point_and_normal(hit_pos, plane_n);

This is my fault on its face, because hey, it does say normal, and normals are traditionally supposed to be normalized. But in keeping with my reference to foreign languages, this is something I really just didn’t think about, because I am used to the way I’ve always done things with planes: nothing is normalized automatically, everything is just left unnormalized, because back in the day, normalization was very expensive and you never did it in a math routine unless it was absolutely necessary.

So plane_from_point_and_normal was actually doing what I thought it would: it just stored plane_n as the a, b, c values, and computed d with the inner product. But get_normal() was not doing what I thought it would. Rather than just returning me the vector {a,b,c}, it was normalizing them before giving them to me, which meant all my calculations with them would no longer agree with the d value computed in plane_from_point_and_normal.

And therein, clearly, lies the entirety of the bug.

Just to underscore how much of a native language issue this really is, when I showed Jon the preview of this article, he said, “If you are thinking of the plane as a geometric object, of course the normal is unit length, what else would it be?? But if you are thinking of it as some specific equation all the time, then it is natural to care about the a, b, c, you put in.” So I can only assume, were he using my math library, he would’ve been tripped up by the fact that I don’t normalize normals automatically, exactly the inverse of the problem I encountered with his.

I honestly don’t believe that this sort of thing is a case of us being presumptuous or sloppy; it’s just the price you have to pay for being able to program complex things at a reasonable speed. If we constantly have to think about what it means to work with a vector, or a plane, or a quaternion, we’re going to be a lot slower. It’s much more efficient to have a consistent perspective, and always rely on it.

And of course, the cost is bugs like this — but that’s a very small cost relative to the benefits.

Partial Credit

So, where does that leave you?

Clearly, automatic partial credit goes to everyone who wrote in suggesting that numerical precision was the problem. You came to the same conclusion I did with the information I knew at the time, which was all you had to go on. You certainly can’t be blamed for not finding the real bug since you would have needed the source code to the Witness math library to even see it. And it is true that the numerical precision of the routine could have been improved, as I believe I was able to verify separately with my test program.

But by the same token, nobody gets automatic full credit. There were a lot of reasons why, even with only the information in the original article, you could have said, “I don’t think it’s numerical precision, Muratori.” Maybe I am ascribing overly heroic attributes to the man, but I feel like Forman Acton would have known, from looking at the plane intersection snippet, that there wasn’t any way to lose enough precision in there to account for the bug as I described it, at least not in any way that would have been substantially improved by the suggested fixes people wrote in (or the fix I actually did, for that matter).

So in order to actually get full credit, you’re going to have to earn it, I think, but on the honor system. Ask yourself, if you had encountered this bug, and you had “fixed” it as I had, would you have gone back and figured out what was really happening before moving on to something else?

If you can honestly answer yes, then you get full credit, for this is about as good as you can hope to do as a working programmer, I think. You’re no Forman Acton, of course. . . but then again, who among us is?

Up until now we have not been in that kind of work mode, because we were figuring out the game, and in that kind of mode, I find schedules to be anti-quality. But once you have everything figured out and need to produce a volume of work, schedules are pro-quality, since they help you gauge how well you are doing, whether you need a reality check on the rate or quality of things you can produce, etc.

Luis has made this handy map that we are using to partition up modeling tasks, which I have then edited to remove spoilers:

I like looking at this map because it reminds me how many distinct areas there are and how they relate to each other.

Today I was showing the game to someone who had not seen it before, and that's always interesting because it helps me see the game through fresh eyes a little. It really feels unlike any other game in the way it is an Open World but also a Small World, where you can be in one location and see several neighboring locations, and that helps keep you oriented, but it also helps you reflect on what you have done and orient on where you are in the course of things you are planning to do.

We still have a ton of work to do, though, and I was reminded of this as well. Not just in detailing all these areas, but in filling out the story, crafting details that further help the areas relate to each other, etc.

It is going to be an interesting time!

Current puzzle count: 505.

The Background

Way back in 2008 when I came upon the core idea of this game, the design was the exciting part. Though I started in games mainly as a technical person who was excited about technical challenges, I have shifted into a mode of thought where, when I program, it's mainly just to get the design done in the most straightforward way.

Since Braid I have viewed programming as mostly pragmatic: I know what I want to do, and I know what kinds of things I have to type in to make it happen, but for the most part, the interesting mental challenges are in design, and programming is rote execution. Once you have enough programming experience, I think 95% of writing a big program is like this. There have been a few times during The Witness when I spent significant effort solving technical problems where I had no idea what the answer would be in advance, like with the terrain manipulation, but even in these cases I treated the technical development in a very pragmatic way: the goal was to get something that is good enough for current needs, not to design a shining jewel of an amazing technical system (back when I was a younger game programmer, I was trying to make amazing technical systems all the time, and it came at the cost of actually getting games done).

Despite the fact that I am trying to focus on design, in order to make this game a reality, I had to start a company, bring people onto the project to build the game, etc. This means that nowadays I am not just wearing the programmer and designer and business development hats, I also wear the producer hat and the (admittedly small at present) HR hat. Maybe if The Witness does well, we can expand the team a little bit so that I don't have to do these things, but for now, that's the reality.

We are very laid back and unstructured in the way we do things, but even so, I feel there's a heavy subconscious load involving running the company, making sure nobody's work is blocked, dealing with money all the time, etc. I often find it hard to do deep design thinking when I am in the office, if only because all these issues are just ambiently around. So, in terms of design, my most productive days tend to be work-from-home days (which for me includes most weekends).

All this up til now is just to explain that if I take a vacation seriously, it can be very relaxing, because that subliminal load is gone, or at least mitigated for a while. The activity of the rest of the company is mostly paused, which removes urgency from my own activities.

At first I didn't know how I would spend vacation; at first I thought maybe I would actually treat it the way most people treat vacation, and not work. Then I thought, hmm, there is a certain sector of the game that I might like to work on, that is definitely very important, and I should get a bunch of that done while everyone is gone, so that when they come back it will be figured out. (I don't want to go into detail about what that section of the game is, because I am avoiding spoilers).

The Vacation

But, inspiration is a funny thing. I worked a little bit on the issues I had planned to work on, but for the first bit of the vacation, up through the 25th, I found myself most motivated by cleaning up old nagging issues on my to-do list, and shipping this improved version of the game to our very few beta players. This involved stuff like improving the feel of tracing lines on the puzzle panels.

A peek at the export script, used to package the game for end-users.

After shipping that beta, I found my attention most strongly drawn toward an unexpected part of the game: an optional area, one that we could completely cut from the game and nobody would know, yet at the same time, it's an area that adds to the overall conceptual arc of the game in a strong way, and would be very rewarding for the small percentage of players who get deeply into the game. (Again, I apologize for the vagueness, but let's avoid spoilers).

I had roughed out a version of this area a long time ago -- maybe a year or two ago -- and had always found the basic idea very exciting. But back then the area wasn't done enough to prove the concept, and over time it decayed while the rest of the game was being worked on (which is natural on a big project like this). But, perhaps because it's an area of the game that nobody else on the team really knows about or understands yet, I had a great motivation to spend vacation time on making this area good. Now that I think about this again it makes a strange kind of sense: when development is pretty much paused, it is somehow appropriate to work on something that nobody else really knows about.

At first I thought the task would just be to make the old stuff work again and then do a straightforward job of the extra functionality I needed to complete the area. Thinking about the programming I would have to do to complete the area, I saw it as a necessary bummer ("I want to be done with this game, but ugh, there is more programming I have to do that isn't even on the mainline to all the things that really need to be done.")

But I started programming that extra functionality, and 10 minutes into it, realized I was not so excited about the design part -- it seemed like a very rote execution, like anyone could do in any game, and The Witness ought to have something better than that. Because there was no pressure, and because I was free to do whatever I wanted, my mind was open enough to instantly see an alternative that would be much truer to the essence of the game. The chance to do this better thing was very exciting. (We'll come back to this subject of being able to see ideas, later).

So I started programming this more-exciting version of the functionality. Then I saw how I could make it even more interesting, and I would start doing that. Then I would see how to make it even more interesting, though that would introduce a lot more programming work. If I were feeling the regular pressure of the game schedule, at this time years in when it is time to be finishing the game, I might have said, "naah, we don't have time add that extra work". But because it was vacation the work was free, so I said screw it.

I worked in a fiercely productive way for a couple of days. New ideas would come. They would raise new issues. I would program some things, and in that way that programming often goes, new problems would come to light that I hadn't thought of, and those problems introduced even smaller 2nd-order problems when confronted. But by this time I was fully energized and just had dogged determination and was fully enjoying the whole process. In the end I programmed at least 5 times as much functionality as I had planned, but rather than being bummed about it, I enjoyed the whole excursion.

Those who know me will tell you that I am not predisposed toward painting things in an unduly positive light -- but I will tell you that these past four days or so have been amazing and wonderful. Here's a shot from what I did (the visuals are 'programmer art', as usual):

It is hard to tell you why this is so cool without huge spoilers.

It's so amazing and wonderful because

it's been a long time since I worked this hard and this effectively. Keeping in mind that memory is sometimes not as reliable as we think, I would tell you that the last time I worked this hard and this well was when starting my first, ultimately doomed, company (with a friend from college and a 3D artist we recruited from that same school and unfortunately put through years of meager-wage toil).

Here are some shots from the first professional game I ever worked on, back before we had a concept of 'indie development'. Software rendered, subpixel-accurate texture mapping, assembly-language inner loop that commandeers ebp in order to utilize a precious extra register, a la Mike Abrash, and oh yeah, it was multiplayer-only, 32-player, over 9600 baud modems:

We were working on this game from 1996 to 1999 or so. 1996 was the year I got into professional game development, and in hindsight, it appears to have been the hardest possible time in the history of video games. Indies of today, y'all have no idea how much easier it is to make games now than it was in 1996. (I hate to be an old fogey who's all "back in my day, we had to walk to school in the snow, barefoot, uphill both ways," but in this case it is absolutely true.)

That game was never very successful. It had design problems, but also, we didn't really know how to talk about or promote a game; lastly, this was before many people played games over the internet, so the potential audience was pretty small. (Trivia Tidbit: There was another online game being supported by the same publishing service as our game, and this other game was more popular, despite being relatively simple in a technical sense, which annoyed the hell out of us on a subliminal level that we wouldn't totally admit. The guys making this game were John Vechey and Brian Fiete; our producer at the publisher was Jason Kapalka, who later went off with these guys to start a company called PopCap, which last I heard had done okay.)

Back in those days I worked very hard. When we started the company I knew nothing about building 3D games; we learned very quickly. I would long hours, late into the night, intensively. At some point we all burned ourselves out, I think.

After we closed down that company, I went on to do a number of contractor things, game industry lecture things, magazine article things, and working on various independent projects that I never finished. Through that time, I was still very burned out. If you have never experienced burnout, it's hard to explain. Burnout is not just "I don't feel like working right now"; it is about your mind refusing to permit you to work, because it has seen what happens when it lets you work.

Eventually, I swore that I would design a very good game and finish it and release it to people, and the result is a game that was successful and that many people like:

A main part of the reason this game exists was that I instilled a certain discipline in myself: I wasn't going to do this game because I was excited about a technical thing, or excited about programming; I was going to minimize the amount of programming, firstly so that the game would get finished at all (I had plenty of experience by this point at not-finishing-projects, as did many of my friends; it was an endemic problem), but also so that I could focus appropriately on the design and build something very special.

Despite the fact that I kept the game technically simple, there was still a huge amount of programming to do. But thankfully, the strategy worked and the game was completed. Eventually.

Even though I'd made a successful game, I was still burned out to some degree. In 1996-1997 I was working at full capacity and was very productive, at least inasmuch as one might expect for a programmer of that age who was first starting to get real-world large-project experience. But after being burned out at that time, I never again worked at full capacity.

What I am getting at

I am saying all these things just as a way to get around to explaining that, during this Christmas vacation, I finally recovered fully from burnout. For about 3 days, I was working at 100% capacity, which I have not done in 15 years. It felt wonderful to have that back, to not succumb to the story that maybe game design and programming are like math of physics where you peak at 25. These few days were way better than when I was 25, because now I am substantially better at programming, and I am good at design also, and games are so much easier to make than they used to be; so the design cylinders are firing and then they turn it over to the programming cylinders, etc, and very quickly a lot of high-quality work gets banged out. And it's not just arbitrary good work, but work that helps to round out the game that I already feel is the best thing I have ever worked on, that helps put grace notes in just the right places, in the manner of exploratory game design that Marc ten Bosch and I spoke about at Indiecade 2011. So this period of intensive work was seasoned and tempered in a way that was not possible when I was 25.

It has been a wonderful time and I look forward to doing more of it. Part of what allowed it to happen this way was that lack of pressure that I have mentioned, due to being on vacation. When a design idea would come to mind, and I would see that it might be cool but would lead to a lot of work, well, this week I was able to let it happen, and I was able to enjoy the challenge of going off on a minor programming rathole in order to do something really good in the design. This is to some degree a reversal of the discipline I instilled to get Braid done; you can think of it as a correction term to the equation. The 0th order term is knowing how to program and design; the 1st order term is having discipline to be able to execute efficiently without being carried away with ideas that will scuttle production; but the 2nd order term is that it is joyful and interesting to have some wiggle room in there, sometimes eroding away a bit of the 1st order term.

There is a kind of open curiosity about programming that I haven't had in a long time, that I allowed myself to re-discover this week in the freedom of exploring that 2nd-order term.

(Richard Feynman has an anecdote about how he was burned out on physics, and how he started to get over it, which I think bears some similarity).

I am not explaining any of this very well. But I am doing the best work of my life, right now, and it feels great, and for the first time I could see myself working in this same fashion for the rest of my life and feeling happy that I am operating at something like my full potential. So there you have it.

I mentioned earlier that I would talk further about the ability to see ideas, but I guess I was lying, because this is already too long and rambling. I'll save that for a future post.

It's worth pointing out that I started Braid while working on what was supposed to be a vacation. And back when I got the impulse to start the Experimental Gameplay Workshop, I was on a plane back from a vacation to Africa. So apparently vacations are good times for me, even if I don't treat them the way Normal People do.

I do not know who made this rock or how long it has been there, but when I first happened upon it, I found that it resides in a nicely isolated part of the island where there’s nothing else in view, so the frame rate is always high even in full debug mode. Plus, it’s big, unobstructed, and easily pickable in the editor without fear of selecting any other objects. These properties make it perfect for testing editing features in progress, so the rock and I have become fast friends.

Normally, I do not want large rocks to move around. If I’m in the game engine, and a giant rock starts moving, I know I’m in for some debugging work. But if I’m in the editor, and I tell a rock to move, I expect it to move. That’s the whole point of having an editor. So when this cliffside rock, which otherwise had been so pleasant, decided that it would only move in certain directions, I knew I had a bit of a mystery on my hands, one that eventually forced my brain to remember some very important things it had forgotten in the ten years since I last did any serious 3D graphics programming.

Foreign Languages

Working in someone else’s codebase is much like moving to a foreign country that happens to use the same alphabet as you do, but not the same language: technically, you can read, but you don’t really have any idea what most things mean. At first, every little thing you do is a painstaking linguistic procedure, and you have to constantly check and recheck things to make sure you actually know what’s going on. Over time, as you gain more experience, you start to intuit the way things work, and you’re able to operate at closer to your full speed.

I’m used to the process of working in unfamiliar codebases since I’ve worked with so many of them over the years. I tend to approach each new one the same way: first, I write something very isolated, and keep it walled off. This lets me start making some contributions right away, while I gradually learn how to interface with the rest of the code. In the case of The Witness, this was Walk Monster. Next, I try to do some performance optimization or bug fixing, so I learn to work locally, but directly, with existing code. That was fixing the five-second stall. Finally, I start picking real features to add to other people’s code, preferably in the systems that I will most likely need to learn well.

Since I’m working on the collision system, and at this point I am certain there will need to be some interactive tools for it, I picked the Witness editor as the best place to start adding real features. After playing with it for a while, I selected two features I thought would be good additions from both the standpoint of learning the code, and for be generally useful for people working on the game. The first feature was compatible camera controls, and the second was a translation manipulator.

By “compatible camera controls”, I mean camera controls that work like common 3D art packages. My understanding was that the Witness team did their modeling in Maya, so I put a toggle in the editor that, when enabled, allowed you to use ALT-drag camera movements that replicate the way Maya moves the camera. After checking this in, I learned from the artists that some of them prefer 3D Studio MAX, so I added another button that emulates its camera controls (once you do one, it’s very little incremental cost to add others, as they’re really all just other sets of bindings for the same movements). All of this went rather smoothly.

Unfortunately the second feature, the translation manipulator, did not go so smoothly at all.

The Translation Manipulator

By “translation manipulator”, I mean the standard three-axis overlay that modern 3D packages have where you can click and drag on any axis and move your selection along that axis. Depending on the package, there are also little squares on each plane that allow planar movement. I chose to include these, but to allow them to be toggled separately since not all packages have them and I didn’t want them to be in the way if an artist wasn’t accustomed to using them.

Compared to the camera controls, it was moderately more work to understand all the intricacies of the editor code involved with translation. But I worked my way through it, and eventually I had a translation manipulator working reasonably well in my test world, which was a completely empty world with just a few boxy entities that I could move around to see if everything was working.

When I was confident everything was solid — undo was working, occlusion was working, etc. — I tried it out on the full Witness island for a bit. Nothing seemed immediately wrong, so I checked in the feature on a toggle switch that defaulted to “off”, and asked Jon to check it out when he got a chance. I figured it was better not to check it in enabled by default and have the artists all start using it, only to find out it had some critical bug that would corrupt things in some irreparable way, losing lots of valuable work in the process. Since Jon does a lot of work in the editor, and he wrote a lot of the editor code originally, I figured he would know the things to try that might break so we could fix them before any real damage was done.

When I heard back from Jon, he said it did seem to be working correctly, but sometimes “moving along the Z axis wasn’t working”.

The Occasionally Movable Rock

I was not at all surprised to hear that Z-axis translation was unreliable. This was something I was expecting to have happen. Surely something was broken in the way my code interacted with the existing planar movement constraints, which were buttons the editor had always provided and which were the only way of constraining interactive movement before I added the translation manipulator. But I checked the code thoroughly, and I just didn’t see any way that I could be mishandling the Z axis in the intermittent way Jon had reported.

I tried playing around with movement more in the editor, to see if I could reproduce the problem myself. Soon, I started to notice that when I played with the solitary rock in the ocean, it wasn’t really “sticking” to the cursor as well as it should be; the code I had written was supposed to feel as if you had “grabbed” the manipulator, and it should follow your cursor rather exactly. It wasn’t doing that.

Then I noticed something even stranger. Depending on how I rotated the camera, I was able to find angles at which Y-axis dragging moved the rock extremely slowly even for large cursor movements. At one particular camera angle, the rock refused to move along the Y axis at all! For example, at this angle, I could move the rock along the Y (green) axis, albeit slowly:

and at this angle, I could not move the rock along the Y axis at all:

even though the other two axes worked fine, and from other camera angles the Y axis worked fine, too. Since the manipulator supported entity-relative axial movement as well as world-relative axial movement, I tried switching to entity-relative axes to test if there was something wrong with the Y axis specifically. I rotated the rock such that the Z (blue) axis was pointing in the direction that the Y axis pointed previously:

To my surprise, it now refused to move along Z! So it really had nothing to do with which semantic axis you were moving, it seemed to be the direction in the world and the angle of the camera alone that mattered for reproducing the bug.

Looking for more ideas about what the bug could be, I tried loading the empty test world again and playing around with the manipulator there. Again, to my surprise, it was completely impossible to reproduce the bug. No matter what camera angle I used, or where the axes were pointing, the always dragged perfectly, and seemingly at the correct speed to boot.

What on earth was going on? Clearly, it was time to think more deeply about the code and the behavior it was exhibiting.

Ray Versus Plane

In most 3D editing operations, you must confront the problem of turning a 2D cursor position into something useful in three dimensions. Since a 2D point on the screen mathematically represents a ray in three dimensions (everything in the world that is “under” that point from the perspective of the user), this typically involves taking the ray and intersecting it with something. For example, to implement picking of objects in 3D, you typically intersect this 3D ray with the objects in the world, and you see which intersection is closest. That will be the closest object under the cursor when the user looks at the screen, and therefore the one that they most often expect to pick when they press the mouse button.

My translation manipulator also works by intersecting the ray implied by the cursor. Knowing which axis the user clicks on is analogous to the picking problem, and I essentially implemented it that way: I pretend that there are three long, skinny boxes that align with the axis arrows, and I test to see if the cursor ray intersects them.

Picking the axis is straightforward, but things get more complicated once dragging begins. Although I know which axis the user started dragging, in order to actually move things properly, I have to have a way of computing exactly how far they have dragged the object along the selected axis. Intuitively you might think well, why not just extend the box used to pick the axis to be infinitely long along the dragging axis, and continue intersecting with it as the user moves the cursor? That’s not a bad idea, and it would work, but it isn’t the best solution.

The reason is because it doesn’t feel very good to use. Although it will feel fine if the user drags right along the axis in question, a lot of times users don’t do that. They tend to drag erratically, and pay more attention to the object’s position than how closely they follow the line implied by the axis. This quickly moves the cursor off the imaginary box entirely, so the ray implied by the cursor won’t intersect it. This causes the manipulator to stop working until the user moves the cursor back onto the axis, resulting in a sticky and unpleasant user interaction.

What we’d rather have happen in this case is for the dragging to happen as if the user had moved the mouse to the point on the axis closest to where they actually dragged, even when they’ve wandered quite far from the axis itself. To see how to solve this problem geometrically, imagine a user dragging the mouse along the X axis of a 2D plane. If the user starts dragging erratically, and moves along Y as well, their cursor will end up out in the middle of the plane somewhere. Figuring out where they would have been on the X axis is quite simple, though: ignore the Y coordinate, and just use the X coordinate. Nothing could be simpler.

But how do we do this operation for an arbitrary 3D axis and a ray extending from the cursor? It turns out you can do it exactly the same way. All you have to do is pick a 3D plane that happens to contain the dragging axis, then intersect the cursor ray with that plane. That gives you a nice 2D coordinate on a 2D plane, where the dragging axis is analogous to the X axis in the simple example. You then “throw out the Y”, and are left with a drag that always feels nice in 3D, sticks the object tightly to the cursor, and does not require the user to drag close to the axis.

You don’t need any fancy math to do this. It’s entirely constructible with simple operations available in any math library: dot product, cross product, and ray-intersects-plane, all three of which were already available in The Witness codebase.

So my translation manipulator was built entirely out of the most basic 3D operations in any math library. Yet somehow it was exhibiting very odd behavior atypical for a simple system. How could this be? What was I missing?

Debug Rendering

My first thought was that I must be picking the dragging plane poorly. Constructing planes in 3D with only partial information is always a little dicey; you need three non-collinear points to define a plane, but often you want to construct one with only one or two points. The translation manipulator is a perfect example: it needs a plane to operate, but it only really has a line to work with, which is equivalent to only two points. There are, in fact, an infinite number of planes that pass through the dragging axis of the translation manipulator. It’s much like a waterwheel: the dragging axis is the center axle, and each blade of the wheel is another valid choice for a plane.

So how do you pick one? The best method I’ve found for translation manipulators is to use the location of the user. Since we want the user to be able to drag around on this plane easily, it makes sense to say that, of all the planes containing the dragging axis, the best one is the one that is most visible to them. This avoids the potentially bad cases you can hit if you pick something independent of the user’s location, such as the resulting plane being edge-on or nearly edge-on so that the user cannot actually drag along it effectively.

Since even simple 3D math like this can be difficult to verify by inspection alone, my first inclination was to add some debugging rendering to verify that I was picking the dragging plane properly. Since the camera angle clearly mattered, and the plane was one of the only things dependent on that angle, it seemed like a likely culprit. So I tried drawing the normal to the plane in yellow at the base of the manipulator:

Even in the cases where the dragging failed, the plane normal looked right. Maybe the positioning of the plane was wrong? I tried computing the location of the plane directly, by moving along the normal by the plane constant d and drawing the normal there. This, of course, put the normal nowhere near the manipulator, so I had to look around for it (the plane equation gives you the point closest to the origin, which has no real relationship to where the manipulator might be, other than that they both lie on the same infinite plane). But once I found it, even though it was far away, looking back toward the manipulator it still did seem like it was the right plane.

Next I moved onto the intersections. I tried drawing the intersection point that I had computed when picking the axis in the first place, and also the point that I used to compute the plane constant in the first place. Both looked correct.

But then I tried drawing the actual intersection computed at each drag update, and that didn’t look correct. In fact, it didn’t really “look” at all. Although I was specifically telling the renderer to draw a white sphere where I’d found the intersection of the cursor line with the plane, no white sphere appeared. I checked the code again, and it looked correct, so I tried doing a few more test drags, this time from angles that had seemed to be working before. Now, the sphere showed up, although it was bigger than I thought it should be, which was odd.

I tried rotating the camera. I wrote the code so as to leave the sphere rendering wherever the last dragging intersection occurred, so if I stopped dragging, it just stayed where it was. This allowed me to rotate the camera and see precisely where the sphere was located, and as I did, I realized that the reason the sphere appeared too large was because it wasn’t even close to being on the plane I thought I was using. It was much closer to the camera. Weirder still was that, at the buggier camera angles, the sphere was jumping around erratically, and wasn’t really sticking to the cursor, which it always should do no matter what plane was being used. I was, after all, intersecting the cursor line with a plane. How could the intersection point possibly lie anywhere but directly under the cursor, regardless of the parameters of the plane?

It was at that moment that my math brain finally woke up. It had been sleeping for about a decade, which is how long it’d been since I’d done any real mathematical programming. In an instant, I immediately knew both what the problem was and how to fix it.

Always Look Behind You

The problem was embarrassingly obvious when you consider the clues I’d been generously given and yet completely ignored. Remember when I said I drew the plane normal at the location indicated by the plane constant, and I had to look “far away” at the manipulator to see if it looked like it was in line with the plane? Well if the generally erratic behavior of the manipulator hadn’t been telling enough, that little episode should have been the dead giveaway.

At this point, every serious 3D programmer who hasn’t been doing other work for ten years surely knows what the bug was. But for the sake of people who are less experienced, I decided to end this article with a bit of a cliffhanger.

I am going to tell you that you can find the bug by looking at this simplified version of the ray_vs_plane function in The Witness that I was calling (the real version does some epsilon checking in case the ray and plane are parallel, but otherwise it is identical):

Vector3 ray_vs_plane(Vector3 ray_origin, Vector3 ray_direction,
                     Vector3 plane_normal, float plane_d)
{
    float denominator = dot_product(plane_normal, ray_direction);
    float numerator = dot_product(plane_normal, ray_origin);
    float t = -((plane_d + numerator) / denominator);
    return ray_origin + t*ray_direction;
}

And I will also show you this screenshot of what it looks like if you turn around 180 degrees from the rock and look back at the island itself:

But, until next time, I’m not going to tell you what the bug actually is. Can you find it, and figure out how to fix it, before then?

Granted, it is not uniformly horrific. It ranges from only slightly scarey (xinput, comdlg32) to full psycho (DirectShow, Event Tracing, TabletPC, etc.). But no matter which part of the spectrum you encounter, programming on Windows tends to be about doing a lot of unnecessary research, experimentation, and debugging due to a confluence of poorly designed APIs.

Since time spent debugging Windows issues isn’t often chronicled, I thought I’d go ahead and describe my experience this Saturday of tracking down a Windows bug in The Witness.

The Symptom

Apparently, not everyone on The Witness team was seeing the symptom. But trust me, it was there.

On both Jon and my machines, there was a period during the initial loading of the game where you couldn’t do things like ALT-TAB back to the debugger. If you set a breakpoint then ran the game, when the breakpoint eventually hit and the debugger came to the front, it would wait about five seconds before it allowed you to actually interact with it. Sometimes, in rare cases, it would go into some sort of psycho mode whereby mouse movements stuttered, and the computer would literally beep at you as the mouse moved, for several seconds. Then everything would return to normal.

While not technically a showstopper, this is the kind of bug that’s worth fixing if you can, because it costs you real development time every day. Five seconds wasted every breakpoint is bad news, and people often underestimate the effects of frustration that can build up over time due to flaky and unpleasant development environments. So I decided to try to track it down.

Because Windows is a closed-source platform, there’s no great way to track down problems of this nature. However, having programmed on Windows for over twenty years now, I have a nose for it, so I felt like I knew the best place to start: since there was definitely a UI blackout somewhere during startup, there had to be something occurring early in the execution of the program that was causing at least that particular symptom. So before doing anything else, I started stepping through the entire startup sequence of the program to see if anything out of the ordinary struck me.

I very quickly came upon this code:

// @@ The hook appears to install properly, but is never invoked :(
keyboard_hook = SetWindowsHookEx(WH_KEYBOARD_LL, LowLevelKeyboardProc,
                                 GetModuleHandle(NULL), 0);

Now, I don’t want to make it sound like the “find the source of the problem” phase of Windows debugging is straightforward. It usually takes a very long time, sometimes multiple days. But as luck would have it, this was literally the first piece of code I suspected, and it turned out to be the culprit.

I commented out the SetWindowsHookEx, ran the program, and poof! All the symptoms were gone.

The Problem

From experience, I’ve learned that it’s always best to fully understand a problem before you fix it. If you just patch over its symptoms but never figure out what the problem really was, it will often come back to haunt you.

In this context, that meant two things: first, I should figure out specifically why the keyboard hook was causing the UI stalls; second, I should determine why the code had been trying to call SetWindowsHookEx in the first place, even though apparently it never quite worked (according to the “is never invoked” comment).

The first part turns out to be obvious in hindsight, but it took me a little while to put all the pieces together and be sure I had a solid explanation for what we were seeing. As with any puzzle, looking at the finished picture is easy, but figuring out what that picture might be when all you have are little pieces coming in one at a time is much more difficult. Here’s the best picture I could come up with, but of course there’s no way to completely verify it external to Microsoft:

Normal Windows hooks are pieces of code which execute in the address space of the eventual message recipient, not the installer of the hook. It is for this reason that, normally, if you want to install a global hook, you must put the hook code in a DLL, because that DLL must be mapped into the address space of every running executable on the machine that can receive Windows messages.

But low-level hooks are completely different. A low-level hook like WH_KEYBOARD_LL is a piece of code that resides in only the process that registers the hook. Windows remembers which thread registered the hook, and when any other executable is about to have a hooked message posted to its message queue, Windows actually waits until it can switch contexts back to the hook thread, run the hook in that context, and then finally deliver the message.

This leads directly to the symptom that we were experiencing. When the game is loading, its main thread isn’t yielding to Windows frequently like it is during normal play. Since the main thread is the one that registered the hook, Windows must wait for that thread to finish all the load-time work it was doing and call a function which yields to Windows in order for Windows to actually call the hook procedure. Since no keyboard processing can occur until the hook procedure has run, the keyboard becomes unresponsive as messages back up waiting to be processed through the hook. This completely explains the UI problems during startup, and fits with the oddity that the mouse never seemed to be affected, just the keyboard.

When the game hit a breakpoint, the keyboard hook couldn’t execute at all because the debugger had halted the thread required to execute it. This would have meant that keyboard input would be completely halted, and the debugger would never be able to receive keystrokes, if it weren’t for the fact that Windows actually uses a timeout on hook calls. If the timeout value is exceeded waiting for the hook thread to become available, the hooking executable is assumed to have crashed, and it will silently remove that hook and continue with normal operation.

That timeout value? Five seconds — right in line with our observed pause (technically, it’s whatever is set in the registry under HKEY_LOCAL_MACHINE\Control Panel\Desktop\LowLevelHooksTimeout, but the default is five seconds).

So, no mystery as to the pathology. But what about the second part? What was the code actually trying to do?

The Windows Logo Keys

Although SetWindowsHookEx might seem like an odd call to make in a game executable, it’s actually quite common. It’s there to prevent the Windows logo keys on modern keyboards from ruining full-screen games. Because the keys are placed in a location that’s easy to hit by mistake, many gamers would accidentally hit them, causing their games to be instantly deactivated by Windows in favor of bringing up the start menu. So widely acknowledged was this problem that, when these keys were first introduced, Microsoft itself published a recommended work-around. From Disabling Shortcut Keys in Games:

This article describes how to temporarily disable keyboard shortcuts in Microsoft Windows to prevent disruption of game play for full screen games... Use a low-level keyboard hook to filter out the Windows key from being processed.

The article goes on to show sample code that is basically the same code that was being used in The Witness. It’s a simple SetWindowsHookEx call with WH_KEYBOARD_LL whose hook routine does nothing but block VK_LWIN and VK_RWIN keys from being processed. The only difference between the code in The Witness and the code in the sample is that The Witness would conscientiously remove the hook when it no longer had focus, and then reinstall the hook when focus was regained. The sample code, by contrast, just set a global variable.

So if this was what the code was doing, why the frowny face? If you look back at the comment, it states very clearly that the hook was never being invoked, even though no error message came back from Windows. And, although I normally remap my Windows keys via the registry, I went ahead and mapped them back to see if The Witness was properly blocking their effects.

Much as the frowny face foretold, it wasn’t.

Obeying the Static Discipline

I don’t know anything about digital circuit design, but I once watched a lecture by Gill Pratt where he explained something called “the static discipline”. In essence, it is the requirement that any component in a circuit must accept voltages with a certain range for “1” and “0”, and then must output its own results in a certain range. These ranges are defined so as to require that components always produce equal or better conditioned voltages than their inputs, thus ensuring that the digital signals being propagated through the circuit don’t degrade into noise due to lots of tiny losses.

For some reason, I took this concept to heart in a programming sense and have found it is a good rule to code by. My version of the static discipline, adapted for software, is that whenever you are making a modification to a piece of code, you should always leave it in a state of stability equal to or better than how you found it. And preferably the latter.

All too often, people go in to fix a problem or add a feature to a piece of code, and they just hammer on it until it does that one new thing. The resulting code is then usually more fragile, less well designed, more unnecessarily complex, etc. To try to prevent myself from having this effect, I try to observe the static discipline.

And so it was with this Windows key situation. I could see what the code was supposed to be doing, and although I could reasonably fix my bug by just commenting out the hook (because it didn’t work), I felt it was the more disciplined thing to do to figure out why the hook wasn’t working, and to implement something that did what it was trying to do. Since there was no obvious bug, I had to start experimenting.

After adding some debug outputs and running Spy to keep an eye on the window messages, the first thing I tried was removing the part of the code that unregistered the hook when his window lost focus. Since I didn’t know exactly how his windowing code worked at all levels, I figured it wasn’t entirely out of the question that it could be getting disabled prematurely.

Although that turned out not to be the case, this experiment did pay off unexpectedly (as many do): although the keyboard hook still never got called when The Witness window had focus, the hook did start getting called when it lost focus. Yes, oddly enough, the hook was working just fine for everyone else’s window, but on the one window where it actually needed to work, it wasn’t working.

This lead to the obvious question, what was The Witness doing differently?

Raw Input

This was a case where having lots of Windows programming experience probably saved a lot of time. Instead of having to experiment blindly, I knew right off the bat that Windows Raw Input probably had something to do with the situation. I’ve debugged lots of Windows front end code in the past, and any time Raw Input, DirectInput, XInput, or anything else with the word “input” in it is involved, you know that things are going to be going a little haywire.

I commented out the device registration call for Raw Input, and lo and behold, the hook started working for The Witness window, no problem. So, although I was completely unable to find any documentation on MSDN that discussed what was going on under the hood, it must be the case that Raw Input processing happens before low-level keyboard hook processing in such as way as to prevent the keyboard hook processing from ever seeing keyboard messages on a Raw Input-registered window. Normally, this wouldn’t be a problem, but in our case, the Microsoft-recommended fix for the Windows logo key problem relies on keyboard hooks. The Witness needs Raw Input to get good low-level relative mouse movements, but it also needs to disable the Windows logo keys; sadly, this is the kind of situation that is all too routine for anyone who programs Windows professionally.

I couldn’t think of any plausible ways that I could convince Raw Input to start using the Windows keyboard hook, since I don’t have access to any of the Windows source code and I have no idea how it works internally. So I focused my efforts on Raw Input.

I experimented with just about everything I could think of. I tried not calling DefWindowProc on WM_INPUT calls, to see if it was the default window processing that was using the Windows logo key events to switch to the start menu. No such luck. I tried using RIDEV_NOLEGACY during Raw Input initialization to see if preventing keyboard messages from being generated would solve the problem. It didn’t. After many failed attempts, and finding no relevant information on the web, I decided to call it a night and sleep on it.

When I woke the next morning, for some reason I remembered the most important thing anyone programming Windows must remember: never, ever trust the documentation. It’s always either inaccurate or incomplete. So when I sat down at my keyboard, the first thing I did was try using the RIDEV_NOHOTKEYS flag when initializing Raw Input.

You see, the previous day I hadn’t bothered to try this flag, because it explicitly says in the documentation that it does not block system-level hotkeys, only application-level hotkeys (like the kind you create yourself with RegisterHotKey). But why was I believing that? That was just something somebody wrote down when they needed to create a documentation page. They probably never even saw the actual code for Raw Input in the first place.

Surprise surprise, RIDEV_NOHOTKEYS worked. After all the fussing, all that had to be done to make The Witness impervious to Windows Logo keys was one little flag during Raw Input initialization. No keyboard hook necessary.

Zero-Sum Game

It’s worth taking a step back for a moment and asking the question, why did all of this happen? Why did I have to spend several hours working on code that has nothing to do with my project, is not valuable in any intrinsic way, and whose specifics are relevant only in a fleeting temporal sense (there will be no need to know anything about RIDEV_NOHOTKEYS when the next user input model for Windows is invariably introduced)?

The answer is the same on Windows as it is on most every other platform that exists today: the platform providers just don’t spend real resources designing and improving their APIs. If there had simply been an API call, WindowIgnoresWindowsLogoKeys(HWND), then everyone would always just call that, and the platform developers and Microsoft could field it and do the right thing, no matter how many times they revised their platform and its infrastructure.

Better yet, way back when they first started catering to games with Windows 95, they could have introduce a call like BeginFullScreenGame(), and that could have done literally everything for you: initialize the window, set up the input stream, prepare the sound device, etc. Then they wouldn’t have had to ever worry about the Windows logo keys at all, because they would have been automatically disabled in any application that initialized itself with that call.

But instead of any of these things, Microsoft chose to ship a new feature (the Windows logo hotkey) without ever thinking about the consequences. Then they patched over its problems by telling developers to install a keyboard hook (SetWindowsHookEx), something that has nothing to do with the actual problem but which was the most expedient way for them to get the problem solved without doing any real work on their platform. Then they broke that method, which they’d previously advocated, when they introduce Raw Input, so (presumably) they introduced a new way to do the same thing (RIDEV_NOHOTKEYS), although they failed to document it. Then they didn’t even bother to publish an obvious tech article explaining this change, or even revise their old documents saying that SetWindowsHookEx was the preferred solution.

As broken as all that sounds, it is absolutely par for the course. On every project, in every discipline, developers just like me spend hundreds of hours doing nothing but diagnosing and repairing problems with the platforms on which we ship our software. It may sound like it’s just “part of the job”, but the pernicious thing about it is that development resources are always finite. External to the platform, development is always a zero-sum game. The hours I spent tracking down this bug were hours that won’t get spent making parts of The Witness better, and there’s nothing me or the team can do about that. If we want to ship a polished project, it’s a sacrifice we have to make.

Right now, we save the thumbnail and the game data as separate files, but the names are mostly the same; so if you go into your game data folder in Windows you can see them living there very plainly:

(It might be interesting to do it like Spore and embed the save data in the image file, so that there's only one file per save. Maybe we'll do this, though it's not a no-brainer; right now the image file is usually bigger than the save data, so this would involve tying a relatively small amount of crucial data to a relatively large amount of cosmetic data. Maybe that doesn't matter, though.)

As I mentioned in the previous post, we are starting to get the game out to a limited number of players. It's been a long time since I've had anyone new play through the game as a whole, so I really have no idea how it plays now, and it's important to find out. However, it's not so easy, because if you sit behind someone and watch them play a game, they are going to play it differently than if you weren't there. You invalidate your own playtest data.

So, I wanted to set up the game to record some minimal playtest data, that would be easy for me to browse through later on, and that would get recorded without me being there to directly watch the play session.

One of the first approaches that programmers often consider is an input recording method. You just journal a stream of all the player's inputs, and you can play that back deterministically to reproduce the entire game session. I have never liked this method. Firstly, you need to play back using exactly the same version of the game executable that the player used, which is a pain; secondly, this kind of playback code is brittle and breaks often. An uninitialized memory read bug can render your playback data useless! (Of course, one reason you might want playback data is to track down an uninitialized memory read bug... so... there's a problem here.)

For these reasons I didn't want to use input journaling. It seemed to me that our saved games are small enough, and the thumbnails are nice enough, that together they almost solve the problem. All I need to do is save a series of snapshots / game states over time, without overwriting previous saves:

Whereas the earlier folder contained a collection of different play sessions, this one contains one play session at different moments in time. Every time the game made an autosave for this play session (overwriting the old one in the normal way), it also also copied the autosave here. The date and time that are in each filename just indicate when that game was started; the number at the end tells you which snapshot this is.

There's one immediate, very nice property of this: I don't even need to run the game in order to get a general idea of how someone's playtest went. Once they send me the files, I can just click on the first one in Windows to invoke the image viewer, and then click the forward/backward buttons in the image viewer to animate my way through the game.

But, if I have questions about what was going on, I can also dig into the save file that corresponds to any given image. I made a mode that lets me browse these saves in-game, load up any particular one into the current world state, and play from there. I invoke it by running a console command:

This loads all the saved games and thumbnails and pops them into a scrolling list at the top of the screen, in chronological order:

I just scroll to the one I want:

... then press Enter to load that game state:

It's very fast to use and it only took a few hours to program.

This whole system is also pretty cool because it's much more robust than the input-recording method. If I want to look into some detail that is super-nitpicky, I may need to use the same build of the game executable that the player used, but this will almost never be necessary. Usually I won't even need to use the in-game browser -- as I mentioned, I can just click through the thumbnails in Windows to get a basic sense of what's going on.

The fact that we only have discrete snapshots is sort of a side-bonus: because we only snapshot every minute or so, the data is not so creepy and spy-like; players can relax while they are playtesting and know that we won't later be obsessing over their every move.

Today we shipped an update, still to the same limited group. We have some initial tech issues to work out before we go wider (game packaging issues, graphics card incompatibilities, and the like).

Here's a desktop screenshot from one of the testers, with someone chiding him about what he's playing:

(The Witness window in this shot is artificially resized to make room on the desktop; by default we launch at 1280x720.)

I know that the impulse is to think "oh wow they are almost done!" Don't think that, it's not the case. We still have a gigantic amount of work to do and it'll be a while before the game is done.

Random Trivia Q&A

Q: How many footstep sound effects are in The Witness?

A: 1,119 so far. They sound really good! We will probably be in the Guinness Book of World Records as the game with the most footstep sounds...

Current puzzle count: 528.

One of my primary design goals, all through development, has been to keep the island spatially small, so that there isn't a lot of wandering through empty or pointless spaces. The idea is to take the best parts of an open-world game, involving choice, exploration and discovery, but compressing these into a maximally dense experience.

That has been going well, but there's an interesting phenomenon where, as we add more detail to the world, it starts to feel a lot bigger. When something is a mostly-featureless plain it is easy to feel like it's small, but once you are passing tiny rocks and plants that are set up very carefully, your brain has more scale reference. So there have been a couple of times during development where we stopped and said "Hey wait a minute, things are just too big."

Last week that happened again; we were looking at two different parts of the island on the same day, and both of them seemed much bigger than we needed. So, we decided to go back to the overall island plan and try to compress it further. We had a discussion about this on Thursday, then the architects talked it over; you see the result of those discussions in the image above. These maps are a handy guide to helping us figure out what we can cut and how we can scoot existing areas around to pack them a little better, and to better-define the areas between them.

This is a little painful, since we almost had the island back into playable shape, and this knocks it back out of playable shape for a little longer. But my hope is that we can just make a mess and then clean it up very quickly.

In other news, there had been a puzzle that I needed to design for two years -- in the sense that I knew I needed to build the puzzle in order for a certain area to be complete, and I knew what that puzzle vaguely had to be about, but didn't have any good ideas on how to build it. Last week I had a breakthrough on that and built it very quickly; it's now one of the best puzzles in the game. So things are coming along well!

The colors are all funny because we haven't written the shaders for the iPad yet, so what you are seeing is the surface normals of the objects in the scene.

We haven't written the shaders yet because they need to be OpenGL ES shaders, and the shaders we have right now are HLSL shaders all written for medium-to-high-end PCs. As the first part of the porting work, Andy made the engine work on OpenGL on the PC, and for that we converted our HLSL shaders to GLSL automatically. But the jump to ES on an embedded device is big enough that it doesn't make sense to try and use the same shaders. Instead we'll have to have some simpler shaders that we substitute.

As you can see from my previous post, I have been working on touch controls (not on the iPad, but on a touchscreen Windows PC). A very rough first cut of that is now working. It's nice to play the game with touch controls, as it feels more immediate than mouse and keyboard (and I have been a first-person shooter player forever!)

So in order to have the game fully interactive on the iPad, we just need to write the bit that translates iPad touch controls to feed them into the rest of the gameplay code, which should not take long.

Then, after that, we have to work out exactly how we are going to crunch all the graphical assets down to portable versions and do the streaming in an appropriate way... which is going to be a bit of work. Salvador built a streaming asset system a while back, and we have been using that daily on the PC, but we haven't tried to set it up on a lighter-weight portable system like the iPad yet. We'll see how it goes!

I am not sure that many people are going to use Windows for touch devices, at least in the Windows 8 timeframe. But at least some people will, so it'd be nice to have a high-quality touch interface built into the game on Windows.

With all this in mind, I started working on some Windows 8 touch hardware a couple of days ago. There is a message format that you can use in Windows 8 to get events from the touch hardware, WM_POINTER, but because I want the touch interface to work in Windows 7 as well, I wanted to use WM_TOUCH messages, which were introduced in Windows 7 and are supported in both operating systems.

It look me less than 20 minutes to get WM_POINTER working. It took many, many hours to get WM_TOUCH working, because WM_TOUCH is some kind of fully insane mess. None of the insanity seems to be documented, or discussed on the internet, so I am laying it out here in case anyone in the future stumbles into this same trap.

A WM_TOUCH message will never be returned by PeekMessage. Spy++ will tell you that WM_TOUCH is being sent to your application, but before the message is delivered, it gets stealthily converted into a mouse motion event. The type of this event will vary depending on whether you have raw input selected: if raw input, it will be WM_INPUT (0x00ff); if not, it will be WM_MOUSEMOTION (0x0200).

If your application handles these events, you may think it's reasonable to complete the handling without calling DefWindowProc. If you do this, you will never see a WM_TOUCH, because something buried down in DefWindowProc stealthily dispatches a WM_TOUCH to your application's windows when it sees a specially-marked WM_INPUT or WM_MOUSEMOVE.

Note that actually, you are supposed to always call DefWindowProc on WM_INPUT (for "cleanup" -- I don't know what this means and why they can't just clean it up when you get the next message -- but whatever); but WM_MOUSEMOVE has no such requirement, and if you believe you handled the event, it is natural *not* to call DefWindowProc. This will result in your never seeing WM_TOUCH. Strictly speaking our game had a bug, in that we were not calling DefWindowProc after WM_INPUT, but the bug was hard to find because it persisted when I tried turning off raw input, because we also were not calling DefWindowProc after WM_MOUSEMOVE (which as I mentioned is, in theory, totally fine.)

Important to note: WM_TOUCH gets dispatched to your windows without you dispatching it! You may think that you can log all messages after they come out of PeekMessage, and that you are in control of your entire message stream, but this is apparently not true. WM_TOUCH will just teleport itself into your window.

In addition to the mouse motion events, your application will get a bunch of other mouse events that are generated by the touch interface, to emulate mouse stuff for programs that don't understand touch. There is no way to tell Windows you don't want these events (unless you do Windows 8 stuff and respond to WM_POINTER; see below). According to Microsoft, the official way you are supposed to detect and ignore these messages is like this:

#define MOUSEEVENTF_FROMTOUCH 0xFF515700
if ((GetMessageExtraInfo() & MOUSEEVENTF_FROMTOUCH) 
        == MOUSEEVENTF_FROMTOUCH) { 
     // Skip message
}

I wish I were joking. It's been this way for years; Casey wrote about this in 2008.

This mask works for events like WM_LBUTTONDOWN, but it does not work for the mouse motion events e.g. WM_INPUT. If you want to take input both from touch and the mouse, you need to disambiguate the real WM_INPUTs from the fake ones. Through a process of experimentation with message flags, I landed on this:


if ((GetMessageExtraInfo() & 0x82) == 0x82) { /* ignore event */ }


It seems to work for now, but who knows if that will mysteriously break in some future version of windows.

Nicely, if your application handles WM_POINTER messages, you will not get any of this mouse emulation stuff or WM_TOUCH, so all this ugliness disappears. If it is okay to make Windows 8 your minimum specification, I recommend using WM_POINTER and staying far away from WM_TOUCH.

I ended up running a straightforward test where we would change the game minimally and build consecutive patches. The patches turned out to be quite large, but upon investigation, it was clear there usually wasn't a good reason for this. Our compiled shaders would contain small differences, as would many meshes in the game.

These looked like uninitialized memory errors and, upon further investigation, that's indeed what they were. They are fixed now, so patches are a lot smaller.

However, patches may never be as small as we would like, and I find there is a pretty deep philosophical issue where.

Right now we have 17MB of files that change in response to minimal changes in the game (say, moving a few entities in the world by small distances). These are mostly cube maps for Light Probe entities, which are used to do lighting on nearby objects. Here's what a Light Probe looks like in the game:

(Well, you don't see that sphere the game, only in the editor!) To generate one of these cube maps, we render the scene in all directions, from the point at the center of that sphere, then apply a blur to the result.

Imagine that we move a couple of entities in the world, even slightly. All Light Probes that can see those entities are going to change, by at least a pixel or two. Because we are applying a big fat blur kernel, a 1-2 pixel change becomes small changes spread across the entire image.

We have an automated script that preps the game for release, clearing out data that we only use for development, and ensuring that automatically-computed things get recomputed (like LODs for faraway objects). Recently we added a step to this process that re-generates all the Light Probes, because it just makes sense to update those automatically. If we have to keep track of them manually somehow we are just going to make mistakes.

What this means is that you can pretty much guarantee that a lot of light probes are going to change if you move some entities and patch the game. If we change anything basic about any of the shaders, which seems very likely, then all of the light probes are going to change (because the output of the shaders feeds back into the light probes).

That said, maybe it's the case now that more light probes are changing than I would expect. I am wondering if other things in the game are causing them to change. For example, when you are playing the game, foliage waves in the wind. I wonder if the animation timer for plants is ticking during the process when we generate light probes, causing them to move over time? This would cause most light probes to be different every time we bake them, regardless of other factors.

Now, there's a bigger issue. In addition to light probes, we have pre-baked lightmaps all over all geometry in the game:

In a way that is reminiscent of the light probes (but a bit different), we generate these by placing a camera at many points along each surface, rendering the scene from each camera, and interpolating those into a final result.

So, just like light probes change when entities move slightly, lightmaps are going to change also. Lightmaps will also change if shaders change, but furthermore, lightmaps will change if light probes change (because light probes determine the look of entities that shine on the surfaces we are lightmapping), and light probes will change if lightmaps change (because the lightmaps influence the objects rendered in the light probes). So we have a circular dependency, which is not so bad in practice, except it means that if you change one entity in the game, then repeat this process iteratively, you may well end up changing most lightmaps and light probes in the game.

The light probes are not so big, but last I checked there are between 500MB and 700MB of lightmaps. However, re-baking the lightmaps is not part of the automated build process; we have to trigger a bake manually. But it seems like good policy to do a full rebake any time we change the game significantly. So, we are probably looking at a minimum patch size in the 500MB range.

I guess that is not such a big deal, since we are not an online game that would be patching all the time. Hopefully once we release the initial version of the game, the "content" won't change very much (sorry, but the word "content" as used in video games is kind of absurd, so it's hard for me to use it without sarcasm quotes; it's almost as bad as "title", which is a word I refuse to use, except to make fun of its use)... the "content" won't change very much, so patches would be rare.

We'll see. We have a while to figure these things out.

From: Ignacio Castaño
Date: 24 August 2012

I just checked in a few changes to the water shader. Note that this is still work in progress, there are some artifacts and unfinished features.

From the art point of view, the main change is that there's now a water_color_map and a water_flow_map that you can use to change the behavior of the shader.

The RGB channels of the color map control the water extinction color and the alpha controls the opacity factor. The previous underwater shading model was simply wrong, the new one works nicely and is easy to tweak. Note that I assume the water color changes very smoothly, avoid sharp color changes!

The flow_map is intended to contain additional water attributes. Right now it's only used for the stillness, but I will certainly add more in the future.

You will also notice that the shoreline now also has a foam layer:

The way it's computed is by comparing the screenspace depth difference. This is not very correct, it looks OK from some angles, but it breaks badly when the underwater geometry has screen-space depth discontinuities:

If the foam is a desirable feature, I'm considering modifying the print_map command to render a foam map, or maybe this is something that can be painted manually and added to the water_flow_map. One of my concerns is that this map may not have enough resolution.

Just Cause 2 seems to use this method to render foam, but they apply a big screenspace distortion to hide the artifacts, I found it sickening when you stare at it closely...

The way the water is shaded has also changed somewhat. First, I now approximate the specular exponent based on the position of the camera with respect to the water and the water roughness. Somewhat like the LEAN paper does, but assuming the roughness of the water is the same for the whole texture. Depending on the value of the exponent I either sample the mirrored reflection map, a filtered environment map, or blend between the two. My goal was to be able to use the mirrored reflection map only on a small subset of the lake entities. The physically based approach does not allow us to cull many, so I'll probably have to introduce some hacks. BTW, I'm doing all these work not just an optimization, but also so that we can use the light probe reflection model in the river and lakes inside the island, where the mirrored reflection map is not available.

The main problem with the filtered environment map is that it does not take local reflections and occlusion from the environment into account. I have two ideas to work around this, one is to store an ambient occlusion map in the lightmaps, this may be a bit too smooth and will lack directional information. The other is to do screenspace raycasting of the depth buffer to decide whether the environment map is occluded or not.

This is somewhat like what Crysis 2 does. Insomniac also uses a similar approach with just a single sample:

http://www.insomniacgames.com/tech/articles/0409/files/water.pdf

I'm hoping we can get away with a crude approximation too. I did a quick test, where the maths are not exactly right, with only 4 samples and no smoothing, and it was starting to look promising:

This won't allow us to use the filtered environment maps closer to the camera, but if the screenspace raycasting works well, I'm thinking about using non-filtered environment maps in place of the mirrored reflection map, whenever those are not available.

Finally, I've also been playing with Valve's water flow model, I have the basic implementation working, but I haven't yet been able to reproduce the workaround for the pulsing and repetition artifacts that Alex proposes.

http://www.valvesoftware.com/publications/2010/siggraph2010_vlachos_waterflow.pdf

A couple of months ago I was playing Super Monday Night Combat; a much-anticipated weekly update came along, but its file size was over 800MB! MNC is an Unreal Engine game and Unreal, like many game engines (and like The Witness), packages its data files together for robustness and speed. This package format interacts with the patch delivery system of any online distribution service, though.

Steam's system performs a binary-data comparison between the old version of your file and the new version, sending only the pieces that have changed. To make this analysis tractable, files are thought of as collections of 1 megabyte chunks; if anything in a particular chunk changes, even one byte, the whole chunk gets included in the patch. If you aren't careful about this, small details of your package format can trigger very large patches, and this is what was happening with Super MNC (not really the developers' fault since they were using a licensed engine; I have heard that the Unreal guys are addressing the problem.)

The Witness packs most of its data into a single 2-gigabyte file stored in the .zip format. Before uploading the initial version to Steam, I took a safeguard to minimize patch sizes, or so I thought: Files are sorted lexicographically and then placed into the zip file in that specific order. (Imagine that files are stored in random order: probably every patch involves every player re-downloading the entire game!)

Last night, as a test, I pushed a new build to Steam. The hope was that the patch would be small, since we had only done two days of work. Instead, here's what we got:

The patch was 660 megabytes. Clearly I had not done enough to prevent this. I opened up a zip file reader and took a look at the file order. In many cases, it was as you'd expect, but in the case of meshes and lightmaps we see this:

'save' is the world name and the 5-digit number is the identifier for each entity. (The world name is redundant right now, since The Witness takes place entirely in one world). We re-bake lightmaps pretty often, but we change meshes much less often. So, the packing order you see here will result in us patching more chunks than we should. Not necessarily by much, in this case, since meshes are much smaller than lightmaps, but that might not be true for all meshes, and other data types may have a similar issue but exhibit more-severe problems. So, I had the thought that instead of sorting things like this:

a.lightmap
a.mesh
b.lightmap
b.mesh


What we really want is to sort by file extension first, then sort by name:

a.lightmap
b.lightmap
a.mesh
b.mesh


... but not always. Sometimes we have multiple files associated with one source file, and that will all change when that source file changes, but that have different extensions. This is the case with shaders:

So I hacked together a solution that sorts anything with an extension of .shader*, in terms of the global file list, as though the extension were only ".shader" -- that means all these files will stick together -- but locally, you still need to keep the full extension, so that the files for one particular shader do not randomize their order every time you make a patch.

The way I do this is to tack a modified extension onto the front of every filename, for sorting purposes only. So "hello.texture" becomes ".texture hello.texture", "whatever.shader_ps_text" becomes ".shader whatever.shader_ps_text", "whatever.shader_dump" becomes ".shader whatever.shader_dump".

If this doesn't make sense to you, don't worry too much about it, because it's not that big of a problem (and I am not even sure if those ps_text and vs_text files are needed for the running game; if we should be discarding them it would certainly mitigate the issue.) These kinds of problems would modestly increase the patch size, but I would not expect the increase to be terrible.

The whole time I was doing this sorting stuff, I knew it wasn't the real problem, and in the back of my mind I was thinking, "I hope the Steam patch mechanism ignores timestamps on files so that we don't have to go touching all files to a known time. (Or I hope we aren't packing a newly-generated build ID into each file or something.)" But this was an example of my brain not working properly: because we are packing all this stuff into a zip file, Steam doesn't know they are separate files! Since the zip file stores timestamps for all the files, of course that data changes every build (since our build process generates new files every time).

At first I was thinking that this wouldn't be the real problem, either, because the zip file format has a main directory that is stored compactly, and that is where timestamps would be, because you want to be able to access that data quickly. So I would have guessed that, if all the timestamps changed, the worst that would happen is that the directory would change every build, but that is at most a couple of megabytes. But looking at the zip file specification, I discovered how wrong I was. Inexplicably, timestamps for each file in a zip archive are stored both in the central directory and in the local header for each file, so if you change timestamps for all your files, you are strewing changes everywhere.

So I made the packaging script set all timestamps to 0, but the zip library we are using didn't like that, so I set the timestamp to January 1, 2012 00:00:00 GMT.

After fixing this, as a test I built two patches in a row from scratch, with the game recompiling all shaders and re-compressing all textures each time. In theory the results should be exactly the same (certainly if this still generated a huge patch, we would still have some kind of big problem). Happily, we have found at least some bit of success:

I am not sure what it is saying there about 2046 of 2262 matching, or 60 changed chunks, but saying it uploaded 0 chunks. In any case it's better than before. I am going to ask the Steam people what these various numbers actually mean.

Feeling good about this, I ran a final test where I moved around a few objects in the world and changed a constant in one shader:

That's once again not so good (140 MB!) I had been sweeping the 18MB from the previous run under the rug, thinking "hey, it says 0 chunks sent, so maybe that 18MB is protocol overhead involved in figuring out which chunks changed, and it doesn't impact patch size." Apparently not. The next step involves asking the Steam folks for advice, hunting around for a binary diff file with which to analyze our own packages to see if we are doing something unsavory, etc.

Here's the current Perl code for the part of our script that builds the zip file:

sub modify_name {
    my $name = $_[0];

    my ($file, $dir, $ext) = fileparse(lc($name), qr/\.[^.]*/);
    my $orig_ext = $ext;

    if ($ext =~ m/^\.shader/) { 
        $ext = '.shader';
    }

    # Throw away 'dir' since we discard it when writing the file, anyway.
    return "$ext $file$orig_ext";
}

sub make_uncompressed_zip {  # Sort zip members alphabetically so that binary diffing will have an easier time.
    my ($source, $prefix, $dest_zip_file) = @_;

    my @unsorted = glob("$source/*.*");
    my @sorted = sort { 
        modify_name($a) cmp modify_name($b)
    } @unsorted;

    print("Sorted $#sorted files.\n");

    use Archive::Zip qw(:ERROR_CODES);
    my $zip = Archive::Zip->new();

    
    # Zip API complains if time is before 1980, so use an arbitrary timestamp in that range (yuck).
    my $timestamp_2012_jan1 = 1325376000;
    foreach my $fullname (@sorted) {
        my ($file, $dir, $ext) = fileparse($fullname, qr/\.[^.]*/);

        # print("Adding '$fullname' as '$prefix$file$ext'\n");
        my $member = $zip->addFile("$fullname", "$prefix$file$ext");
        $member->setLastModFileDateTimeFromUnix($timestamp_2012_jan1);  # Clobber timestamp. 
    }

    # Set the compression level on all the members.

    my @members = $zip->members();
    foreach my $member (@members) {
        $member->desiredCompressionMethod(COMPRESSION_STORED);
    }

    $zip->writeToFileNamed($dest_zip_file);
}

We finally got that working well enough; here's me getting ready to download the game on Steam:

And here I am after having launched the game:

I don't want people out there to get too excited about these screenshots: this does not mean the game is close to release. You wouldn't want to play it right now. But this will help us a lot when it comes to getting the game out to playtesters and keeping it updated.

For actually shipping our game off to Steam, we are using the new version of the Steam developer tools, and they are really good. The old stuff (which I had used for Braid) was serviceable but had a lot of undesirable things going on. The new stuff is exactly what you want: you write a couple of very small and easy to read config files, then you run a command and it uploads your game. This process is now fully-automated, which means developers can modify their game and push changes out to users faster and with less hassle than with any other online distribution service. (It is basically as fast as if you host the game yourself).

Lately, Ignacio and Salvador took the features all the way so that we can use them easily in normal workflow. (This happened over some period of time, but the last parts came together this week). Now you can take a screenshot, screw around with the colors in Photoshop, and load it back into the game to control the grading. I am pretty sure this is not our idea, and we got it from the Uncharted guys or someone like that (Update: The idea was developed by Naty Hoffman and others; see the comments to this article).

Here's how it goes. First, I find an area the look of which I would like to change:

Next I hold down the control key while taking the screenshot, causing it to be saved with a special color bar along the top of the image:

Then I go into Photoshop and screw around with the colors. Here I am using the Hue/Saturation tool to shift the palette away from red:

Or, instead, I may wish to shape the histogram of the image:

That color bar at the top of the screen acts as a look-up table. The game doesn't need to understand what operations we have performed in Photoshop; it just knows what that color bar looked like when the screenshot was saved out, and, by comparing it to the colors in the result, knows how to transform any color in the rendered scene.

In the game, you can now attach these grading operations to spatial markers, so that color grading kicks in when you are inside specific areas. Here I am setting up some markers in the editor (the marker is the big white box):

After setting up two of these boxes, one for each of the two weird color operations I created, then I can walk around in the game and have these operations apply to everything:

This gives us a really nice degree of control over how the game looks. We plan to use it to help with the color theming that will tie together specific areas.

Originally I wasn't sure whether I wanted to implement subtitles or not. It's important to me that the game have an overall aesthetic that contains no visual language. I was worried that if we were to draw subtitles for people who don't speak English, it might ruin this. But, a year ago, I was talking to a friend who has experience in film, and he convinced me that people just understand that subtitles are a different thing, that they are a layer on top of the game or movie or whatever.

So, I am implementing this, and maybe it's good, because if I had been hard-line about things, it would mean that most people in the world would not be able to understand the recordings. That's not terrible, because they are not necessary to complete the game, but as long as I don't feel like, ultimately, the subtitles are ruining things, it is probably better to have them.

(It's going to be an interesting time when I go try to implement the main menu without any written words...)

It's been a very productive day. I implemented the basic subtitling system in less than 3 hours (making a subtitle data file for one of the voice tracks, loading it, detecting in-game when a voice track is playing, finding the appropriate piece of subtitle for the current time, and drawing it on the screen). Here's the result:

The data file is a very simple format that I just made up. Taking my experience in making Braid multi-language, and looking at the kind of text we will be dealing with in The Witness, it was clear that I needed a format that was mostly about the text, with very minimal markup. Right now the main markup we need is just "which recording is the following text for" and "what time in the recording should the following lines be displayed". I can imagine that later on we will add some more features to it, but probably not many.

Since The Witness inherits code from Braid, we already have handling for other languages in our font rendering, so I pumped the text through Google Translate to test a couple of other languages:

If you are playing in English, subtitles are OFF BY DEFAULT. It amazes me how badly many games handle subtitles, most notably, that a great many games seem to have voice acting in English and then also default to subtitling in English. This is terrible because people read at a different speed than they listen, and they probably read faster. It ends up dividing peoples' attention in a weird way. I don't know why so many games do this except that they aren't paying enough attention to user experience. So to have a good experience you have to go turn subtitles off, but sometimes that doesn't even work! A couple of days ago, I was playing Xenoblade Chronicles, and when I turn off subtitles, only about half of the subtitles in the game go away. Apparently there are two different classes of subtitles and one of them you cannot even turn off! I think they never even tested it...

Every week we have a meeting with the architects where we plan the structures and landscapes on the island. Most of the week, the architects are generally off thinking about this stuff while we are building the game. During the weekly meeting we get together, talk about what concepts the architects have come up with, figure out how those might integrate with the game, make decisions about which ones to pursue, and iterate in detail on concepts we have already decided to pursue.

A while back we took some photos of this meeting, so I figured it would be fun to post them.

Here we are looking at a number of different possibilities for one of the areas in the game. I had already done the gameplay design, so we had a working location that was built out of blocky programmer stuff (if you see blocky things in the island update screenshots, it was something like that). So we knew the general shape of what we were trying to build and how it had to behave; the architects started with that and made proposals about what the location might be like in terms of its basic construction.

We chose one of these ideas and iterated on it a while. Other concerns, having to do with the overall layout of elements on the island, then caused us to change the size and shape of this location; once we had done this, the original design was felt to be a bit weak at the new size and shape, so we went back and heavily modified the concept. Only this past week have we converged on the final design and gotten a good first approximation into the game in a playable way. It's pretty cool.

Because we're working on a lot of things at once, we usually talk about a few different locations that are at different stages of development. At this particular meeting we were also looking at some early concepts of the visual design and modeling of the area where you start the game:

Just before I left my last job I was working as a Lead/Art Director on a marketing driven project focused on selling as many units as possible. On one side I had high level decisions being made by stockholders whose focus is on turning a profit while on the other side there was the development team who truly believed they would be able to make a good game.

Also I was surrounded by schedules, performance reviews, hierarchical titles, producers, human resources, bonus system and all the other corporate systems that eventually dilute the organic process of doing something creative. While I understand the logic behind it I  realized that is not the reason why I joined the video games industry and that something wasn’t right.

Now, after these few months, the answers for these questions became clear. Its all based on trust. In this project, all the decisions we arrive at are very organic. Jonathan has a very clear vision of what he wants and the fact that we can see it and believe in it makes it extremely logical and simple.

Our passion for what we do (and what we would probably be doing on our free time if we weren’t being paid for it) gets distilled directly into the game in a very rewarding process.How much and how we give it depends entirely on us and that creates a natural challenge and pressure that most AAA companies cannot provide.

This trust leads to respect and it creates a very unique work environment. This is something that I’ve never seen with all the project management, task scheduling and scrum meetings that try to fix an inherently broken game. I don’t think video games can be produced like an assembly line, at least not games that have something meaningful to say, and I’m glad that I found people that are doing it the right way.

All this may sound trivial but probably 90% of the game developers that I know will understand how big of an issue this is. I still believe big companies can change their ways and make the same profit if not more, but it has to start from the bottom, the people who are hard at work making the games.

As for the second part, I would like to share what we have been trying to do with the many island panels. As you might already know the player has to solve different puzzles using these panel interfaces scattered around the island.

I started my research by looking at what materials they could be made of, what technology they would use and how they would be assembled and how all this could relate with the different areas and narrative of the game.

This first set of images shows some proposals of how the panel screens could possibly look. I tried not to focus so much on the gameplay but more on ideas that could lead to further discussions among the team.

After some solid feedback and a couple of art meetings I was able to understand what was working or not and decided to do a new set of proposals. Also this time I tried to focus on the more technical aspect of getting these panels to work in the game and if the effort we would have to put into them would be worthwhile.

The same amount of work went into the structure of the panels and how they would be assembled. This time, working with the architects gave valuable input into understanding how they would actually be built in the environment and the amount of work required. On this image you can see a screen capture of several prototypes before being tested ingame.

These are usually modelled very fast with just base colors so I can do several iterations until I find something that I'm happy with.
And finally here is how some of them are looking in-game. They are still in concept but by placing them in the correct context makes it a lot easier to understand what is working or not.

We had to do a lot of work to get the game back into playable condition; a lot of things had been broken since the press tour last August / September. (When you are making sweeping changes to a big and complicated game world, it is pretty easy for things to get bent out of shape). So the GDC served as a good internal deadline to have the game banged back into shape.

I didn't expect a lot of press coverage to come out of this, because at the GDC there's just a lot of stuff going on, a lot of things for people to pay attention to and write about. So I had been doing this mostly as an opportunity to keep the press up-to-date with the game, see how it is evolving graphically, etc. But, to my surprise, we got a lot of coverage. Here are links to what people are saying about the game lately:

First, a video interview on Gamespot (excerpted; we probably talked for 45 minutes or an hour).

Kirk Hamilton at Kotaku finds a musical treatment for The Witness' game design philosophy.

Ben Kuchera at the Penny Arcade Report, with a piece about online distribution channels, and another biz-focused one.

Ben Gilbert at Joystiq, with a design-oriented posting and also a more biz-like one.

Alex Rubens from G4 with an overall preview of the game.

Daniel Starkey of Destructoid, with a preview as well.

As you can see from the write-ups, people seemed to really dig the game. So that is all good.

Keep in mind that, because I am known for my previous game and and the reporters are there talking directly to me, the tone that tends to get adopted in these write-ups makes it sound as though I were making the game myself. That's not the case; there's a good-sized team of very talented people building the game! (This reminds me that we need to update the About page to list everyone who is currently working on the project.)

The current puzzle count is 440.

As expected in the previous update, over the past week or so I went back and worked on the endgame again. I built an original version of the endgame last summer; I had one friend playtest it, and it worked out pretty well, though I felt it wasn't quite epic enough. I didn't know what more to do with it at the time, so I just let it sit, and went to thinking about other parts of the game.

Lately I had more ideas for the endgame, so I went into personal crunch mode and rebuilt into something more epic. This was one of the most productive periods of my working life (maybe the most productive) and I think this area, all taken together, may be the most interesting thing I have ever designed. However, it may be a little too difficult for the player, at the moment! Today I playtested this endgame and one other area of the game, with a game designer friend of mine, and it took nine and a half hours to play through these two areas (6 hours for the endgame, 3.5 hours for the other area). I think this is a little long, so I am going to be tuning these areas and cleaning them up a little bit. This weekend hopefully I can get in some playtest time with Jeff and Casey of Jeff and Casey Time, so if that happens we will see how it goes.

(It's interesting; I haven't done a playtest of The Witness for a while, but I was still thinking of it as a 10-15 hour game, and that's what I tell people when they ask. But if these two areas by themselves, even after being cleaned up, are 6-7 hours of playtime, that means the whole game is ... ... much longer. Sometime in a month or two, I'll do some playtests with people who have never seen the game before, and we'll see how long it really is.)

Meanwhile, on the art side, we've been working on the style of the game; expect a future blog post as we come to conclusions on that.

Tech-wise, Andy has been working on performance stuff. First he made raycasting much faster; til now the game had been relatively lazy about spatial organization. Andy implemented a quadtree for finding entities in the world and enabled k-d tree generation and serialization for casting rays against individual meshes. (We had k-d tree code already as part of a library but weren't using it.) Salvador is working on in-game animation playback, as well as exporting; until now we have been making do without animations, but once this is in the game it will help us with a number of issues.

This is because this corner of the world is one of the last areas that we haven't really figured out, in terms of what goes where. There is sort of just some random stuff dumped there right now. That will change pretty soon.

Expect the next few island snapshots to show a lot of big changes. We've reached the point where the basic gameplay is settled, I'm refining many of the specific puzzles, and we've got at least rough drafts of many of the architectural layouts. That means we can start planning the island a bit better. Over the next month or so, we're going to be changing the shape of the island to something a little more geologically plausible, something that carries more information. It's going to be interesting to do this without disrupting the gameplay layout of the island too much.

The current puzzle count is 427. In January I scrapped and re-designed an area and its 60-or-so puzzles. The previous area was inside dark caves, which is a bit cliche, and the gameplay hook that motivated my setting that area in caves, which I had hoped would be cool, didn't work very well. In addition, I felt that the puzzles themselves, while fitting into the rest of the game, were the weakest part of the game. I'd been feeling this way for a long time, but I didn't have a good solution for it til late December. I moved the location and redesigned the puzzles. Now the theme of the location is totally different, has a gameplay hook that works way better than the cave thing, and is very cool. The puzzles situated in this area are also much stronger than before, because I found a way to change the rules of those puzzles to make them much tighter. They'd felt wishy-washy before (people would play through that area and, after finishing, lack clarity on what the puzzles were saying. That's no good.)

We certainly could have shipped the game with the old area in the caves with the old puzzles, and probably shipped the game at least a month sooner, but an important skill in building a beautiful game is knowing when something just isn't good enough.

Next I am going back to the endgame, which I did my first pass at last summer sometime; I've got a gameplay hook to add which I think will bring it over the top and make it extra-good. I'll be working on that over the next few weeks while also working with the landscape architects on the island revision.

Also, we've got a new programmer starting soon (this week or next!) Combined with our other new addition, who's been here a few weeks (Andrew Smith, ex-Zipper and -Oddworld; hi Andy!) we will have quite a bit of tech manpower to bring to bear on finishing this game.

When we sat down to start working with the architects, one of the first things we did was put together a history of the island where The Witness takes place: how long has the island been there?  What groups of people lived there, and what kind of structures did they build?  How did later civilizations use the structures built by earlier groups?

Quickly we developed a running theme of sites that are old and have over time been used in two or three different ways.  When entering a new location, you probably first notice the most basic elements of its structure: where you can walk to easily, versus where you can't; what puzzles or goals are calling for your attention, and what walls or doors might be preventing you from getting to them.  If this is all you care about, you can then play through the area in a very utilitarian fashion.  But if you have an eye for detail, you may notice the elements of modern construction shoring up the design of an older structure with a different purpose.  The design of the new structure tells you something about what the newer guys were doing there, and how it differs from what the older guys were doing there.  And if you look really closely, you can see traces of the original footprint of the site (perhaps these are very subtle, some bricks just rising above the dirt in a few places), and infer the purpose of this very old site too.

If you see the different civilizations that came to this island as embodying different philosophies; and you see the structures they built as representative of the way these philosophies led them to interact with the world; and you see further that when they replaced a site, it represents the rejection of some older worldview that they consider no longer useful, then perhaps you start to get some idea of the amount of backstory that can be encoded into the world, nonverbally.

All of this backstory is also directly relevant to the main story -- who you are, why you are on this island, how you may get back home.  So, to put it another way, the game is constructed so that the more you pay attention to tiny details during your travels, the more insight you will have to the central story, even though it may not be obvious at any given time what a particular detail has to do with that story.

The most-recent issue of Game Informer printed an article that showed some of the first screenshots of the newly-architected areas appearing in the game engine.  It is an interesting article; pick up the issue if you haven't!  They also put up a companion article online (but which not as extensive) here:

(Summary article at Game Informer)

The area featured in this article is the one that we have taken furthest so far in terms of modeling and texturing.  Here are some other shots of it:

The Keep, before and after

The Keep, close-up 1

The Keep, close-up 2

The Keep, close-up 3

We've done basic planning for some of the other areas, so I thought it would be nice to show some screenshots of work in progress.  For each of these, there's a 'before' version (when we had a fully-playable game, but basic placeholder structures) and after (once we have incorporated the architects' designs, revised the site plans and in some cases even the puzzles, etc):

 

The Compound, before and after

Glass Factory, before and after

Peninsula, before and after

Vault, before and after

Don't regard these "after" images as representative of what the game will look like!  They are still works in progress, and the final game will look substantially better: scenes like these will feel richer due to the placement of detail objects, further care put into the texture mapping and lighting, refinement of the still-rough-drafty terrain features around these structures, and many other bits of subtle game development magic.  But I wanted to show these now since they provide a good taste of the game's flavor, and they show how much structural intelligence is brought to the table by working with the architects.  Yes, we could have started with the placeholder structures and made them more elaborate and better-looking, in a general video-game-level-design way, but that's different from having well-thought-out ideas subtly embodied in the structures of the areas, which is what we are going for.

Eventually, these things have to be figured out. Over the past month or so we've done some of that figuring. We nailed down the architecture for a couple of areas (including the all-important area where the player starts the game). We've done this in a way where the architecture and the puzzle ideas have synergy, which is always nice. In particular, there was one location that, from the beginning of development, was basically a big box with a window cut into it and some puzzles stuffed inside. For a long time (maybe 2 years!) I was never sure what to do with it, but eventually the architects suggested an idea for that location. I mocked it up and didn't like it. They suggested another one. This one sparked an idea for how to make the structural properties resonate with the puzzle theme, and when I built even just a rough-draft of the building, I found the result to be strong. Lately we've been finalizing that location too.

So anyway, there are all these spots on the island that represented vague ideas, with puzzles there that were just in rough form, or that weren't even implemented fully, but just served as reminders to do something there. I've been going through and filling in a bunch of those.

I have also been working with some other puzzles, which seemed "done enough to ship", and thinking hard about how to make them better. Braid benefited from a lot of this kind of attention -- a lot of games would have shipped with just early versions of the puzzles in that game, considering them to be just fine; but the more I worked with the puzzles in Braid, the more I found better, simpler, more-to-the-point versions of them. The same thing is happening here (though The Witness is a much bigger, more-complicated game, so sometimes it is challenging to get to that same kind of clarity.)

With regard to the visuals (modeling and texturing), things have been coming along nicely as well. Pretty soon a high-profile game magazine will publish some screenshots of one of the most-finished areas of the game, showing the look we are targeting and giving a strong hint of what the game will be like stylistically. When the magazine is out there we will link it here.

(This has not been a particularly coherent update; just wanted to let you all know what is up!)

Any system of interactivity can of course be explored: If X happens, what are the consequences? What are all the ways in which pattern Y expresses itself, and to what do those expressions lead? By inspecting the structure of a system in this way, we can find the core ideas of the system, and see how those ideas illustrate fundamental truths of our universe. We present a game design aesthetic that values looking for systems that express these truths in the cleanest possible way. We explain how this is different from more-traditional combinatoric design techniques; we show examples from our games and describe a method for applying the aesthetic in general.

Here's a link to the Indiecade page, on which there's also a presentation by Richard LeMarchand, and more to come.

Thanks very much to Ida C. Benedetto for recording the talk!

I'll post something more concrete before too long, but I wanted to at least say something, since it's been a while.

Team-wise, we have just hired two people (a 3D artist and a programmer), and we plan to hire another programmer as well, once his visa clears and he can get into the country. This is a pretty big staffing-up, percentage-wise, and will allow us to make things quite nice. The new folks will be starting over the next couple of months (they need to wind down the things they are working on currently!)

The difference is not too pronounced in this scene, but with smoother-colored textures it becomes a lot more obvious (and I do want to use more smoother-colored textures, art-direction-wise!)

The situation is, the lightmaps are HDR (for some definition of "high"), so we needed a way to encode those values reasonably into a bitmap. We were using RGBM where M is just an overall brightness factor; you read RGB from the texture then multiply by M to get the output color. We used to just encode this naively, but the problem is, when M is less than 1 then it actually kills precision in your RGB, so the solution is, well, don't ever let M go below 1; just clamp it. This is not rocket science and in fact is the kind of thing that is a little funny, as one starts wondering why we didn't think of that a long time ago, but hey, that's how it is sometimes. The results made me happy, though, so I am posting them.

The improved image quality may enable us to have higher dynamic range throughout the game (this is probably something we will evaluate later on), which would be very nice!

The talk's about 38 minutes long, with about 40 minutes of Q&A afterward. Here's a slideshow:

If the above embedded player doesn't work for you, or you prefer to download the files directly, here they are:

audio track
slides (ppt format)

I've worked on trees for games over the years, and I've come to a conclusion: There's no good way to make them currently. Trees reject the idea of low poly versions of themselves by their very nature, and what you're left with is many unsatisfactory ways of making them. It's up to the artist to pick which one of these techniques to use and to do their best to make it work. That choice depends on a lot of things. Does it have to be real-time, or can it be pre-rendered? Where will the player be when he sees the tree? Up close, far away, underneath, from above, or (in the worst case) all of these? Will there be a few trees or a forest of them? Then there's the engine constraints.  Overdraw, poly counts, texture space, LOD systems, lighting and light maps; all these affect how the trees need to be created. And none of that is even scratching the surface on the style you want the trees to have.

Still, I like a challenge as much as the next person.

Here's an early visual mockup of the game that contains some trees. I didn't actually make this, but it did give me some direction is where to go. As you can see, the trees are very colorful and full of volume. I wasn't sure how close I could get to this, but I was going to give it a try. The problem with most low-poly game trees is that they are made with large sheets of polygons with branch and bough textures on them...and while this is certainly a way of making game trees, they usually can't be considered volumetric.

The current trees I have right now started from a lucky accident. Early on the engine couldn't handle alpha maps, so I want to make a bough structure that would look vaguely leafy without being too many polys and without using opacity maps. That's how I came up with this.

It's basically a ton of triangles sitting in a bowl. The bowl is there to simplify the bottom and hide the triangle nature of the triangles. It worked fine for the time I needed it. Recently I started adding a bunch of trees to the world and I wanted something fairly lightweight for even the large trees (the one I had done just before that was 40,000 polys, which was way too big). So I went to the old non-opacity tree, added some opacity maps and tried it out. And, surprisingly, it worked really well. It was giving me a volume I didn't expect, and it looked really good at a distance.

Not only that, I took a look at them with only the lightmaps on and the structure looked really nice...especially the branches underneath.

They were still too heavy though, being 12 thousand polys. Worse, they looked really bad up close and you could see that, what read as a nice, thick foliage from a distance, was really a mess of triangles.

So the past few days I've been trying to make the trees look better from a close distance and be lower poly without losing what I liked about the previous ones. After a bit of struggle, I got something that seems to do the trick. Ignacio helped a lot, making a shader that made polys disappear when they became too edge on to the camera, which eliminated many ugly artifacts. The foliage is half the polys of the other trees, plus it looks a lot better close up. Here's a close shot of it, plus a comparison shot with an older tree.

I still have a bunch of other ideas I want to add to these to make them look better, but I think we're on the right track to getting trees that really do have a nice amount of volume and harken back to the early mockups.

But we also figured, hey, why not play around with the color of sunlight as well? The idea is that as you walk around in the game, the color of the sun and ambient light will interpolate as you move. We haven't hooked it up that way yet; what you see below is just done by tweaking global variables in the game. Well be doing more experiments in this direction...

Much denser and more visually interesting; it's starting to look like maybe a good game!

Here are some close-ups of different areas now.

Autumnal:

Desert:

Foresty (there is a lightmapping problem on some of these trees so they are brighter than they should be!):

But now we are thinking about how to create colorful scenes, like in the previous post, and this style of blend-mapping did not seem like a very good idea. In order to create gradients across the scene, we would need to have a lot of blend layers; this would be slow and use a lot of texture memory (since now we are storing a lot of individual blend maps). Even once we did that, it would be laborious to control, and it's unclear that it would look good.

Ignacio mentioned that the Wolfire guys were just dropping one color map across their whole terrain. That seemed like a good idea to me -- I have done that before, but in contexts where we were trying to really crank up the resolution to levels that were ridiculous at the time, like 8192x8192 texture maps back in 2001. Rather than doing that, the idea now is to just use what is a reasonable-sized texture on a modern machine, and see what it looks like. So Shannon painted a 2048x2048 texture of the island (of which we are only really using less than half right now) and here's how it looks from up high:

Immediately this strikes me as much nicer, and much more in-tune with the art direction we want for the game. The buildings and other meshes now stick out like sore thumbs, but that is just what happens when you improve only a part of the scene. We'll be working on this.

Here's a scene from closer in:

As you can see, we have less detail close-up on the ground; things look a bit blurry. However, the colors are a lot better, and I think if we can adapt this scheme into something that permits sharper details up close, we will be able to make something that has a lot of character and looks special. In these shots, we are still using the old blend map system to tile high-detail normal maps across the terrain; that helps, but by itself it's not enough. (In recent videos for Overgrowth, like this one, you can see that they are doing something that provides close-up details, maybe just intensity maps that are being modulated by the color map). So we'll keep working on this and see how it goes.

Meanwhile, Ignacio has been revising the shadow map rendering. We used to have very speckly shadow maps, as I was using a 4-sample Poisson disc to sample the shadow map. Ignacio recently switched this to a Gaussian that has more taps, then applied some math tricks to reduce the number of taps. This results in softer shadows:

At some angles, shadow map jaggies still show up pretty clearly:

This is something we might work around in the final game by changing the scene when this happens! Or, it's possible the shadows will be modified further so that they look smoother in cases like this. Who can say?

Shannon did a few color studies by taking screenshots of the game world as it stands today, then painting over them to build scenes with different palettes. Here's what he came up with:

I think it looks quite nice. Don't assume much based on the painterly style of these images; they were intentionally done at a low level of detail, so that we could quickly get a feel for the colors. We don't know yet exactly what the rendering style of the game will be in the end (but that is something we will start working on soon!)

The puzzle count is now 285.

This means I am getting closer to my Gameplay Complete milestone. The bulk of the game design is now more-or-less there, in the sense that you can go and solve all the puzzles and they are basically the body of puzzles that will be there for the final game (though in the final game they will be refined, and more will be added -- but if we had to ship with only the puzzles that are in there now, we could do that, and it would be a good game). However, there's no endgame or winning condition, so I will be working on this next.

I think the endgame is going to be a bunch of work -- the original idea was ambitious, and I need to do a bunch of redesign on it since the game as a whole has evolved since the time the endgame was originally conceived. We'll see how it goes.

The visuals are still placeholder. In a couple of weeks we have some architects coming in to start designing buildings and terrain; at that time our production pipeline will change and we'll be moving a little closer to "the real visuals".

As a result of these sessions it immediately became clear how to redesign some of the areas to improve pacing. So I started radically restructuring three of the major areas, as well as rearranging the relative positions of areas on the map. That process isn't done yet, but it is starting to converge (just one more area left to redesign). I am very happy with the two areas already redesigned; they are obviously far better than the original versions. It feels very rewarding when you feel good about a game already, and then do something to improve it so much.

Also, I've shrunk the island again. Here's what it looks like now:

The current puzzle count is 275.

Also, we are hiring.

Here's what it looks like now:

It does not look shipping-quality, but then, almost nothing in the game does! As the first results obtained by just slapping some pre-existing grass meshes down on the terrain without trying too hard, I find it highly encouraging.

Note that grass doesn't cast shadows right now. There are two reasons for that: rendering speed and aliasing of the shadow map. I suspect we will try to make grass cast shadows before we ship, though, and if we make something good happen there, I think it will really boost the visual quality.

The one piece of code that pushed the grass over the threshold into general usability is that it now has the ability to sample terrain blend maps. (The terrain uses a blend shader, where a non-visible blend texture map controls the degree of mixing of different terrain materials. This is becoming a common thing in 3D games; Uncharted 2 and Crysis 2 do it, among others.) The grass can now peek at the blend map and reduce its density in areas of particular blend values, which is why in this screenshot that path going up the hill is free of grass. This tremendously reduces the amount of manual tweaking required to make the grass not do obviously-bad things.

However, the grass doesn't check for buildings yet, which means if you place a field of grass across the area of a building, the grass will probably stick up through the floor inside the building! Fixing that is next on my to-do list, but it involves a lot of raycasting, so hopefully it doesn't make grass placement too slow.

In the future I plan to try using this same "grass" system for other kinds of scattered ground objects (fallen leaves, twigs, small rocks, etc).

I was talking to David Hellman about the game the other day, and he was asking when the island was going to become less like a flat plane where you can just walk to any destination from anywhere else.

It's been on my to-do list for a while to add more topology to the island. That week I had been working on an area full of puzzles, and I wanted the buildings that define this area to have more of a purpose than they currently do; a topological purpose seemed like the right thing.

So I figured I'd add a canyon, and the building containing these puzzles would also serve as a bridge across the canyon.

The natural way to build the canyon in this engine is to shape it out generally with terrain control points, and then make the walls steep by putting mesh objects in there (so that you can't walk up them, and so that the walls of the canyon won't look super-distorted the way heightfield terrains usually do when there are steep slopes.)

Here was the first cut at it:

As you can see, there are a lot of weird ripples and curves happening there. This is what signal processing people might call "ringing". What's happening is that the Kriging system that we are using for terrain modeling is always trying to reconstruct a smooth function, but what we are trying to build here is a discontinuous function (or else a function so steep that it has a lot of high frequencies in it).

There are a number of ways that one might fix this, but the cheap and hacky one, which would work not only for Kriging but for any other terrain modeling stuff we might stick in in the future, is to add the notion of separating planes that represent discontinuities in the terrain. When evaluating the height of the terrain at any point, we just ignore any control points that are on the other side of the separating plane.

Adding discontinuities has been on my to-do list since last year when Kriging was added to the engine. It was pretty obvious that it would be needed eventually, but in the spirit of getting things done and not rat-holing on technology, I chose to put it off for as long as possible. But it makes sense that if we are starting to add more topology to the island, we are going to want to have more steep places, and so it's time to add this.

(We've already had this steepeness problem before, though: if you look at this plateau that is near the camera in our regular island snapshots, you'll see that the whole thing is made out of mesh objects, rather than terrain; we did this in order to avoid this very same ringing problem:)

It didn't take long to add discontinuity entities into the editor; here's how they are represented:

To start with, I just used a really dumb heuristic to decide when control points should be excluded. When we use this, here's the result:

There is still a bunch of heightfield terrain sticking out. What's going on? If we look back at our red discontinuity planes, it is clear that there are some terrain sample points inside the canyon that are getting treated as if they were outside:

What's going on is that my simple criterion said "if you are on the opposite side of a discontinuity plane from a control point, ignore that control point". The problem is that I am treating them as infinite planes, so those points that you see, which are in front of one plane, are actually outside a neighboring plane, if you were to extend that plane infinitely. This is no surprise; I knew it would be a problem, but I am always about implementing the simple thing first and seeing what happens. But now I need a heuristic that will fix this problem.

One solution would be to combine the discontinuity objects into sequences of points, and then do some more-heuristicky things that may be easier because we have an ordering between the points.  But I don't want to resort to that if I don't have to, because that may make the system less general, and may make it harder to edit.  (Then again it may make it easier to edit, because we have editor support for paths!  But if we want to, we can easily go the other way, from a path down to individual slabs, so it's better to take the most-primitive thing as input if we are able to.)

I added a check that would make discontinuity planes irrelevant if the sample points are closer to the side planes of these very thin slabs than they are to the face planes.  In theory that would at least help, but upon running this, the scene looked almost exactly the same.

Then upon further investigation I found that I was computing totally wrong planes in the Kriging code; I was transforming the coordinates incorrectly, and so the planes were not in fact where the red slabs in the picture would show them to be.  I fixed this:

Ahhh, much better.  This seems like it will be good enough for now.  There's still the question of how to seamlessly blend the heightfield terrain into the mesh objects, but that can wait until closer to ship.  For now, it doesn't need to be pretty; it just needs to let us build things of roughly the right shape.

The heuristic I use for discarding points is still pretty simplistic and may cause problems.  My hope is that most of those problems can be worked around just by changing the planes a bit.  If that turns out not to be true, a little more work will need to be done here!

The general requirement was that we need to be able to capture a pre-rendered scene with a really wide viewing angle. At first I was confused about the technical aspects of the problem (hopefully forgivable, since I spend most of my time thinking about gameplay these days, so my tech is a little rusty), so I thought that we might need to have a linear projection in order to solve the specific problem under consideration. So Shannon made a mock-up scene in 3D Studio MAX and we started making prerenders with increasingly wide camera angles, in order to test our special environment mapping. Here are three shots of the same scene with fields-of-view of 90 degrees, 120 degrees, and 150 degrees:

(Click on the images to see actual sizes.)

It's clear that as the field of view becomes wider, we can see more of the scene at once, laterally. This is good for our application, because we want to see as much as possible! But at the same time, when the angle is wide, objects at the center of the scene appear further away, occupying fewer pixels in the rendered image. This is bad because it means we only have low-resolution imagery for the most important things in the scene! (These screenshots are also badly artifacted because for some reason the avi encoder we were using interlaces the output by default, and we couldn't find a way to not interlace, without using a different codec! Hint to anyone making a video codec, anywhere in the past or future of this universe or any other: NOBODY LIKES INTERLACING. IT IS UGLY AND CAUSES A LOT OF PROBLEMS. DON'T DO IT. Please do your part to make the world a better place.)

It's around this time that I realized we didn't have to use a linear projection, which was good -- if we can warp the image however we need to, then we can have the important parts of the scene landing really big in the middle of the texture, using lots of pixels, and squeeze the rest of the scene nonlinearly around the edges of the bitmap, using fewer pixels.

A 180-degree fisheye lens seemed like the right tool to do this. For other reasons that I won't go into, another nice property of the fisheye lens (as opposed to some arbitrary distortion) is that it is physically plausible -- you can mount a fisheye lens onto a physical camera and generate a similar image.

We found some plugins for Mental Ray (a renderer that you can use from Max or Maya) that seemed to do the trick. Here's the output of one of the plugins we found (this is from Jan Sandström's JS_fisheye.c, which appears to be a modification of a simple fisheye lens shader in the Mental Ray reference manual):

Unfortunately the field of view in this image we saved is less than 180 degrees, which makes it hard to directly compare with the later images, but just look at the basic character of it for now.

At first I assumed that this shader was implementing actual fisheye lens math, though there were no real comments to go on. Here's the math used in the stripped-down Mental Ray Reference Manual:

    mi_vector_to_camera(state, &camdir, &state->dir);
    t = state->camera->focal / -camdir.z /
           (state->camera->aperture/2);
    x = t * camdir.x;
    y = t * camdir.y * state->camera->aspect;
    r = x * x + y * y;
    if (r < 1) {
        dir.x = camdir.x * r;
        dir.y = camdir.y * r;
        dir.z = -sqrt(1 - dir.x*dir.x - dir.y*dir.y);
        mi_vector_from_camera(state, &dir, &dir);
        return(mi_trace_eye(result, state, &state->org, &dir));
    }

(JS_fisheye.c has more parameters but is basically the same thing.)

Essentially, this code takes an input texture coordinate and converts it into a view vector in 3D space. In order to use the rendered image from within a shader in the game, I need to be able to invert this function: turn it from a view vector into a 2D texture coordinate. But when I tried to do this, I had all kinds of problems: the math was messy and ugly. I got the inkling that this shader may not be acting as a physical lens really would, but rather, was a 2D image-warping effect that gives the same general impression as a fisheye lens.

(The reason for thinking this is that light behaves bidirectionally; there's no difference between light moving forward and light moving backward. So if something is simple when light is going in one direction, it ought to be simple in the other direction too. If the equation starts looking a lot more complicated, that is sort-of a violation of the way that the physical universe works, mathematically, and so one starts thinking that something is wrong. That was the intuition I had, anyway.)

Over on this page I found a nice reference for the way that a real fisheye lens bends light. Indeed the equation is very simple (though as it is written on that page, it's not quite in the proper form for us to use). Based on that, I wrote a new Mental Ray shader that looks like this:

    mi_vector_to_camera(state, &camdir, &state->dir);
    miScalar x = state->raster_x / state->camera->x_resolution * 2 - 1;
    miScalar y = state->raster_y / state->camera->y_resolution * 2 - 1;
    
    miScalar r2 = x * x + y * y;

    if (r2 < 1) {
        miScalar c = 1 - r2;
        miScalar s = sqrtf(2 - r2);

        dir.x = x * s;
        dir.y = y * s;
        dir.z = -c;  

        mi_vector_from_camera(state, &dir, &dir);
        return mi_trace_eye(result, state, &state->org, &dir);
    }

Superficially it doesn't look too different from the previous example, but in fact this version refracts light in a physical way and is invertible. Here's what a render looks like using this lens:

As I mentioned, it's hard to compare with the earlier shot because the field-of-view is different (sorry about that), but I think it's evident that the nature of the distortion is fairly different between the two shots. In our version, it feels milder.

Here's the runtime pixel shader code that inverts it:

    // (xprime, yprime, zprime) is the view vector in the same space where the environment map was rendered.

    float c = abs(zprime);

    video_uv.xy = float2(xprime, yprime);
    float scale = 1 / sqrt(1 + c);
    video_uv.xy *= scale;

    // Now we have video_uv in [-1, 1]; for [0, 1], scale appropriately.

It's very simple, and the only real math required is a multiplication by a reciprocal square root (very fast for shaders!) So that was pleasant.

Once we had these both hooked up and working, the scene rendered perfectly, and we knew that the effect we wanted to create was achievable. By way of improving it, Ignacio suggested using a cylinder map instead of a fisheye lens, because that is better for this particular shot: we need to render a wide room, and see a lot laterally, but the vertical span is roughly constant and much shorter than the horizontal span. (The fisheye lens is more general, and we can use it for any scene, but to optimize texture resolution for specific cases, we might go to other things like the cylinder map. You can think of the cylinder map as being fisheyed along only one axis.)

Here's the cylinder map version of the scene:

This is what we are going with for now. Problem solved, job seemingly well-done.

Ignacio put the cylinder mapping code into the same shader as the fisheye lens, and also added a latitude-longitude distortion.

Our work was sped up drastically by the fact that we were able to find JS_fisheye.c free on the internet, as well as information for the way a fisheye lens works. So here's our attempt to give back a little: our final Mental Ray shader and the .mi file that defines the interface for it:

witness_fisheye.c
witness_fisheye.mi

If you are an experienced graphics programmer asking yourself why the hell didn't they just use a cube map for this, well, there is a very good answer to that, but it involves the spoiler. You'll see when the game is released!

I had been viewing a bunch of online portfolios of 3D modelers (and lamenting the quality of these portfolios) with an eye to hiring someone. Sometimes these portfolios are shown in steps: here's the wireframe, here's the filled-in geometry, here it is with lighting, and here is the final thing after it's been textured. Casey pointed out something interesting about these that I knew was true as soon as he said it: most of these portfolios look worse than they need to because the texturing ruins them. When you run a radiosity algorithm on a scene full of unpainted geometry, it has an elegant, sophisticated, scent-of-the-real kind of thing going on; but it is very easy to spoil this sophistication through texturing. And, it seems, in video games we have developed a tradition of "bad" texturing, whatever that means: unsubtle, unrefined, or just plain ugly.

We're on a low budget here, being an indie game; so not only was it an open question how to solve the Ugly Texturing problem, but we also needed to solve it with relatively little manpower.

I knew that, ultimately, we could just fall back on untextured geometry with lighting, if we had good precomputed lighting. That's why we spent so much time and effort putting together our lighting system (as has been written about many times in this blog). The question was, would we stick to vertex colors on most objects, or would we use some kind of texturing?

Early on, we identified a texturing style that would work for human-sized objects and would gel with the gameplay of The Witness. The style prioritizes color, and tries not to break up the geometry of the object by imposing conflicting details through the texture. We did some objects like a 1-person boat in this style in order to refine it. [I'd like to include a screenshot of the boat here to illustrate it, but the boat is messed up right now, so I'll just have to be vague.]

Because that style tries to use the texture map to accentuate the object, the contents of the texture must change with the local neighborhood of the object, which means you can't use tiling textures very much. This means it would be very expensive to texture something like a big building, since you'd need unique texels everywhere.

Thankfully, the Western game development world has a tradition of developers being very open with their techniques and processes, and communicating with other developers very honestly about what is effective. (This is not something that one finds elsewhere in the world, and we should work hard to keep that spirit alive because it benefits everyone; but that's a totally separate rant.) Here's one good example of this kind of information exchange: at the GDC this year, some guys from Naughty Dog gave a talk on the art direction for Uncharted 2 (pdf link to their slides here.) Somewhere around page 220 of this very informative slideshow, they start talking about the blend shaders that they use in order to give more character to buildings and other geometry while still using tiling textures. The idea is that you interpolate between two (or more) textures with a blend function that fits the shape of your building nicely. You put some smarts into the interpolation so that it produces something that is visually interesting, rather than the blurry mess you would get via naive interpolation.

For The Witness it seemed like a good idea to use this kind of technique, but in a minimalist way, to create patterns that accentuate the shapes of buildings, and help tie them together visually, without stumbling into the realm of Ugly Texturing.

(There are a lot of reasons for the minimalism; one reason, though, is to provide clarity about what you can interact with, versus what is inert. Interactive things are colored in a way that makes them stand out in an obvious way. Think of it as something like what Mirror's Edge did, but in a different way.)

We've started experimenting with these blend shaders; here are some images:

The upper-left shows a scene before we implemented the blend shader for buildings. In the upper-right, we are using a blend shader that blends the original texture with an alternate one that is a bit darker and rougher (but the difference is a bit subtle, by game standards at least). Below these images are the same scenes with precomputed lighting mixed in.

It is pretty clear to me that the lighting is actually the biggest ingredient in tying the scene together, but the blend shader is, I think, a good extra kick. It helps the area feel like there is a little bit of life and history to it (for example, the lighter-colored paths near the puzzle panels in the foreground, which imply that people have been walking that way). What you see here is not of shipping quality, but I see how we can reach shipping quality by iterating on this style and putting production-level effort into the assets.

Here's a close-up of what the stairway looks like:

As you can see, the texture maps have some grain to them; from far away, the grain melts away and the structure is mostly a solid color, accented by lighting.

These experiments are still early yet but I find them very promising. The plan is that we will continue to hone the style and end up with something very special.

I've seen mixed reactions to these mazes and to the gameplay shown in the video. Some people seem to have jumped to the conclusion that the game is going to be about solving simple maze puzzles, and thus not be particularly exciting.

If that thought crosses your mind, all I can ask is to have faith. I wouldn't make a game about solving a series of rote puzzles. The footage came from the beginning of the game because we don't want to spoil anything major. But even this beginning-of-the-game stuff has more to it than meets the eye.

It's like Braid, in a way. Back before Braid was released, I never made a trailer video for it, because I couldn't come up with a way to show in video form what the game was about. You can show some rewinding happening, or other funky time stuff, sure, but what is cool about Braid is not the mechanics of those things or how they look on the screen. What is cool about Braid is what happens in the player's mind during the puzzle-solving process. The time stuff is the method by which that is implemented, but the rules of time manipulation do not contain the magic. Watching a video, it's nearly impossible to see the magic. You have to play the game to see the magic.

The Witness is like that too. The blue mazes are a major aspect of the game, but they are not the point. The point is the magic that happens in the player's mind when he understands the subtle things that the mazes are saying -- because the mazes aren't just puzzles, they are lines of communication that aggregate, become more complex and eventually say surprising things. This can't be seen in a video; you have to play it to have the experience. The magic is in the player's head, not on the screen.

The only two guys who have played The Witness to completion have said it will be better than Braid when it is done. I am certainly not going to jump up and down and say "hey this game is better than Braid", or even claim that Braid is good. But I just want to put that out there as reassurance to those of you who are worried about that gameplay video.

I have a file with about 80 game ideas in it, most of which would probably make really good games. But I can only make one game at a time right now, so I had to pick one. I picked The Witness, even though it more difficult and expensive to make than most of the other ideas, because I was more excited about the core concept of The Witness than about the other 79 ideas. This is a highly ambitious game that stretches my ability as a game designer to its limit.

So if you don't yet see why the game is ambitious or exciting, please have patience! We'll be releasing more information as the game comes closer to completion...

And whatever you think about what you've seen so far, I'd like to thank you for your interest in the game!

For all three days of PAX 2010, The Witness was publicly playable by anyone who came by the booth.  However, it was unmarked and unattended, so it was easy to miss (as many people did; but then, plenty of people played it, too).

Here's a close-up, with the opening scene of the game, as it silently waits for someone to sit down and play:

I had several reasons for wanting to show the game this way.  Firstly: At a show full of companies trying to capture your attention and sell you things, I wanted to do something that is subtle, and a surprise -- if you notice it, and decide to investigate, you find something unexpected.

Also, I wanted people to be able to play the game for as long as they want, not feeling pressured to stop playing because of a huge line of antsy people waiting behind them.

Before PAX, only two people had ever played the game; those people were game developers, and the versions they played were very rudimentary.  This was the first time the game had ever been played by general players, and I wanted to see how people would react to it, as honestly as possible.  I didn't want to give them any reason to think that this was a good game, or was by any designer they might have heard of.  So I just left the game running, and would occasionally peek at it from afar to see who was playing and how they were doing.

PAX is a huge show, with tons of stuff to see everywhere; if the game could hold peoples' attention under those conditions, purely on its own merits rather than by hype or pedigree, then I would know that it was really working.

Happily, the game worked very well -- despite the fact that the visuals are still in an early prototype stage.  People tended to play for a long time, and they liked what they played.  Here's a picture from Sunday; these guys played for a long time, perhaps an hour and a half:

Some game designers did recognize the game; here's Dylan Fitterer (of AudioSurf) playing:

Stephen Totilo of Kotaku recognized the game early, on Friday, and found me camping out on the other side of the booth.  Thankfully, though, he didn't spill the beans.

I am very happy with how this playtest session went.  I am glad I was able to bring a game directly to players, as a surprise.

The common wisdom among game designers is that the first time you put your game in the hands of players, it totally falls apart, and nothing works the way you expected.  Thankfully, that didn't happen here.  The game does work as expected, despite being kind of crazy and experimental, and people enjoy playing it.

All that's left now is just to do all the hard work required to finish the game!

Update: Stephen Totilo has posted his impressions, along with a gameplay video, at Kotaku.

A few of the many, many ideas we looked at:

Here's David, looking like he's at the top of his game:

We are starting to tinker around with the sky (this one is way too dark). Also, moving around a bunch of locations and adding some new ones.

I'm doing a couple of weeks' worth of serious map organization / reorganization in order to get the game to a point where we can playtest it at pretty much any time, and whoever sits down to play will have a reasonable experience. The game was in this kind of shape back in January, but we had to do a lot of work on game internals in order to get things where they needed to be, interactivity-wise, for the mid-stage of development. Now that most of that stuff is done, it's time to put the world/content back in playable shape; from there, new stuff can be added in a way that continuously fills out the world. Once I feel like the world is "full enough" we can leave the preproduction phase and start going into production (which is where we start caring about what the graphics look like, start seriously doing voice recordings, etc).

One area I've been working on for the past few days is the start of the game. It is meant to be a semi-guided experience that clues the player into what the game is about. (If you remember the beginning of Braid, it's something like that.) You may have seen the interior of the house (the white building on the left) in earlier blog posts:

The area just outside the house is a constrained path, but very soon the player exits this area and can freely roam an open world.

Most open-world games want to impress you with the hugeness of the world, and the corollary to that is that you spend a lot of time walking / riding / driving between places. The Witness takes the opposite philosophy: it wants the world to be as high-density as possible, just packed full of interesting things. As the island develops, we will be placing areas as close to each other as we can, within constraints. (Braid had a similar high-density philosophy -- though it manifested in a different way -- and many people appreciated that).

The Witness is an exploration-puzzle game, to be released on multiple platforms — whatever makes sense in late 2011, when the game will hopefully be finished.

The game is being developed by a small, independent team, a mix of full-timers and contractors. Here's a list:

Jonathan Blow
Jonathan is a designer-programmer whose goal is to make games that are mind-expanding in ways special to this medium. He is best known for the well-received game Braid. He is also a partner in the Indie Fund, an initiative to help creative new developers grow stronger while remaining independent. He speaks frequently at conferences and universities on the advancement of game design as an art form.

Ignacio Castaño
Ignacio started programming at an early age and became obsessed with graphics. That led him to work at several game companies including Crytek, Relic and Oddworld. More recently he worked at NVIDIA, where he contributed to the design of modern graphics hardware, APIs, and programming models. Today he's the father of a 3 year old boy and embraces all aspects of human experience.

Shannon Galvin
Shannon Galvin has worked on games for over 16 years, having started on a game called 'SimTown' and then adding pretty pictures to many others...including SimCity 2000, Sims 2 and Spore. When not doing art, he often wanders around games looking at environments and using cheat codes when monsters try to interrupt his art appreciation. Oh, and he's got a ceramic kiln in his living room. Those things can vaporize metal! (Caution: Kilns should not be used to vaporize metal).

Andrew Hynek
Andrew Hynek has been a game programmer since the early PS2 days. Having worked at small and large studios on a variety of genres, he is trying out the indie / freelancing life at his new company, Atrophy Studios.

Andrew Lackey
Andrew loves sound, nails on chalk board and all. As a gamer and indie music/film/game fan, he's thrilled to connect with the team on the Witness. Music and sound have been a part of his life since day one, but he got his professional start in Los Angeles on feature films. Go!, The Matrix Trilogy, 8 Mile, and Treasure Planet were among the 30+ films he worked on as a sound editor and foley supervisor. In 2004, he defected to San Francisco and shortly thereafter... the game industry. After a brief stint at Skywalker Sound, he went to work for Electronic Arts as a senior sound designer. He worked on several titles including the creature voices and sound design for Spore and Dead Space...although in decidedly different veins. In late 2008, he went indie and started Wabi Sabi Sound. It's been a great ride. He's contributed to BioShock 2, Left 4 Dead 2, Dante's Inferno, and currently The Witness. The nonconforming push of the envelope is a space he's always thrived in. The Witness challenges him to rethink the fundamentals of a craft he's been doing for over 10 years.... he loves that!

Nicholas "Indy" Ray
Nicholas has been programming professionally for 8 years. He first worked as a game developer at the U.S. Army-funded "America's Army" video game project. He has wide programming experience, ranging from compiler design to animation, and spends much of his time time trying to come up with unique solutions to problems, while stubbornly holding onto the conviction that byte-level data transformation is what programming is all about.

When not programming, he enjoys rock climbing in the summer, various snow sports in the winter, getting lost, and most anything that involves making a fool of himself.

Eric Urquhart
At the age of 7 when Eric won his local community dental art contest by designing a fluffy rabbit with a big shiny tooth. It was then when he knew he could be on to something. So he started practicing painting on his bedroom walls. That wasn’t a big hit with the parents but did serve as good experience for his future as a mural artist. Eric quickly became sought after for his mural work, being flown around the nation to add his personal touch to structures all around the country. Six years ago he set down his analog brush and paints for a Wacom tablet and Photoshop. Since then he has been unleashing his artistic talent in feature film, television and the gaming industries. Past companies and clients Eric has worked for and with include Sony Pictures Imageworks, Warner Bros., Fuel and Brand New School.

[See image gallery at the-witness.net]

It may not look tremendously different from the last one, aside from the translucent water and added objects at the near corner.  However, a number of locations have been worked on in the past few weeks, which happen to be either far across the map or underground, so I figured I'd post some highlights.

Keep in mind that all of this is placeholder graphics; because we consider gameplay to be very important, we are planning out the gameplay carefully before we go into full production.

This area is an old ruin carved from stone; as the world gets developed, the area around it will become more deserty.  In a later blog update I'll show this area more extensively:

[See image gallery at the-witness.net]

Here's  a cave at a junction of underground tunnels, lit by colored lights so that you know which tunnel is which:

[See image gallery at the-witness.net]

A cathedral, built on a hill; its walls are old and crumbling, but it's been well-cared-for: