There's one other place where fog snaps right now, and that is in time -- it is probably pretty easy to fix, though the explanation might be a little unexpected so I am Ccing it to the team as a general knowledge transfer thing.

In general, with floating point numbers, there is a problem with loss of precision when you do computations involving large numbers added to small numbers; the small numbers tend to get dropped and go toward zero. This is a general thing about computers, not just graphics (it is the price we pay for being able to pretend we can store real numbers in small amounts of memory). Most of the time it's not an issue and programmers don't think about it, but there are some times when it can be an issue.

In shaders, we often want to have a time variable, controlled by the CPU, that we can use to help generate effects. That time variable counts upward, maybe from the time the level started, or the time the game started. If that number is measured in seconds, then after 10 minutes it will be around 600, after 2 hours it will be 7200, etc. If someone leaves the game running for 3 days (a certification requirement on consoles!), it would get up over 250,000.

Those numbers don't seem super big, but at the same time, we also deal with numbers that are small. If the game is running at 60fps (which on PC these days is considered maybe a slow frame rate!), that's 0.017 seconds per frame, and we might be doing math that involves both this dt and the current time. At which point the ratio of the current time to the frame time is like 250,000 / (1/60) = 15 million. That is really very high and a recipe for all kinds of numerical problems you don't expect -- if you add the delta time to the current time, in a 32-bit float, the delta time is almost completely destroyed because there is not enough precision in the number. Long before this time, you'd start to see jitteriness in animation, things not matching up to where you'd expect them to be, etc.

One way to solve this is not to let the current time grow indefinitely -- at some point, you reset it to 0. You could pick a fixed amount of time, like I don't know, every minute, and at the end of the minute the time is back at 0 and you keep going. But if you want your effects to be seamless, they all have to match up exactly at the 1-minute boundary, which makes tuning things difficult and annoying. For example, if you want to make a cosine wave that is seamless across time, you can make the wave with a period of 1 minute, or half a minute, or 1/3 a minute, etc, and firstly this is annoying because when designers want to tune stuff, they have to know this, and it's unwieldy and confusing, and secondly, these numbers cannot be represented exactly in floating-point numbers either, so you're adding imprecision that way.

But there is an old graphics programmer trick that solves this very cleanly, that I think has been around for a long time (I first heard of it from Ignacio when we were working on The Witness).

For our cosine wave example mentioned above, we are going to be evaluating some function like cos(freq*time*2*PI), where 'time' is just the wall clock time according to the game, and 'freq' is some parameter controlling the frequency of the cosine wave, that we want it to be very easy for designers to adjust without introducing problems.

The constraint you have to meet, for your cosine waves to match up when the timer resets, is that they all have to evaluate to 1 at whatever the max time is -- because when you reset to time = 0, you will get cos(freq*0*2*PI), which is 1. So cos(freq*time_max*2*PI) also has to be 1, in other words, the cosine has to have gone around the circle an integer number of times. Since we have a factor of 2*PI in there, this means that our wave will be flawless any time freq*time_max is an integer.

How do we ensure that, easily, in a way that people don't have to think about? Pick, for time_max, a number of seconds that is 10 to some integer power -- for example, 1000. Then, any number you type into a tweak file, that has no more than 3 digits after the decimal, will result in an integer number of times around the circle. For example, if freq = 9.876, we get cos(9.876*1000*2*PI) == cos(9876*2*PI) == 1. This works for any number you pick with 3 digits after the decimal. If you need 4 digits, make time_max 10000, and so forth. If we are reading values from a tweak file, we can round them to the nearest 3 digits before sending them to the shader (and maybe output a warning, that the digits aren't being used, if the rounded number is appreciably different from the input number, to remind people of this system).

This works with any periodic function, we just used cosine here as an example. For linear shifts, like scrolling uv coordinates or whatnot, it's similar -- if you have uv coordinates from 0 to 1, you just need to make sure shift_rate*time_max is an integer in order for the scrolling to be perfect when the time resets. Similarly for the time parameter used to index a flipbook animation.

So the reason the fog is snapping after you play for a while is that the time sent to the shader system gets reset to 0 every 1000 seconds but the fog doesn't know about this. And the reason for the resetting is to make sure we maintain good precision in the time-based effects in the shaders.

The Witness requires Metal for rendering. Metal is a graphics technology from Apple that allows faster and more fluid graphics. These are the Mac computers that support Metal:

• MacBook (Early 2015)
• MacBook Air (Mid 2012 or newer)
• MacBook Pro (Mid 2012 or newer)
• Mac mini (Late 2012 or newer)
• iMac (Late 2012 or newer)
• Mac Pro (Late 2013 or newer)

https://support.apple.com/en-us/HT205073

It's possible to build custom Mac systems with Metal support, either using an old Mac Pro and upgrading the graphics card or building it from scratch using compatible parts (Hackintosh). These configurations are unsupported, but some customers have found them to work.

If using an AMD GPU, the only supported add-in card with Metal support is:

AMD Radeon 7950 (Tahiti Pro) Sapphire card.

If using an NVIDIA GPU, any GeForce 6xx or newer should work, but it is necessary to install the latest drivers from:

http://www.nvidia.com/object/mac-driver-archive.html

System Requirements:

Minimum:

* OS: El Capitan 10.11.6
* Processor: Intel Core i5 @ 1.8 GHz
* Memory: 4 GB RAM
* Graphics: 1 GB of video memory
- Intel: HD Graphics 4000
- AMD: Radeon R9 M290
- NVIDIA: GeForce 640M
* Storage: 5 GB available space

Recommended:

* OS: Sierra 10.12.4
* Processor: Intel Core i7 @ 2.4 GHz
* Memory: 8 GB RAM
* Graphics: 2 GB of video memory
- AMD: Radeon R9 M390
- NVIDIA: GeForce 780
* Storage: 5 GB available space

The list of Apple systems with AMD graphics that support Metal is:

Mac Pro 6,1 (Radeon FirePro D300, D500, D700) - cylinder
iMac 15,1 (Radeon R9 M290, M295X)
iMac 17,1 (Radeon R9 M380, M390, M395, M395X)

MacBook Pro 11,5 (Radeon R9 M370X)
MacBook Pro 13,3 (Radeon Pro 460)

The list of Apple systems with NVIDIA graphics that support Metal is:

MacBook Pro 9,1 (GeForce GT 650M)
MacBook Pro 10,1 (GeForce GT 650M)
MacBook Pro 11,3 (GeForce GT 750M)
iMac 13,1 (GeForce GT 640M, 650M)
iMac 13,2 (GeForce GTX 660M, 675M)
iMac 14,2 (GeForce GT 755M, 775M)
iMac 14,3 (GeForce GT 750M)

To celebrate, the game is on sale for 50% off on Steam, here: http://store.steampowered.com/app/210970/

Braid is also on sale, and if you already own it, you get an extra discount on The Witness: http://store.steampowered.com/bundle/2501/

Meanwhile, we are working on The Witness for MacOS, and it should be done pretty soon. iOS is also coming up but is a bit further in the future.

Happy 2017!

(That is all!)

That was a work-in-progress, and I said at the time I'd post final specs as soon as we know them, which we now do.

We are definitely supporting HDR, on both the PS4 Pro and the base model PS4. If the game detects an HDR TV, it will start in HDR mode. HDR works in all resolutions mentioned below.

If you have a PS4 Pro, and are outputting to a 1080p TV, The Witness will render at 1080p, 60fps, 4x MSAA (which is substantially higher than the base PS4's 900p, 60fps, 2x MSAA). This is the same as we announced in the earlier posting.

If you have a PS4 Pro, and are outputting to a 4k TV, then you get a choice of two modes from the options menu:
(a) 1440p 2x MSAA rendering of the game world, upscaled to 4k, text and UI rendered at 4k, 60fps
(b) 4k 2x MSAA rendering for everything, 30fps 

We added option (b) because a lot of people seem to want it, and it does look really good!

I don't yet know the exact release date for the patch, but it is coming pretty soon, in November sometime. We are now in the process of nailing down all the little details and making sure they are good. As soon as we know the release date I will announce it here.

The Witness already runs at a consistent 60fps on the base PlayStation 4, so rather than increasing frame rate, our patch is about visual quality. Please don't take the specifics mentioned here as absolute promises; things might change by release. But here's what we are aiming for:

When you start up the game on a PS4 Pro, it will go into one of two modes, depending on the current display device:

* When using PS4 Pro with a 4k display, we will render at 1440p or possibly higher, all while keeping a consistent 60 frames per second. This image will then be upscaled to 4k, and we will draw text, menus, and other UI at native 4k resolution. (This is similar to what happens on the base PS4, where we render the 3D scene at 900p, then upscale to 1080p and draw UI at 1080p). We might be able to achieve resolutions higher than 1440p based on various engineering tradeoffs (for example, if we boost the resolution of the main scene to a greater degree than we boost the resolution of the reflections in water, which tend to be distorted anyway. Also, we'll evaluate checkerboard rendering -- we're not sure if it's the right way to go for this game, but we'll see.) We'll post again when we finalize these specs.

* When using PS4 Pro with a display that is 1080p or lower, we will render the scene at 1080p (instead of the old 900p), and increase antialiasing quality from 2x to 4x MSAA, again while keeping a solid 60 frames per second. We are exploring additional visual quality increases, like increased streaming distance for high-resolution meshes, or decreasing the noticeability of the fade-in between faraway LODs and high-res close-up meshes. Again, we'll post details when this is finalized!
We're also working on HDR support. Here in the office we just got a new LG B6P television, which is very bright and has low black levels, providing a lot of dynamic range, and it's been fun so far tweaking the look of the game on HDR displays like this.

We thought we were working on HDR support only for the PS4 Pro, but yesterday Sony announced that they will be enabling HDR support on all PS4s via a firmware update. Sometimes Sony even surprises developers! (Back when the PS4 was originally announced, we were surprised by some of the final specs too ... and I was on stage in that show!)

We think it's great that Sony are doing this. We don't know the technical details of how it will work, yet, but provided that nothing prevents us, the Witness patch will use HDR on all PS4s when available. (You will, of course, need an HDR TV or monitor for HDR mode to kick in!)

If all goes well, this patch will be downloadable on November 10 when the PlayStation 4 Pro is released.

We'll keep you posted as we continue work on this patch!

Click here to view the video on YouTube.

 (Alcohol Reference)

This will be Friday, February 12th, starting at 1pm Eastern (10am Pacific) and going for ...  a while?

I have made a new reddit account with the fabulous name of "Jonathan_Blow" for these purposes. (Note the underscore! There are some other accounts with similar names!)

Maybe we can also get some other people from the team in there answering questions.

The thread is now up.

There is a wrinkle, which is that I can't be too precise about the sales results on specific platforms, because often when we sign a deal with a particular store, we agree not to reveal their sales numbers. I am actually not sure at this time which stores we are allowed to be specific about and which we aren't, and I would have to dig up and sort through a number of contracts to be sure about it; but that is not a good use of time right now, since I am spending most of my day supporting the users who have technical problems (shipping games on PCs these days is really not fun). The situation can be thorny in subtle ways, too, because if I post information about all stores but one, then I am implicitly revealing the sales figures for that last store, which is not allowed.

Finally, I want to make clear that we did not make this game in order to make money. We were trying to build a beautiful / interesting / intricate thing, first and foremost. The money just helps us stay in business in order to build new things. It is very easy on the Internet to read a financial posting like this cynically, so I urge folks out there not to do that.

Okay, so here's what I can say:

Across all platforms, The Witness has totalled over $5 million USD gross revenue in the first week, and it has sold substantially more than 100,000 units.

This is a good chunk more revenue in one week than Braid made in its entire first year, from August 2008-September 2009. (Braid initially launched on XBLA in August 2008, and it came to Steam in April 2009). Braid was considered a hit independent game at the time.

We can also compare sales by units instead of revenue; this is a little more of an apples-to-oranges comparison because The Witness has a higher price than Braid did (Braid launched at $14.99 [$16.50 when inflation-adjusted to 2015 dollars], and The Witness launched at $39.99). By number of units, the first week of each individual platform handily beats Braid's first week of sales. (Witness on PC by itself beat Braid's first week by a decent margin, and Witness on PSN by itself beat Braid's first week by a decent margin, counting only by number of units). This is great because as price goes up, naturally the number of units sold goes down. So the fact that we beat Braid by units, more than doubly, is a really nice success.

The Witness launched on two platforms, PSN and Windows PC. Neither of these platforms dominates our sales; PC is very strong for us, and PSN is very strong for us.

There are some publicly-available guesstimates for specific platforms on sites such as SteamSpy, but the numbers that SteamSpy is reporting for The Witness are a bit too low (though this is kind of to be expected, I guess, from the way that site works).

So, the game is doing great. That doesn't mean we have broken even on our development cost yet! Because our development budget was so high, $5 million in revenue is not enough to recover it yet (because we split that revenue with the storefronts, we have to subtract VAT in Europe, etc). However, it is looking like, as time goes on, we should break even and make a comfortable safety margin on top of that, which will allow us to make more nice games in the future -- unless some kind of world economic disaster happens.

As I mentioned, right now we are dealing mostly with PC graphics driver problems, and we are also working on adding some features to the game about configurable controls and rendering options, for PC and PS4.

After this, in the near future, we will start investigating the bringing the game to other platforms. Under serious consideration are: iOS, Android, Xbox One, OS X. We will provide more-concrete information about these as it becomes available!

It should go up on Steam this morning around 9 or 10am Pacific time.

I made a posting on the PlayStation Blog: https://t.co/ChsVlBzhVw

Thanks to everyone who's been following along on these forums, and everyone who's been interested in the game generally!

We'll see what happens tomorrow!

A lot of people have been asking about preorders too. Preorders will go live on Steam and the Humble Store (and on this web site) today at noon PST. Unfortunately we won't have preorders on PSN, because apparently developers have to set those up months in advance and we didn't know that!

The preorders are regular price! They contain an amazing bonus pack-in: the warm fuzzy feeling that you have pre-ordered the game. (And that's all). So it's exactly the same as buying the game at launch, except earlier.

If you buy from the Humble Store, 10% of the price goes to the Against Malaria Foundation. For a lot of us in developed nations, malaria seems like a far-away thing that you only think about when you go on a really adventurous trip, but in many parts of the world it's a serious problem. The good news is that we are making great progress in reducing incidence of malaria, see this report:

The Against Malaria Foundation is rated one of the most cost-effective charities in the world by both Giving What We Can and Givewell, two organizations focused on ensuring that charitable donations do maximum good in the world. Giving What We Can has a good high-level description of the philosophy behind this movement.

The Witness deliberately riffs on several traditional elements of game design, and one of those elements is the classic audio log, where you are wandering along and you find a recording that was placed there by someone before. The Witness has recordings like this, but let's just say they are done in a highly non-traditional way. When you first encounter them you won't find much in the way of answers, but, if you keep looking, you will eventually be able to piece together what's going on.

These are the folks whose fine voicework you will hear in these recordings:

Ashley Johnson, who you may know as Ellie from The Last of Us, or from her roles in Tales from the Borderlands and Infamous First Light. She also plays the smartest person on the hit TV show Blindspot. You can find Ashley on Twitter.

Phil LaMarr, who I best know as Marvin from Pulp Fiction, but who you may know as Vamp in Metal Gear Solid 2 and 4, or as Ratbag from Shadow of Mordor. He also played Hermes in Futurama and Ollie Williams in Family Guy! You can follow his random musings here.

Matthew Waterson, who played very English person Sir Lucan in The Order: 1886, an has recently acted in Fallout 4 and Halo 5. Check out Matthew's amazing Sean-Connery-themed profile picture.

Terra Deva, an oft-performing musician, including recent hit single "At Night" with the swiss group Shakedown (though she doesn't play music in The Witness because we don't have music in The Witness! She is acting!) and veteran of... The Mickey Mouse Club!!!! (Among many other places!) Terra's Twitter page is here.

We recorded these sessions at Warner Bros. Studios. Our voice director was Liam O'Brien, who has also done quite a bit of acting in games himself. Pierce O'Toole acted as the great force of organization throughout the process and many things would have gone worse were it not for him!

Having said all this, I will warn you that the story in The Witness, such as it is, is fairly subtle and happens mostly implicitly. You really have to explore and read between the lines to know what's going on. So do not expect a traditional game story like The Last of Us or The Order! All that said, we are very happy to have these actors lending their talents to our game, and I look forward to release when you'll see what we've been up to.

Well, I think we will post a little more frequently, because we'll have increasing amounts to say about the game as we get closer to release. In the meantime, here's:

(It is longer than Long Screenshot #1!)

Voiceover

This week we just finished recording the majority of the final voiceover. We're going to do a little more in early January, but most of it is in the can. After New Year's we'll make some postings where we announce who the voice actors are (you may know some of them from other recent games!)

Physical Box Release

We've had some people asking us whether there will be a physical retail release of The Witness. It's seeming likely, as we have been in talks with a couple of parties for a while about that possibility. *However*, it looks like if there is a retail release, it won't happen until a little bit after the digital release. The reason is just that we need all the time we can get up until the release date in order to get all the finishing touches into the game; but because of the lead-time involved in physical manufacturing and distribution, we would have to freeze the game right about now in order to have it on store shelves for the release date. *Or* we could delay the digital release date in order to wait for the physical. But I don't see a good reason to do either of those things. I would rather just focus our energy on making the game as good as we can make it, and a retail version can lag behind by a little bit.

What Else is Going On

We've gotten the bulk of the translations into the game for all the languages mentioned in the previous blog posting. There is a small percentage of text that will come in a little later, but for the most part those languages are in. Also, we will be adding Hungarian to the list of languages suppoorted at release.

Here are some screenshots of Arabic and Russian:

Of all the languages, supporting Arabic took by far the most programming work, and as you can see from some gaps between the letters, and a couple of cases where letters are too close to each other, it's not quite done yet. I will be working on that as soon as I finish with this blog posting! But I think even with these issues it is starting to look nice.

We're doing *lots* of little gameplay fixes all around the island, like, what does collision feel like in this area when you bump into things, what happens if you are standing in this doorway and the door tries to close on top of you, all that kind of thing.

Visuals are mostly locked down at the moment; we are no longer changing big things about how the game looks, we're more making spot-fixes here and there every time we see something we don't totally like.

Music and Sound Effects

Once in a while I get a question, from someone who really liked the music in Braid, what the soundtrack in The Witness is like. The answer to this question is that there isn't one. There is (almost) no music in the game. This is not an arbitrary decision, but is in fact very important to the coherence of the thing we are making.

The Witness is a game about being perceptive: noticing subtleties in the puzzles you find, noticing details in the world around you. If we slather on a layer of music that is just arbitrarily playing, and not really coming from the world, then we're adding a layer of stuff that works against the game. It'd be like a layer of insulation that you have to hear through in order to be more present in the world.

Instead, we put a great deal of care into the sounds of the world around you, in a way that maximizes immersion in the game. This is made trickier by the game's setting: you are alone on this island, and there are not even any other animals. There are no birds in the trees! In everyday life if we imagine the sound of nature, we'll think of some elements that have no place on the island: a forest naturally has the sounds of birds, plains with smaller shrubbery will have crickets, a marsh will have the sounds of many insects. There's none of that in this game because in this game you are really alone, and it has forced us to be very creative with the audio in order to ensure things have depth and texture to them. This work is being done by Wabi Sabi Sound, who did the sound for the very atmospheric Dead Space series, and more recently some smaller, artier games like Ori and the Blind Forest.

We thought it would be fun to release a few more small videos to give you more of a feel of what the world of the game is like, so here's:

Story and Voicework

Lately I have been doing a lot of work on story and voice acting stuff. When wandering around the island, you may find voice recordings that were placed by ... well, at the outset of the game, you don't know who; and as the game goes along, there is an interesting mystery to unfold about who these people are or were. In the game currently we have what is known as "temp voice", which is me reading all the parts under very noisy conditions and just getting them into the game so we can make sure the data works properly with the engine, to get all the subtitles into the game and roughly synced with the audio (which we need to do sooner than later so we can give the translators those subtitles, which they will translate into many different languages).

We have cast the real actors and starting tomorrow we'll do our first test readings with those actors. We'll use this to get a new layer of temp voice in the game, which will get it closer to final; this will allow me to see how the game really feels with this stuff in there, possibly to adjust the nuances of the way the recordings appear in the game and the way they are played back. When everything feels good we'll do a set of final recording sessions. There's not a whole lot of time until release, and we have several holidays coming up, but we'll get it done!

We are working with Warner Brothers Game Audio in Burbank, California to do the voice stuff; they are providing the facilities and voice direction and production assistance. After we do a few recording sessions I will have more details to share in this department.

Localization

Earlier, I mentioned subtitles. As many games do, we will be localizing our game so that people who speak many different languages will be able to understand the words. We are not recording voice acting in different languages, mainly because it is pretty hard to make sure quality is high when you do that. It is hard enough making sure acting is good in just *one* language. We generally seem to feel the same way about movies: a foreign-language dub of a movie is generally considered to be inferior to a subtitled version, because the acting is so important.

Our localization efforts will involve menu text and subtitles, just as for a good movie.

This is not 100% final, but here's the list of languages we are planning to have translations for, at launch:

• English
• Italian
• French
• German
• Spanish (Spain)
• Spanish (Latin America)
• Portuguese (Portugal)
• Portuguese (Brazil)
• Arabic
• Russian
• Chinese (Simplified)
• Chinese (Traditional)
• Japanese
• Korean
• Polish
• Indonesian

We may add more languages later but man this stuff is expensive, so there is a limit to how many languages we can do at once.

We will have more news for you before too long!

You can read it here.

Here's my update on the PlayStation Blog.

Also, we have some interviews and previews of the game:

Gamespot
Ars Technica
Polygon
Wired
PlayStation Blog

The PlayStation Blogcast had a nice ten-minute discussion of the game.

It will not only show the game in a further state of visual development than the previous trailer (which is two and a half years old now -- wow!), it'll also show lots of scenes you probably haven't seen before.

The trailer isn't done yet, but here are a few stills:

Ignore the debug text in the upper-right (including the frame rate; these were taken on a monitor that's locked at 30Hz; the game runs much faster on higher-end PCs.) Also, the actual trailer will be produced at much sharper image quality and higher resolution. These are just snippets of the non-final footage we are using to figure out the composition and flow of the trailer.

It does *not* mean that the game is done. We still have a lot to do! But it *does* mean that the nature of the work changes and becomes simpler, because we don't have to be making high-level creative decisions any more. It is now much more about turning the finish-the-game crank (making sure stuff plays well and polishing it up) for anything related to game design, modeling and texturing.

There are a few categories that still need creative decisions (story stuff, the menu system, etc), but for those of you who are awaiting the game's release, this is pretty good news.

There are currently 677 puzzles in the game. That number might be slightly different when it ships (we might cut a few, for example!)

We've been revising the modeling and texturing for areas near the end of the game. There are some nice treats in store!

We've been working on some of the "special effects" in the game, most notably particle effects. Our particle system was old and slow, so we are speeding this up, and then we will make some modifications to help make particle authoring an easier process.

We've been making some baseline functionality smoother. For example, so that you don't need to think about saving your game, The Witness autosaves your progress every minute or so. On a high-end PC, you don't notice the autosave, but on a low-end PC or on a console (because consoles tend to have different memory access properties than PCs), scanning the world for stuff that needs to be saved can be pretty slow, like, it could take 1/10 of a second, which would result in a hitch in the frame rate happening every minute. So we are hammering on this to make it faster, which should also result in saved game files being smaller, since we'll be excluding a lot of things that don't need to be saved.

Ignacio has posted the conclusion of his series on irradiance caching -- at long last! -- and released the source code.

This code is a significant chunk of what makes lighting in The Witness as nice as it is.

This is of course a very crude mockup of what the new starting area will be. It's smaller than the old one and should work a little better play-wise. We have yet to reshape the terrain around it, so there are huge holes. Our deadline to have this area fully rebuilt is the 6th of June!

Current puzzle count is 627.

http://mollyrocket.com/casey/stream_0019.html

Meanwhile, we've had a lot of Witness Wednesday posts in a row on this blog. I'll have to stop slacking and write something about the game.

Here's the blog post.

Here's the video:

Update: And here's another post, complete with a video showing in detail how the grass planting works in the actual game.

Working on The Witness, Part 5

Working on The Witness, Part 6

In this one he talks about how to distribute grass. Legally. As well as the different colors of noise one might want to be familiar with in order to do this well.

Last year I was pessimistic about VR systems in the near future, since the ones I had tried didn't seem to offer much (certainly not enough to justify wearing a bulky headset all the time and fighting back feelings of nausea).

But I was fortunate enough to get a sneak peek at Valve's virtual reality system. It is so much better than anything else I had used that I was instantly very excited by it. Right away I could see games you might design for this system that had been impossible before; so it isn't just a matter of the system being very immersive (which it is) but that it can take video games in general to a new place.

We worked with Valve to get support for the system in The Witness, and that's where those screenshots come from (but when I took the shots, an HMD was not plugged in, and as a result what you see is a dummy calibration that does not correspond to any useful image-warping; the screenshot was really more about just being happy about the stereo display capability.) In late November I spent a few days at Valve's offices working with Atman Binstock and Doug Church to adapt the HMD's input system more thoroughly for The Witness and to build a few scenes that are playable on the device.

(In theory the whole game is playable on the device, but when you are rendering for VR the performance demands become much higher, because you are rendering two scenes at higher resolution than you might previously have been rendering one scene. So the full game world with all content currently does not run fast enough for this device, but that is the kind of problem that gets solved over time).

We had to keep this under wraps, though, because Valve's system was not announced. But this month at Steam Dev Days they revealed details of the system.

For more details on the Valve system, Lee Vermeulen wrote up a nice post of his impressions:

http://www.3delement.com/?p=332

So, we will be supporting this device (and any similar devices) with The Witness. But I am really looking forward to see what gets designed specifically for devices like this.

Luis has been doing a lot of work lately on the mountain and especially the stuff at the top.

Other folks have been doing a bunch of work on things that are not as obvious in this picture!

Current puzzle count: 595

Click here to view the video on YouTube.

(I didn't name this speech; the title given here was picked by Creative Mornings after the speech was given.)

Click here to view the video on YouTube.

Discussion on PlayStation Blogcast (Witness talk starts around 26:25):

http://blog.us.playstation.com/2013/10/18/playstation-blogcast-094-witness-the-daylight/

(From the way the PS guys talk, it sounds like it is all me making this game, but as always on games of this size, there is a whole team making it happen!)

Interview with Gametrailers, also from the PlayStation event (you may have to sit through a commercial with army dudes to see it):

http://www.gametrailers.com/videos/aeeuwu/the-witness-the-puzzle-of-making-puzzles

The Witness was among the games being shown! We were the only third-party game there. Also shown were a whole lot of Killzone: Shadow Fall, Knack, PS4 versions of Sound Shapes and Flower, and Resogun.

I did some interviews and talked to a number of press people who are very interested in The Witness. I also took some photos.

Brad from Giant Bomb playing The Witness:

The room we were in:

Flower at 1080p and 60Hz, with the new, more-sensitive controller, feels nice to play:

Sound Shapes:

Normally I would post an Island Snapshot, but the viewpoint we have been using for Island Snapshots is a little bit unfortunate lately, since the main thing taking up the view is the mountain, which is an area we haven't worked on modeling-wise lately. (It is a big area so we are going to do a concentrated push on it in a little bit, when some other things get locked down.)

So, here are some screenshots of other stuff we have been working on lately. Both areas are in progress and will look better than this in the final game!

A lake that's central to the island...

A swampy area with light buildings and walkways. The vegetation you see here is temporary! But I am happy we are finally starting to work on this area; if you look at it in the trailer, for example, it's obviously very much more temporary.

It's been revamped by Alex, who is new on the team, helping us with the huge amount of modeling we still need to do to finish the game. Hopefully we'll get a bio paragraph for him soon on the About page.

The current puzzle count is 590.

The current footstep count is 1756.

P.S.: Last week we showed The Witness at E3. This was the first time we have ever shown the game running on the PlayStation 4. We got some nice write-ups about the game; here are just a few:

Steven Hansen from Destructoid really enjoyed the game!

Tara Long from Revision 3 really enjoyed the game! (She also gave us an award at the show!)

Darren Franich from Entertainment Weekly put us at #1 on the list of most exciting games of E3.

Matt Miller of Game Informer seemed to really like the game, and put together a list of defining features.

Chris Leggett of Gameplanet says The Witness was his personal favorite game of E3. (The folks at Gameplanet who chose the photos to go on the article used some unfortuante, super-old development photos though!)

Tom McShea and Shaun McInnis of Gamespot gave a video summary from the show, seeming to like the game a lot as well!

If you watch carefully and don't blink, and look past all the balloons that are blocking it, you can see the poster (in different proportions) starting around 0:57 in this commercial: http://www.youtube.com/watch?v=QOdW1OuZ1U0

Hey, it is what it is.

Because we picked this one as the final, we have it in a much higher resolution than the other options. If you like it, feel free to use it as a desktop image, or whatever!

... which doesn't look too much different from last time's. Partially this is because we've been working on things that you don't see from this angle.

Here's a shot from a different angle:

We still have so much to do!

On my end, I've been working for the past couple of weeks on UI-related things. For a week I was doing better touch controls (perhaps we'll post a video of this soon), and for a week I was somewhat-rewriting the code that implements tracing lines on panels. This can get surprisingly complicated, and I've rewritten it a few times. I think this is close to the last time, though.

The game tells me the current puzzle count is 574, which is higher than I expected, but hey.

Feel free to use these as desktop images or whatever. Unfortunately we didn't mock these up at high resolution; what you see is all there is!

Most of what we've been working on lately is not visible in this shot, though! The biggest obvious changes are probably in the forest and beach on the left-hand side.

There are some basic hacks going on; for example, right now, we are not drawing the real sky; it's just solid blue. Also, right now we are just loading this starting area, but it should not be long before we can load the whole game (Salvador wrote the streaming system a while back and we have been testing it extensively on the PC, so for the iPad it should Just Work. Famous last words.) Also, as you can see, the panel-tracing controls could use some work.

When we can wander around the island generally, I'll post another video update!

Well, you got a pretty good look at the island in the trailer we recently released in conjunction with the PlayStation 4 announcement. That version has a lot of new modeling compared to the January snapshot, because we were working on the trailer pretty hard. But some new modeling has happened since then, and there are some subtle but nice things to see here.

Since someone is going to ask: The current puzzle count is 499.

Our deal with Sony is a limited-time exclusivity that applies to competing console platforms. Basically it is that you will see the game on the PlayStation 4 for a while before you will see it on the Wii U or the rumored next Microsoft console. (We haven't been disclosed on Microsoft's next console, so I don't know anything about that platform besides the rumors that are on the internet).

This exclusivity does not prevent us from being on the PC or iOS at launch, for example.

People have been speculating that we got paid for PS4 exclusivity, but that is not the case. There was no money involved.

The real situation is that, because we are a small developer, we only have the ability to launch on a small number of platforms at once. We liked the idea of being on a console, and originally we thought we might be on the PS3 or Xbox 360, but eventually we decided not to target either of those, due to the relatively low system specs and the work required to do the ports. After some more time went by, and our release date drifted further into the future, we realized that the next-generation console launch time might be a good time to release the game. (For a while we were hoping to be out substantially earlier than the next consoles, e.g. right now, so we didn't start thinking seriously about this until recently).

To launch a high-spec version of the game on a console, it came down to a choice between the next Microsoft console or the next Sony console. There were people at Sony who really liked the game and were keeping in touch with us about it, and so we naturally started going to their PS4 developer events, got a dev kit, and started playing with it. I don't have good communication with anyone at Microsoft right now, but all our technical people like the PS4 specs a lot more than the leaked Durango specs, and we like the positioning of the PS4 (it's about games) more than what we perceive Microsoft's positioning is going to be.

So we just found ourselves doing a PS4 port. And once we are doing that, it is not practical for us also to think about another console port. We were going to be a de facto console exclusive for Sony no matter what. As I mentioned, there are people at Sony who are very interested in The Witness, so they somehow percolated up through the ranks the idea of showing the game in the launch show.

Of course, Sony wants that show to point out things that are going to be exclusive or special to their console. If we are in their show, but next week we turn around and announce "we changed our mind, we are exclusive on Durango forever," that would be a bit weird. So to be in the show we signed a timed exclusivity for competing consoles. But this was just a formalization of something that was already going to be the case. We like the PS4 and we like the people at Sony we are working with, so it was an easy choice to make the agreement.

Exact timing of our release on launch platforms has yet to be figured out. It just depends on how much we can do and when; we'll know more as we get much closer to the game being done.

(Watch it in HD + fullscreen if you have the bandwidth!)

Here's the interview.

Click here to view the video on YouTube.

The game's web site.
The Steam page.

The game is primarily the work of one designer, Alexander Bruce, who has been working on it for six years. Players seem to be really into the game, and it's been the top-selling game on Steam for two days so far.

Up until now we have not been in that kind of work mode, because we were figuring out the game, and in that kind of mode, I find schedules to be anti-quality. But once you have everything figured out and need to produce a volume of work, schedules are pro-quality, since they help you gauge how well you are doing, whether you need a reality check on the rate or quality of things you can produce, etc.

Luis has made this handy map that we are using to partition up modeling tasks, which I have then edited to remove spoilers:

I like looking at this map because it reminds me how many distinct areas there are and how they relate to each other.

Today I was showing the game to someone who had not seen it before, and that's always interesting because it helps me see the game through fresh eyes a little. It really feels unlike any other game in the way it is an Open World but also a Small World, where you can be in one location and see several neighboring locations, and that helps keep you oriented, but it also helps you reflect on what you have done and orient on where you are in the course of things you are planning to do.

We still have a ton of work to do, though, and I was reminded of this as well. Not just in detailing all these areas, but in filling out the story, crafting details that further help the areas relate to each other, etc.

It is going to be an interesting time!

Jeff asked me about this part of the iOS App Store's Developer Terms of Service:

We view Apps different than books or songs, which we do not curate. If you want to criticize a religion, write a book. If you want to describe sex, write a book or a song, or create a medical app. It can get complicated, but we have decided to not allow certain kinds of content in the App Store.

And here is what I said:

I think this is the wrong attitude about games, but look, ultimately it is game developers' fault, not Apple's. Apple is treating games as shallow commercial entertainment experiences because they have been taught by game developers that that is what games are.

If we had built a world where games routinely work with serious issues in ways that people care about, Apple would not be able to take this stance, because it would not make any sense.

Why do they say "If you want to criticize a religion, write a book"? It's because it's obvious that banning books is bad, because there have been a lot of books that people find important (and we have had many cultural cycles involving people attempting to ban books, and culture has worked out ultimately that this is not a good thing). Games do not have this history. Right now Apple thinks a game is Angry Birds or maybe Infinity Blade. So who can blame them?

Apple may be badgered enough to change their policy someday, or they may not. But that doesn't matter very much because really it is just a reflection of the general cultural idea about what games are. The only way that idea will change is if a lot of developers make a lot of serious/deep/honest/touching/intrepid games for a long time. I don't know if that will ever happen. How many games can you think of to which you can seriously apply these adjectives? Certainly to none of the top-selling games on the App Store, and certainly certainly to none of those big free-to-play games that are raking in all the cash.

So, game developers are just sort of reaping what they have sown. What else would you expect?

I have not played Endgame: Syria but maybe it is a step in this direction. But to change these attitudes we need a lot of steps, consistently, not just a token step now and again.

Current puzzle count: 505.

The Background

Way back in 2008 when I came upon the core idea of this game, the design was the exciting part. Though I started in games mainly as a technical person who was excited about technical challenges, I have shifted into a mode of thought where, when I program, it's mainly just to get the design done in the most straightforward way.

Since Braid I have viewed programming as mostly pragmatic: I know what I want to do, and I know what kinds of things I have to type in to make it happen, but for the most part, the interesting mental challenges are in design, and programming is rote execution. Once you have enough programming experience, I think 95% of writing a big program is like this. There have been a few times during The Witness when I spent significant effort solving technical problems where I had no idea what the answer would be in advance, like with the terrain manipulation, but even in these cases I treated the technical development in a very pragmatic way: the goal was to get something that is good enough for current needs, not to design a shining jewel of an amazing technical system (back when I was a younger game programmer, I was trying to make amazing technical systems all the time, and it came at the cost of actually getting games done).

Despite the fact that I am trying to focus on design, in order to make this game a reality, I had to start a company, bring people onto the project to build the game, etc. This means that nowadays I am not just wearing the programmer and designer and business development hats, I also wear the producer hat and the (admittedly small at present) HR hat. Maybe if The Witness does well, we can expand the team a little bit so that I don't have to do these things, but for now, that's the reality.

We are very laid back and unstructured in the way we do things, but even so, I feel there's a heavy subconscious load involving running the company, making sure nobody's work is blocked, dealing with money all the time, etc. I often find it hard to do deep design thinking when I am in the office, if only because all these issues are just ambiently around. So, in terms of design, my most productive days tend to be work-from-home days (which for me includes most weekends).

All this up til now is just to explain that if I take a vacation seriously, it can be very relaxing, because that subliminal load is gone, or at least mitigated for a while. The activity of the rest of the company is mostly paused, which removes urgency from my own activities.

At first I didn't know how I would spend vacation; at first I thought maybe I would actually treat it the way most people treat vacation, and not work. Then I thought, hmm, there is a certain sector of the game that I might like to work on, that is definitely very important, and I should get a bunch of that done while everyone is gone, so that when they come back it will be figured out. (I don't want to go into detail about what that section of the game is, because I am avoiding spoilers).

The Vacation

But, inspiration is a funny thing. I worked a little bit on the issues I had planned to work on, but for the first bit of the vacation, up through the 25th, I found myself most motivated by cleaning up old nagging issues on my to-do list, and shipping this improved version of the game to our very few beta players. This involved stuff like improving the feel of tracing lines on the puzzle panels.

A peek at the export script, used to package the game for end-users.

After shipping that beta, I found my attention most strongly drawn toward an unexpected part of the game: an optional area, one that we could completely cut from the game and nobody would know, yet at the same time, it's an area that adds to the overall conceptual arc of the game in a strong way, and would be very rewarding for the small percentage of players who get deeply into the game. (Again, I apologize for the vagueness, but let's avoid spoilers).

I had roughed out a version of this area a long time ago -- maybe a year or two ago -- and had always found the basic idea very exciting. But back then the area wasn't done enough to prove the concept, and over time it decayed while the rest of the game was being worked on (which is natural on a big project like this). But, perhaps because it's an area of the game that nobody else on the team really knows about or understands yet, I had a great motivation to spend vacation time on making this area good. Now that I think about this again it makes a strange kind of sense: when development is pretty much paused, it is somehow appropriate to work on something that nobody else really knows about.

At first I thought the task would just be to make the old stuff work again and then do a straightforward job of the extra functionality I needed to complete the area. Thinking about the programming I would have to do to complete the area, I saw it as a necessary bummer ("I want to be done with this game, but ugh, there is more programming I have to do that isn't even on the mainline to all the things that really need to be done.")

But I started programming that extra functionality, and 10 minutes into it, realized I was not so excited about the design part -- it seemed like a very rote execution, like anyone could do in any game, and The Witness ought to have something better than that. Because there was no pressure, and because I was free to do whatever I wanted, my mind was open enough to instantly see an alternative that would be much truer to the essence of the game. The chance to do this better thing was very exciting. (We'll come back to this subject of being able to see ideas, later).

So I started programming this more-exciting version of the functionality. Then I saw how I could make it even more interesting, and I would start doing that. Then I would see how to make it even more interesting, though that would introduce a lot more programming work. If I were feeling the regular pressure of the game schedule, at this time years in when it is time to be finishing the game, I might have said, "naah, we don't have time add that extra work". But because it was vacation the work was free, so I said screw it.

I worked in a fiercely productive way for a couple of days. New ideas would come. They would raise new issues. I would program some things, and in that way that programming often goes, new problems would come to light that I hadn't thought of, and those problems introduced even smaller 2nd-order problems when confronted. But by this time I was fully energized and just had dogged determination and was fully enjoying the whole process. In the end I programmed at least 5 times as much functionality as I had planned, but rather than being bummed about it, I enjoyed the whole excursion.

Those who know me will tell you that I am not predisposed toward painting things in an unduly positive light -- but I will tell you that these past four days or so have been amazing and wonderful. Here's a shot from what I did (the visuals are 'programmer art', as usual):

It is hard to tell you why this is so cool without huge spoilers.

It's so amazing and wonderful because

it's been a long time since I worked this hard and this effectively. Keeping in mind that memory is sometimes not as reliable as we think, I would tell you that the last time I worked this hard and this well was when starting my first, ultimately doomed, company (with a friend from college and a 3D artist we recruited from that same school and unfortunately put through years of meager-wage toil).

Here are some shots from the first professional game I ever worked on, back before we had a concept of 'indie development'. Software rendered, subpixel-accurate texture mapping, assembly-language inner loop that commandeers ebp in order to utilize a precious extra register, a la Mike Abrash, and oh yeah, it was multiplayer-only, 32-player, over 9600 baud modems:

We were working on this game from 1996 to 1999 or so. 1996 was the year I got into professional game development, and in hindsight, it appears to have been the hardest possible time in the history of video games. Indies of today, y'all have no idea how much easier it is to make games now than it was in 1996. (I hate to be an old fogey who's all "back in my day, we had to walk to school in the snow, barefoot, uphill both ways," but in this case it is absolutely true.)

That game was never very successful. It had design problems, but also, we didn't really know how to talk about or promote a game; lastly, this was before many people played games over the internet, so the potential audience was pretty small. (Trivia Tidbit: There was another online game being supported by the same publishing service as our game, and this other game was more popular, despite being relatively simple in a technical sense, which annoyed the hell out of us on a subliminal level that we wouldn't totally admit. The guys making this game were John Vechey and Brian Fiete; our producer at the publisher was Jason Kapalka, who later went off with these guys to start a company called PopCap, which last I heard had done okay.)

Back in those days I worked very hard. When we started the company I knew nothing about building 3D games; we learned very quickly. I would long hours, late into the night, intensively. At some point we all burned ourselves out, I think.

After we closed down that company, I went on to do a number of contractor things, game industry lecture things, magazine article things, and working on various independent projects that I never finished. Through that time, I was still very burned out. If you have never experienced burnout, it's hard to explain. Burnout is not just "I don't feel like working right now"; it is about your mind refusing to permit you to work, because it has seen what happens when it lets you work.

Eventually, I swore that I would design a very good game and finish it and release it to people, and the result is a game that was successful and that many people like:

A main part of the reason this game exists was that I instilled a certain discipline in myself: I wasn't going to do this game because I was excited about a technical thing, or excited about programming; I was going to minimize the amount of programming, firstly so that the game would get finished at all (I had plenty of experience by this point at not-finishing-projects, as did many of my friends; it was an endemic problem), but also so that I could focus appropriately on the design and build something very special.

Despite the fact that I kept the game technically simple, there was still a huge amount of programming to do. But thankfully, the strategy worked and the game was completed. Eventually.

Even though I'd made a successful game, I was still burned out to some degree. In 1996-1997 I was working at full capacity and was very productive, at least inasmuch as one might expect for a programmer of that age who was first starting to get real-world large-project experience. But after being burned out at that time, I never again worked at full capacity.

What I am getting at

I am saying all these things just as a way to get around to explaining that, during this Christmas vacation, I finally recovered fully from burnout. For about 3 days, I was working at 100% capacity, which I have not done in 15 years. It felt wonderful to have that back, to not succumb to the story that maybe game design and programming are like math of physics where you peak at 25. These few days were way better than when I was 25, because now I am substantially better at programming, and I am good at design also, and games are so much easier to make than they used to be; so the design cylinders are firing and then they turn it over to the programming cylinders, etc, and very quickly a lot of high-quality work gets banged out. And it's not just arbitrary good work, but work that helps to round out the game that I already feel is the best thing I have ever worked on, that helps put grace notes in just the right places, in the manner of exploratory game design that Marc ten Bosch and I spoke about at Indiecade 2011. So this period of intensive work was seasoned and tempered in a way that was not possible when I was 25.

It has been a wonderful time and I look forward to doing more of it. Part of what allowed it to happen this way was that lack of pressure that I have mentioned, due to being on vacation. When a design idea would come to mind, and I would see that it might be cool but would lead to a lot of work, well, this week I was able to let it happen, and I was able to enjoy the challenge of going off on a minor programming rathole in order to do something really good in the design. This is to some degree a reversal of the discipline I instilled to get Braid done; you can think of it as a correction term to the equation. The 0th order term is knowing how to program and design; the 1st order term is having discipline to be able to execute efficiently without being carried away with ideas that will scuttle production; but the 2nd order term is that it is joyful and interesting to have some wiggle room in there, sometimes eroding away a bit of the 1st order term.

There is a kind of open curiosity about programming that I haven't had in a long time, that I allowed myself to re-discover this week in the freedom of exploring that 2nd-order term.

(Richard Feynman has an anecdote about how he was burned out on physics, and how he started to get over it, which I think bears some similarity).

I am not explaining any of this very well. But I am doing the best work of my life, right now, and it feels great, and for the first time I could see myself working in this same fashion for the rest of my life and feeling happy that I am operating at something like my full potential. So there you have it.

I mentioned earlier that I would talk further about the ability to see ideas, but I guess I was lying, because this is already too long and rambling. I'll save that for a future post.

It's worth pointing out that I started Braid while working on what was supposed to be a vacation. And back when I got the impulse to start the Experimental Gameplay Workshop, I was on a plane back from a vacation to Africa. So apparently vacations are good times for me, even if I don't treat them the way Normal People do.

The current puzzle count is 497. (This is lower than last time; last time was an overcount, as we had some temporary duplicates in the world that were being counted mistakenly.)

Right now, we save the thumbnail and the game data as separate files, but the names are mostly the same; so if you go into your game data folder in Windows you can see them living there very plainly:

(It might be interesting to do it like Spore and embed the save data in the image file, so that there's only one file per save. Maybe we'll do this, though it's not a no-brainer; right now the image file is usually bigger than the save data, so this would involve tying a relatively small amount of crucial data to a relatively large amount of cosmetic data. Maybe that doesn't matter, though.)

As I mentioned in the previous post, we are starting to get the game out to a limited number of players. It's been a long time since I've had anyone new play through the game as a whole, so I really have no idea how it plays now, and it's important to find out. However, it's not so easy, because if you sit behind someone and watch them play a game, they are going to play it differently than if you weren't there. You invalidate your own playtest data.

So, I wanted to set up the game to record some minimal playtest data, that would be easy for me to browse through later on, and that would get recorded without me being there to directly watch the play session.

One of the first approaches that programmers often consider is an input recording method. You just journal a stream of all the player's inputs, and you can play that back deterministically to reproduce the entire game session. I have never liked this method. Firstly, you need to play back using exactly the same version of the game executable that the player used, which is a pain; secondly, this kind of playback code is brittle and breaks often. An uninitialized memory read bug can render your playback data useless! (Of course, one reason you might want playback data is to track down an uninitialized memory read bug... so... there's a problem here.)

For these reasons I didn't want to use input journaling. It seemed to me that our saved games are small enough, and the thumbnails are nice enough, that together they almost solve the problem. All I need to do is save a series of snapshots / game states over time, without overwriting previous saves:

Whereas the earlier folder contained a collection of different play sessions, this one contains one play session at different moments in time. Every time the game made an autosave for this play session (overwriting the old one in the normal way), it also also copied the autosave here. The date and time that are in each filename just indicate when that game was started; the number at the end tells you which snapshot this is.

There's one immediate, very nice property of this: I don't even need to run the game in order to get a general idea of how someone's playtest went. Once they send me the files, I can just click on the first one in Windows to invoke the image viewer, and then click the forward/backward buttons in the image viewer to animate my way through the game.

But, if I have questions about what was going on, I can also dig into the save file that corresponds to any given image. I made a mode that lets me browse these saves in-game, load up any particular one into the current world state, and play from there. I invoke it by running a console command:

This loads all the saved games and thumbnails and pops them into a scrolling list at the top of the screen, in chronological order:

I just scroll to the one I want:

... then press Enter to load that game state:

It's very fast to use and it only took a few hours to program.

This whole system is also pretty cool because it's much more robust than the input-recording method. If I want to look into some detail that is super-nitpicky, I may need to use the same build of the game executable that the player used, but this will almost never be necessary. Usually I won't even need to use the in-game browser -- as I mentioned, I can just click through the thumbnails in Windows to get a basic sense of what's going on.

The fact that we only have discrete snapshots is sort of a side-bonus: because we only snapshot every minute or so, the data is not so creepy and spy-like; players can relax while they are playtesting and know that we won't later be obsessing over their every move.

Today we shipped an update, still to the same limited group. We have some initial tech issues to work out before we go wider (game packaging issues, graphics card incompatibilities, and the like).

Here's a desktop screenshot from one of the testers, with someone chiding him about what he's playing:

(The Witness window in this shot is artificially resized to make room on the desktop; by default we launch at 1280x720.)

I know that the impulse is to think "oh wow they are almost done!" Don't think that, it's not the case. We still have a gigantic amount of work to do and it'll be a while before the game is done.

Random Trivia Q&A

Q: How many footstep sound effects are in The Witness?

A: 1,119 so far. They sound really good! We will probably be in the Guinness Book of World Records as the game with the most footstep sounds...

Current puzzle count: 528.

They may not look tremendously different from this off-island vantage point, but when you are down there moving around, the difference is substantial. I feel a lot better about the layout now.

One of my primary design goals, all through development, has been to keep the island spatially small, so that there isn't a lot of wandering through empty or pointless spaces. The idea is to take the best parts of an open-world game, involving choice, exploration and discovery, but compressing these into a maximally dense experience.

That has been going well, but there's an interesting phenomenon where, as we add more detail to the world, it starts to feel a lot bigger. When something is a mostly-featureless plain it is easy to feel like it's small, but once you are passing tiny rocks and plants that are set up very carefully, your brain has more scale reference. So there have been a couple of times during development where we stopped and said "Hey wait a minute, things are just too big."

Last week that happened again; we were looking at two different parts of the island on the same day, and both of them seemed much bigger than we needed. So, we decided to go back to the overall island plan and try to compress it further. We had a discussion about this on Thursday, then the architects talked it over; you see the result of those discussions in the image above. These maps are a handy guide to helping us figure out what we can cut and how we can scoot existing areas around to pack them a little better, and to better-define the areas between them.

This is a little painful, since we almost had the island back into playable shape, and this knocks it back out of playable shape for a little longer. But my hope is that we can just make a mess and then clean it up very quickly.

In other news, there had been a puzzle that I needed to design for two years -- in the sense that I knew I needed to build the puzzle in order for a certain area to be complete, and I knew what that puzzle vaguely had to be about, but didn't have any good ideas on how to build it. Last week I had a breakthrough on that and built it very quickly; it's now one of the best puzzles in the game. So things are coming along well!

The colors are all funny because we haven't written the shaders for the iPad yet, so what you are seeing is the surface normals of the objects in the scene.

We haven't written the shaders yet because they need to be OpenGL ES shaders, and the shaders we have right now are HLSL shaders all written for medium-to-high-end PCs. As the first part of the porting work, Andy made the engine work on OpenGL on the PC, and for that we converted our HLSL shaders to GLSL automatically. But the jump to ES on an embedded device is big enough that it doesn't make sense to try and use the same shaders. Instead we'll have to have some simpler shaders that we substitute.

As you can see from my previous post, I have been working on touch controls (not on the iPad, but on a touchscreen Windows PC). A very rough first cut of that is now working. It's nice to play the game with touch controls, as it feels more immediate than mouse and keyboard (and I have been a first-person shooter player forever!)

So in order to have the game fully interactive on the iPad, we just need to write the bit that translates iPad touch controls to feed them into the rest of the gameplay code, which should not take long.

Then, after that, we have to work out exactly how we are going to crunch all the graphical assets down to portable versions and do the streaming in an appropriate way... which is going to be a bit of work. Salvador built a streaming asset system a while back, and we have been using that daily on the PC, but we haven't tried to set it up on a lighter-weight portable system like the iPad yet. We'll see how it goes!

I am not sure that many people are going to use Windows for touch devices, at least in the Windows 8 timeframe. But at least some people will, so it'd be nice to have a high-quality touch interface built into the game on Windows.

With all this in mind, I started working on some Windows 8 touch hardware a couple of days ago. There is a message format that you can use in Windows 8 to get events from the touch hardware, WM_POINTER, but because I want the touch interface to work in Windows 7 as well, I wanted to use WM_TOUCH messages, which were introduced in Windows 7 and are supported in both operating systems.

It look me less than 20 minutes to get WM_POINTER working. It took many, many hours to get WM_TOUCH working, because WM_TOUCH is some kind of fully insane mess. None of the insanity seems to be documented, or discussed on the internet, so I am laying it out here in case anyone in the future stumbles into this same trap.

A WM_TOUCH message will never be returned by PeekMessage. Spy++ will tell you that WM_TOUCH is being sent to your application, but before the message is delivered, it gets stealthily converted into a mouse motion event. The type of this event will vary depending on whether you have raw input selected: if raw input, it will be WM_INPUT (0x00ff); if not, it will be WM_MOUSEMOTION (0x0200).

If your application handles these events, you may think it's reasonable to complete the handling without calling DefWindowProc. If you do this, you will never see a WM_TOUCH, because something buried down in DefWindowProc stealthily dispatches a WM_TOUCH to your application's windows when it sees a specially-marked WM_INPUT or WM_MOUSEMOVE.

Note that actually, you are supposed to always call DefWindowProc on WM_INPUT (for "cleanup" -- I don't know what this means and why they can't just clean it up when you get the next message -- but whatever); but WM_MOUSEMOVE has no such requirement, and if you believe you handled the event, it is natural *not* to call DefWindowProc. This will result in your never seeing WM_TOUCH. Strictly speaking our game had a bug, in that we were not calling DefWindowProc after WM_INPUT, but the bug was hard to find because it persisted when I tried turning off raw input, because we also were not calling DefWindowProc after WM_MOUSEMOVE (which as I mentioned is, in theory, totally fine.)

Important to note: WM_TOUCH gets dispatched to your windows without you dispatching it! You may think that you can log all messages after they come out of PeekMessage, and that you are in control of your entire message stream, but this is apparently not true. WM_TOUCH will just teleport itself into your window.

In addition to the mouse motion events, your application will get a bunch of other mouse events that are generated by the touch interface, to emulate mouse stuff for programs that don't understand touch. There is no way to tell Windows you don't want these events (unless you do Windows 8 stuff and respond to WM_POINTER; see below). According to Microsoft, the official way you are supposed to detect and ignore these messages is like this:

#define MOUSEEVENTF_FROMTOUCH 0xFF515700
if ((GetMessageExtraInfo() & MOUSEEVENTF_FROMTOUCH) 
        == MOUSEEVENTF_FROMTOUCH) { 
     // Skip message
}

I wish I were joking. It's been this way for years; Casey wrote about this in 2008.

This mask works for events like WM_LBUTTONDOWN, but it does not work for the mouse motion events e.g. WM_INPUT. If you want to take input both from touch and the mouse, you need to disambiguate the real WM_INPUTs from the fake ones. Through a process of experimentation with message flags, I landed on this:


if ((GetMessageExtraInfo() & 0x82) == 0x82) { /* ignore event */ }


It seems to work for now, but who knows if that will mysteriously break in some future version of windows.

Nicely, if your application handles WM_POINTER messages, you will not get any of this mouse emulation stuff or WM_TOUCH, so all this ugliness disappears. If it is okay to make Windows 8 your minimum specification, I recommend using WM_POINTER and staying far away from WM_TOUCH.

I ended up running a straightforward test where we would change the game minimally and build consecutive patches. The patches turned out to be quite large, but upon investigation, it was clear there usually wasn't a good reason for this. Our compiled shaders would contain small differences, as would many meshes in the game.

These looked like uninitialized memory errors and, upon further investigation, that's indeed what they were. They are fixed now, so patches are a lot smaller.

However, patches may never be as small as we would like, and I find there is a pretty deep philosophical issue where.

Right now we have 17MB of files that change in response to minimal changes in the game (say, moving a few entities in the world by small distances). These are mostly cube maps for Light Probe entities, which are used to do lighting on nearby objects. Here's what a Light Probe looks like in the game:

(Well, you don't see that sphere the game, only in the editor!) To generate one of these cube maps, we render the scene in all directions, from the point at the center of that sphere, then apply a blur to the result.

Imagine that we move a couple of entities in the world, even slightly. All Light Probes that can see those entities are going to change, by at least a pixel or two. Because we are applying a big fat blur kernel, a 1-2 pixel change becomes small changes spread across the entire image.

We have an automated script that preps the game for release, clearing out data that we only use for development, and ensuring that automatically-computed things get recomputed (like LODs for faraway objects). Recently we added a step to this process that re-generates all the Light Probes, because it just makes sense to update those automatically. If we have to keep track of them manually somehow we are just going to make mistakes.

What this means is that you can pretty much guarantee that a lot of light probes are going to change if you move some entities and patch the game. If we change anything basic about any of the shaders, which seems very likely, then all of the light probes are going to change (because the output of the shaders feeds back into the light probes).

That said, maybe it's the case now that more light probes are changing than I would expect. I am wondering if other things in the game are causing them to change. For example, when you are playing the game, foliage waves in the wind. I wonder if the animation timer for plants is ticking during the process when we generate light probes, causing them to move over time? This would cause most light probes to be different every time we bake them, regardless of other factors.

Now, there's a bigger issue. In addition to light probes, we have pre-baked lightmaps all over all geometry in the game:

In a way that is reminiscent of the light probes (but a bit different), we generate these by placing a camera at many points along each surface, rendering the scene from each camera, and interpolating those into a final result.

So, just like light probes change when entities move slightly, lightmaps are going to change also. Lightmaps will also change if shaders change, but furthermore, lightmaps will change if light probes change (because light probes determine the look of entities that shine on the surfaces we are lightmapping), and light probes will change if lightmaps change (because the lightmaps influence the objects rendered in the light probes). So we have a circular dependency, which is not so bad in practice, except it means that if you change one entity in the game, then repeat this process iteratively, you may well end up changing most lightmaps and light probes in the game.

The light probes are not so big, but last I checked there are between 500MB and 700MB of lightmaps. However, re-baking the lightmaps is not part of the automated build process; we have to trigger a bake manually. But it seems like good policy to do a full rebake any time we change the game significantly. So, we are probably looking at a minimum patch size in the 500MB range.

I guess that is not such a big deal, since we are not an online game that would be patching all the time. Hopefully once we release the initial version of the game, the "content" won't change very much (sorry, but the word "content" as used in video games is kind of absurd, so it's hard for me to use it without sarcasm quotes; it's almost as bad as "title", which is a word I refuse to use, except to make fun of its use)... the "content" won't change very much, so patches would be rare.

We'll see. We have a while to figure these things out.

Even the idea of UFCS is just a massive overcomplication. It only seems simple compared to baseline C++.

I should say that the biggest problem I find with the idea of member functions is that they are extremely anti-code-reuse. I know this may seem paradoxical, because the whole point is to encapsulate code in an easy-to-reuse way, but I'll give an example.

Let's say you have some basic data type like a 3D Vector called Vector3. You want everyone to be motivated to use Vector3 so that there's no friction when passing data between the main program and various libraries, etc.

All that is necessary for compatibility in a fast program is that everyone agrees on the data layout of Vector3. But as soon as you tie member functions to the data, you are insisting that everyone needs to think the same way about the data that is in a Vector3, and operate on it the same way. And that will never be true, because different code needs to think about data types in different ways; and also because programmers express their personality in programs, and different programmers want to think in different ways about the most common data types.

So if you force people to think a certain way about Vector3 or other basic data types, you are motivating them (or *requiring* them) to go against that grain and make their own alternatives. Then the system is fragmented and you have all these data types that need to be converted again. Or, if people don't do that, they just end up passive-aggressively hating the task of programming, which is not what you want.

So in reality, you want Vector3 to define only a common piece of data to be exchanged. You can provide default functions for operating on Vector3, but in a way that people can easily shadow them with their own functions or else use a completely disjoint set. That is just not what most "object models" encourage. Thus they put huge amounts of friction onreuse in this particular kind of case, which is very common and very important.

(I wish people would stop telling me that member functions are good because you can do obj . and get a list of procs in the IDE after typing the dot. You can obviously do this with flat functions too. Please stop telling me this kthx.)

Because this came out as a series of tweets, it is probably less coherent than what I would have written if it were a straight-up essay, but hey, this is how it went.

September 2011:

August 2012:

You can see Ignacio's new water in this shot (which will continue to improve, and already we have a lot of control over water visuals that we are not using yet). There's a bunch of new audio in the game. Art folks are working on revising various areas and getting them into fully playable shape. Lately I have been thinking hard about how to improve various puzzles (a couple of puzzles already just got *way* better this week), and I have also started rearranging the jungle area, which you see in the left foreground. We've had a lot of discussions with the landscape architects, and they've given us a detailed topography for the island that helps define some of the features we think will be good to have, like this river you can see to the left of the mountain (solid blue for now; it's temporary!).

Also, a couple of us have been on vacation!

From: Ignacio Castaño
Date: 24 August 2012

I just checked in a few changes to the water shader. Note that this is still work in progress, there are some artifacts and unfinished features.

From the art point of view, the main change is that there's now a water_color_map and a water_flow_map that you can use to change the behavior of the shader.

The RGB channels of the color map control the water extinction color and the alpha controls the opacity factor. The previous underwater shading model was simply wrong, the new one works nicely and is easy to tweak. Note that I assume the water color changes very smoothly, avoid sharp color changes!

The flow_map is intended to contain additional water attributes. Right now it's only used for the stillness, but I will certainly add more in the future.

You will also notice that the shoreline now also has a foam layer:

The way it's computed is by comparing the screenspace depth difference. This is not very correct, it looks OK from some angles, but it breaks badly when the underwater geometry has screen-space depth discontinuities:

If the foam is a desirable feature, I'm considering modifying the print_map command to render a foam map, or maybe this is something that can be painted manually and added to the water_flow_map. One of my concerns is that this map may not have enough resolution.

Just Cause 2 seems to use this method to render foam, but they apply a big screenspace distortion to hide the artifacts, I found it sickening when you stare at it closely...

The way the water is shaded has also changed somewhat. First, I now approximate the specular exponent based on the position of the camera with respect to the water and the water roughness. Somewhat like the LEAN paper does, but assuming the roughness of the water is the same for the whole texture. Depending on the value of the exponent I either sample the mirrored reflection map, a filtered environment map, or blend between the two. My goal was to be able to use the mirrored reflection map only on a small subset of the lake entities. The physically based approach does not allow us to cull many, so I'll probably have to introduce some hacks. BTW, I'm doing all these work not just an optimization, but also so that we can use the light probe reflection model in the river and lakes inside the island, where the mirrored reflection map is not available.

The main problem with the filtered environment map is that it does not take local reflections and occlusion from the environment into account. I have two ideas to work around this, one is to store an ambient occlusion map in the lightmaps, this may be a bit too smooth and will lack directional information. The other is to do screenspace raycasting of the depth buffer to decide whether the environment map is occluded or not.

This is somewhat like what Crysis 2 does. Insomniac also uses a similar approach with just a single sample:

http://www.insomniacgames.com/tech/articles/0409/files/water.pdf

I'm hoping we can get away with a crude approximation too. I did a quick test, where the maths are not exactly right, with only 4 samples and no smoothing, and it was starting to look promising:

This won't allow us to use the filtered environment maps closer to the camera, but if the screenspace raycasting works well, I'm thinking about using non-filtered environment maps in place of the mirrored reflection map, whenever those are not available.

Finally, I've also been playing with Valve's water flow model, I have the basic implementation working, but I haven't yet been able to reproduce the workaround for the pulsing and repetition artifacts that Alex proposes.

http://www.valvesoftware.com/publications/2010/siggraph2010_vlachos_waterflow.pdf

A couple of months ago I was playing Super Monday Night Combat; a much-anticipated weekly update came along, but its file size was over 800MB! MNC is an Unreal Engine game and Unreal, like many game engines (and like The Witness), packages its data files together for robustness and speed. This package format interacts with the patch delivery system of any online distribution service, though.

Steam's system performs a binary-data comparison between the old version of your file and the new version, sending only the pieces that have changed. To make this analysis tractable, files are thought of as collections of 1 megabyte chunks; if anything in a particular chunk changes, even one byte, the whole chunk gets included in the patch. If you aren't careful about this, small details of your package format can trigger very large patches, and this is what was happening with Super MNC (not really the developers' fault since they were using a licensed engine; I have heard that the Unreal guys are addressing the problem.)

The Witness packs most of its data into a single 2-gigabyte file stored in the .zip format. Before uploading the initial version to Steam, I took a safeguard to minimize patch sizes, or so I thought: Files are sorted lexicographically and then placed into the zip file in that specific order. (Imagine that files are stored in random order: probably every patch involves every player re-downloading the entire game!)

Last night, as a test, I pushed a new build to Steam. The hope was that the patch would be small, since we had only done two days of work. Instead, here's what we got:

The patch was 660 megabytes. Clearly I had not done enough to prevent this. I opened up a zip file reader and took a look at the file order. In many cases, it was as you'd expect, but in the case of meshes and lightmaps we see this:

'save' is the world name and the 5-digit number is the identifier for each entity. (The world name is redundant right now, since The Witness takes place entirely in one world). We re-bake lightmaps pretty often, but we change meshes much less often. So, the packing order you see here will result in us patching more chunks than we should. Not necessarily by much, in this case, since meshes are much smaller than lightmaps, but that might not be true for all meshes, and other data types may have a similar issue but exhibit more-severe problems. So, I had the thought that instead of sorting things like this:

a.lightmap
a.mesh
b.lightmap
b.mesh


What we really want is to sort by file extension first, then sort by name:

a.lightmap
b.lightmap
a.mesh
b.mesh


... but not always. Sometimes we have multiple files associated with one source file, and that will all change when that source file changes, but that have different extensions. This is the case with shaders:

So I hacked together a solution that sorts anything with an extension of .shader*, in terms of the global file list, as though the extension were only ".shader" -- that means all these files will stick together -- but locally, you still need to keep the full extension, so that the files for one particular shader do not randomize their order every time you make a patch.

The way I do this is to tack a modified extension onto the front of every filename, for sorting purposes only. So "hello.texture" becomes ".texture hello.texture", "whatever.shader_ps_text" becomes ".shader whatever.shader_ps_text", "whatever.shader_dump" becomes ".shader whatever.shader_dump".

If this doesn't make sense to you, don't worry too much about it, because it's not that big of a problem (and I am not even sure if those ps_text and vs_text files are needed for the running game; if we should be discarding them it would certainly mitigate the issue.) These kinds of problems would modestly increase the patch size, but I would not expect the increase to be terrible.

The whole time I was doing this sorting stuff, I knew it wasn't the real problem, and in the back of my mind I was thinking, "I hope the Steam patch mechanism ignores timestamps on files so that we don't have to go touching all files to a known time. (Or I hope we aren't packing a newly-generated build ID into each file or something.)" But this was an example of my brain not working properly: because we are packing all this stuff into a zip file, Steam doesn't know they are separate files! Since the zip file stores timestamps for all the files, of course that data changes every build (since our build process generates new files every time).

At first I was thinking that this wouldn't be the real problem, either, because the zip file format has a main directory that is stored compactly, and that is where timestamps would be, because you want to be able to access that data quickly. So I would have guessed that, if all the timestamps changed, the worst that would happen is that the directory would change every build, but that is at most a couple of megabytes. But looking at the zip file specification, I discovered how wrong I was. Inexplicably, timestamps for each file in a zip archive are stored both in the central directory and in the local header for each file, so if you change timestamps for all your files, you are strewing changes everywhere.

So I made the packaging script set all timestamps to 0, but the zip library we are using didn't like that, so I set the timestamp to January 1, 2012 00:00:00 GMT.

After fixing this, as a test I built two patches in a row from scratch, with the game recompiling all shaders and re-compressing all textures each time. In theory the results should be exactly the same (certainly if this still generated a huge patch, we would still have some kind of big problem). Happily, we have found at least some bit of success:

I am not sure what it is saying there about 2046 of 2262 matching, or 60 changed chunks, but saying it uploaded 0 chunks. In any case it's better than before. I am going to ask the Steam people what these various numbers actually mean.

Feeling good about this, I ran a final test where I moved around a few objects in the world and changed a constant in one shader:

That's once again not so good (140 MB!) I had been sweeping the 18MB from the previous run under the rug, thinking "hey, it says 0 chunks sent, so maybe that 18MB is protocol overhead involved in figuring out which chunks changed, and it doesn't impact patch size." Apparently not. The next step involves asking the Steam folks for advice, hunting around for a binary diff file with which to analyze our own packages to see if we are doing something unsavory, etc.

Here's the current Perl code for the part of our script that builds the zip file:

sub modify_name {
    my $name = $_[0];

    my ($file, $dir, $ext) = fileparse(lc($name), qr/\.[^.]*/);
    my $orig_ext = $ext;

    if ($ext =~ m/^\.shader/) { 
        $ext = '.shader';
    }

    # Throw away 'dir' since we discard it when writing the file, anyway.
    return "$ext $file$orig_ext";
}

sub make_uncompressed_zip {  # Sort zip members alphabetically so that binary diffing will have an easier time.
    my ($source, $prefix, $dest_zip_file) = @_;

    my @unsorted = glob("$source/*.*");
    my @sorted = sort { 
        modify_name($a) cmp modify_name($b)
    } @unsorted;

    print("Sorted $#sorted files.\n");

    use Archive::Zip qw(:ERROR_CODES);
    my $zip = Archive::Zip->new();

    
    # Zip API complains if time is before 1980, so use an arbitrary timestamp in that range (yuck).
    my $timestamp_2012_jan1 = 1325376000;
    foreach my $fullname (@sorted) {
        my ($file, $dir, $ext) = fileparse($fullname, qr/\.[^.]*/);

        # print("Adding '$fullname' as '$prefix$file$ext'\n");
        my $member = $zip->addFile("$fullname", "$prefix$file$ext");
        $member->setLastModFileDateTimeFromUnix($timestamp_2012_jan1);  # Clobber timestamp. 
    }

    # Set the compression level on all the members.

    my @members = $zip->members();
    foreach my $member (@members) {
        $member->desiredCompressionMethod(COMPRESSION_STORED);
    }

    $zip->writeToFileNamed($dest_zip_file);
}

We finally got that working well enough; here's me getting ready to download the game on Steam:

And here I am after having launched the game:

I don't want people out there to get too excited about these screenshots: this does not mean the game is close to release. You wouldn't want to play it right now. But this will help us a lot when it comes to getting the game out to playtesters and keeping it updated.

For actually shipping our game off to Steam, we are using the new version of the Steam developer tools, and they are really good. The old stuff (which I had used for Braid) was serviceable but had a lot of undesirable things going on. The new stuff is exactly what you want: you write a couple of very small and easy to read config files, then you run a command and it uploads your game. This process is now fully-automated, which means developers can modify their game and push changes out to users faster and with less hassle than with any other online distribution service. (It is basically as fast as if you host the game yourself).

Lately, Ignacio and Salvador took the features all the way so that we can use them easily in normal workflow. (This happened over some period of time, but the last parts came together this week). Now you can take a screenshot, screw around with the colors in Photoshop, and load it back into the game to control the grading. I am pretty sure this is not our idea, and we got it from the Uncharted guys or someone like that (Update: The idea was developed by Naty Hoffman and others; see the comments to this article).

Here's how it goes. First, I find an area the look of which I would like to change:

Next I hold down the control key while taking the screenshot, causing it to be saved with a special color bar along the top of the image:

Then I go into Photoshop and screw around with the colors. Here I am using the Hue/Saturation tool to shift the palette away from red:

Or, instead, I may wish to shape the histogram of the image:

That color bar at the top of the screen acts as a look-up table. The game doesn't need to understand what operations we have performed in Photoshop; it just knows what that color bar looked like when the screenshot was saved out, and, by comparing it to the colors in the result, knows how to transform any color in the rendered scene.

In the game, you can now attach these grading operations to spatial markers, so that color grading kicks in when you are inside specific areas. Here I am setting up some markers in the editor (the marker is the big white box):

After setting up two of these boxes, one for each of the two weird color operations I created, then I can walk around in the game and have these operations apply to everything:

This gives us a really nice degree of control over how the game looks. We plan to use it to help with the color theming that will tie together specific areas.

As you'll notice, we are starting to work on that big mountain that is closest to the camera. (Finally!) This is still an early concept; it will change a lot.

Ignacio recently implemented a clean way to print an overhead map of the island, in flat colors, which is probably useful for marking things up:

On the modeling side, Eric has been refining an area that's far from the camera in these shots (you'll see some screenshots of the interior soon). Luis has been working on the style of our outdoor scenes; we have interesting problems that involve maintaining a visual style that is good for gameplay while also creating images that are most striking. Orsi has been working on refinements to a few particular areas, particularly, adding some deeper puzzle complexity to the area with the tall trees (off to the right in the screenshot image).

Until recently I had been bothered by some unanswered questions regarding the endgame. There were things I wanted to do that didn't fit together and that felt clumsy. But thankfully, after taking a relaxing weekend where I barely worked on the game at all, on Monday solutions to most of these problems just popped into my head. So that's cool.

There are still some questions about the gameplay but I think the biggest ones are now nailed down. Over the next couple of weeks I will be implementing the remaining parts of this endgame.

Tech-wise, Andy is pretty far along with the OpenGL renderer, which will help us hit other platforms, and Salvador has tied up the basics of asset streaming and is doing some rendering optimization stuff. Ignacio is refining the LOD system in addition to many miscellaneous tasks.

The original letter went through a few examples of specific issues of certification and how they could be handled better, to help substantiate why I think these processes are so broken. These are not the kinds of details you would expect to make it into a short article for a general audience, but having typed them up, I think some people out there might be interested (especially developers who have not done console games before but are thinking about it). So, below is my original letter (edited a little bit for clarity):

-----

Hi Kyle,

I have gone on record a number of times, saying that the XBLA certification process makes games worse rather than better. But that is a pretty old experience for me now (4 years ago), and they are unlikely to change any of that stuff until the new console comes out anyway. Keep in mind that the Xbox people as a whole do not think of XBLA as a high priority (you can get a glimpse of this from the E3 press conferences, etc).

But, I think the more important issue applies globally, to all of Microsoft, Sony and Nintendo (and not just to smaller downloadable games, but to every game regardless of size). The certification processes of all these platform holders were based on the idea that all these steps they test are absolutely necessary for software to run robustly, and that software robustness is super-important for the health of their platform and its perception by customers.

But, look at iOS. There is almost no certification process for iOS, so by the Microsoft/Sony/Nintendo theory, the apps should be crashing all the time, everyone should think of iOS as sucky, etc. But in fact this is not what is happening. There is no public outcry for more testing and robustness of iOS software.

Part of this may be that iOS software is so easily patched; so maybe a heavy cert process made sense back in the disc-only days, but as we go into the next console generation it becomes unnecessary. But something tells me that Microsoft/Sony/Nintendo are not really going to let up on cert, even though they say they will. It's just "not in their DNA" as they say. The proof of this is the bizarre over-complexification that already happens with console games today, and is baked into the current certifications, that any of them could easily fix, but none of them do, because they don't care.

For example: every single game is REQUIRED to say on startup, "sometimes this game saves, when you see this animated icon in the corner, DO NOT TURN OFF YOUR CONSOLE, etc". This is something that developers have to implement and that has to be tested, which costs significant time and money, but worse than all that, it impacts the user experience, because the startup of the game becomes just a little more bureaucratized, and also -- this is supposed to be a fun experience, so why are you issuing warnings and strict instructions? (Just one thing like this may not seem like too much, but combined with everything else, it is a lot; I am using it as just one specific example).

Why is this DO NOT TURN OFF YOUR CONSOLE warning there? It's because if the game is saving a save file, overwriting an old one, and you turn off in the middle, you might corrupt the saved game. Well, guess what... the solution to this is to IMPLEMENT A MORE ROBUST SAVE SYSTEM. You save the new file next to the old file, flush the I/O, and only delete the old file once the integrity of the new one has been verified (or just don't remove it, and keep two copies of the save at all times). On load you just make sure you load the intact save file if one is corrupted. This is not hard to implement; I did it in Braid. But if consoles cared about this kind of thing, it would be built into their basic save-file API, so that it would always work perfectly and no developers would ever have to think about it.

If they did this, there would be fewer things to certify, certification would cost a little less and take a bit less time. Let's say you save 3 days of development and testing per game (this is conservative; the real amount can be substantially higher when you factor in the discussions and coordination about how the save notice should look, etc). Now add up how many games have been released just on the Xbox 360, multiply that number by 3 days, and what you get is probably OVER A DECADE OF DEVELOPER TIME that was wasted. Just on this one little requirement. For something that should just be built into the system by one person in a couple of weeks. Okay, that was only the Xbox, so now add in all the PS3 and Wii games, and see what a huge waste of time that represents.

Like I said, just this one thing by itself would not be too bad, but it is only an example. Another example that all three certification processes have is the "title-safe region" restriction. This basically means that you are not allowed to draw text or gameplay-critical information near the edge of the screen (about 10% of screen width in a margin around the edge) because you don't know if the user's TV is displaying the whole picture (even modern HDTVs often cover the edge of the display with a bezel or else just crop it entirely, which is completely stupid).

Okay, so at first blush this seems to make sense, so to make sure players can see everything, we have to go through each screen and each mode in every game we ever release and certify that important things don't get drawn too far toward the edge. Well, as you can extrapolate from my previous example, this also has wasted DECADES OF DEVELOPER TIME (probably many decades, because dealing with title-safe probably takes a lot more time than save/load). And it is also something that these consoles could handle at the system level if they actually cared about the problem. Just put a screen calibration menu in the dashboard, and then initialize it properly so that players would almost never need to use it (when they are using HDMI output, initialize the settings using the display's EDID; if they are outputting a different way, just default to shrinking the display a little bit).

10% at the edges of the screen may not sound like much, but it is actually huge. Suppose a game is rendering at 1280x720... you are only really able to use the interior 1024x576, with everything outside that just being decoration. 1280x720 is 921600 pixels; 1024x576 is 589824 pixels, in other words, you only get to use 64% of the screen area for real! Everything outside that is only allowed to be decorative. You know how the 360's new Metro interface seems to use an absurdly small amount of real estate in the middle of the screen? That is due in large part to title-safe region. You know how games always seem to have their HUD elements placed too far inward when they could be nicer, further off to the sides? Same.

This would also have the benefit of saving work for most people who implement 2D games. In Braid we had to program a screen calibration menu. I notice there's one at the beginning of Dyad, and one at the beginning of PixelJunk Shooter. For God's sake why??

This makes all games worse all the time, and makes developers have to do a lot more work, but ultimately could be fixed very easily by any of the platform holders. They just don't care enough to do it. I think they will have a hard time getting rid of these rules because bureaucracy is in their DNA.

These are just two examples of many... I am going to stop ranting here, but this is just the tip of the iceberg. Most certification requirements are like this.

The edge that both Apple and Valve have going into the future is that they both genuinely care about the end-user experience and want to make it as good as possible. (Their end-user experience is already way better than any of the consoles and they are always working to improve it). Which coincidentally seems to be the place that these consoles are handicapped due to their corporate culture. Can anyone look at the current 360 or PS3 dashboards and legitimately say that those are products of an entity that deeply cares about user experience?

A question a lot of developers have is: When the 720 / PS4 get launched, how many people are really going to care and buy games for those systems? Obviously some people will, but if it is less than bought games this generation -- which a lot of developers think is very possible -- then look for a "peak oil" kind of crash where a lot of too-big publishers and developers fight over a shrinking market. If the actual way the next-gen consoles work is much like now, they will be a functionally archaic in the marketplace (keep in mind that they have to compete with the iPad 4, 5, 6, 7, 8, and 9. Have you got any idea what the iPad 5 or 6 are going to look like, how powerful they are going to be, what other user experience benefits they are going to have? I sure don't.)

-----

A few days after having written this letter, I just have one more thought to add: I know that Microsoft thinks they are fixing these issues with their next console, and creating an environment more suitable to free-to-play games and downloadable things in general; I assume Sony is thinking similarly. I will be very surprised if either of them succeeds in fixing these problems. Large institutions always undercorrect; they always think they are being radical and risky when in fact they are doing something tame that is just a modified version of the status quo. When large institutions need to change course by 95% in order to do well, even if they know it's an emergency and are totally panicked, they can probably only manage about 35%. For recent illustrations of this, look at Nokia and RIM (or, uhh, look at the large governments of Earth in dealing with global warming).

So I would bet that Microsoft and Sony believe they are streamlining their online stuff, because they are implementing an expedited, simplified cert process for patches and content updates, or something, and that because of this they will be in a great position to compete with Facebook and Apple and Steam and whoever. Whatever they do is very likely not to be enough. Their competitors are not stopping either. (Steam, which was already pretty painless in terms of updating games, recently revamped their system; the new thing is way better than the old thing, which was already way better than what the consoles do.)

Originally I wasn't sure whether I wanted to implement subtitles or not. It's important to me that the game have an overall aesthetic that contains no visual language. I was worried that if we were to draw subtitles for people who don't speak English, it might ruin this. But, a year ago, I was talking to a friend who has experience in film, and he convinced me that people just understand that subtitles are a different thing, that they are a layer on top of the game or movie or whatever.

So, I am implementing this, and maybe it's good, because if I had been hard-line about things, it would mean that most people in the world would not be able to understand the recordings. That's not terrible, because they are not necessary to complete the game, but as long as I don't feel like, ultimately, the subtitles are ruining things, it is probably better to have them.

(It's going to be an interesting time when I go try to implement the main menu without any written words...)

It's been a very productive day. I implemented the basic subtitling system in less than 3 hours (making a subtitle data file for one of the voice tracks, loading it, detecting in-game when a voice track is playing, finding the appropriate piece of subtitle for the current time, and drawing it on the screen). Here's the result:

The data file is a very simple format that I just made up. Taking my experience in making Braid multi-language, and looking at the kind of text we will be dealing with in The Witness, it was clear that I needed a format that was mostly about the text, with very minimal markup. Right now the main markup we need is just "which recording is the following text for" and "what time in the recording should the following lines be displayed". I can imagine that later on we will add some more features to it, but probably not many.

Since The Witness inherits code from Braid, we already have handling for other languages in our font rendering, so I pumped the text through Google Translate to test a couple of other languages:

If you are playing in English, subtitles are OFF BY DEFAULT. It amazes me how badly many games handle subtitles, most notably, that a great many games seem to have voice acting in English and then also default to subtitling in English. This is terrible because people read at a different speed than they listen, and they probably read faster. It ends up dividing peoples' attention in a weird way. I don't know why so many games do this except that they aren't paying enough attention to user experience. So to have a good experience you have to go turn subtitles off, but sometimes that doesn't even work! A couple of days ago, I was playing Xenoblade Chronicles, and when I turn off subtitles, only about half of the subtitles in the game go away. Apparently there are two different classes of subtitles and one of them you cannot even turn off! I think they never even tested it...

The current puzzle count is 415 (I think we have been over-counting a little recently; this is a more-accurate count).

In case you missed it, I did a live Q&A with Chris Hecker of Spy Party today on Kotaku.

In terms of world-building, lately Luis, Shannon, Eric and Orsi been doing a few things: (1) revisiting the most complicated areas in the game and making them playable with something resembling shipping art assets [until now they were playable only with placeholder pieces]; (2) starting to figure out what the game will look like, outdoor-wise; (3) stuff I can't spoil in a post like this.

On the tech side Ignacio is working on optimizing the game, as often. A big part of this has been working on the mesh LOD system, but he's also looking at performance issues related to specific subsystems like grass rendering. Salvador has done a bunch of work on asset streaming, which seems to be pretty solid so far. Andy made a number of improvements to the collision system and audio system, and has now started work on an OpenGL port of the rendering, which, when done, is a big foothold toward portability to other platforms besides Windows.

The audio guys, Andrew and Geoff, have done some really nice stuff in terms of sound. The audio aesthetic for the game is starting to come together. We have lots of rad footsteps and are doing some very interesting things in terms of environment sounds.

This is not a comprehensive account of everything everyone's been doing; it is just what leaps to mind!

Every week we have a meeting with the architects where we plan the structures and landscapes on the island. Most of the week, the architects are generally off thinking about this stuff while we are building the game. During the weekly meeting we get together, talk about what concepts the architects have come up with, figure out how those might integrate with the game, make decisions about which ones to pursue, and iterate in detail on concepts we have already decided to pursue.

A while back we took some photos of this meeting, so I figured it would be fun to post them.

Here we are looking at a number of different possibilities for one of the areas in the game. I had already done the gameplay design, so we had a working location that was built out of blocky programmer stuff (if you see blocky things in the island update screenshots, it was something like that). So we knew the general shape of what we were trying to build and how it had to behave; the architects started with that and made proposals about what the location might be like in terms of its basic construction.

We chose one of these ideas and iterated on it a while. Other concerns, having to do with the overall layout of elements on the island, then caused us to change the size and shape of this location; once we had done this, the original design was felt to be a bit weak at the new size and shape, so we went back and heavily modified the concept. Only this past week have we converged on the final design and gotten a good first approximation into the game in a playable way. It's pretty cool.

Because we're working on a lot of things at once, we usually talk about a few different locations that are at different stages of development. At this particular meeting we were also looking at some early concepts of the visual design and modeling of the area where you start the game:

Last week, I got together with three other designers for a four-day intensive design retreat known as the Depth Jam. The other attendees were: Chris Hecker of Spy Party, Marc ten Bosch of Miegakure, and Daniel Benmergui of Storyteller.

This event was an experiment. Chris and I had been discussing the idea for a while, but it took us a couple of years to get around to it. One of my personal goals in setting up this event was to find a new way to stimulate my professional development, because the old ways were not cutting it any more.

The Depth Jam was designed in reaction to shortcomings of other game-related events. In order to explain the design choices behind the Depth Jam, I will speak critically of these other events, in order to highlight the problems that the Depth Jam is meant to address. If you are a fan of these events, organize some of them, or otherwise identify closely with them, then this will be uncomfortable. The best I can do here is to assure you that this isn't attack-style negativity; it is criticism that comes from years of carefully considering these situations and thinking hard about how to make things better.

Conferences

When I first started working in video games I learned a lot from conferences and lectures. The few days I spent at the Computer Game Developers' Conference in 1996 were eye-opening, even though I wasn't comfortable enough as a game developer to know how to make effective use of that time. As years go by and we get better at what we do, a natural shift occurs: in the beginning, we are mostly deriving benefit from other attendees and presenters; later on, we are mostly providing benefit to other attendees, getting little out of it ourselves. I have been to the Game Developers Conference 17 times now. I find that I still do get something from attending, but it really takes a lot of work and there are very few people I can expect to learn from.

All the smart programmers I know complain about conferences and consider them basically useless (except for the smart programmers who are also on conference advisory boards). I certainly developed this kind of frustrated "there's nothing good here to see" attitude in the early 2000s, but to mitigate this situation I shifted into being much more of a conference presenter than an attendee. A lot of creative energy went into planning new conference sessions and making them good. This helped extend the useful life of conferences, because I was learning a great deal by running sessions. After about eight years, though, this ran its course and I had gotten the bulk of what I was going to get from this arrangement.

Game Jams

In a typical game jam, developers gather for 2-4 days to do a working sprint, the goal being to produce a finished game entirely during the event. I was lucky enough to be around for the Indie Game Jam, which probably started the game jam trend (though I was often too busy working on last-minute GDC lectures to make jam games, sadly!)

From the first Indie Game Jam. Pictured: Brian Jacobson, Sean Barrett, Ken Demarest, Charles Bloom, Jonathan Blow, Brian Sharp, Doug Church.

The Indie Game Jams were very different from the jams we see today. The IGJ was founded on the idea of exploring the design ramifications of a crazy technical question; we would provide attendees with some code to accomplish some technical feat, then they would see where that would lead in terms of design. For example, the question for the first IGJ was "Graphics hardware is pretty fast now; what will people design if we give them an engine that can draw 100,000 little sprite guys on the screen at once?" We were picky about who we invited: attendees had to be designer-programmers who were actually good at programming, because dealing with new technology on a short timeframe can be very challenging.

Also from the first Indie Game Jam. Pictured: Art Min, Charles Bloom, Robin Walker, Thatcher Ulrich, Brian Jacobson, Zack Booth Simpson, and ... I don't know, Justin Hall? Charles Bloom again?

In contrast, contemporary game jams are more open. The idea is that it's great to make a game, any game, and that even if you don't manage to finish, you were still part of a community. This community spirit is often upheld as the best part of a game jam.

I think these jams can be really nice for beginners, to show people that making a game is something within reach, and to help them meet other people interested in making games. For experienced developers, though, I think these jams are not so good, because the jams are part of an overall social context that supports stagnation.

Experienced developers would do well to continually hone their craft and push beyond their comfort zone, but jams celebrate low expectations and provide Warm Community Feelings that create a comfort coccoon. "You made a game that is not very interesting? You didn't finish a game at all? Well, you are still part of a community that likes you because you are participating in a game jam, and that is what is truly important."

I don't think there is anything inherently wrong with desiring a feeling of community, but we must take care to separate the notion of participating in the community from the notion of success. Participating in a game jam is not an indicator of success at game development -- nor is attending the Independent Game Summit or Indiecade. I think that nice feelings are nice, but you want to build an ecosystem where the nice feelings coincide with behavior that will make developers robust and powerful and interesting in the long term. You don't want nice feelings to act as a soporific. Activities that are good for beginners are often stagnant for intermediate developers.

What to do about this? I think if there were a category of game jams with higher expectations, so that advanced jams were differentiated from beginner jams, that would be a nice start. The IGJ model definitely asked more of participants, but I don't think the IGJ is the right thing going into the future, because even if you are playing with challenging technology, the time-limited format prevents you from going deep. IGJ was nice in 2002, but today it would just contribute further to the problem outlined in Chris Hecker's rant Please Finish Your Game. We see lots of wacky but shallow game designs all over the web, and it feels like a problem, or at least a vast sea of potential unreached.

As Chris says in his write-up about the Depth Jam, "game jams are shallow by design." Because they are shallow, I don't feel they are the right place for me to develop as a game design practitioner.

Retreat-Like Gatherings

There have been a few retreat-like gatherings for forward-thinking game designers: see for example Project Horseshoe and Phrontisterion. I have never been to these. I like the basic idea behind these kinds of events, but the execution seems to have endemic problems. I know people who have been to both of these events, but when I ask, they do not recommend attending the events. Certainly I have found Project Horseshoe's written reports unhelpful.

The obvious problem with these events is that they are mostly just a bunch of talking. When people get together for a bunch of talking, most of them just say a bunch of bullshit (here I mean bullshit in the Harry G. Frankfurt sense) that is unfocused and untethered to reality. If it is not of critical importance whether what people say is right, then most of it is not going to be right.

I really like the retreat model as a basic template. Over the past few years I've been to a number of retreats, mostly for things like meditation or silent existential contemplation. There's a lot good about going to a far-away place where you are not bothered by the concerns of everyday life. But looking at Phrontisterion and Horseshoe, the question arises: how does one design a retreat like this that is not just a bunch of talking?

I knew some of the answer, because I've seen success in dealing with a similar question in a neighboring realm:

Local Developer Meet-Ups

Many cities around the world have regular meet-ups; once a month or so, game developers get together at a bar or something, and just chat and be social. I generally don't go to these because bars are antagonistic to interesting conversations and because attendees tend toward the neophyte side, which means we have the same problem as at conferences: there's little benefit to be had for an experienced developer. But these events also have the Horseshoe problem, in that the conversation is not focused and doesn't really matter, so people just say a bunch of nonsense. If all you want is to get drunk and be social, these events are fine, but they don't do much for professional development.

Two years ago I started a series of monthly developer meetups in the San Francisco Bay Area; the idea was to keep discussion quality high by (a) holding the meetings in quiet places conducive to good discussion, like someone's house (b) inviting only active game designers [this was not a "game industry" meeting, it was a "thoughtful game designer" meeting], and (c) starting each meeting with one of the attendees presenting their own work. After the presentation, we discuss that work specifically for a while; then at some point, the discussion naturally dissolves into separate, more-general discussions.

(I have been to some developer meet-ups that also began with presentations, but which to me did not manage to establish a culture of quality. One example was the Austin IGDA meetings back in 2002-2003. I think there were many subtle things preventing quality from rising, mostly having to do with intention of the event: the goal of the meeting was to drive attendance, not to be deeply interesting; also, they were "game industry" meetings, with all the associated issues. Also, scale matters: the Austin IGDA meetings were bigger, and they occurred in offices or at places like Dave & Buster's, which did not encourage a personal connection to the presentation or the other attendees).

The meetings in the Bay Area were very successful at keeping discussion quality relatively high. The situation wasn't perfect, but it was much better than your typical bar meeting. (I would have attended these regularly even if I were not involved in organizing them). Other attendees really enjoyed the meetings as well. After about a year, though, I stopped arranging the meetings because we seemed to have exhausted the supply of people willing to present, and I didn't want to start having meetings that were not kicked off by solid presentations. Because everyone had a good time, there's a reasonable probability that in the future we will pick these up again. I think we might be able to improve the discussions further by reducing the presenter-vs-audience asymmetry somehow (see the Depth Jam section below).

Key to the success of these events was having specific, concrete issues to talk about: a specific game being presented, so that discussion could be anchored to the details of that specific game. It's also important that the game was being presented by its author; if the session is just someone talking about someone else's game that they liked or didn't like or just want to say something about, it is too easy for the discussion to be useless bullshit.

It seemed like this model could be applied to ground a retreat so that it would not just be a bunch of talking.

The Depth Jam idea

Now, we come to the actual Depth Jam. We settled on the basic idea very quickly: the jam would occur in a relaxing retreat-like environment. There would be a limited number of participants; four seemed like a good number. Each participant would have a good game that he has already been working on for a while, and which presents some deep and interesting problem he would like to solve; this problem serves as a focus for discussion. The fact that every particpant has a game under discussion means that every participant has "skin in the game", which keeps discussion tethered and unfrivolous.

I'd like to emphasize this last point because it can be subtle but it is crucial. Suppose some people are showing their games and being criticized and generally having a rough time due to all the stress that happens naturally when having one's creation dissected; and meanwhile, the people who are criticizing do not have any obligations, and they are just tossing in comments from the peanut gallery. This situation creates a weird imbalance. The comments and criticism will not be as thoughtful as they could be, yet they will be taken very hard by the people showing the games.

If everyone is having their creations dissected, there's only one class of attendee instead of two. It is easier to empathize and avoid unnecessary harshness. People are going to be more careful that their ideas and criticisms are thoughtful, because they are acutely aware of wanting careful input when it comes to their own game.

The Format

Our first proposal for the Depth Jam had us spending one entire day on each game, so that we could dive into each game at maximum depth without context-switching. However, as the idea churned, we decided it would be beneficial to allow some iteration. Why not split the day into two time slots, so that each game gets half a day and we cycle through the four games twice? This would allow time to modify the games based on the discussion, which would give the discussion even more teeth: now we're not just talking about a particular problem in a particular game (with some kind of tendency to wax philosophical), we're actually figuring out how the author will address the problem here at the retreat, with the results of that approach to be plainly visible in the next session.

In our pre-jam planning meeting, we decided to go even further this way, breaking each day into four slots, cycling through all the games four times (so that each game had a two-hour slot each day). This seemed to work pretty well and it allowed a lot of iteration, but it might have been excessive. Daniel thinks there was too much context-switching and he might have been able to think more effectively if given more stability. My impression is that the talking was great for the first couple of days and degraded in quality toward the end (but was still worthwhile even then). At least two attendees did not have a problem with this, though, and found that the final discussions were very valuable for them.

For the next jam I would propose a mixed format: we'd start with two 4-slot days, just like we did this time, followed by a day with no-talking, all-working-and-quiet-and-taking-a-walk, followed by a final 4-slot day. (Or, perhaps I would lengthen the retreat to 5 days and put the rest day in the middle.)

The nice thing about the 4-slot-per-day format, which would not have been true of the 2-slot-per-day format, is that it gives us the maximum amount of calendar time for ideas to stew between iterations. I have long been appricative of the role of calendar time in good design. Sometimes it doesn't matter how many hours you pack in trying to get things done; the good ideas will arrive unpredictably, and maybe you just need to allow time for this to happen. I was curious whether this principle would also be true on the timescale of a 4-day retreat. For me, at least, it was; I got my best idea in the shower on the morning of day 3, in response to discussions we'd had on day 2. During my session on day 3 we discussed and refined the idea, and on day 4 I showed an implementation of it.

Creature Comforts

We spent some money renting a nice beach house and ordering catered food; the cost for the event was around $5000. Chris goes into more detail about this in his write-up. You don't need to spend any money on an event like this, but if you can afford it I recommend spending some money to help create a nice environment that minimizes stress and factors away concerns like "what are we going to eat for dinner" and "who is going to do the dishes". The purpose is twofold: first, it helps you focus on the subject at hand; second, the minimization of external stresses helps you deal with the potential added stresses of working really hard and disagreeing with people all the time.

If you think that the Depth Jam will help you make more headway on even one deep problem than you would have otherwise, and thus make your game better, it's easy to think that the game will also sell a little better and that costs in the neighborhood of $1250 per attendee are easily justified.

Attitude

Discussions can easily turn into arguments, or at least energy-sucking disagreements. As peoples' energy gets drained, they become more irritable, so there's a feedback loop lurking here. A little bit of this happened at our Depth Jam, but we saw it happening and course-corrected, so that the final day's discussion was reasonable.

Prior to the jam, though, we had not thought much about this. I think it would be helpful in future for the attendees to go in knowing that irritability is likely to happen; the mere fact of this awareness probably helps the situation, and anyway, a little bit of psychologically-aware pacing at the beginning would have gone a long way.

Games and Attendees

It's very important the attendees be capable both of participating in good discussions and acting on the discussions to improve their games within a short timeframe. This latter requirement basically limits attendees to being competent designer/programmers. If someone can't program, it's hard to see how he can participate meaningfully in this style of depth jam, because it would be very hard to iterate. Possibly if someone is a level/world/puzzle designer who can build scenarios quickly in UDK or Unity or whatever, it can be made to work, but I still think that person would be feeling the limitations of being unable to make algorithmic changes.

I don't think that having teams of people would work, at least not for the format described here, because it would dilute the energy and bog down the iteration process. If you have one designer and one programmer trying to do the job that the other people are doing as single designer/programmers, your duo is probably going to have a hard time keeping up. If the jam is made up entirely of duos, it's going to make discussions a lot messier and lower-energy (twice as many people talking about the same number of games). At least one of us felt that four people was already too many for high-quality discussions, because you keep having interesting ideas but have to wait for other people to stop talking in order to say them, and by that time the discussion may have moved on to a different topic.

We have tossed around some ideas about how to scale the jam to larger groups, but haven't come up with anything that is fully convincing yet.

It's important that the games be high-quality efforts that pose problems that everyone will be interested in. We were fortunate to have four very interesting games: Miegakure, a puzzle-platformer in four dimensions; Storyteller, a game about building stories through character interactions; Spy Party, a heavily player-skill-oriented game about subtlety and deception in human behavior; and The Witness, a first-person puzzle game with a heavy emphasis on nonverbal communication.

If time permits I may do a detailed write-up of the issues we discussed for each game and the resolutions we reached (though care must be taken here, as we don't want to disclose aspects of these games that the designers would rather keep secret).

Conclusions

I am happy with the way this first Depth Jam went. I think we can certainly tweak the format to improve it, but already it is a useful tool in my further development as a game designer. I got much more out of this four-person, four-day event than I do from attending a conference. It seems appropriate to me to do a Depth Jam every six months. Provided we are organized enough to get the next one together, we'll adjust the format and see how it goes!

See also:

Chris Hecker's write-up
Daniel Benmergui's write-up
Miegakure web site

In a strange twist of fate, the corner of the island that I chose long ago to be closest to the camera is the place that has taken longest to develop. As you can see, we are continually modifying this area but it's still in a very rough state. Most other areas of the island are further along.

Lately I have been doing a little bit of playtesting and high-concept discussion (more on this later in the week). I've also been doing more work on the endgame. The modelers have been cranking away at making areas look better. I think we have another engine technology posting coming down the pipe.

In this instalment, Thiel takes the question of what kind of company to start and reframes it in terms of secrets.

Thiel's point of view reminds me of the way I think about games in general. Quite often I go out and give speeches about game design or business, and I say things that some think are overly idealistic, or oblivious to the necessities of recipients' situations, or just plain wrong. Certainly my advice usually runs counter to conventional wisdom. (Yet somehow, by following these principles, I seem to do okay.)

Thiel begins with the question: "What important truth do very few people agree with you on?" and goes on to define secrets, in this context, as "unpopular or unconventional truths". If everyone knew these things and believed them, they wouldn't be secrets.

I recommend this write-up to anyone who wants to think unconventionally about game design.

The changes are a little bit drastic in some spots, but overall not as drastic as I had expected. I had been worried about some issues in the natural flow of where players would naturally end up going, especially early in the game. It turns out we were able to solve these without changing too much. So between last island update and this one, for the most part, some areas were moved laterally or expanded, and some new elevation changes were introduced, but areas were not shuffled around; they are all more-or-less in the same place with relation to each other. So for the most part, things are converging, which is a good sign.

And, oh yeah, we are making that mountain a lot bigger. It is the messiest part of the island right now, and has been for a while. There are still a few things gameplay-wise to figure out here, then it will start to come together. My biggest questions in the area have to do with that white building near the bottom of the image; the puzzles inside that building will determine its overall structure, which will determine how it is worked into the surrounding terrain of the mountain. We have some proposals from the architects on what this building will look like, but it is waiting on me to finish the puzzle design.

Because we haven't done it in a while, here's the island from a different angle:

The current puzzle count for the whole game is 450.

But today we are going to have the first in a series of meetings wherein we think pretty hard about the topography of the island and what the flow of natural curious investigation will tend to be from various spots. Expect the island to look pretty different in the next update! (Or maybe it will turn out that things are mostly fine...)

Lately, I have been working on some of the more sophisticated puzzles in the game. These are things that were at least sort-of done already, but I wanted to revise the designs to improve them -- to ensure that the puzzles are as interesting as possible and built as closely as possible around the core themes of the areas that contain them. I am happy with the way these are coming along so far, but there is definitely still some work to do here.

The art team has been working further to nail down the style of the game, now focusing on cliffs and plants and other natural things, as well as control panels and other bits of machinery. We're also taking a few buildings that the architects have finished designing, and starting to build out the geometry as one really would for a video game. (The architects give us buildings modeled in SketchUp, and they tend to come into the game a bit messy, but also, not modeled the way an experienced video game modeler would do it).

On the programming side, we've got lots done lately. Andy has added a software occlusion culling system, which should help with frame rates in the game. He's also been playing with culling for water reflections. Salvador got animations working in the exporter and in the game, so now we can play animations on entities, which is very useful. (We worked on the game for years and built something fully playable without this ability!) I have been going back and doing the gameplay work for animated entities, as some things have to change in the gameplay logic. In all it is a very nice improvement. Salvador is now working on asset streaming, which will help the game start up faster and run better (and help it run at all on lower-end platforms like the iPad). Ignacio's been doing a whole lot of fixes to various systems and hammering away at the todo list. One of these things has been better control for the tone mapping process. We now get a graph of tone mapping curve:

And we also get the ability to export screenshots into Photoshop that have a color key on them; then the colors can be tweaked in Photoshop and re-imported into the game to alter tone mapper settings. This is in many cases more convenient than playing with those sliders.

We had to do a lot of work to get the game back into playable condition; a lot of things had been broken since the press tour last August / September. (When you are making sweeping changes to a big and complicated game world, it is pretty easy for things to get bent out of shape). So the GDC served as a good internal deadline to have the game banged back into shape.

I didn't expect a lot of press coverage to come out of this, because at the GDC there's just a lot of stuff going on, a lot of things for people to pay attention to and write about. So I had been doing this mostly as an opportunity to keep the press up-to-date with the game, see how it is evolving graphically, etc. But, to my surprise, we got a lot of coverage. Here are links to what people are saying about the game lately:

First, a video interview on Gamespot (excerpted; we probably talked for 45 minutes or an hour).

Kirk Hamilton at Kotaku finds a musical treatment for The Witness' game design philosophy.

Ben Kuchera at the Penny Arcade Report, with a piece about online distribution channels, and another biz-focused one.

Ben Gilbert at Joystiq, with a design-oriented posting and also a more biz-like one.

Alex Rubens from G4 with an overall preview of the game.

Daniel Starkey of Destructoid, with a preview as well.

As you can see from the write-ups, people seemed to really dig the game. So that is all good.

Keep in mind that, because I am known for my previous game and and the reporters are there talking directly to me, the tone that tends to get adopted in these write-ups makes it sound as though I were making the game myself. That's not the case; there's a good-sized team of very talented people building the game! (This reminds me that we need to update the About page to list everyone who is currently working on the project.)

The current puzzle count is 440.

As expected in the previous update, over the past week or so I went back and worked on the endgame again. I built an original version of the endgame last summer; I had one friend playtest it, and it worked out pretty well, though I felt it wasn't quite epic enough. I didn't know what more to do with it at the time, so I just let it sit, and went to thinking about other parts of the game.

Lately I had more ideas for the endgame, so I went into personal crunch mode and rebuilt into something more epic. This was one of the most productive periods of my working life (maybe the most productive) and I think this area, all taken together, may be the most interesting thing I have ever designed. However, it may be a little too difficult for the player, at the moment! Today I playtested this endgame and one other area of the game, with a game designer friend of mine, and it took nine and a half hours to play through these two areas (6 hours for the endgame, 3.5 hours for the other area). I think this is a little long, so I am going to be tuning these areas and cleaning them up a little bit. This weekend hopefully I can get in some playtest time with Jeff and Casey of Jeff and Casey Time, so if that happens we will see how it goes.

(It's interesting; I haven't done a playtest of The Witness for a while, but I was still thinking of it as a 10-15 hour game, and that's what I tell people when they ask. But if these two areas by themselves, even after being cleaned up, are 6-7 hours of playtime, that means the whole game is ... ... much longer. Sometime in a month or two, I'll do some playtests with people who have never seen the game before, and we'll see how long it really is.)

Meanwhile, on the art side, we've been working on the style of the game; expect a future blog post as we come to conclusions on that.

Tech-wise, Andy has been working on performance stuff. First he made raycasting much faster; til now the game had been relatively lazy about spatial organization. Andy implemented a quadtree for finding entities in the world and enabled k-d tree generation and serialization for casting rays against individual meshes. (We had k-d tree code already as part of a library but weren't using it.) Salvador is working on in-game animation playback, as well as exporting; until now we have been making do without animations, but once this is in the game it will help us with a number of issues.

Our gift to you, on this very special Valentine's Day, is the worldwide release of the long-awaited game Dear Esther.

If you haven't heard of Dear Esther, watch this:

Click here to view the video on YouTube.

(Or hey, watch the trailer even if you are quite familiar with the game; the trailer is beautiful and worthy of multiple viewings.)

We expect public reception of this game to run wide: some will love it, and others will be very concerned about whether this thing can be called a game and what that means. So far, this has certainly been the case in pre-release reviews.

Game Informer scored the game an 8/10, saying: "You should consider checking out Dear Esther the same way you’d appraise a film. If you’re interested in absorbing an intellectual story and gorgeous visuals without having to exert a drop of effort, take a chance on this curious experiment."

VideoGamer.com scored the game a 9: "Discovery is such an important part of Dear Esther, especially when everything is so phenomenally pretty."

Meanwhile, Destructoid gave the game a lowly 4.5/10: "It’s as if it wants to be a part of this wonderful medium of ours without asking itself why, which is exactly why you should seek it out and learn from its failures as a game enthusiast, critic, or developer."

We like that there's such a big difference of opinion because it means the game is breaking new ground. It's playing in territory that is not safe; there is no established understanding there.

Dear Esther is a game that no publisher would have funded. Dan, Rob, Jessica and the other associates of thechineseroom have done an excellent job putting together a beautiful game. We are happy to be backing it; we hope you enjoy playing it.

If you'd like more information about Dear Esther, here's an interview with Dan, and here's a link the game's page on Steam.

Update: Dear Esther has been wildly successful, selling 16,000 copies in under 24 hours. As an investment, it reached profitability in 5.5 hours. More details are available at a new post on the Indie Fund blog.

This is because this corner of the world is one of the last areas that we haven't really figured out, in terms of what goes where. There is sort of just some random stuff dumped there right now. That will change pretty soon.

Expect the next few island snapshots to show a lot of big changes. We've reached the point where the basic gameplay is settled, I'm refining many of the specific puzzles, and we've got at least rough drafts of many of the architectural layouts. That means we can start planning the island a bit better. Over the next month or so, we're going to be changing the shape of the island to something a little more geologically plausible, something that carries more information. It's going to be interesting to do this without disrupting the gameplay layout of the island too much.

The current puzzle count is 427. In January I scrapped and re-designed an area and its 60-or-so puzzles. The previous area was inside dark caves, which is a bit cliche, and the gameplay hook that motivated my setting that area in caves, which I had hoped would be cool, didn't work very well. In addition, I felt that the puzzles themselves, while fitting into the rest of the game, were the weakest part of the game. I'd been feeling this way for a long time, but I didn't have a good solution for it til late December. I moved the location and redesigned the puzzles. Now the theme of the location is totally different, has a gameplay hook that works way better than the cave thing, and is very cool. The puzzles situated in this area are also much stronger than before, because I found a way to change the rules of those puzzles to make them much tighter. They'd felt wishy-washy before (people would play through that area and, after finishing, lack clarity on what the puzzles were saying. That's no good.)

We certainly could have shipped the game with the old area in the caves with the old puzzles, and probably shipped the game at least a month sooner, but an important skill in building a beautiful game is knowing when something just isn't good enough.

Next I am going back to the endgame, which I did my first pass at last summer sometime; I've got a gameplay hook to add which I think will bring it over the top and make it extra-good. I'll be working on that over the next few weeks while also working with the landscape architects on the island revision.

Also, we've got a new programmer starting soon (this week or next!) Combined with our other new addition, who's been here a few weeks (Andrew Smith, ex-Zipper and -Oddworld; hi Andy!) we will have quite a bit of tech manpower to bring to bear on finishing this game.

The trailer has just been released:

Click here to view the video on YouTube.

See more about Dear Esther at the official site.

This will be the second Indie Fund game to release. Just a few days ago we announced that Q.U.B.E., the first funded game, is already a financial success. It appears obvious that Dear Esther will be as well.

A small team with limited manpower, we wanted a low-effort way to use our existing version control system (svn) to facilitate concurrent world editing for our open-world 3D game (i.e. we desire that multiple users can edit the world at the same time and merge their changes).  We developed a system wherein the data for each entity is stored as an individual text file.  This sounds crazy, but it works well: the use of individual files gives us robustness in the face of merge conflicts, and despite the large number of files, the game starts up quickly.  Entities are identified using integer serial numbers.  We have a scheme for allocating ranges of these numbers to individual users so there is no overlap (though this scheme could certainly be better).  The system seems now to be robust enough to last us through shipping the game, with seven people and one automated build process concurrently editing the world across 18 different computers.

We don't know whether this method would work for AAA-sized teams, though it seems that it might with some straightforward extensions.  But for us the method works surprisingly well, given how low-tech it is.

About the Game

We have been developing a 3D open-world game called The Witness since late 2008.  For much of that time, development happened in a preproduction kind of way, where not many people were working on the game.  Recently, though, we went into more of a shipping mode, so the number of developers has grown: we're up to about 10 or 11 people now (pretty big for a non-high-budget, independent game).  Not all of these people directly touch the game data, but about 7 do now, and this number will rise to 9 in the near future.

The design of the game requires the world to be spatially compact, relative to other open-world games, so it is far from Skyrim-esque in terms of area or number of entities.  At the time of this writing, there are 10,422 entities in the game world.  I expect this number to grow before the game is done, but probably not by more than 2x or 3x the current amount.

Because the world is not too big, and some game-level events require entities to coordinate across the length of the world, it makes sense to keep all the entities in one space, that is to say, there is no zoning of the world or partitioning of it into separate sets of data.  This makes the engine simpler, and it reduces the number of implementation details that we need to think about when editing the world.  It also introduces a bit of a challenge, because it requires us to support concurrent editing of the world without an explicit control structure governing who can edit what.  One such control structure might be: if you imagine the world were divided into a 10x10 grid; so there are 100 different zones; and where anyone editing the world must lock whatever zones they want to edit, make their changes, check them in, then unlock those zones; this would provide a very clear method of ordering world edits.  (Because of the locking mechanism, two people would not be able to change the same world entity at the same time, so there doesn't have to be any resolution process to decide whose changes are authoritative). However, the straightforward implementation of such a control scheme would make for inconvenient workflow, as it is tedious to lock and unlock things.  Even with further implementation work and a well-streamlined UI, there would still be big inconveniences (I really want to finish building a particular puzzle right now, but a corner of that puzzle just borders on some region that a different user has locked, and he is sick this week so he probably won't finish his work and unlock it until next week.  Or, if my network connection is flaky then I may not be able to edit; if I am off the net then I definitely could not edit.)

Thus, rather than implementing some kind of manual locking, I found it preferable to envision a system that Just Magically Works.  Since we already used the Subversion source-control system to manage source code and binary source assets such as texture maps and meshes, it didn't seem like too big of a stretch for Subversion to handle entity data as well.

The World Editor and Entity Data

Our world editor is built into the game; you press a key to open it at any time.  Here's what it looks like:

The editor view is rendered in exactly the same way as the gameplay view, but you fly the camera around and select entities (I have selected the red metal door at the center of the scene, so it is surrounded by a violet wireframe).  In the upper right, I have a panel where I can edit all the relevant properties of this Door; the properties are organized into four tabs and here you can see only one, the "Common" properties.

These properties, which you can edit by typing into the fields of this Entity panel, are all that the game needs to instantiate and render this particular entity (apart from heavier-weight shared assets like meshes and texture maps).  If you serialize all these fields, then you can unserialize them and generate a duplicate of this Door.  That's how the editor saves the world state: serialize every entity, and for each entity, serialize all fields.

Originally, the game saved all entities into one file in a binary format.  This is most efficient, CPU-performance-wise, and the natural thing that most performance-minded programmers would do first.  Unfortunately, though, svn can't make sense of such a file, so if two people make edits, even to disjoint parts of the world, svn is likely to produce a conflict; once that happens, it's basically impossible to resolve the situation without throwing away someone's changes or taking drastic data recovery steps.

Our natural first step was to change this file from binary into a textual format.  Originally I had worries about the load-time performance of parsing a textual file, but this turned out not to be an issue.  I decided to use a custom file format, because the demands in this case are very simple, and we have maximum control over the format and can modify it to suit our tastes.

Many modern programmers seem to have some kind of knee-jerk inclination to use XML whenever a textual format is desired, but I think XML is one of the worst file formats ever created, and I have no idea why anyone uses it for anything at all, except that perhaps they drank the XML Kool-Aid or have XML Stockholm Syndrome.  Yes, there are XML-specific tools out there in the world, and whatever, but I didn't see how any such tool would be of practical use to us.  One of our primary concerns is human-readability, so that people can make intelligent decisions when attempting to resolve revision control conflicts.  XML is not particularly readable, but we can modify our own format to be as readable as it needs to be in response to whatever real-world problems arise.  You'll see a snippet below of the format we use.

Evolution of the Textual Format

The first version of our textual format was a straightforward adaptation of our binary format.  The gameplay-level code that saves and loads entities is the same for either format; the only difference is just a boolean parameter, choosing textual or binary, which is passed to the lower-level buffering code.  After the buffer is done being packed, it's written to a file.

In our engine, entities are generally referenced using integer serial numbers (that act as handles).  These numbers are the same when serialized as they are at runtime.  Originally, in the binary file, the order in which the entities were written didn't matter.  But if we want to handle concurrent edits, we want to make merging as easy as possible for the source control system, so we sort entities by serial number, writing them into the file in increasing order.  Thus we didn't generate any spurious file changes due to haphazardly-changing serialization order.

In theory, Subversion could now handle simultaneous world edits and just merge them, the same way it does with our source code.  Unsurprisingly, though, this initial attempt was not sufficient.  Though the file was now textual, it contained a bare minimum of annotations and concessions to readability.  Though I knew I would eventually want to add more annotations, I didn't want to go overboard if it wasn't necessary, because these inflate the file size and the time required to parse the files (though I never measured by how much, so it's likely the increases are negligible performance-wise and I was engaging in classic premature optimization).  As it happens, the kind of redundant information that increases human-readability also helps the source control system understand file changes unambiguously, so as we made the format more-readable, the number of conflicts fell.

Here's what our file format currently looks like, for entity #6555 (the Door we had selected in the editor screenshot).

Door
72
6555
; position
    46.103531 : 42386a04
    108.079681 : 42d828cc
    8.694907 : 410b1e57
; orientation
    0.000000 : 0
    0.000000 : 0
    -0.069125 : bd8d9184
    0.997608 : 3f7f633d
; entity_flags
    160
; entity_name
    !
; group_id
    4059
; mount_parent_id
    6554
; mount_position
    0.035274 : 3d107ba5
    0.006069 : 3bc6e19c
    0.000038 : 381ffffc
; mount_orientation
    0.000000 : 0
    0.000000 : 0
    0.000027 : 37e5f004
    1.000000 : 3f800000

(The first line, "Door", is the type name; 72 is the entity data version, which is used to load older versions of entities as their properties change over time; 6555 is the entity's ID.  root_z and cluster_id (seen in the screenshot) are not saved into the file, because these properties have metadata that say they are only saved or loaded under certain conditions which are not true here.  For the sake of making the listing short, I've shown only the properties listed on that first tab in the screenshot.)
You'll note that the floating-point numbers are written out a bit oddly, as in this first line under 'position':

    46.103531 : 42386a04

This is how we preserve the exact values of floating-point numbers despite saving and loading them in a text file.  If you do not take some kind of precaution, and use naive string formatting and parsing, your floating-point values are likely to change by an LSB or two, which can be troublesome.

The number on the left is a straightforward decimal representation of the value, written with a reasonable number of digits.  The : tells us that we have another representation to look at; the number on the right, which the : introduces, is the hexadecimal representation of the same number.  The IEEE 754 floating-point standard defines this representation very specifically, so we can use this hexadecimal representation to reproduce the number exactly, even across different hardware platforms.  The decimal representation is mostly just here to aid readability.  However, if we decide we want to hand-edit the file, we can just delete the : and everything after it, changing that first number to a 47.5 or something; seeing no :, the game will parse the decimal representation and use that instead.

There are other ways to solve this floating-point exactness problem; for example, you can implement an algorithm that reads and writes the decimal representation in a manner that carefully preserves the values.  This is complicated, but if you're interested in that kind of thing, here's what Chris Hecker says:

Everybody just uses David Gay's code:
http://www.netlib.org/fp/dtoa.c
http://www.netlib.org/fp/g_fmt.c

You need to get the flags set right on the compiler for it to work. /fp:precise at least, I also do #pragma optimize ("p",on), and make sure you write a test program for it with your compiler settings.

(Twitchy, and look at how much code that is!  What we did is a couple of lines.) In a document on his web site, Charles Bloom discusses a scheme that was in use at Oddworld Inhabitants, similar to what we're doing in The Witness but involving a comparison between the two representations:

... Plain text extraction of floats drifts because of the ASCII conversions; to prevent this, we simply store the binary version in the text file in addition to the human readable version; if they are reasonably close to each other, we use the binary version instead so that floats can be persisted exactly.

Subversion Confusion

After switching to this format, conflicts still happened.  I categorize conflicts into two types: "legitimate", where two people edit the same entity in contradictory ways and the source control system doesn't know who to believe; and "illegitimate", where Subversion was just getting confused.  We did have a number of legitimate conflicts for a while, yet which were not the users' fault, because our world editor tended to do things that are bad in a concurrent-editing environment.  (See the Logistics section at the end of this article for some examples).  But even after we ironed these out, we still found that Subversion was still getting confused.

Here's an example of a common problem: suppose we start with a world containing data on entities A, B, and D, so our file contains: ABD.  You edit the file and delete entity B.  I edit the file and also delete entity B, but add another entity C.  (That we both deleted B may be a very common use-case, especially if this is due to some automatic editor function).  So, you want to check in a file that just says: ABD - B = AD, so the new result is AD.  I want to check in a file that says: ABD - B + C = ACD (the C goes in the middle, because remember, the entities are sorted!)  In theory this is an easy merge: the correct result is just ACD.  But Subversion would get confused in cases like this, requiring the user to hand-merge the file.

This is a disaster, because even with a visual merge tool such as the one that comes with TortoiseSVN, you don't want non-engine-programmers trying to merge your entity data file.  Even if that file is extremely readable, at some point someone will make a mistake; the file will fail to parse, and then you have a data recovery problem that requires the attention of an engine programmer.  When this happens it kills the productivity of two people (the engine guy, because he has to merge the data, and also the guy who generated the conflict, because until it gets resolved his game is in an un-runnable state, and anyway, the engine guy has probably commandeered the other guy's computer to fix this).

After we had this problem a few times too many, Ignacio suggested moving to a more-structured file format like JSON, which, in theory, source-control systems would handle more solidly.  But I thought that this would only reduce the frequency of the problem, but not solve anything fundamental; even if we achieved a situation of only generating legitimate conflicts, each conflict would still be a potential game-crippling disaster.

Splitting the File

It occurred to me, finally, that it could hardly get cleaner than to store each entity in its own file.  This initial thought might cause a performance-oriented programmer to recoil in horror; we have 10,422 entities right now, and it must be very slow to parse and load 10,000 text files every time the game starts up!

But I knew that we already had over 4,000 individual files laying around for assets like meshes and texture maps, currently loaded individually, the idea being that at ship time we would combine these into one package so as to avoid extra disk seeks and the like.  But on our development machines, these files still loaded quickly.  I figured that if we can already load 4,000 big files, why not try loading an extra 10,000 small files?  If loading all these files turned out to be slow, we could resort to this only when entity data has changed via a source control update; after loading all the text files, we can save out a single binary file containing all the entity data, which would be much faster to load, just like we used to do before we supported concurrent editing.

I tried this and it worked great.  So, now we have a data folder called 'entities', which holds one file per serialized entity.  The Door selected in the editor screenshot is stored in a file named after its entity ID, "6555.entity_text", which contains the text you see in the listing above.

As mentioned, there are currently 10422 entity files; the text files take up about 8MB of space, whereas the binary file is 1.4MB.  We're not making any effort to compress either of these formats; they are very small compared to the rest of our data.

Benefits of Using Individual Files

Storing each entity in a separate file solved our problems with merging: both the illegitimate source control confusions as well as the problems due to hand-guided file merging.

The illegitimate conflicts go away because transactions become very clear.  If I delete an entity, and you delete an entity, then we both just issue a file delete command, and it's very easy for the source control system to see that we agree on the action.  Furthermore, if we both delete B but I add C, because the change to C is happening in a completely separate file, there is no way for the system to get confused about what happened.

Legitimate conflicts do still happen, as multiple people edit the same entities, but these cases are much easier to handle than they were before.  The usual response is to either revert one whole entity or stomp it with your own.  Because we usually want to treat individual entities atomically -- using either one person's version of that entity or the other's -- individual files are a great match, because now the usual actions we want to perform are top-level source control commands (usually from the TortoiseSVN pulldown menu); before, this wasn't true, and merging a single entity in an atomic way required careful use of the merge tool.  If an entity ever gets corrupted somehow, for whatever reason, we can just revert that one file, knowing the damage is very limited in scope; before we've reverted it, the worst that can happen is that that single entity will not load; it won't prevent the rest of the entities in the world from loading.

It would be difficult to overemphasize the robustness gained.  I feel that these sentences do not quite convey the subtle magic; it feels a little like the state change when a material transitions from a liquid to a solid.

Sometimes, someone will update their game data in a non-vigilant way; not noticing that a conflict happened, so they won't resolve it; and then they will run the game.  Before, this would have been a disaster; Subversion inserts annotations into text files to try to help humans manually resolve conflicts (this is an unfortunate throwback to programs like SCCS), which would have caused much of the world to fail to parse.  Now, though, the damage from these annotations is constrained to individual entities. It's also easier to detect: if there is a conflict in the file for our Door #6555, then in addition to writing these annotations into 6555.entity_text (making it unparseable by the game), Subversion will also write out files 6555.entity_text.mine and 6555.entity_text.theirs, which contain the two intact but conflicting versions of the file.  The game can detect these very straightforwardly, put an error message on the HUD, and even choose one to load by default.  It is all very clean.

This is one of those interesting cases where reality-in-practice is starkly different from computer-science-in-theory.  In theory, a directory is just a data structure organizing a bunch of black-box data, and we happen to view this organization in a way that looks like a list of files; we could just do the same thing interior to our own file format, and it would be no big deal.  In practice, due to the way our tools work, these two arrangements are tremendously different.  The individual files just feel solid and reliable to work with, whereas one big structured file is always going to be precarious.

Startup Procedure

When the game starts up, we need to load all the entities from disk.  In the simplest case, this is straightforward: we just iterate through every file in the entities directory and load it.  However, as I mentioned, we wanted to keep a binary version of the entities that is preferred by the game if it exists and is up to date; partly, this might help startup speed a little (we don't have to parse all these strings), but mainly, that is the file format and code path we are going to use when the game ships, and we don't want that code to rot in the meantime.  Using it every day is the best way to prevent it from rotting.

So: when the game first starts up, we check for the binary entity package.  If it's not there, we load all the individual entities files, then immediately save out the binary package.  (This binary package is a private, local file; if we were to add it to source control, everyone would always conflict with everyone else.)  But suppose we start up and the binary package is already there (which it usually will be).  We need then to determine if any of the textual entities have changed since the binary package was built.  The most-general way to do this is just by looking at the file timestamps; if we were to rely on some other signal being written to the disk that says "hey, entity data has changed," then we would only be allowed to modify the entity files using sanctioned and customized tools; that would be annoying and would cause all kinds of robustness issues.

In concrete terms: we want to know the timestamp of the newest .entity_text file, inclusive of the directory containing these files (if we just did a Subversion update that only deleted entities, then all the .entity_text timestamps will be old, but the directory timestamp will be new; we want to ensure we properly rebuild the binary file in this case!)  To find the newest timestamp, we just iterate through all the timestamps in a brute-force way:

bool os_get_newest_file_modification_time(char *input_dir_name, Gamelib_Int64 *file_time_return) {
    //
    // We include the modification time of the directory as well.
    //

    char *dir_name = mprintf("%s/*", input_dir_name);
    Free_On_Return(dir_name);
    if (strlen(dir_name) >= MY_MAX_PATH - 5) return false;

    //
    // best_time starts as the time for the directory (if it exists!)
    //
    Gamelib_Int64 best_time = 0;
    bool success = get_file_modification_time(input_dir_name, &best_time);
    if (!success) {
        Log::print("Unable to get modification time for directory '%s'!\n", input_dir_name);
    }

    //
    // Now consider the file timestamps.
    //
    WIN32_FIND_DATA find_data;
    HANDLE handle = FindFirstFile(dir_name, &find_data);
    if (handle == INVALID_HANDLE_VALUE) return false;

    while (1) {
        if (find_data.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) {
        } else {
            Gamelib_Int64 file_time = to64(find_data.ftLastWriteTime);
            if (file_time > best_time) best_time = file_time;
        }

        BOOL success = FindNextFile(handle, &find_data);
        if (!success) break;
    }

    FindClose(handle);

    if (file_time_return) *file_time_return = best_time;

    return true;
}

bool get_file_modification_time(const char *name, Gamelib_Int64 *result_return) {
    HANDLE h = CreateFile(name, 0, FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE,
                          NULL, OPEN_EXISTING, FILE_ATTRIBUTE_READONLY | FILE_FLAG_BACKUP_SEMANTICS,
                          NULL);
    if (h == INVALID_HANDLE_VALUE) return false;

    FILETIME write_time;
    GetFileTime(h, NULL, NULL, &write_time);
    CloseHandle(h);

    if (result_return) *result_return = to64(write_time);
    return true;
}

You'll notice that this is Windows-specific code, but we don't have to worry about porting it, because we only run the world editor on Windows.  Fortunately, in Windows, the file timestamp is right there in the directory data, so we don't have to open each file.  (At least in NTFS this is true, rather than being an API illusion; the timestamps are stored in the Master File Table [MFT].)

What's the performance like?  I typically run the game on two different machines: a high-performance Tower PC (CPU: Intel Core i7-970, hard drive: OCZ Vertex 3 SSD, RAM: 12GB) and a gaming Laptop (CPU: Intel Core i7-2820QM, hard drive: Intel SSD 500 [SSDSC2MH250A2], RAM: 12GB).

The time spent in os_get_newest_file_modification_time is:  Tower: .0095 sec, +/- .0015 sec; Laptop: .039 sec, +/- .005 sec.  (Timings were computed by averaging six trials, after a discarded warm-up run.)  This is definitely one of those cases where I'm glad I didn't prematurely optimize, as I would have expected this to be much slower!

We typically install SSDs in all our work machines, but even when people have home machines that use spinning-platter hard drives, startup times are not much slower (likely due to the MFT data staying in cache).

If for a large enough number of entities this timestamp-gathering process becomes too slow, we can always write some code to read the timestamps out of the MFT directly, avoiding all the API call overhead, as in this forum posting.  To date we've had no need to do this.

Some people have reported performance problems when Windows has more than 100,000 files in one directory, but this can be worked around: just make 10 directories, with IDs ending with each digit from 0-9 grouped in common directories.  (Why use the least-significant "ending" digit, rather than the most-significant "starting" digit, which might be the impulsive choice?  Because if you do it that way, Benford's Law says your entities are likely to be unevenly distributed across the directories.  Why not just hash them into a folder?  Because it's useful to be able to manually navigate to the entity you want, without searching.  (However, partitioning by least-significant digit may be inconvenient if you want to manually edit a sequential range of entities for some reason, because that range will span many directories.  If this is an important use case, perhaps there's a good compromise like using the tens digit or hundreds digit -- though I have the itching feeling that a clearly superior solution would exist.))

As things stand in terms of editing the game world, we are probably not yet at the peak entity count, but there's unlikely to be more than 3x the number of entities we have now.  Given how low the timing measurements are now, it's clear that performance will not be a factor in the ability to use this system up until shipping the game.

Partitioning the ID Space

As I mentioned, in our engine, entity IDs are integer serial numbers.  We like this because they are easy to type (we have to type them in the editor sometimes) and at runtime, looking up an entity compiles to a fast indexing of a flat array, with a bounds check.

This introduces another problem for us to deal with in world editing, though: we need to ensure that two users, who are both creating new entities, do not both try to use the same number.  This would result in both users attempting to add completely incompatible .entity_text files with the same name.

One obvious and bulletproof solution is to have the editor connect to the network and atomically grab an ID from a central server each time one is needed.  But again, we don't want to require users to be on the network in order to edit.

My solution was a somewhat lazy one: we allocate a range of IDs to each computer that edits the world.  These are kept in a file called "id_ranges"; at startup time, the game queries its machine's MAC address, then looks in the file for the corresponding ID range; if not found, it claims a new range and adds a new line to the file:

 6000 10000 BC-AE-C5-66-28-E9 #Jonathan_office
10000 13000 BC-AE-C5-66-28-48 #Ignacio_office
13000 16000 00-1C-7B-75-00-D8 #Jonathan_laptop
16000 19000 BC-AE-C5-66-28-93 #Shannon_office
19000 22000 BC-AE-C5-56-42-51 #Jonathan_home
22000 25000 C0-3F-0E-AA-F3-F2 #icastano_home
25000 28000 F4-6D-04-1F-D4-36 #Shannon_home
28000 31000 Baker #Baker
31000 34000 BC-AE-C5-56-43-FE #Misc_PC_office
34000 37000 00-E0-81-B0-7A-A9 #eaa
37000 40000 00-21-00-A0-8F-E3 #Orsi
40000 43000 64-99-5D-FD-97-64 #Jonathan
43000 46000 08-86-3B-42-0B-8D #Misc
46000 49000 90-E6-BA-88-9B-B7 #andrewlackey
49000 52000 48-5B-39-8C-80-5E #eaa
52000 55000 50-E5-49-55-64-BF #facaelectrica
55000 58000 00-FF-D4-C7-15-A3 #facaelectrica
58000 70000 Clusters #Clusters
71000 75000 Overflow #Overflow
75000 78000 F4-6D-04-1F-CC-77 #Nacho
78000 81000 BC-AE-C5-74-7B-A3 #Thekla

On each line, everything after the # is just a comment telling us in a human-readable way whose machine is responsible for that range (it defaults to the Windows username, but we can hand-edit that if we want).

("Baker" is for an automated process, so I made a special case that uses the string "Baker" instead of the MAC address, just to minimize the potential for the build machine to claim a new range without us noticing and failing to check that in; see below.  "Clusters" is for an editor operation that is also run in an automated process, but which can also be done manually by anyone; cluster generation removes all previous clusters from the world, so if clusters didn't have their own space, you would end up frequently deleting entities in other peoples' ranges; this seems like it might be a bad idea, though maybe, in reality, it's fine and there would be no problem.  But as it stands we know that anything in the Clusters range is completely disposable, so if a conflict ever arises there we can just revert or delete those entities without a second thought.)

This ID space is naturally going to have a lot of big holes in it, but before we ship we will renumber the entities so that we can pack them tightly into an array.  In the meantime, it is sparse but not pathologically so; we can still get away with using a linear array to index the entities, though it will use more memory now than at ship.

This id_ranges file is a bit of a lazy solution, and it causes a few annoyances.  The most annoying thing is that, any time someone runs the editor for the first time on a new machine, they need to immediately check in the id_ranges file to publish their allocation to the rest of the team; and they need to coordinate so that nobody else is trying to do the same thing.  Because the editor just allocates the next 3000 IDs starting with the highest ID in the file, if two people try to allocate IDs simultaneously, they will both claim the same range.  We could solve this by requiring the editor to hold a source control lock to modify the id_ranges file, but I just haven't gotten around to it, because this hasn't been a big-enough problem.  Furthermore, we could have the editor automatically commit the changes to the file; but currently, nothing is ever published to the rest of the team automatically -- publishing always requires manual action, and this seems like it may be a good thing to preserve.

But, not having implemented these solutions, we have been taken by surprise a couple of times by interactions with a second lazy choice of this system: the fact that we file only records one MAC address per computer.

Most modern computers have multiple MAC addresses (for example: if your computer has a Wifi adapter and two Ethernet ports, it's probably got at least 3 MAC addresses).  My lazy code just asks Windows to iterate the MAC addresses and reports the first one for use in the file.  Sometimes, though, the reported address will change, which causes someone's machine to modify the id_ranges file when they didn't expect it; we had this problem once when an audio contractor, who works offsite, installed some VPN software.  If that user is not vigilant about source control (which non-technical people often aren't), it can be a while before this problem is discovered; meanwhile, that user has been creating entities that may conflict with someone else's.

Most or all occurrences of this problem could be solved by recording all of the user's MAC addresses in the file, instead of one; I just haven't done so.

If we weren't using integer serial numbers, we wouldn't really need id_ranges at all.  For example, if we looked up entities at runtime by hashing unique string identifiers, then we would only need to ensure that each user has a unique string prefix.  We take a slight overkill approach, where each entity ID is a somewhat long string starting with the user's MAC address; or we could even generate a Windows GUID for each entity.  MAC addresses are 48 bits wide.  If you concatenate the MAC address with a 4-byte integer, that's just 10 bytes per entity ID, which isn't much (we currently use 4, already!).  Of course encoding these as strings and storing them in a hash table is going to require more space, but it isn't ridiculous.  Looking things up in a hash table all the time is slower than indexing a flat array, so we are sticking with the way we do things for now; but this would solve some problems and simplify the overall system, so it sounds like the kind of decision I would change my mind on later, as computers get even faster than they currently are.  So if you implement a scheme like this, you can ignore everything I said about id_ranges.  The biggest problem becomes making it easy to type longer entity IDs into the in-game editor.

Logistical Issues in Editor and File Format Design

In closing, I want to discuss changes we made to the editor and file format, in order to make them compatible with the paradigm of concurrent editing.

Arrays

Just about every game wants to serialize arrays into files.  When transitioning from a binary format, the most natural thing for a performance-minded programmer is to write the array length out first, then write each member of the array.  That way, when reading the file back, we know how much memory to allocate, and we avoid all the slow things one has to do when reading an array of unknown length.  However, encoding arrays in this way results in a textual format that always conflicts, because even when people are making compatible edits to an array, nobody is going to agree on what that array count should be; also, it becomes impossible to manually edit conflicts without typing the count in: if two people add elements to an array, the merged array will contain all the items, but neither version of the file will contain the correct count!

To solve this, of course, we go to the common text-file convention of using a syntactic designation that an array is now over, rather than pre-declaring how many elements there are.  This is obvious, but I mention it here because it is a relatively universal concern.

Automatic Changes

You will also want to minimize or work around cases where the editor automatically changes entity properties without user intervention, because often these will lead to conflicts.  A simple example from our game world: the sun is represented by a few entities that are instantiated in the world, a glowing disc and a couple of particle systems.  Every frame, the gameplay code positions these entities with respect to the current camera transform so that, when the player moves around the world, the sun will seem to be infinitely far away.  This happens in the editor too, because when editing we want to see the world exactly the way it will look in-game.  So, as soon as we enabled concurrent editing, everyone was checking in different coordinates for the sun (depending on where the camera happened to be when they saved!) and this caused a lot of conflicts.  My first solution was to add a hack that zeroes the position of sun entities when saving, but upon loading, this interacted badly with the way particle systems interpolate their positions.  Instead, I added an entity flag that signifies that an entity's pose doesn't matter, with respect to thinking about whether it has changed or not, so we will only modify the sun's .entity_text files if some more-important properties are modified.  (This article is already long, but I'll say that in general, we detect differences between an entity's current state and its state when it was loaded, and we only write out .entity_text files when properties have changed.)

Relative Transforms

In our engine, all entities have 'position' and 'orientation' properties, which always represent the entity's frame in worldspace, flat and non-hierarchically.  If we want an entity to be attached to another entity's transform, we set the properties 'mount_parent_id', 'mount_position', and 'mount_orientation' (you can see all these properties in the listing for Door #6555 above); once per frame, the game engine computes the entity's new worldspace transform based on its mount information, properly handling long dependency chains.  Once we started using textual entity files, we found that entities' worldspace coordinates would often differ by an LSB or two, even when their parents had not moved (differences especially happening, for example, between Debug and Release builds of the game).  Even without concurrent editing, it's good to clean this up as a matter of basic hygiene, because otherwise you're checking in spurious changes all the time, increasing load on the source control system and making it harder for a human to see useful information in the change list.  I chose to solve this problem by making the mount-update code more complex: it detects whether an entity has moved of its own accord, and only updates child entities' worldspace positions if the parent entity really moved (or if the child entity really moved!).

Terrain Editing

There is one piece of editing workflow that used to make sense before concurrent editing, but which now does not.  We have a terrain system wherein we can move control points around in worldspace, then select a terrain entity, which has a height-field mesh, and press a key to recompute that height field with respect to the new control points.  I originally made this recomputation happen only on manually-selected entities, rather than automatically on the entire world, for UI smoothness purposes: the recomputation takes some time.  But when you only recompute one block of terrain and not its neighbors, you get discontinuities in the terrain; the longer things go on, the more discontinuities you get.  Once in a while I would fix this by selecting all terrain entities in the world, recomputing them simultaneously.

A terrain discontinuity

The catch is that when terrain changes, it affects other entities.  We re-plant grass, so that the grass matches the new elevations; we change the positions of trees and rocks and other objects that are rooted to the terrain at a predetermined height.  Whenever you recompute some terrain, we collect all grass/trees/rocks within a conservative bounding radius, as these are all the things that might possibly change.  When dealing with small groups of terrain blocks, this already is a bit of a recipe for trouble (it is not too hard to overlap someone else's edits this way); but it also means that if I recompute the entire terrain to eliminate those discontinuities, I am in turn editing all grass, trees, and rocks in the world, and will inevitably conflict with anyone else who has moved any of them. For now, we simply shy away from this kind of global operation; when you get a conflict in some entity that you probably didn't care about, like grass, it is easy just to accept the remote version of the file.  Long-term, I expect some kind of heavier-weight solution would have to happen here, though I hope it does not involve much locking.

Issuing Source Control Commands

Our editor automatically issues source control commands when entities are created or destroyed, because it would be extremely tedious and error-prone to expect users to do this.  Initially we had some problems with source control commands unexpectedly failing, which quickly leads to chaos.  After some work, issuing source control commands seems to work with full consistency, but we are still a little wary of it; so we don't, for example, want to issue the commands via a background process, adding more complexity to the system.  The problem with this is that svn commands are a bit slow.  Sometimes when I have been editing the world for a while, it can take 15 to 30 seconds to save my changes, which would have happened instantaneously without the svn commands.

Conclusion

We've put together a system that lets us concurrently edit an open world, using source control to manage our world edits.  It works well enough for us to ship the game, so I am happy with it.  If the scheme ever fails in some way, or requires heavy modification, I will post an update!

When we sat down to start working with the architects, one of the first things we did was put together a history of the island where The Witness takes place: how long has the island been there?  What groups of people lived there, and what kind of structures did they build?  How did later civilizations use the structures built by earlier groups?

Quickly we developed a running theme of sites that are old and have over time been used in two or three different ways.  When entering a new location, you probably first notice the most basic elements of its structure: where you can walk to easily, versus where you can't; what puzzles or goals are calling for your attention, and what walls or doors might be preventing you from getting to them.  If this is all you care about, you can then play through the area in a very utilitarian fashion.  But if you have an eye for detail, you may notice the elements of modern construction shoring up the design of an older structure with a different purpose.  The design of the new structure tells you something about what the newer guys were doing there, and how it differs from what the older guys were doing there.  And if you look really closely, you can see traces of the original footprint of the site (perhaps these are very subtle, some bricks just rising above the dirt in a few places), and infer the purpose of this very old site too.

If you see the different civilizations that came to this island as embodying different philosophies; and you see the structures they built as representative of the way these philosophies led them to interact with the world; and you see further that when they replaced a site, it represents the rejection of some older worldview that they consider no longer useful, then perhaps you start to get some idea of the amount of backstory that can be encoded into the world, nonverbally.

All of this backstory is also directly relevant to the main story -- who you are, why you are on this island, how you may get back home.  So, to put it another way, the game is constructed so that the more you pay attention to tiny details during your travels, the more insight you will have to the central story, even though it may not be obvious at any given time what a particular detail has to do with that story.

The most-recent issue of Game Informer printed an article that showed some of the first screenshots of the newly-architected areas appearing in the game engine.  It is an interesting article; pick up the issue if you haven't!  They also put up a companion article online (but which not as extensive) here:

(Summary article at Game Informer)

The area featured in this article is the one that we have taken furthest so far in terms of modeling and texturing.  Here are some other shots of it:

The Keep, before and after

The Keep, close-up 1

The Keep, close-up 2

The Keep, close-up 3

We've done basic planning for some of the other areas, so I thought it would be nice to show some screenshots of work in progress.  For each of these, there's a 'before' version (when we had a fully-playable game, but basic placeholder structures) and after (once we have incorporated the architects' designs, revised the site plans and in some cases even the puzzles, etc):

 

The Compound, before and after

Glass Factory, before and after

Peninsula, before and after

Vault, before and after

Don't regard these "after" images as representative of what the game will look like!  They are still works in progress, and the final game will look substantially better: scenes like these will feel richer due to the placement of detail objects, further care put into the texture mapping and lighting, refinement of the still-rough-drafty terrain features around these structures, and many other bits of subtle game development magic.  But I wanted to show these now since they provide a good taste of the game's flavor, and they show how much structural intelligence is brought to the table by working with the architects.  Yes, we could have started with the placeholder structures and made them more elaborate and better-looking, in a general video-game-level-design way, but that's different from having well-thought-out ideas subtly embodied in the structures of the areas, which is what we are going for.

Click here to view the video on YouTube.

Q.U.B.E. will be available on Steam.

(Web site for the game)

(Indie Fund announcement page)

Eventually, these things have to be figured out. Over the past month or so we've done some of that figuring. We nailed down the architecture for a couple of areas (including the all-important area where the player starts the game). We've done this in a way where the architecture and the puzzle ideas have synergy, which is always nice. In particular, there was one location that, from the beginning of development, was basically a big box with a window cut into it and some puzzles stuffed inside. For a long time (maybe 2 years!) I was never sure what to do with it, but eventually the architects suggested an idea for that location. I mocked it up and didn't like it. They suggested another one. This one sparked an idea for how to make the structural properties resonate with the puzzle theme, and when I built even just a rough-draft of the building, I found the result to be strong. Lately we've been finalizing that location too.

So anyway, there are all these spots on the island that represented vague ideas, with puzzles there that were just in rough form, or that weren't even implemented fully, but just served as reminders to do something there. I've been going through and filling in a bunch of those.

I have also been working with some other puzzles, which seemed "done enough to ship", and thinking hard about how to make them better. Braid benefited from a lot of this kind of attention -- a lot of games would have shipped with just early versions of the puzzles in that game, considering them to be just fine; but the more I worked with the puzzles in Braid, the more I found better, simpler, more-to-the-point versions of them. The same thing is happening here (though The Witness is a much bigger, more-complicated game, so sometimes it is challenging to get to that same kind of clarity.)

With regard to the visuals (modeling and texturing), things have been coming along nicely as well. Pretty soon a high-profile game magazine will publish some screenshots of one of the most-finished areas of the game, showing the look we are targeting and giving a strong hint of what the game will be like stylistically. When the magazine is out there we will link it here.

(This has not been a particularly coherent update; just wanted to let you all know what is up!)

Any system of interactivity can of course be explored: If X happens, what are the consequences? What are all the ways in which pattern Y expresses itself, and to what do those expressions lead? By inspecting the structure of a system in this way, we can find the core ideas of the system, and see how those ideas illustrate fundamental truths of our universe. We present a game design aesthetic that values looking for systems that express these truths in the cleanest possible way. We explain how this is different from more-traditional combinatoric design techniques; we show examples from our games and describe a method for applying the aesthetic in general.

Here's a link to the Indiecade page, on which there's also a presentation by Richard LeMarchand, and more to come.

Thanks very much to Ida C. Benedetto for recording the talk!

I'll post something more concrete before too long, but I wanted to at least say something, since it's been a while.

Team-wise, we have just hired two people (a 3D artist and a programmer), and we plan to hire another programmer as well, once his visa clears and he can get into the country. This is a pretty big staffing-up, percentage-wise, and will allow us to make things quite nice. The new folks will be starting over the next couple of months (they need to wind down the things they are working on currently!)

The current puzzle count is 350.

Some parts of the island are pretty messed up in this screenshot. For the press tour, we made sure the whole game was more-or-less solid and playable. Just after that, though, we were free to tear things apart, move things around, change the sizes of areas, add new areas, and all that. We're still in the middle of that process and not everything has been made playable again.

Here's the link.

The difference is not too pronounced in this scene, but with smoother-colored textures it becomes a lot more obvious (and I do want to use more smoother-colored textures, art-direction-wise!)

The situation is, the lightmaps are HDR (for some definition of "high"), so we needed a way to encode those values reasonably into a bitmap. We were using RGBM where M is just an overall brightness factor; you read RGB from the texture then multiply by M to get the output color. We used to just encode this naively, but the problem is, when M is less than 1 then it actually kills precision in your RGB, so the solution is, well, don't ever let M go below 1; just clamp it. This is not rocket science and in fact is the kind of thing that is a little funny, as one starts wondering why we didn't think of that a long time ago, but hey, that's how it is sometimes. The results made me happy, though, so I am posting them.

The improved image quality may enable us to have higher dynamic range throughout the game (this is probably something we will evaluate later on), which would be very nice!

Previews of the game are now being published, and readers of this blog have been asking for gameplay details for a while. So it seems like a good idea to list the previews here. I'll keep updating this post as I find more.

on Kotaku, by Stephen Totilo
on Joystiq, by Ben Gilbert
on Gamespot, by Sophia Tong
on IGN, by Anthony Gallegos
on Gamasutra, by Christian Nutt
on 1UP, by Matt Leone
on GamesRadar, by Charlie Barratt
on Giant Bomb, by Ryan Davis
on Giant Bomb, by Patrick Klepek (interview-ish)
on Time Techland, by Evan Narcisse
on IFC, by Evan Narcisse (interview)
by Dan Griliopoulos on Rock, Paper, Shotgun

interview with GamesRadar
interview with Cory Cohen of OXM
long interview with Dan Griliopoulos of Rock, Paper, Shotgun (contains a bunch of stuff not elsewhere)

interview with Christian Donlan of Eurogamer

(Sometimes these previews, from the way they are worded, make it seem like I made the entire game with my own bare hands. That is not the case; it's just how this kind of article tends to sound. There are a number of people working on this game, who have creative input in various ways; see the About page for a partial list!)

Sun beams. These are accomplished with a radial blur centered on the sun, during postprocessing. The nice thing about this effect is how dynamic it is -- as you walk beneath the trees, the shape and intensity of the beams are changing all the time.

Due to the way the effect works, sometimes beams are drawn that are a little bit unphysical. You probably have to be looking closely to see that, but I am a little bit uncomfortable with it given that the game is all about the opposite of that. We'll see how this evolves. We've been talking about augmenting this with other effects.

A revised water reflection effect.

Softening of the shoreline, so that it doesn't look like one big alpha-blended water polygon intersecting the ground.

Here's what the island looks like now:

It's fun to look at this compared to another shot of the island, say, from about a year ago:

The current puzzle count is 328.

The talk's about 38 minutes long, with about 40 minutes of Q&A afterward. Here's a slideshow:

If the above embedded player doesn't work for you, or you prefer to download the files directly, here they are:

audio track
slides (ppt format)

But we also figured, hey, why not play around with the color of sunlight as well? The idea is that as you walk around in the game, the color of the sun and ambient light will interpolate as you move. We haven't hooked it up that way yet; what you see below is just done by tweaking global variables in the game. Well be doing more experiments in this direction...