Interview: The man that made Alan Wake's face work

Mark Carson is the founder of CaptiveMotion, the motion capture studio that completed the facial animation work in Remedy's Alan Wake. The company brought the thriller's virtual actors to life some six months after the movement on their bodies had been finished, shooting Wake's expressions with up to sixteen times the amount of markers normally used for this type of project. Exacting bunch, Remedy.

Here Mark talks about the present and future of motion capture in games, crossing the uncanny valley and how Heavy Rain was "visually uneven".

Interview by Nathan Grayson.

VG247: Motion capture’s a big part of many videogames today. What makes your work on Alan Wake different? How has your approach improved upon those used by other videogame-centric mo-cap studios?

Mark Carson: To me the biggest difference between our work and tradition mo-cap is the fidelity. When it comes to recreating a performance it’s all about the amount of data you capture and how good that data turns out. For example, traditional facial motion capture uses around 80-100 markers. We shot Alan Wake with over 1,600 markers.

But it’s also the quality of the capture. Tradition facial motion capture uses little reflective beads about the size of a BB. The problem is that data captured is it at the top of the BB and not at the skin level so it basically creates a transitional effect similar to the actor wearing a latex mask. Our markers are flat to the skin so it significantly raises the quality level. Ultimately the final product is dependent on a lot of factors but we designed our technology to not be the limiting factor. And I think that is what really differentiates us: we built a technology from scratch specifically for the needs of next generation gaming.

What drew you to Alan Wake, specifically? How did your studio get involved with the project?

Mark Carson: Alan Wake was brought to us by our European partner metricminds. Metricminds is an mo-cap facility and animation house who spotted our technology early and became a large advocate of our products. We are currently in the process of building an addition to their studio in Frankfurt to house our next generation equipment so the European market will have better access to our software, Embody.

How much creative input did you have on Alan Wake? Were you allowed to improv a bit – do your own thing to spice up the characters and their interactions? Or were you simply handed a list of things to do and told to get to it?

Mark Carson: We had absolutely no creative input and a narrow technical input path into Alan Wake. Alan Wake was a long project from a team that carefully crafts every element of the story. Our part of the project was just a small cog in a very big machine. Consider us ‘spotlight operator #2’ at the Royal Ballet: we know our jobs and try not to make suggestions regarding what happens on stage.

Do you think videogame characters and their interactions will ever cross the uncanny valley and become indistinguishable from actual, flesh-and-blood people? If so, when do you think that will occur?

Mark Carson: Absolutely, in my opinion spanning the uncanny valley has already happened in movies like Benjamin Button and Avatar. I think the bigger question is when it will become commonplace. That’s really where CaptiveMotion comes in. Our system doesn’t do anything that James Cameron hasn’t accomplish with a bucket of money and many years. However, the bucket of money required for our system is a lot smaller and we can process the data a lot faster. That comes from our videogame roots: 90 minutes of animation in 10 years with $300 million is easy. Videogames need 300 minutes in four months for a tiny fraction of the Avatar budget. That’s us.

Which do you think will happen first: photorealism or completely believable, natural facial expressions and movement allowing characters to not look like “dolls”?

Mark Carson: It’s really part of the same problem. Some of the companies we work with like Lightstage have made huge strides in photorealism, and we have increased our detail level 300 percent over the last few years. But each significant advancement by one technology just starts bringing out flaws in the others. I think we will probably get there together and ultimately the final hurdle will be accomplished by a combination of pipeline, process and artistic talent; most of the tech required already exists.

On that note, do you think that sort of graphical leap will be a good thing? Today’s game designers have to be creative in order to craft truly expressive characters. Do you think true photorealism could breed laziness in videogame art styles?

Mark Carson: There is always a side-effect to new art, but the spray can didn’t ruin traditional art. Sure, we got a lot graffiti as a result but we also got Mark Ecko, so everything comes with its pros and cons. Ultimately, new technology will always foster new art that we can’t even imagine today.

Have you had a chance to check out Heavy Rain on the PS3? Pretty impressive, don't you think? How do you think Alan Wake stacks up against it? Would you say Heavy Rain’s characters are the most convincing ever seen in a videogame?

Mark Carson: Unfortunately I don’t own a PS3, but I’ve pored over the reviews and extended play video from the various sites. It’s beautiful and brilliant but visually uneven at points, which unfortunately is the nature of the industry because of the scope, budgets and timeframes required. I would have loved to have worked with them on that project. I think we could have help with that. In the end it’s a very different product from Alan Wake and I think everyone will agree once it comes out. The exciting thing is that as an industry we are seeing the products; character driven, empathic and cinematic projects. It only gets better from here.

Heavy Rain’s also an interesting example in that the game’s writing actually isn’t all that incredible. Sure, the story and choices are revolutionary, but the writing itself is a bit weak. However, the verdict seems to be that the game’s characters are incredibly believable. How much do you think accurate modeling and mo-cap played a role in this?

Mark Carson: Since I haven’t played the game I can’t talk to the writing but that is a common issue. In the larger context, story-writing in videogames is a very new genre, so we haven’t really developed a complete understanding of the process and tools. Writing and storytelling is still very experimental but visually I think the path is clear. Accurate modeling, more specifically high resolution scanning and mo-cap is critical. The reason is that an artist always views the world through his own experience and can only try to replicate human actions. I think the most evidence of this is found in the eyes, because when you really look at the detailed movement of eyes it seems rather unnatural. The reality is that we actually do crazy things with eye movement and this is counter-intuitive to us as people. To really introduce natural motions into digital media requires good scanning and motion capture.

As a side note, this is one of the things that always jumps out at me, using an artist to replicate human movement. Artists are great creative minds and amaze me all the time. I think it’s a waste of their talents to try and replicate something that already exists and is easily created with sampling the real world. We have actors and they are professionals at human movement and conveying human emotion: use them and let the artists focus on what they do best.

How long did the motion capture work for Alan Wake take? Weeks? Months? How many people worked on that portion of the game? How does this compare to “normal” motion capture methods?

Mark Carson: Shooting took about a week. Cleaning and processing the data took about seven weeks, but a lot of that is set-up and iteration on the final product. For example, the first three weeks we did about three minutes of animation as we set up the pipeline. The final week we were processing almost five minutes a day. So, like any project, getting off the ground is half the battle. Those metrics are with our internal team: about seven people were directly involved with production.

Normal full body motion capture and our facial capture are comparable in time-frames, but when it comes to traditional facial capture versus CaptiveMotion we are really much, much faster. The fact that we capture so much more information than traditional facial motion capture is rather significant. We capture ten times the detail and capture eye movement and color differentiation. As a result we have a lot more data when we do our predictive modeling, so per marker error rate is significantly smaller than traditional methods and a lot faster as a result.

Obviously, new videogame consoles allow for higher-fidelity graphics. Unlike previous five-year console cycles, however, this time around Sony and Microsoft have elected to stick with the same hardware for an extended period of time. Do you think this lack of forward motion could hold back motion capture in videogames?

Mark Carson: From my perspective, developers are masters at efficiency, moving around the blocks to pack more material into a smaller area. To some degree the hardware cycle will definitely hold back high-end motion capture but I think there is still plenty of room from creative expansion. From CaptiveMotion’ s perspective its really two questions. Can the developers currently use our full capabilities? No, we can capture an insane level of detail. Can they significantly improve their existing content? Absolutely.

What are your thoughts on games like Lionhead’s Project Natal “virtual pet” Milo? It’s a game all about interacting with a character who at least seems to be living, breathing, and thinking. Do you think games like that have a future in the medium outside of being glorified tech demos?

Mark Carson: If you read science fiction you may be familiar with Jack McDevitt or Vernor Vinge. Both see the natural evolution of entertainment being interactive movies, the ‘HoloDeck’ if you will. Project Natal and Milo are how we get there, and each is a part of that puzzle. I don’t see them as standalone projects, but more as fuel for the creative fires. The fact that people like Molyneux continue to put these ideas out to the community and say "look at what you can do with this technology" is critical for innovation.

Finally, do you have any other videogame mo-cap projects that you’re working on right now that you can talk about? Or, if not, can you give us any hints?

Mark Carson: Well, we are currently getting ready for a project that hasn’t been announced, and in fact has been denied several times. Then again, that’s really not much of a hint, is it? The funny thing about this industry is that generally speaking you can’t talk about anything until it is ancient history. Other than the project that ‘doesn’t exist’, we also have a pretty deep group of close partners and we are working with them on some new tech demos, a couple of proof of concept projects and a several game and movie trailers.

Alan Wake's due out on May 21 in the UK, and May 18 in the US release for Xbox 360.