How Hard is to Develop for the 4K-Ready Xbox One X and PS4 Pro? We Asked the Experts

This article first appeared on USgamer, a partner publication of VG247. Some content, such as this article, has been migrated to VG247 for posterity after USgamer's closure - but it has not been edited or further vetted by the VG247 team.

The industry is going through a shift in terms of game development. The Xbox One X and PlayStation 4 Pro are both pushing Ultra HD 4K (3840 × 2160) resolution support, high dynamic range (HDR) color, and enhanced graphical effects as their core offering. These additions come alongside lower prices for 4K televisions; whereas these TVs were previously for high-end buyers only, you can now buy smaller models for reasonable prices.

The industry has jumped from 8-bit to the high color era of 16-bit, from 16-bit to early polygonal efforts of the 32-bit efforts, and from that to the HD era of the last console generation. It's even dived into the speedbumps and tangents of virtual reality and stereoscopic 3D.

But one growing issue has been cost. As graphics technology and game presentation have improved, so too have costs risen. Now, with new high standards, developers are creating fully-voiced characters with photo-realistic clothes and hair. Their worlds have a fidelity that borders on film CGI at times. And all of this requires money and time.

Back in 2013, Nintendo's Shigeru Miyamoto admitted that HD game development on the Wii U took the company by surprise and required "twice the human resources" than past efforts. In an answer to a question about the rising costs of HD game development, Undead Labs technical art director Steve Theodore, who was previously a technical director on Halo 3 and an artist on Half-Life, noted that similar enemy creatures required far more time. A Half-Life Houndeye took two weeks to finish in 1997, while a similar Hivehound enemy made in 2007 took seven weeks. (And the design almost didn't make it into the game at all.)

That was all for the jump to HD. What about the move to Ultra HD?

Not every developer is using the Xbox One X and PlayStation 4 Pro in the same way, but we reached out to a number of developers who have games that aim for 4K resolution. The folks who were kind enough to answer our questions include Ubisoft Technology Director Sylvain Trottier (Assassin's Creed Origins), Monolith Productions Director of Technical Art Matt Allen (Middle-earth: Shadow of War), and The Coalition Technical Director Mike Rayner and CG Supervisor Colin Penty (Gears of War 4).

Creating The Characters And Worlds of Tomorrow, Today

Right from the beginning, everyone who responded downplayed the issue of authoring content in 4K resolution versus making games for a lower resolution like 1080p. The reason? Studios are already developing their assets at very high resolutions. Even back in 2013, Assassin's Creed IV: Black Flag was easily able to be in 4K on PC.

"Since the 4K generation is less about a major game hardware switch, and by that I mean a brand new console from the ground up, there haven't been very many big changes in how we build content," says Monolith's Matt Allen. "For years our artists have had to build their content at a high enough fidelity to run on the most powerful machines we can find. This is because it is easier to remove details and resolution than to add it. As a side note, even now most of our models and textures are still being made at a resolution that modern consoles can't run effectively."

Development costs aren't rising due to the move to 4K, because developers have been working with much higher resolutions for some time now. The real cost in terms of visual presentation comes down to optimization: figuring out which assets can be scaled down to run on Xbox One X and PlayStation 4 Pro, or their weaker counterparts.

"Even now most of our models and textures are still being made at a resolution that modern consoles can't run effectively."

- Monolith Productions Director of Technical Art Matt Allen

"The primary content creation overhead for the Xbox One X version of the game was on the Technical Art team to fully maximize visual features and up-rez our existing content that had already been created to 4K spec for the PC version of the game," says The Coalition's Colin Penty. "The Xbox One version of the game was very carefully assembled to ensure optimal performance and visual quality on that platform, so there was a re-balancing required for the Xbox One X version of the game once we began to modify the visual features list and content."

Monolith's Matt Allen adds that for a number of game artists, they've been creating game characters and worlds in a fidelity that most players would never see. Generally, unless a player moves back and forth between 1080p and 4K resolutions, they'd never notice the difference as the brain normalizes to whatever resolution you normally play at. Certain aspects of asset creation were being completely missed by players, and many of the post-processing effects used on games actually look better at higher resolutions.

"In short, having all of that post-processing happen at a higher resolution on more pixels is insanely important, but it's a super unsung hero," Allen says. "Everyone sees the 300K triangle models or the 8K textures and how great they look, but all the behind-the-scenes calculations add just as much 'wow factor.'"

Tools of the 4K Trade

Rayner says that the biggest thing players will never notice is the part they'll never see: the enhanced development tools and technology used to make all of these assets. For Assassin's Creed: Origins, the game uses Dynamic Resolution Rendering to preserve the framerate. The game isn't always at a native 4K resolution on Xbox One, but if Trottier and his team at Ubisoft did their jobs correctly, the average player never noticed. "Players might not notice our efforts on that front, but that's a good thing," he tells me.

One tool of the trade that some developers have been using recently is photogrammetry. The technique involves taking pictures of real world objects and then scanning them into the game directly as 3D models and high-resolution textures. It was used to create the fairly-realistic models in Capcom's Resident Evil 7: Biohazard.

"To create realistic terrains for Assassin's Creed Origins we also used photogrammetry," says Trottier. "We sent a team of experts to locations where the rocks, pebbles and ground resemble the ones we wanted to feature in Origins. They took dozens of super high-res pictures that we then imported in the engine to create a wide diversity of terrain textures that look so close to the real thing that it's sometimes very difficult to tell the picture from the in-game asset."

Being an internal development studio for Xbox One X platform holder Microsoft, The Coalition has access to specified tools for the console that help. One tool is the Performance Investigator for Xbox (PIX), which captured data from Gears of War 4 during development and helped The Coalition to optimize its code further. "The ease of development and powerful tools gave us the extra time and support to deliver the new 'Performance' mode [for Gears of War 4], which ultimately gives more options to gamers on how they want to use the power of Xbox One X," says Rayner.

Part of the problem with 4K asset creation isn't just in the development, it's the delivery. Internally, developers are sharing absolutely massive art assets around the world. None of that is cheap or easy.

"We realized how much disk space and bandwidth we were taking up," says Monolith's Allen. "Even a single five minute 6K HDR cinematic is massive, let alone two hours. On top of this, not everyone needs all of those high fidelity source assets on their machines. A lot of work is done on our internal infrastructure, both the PC hardware we are using and also the servers themselves. As things get faster and teams get larger, managing all of that content correctly has become one of the new challenges, and it will only get more complicated as we move forward."

Making Space For Massive Worlds

The storage space issue extends to players too. 4K is causing game sizes to balloon, with some games reaching and surpassing 100GB with all their 4K assets included. Forza Motorsports 7 with its Xbox One X enhancements takes up 95GB of space. Gears of War 4 is 103GB. Downloading a few of these games can absolutely hurt any players that have a data cap on their internet service provider. According to Assassin's Creed Origins' technology director Trottier, this is down to everything in a game: Models, textures, cinematic scenes, and even lighting.

"Constant innovation in compression techniques is one example of how we can work around sizing challenges," says Trottier. "For example, lighting data size was an issue at the beginning of the project. We put our best engineers on the issue and they came up with innovative solutions to distribute the data in our world and compress it based on the region of the world without sacrificing quality."

Another answer is the use of modular downloads, with players only downloading what they need. If you're not using 4K resolution, then there's no need to download 4K textures and cinematics. If you don't need HDR, that can save space too. Maybe in the future, modular downloads will become more the norm, allowing players to skip downloading single-player campaigns or multiplayer modes if they're not going to use them. Gears of War 4, for example, has varying download sizes for its Xbox One and Xbox One X versions. Middle-earth: Shadow of War has an additional, optional download for 4K cinematics, letting players choose if they want to download the extra 10-to-15GB.

"This problem is just going to get worse as we push more resolution and wider dynamic ranges. There are a number of possible solutions to these issues, and certainly digital downloading of assets, à la carte, can help," says Allen. "Ultimately though, that’s just custom content management, not a new, or fundamentality different content creation strategy."

During the interviews, procedural generation was one method brought up to bring content sizes down. The upside is the games themselves end up smaller as a result of simply having to generate worlds from a set amount of game assets. The downside is procedural content doesn't have the same intent and design of created world content; sometimes those perfect game moments happen in-game because someone designed the environment to funnel you towards a specific moment or vista.

"We share a lot. We also work together between all the teams involved so that the knowledge transfer is easier and the code is mastered in multiple studios."

- Ubisoft Technology Director Sylvain Trottier

Sharing Is Caring (and Cost-Effective)

One way developers can keep costs and development time down is to share certain assets from project to project. While not everything can be brought over, a studio can sometimes use certain environment textures and assets from game to game. Being able to carry forward a character model from a previous entry with some improvements and changes is easier than crafting an entirely new model. Ubisoft is one publisher known to share art between studios and titles, as all of its studios are internal.

"We share a lot," says Trottier. "We share the same core engine from one Assassin's Creed game to the other, with some improvements and evolutions of course. We also work together between all the teams involved so that the knowledge transfer is easier and the code is mastered in multiple studios."

Not all games share assets though. For Middle-earth: Shadow of War, Monolith brought over some of the older content from Shadow of Mordor, but ultimately ended replacing those assets during development. Meanwhile The Coalition is a brand-new studio, but it's full of veteran developers who have an eye towards the future—even in original asset creation. Some content can be brought over from previous titles, but at a certain point, things have to be left behind.

"You always make the content with the intention of it being used for as long as possible."

- The Coalition CG Supervisor Colin Penty

"You always make the content with the intention of it being used for as long as possible, but inevitably some new technologies or process will be discovered that will require old content get refactored to stay relevant," says Gears of War 4 CG Supervisor Colin Penty. "If the refactor is significant enough, then it may not be worth keeping the old content and just re-building it. I think we strike a strong balance between the three R's—Refactoring, Reusing, Rebuilding—to achieve a really high-quality product in a reasonable amount of time. Additionally, each game we make has a unique visual concept and style, so the vast majority of content is built specifically for each title."

We've just begun the 4K era of home console development. Publishers and developers are improving certain past titles, but most major AAA multiplatform games moving forward will ship with enhancements catering to the Xbox One X and PlayStation 4 Pro. Both consoles represent a mid-generation leap for their respective platforms; we don't know if there will be another iterative leap in the future or entirely new platforms. Regardless, those platforms will likely start with 4K and HDR support as part of the package and move up from there.

All that is before you even move beyond 4K. The HDMI Forum, the group behind television input standards, announced the official details for the HDMI 2.1 specification. These details include support for 8K resolution (7680 × 4320), dynamic HDR, and variable refresh rate. Yeah, 8K. Back in September, Sharp announced its first 8K television set for commercial release, the 70-inch Aquos 8K TV. Sure, there's no 8K content, but the TV is coming to China and Japan later this year.

Technology is moving faster than ever. 4K is here to stay, and 8K is coming sooner rather than later. In the middle of this, developers have to be ready to make games that stand up to the technology. How far can the industry go before things hit a breaking point? It's ready for 4K, but for the next step and the one after, that question is left to be answered.