Ray Tracing explained: how the lighting tech coming to PS5 can change games
As we speed towards PlayStation 5 and a next-gen Xbox kicking off a new console hardware generation, you're going to hear the term ray tracing an awful lot. But what is this hot new graphics technology - and should you give a damn about it?
Every generation of gaming hardware always has its buzz words. With the PS4 Pro and Xbox One X, 4K and HDR were the ultimate features, and they'll still be a major part of the next generation. As the PS5 and next Xbox draw ever closer, another feature joins those three as a vital selling point - ray tracing. It's a sweet new lighting technique that unlike 4K and HDR can drastically improve games without the player splashing out on an expensive new display as well as a new console or graphics card.
Ray tracing will very likely form a major front in the battle between Microsoft and Sony's next generation gaming hardware. Sony has already confirmed PS5 will support ray tracing, while over on PC it's already been a major point in the GPU war between AMD and Nvidia, with Nvidia setting their products apart with a distinct focus on the feature.
It's clearly a big deal, but why is everybody crazy about it? On this page we'll try to explain why ray tracing is exciting and how exactly it works in layman's terms.
What is Ray Tracing?
As you might have gathered from the name, ray tracing is all about lighting. We're reaching a point where there's diminished returns on simply pushing more polygons and higher quality textures; for enhanced realism, you need to improve things like lighting to make a scene more natural and realistic. Right now ray tracing is the holy grail of real-time lighting in computer graphics.
Traditionally, most video games use Rasterization to handle lighting, shading and other similar effects. This technique essentially takes the 3D objects created by a computer and calculates key information like the distance between objects before squishing the scene down into what you see on your 2D display. You're basically taking a 3D shape, squashing it down to 2D, then running a variety of shaders to colour and process the image including approximated lighting and shading based on the scene conditions as determined by the previously calculated information.
This is a fast but intense process, and has allowed for a solid balance of visuals and performance. Over the years developers have employed additional tricks here and there to give a better impression of more realistic real-time lighting - but in the end it is all still fakery.
You might notice that description of Rasterization has no mention of how light travels in the real world. That's because in the grand scheme of things it doesn't really matter - rasterization is all about an estimation of what the light might do in reality. This is where ray tracing is different - it's a more realistic representation of how light works in reality, and therefore can result in much more realistic-looking scenes such as the screenshot above - an incredible real-time tech demo that movie CG powerhouse ILM produced in partnership with Nvidia to showcase their live, PC-based ray tracing technology.
There's lots of other impressive tech demos of ray tracing in action, but perhaps the most mindblowing is the simplest - take a look at Digital Foundry playing Minecraft with ray tracing cranked all the way up. It looks insane.
How does Ray Tracing work?
When you focus your gaze on an object, you're seeing it because light is bouncing off it and into your eyes. If you imagine an individual ray of light as visible, look at an object near you now and then try tracing back the route that beam of light took - out of your eye, back to the object, bouncing off the object, this way and that, until it reaches the light source. You've just done some ray tracing, albeit in reverse.
Put simply, ray tracing is the act of simulating beams of light and the way they move in order to create a more realistic representation of light and other effects impacted by light such as shadows. It alone can truly transform a game's visuals.
You're probably more familiar with ray tracing than you know, as it's been used in Hollywood movie CG work for a very long time now. That ultra-realistic, barely visible reflection in Iron Man's armor? That's thanks to ray tracing. Pixar has been using it for an age, and there's deep dives into how ray tracing worked on movies like 2006's Cars and how it was more recently refined for 2013's Monsters University available online.
There's a reason Hollywood has been able to use this technique while games haven't, of course, and it's all about power. Hollywood movie studios use huge server farms thousands of times more powerful and expensive than our game consoles, and they also have time on their side. A single frame of some Pixar films can take a day or more to render, and that's in part thanks to excruciatingly realistic ray traced lighting. For a game, you need to render thirty to sixty such frames a second. Thus, it has so far been impossible.
The next generation of hardware obviously won't be hundreds of times more powerful than the PS4 and Xbox One, so what's changed to make it possible? Mostly it's a matter of scale. Just like how games 'cull' out geography and detail that's out of sight in order to maintain solid performance, the games-friendly real-time ray tracing pulls back on how many individual rays are used. This allows for a good boost in visual fidelity without making real-time gaming performance impossible.
In real terms, what this means is that this new version of ray tracing only traces the light beams that directly hit a virtual camera in the scene, which itself represents what the player can see. Rays aren't bouncing around off-screen in ways that are entirely realistic as with the heavy-duty movie representations. In reality, there are billions of rays - more - at any given time. In this version, the GPU might be tracking a couple of hundred. There's a lot else going on around this - AI deep learning is used to clean up the scene and fill in the gaps, for instance - but it's all centered around those traced rays heading fom a light source, bouncing their way to that virtual camera as they do to your eyeballs in real life.
Despite being a significantly pared-back version of the technology, the ray tracing that Nvidia has been able to showcase on its RTX series graphics cards is seriously impressive and leaves a strong impression on the visuals of games updated to leverage it. Better still is that in some instances this sort of technology can even have an effect on in-game systems and tactics. There's already some incredible examples of this in action on PC, and if both Sony and Microsoft can offer ray tracing in the PS5 and next Xbox, the pros and cons of each implementation is sure to become a next-gen head-to-head graphics comparison sticking point.
How ray tracing impacts PC gaming performance now - and how it might on next-gen consoles
Let's get the main thing you need to know out of the way up top: ray tracing looks absolutely amazing. It doesn't come without a cost, as PC players have learned. First is the real cash-money cost of the hardware, as those with dedicated ray tracing cores are expensive. Second is in performance, as even on the specialist cards such as Nvidia's RTX series GPUs you can expect to see lower frame rates with ray tracing enabled, with the simplest solution to that problem being to lower the in-game resolution.
On console at least one of these costs will be less visible - the price of the GPU will be amortized by the overall cost of the console and the other money-making aspects of that ecosystem such as licensing fees. The latter cost has also already been significantly reduced over on PC, with tweaks and optimization boosting Battlefield 5 ray tracing performance by 50% just a few weeks after its original launch. Both points are still factors in this young technology's status, however, and you can still expect to suffer a lower frame rate or resolution with full ray tracing features enabled.
It feels likely that, as on PC, next-gen players will be left to choose between 4K presentation and full ray tracing with in-game performance options. It speaks to ray tracing's qualities that in many cases I would recommend taking the hit and playing at a lower resolution.
As some of the demos littered on this page demonstrate, ray tracing can have a real impact on how games actually look. The Metro Exodus footage embedded above this section really drives home how good this technology has the potential to be - it completely transforms the game both out in the open world with sunlit areas as well as in the dark, claustrophobic underground zones. Ray tracing actually works to enhance elements of the game, too - more realistic shadows mean enemies can be lurking in spots completely hidden from sight where in a rasterized version they might be visible. There's more nuance to the presentation in general, and it's all thanks to the shift in lighting. It's fair to say that with ray tracing on Metro Exodus looks like a generational leap - despite in all other senses being a game that runs just fine on current hardware.
For an example of ray tracing actually potentially changing moment-to-moment play, look no further than Battlefield 5, one of the first games to support the technology. As well as using ray tracing to make a huge difference to the overall look of the game, ray tracing also has a role to play in determining in-game reflections. Typically, when you see a mirror in a game it's either crude or cleverly faked with a 'mirrored' version of the scene - a second room, with a second you moving around in it, copying your movements. With ray tracing high quality, accurate real time reflections are possible - and that can actually have major implications on the certain types of game.
A stealth game would be able to use ray tracing to enable the player to see what's coming from around a corner, or keep an eye on a situation from an out-of-sight vantage point. In the earliest ray tracing demos, Battlefield developer DICE showcased how in a ray tracing enabled version of Battlefield 5 a player might actually see an online rival approaching from around the corner thanks to a reflective surface like a car door or shop window. That's a potentially huge change - and if every PS5 or Xbox comes with this technology enabled, it could be leveraged in games in a much more broad manner than it currently is on PC.