Raytracing vs Rasterisation
It's interesting to note the ongoing debate about Raytracing vs Rasterisation in the tech media, it seems to be as old as the hills, but the face off has become increasingly frequent with recent technology and technique advancements which seem to hold promise of enough processing power to make quality real time Raytracing possible for the masses.
I got thinking that in some ways it's similar to the Electric Car vs Fossil Fuel Car. Both go back a long way, but recent advances in electric drive systems, battery technology, ultra capacitors and hydrogen fuel cells, are making them inch ever closer to a practical solutions for all.
The analogy turns up some similarities. Fuel driven engines are a real hodgepodge of techniques trying to make the best of something that is hard to make the best of.
Conceptually you are vaporising fuel, and mixing it with air, drawing it into a confined space and compressing it, setting light to it, letting it explode, then using the force of the explosion to push another cylinder of exhaust out of the engine into the atmosphere. Noisey too! The engine requires an energy injection to start, and will stall if not kept running, even when the car is not moving. A poor implementation is very wasteful of both energy & fuel, produces lots of pollution and requires a great deal of maintenance to work reliably. Plus to get any real speed, you need complex gearing. Add to that you need fluid filled radiators to keep it from melting!
Over the years manufacturers have come up with a whole bag of tricks to make them work better. Complex engine management systems which monitor air pressure, temperature, load, engine speed, oxygen level in the exhaust, variable valve timing, complex oil formulations, ignition timing maps, the list goes on!
This has resulted in the fairly powerful cars we have today. They require regular maintenance or they break down, they still produce pollution (although much less than they used to) and are expensive to own and maintain.
Contrast that with electric vehicles. They produce no direct pollution at all, the actual engine is pretty much just an electromagnet suspended between two solid state magnets (oversimplification), and can even be incorporated into the wheels! They do not require much engine management, are relatively low maintenance and fairly quiet.
So why isn't the world filled with Electric Cars?
a) Performance - Up until recently electric cars were slow
b) Performance - Up until recently electric cars had very low range
c) Performance - They take time to charge and cannot be refuelled instantly
d) Infrastructure - The world isn't set up to use them yet, so no charging facilities etc.
As pressure with the existing fossil fuel system mounts though, a change is coming.
Ok, back to Raytracing vs Rasterisation..
Rasterisation is like the fossil fuel driven engine. It developed long ago because it was possible to make it work! It involves taking a large set of 3D vertices and transforming them into 2D points (remembering the Z as a function of distance from the viewpoint). Take three at a time, then you make a triangle. Then a set of 2D Rasterisation routines draws and fills the triangle interpolating between the 2D points and marking the interpolated Z into the Z-Buffer. As you go through all the triangles, one by one, you check each point you're drawing to make sure the Z is closer than the one in the Z Buffer, so that nearer triangles obscure the ones further away.
In its raw form, image quality is a bit lacking and is quite wasteful in writing all triangles, regardless of what can be seen, etc. You get the point..
Of course, like the car, it has seen some *serious* work over the years. Especially since the hardware people started developing it, a whole technological hodgepodge of techniques have been used to make it work ever better.
On the hardware side, Zbuffers have been joined by, stencil buffers, texture memory, mip mapping, filtering, antialiasing, shaders, backface culling, normal mapping, Z-Sorted alpha transparency <insert really long list>, etc.
On the software side LOD, visibility portals, occlusion testing, <insert realllly long list>, etc.
The result is that we have huge GPUs with 10x - 20x the processing power of the average CPU, being driven by hugely complicated multi-pass software engines utilising large & complicated APIs (DirectX for instance).
It costs an obscene amount of money to make a high end game. Much of this is due to work required by programmers, artists, level designers, who need to apply new tricks to align the needs of the game to the limitations of the technique.
Where is Raytracing then?
The technique is conceptually fairly simple, you have a list of triangles. You fire a ray for each pixel on the screen, and trace which triangle it hits. From there you trace additional rays to light sources. For transparency, you continue the trace through the object, for refraction, you adjust the direction of the ray as it passes through, for reflection, you continue the trace at an angle. Repeat for each pixel.
You don't need a Z-Buffer, since each pixel is traced indiviually.
You don't need to Z-Sort triangles since transparency is traced in order.
You don't need a stencil buffer for shadows (I know there a lots of other techniques), they're raytraced per pixel.
You don't need a LOD system, since visibility is defined per pixel rather than per triangle.
You don't need other visibility culling techniques, since triangle count is not so important as RT scenes are spatially subdivided.
So why isn't the 3D computing world filled with raytraced games and applications?
a) Performance - Up until recently, raytracing was strictly an 'Offline' thing. Too slow!
b) Performance - Up until (relatively) recently, raytracing had little in the way of acceleration techniques available to them.
c) Performance - Up until recently, processors (and memory speeds) have not been powerful enough to handle the load of tracing millions of rays per frame!
d) Performance - With many core processors on the GPU and now CPU, things are finally falling into place.
The most encouraging thing about Raytracing though, is that it employs one technique, and uses it in increasing quantities to do more and more.
You want polygons with LOD = 1 Ray
You want Per Pixel Lighting + 1 Ray per light source
You want Hard Shadows + 1 Ray per light source
You want Soft Shadows + n Rays for sampling
You want Antialiasing + n Rays for adaptive supersampling
You want Radiosity + n Rays fired at random for environment sampling
You want per pixel collision detection + n Rays each object.
You want Subsuface Scattering + n Rays per hit.
Couple that with the fact that raytracing can handle large loads of Triangles with relative ease, and you can see why it is a desirable upcoming technology.
Now there's no doubt that all those rays add up! But the potential is there from the start.
It is quite possible that a Raytracing API could be created that gives the option to use enable of these image quality settings (even if it would be a 10 second per frame slideshow to use them all). Then as new hardware to accelerate it becomes available, even older games would begin to take on a new life. You see, all they needed was more power, not new techniques!
Imagine playing something like Oblivion with no LOD popping in your face, a significantly higher polycount, perfect shadows and reflections, proper refraction one year. Then upgrading your GFX card later in the year and when you load up the same game, being able to enable cool Global Illumination effects, then upgrading perhaps the next year, and being able to enable subsurface scattering to improve the skin and other textures.
Rather than making games to target a specific performance group, you could make one that covers from mainstream to extreme simply by increasing or decreasing the number of rays it can use.
Then the engine you use to program the game will not be so significant, since the software engine will not be the all singing, all dancing bag of magic tricks it now is, but all the magic can be in the API, but essentially latent, until the advancement of processing power unlocks their use.
What else could be done to make raytracing faster? How about upscaling, like video? Rendering at a little lower resolution, then interpolating this to a larger one with a powerful upscaling filter.
Rasterisation has wads of long term investment on its' side, but I believe it may not hold the crown forever.
Let the Flaming commence ;)
It's interesting to note the ongoing debate about Raytracing vs Rasterisation in the tech media, it seems to be as old as the hills, but the face off has become increasingly frequent with recent technology and technique advancements which seem to hold promise of enough processing power to make quality real time Raytracing possible for the masses.
I got thinking that in some ways it's similar to the Electric Car vs Fossil Fuel Car. Both go back a long way, but recent advances in electric drive systems, battery technology, ultra capacitors and hydrogen fuel cells, are making them inch ever closer to a practical solutions for all.
The analogy turns up some similarities. Fuel driven engines are a real hodgepodge of techniques trying to make the best of something that is hard to make the best of.
Conceptually you are vaporising fuel, and mixing it with air, drawing it into a confined space and compressing it, setting light to it, letting it explode, then using the force of the explosion to push another cylinder of exhaust out of the engine into the atmosphere. Noisey too! The engine requires an energy injection to start, and will stall if not kept running, even when the car is not moving. A poor implementation is very wasteful of both energy & fuel, produces lots of pollution and requires a great deal of maintenance to work reliably. Plus to get any real speed, you need complex gearing. Add to that you need fluid filled radiators to keep it from melting!
Over the years manufacturers have come up with a whole bag of tricks to make them work better. Complex engine management systems which monitor air pressure, temperature, load, engine speed, oxygen level in the exhaust, variable valve timing, complex oil formulations, ignition timing maps, the list goes on!
This has resulted in the fairly powerful cars we have today. They require regular maintenance or they break down, they still produce pollution (although much less than they used to) and are expensive to own and maintain.
Contrast that with electric vehicles. They produce no direct pollution at all, the actual engine is pretty much just an electromagnet suspended between two solid state magnets (oversimplification), and can even be incorporated into the wheels! They do not require much engine management, are relatively low maintenance and fairly quiet.
So why isn't the world filled with Electric Cars?
a) Performance - Up until recently electric cars were slow
b) Performance - Up until recently electric cars had very low range
c) Performance - They take time to charge and cannot be refuelled instantly
d) Infrastructure - The world isn't set up to use them yet, so no charging facilities etc.
As pressure with the existing fossil fuel system mounts though, a change is coming.
Ok, back to Raytracing vs Rasterisation..
Rasterisation is like the fossil fuel driven engine. It developed long ago because it was possible to make it work! It involves taking a large set of 3D vertices and transforming them into 2D points (remembering the Z as a function of distance from the viewpoint). Take three at a time, then you make a triangle. Then a set of 2D Rasterisation routines draws and fills the triangle interpolating between the 2D points and marking the interpolated Z into the Z-Buffer. As you go through all the triangles, one by one, you check each point you're drawing to make sure the Z is closer than the one in the Z Buffer, so that nearer triangles obscure the ones further away.
In its raw form, image quality is a bit lacking and is quite wasteful in writing all triangles, regardless of what can be seen, etc. You get the point..
Of course, like the car, it has seen some *serious* work over the years. Especially since the hardware people started developing it, a whole technological hodgepodge of techniques have been used to make it work ever better.
On the hardware side, Zbuffers have been joined by, stencil buffers, texture memory, mip mapping, filtering, antialiasing, shaders, backface culling, normal mapping, Z-Sorted alpha transparency <insert really long list>, etc.
On the software side LOD, visibility portals, occlusion testing, <insert realllly long list>, etc.
The result is that we have huge GPUs with 10x - 20x the processing power of the average CPU, being driven by hugely complicated multi-pass software engines utilising large & complicated APIs (DirectX for instance).
It costs an obscene amount of money to make a high end game. Much of this is due to work required by programmers, artists, level designers, who need to apply new tricks to align the needs of the game to the limitations of the technique.
Where is Raytracing then?
The technique is conceptually fairly simple, you have a list of triangles. You fire a ray for each pixel on the screen, and trace which triangle it hits. From there you trace additional rays to light sources. For transparency, you continue the trace through the object, for refraction, you adjust the direction of the ray as it passes through, for reflection, you continue the trace at an angle. Repeat for each pixel.
You don't need a Z-Buffer, since each pixel is traced indiviually.
You don't need to Z-Sort triangles since transparency is traced in order.
You don't need a stencil buffer for shadows (I know there a lots of other techniques), they're raytraced per pixel.
You don't need a LOD system, since visibility is defined per pixel rather than per triangle.
You don't need other visibility culling techniques, since triangle count is not so important as RT scenes are spatially subdivided.
So why isn't the 3D computing world filled with raytraced games and applications?
a) Performance - Up until recently, raytracing was strictly an 'Offline' thing. Too slow!
b) Performance - Up until (relatively) recently, raytracing had little in the way of acceleration techniques available to them.
c) Performance - Up until recently, processors (and memory speeds) have not been powerful enough to handle the load of tracing millions of rays per frame!
d) Performance - With many core processors on the GPU and now CPU, things are finally falling into place.
The most encouraging thing about Raytracing though, is that it employs one technique, and uses it in increasing quantities to do more and more.
You want polygons with LOD = 1 Ray
You want Per Pixel Lighting + 1 Ray per light source
You want Hard Shadows + 1 Ray per light source
You want Soft Shadows + n Rays for sampling
You want Antialiasing + n Rays for adaptive supersampling
You want Radiosity + n Rays fired at random for environment sampling
You want per pixel collision detection + n Rays each object.
You want Subsuface Scattering + n Rays per hit.
Couple that with the fact that raytracing can handle large loads of Triangles with relative ease, and you can see why it is a desirable upcoming technology.
Now there's no doubt that all those rays add up! But the potential is there from the start.
It is quite possible that a Raytracing API could be created that gives the option to use enable of these image quality settings (even if it would be a 10 second per frame slideshow to use them all). Then as new hardware to accelerate it becomes available, even older games would begin to take on a new life. You see, all they needed was more power, not new techniques!
Imagine playing something like Oblivion with no LOD popping in your face, a significantly higher polycount, perfect shadows and reflections, proper refraction one year. Then upgrading your GFX card later in the year and when you load up the same game, being able to enable cool Global Illumination effects, then upgrading perhaps the next year, and being able to enable subsurface scattering to improve the skin and other textures.
Rather than making games to target a specific performance group, you could make one that covers from mainstream to extreme simply by increasing or decreasing the number of rays it can use.
Then the engine you use to program the game will not be so significant, since the software engine will not be the all singing, all dancing bag of magic tricks it now is, but all the magic can be in the API, but essentially latent, until the advancement of processing power unlocks their use.
What else could be done to make raytracing faster? How about upscaling, like video? Rendering at a little lower resolution, then interpolating this to a larger one with a powerful upscaling filter.
Rasterisation has wads of long term investment on its' side, but I believe it may not hold the crown forever.
Let the Flaming commence ;)