The evolution of parallax

Once upon a time, videogames were bidimensional, and it was fine.
One day, in order to give some depthness to the scene, programmers introduced a technique called parallax scrolling: by moving the elements of the background at different speed, a sense of spatiality and immersivity emerged.

The sensation was outstanding for the age, but the drawback was a very high computational cost, that kept this strategy confined on 16bit machines. With the arrival of 3D graphics, parallax scrolling was abandoned.

In 2001, a Japanese researcher named Tomomichi Kaneko introduced a brilliant modification of texture mapping able to provide "the capability to represent the motion parallax effect to the textures placed on 3D objects". His algorythm was basically a normal mapping, whose strategy is to use a texture's RGB channels to map a 3D normal, overriding the vectors on surfaces; Tomomichi basically suggested to enhance the feeling of movement by displacing textures by a function of the view angle in tangent space.

The tangent space itself is the trick behind all, and let us compute stuff in a comfortable space: instead of doing tons of computations in world space, we "look at things" from the top of the texture.

The generation of the special coordinates (normal, tangent and binormal) is done "client side" during the geometries loading, and sent to the server as vertex attributes.

The normal mapping itself requires at least a couple of textures to be sent and, guess what, I faced again the limits of DirectX9 compliant boards. I wanted to keep at least 4 lights on the scene, and use three textures (diffuse, normal map, and a gloss map). I quickly overflowed the capabilities of my VGA card and learnt an interesting thing.

GLSL is paranoid

I'm not sure it's a GLSL thing, but that's it: it's paranoid. It expects the worst scenario to happen. And therefore does not optimize a thing. I explain.

Suppose you have a code similar to this:

void DoSomething(int number_of_light) {
  ...
  Use( gl_LightSource[number_of_light] );
  ...
}

void main() {
  ...
  for (int i=0; i<TOTAL; i++) dosomething(i); 
  ...
}

What do you expect? You may think that your GLSL compiler will smartly recognize the boundaries of your  request and will fetch from memory just the informations about the TOTAL lights you're going to use. Well, wrong: it will allocate all MAX_GL_LIGHTS informations as uniforms. A mess. By keeping your code so general you instantly reach the limits of the card and the program does not link anymore. I didn't know about this (never read about that) and there's no way to optimize it. You must do several fetches to each specific light source you want to use on the scene, and that way you keep the uniforms count down. Weird.

Does your shader seamlessly compile, but inexplicably doesn't link?

Maybe it's due to the GL_MAX_VARYING_FLOATS limit.

If you go crazy obtaining the error "Fragment shader(s) failed to link", try reducing your explicit -and implicit- varyings variables.

It turns out that, for instance, you cannot combine 8 textures and 8 "blinn-phong light sources" inconsiderately, because if you would like to interpolate textures and light vectors, you end up using (8+8)*4 = 64 varyings floating point variables!

Keeping out useless textures coordinates (z,w) and the alpha channel, by using custom varyings instead of the built-in ones, you can go down to 8*2+8*3 = 40, which is lower than 44, limit for most common DX9 compliant boards.

Fixed functionality beats programmable pipeline 10:0?

Something's terribly wrong somewhere... Blinn-phong shader is wasting all GPU's power!

I've done some optimization, but I'm afraid that writing general code, good for all seasons is not the best way to achieve performance.

I can't imagine how many compromises serious 3D engines hide under the hood!

We meet again, dear wheel

Soon after my experience at CINECA, I took the decision to implement my own 3D engine.

I'm not trying to develop the ultimate graphic experience, of course. I just wanna own a set of libraries useful for easily handling and displaying scientific information, without too much effort.

Actually, I tried to avoid the huge effort of programming a whole 3D framework from scratch by using OpenSceneGraph; well, I tried, but it turned out that making it work was going to take more time than coding my own! Furthermore, I probably won't make always use of a scenegraph for my representations, so here we go.

The first thing I focused on are shaders: PLT is going to be a GLSL-centric engine. The fixed pipeline is still supported, more or less, but there's no future for it, and it's time to leave it alone.

Besides the qualitative leap that shaders allow (the screenshot shows a comparison between fixed functionality and a phong-blinn lighting), there's another huge benefit: the engine can easily be ported onto embedded systems. That's the second aspect I'm focusing on: portability. I want PLT to run on every system I could code on: Windows, Linux, and Windows Mobile.

What misses at the moment? A true geometry loader, for instance. There's already full support for vertex buffer objects, and I can manually load vertexes, normals and so on, but I must code at least an handy .OBJ loader.

As I told, there's no a scene graph, not even a rudimentary one. I'll surely need a bunch of matrix functions for manipulating my objects loaded server-side.

And naturally, I'm going to extend the PLTlights classes for shadow maps support as soon as possible! :)

Unnatural convolution

By using a simple convolution filter, with a small 5x5 kernel running on a fragment shader, one can obtain a nicer and softer shadows.

This trick was invented by Anirudh Shastry; it's not physically correct, it's prone to artifacts (and that's why I'm gonna use stencils) but it's good to see and pretty fast. Moreover, I didn't have to move the whole rendering pipeline in one shader, which is good.

Don't you know, talking 'bout the convolution it sounds like a whisper

Hard shadows destroy realism. It's a matter of fact, they look too unnatural, expecially if the rest of the scene appear softly shaded and we overlap a simulated cone of light.

Plenty of solutions have been proposed. Furthermore, Nvidia cards are so smart that they automatically apply the percentage closer filtering on shadow maps. ATI cards don't, but proposed a pretty way to implement it (by using their proprietary multiple fetching primitives).

I don't want to use proprietary features, which could lead to unexpected situations, so I took the decision to try something new: by using stencil buffers, I'm gonna separate "potentially shadowed" texels and blur them, with some kind of convolution filter:


Pros: it's REALLY fast (dirt work is done via fragment shader), it's lightweight (just an additional RGB texture is required), and it's image-space based, so I don't need to program lighting equations for each texture unit in the shader!

Cons: additional rendering steps are needed, one for each light; self-shadowing artifacts are supposed to emerge and there's no difference between shadows near and far away from light (which is best aspect of smarter algorythms like PCSS or VSM).

We'll see. In the meanwhile, I express my disappointment for ATI's engineers who didn't really implement glConvolution*D() in their drivers :| .