Stencil shadow drawing?

Does anyone have any example code to setup the stencil buffer, and draw stencil shadows correctly? I have got the stencil volumes created, and just need to render them now to look like shadows, instead of big solid volumes.

Is your shadow color pure black?

If I recall correctly, in my old B3D stencil demo, the final step was to black quad over the screen and render with alpha blending on, the amount of alpha controlled how dark the stencil shadow was.

You're using GL b.t.w, right?

Trying to render them now, but nothing appears. This is the code from the NeHe tutorial. If I just render them without stencils, it appears as it does in the image at the top of this page.

I also set the stencil bits to 8 in the setpixelformat code. I made no other changes:

gldisable GL_LIGHTING
										glcolor3f 0,0,0

										glPushAttrib(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT|GL_ENABLE_BIT|GL_POLYGON_BIT|GL_STENCIL_BUFFER_BIT)
										glDisable( GL_LIGHTING )'									Turn Off Lighting
										glDepthMask( GL_FALSE )'									Turn Off Writing To The Depth-Buffer
										glDepthFunc( GL_LEQUAL )'
										glEnable( GL_STENCIL_TEST )'								Turn On Stencil Buffer Testing
										glColorMask( GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE )'		Don't Draw Into The Colour Buffer
										glStencilFunc( GL_ALWAYS, 1, HexToInt("0xFFFFFFFFL"))

										'First Pass. Increase Stencil Value In The Shadow
										glFrontFace(GL_CCW)
										glStencilOp(GL_KEEP,GL_KEEP,GL_INCR)
										lightlink.shadowvolume.draw()
										
										'Second Pass. Decrease Stencil Value In The Shadow
										glFrontFace(GL_CW)
										glStencilOp(GL_KEEP,GL_KEEP,GL_DECR)
										lightlink.shadowvolume.draw()

										glFrontFace( GL_CCW );
										glColorMask( GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE )' 			Enable Rendering To Colour Buffer For All Components
									
										'Draw A Shadowing Rectangle Covering The Entire Screen
										glColor4f(0.0,0.0,0.0,0.4 )
										glEnable(GL_BLEND)
										glBlendFunc(GL_SRC_ALPHA,GL_ONE_MINUS_SRC_ALPHA)
										glStencilFunc(GL_NOTEQUAL,0,HexToInt("0xFFFFFFFFL"))
										glStencilOp(GL_KEEP,GL_KEEP,GL_KEEP)
										glPushMatrix()
										glLoadIdentity()
										gldisable GL_CULL_FACE
										glBegin(GL_QUADS);
											tformpoint 10,10,10,0,camera
											glvertex3f tformedx(),tformedy(),tformedz()
											tformpoint -10,10,10,0,camera
											glvertex3f tformedx(),tformedy(),tformedz()
											tformpoint -10,-10,10,0,camera
											glvertex3f tformedx(),tformedy(),tformedz()
											tformpoint 10,-10,10,0,camera
											glvertex3f tformedx(),tformedy(),tformedz()																														
										glEnd()
										glPopMatrix()
										glPopAttrib()

										glcolor3f 1,1,1
										glenable GL_LIGHTING
										glenable GL_CULL_FACE
										gldisable GL_BLEND

I'm a little closer now.

Should I draw one textured quad across the screen at the end of the rendering routine, or should I draw one for each shadow?

Would go for "per entity" for 2 reasons:

1. Normally not the whole screen is filled with shadows (at least outside Doom3), so this will save some bandwidth

2. It allows per entity shadow rendering passes. Usefull if there are situations where some shadows change but others don't and the like ...

I found that by doing a per-entity shadow pass, I could fade the shadow with distance...as the light gets further away, and the area darkens, the shadow lightens...it's a pretty good way to blend shadows in and out.

Here is the current state. The boxes appear correctly, but the oildrums clearly have some bad mistakes. Can anyone tell what is going on here?

Would say that the function that calculates the outline breaks on the concave top / bottom.
The reason I think so is the shadow part in the middle, right, where you see the bottom shadow stretch out. But you see as well, that a part is missing of it although the outline of the barrel is not "inexistant" there. It looks like the concave part was just cut out on shadow calculation.

Edges appear to be calculating correctly:

It looks like parts that are self shadowing only are projecting out of the object onto the wall behind as well.

And that they even substract from the original shadow (see the bright spot within the upper part of the barrel shadow or the backside of the barel, where the selfshadowing is broken)

Hmm, or is it the lit part of the concave lid thats projecting a shadow through and causing the problem, yet i cant see an outline displayed for that volume.

Do you have to eliminate edges that are within the bounds of other edges? I found the edges between triangle that face towards and away from the light, but I did not do any elimination of edges that fall within the outline of the object silhouette.

When creating the closed volume to project the shadow it looks like the common edge of top of the can and of the back of the can have made 1 linked volume? and to make this a closed loop to project the edge of the volume has had to go into itself.

Lets see if i can do some ascii art to represent it. :)

xxxxxxxxxxx
x00000xx00x
x00000xux0x
x0000xuux0x
x000xuuux0x
x000xxxxx0x
x000000000x
xxxxxxxxxxx

the section where the '0' are would be the volume projected by the back of the can,

the section where the 'u' are would be the volume projected but the lit section on top of the can.

the 'x' are the outline.

because these have a common edge have these volumes linked to become one volume, thus when the outline is generated it has to follow the 'x' round. The 'u' area then is outside the shadow volume.

I havent tried shadows myself yet and what i've just typed is general stuff i've just picked up from articles on gamedev.net and nvidia developer section so please correct me if im talking nonsense:)

So what do I do about it?

To be honest i havent really looked into this much before and im just googling for answers for you.

there is a fair bit of information out there on self intersection of silhouette loops and how this can cause the problem you are seeing.

http://graphics.cs.lth.se/theses/projects/shadows/so_thesis_lq.pdf

is someones thesis i found detailing some info on shadow volumes that may be of some use.

There are some idea's i've seen while googling around about projecting your volume onto a 2d plane that faces the light sorce, then using faster 2d techniques to check for self intersections of the loop. But alot of the maths involed on those pages were beyond me and i got bored fast and moved onto other pages.

Sorry I couldnt be more help but im just googling around and understanding little(im more into creating 2d platformers etc, i havent reached the 3d age yet)

Aw crap.

To decide which way the quads projected from the edges face, I calculated the normal of the quad (based on the first three vertex positions) and compared that to an average of the normal of the two triangles the quad's edge lies between. If the dotproduct was more than zero, I flipped the quad face. So all my volume edges are pointing on the same direction that the edge's triangles are pointing, and not in the opposite direction.

This works great for a convex outline. The outside of my shadow volume is perfect. However, it doesn't work with the smaller volume made out of the inside of the lip at the top. Although the edges form a complete loop, the sides of the volume point inward, and the bottom points up.

Is this indeed the problem?

How do I possibly figure out which direction to make the quads point? The only way I know how to is to make a list of all vertices for the loop, order them, calculate the angle change between each vertex, and add the changes up...or something like that. It sounds horribly complicated.

HA!

The problem was which way the quads were facing.

To determine this, you take another vertex from one of the triangles, a vertex which does not form part of the edge. That point needs to lie behind the quad. Do a plane distance test with the plane equation you get from the first three vertices of the projected quad. If the distance is less than zero, reverse the order of the quad vertices.

Thanks to all that contributed.

Looks pretty nice. What about antialiasing and soft shadows?

AA is already in, I just haven't been running it. The command accepts a parameter for the aa level, like 2, 4, 8 etc., and returns a value to tell if the mode was set successfully.

hey, great wok, i must say :) ... could you give me the code for the shadow volumes, please?

edit: oh shit this is bmx, not bb, sorry. forget this post

what type of FPS are you gettin on what computer specs?

what have resolved, position camera in to cone shadow?