Strategy to render shadows

Hi,

I have a problem rendering my shadows and got the feeling I'm doing it the wrong way. Here's how I do it now:

Background-"Layer" rendered first
B/W version ("created" with glBlendFunc GL_ZERO,GL_ONE_MINUS_SRC_ALPHA) of Sprites blended on top of this with a small offset (to emulate shadow)
Sprites drawn at last.

This method is very fast and looks nice (well, mostly). The problem occurs if two shadows overlap. The blendmode used cause the shadows to darken in this areas. Using "black"(alpha 1.0) as shadowcolor would help but doesn't look good.

Is this something I have to live with, or are there better solutions to draw shadows of alphablended sprites?
How do you do it in your games?

Jake

PS: I'm not limited to Max2D-commands, my engine use both Max2D and raw commands, but any solution should work under DX and OGL.

No one uses dropshadows in his game? Maybe the solution is so simply that I'm unable to see it, but I'm having a really hard time with this. I don't ask for a ready chewn piece of code, just a way to get overlapping dropshadows that "clamp" at a given alphalevel instead of darken themselves.

Hopefully someone can help me,

Jake

Won't be much help but the Zombieblast demo has the same problem. I'd suggest putting something on gamedev.net as, I think, you'll have to get into the 'SetRenderState' (for DX) level.
<edit> For OGL glBlendFunc( GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA )?
<edit2> Shame there was no response to this

Can't you do this by enabling GL_DEPTHTEST and not writing when depth values are equal?

I've knocked up an example of what I'm talking about :-

It uses a depth test (so the shadow pixel doesn't overwrite one already there) and an alpha func which prevents pixels being written if they have an alpha below a .25 threshold.

SuperStrict


Type TBall
	Global i:Int = 0
	Field _r:Int, _g:Int, _b:Int
	Field _x:Float, _y:Float
	
	Method Update()
		_y :+ (10 * Sin(i + _x))
		i:+1
	EndMethod
	
	Function Create:TBall(x:Float, y:Float, r:Int, g:Int, b:Int)
		Local ball:TBall = New TBall
		ball._x = x
		ball._y = y
		ball._r = r
		ball._g = g
		ball._b = b
		Return ball
	EndFunction
EndType


SetGraphicsDriver GLMax2DDriver()
Graphics 800,600

' Make image to draw with
Global image:TImage = CreateImage( 100,100 )
Cls
SetColor 255,255,255
DrawOval 50,50,50,50
GrabImage image,0,0


Global ballList:TList = New TList
Global shadowMode:Int = 0

' Make some objects to render
ballList.AddLast( TBall.Create( 30,150, 255,0,0 ) )
ballList.AddLast( TBall.Create( 50,150, 0,255,0 ) )
ballList.AddLast( TBall.Create( 70,150, 0,0,255 ) )
ballList.AddLast( TBall.Create( 90,150, 255,0,255 ) )
ballList.AddLast( TBall.Create( 110,150, 255,255,0 ) )


Function RenderBackground()
	For Local i:Int = 0 To GraphicsHeight() Step 10
		SetColor Rand(100,155), Rand(100,155), Rand(100,155)
		DrawRect 0, i, GraphicsWidth(), i + (GraphicsHeight()/10)
	Next
EndFunction

Function RenderObjectDepthTestShadows()
	' Draw shadows
	glEnable GL_DEPTH_TEST
	SetColor 0,0,0
	SetAlpha 0.3
	glEnable GL_ALPHA_TEST
	glAlphaFunc GL_GREATER, 0.25

	For Local ball:TBall = EachIn ballList
		DrawImage image, ball._x + 20, ball._y + 20
	Next
	
	glDisable GL_ALPHA_TEST
	glDisable GL_DEPTH_TEST
EndFunction

Function RenderObjectShadowsNoDepthTest()
	' Draw shadows
	SetColor 0,0,0
	SetAlpha 0.3
	For Local ball:TBall = EachIn ballList
		DrawImage image, ball._x + 20, ball._y + 20
	Next
EndFunction

Function RenderObjects()
	SetAlpha 1.0
	
	For Local ball:TBall = EachIn ballList
		SetColor ball._r, ball._g, ball._b
		DrawImage image, ball._x, ball._y
		If Not KeyDown( KEY_SPACE )
			ball.Update()
		EndIf
	Next
EndFunction


SetBlend ALPHABLEND
glClearColor 255, 255,255,255
While Not KeyHit( KEY_ESCAPE )
	glClear GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT
	RenderBackground()
	
	If shadowMode
		RenderObjectDepthTestShadows()
	Else
		RenderObjectShadowsNoDepthTest()
	EndIf
	
	RenderObjects()	
	
	SetColor 255,255,255
	DrawText "DepthTest/AlphaTest shadow rendering - hold SPACE to pause", 100, 10
	DrawText "Press M to toggle Depth/Alpha shadow mode on/off", 100, 30
	Flip
	
	If KeyHit( KEY_M )	
		shadowMode = 1-shadowMode
	EndIf
Wend

Have a look into your replicator, there should be a bunch of flowers, a big pepperoni-onion pizza and a sixpack of beer. Enjoy ;)

I played with your example, read the OGL-docs and now I understand what your example does and how this work.

This is as brilliant as its simple. Thank you again!

Jake

Got them thanks! ;)

No worries, glad to be of service :) - I wrote that while half asleep last night, it was only the flashy background keeping me awake - I should have put an epilepsy warning up.

The same principles apply if you need to convert it to DX.

Hi, it's me again. I don't know why but my DX7-version won't run. I think the ZBuffer check fails, but that's just a guess.

Beside of changing Initialization and CLS to handle Depthbuffer, I changed the following GL-commands to DX:

glEnable GL_DEPTH_TEST
glEnable GL_ALPHA_TEST
glAlphaFunc GL_GREATER, 0.25

Based on my DX7-docs I converted this to:

primarydevice.device.SetRenderState  D3DRS_ZENABLE,true
primarydevice.device.SetRenderState  D3DRS_ZFUNC, D3DCMP_LESS
primarydevice.device.setrenderstate D3DRS_ALPHATESTENABLE, true
primarydevice.device.setrenderstate D3DRS_ALPHAFUNC, D3DCMP_GREATER
primarydevice.device.setrenderstate D3DRS_ALPHAREF,$40

but this won't do anything, so I must miss something.

Here is the sample from above altered for DX use:

SuperStrict


Type TBall
	Global i:Int = 0
	Field _r:Int, _g:Int, _b:Int
	Field _x:Float, _y:Float
	
	Method Update()
		_y :+ (10 * Sin(i + _x))
		i:+1
	EndMethod
	
	Function Create:TBall(x:Float, y:Float, r:Int, g:Int, b:Int)
		Local ball:TBall = New TBall
		ball._x = x
		ball._y = y
		ball._r = r
		ball._g = g
		ball._b = b
		Return ball
	EndFunction
EndType


'SetGraphicsDriver GLMax2DDriver()
SetGraphicsDriver D3D7Max2DDriver()
Graphics 800,600,0,0,GRAPHICS_BACKBUFFER | GRAPHICS_DEPTHBUFFER
' Make image to draw with
Global image:TImage = CreateImage( 100,100 )
Cls
SetColor 255,255,255
DrawOval 50,50,50,50
GrabImage image,0,0


Global ballList:TList = New TList
Global shadowMode:Int = 0

' Make some objects to render
ballList.AddLast( TBall.Create( 30,150, 255,0,0 ) )
ballList.AddLast( TBall.Create( 50,150, 0,255,0 ) )
ballList.AddLast( TBall.Create( 70,150, 0,0,255 ) )
ballList.AddLast( TBall.Create( 90,150, 255,0,255 ) )
ballList.AddLast( TBall.Create( 110,150, 255,255,0 ) )


Function RenderBackground()
	For Local i:Int = 0 To GraphicsHeight() Step 10
		SetColor Rand(100,155), Rand(100,155), Rand(100,155)
		DrawRect 0, i, GraphicsWidth(), i + (GraphicsHeight()/10)
	Next
EndFunction

Function RenderObjectDepthTestShadows()
	' Draw shadows
	'glEnable GL_DEPTH_TEST
	primarydevice.device.SetRenderState  D3DRS_ZENABLE,true
	primarydevice.device.SetRenderState  D3DRS_ZFUNC, D3DCMP_LESS
	SetColor 0,0,0
	SetAlpha 0.3
	'glEnable GL_ALPHA_TEST
	primarydevice.device.setrenderstate D3DRS_ALPHATESTENABLE, true
	
	'glAlphaFunc GL_GREATER, 0.25
	primarydevice.device.setrenderstate D3DRS_ALPHAFUNC, D3DCMP_GREATER
	primarydevice.device.setrenderstate D3DRS_ALPHAREF,$40
	
	For Local ball:TBall = EachIn ballList
		DrawImage image, ball._x + 20, ball._y + 20
	Next
	primarydevice.device.SetRenderState  D3DRS_ZENABLE,false
	primarydevice.device.setrenderstate D3DRS_ALPHATESTENABLE, false
	'glDisable GL_ALPHA_TEST
	'glDisable GL_DEPTH_TEST
EndFunction

Function RenderObjectShadowsNoDepthTest()
	' Draw shadows
	SetColor 0,0,0
	SetAlpha 0.3
	For Local ball:TBall = EachIn ballList
		DrawImage image, ball._x + 20, ball._y + 20
	Next
EndFunction

Function RenderObjects()
	SetAlpha 1.0
	
	For Local ball:TBall = EachIn ballList
		SetColor ball._r, ball._g, ball._b
		DrawImage image, ball._x, ball._y
		If Not KeyDown( KEY_SPACE )
			ball.Update()
		EndIf
	Next
EndFunction


SetBlend ALPHABLEND
'glClearColor 255, 255,255,255
While Not KeyHit( KEY_ESCAPE )
	'glClear GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT
	Primarydevice.device.clear (0,null,D3DCLEAR_TARGET | D3DCLEAR_ZBUFFER, 0,0,0)
	RenderBackground()
	
	If shadowMode
		RenderObjectDepthTestShadows()
	Else
		RenderObjectShadowsNoDepthTest()
	EndIf
	
	'RenderObjects()	
	
	SetColor 255,255,255
	DrawText "DepthTest/AlphaTest shadow rendering - hold SPACE to pause", 100, 10
	DrawText "Press M to toggle Depth/Alpha shadow mode on/off", 100, 30
	Flip
	
	If KeyHit( KEY_M )	
		shadowMode = 1-shadowMode
	EndIf
Wend

I revisited this problem yesterday and think I know why the DX-version fails. Seems like I have to create a zbuffer-surface and attach it to the backbuffer. At least this is what I get from the DX7 docs. I don't know if it's allowed to do this on the fly, but will give this a try.

Anyone knows how to access the backbuffer-surface?

Just live with it. Every game that uses projection shadows has this error.