Stencil shadow rendering(question to tom)

Blitz3D Forums/Blitz3D Programming/Stencil shadow rendering(question to tom)

Devils Child
 #1
hello everybody...or rather: hello tom!

i have got a question to toms dx7 dll:

well, i have managed a first little system in my team, and now i have got following problem: if the camera gets inside the volume, the shadows are looking corruptly... have you any idea??

Function Render(shadows = False)
If shadows Then
	;Build volumes
	ClearSurface VolumeSurface
	sl.ShadowLight = First ShadowLight
	For sm.ShadowMesh = Each ShadowMesh
		If sm\casting Then CreateVolume(sm\ent, sl\ent, sl\parallel)
	Next
	;Render Shadow
  HideEntity ShadowPlane
	HideEntity VolumeMesh
	For sm.ShadowMesh = Each ShadowMesh
		ShowEntity sm\ent
	Next
	SetRenderState(Direct3DDevice7, D3DRS_ZWRITEENABLE, True)
	SetRenderState(Direct3DDevice7, D3DRS_STENCILENABLE, False)
	SetRenderState(Direct3DDevice7, D3DRS_ALPHABLENDENABLE, False)
  DeviceClear(Direct3DDevice7, 7, $00008080, 1, 0)
  CameraClsMode ShadowCam, True, True
  UpdateWorld
  RenderWorld
  CameraClsMode ShadowCam, False, False
	For sm.ShadowMesh = Each ShadowMesh
		HideEntity sm\ent
	Next
	ShowEntity VolumeMesh
	SetRenderState(Direct3DDevice7, D3DRS_ZWRITEENABLE, False)
	SetRenderState(Direct3DDevice7, D3DRS_STENCILENABLE, True)
	SetRenderState(Direct3DDevice7, D3DRS_STENCILFUNC, D3DCMP_ALWAYS)
	SetRenderState(Direct3DDevice7, D3DRS_STENCILREF, 1)
	SetRenderState(Direct3DDevice7, D3DRS_STENCILMASK, $FFFFFFFF)
	SetRenderState(Direct3DDevice7, D3DRS_STENCILWRITEMASK, $FFFFFFFF)
  SetRenderState(Direct3DDevice7, D3DRS_CULLMODE, D3DCULL_CCW)
	SetRenderState(Direct3DDevice7, D3DRS_STENCILZFAIL, D3DSTENCILOP_KEEP)
	SetRenderState(Direct3DDevice7, D3DRS_STENCILFAIL, D3DSTENCILOP_KEEP)
	SetRenderState(Direct3DDevice7, D3DRS_STENCILPASS, D3DSTENCILOP_INCR)
	SetRenderState(Direct3DDevice7, D3DRS_ALPHABLENDENABLE, True)
	SetRenderState(Direct3DDevice7, D3DRS_SRCBLEND, D3DBLEND_ZERO)
	SetRenderState(Direct3DDevice7, D3DRS_DESTBLEND, D3DBLEND_ONE)
	RenderWorld
	SetRenderState(Direct3DDevice7, D3DRS_STENCILPASS, D3DSTENCILOP_DECR)
	SetRenderState(Direct3DDevice7, D3DRS_CULLMODE, D3DCULL_CW)
	RenderWorld
	SetRenderState(Direct3DDevice7, D3DRS_CULLMODE, D3DCULL_CCW)
	HideEntity VolumeMesh
	SetRenderState(Direct3DDevice7, D3DRS_ZWRITEENABLE, True)
	SetRenderState(Direct3DDevice7, D3DRS_STENCILENABLE, False)
	SetRenderState(Direct3DDevice7, D3DRS_ALPHABLENDENABLE, False)
	SetRenderState(Direct3DDevice7, D3DRS_ZENABLE, False)
	SetRenderState(Direct3DDevice7, D3DRS_STENCILENABLE, True)
	SetRenderState(Direct3DDevice7, D3DRS_ALPHABLENDENABLE, True)
	SetRenderState(Direct3DDevice7, D3DRS_SRCBLEND, D3DBLEND_SRCALPHA)
	SetRenderState(Direct3DDevice7, D3DRS_DESTBLEND, D3DBLEND_INVSRCALPHA)
	SetRenderState(Direct3DDevice7, D3DRS_STENCILREF, 1)
	SetRenderState(Direct3DDevice7, D3DRS_STENCILFUNC, D3DCMP_LESSEQUAL)
	SetRenderState(Direct3DDevice7, D3DRS_STENCILPASS, D3DSTENCILOP_REPLACE)
	ShowEntity ShadowPlane
	RenderWorld
	SetRenderState(Direct3DDevice7, D3DRS_ZENABLE, True)
	SetRenderState(Direct3DDevice7, D3DRS_STENCILENABLE, False)
	SetRenderState(Direct3DDevice7, D3DRS_ALPHABLENDENABLE, True)
	HideEntity ShadowPlane
	SetRenderState(Direct3DDevice7, D3DRS_ALPHABLENDENABLE, False)
Else
	HideEntity ShadowPlane
	CameraClsMode ShadowCam, True, True
	RenderWorld
EndIf
End Function


Tom
 #2
The solution is commonly known as 'ZFail method' or 'Carmacks reverse'. It's the slower method of doing stencil shadows but it's more robust & accurate.

Also, there are problems you'll come across where the near & far clip planes (camera Near & Far settings) can clip your mesh volumes. If you use the ZFail method, there is an easier solution to that also (infinite far clip plane)

Check google for many explanations.


There are still isues with doing stencils, I'm hoping BRL will do something about it, info here:
http://www.blitzbasic.com/Community/posts.php?topic=59602

Please, if you think this is a good idea (hell yeh! :) then drop a post supporting it :)


While I have your attention, here is my latest DLL. I've made some better functions for doing renderstates because SetRenderState is too much typing :)
http://www.tomspeed.com/dx7dll/

Some things you should know:

Blitz3D, as of v1.96, doesn't specifically create a stencil capable ZBuffer. Out of pure luck, NVidia always seems to create one, ATI cards don't. (This is why the old DeviceClear never worked correctly with the +4 flag)

Skidracer has said he'd look into changing this so Blitz3D always creates one if possible. Until then, you can check for the stencil buffer depth, and create a new one if neccessary, like so:
;We need a stencil buffer greater than 1bit (idealy 8bits) to do a viable Carmacks Reverse
If GetStencilBitDepth() < 8
  CreateStencilBuffer()
  If GetStencilBitDepth() < 8 RuntimeError "Card does not support >1bit Stencil Buffers"
End If


DeviceClear() now works correctly when clearing the stencil buffer if you create one yourself on ATI cards, and it no longer takes a RECT Type in the parameters:

Example, clear color to red, zbuffer to 1.0, stencil to 0

Const COLBUFF=1
Const ZBUFF=2
Const STENBUFF=4
DeviceClear COLBUFF+ZBUFF+STENBUFF,$ff0000,1.0,0


Before using any of the functions in the DLL you need to initilize it like so:
;----------------
;Init the DX7 DLL
;----------------
DX_D3D=SystemProperty$("Direct3D7")
DX_DEV7=SystemProperty$("Direct3DDevice7")
DX_DRAW7=SystemProperty$("DirectDraw7")
DX_HWND=SystemProperty$("AppHWND")
DX_INSTANCE=SystemProperty$("AppHINSTANCE")
If SetSystemProperties(dx_d3d,dx_dev7,dx_draw7,dx_hwnd,dx_instance) RuntimeError "Error with 1 or more SystemProperty vars"


Here are some of the new functions, much easier to use! Check the .decls file for others.

;D3DSTENCILOP
Const SOP_KEEP = 1
Const SOP_ZERO = 2
Const SOP_REPLACE = 3
Const SOP_INCRSAT = 4
Const SOP_DECRSAT = 5
Const SOP_INVERT = 6
Const SOP_INCR = 7
Const SOP_DECR = 8

;D3DCMPFUNC
Const CMP_NEVER = 1
Const CMP_LESS = 2
Const CMP_EQUAL = 3
Const CMP_LESSEQUAL = 4
Const CMP_GREATER = 5
Const CMP_NOTEQUAL = 6
Const CMP_GREATEREQUAL = 7
Const CMP_ALWAYS = 8

EnableStencil
DisableStencil
SetStencilFunc CMP
SetStencilPass SOP
SetStencilFail SOP
SetStencilZFail SOP
SetStencilRef VAL#
;...

;Note, these do work, but RenderWorld() overrides their
;settings :| I'm hoping BRL will fix this behaviour as
;they're crucial to implementing good stencil shadows!
EnableZBuffer
DisableZBuffer
EnableZWrites
DisableZWrites
EnableAlphaBlending
DisableAlphaBlending


Devils Child
 #3
ok, i'll check it out when i get back from school :)
ths for your new dll and for your support! and if you are interested into my and dare devils 's volumes, just have a look into my worklogs :)

Devils Child
 #4
ok, i'm home now...

i am a lil bit confused now, and i don't know how to write the render() function now?

function render(shadows = false)
????
end function