Vertex/image shading (in 2d, based on faked 3d..)

This screenshot is from my tilemap engine, and, as you can see, has no sort of shading to give the viewer any sort of depth perception (except for the lines, obviously).


This is what I would like to achieve (this is from a game, it uses the same basic 2d rendering technique I use).


I render tiles by the vertex, so I think I could actually determine whether a vertex is on a down slope away from or toward the global lighting (South-West) then raise/lower the color values.

Any ideas?

This looks like you could use a psudo 3d coordinate system projected onto 2d like those used by isometric engines, although it's probably easier to pre-render tiles in 3d then just draw them in place with lighting?

Of course the next step would be to use 3D.

Drawing an actual shaded tile can maybe be achieved simply by using SetColor, though I don't know how you're drawing them.

Now, to calculate how much to shade each tile, I can help you.
Define a light source vector coming down at a slight angle, like (1/sqrt(3))*(1,1,1). (The 1/sqrt(3) is to make the vector have length 1).

Now, for each tile, consider it as a quadrilateral defined by points ABCD. It doesn't matter which way round you label the points.
Find the normal of the tile by finding the cross product of the vectors A->B and A->D. Make the normal a unit vector by dividing each component by the length of the vector.

Next, take the dot product of the light vector and the normal. If the tile is directly facing the light source, the dot product will be 1. If it is exactly at right angles the dot product will be 0, and if it is facing away from the light source the dot product will be negative, which you can take to mean there is no light on the tile. You might want to add an ambient amount of light though, so every tile can be seen, even if it's facing away from the light.

for example, skipping out most of the vector operations:

dp#=dotproduct(normal,light)
if dp<0 then dp=0   'can't subtract light!
light#=dp*.6+.4   'unlit tiles still have some ambient light on them
setcolor light*255,light*255,light*255

If you need help with vector operations, I've written a big explanation somewhere on the forum in the past, but I can't remember where. I'll do it again if you ask nicely.

Warpy's right on the money. Each tile's rgb should be manipulated in realtime based on it's height value. Should be a piece of cake. ;)

That is entirely unfeasible. My maps can have a huge range of different heights, from zone to zone. Also, being able to change the height of any tile in realtime is a big plus.

Great, Warpy, I will try this soon.

Like I said, I have all four positions of the quad for a tile, and I render them using the image frame in a TImage (same code from the TImageFrame.Draw method, but I needed to define the points exactly without transformation):

'DirectX (notice the large slowdown because I have to get the driver each tile render - for OpenGL all I need is the imageframe)
Local dxframe:TD3D7ImageFrame, gpx:TD3D7Max2DDriver
dxframe = TD3D7ImageFrame(iframe)
gpx = TD3D7Max2DDriver(TMax2DGraphics.Current()._driver)

Local uv:Float Ptr, c:Int Ptr
uv = dxframe.xyzuv
c = Int Ptr(uv)

uv[0] = _matrix.topleft.x
uv[1] = _matrix.topleft.y
c[3] = gpx.drawcolor
uv[6] = _matrix.topright.x
uv[7] = _matrix.topright.y
c[9] = gpx.drawcolor

uv[12] = _matrix.bottomleft.x
uv[13] = _matrix.bottomleft.y
c[15] = gpx.drawcolor

uv[18] = _matrix.bottomright.x
uv[19] = _matrix.bottomright.y
c[21] = gpx.drawcolor

gpx.SetActiveFrame(dxframe)
gpx.device.DrawPrimitive(D3DPT_TRIANGLEFAN, D3DFVF_XYZ | D3DFVF_DIFFUSE | D3DFVF_TEX1, uv, 4, 0)


'OpenGL
glEnable(GL_TEXTURE_2D)
glBindTexture(GL_TEXTURE_2D, glframe.name)
glBegin(GL_QUADS)

glTexCoord2f(0.0, 0.0)
glVertex2f(_matrix.topleft.x, _matrix.topleft.y)

glTexCoord2f(1.0, 0.0)
glVertex2f(_matrix.topright.x, _matrix.topright.y)

glTexCoord2f(1.0, 1.0)
glVertex2f(_matrix.bottomleft.x, _matrix.bottomleft.y)

glTexCoord2f(0.0, 1.0)
glVertex2f(_matrix.bottomright.x, _matrix.bottomright.y)

glEnd()
glDisable(GL_TEXTURE_2D)

Since I have access to each vertex of the quad, would it look best to give color to each vertex instead of the whole tile? (somehow I think it wouldn't)

Will try! Thanks.

Are you speaking of 3d vectors? (x,y,z)

Currently I have the tile quad completely pre-calculated into a 2d vector (one for each corner of the quad). So far it doesn't seem to be drawing correct.. I'm going to try using 3d vectors where each corner of the quad also has the height.

I'll post the code/technique if it doesn't work.

Yes, 3d vectors. The shading won't depend on where the tile is projected, but on its coordinates in 3d space.

I've got somewhere!
The tiles are shaded now, but they always have the same shading, regardless of slope.

http://img413.imageshack.us/img413/4995/mapeditorshading1fh7.png

Determining color:

Local normal:TVec3, lvec:TVec3, light:Float, dp:Float
lvec = New TVec3.Create(1 / Sqr(3), 1 / Sqr(3), 1 / Sqr(3))

normal = _quad.topright.Copy()
normal.MultiplyVec(_quad.topleft)
normal.MultiplyVec(_quad.bottomright)
normal.Normalize()

dp = normal.DotProduct(lvec)

If dp < 0 Then dp = 0
light = dp * .6 + .4
SetColor(light * 255, light * 144, light * 144)

Vector type (mostly Yahfree's vector module, but with changes from around the forums - and obviously in 3 dimensions):

Rem
	bbdoc: The 3d (x, y & z) vector module.
End Rem
Type TVec3
	
	Field x:Float, y:Float, z:Float
		
		Rem
			bbdoc: Create a 3d vector.
			returns: The created vector (itself).
		End Rem
		Method Create:TVec3(_x:Float, _y:Float, _z:Float)
			
			x = _x
			y = _y
			z = _z
			
			Return Self
			
		End Method
		
		Rem
			bbdoc: Get a copy of this vector.
			returns: A clone of this vector.
		End Rem
		Method Copy:TVec3()
			
			Return New TVec3.Create(x, y, z)
			
		End Method
		
		Rem
			bbdoc: Get the vector's values.
			returns: Nothing. @_x, @_y and @_z will contain the values of the vector.
		End Rem
		Method Get(_x:Float Var, _y:Float Var, _z:Float Var)
			
			_x = x
			_y = y
			_z = z
			
		End Method
		
		Rem
			bbdoc: Set the vector's values.
			returns: Nothing.
		End Rem
		Method Set(_x:Float, _y:Float, _z:Float)
			
			x = _x
			y = _y
			z = _z
			
		End Method
		
		Rem
			bbdoc: Get the vector's angle.
			returns: The angle of the vector.
		End Rem
		Method GetAngle:Float()
			
			Return ATan2(y, Sqr(x * x + z * z))
			
		End Method
		
		Rem
			bbdoc: Add the given values to this vector.
			returns: Nothing.
		End Rem
		Method Add(_x:Float, _y:Float, _z:Float)
			
			x:+_x
			y:+_y
			z:+_z
			
		End Method
		
		Rem
			bbdoc: Add the given vector to this vector.
			returns: Nothing.
		End Rem
		Method AddVec(vec:TVec3)
			If vec = Null Return
			
			x:+vec.x
			y:+vec.y
			z:+vec.z
			
		End Method
		
		Rem
			bbdoc: Subtract this vector by the values given.
			returns: Nothing.
		End Rem
		Method Subtract(_x:Float, _y:Float, _z:Float)
			
			x:-_x
			y:-_y
			z:-_z
			
		End Method
		
		Rem
			bbdoc: Subtract this vector by another vector.
			returns: Nothing.
		End Rem
		Method SubtractVec(vec:TVec3)
			If vec = Null Return
			
			x:-vec.x
			y:-vec.y
			z:-vec.z
			
		End Method
		
		Rem
			bbdoc: Multiply this vector by the values given.
			returns: Nothing.
		End Rem
		Method Multiply(_x:Float, _y:Float, _z:Float)
			
			x:*_x
			y:*_y
			z:*_z
			
		End Method
		
		Rem
			bbdoc: Multiply this vector by another vector.
			returns: Nothing.
		End Rem
		Method MultiplyVec(vec:TVec3)
			If vec = Null Return
			
			x:*vec.x
			y:*vec.y
			z:*vec.z
			
		End Method
		
		Rem
			bbdoc: Divide this vector by the values given.
			returns: Nothing.
			about: This does check if the divisors are zero.
		End Rem
		Method Divide(_x:Float, _y:Float, _z:Float)
			
			If _x <> 0 And _y <> 0 And _z <> 0
				x:/_x
				y:/_y
				z:/_z
			End If
			
		End Method
		
		Rem
			bbdoc: Divide this vector by another vector.
			returns: Nothing.
			about: This does check if the given vector's values are zero.
		End Rem
		Method DivideVec(vec:TVec3)
			If vec = Null Return
			
			If vec.x <> 0 And vec.y <> 0 And vec.z <> 0
				x:/vec.x
				y:/vec.y
				z:/vec.z
			End If
			
		End Method
		
		Rem
			bbdoc: Get the dot product of this vector and the vector given.
			returns: The dot product of the two vectors.
		End Rem
		Method DotProduct:Float(vec:TVec3)
			
			Return x * vec.x + y * vec.y + z * vec.z
			
		End Method
		
		Rem
			bbdoc: Get this vector reflected upon another.
			returns: A new vector that is the result of this vector reflecting off the given vector.
		End Rem
		Method Reflected:TVec3(vec:TVec3)
			Local vecn:TVec3 = vec.Normalized()
			Local vec1:TVec3 = Copy()
			Local vecn_DOT_vec1:Float = vecn.DotProduct(vec1)
			
			vecn.Multiply(2 * vecn_DOT_vec1, 2 * vecn_DOT_vec1, 2 * vecn_DOT_vec1)
			
			vec1.SubtractVec(vecn)
			
			Return vec1
			
		End Method
		
		Rem
			bbdoc: Get a normalized vector of this vector.
			returns: This vector normalized.
			about: See also #Normalize.
		End Rem
		Method Normalized:TVec3() 
			Local vector:TVec3 = Copy()
			
			vector.Normalize()
			
			Return vector
			
		End Method
		
		Rem
			bbdoc: Normalize this vector.
			returns: Nothing.
		End Rem
		Method Normalize() 
			Local magn:Float = GetMagnitude()
			
			If magn <> 0
				
				x:/magn
				y:/magn
				z:/magn
				
			End If
			
		End Method
		
		Rem
			bbdoc: Get the length (magnitude) of the vector.
			returns: The magnitude of the vector.
		End Rem
		Method GetMagnitude:Float()
			
			Return Sqr(x * x + y * y + z * z)
			
		End Method
		
End Type

Unless I'm missing something, you're doing the cross product wrong. You want to take the cross product (topleft - topright) x (bottomright - topright)

And MultiplyVec in Yahfree's code isn't a cross product operation. In fact, I can't think what it could be for. Wikipedia has the proper definition for you, and here's a function you can add to your code:

Function CrossProduct:TVec3( vec1:TVec3, vec2:TVec3 )
	Return TVec3.Create( vec1.y*vec2.z - vec1.z*vec2.y, vec1.z*vec2.x - vec1.x*vec2.z, vec1.x*vec2.y - vec1.y*vec2.x )
End Function

So:

Local normal1:TVec3, normal2:TVec3, normal:TVec3
normal1 = _quad.topleft.copy()
normal2 = _quad.bottomright.copy()
normal1.SubtractVec(_quad.topright)
normal2.SubtractVec(_quad.topright)

normal = TVec3.CrossProduct( normal1, normal2 )

By the way, I think that in defining vector types, the operators should return new vector instances, not modify existing ones. Maybe it would be slow if you're doing loads of vector work, but in that case you should probably think about representing them with float triples.

Thanks Warpy!

This code works perfect (light vector and colors will be initiated once in the near future):

Local lvec:TVec3, light:Float, dp:Float
lvec = New TVec3.Create(1 / Sqr(3), - 1 / Sqr(3), 1.0)

Local normal1:TVec3, normal2:TVec3, normal:TVec3
normal1 = _quad.topright.Copy()
normal2 = _quad.bottomright.Copy()
normal1.SubtractVec(_quad.topleft)
normal2.SubtractVec(_quad.topleft)

normal = TVec3.CrossProduct(normal1, normal2)
normal.Normalize()
dp = normal.DotProduct(lvec)

If dp < 0 Then dp = 0
light = dp * 0.4 + 0.6
SetColor(light * 255, light * 255, light * 255)

I also changed the quad positions used because they were slightly wrong (topleft = A, topright = B, bottomleft = C, bottomright = D).
I still haven't found the best light position, some seem to make the color change too intense, some not enough.
http://img252.imageshack.us/img252/2800/earlywithshadingmoew5.png
http://img252.imageshack.us/img252/7167/earlywithshadingdebuglizx0.png

Glad it works!

As you're doing this in realtime, why not add a little control to change the angle of the light vector until you get something you like?

Good idea!

Is the Normalize method correct in my vector type?:

		Method Normalize() 
			Local magn:Float = GetMagnitude()
			
			If magn <> 0
				
				x:/magn
				y:/magn
				z:/magn
				
			End If
			
		End Method

Curious because of the method minib3d uses (klepto's version):

	Method Normalize()
	
		Local d#=1/Sqr(x*x+y*y+z*z)
		x:*d
		y:*d
		z:*d
		
	End Method

Also, what would I have to do to smooth the light coloring over a neighboring tiles?

Yep, same thing. sqr(x*x+y*y+z*z) is the magnitude, and you're dividing by it and klepto is multiplying by (1 dividided by the magnitude). 6 and two 3s.

To smooth the light colouring, you'll have to look into vertex colouring. I don't know OpenGL or Direct3d, so you're on your own on that one.

Ah, righto.. Thanks.