AntiAlias

Ok I know this question has been asked before because I searched it, but so far i havent realy goten much help found. All i want to do is lessen the jaged edges of a black border (cel-shaded) 3D object. Its realy anoying to look at. Now before anyone gets into it im not loooking for any reasons why i should NOT have Antialising due to memory or render time or whatever. ok. The closest thing i found was an algorythem created by someone (using an example of orange and grey pipes) that softened the edges of pixles that were to different. I never got to test this as some of the functions were missing (bufferwidth,height,red,blue,green...)
I also looked into bloom filters.. have one that works but it doesnt work on darker colors so its sorta useless of this. Any help is fine thanks.

You could copy the buffer to a texture and use the blur filter in the code archives... That might kill the frame rate though...

Well, I'm working on an experiment that will do a raster edge detection along selected edges for cel-shading. (Hopefully way way faster than applying an edge detection to the whole screen!)

It is almost certainly going to be really, really slow, but if, in the remarkable event, it isn't too slow then it would probably be ideal.

You're currently using inverted mesh for the black borders, right?

Im sure there was also a DLL somewhere lying around that was a sort of helper to properly enable AA for a B3D app, dunno where its got too though :(

Doesn't "AntiAlias True" work?

You'll be hard pushed to get any AA working effectively without proper hardware support. The best solution perhaps would be to render the scene in double resolution, perhaps in chunks, it textures and have the hardware scale it onto a screen-sized textured quad. Or if it's for black lines only you could potentially use the vertex data to draw 2D antialiased lines along the black edges. That might well work. At least for the outside edge of the black object. The inside edge would be harder.

Still if anyone finds a working solution I'd love to know as my program's using cartoon shading. Though remember you can set your card drivers to forced AA. On a Ti4200 anything above 2x drops the framerate though and it doesn't make a huge difference.

In fact 4x AA, 8xAA still looks jaggied. It's just unavoidable with black lines against bright colours unless you draw the lines with a 2D antialising algorithm.

Ross: That sounds simple. i didnt know there wasa "Blur" filter. ill look there also.
Mr Snidesmin: Yah im using a black inverted mesh. And on the BG it REALY looks bad.

All in all im trying to find something that doesnt rely on hardware at all. Because the fact is is if you dont program or arent a serious gamer you usualy dont HAVE a graphics card. most times its onboard and i like to make things compatable for a wide range. But if no solution in algorithems is goodenough with a good FPS then ill dive into other relms. Thanks all

I think halo did an antialiasing system...

AFAIK there's no reliable antialias system for Blitz3D.
It makes me sad! :(

I'll post Halo's antialiasing code when I get home. Pretty results. A tad slow in realtime, but.

Oh yeah - I'm back! >_< 8 months in Thailand with no internet was a pain. But at least I got some coding done. News soon. :)

+BlackD

Fred, Blitz AA has been broken for ages... and there is NO sure way to set it in DX anyway; even our 3DMarkXXXX doesn't have AA as default setting although we would like that.

Reason for this is that there's no way to make sure (within code) that driver actually enabled AA - yes, it may may say that yes I can and will do that but then just promtly ignore the AA request. Also there's now so many ways to do AA that findng equal or comparable (speed/quality wise) setting for all is close to impossible.

But it WOULD be nice to have Blitz3D even try to enable whatever AA user driver has as a default setting... and it would be nice to have DXTC too, but that's dreamworld hallucination. :)

Halo's antialias. I think it's more for static-image antialiasing rather than in-game, maybe for a render preview in a 3d modeller like.. ooh say.. Cartography Shop?

Function AntiAliasBuffer(buffer,passes=1,threshhold=20)
gfxbuffer=GraphicsBuffer()
SetBuffer buffer
LockBuffer buffer
w=BufferWidth(buffer)
h=BufferHeight(buffer)
For p=1 To passes
	For x=0 To w-1
		For y=0 To h-1
			hue=ReadPixelFast(x,y)
			r=Red(hue)
			g=Green(hue)
			b=Blue(hue)
			count=BottomLeftPixelsAntialiased(buffer,x,y,threshhold)
			For n=1 To count
				perc#=Float(n)/Float(count)
				hue=ReadPixelFast(x-n,y)
				lr=Red(hue)*perc+r*(1.0-perc)
				lg=Green(hue)*perc+g*(1.0-perc)
				lb=Blue(hue)*perc+b*(1.0-perc)
				WritePixelFast x-n,y,RGB(lr,lg,lb)
				Next
			count=BottomRightPixelsAntialiased(buffer,x,y,threshhold)
			For n=1 To count
				perc#=Float(n)/Float(count)
				hue=ReadPixelFast(x+n,y)
				lr=Red(hue)*perc+r*(1.0-perc)
				lg=Green(hue)*perc+g*(1.0-perc)
				lb=Blue(hue)*perc+b*(1.0-perc)
				WritePixelFast x+n,y,RGB(lr,lg,lb)
				Next
			count=VerticalBottomLeftPixelsAntialiased(buffer,x,y,threshhold)
			For n=1 To count
				perc#=Float(n)/Float(count)
				hue=ReadPixelFast(x,y-n)
				lr=Red(hue)*perc+r*(1.0-perc)
				lg=Green(hue)*perc+g*(1.0-perc)
				lb=Blue(hue)*perc+b*(1.0-perc)
				WritePixelFast x,y-n,RGB(lr,lg,lb)
				Next
			count=VerticalBottomRightPixelsAntialiased(buffer,x,y,threshhold)
			For n=1 To count
				perc#=Float(n)/Float(count)
				hue=ReadPixelFast(x,y+n)
				lr=Red(hue)*perc+r*(1.0-perc)
				lg=Green(hue)*perc+g*(1.0-perc)
				lb=Blue(hue)*perc+b*(1.0-perc)
				WritePixelFast x,y+n,RGB(lr,lg,lb)
				Next
			Next
		Next
	Next
UnlockBuffer buffer
SetBuffer gfxbuffer
End Function

Function BottomLeftPixelsAntialiased(buffer,x,y,threshhold)
If x=0 Return
hue=ReadPixelFast(x,y)
r=Red(hue)
g=Green(hue)
b=Blue(hue)
w=BufferWidth(buffer)
h=BufferHeight(buffer)
Repeat
	found=False
	hue=ReadPixelFast(x-1,y)
	lr=Red(hue)
	lg=Green(hue)
	lb=Blue(hue)
	If Abs(lr-r)>threshhold Or Abs(lg-g)>threshhold Or Abs(lb-b)>threshhold
		If y<h
			hue=ReadPixelFast(x-1,y+1)
			blr=Red(hue)
			blg=Green(hue)
			blb=Blue(hue)
			If Abs(blr-r)<threshhold And Abs(blg-g)<threshhold And Abs(blb-b)<threshhold
				found=True
				EndIf				
			EndIf
		EndIf
	If found
		count=count+1
		Else
		Exit
		EndIf
	If x=0 Exit
	x=x-1
	Forever
Return count
End Function

Function BottomRightPixelsAntialiased(buffer,x,y,threshhold)
hue=ReadPixelFast(x,y)
r=Red(hue)
g=Green(hue)
b=Blue(hue)
w=BufferWidth(buffer)
h=BufferHeight(buffer)
If x=w Return
Repeat
	found=False
	hue=ReadPixelFast(x+1,y)
	lr=Red(hue)
	lg=Green(hue)
	lb=Blue(hue)	
	If Abs(lr-r)>threshhold Or Abs(lg-g)>threshhold Or Abs(lb-b)>threshhold
		If y<h
			hue=ReadPixelFast(x+1,y+1)
			blr=Red(hue)
			blg=Green(hue)
			blb=Blue(hue)
			If Abs(blr-r)<threshhold And Abs(blg-g)<threshhold And Abs(blb-b)<threshhold
				found=True
				EndIf				
			EndIf
		EndIf
	If found
		count=count+1
		Else
		Exit
		EndIf
	If x=w-1 Exit
	x=x+1
	Forever
Return count
End Function

Function VerticalBottomLeftPixelsAntialiased(buffer,x,y,threshhold)
If y=0 Return
hue=ReadPixelFast(x,y)
r=Red(hue)
g=Green(hue)
b=Blue(hue)
w=BufferWidth(buffer)
h=BufferHeight(buffer)
Repeat
	found=False
	hue=ReadPixelFast(x,y-1)
	lr=Red(hue)
	lg=Green(hue)
	lb=Blue(hue)
	If Abs(lr-r)>threshhold Or Abs(lg-g)>threshhold Or Abs(lb-b)>threshhold
		If x>0
			hue=ReadPixelFast(x-1,y-1)
			blr=Red(hue)
			blg=Green(hue)
			blb=Blue(hue)
			If Abs(blr-r)<threshhold And Abs(blg-g)<threshhold And Abs(blb-b)<threshhold
				found=True
				EndIf				
			EndIf
		EndIf
	If found
		count=count+1
		Else
		Exit
		EndIf
	If y=0 Exit
	y=y-1
	Forever
Return count
End Function

Function VerticalBottomRightPixelsAntialiased(buffer,x,y,threshhold)
If y=h Return
hue=ReadPixelFast(x,y)
r=Red(hue)
g=Green(hue)
b=Blue(hue)
w=BufferWidth(buffer)
h=BufferHeight(buffer)
Repeat
	found=False
	hue=ReadPixelFast(x,y+1)
	lr=Red(hue)
	lg=Green(hue)
	lb=Blue(hue)
	If Abs(lr-r)>threshhold Or Abs(lg-g)>threshhold Or Abs(lb-b)>threshhold
		If x>0
			hue=ReadPixelFast(x-1,y+1)
			blr=Red(hue)
			blg=Green(hue)
			blb=Blue(hue)
			If Abs(blr-r)<threshhold And Abs(blg-g)<threshhold And Abs(blb-b)<threshhold
				found=True
				EndIf				
			EndIf
		EndIf
	If found
		count=count+1
		Else
		Exit
		EndIf
	If y=h Exit
	y=y+1
	Forever
Return count
End Function

+BlackD