Mesh creation question & feature request

I've been putting the low-level Blitz mesh creation functionality through some paces lately and I've found that they're a bit on the slow side: in an effort to build a very large number of primitives into a single surface, the code churns anywhere from two to twenty or more times longer when building a primitive manually into a single surface than when calling the built-in Blitz primitive commands to create separate-surface primitives. This doesn't seem so bad if all you're doing is building a couple spheres to toss around, but if you're trying to build several thousands of primitives manually it can slow things down tremendously.

(Side note: please don't ask whether I really need that many objects to toss around. The answer is that yes I do, that it's not for a game, and that preliminary tests with even separate-surface primitives shows that I basically can get the results I want; I just want to speed things up even more by using a minimal number of surfaces.)

So for the question part: has anyone had any success optimizing the built-in mesh creation commands? As far as I can tell I'm building them as fast as can be expected, one tri or quad at a time as necessary, and all inline within a single CreatePrimitive() function.

The feature request part: if it's reasonable to expect that the manual mesh building commands are not meant to compete for speed with the built-in blitz commands, would it be possible to add an AddSurface command, so that once you've modified a specific mesh with positional, scaling and rotation commands you can combine any two objects' surfaces into one?

(note: yes, I can do an 'addsurface' command myself, but the point is speed, and blitz's low-level routines aren't anywhere near fast enough for a solution in Blitz code -- this has to come either from some incredible optimization I'm unaware of, from an addition to the language, or not at all.)

EDIT: as a comparison, to generate ~2500 primitives within the same loop with no other differences takes:

27 seconds with blitz internal primitive commands
61 seconds to generate single surface quads, and
652 seconds (!) to generate single surface primitives.

Obviously the loop does other things as well, but the calls to generate the primitives are the =only= things that change.

I think that when building surfaces blitz assigns memory in chunks for the vert and the tri list. These chunks start out fairly small, and if exceeded a new larger chunk is assigned and data is copied to the new chunk. Not much you can do about that and I am not sure how much it slows things down though maybe a feature to specify the initial number of verts and tris expected when you define a surface would help.

Maybe resuse large surfaces using clearsurface? Would blitz reserve the surface memory though? I doubt it.

That 652 second time is well slow compared to the others. I think you need to whip up some code to demonstrate, and encourage further investigation.

Edit. I have just done a quick test (with debug off!) just adding some verts and triangles and it is slightly faster than creating an equivalent 64 seg sphere primitive. I would say times are comparable. Adding this mesh 10 times to create a larger mesh with 10 surfaces (by painting each sphere mesh a different color) takes about the same amount of time as creaing just one sphere.

It occurs to me if you are also using positionmesh and rotatemesh and scalemesh on large meshes then you are performing lots of accesses to the vertex data and memory. Your own routine may be able to speed things up in this respect, though you would have to code it so it will do them in the order you want. Of course taken individually blitz native code will perform these faster than anything you could write, but maybe (just maybe) if you are doing all these transforms then it may be faster to do them youself in a more optimised fashion (also when combining surfaces as part of the copy process). Taking this to extremes I guess you could read the vertex data into a bank and pass it to a C routine to do the transforms. A lot of hassle though, so if it still turns out slower don't blame me!

Good luck with the feature request! (Although Addmesh does merge two surfaces into one if they have the same properties - paint them with the same brush to see this.)

BTW, BRL. IMO addmesh should really be checking if the vert or tri limit would be exceeded when combining surfaces and if so create a new one.

Isnt it possible to use a few bits of code lying around here (something like "blitz hacked". i cant remember where it was) to directly manipulate the vertex information? as far as i see , using the vertex commands seems to reupload the mesh every time you call them (i hope thats not the case).

When the information has been played with, call an "updatemesh" function which may only just be a single "setvertexcolor" command or something to send the info across just the once?

Ive not tested this yet, but im not realy sure how blitz is doing things so i cant say if that would help.

I am pretty sure is not a case of sending the information to the card that is slowing this down. I suspect that happens at renderworld if the mesh entity is not culled. Some other stuff happens at renderworld as well, mesh bounds are recalculated for instance. (If the mesh has been changed - the MeshWidth function and its cohorts cause that as well).

Anyway yes, you can directly access the vertex information, but it gets complicated when you want to add new verts and tris because of the way memory is assigned as I mentioned above. it may (just may) be possible to tell the vert and tri lists to assign a much bigger memory block on the next expansion, which would cut down on the number of times vert and tri data is copied in memory. Although i noticed this happening when I was poking about I did not try that.

It may be a bit quicker to access this directly for reading and writing vertex data to apply transforms. Presumably the vertex functions have to go around the houses a bit, since they have to lookup the location of the vert and tri data from the surface structure each time. I would not have thought there was a lot in it though.

Anyway I seem to recall Halo mentioning he had done a lot of stuff involving direct access to the vert and tri data so he may be able to offer some insight into that.


I did some tests...

Graphics3D 640,480,0,2

Global surfs,verts,tris,time

Segs=32

Print "Create 10 sphere primitves"
time=MilliSecs()
For i=1 To 10
	bodacious = CreateSphere(segs)
Next
time=MilliSecs()-time
Print "time "+time
printmeshinfo(bodacious)
Print

Print "Create 10 equivalent size meshes"
time=MilliSecs()
For i=1 To 10
	mesh=largemesh(verts,tris)
Next
time=MilliSecs()-time
Print "time "+time
printmeshinfo(mesh)
Print

brush=CreateBrush()
Print "Transform & combine sphere mesh to single Surface 10 times"
time=MilliSecs()
mesh=CreateMesh()
For i=1 To 10
	ScaleMesh bodacious,1.1,1.1,1.1
	RotateMesh bodacious,i*45,i*45,0
	PositionMesh bodacious,2,2,2
;	BrushColor brush,i,0,0 ; uncomment and each mesh will get its own surf
	PaintMesh bodacious,brush 
	AddMesh bodacious, mesh
Next
time=MilliSecs()-time
Print "time "+time
printmeshinfo(mesh)



WaitKey()
End

Function PrintMeshinfo(Mesh)
	verts=0 : tris=0 : surfs=CountSurfaces(mesh)
	For s=1 To surfs
		surf=GetSurface(mesh,s)
		verts=verts+CountVertices(surf)
		tris=tris+CountTriangles(surf)
	Next
	Print "Surfs "+Surfs
	Print "Verts "+Verts
	Print "Tris  "+tris
End Function


Function largemesh(verts,tris)
	Local mesh=CreateMesh()
	surf=CreateSurface(mesh)
	For v=1 To verts
		VertexNormal(surf,AddVertex(surf,v,v*2,v*3),0,0,1) 
	Next
	For t=1 To tris
		AddTriangle(surf,0,0,0)
	Next
	Return mesh
End Function

If nothing else this shows that primitive creation and manual mesh creation times are comparable. Obviously if you were creating a sphere yourself you would probably find the blitz one was a bit quicker, but that is to be expected.

So I guess you have other issues. Maybe you are creating so much stuff you are running out of memory, or perhaps you have a leak, or maybe (just maybe) you were in debug mode?

Many thanks for the cycles David; I had no idea that AddMesh could combine surfaces as well, so that's another possibility for me to investigate.

That said -- your own code demonstrates my =exact= problem if you up the instances required from 10 to 1000. On my rig it takes 24668ms to generate 1000 sphere primitives, while it takes 48242ms to generate 1000 equivalent sized meshes -- which suggests that something nasty's happening when you start doing lots and lots of custom mesh work. Blow that out to 10000 instances and you start to see my problem.

My bad. Try this version.

Graphics3D 640,480,0,2

Global surfs,verts,tris,time

Segs=32
N=1000

Print "Create "+N+" And free "+(N-1)+" sphere primitves"
time=MilliSecs()
For i=1 To N
	bodacious = CreateSphere(segs)
	If i<N FreeEntity bodacious
Next
time=MilliSecs()-time
Print "time "+time
printmeshinfo(bodacious)
Print

Print "Create "+N+" And free "+(N-1)+" equivalent size meshes"
time=MilliSecs()
For i=1 To N
	mesh=largemesh(verts,tris)
	If i<N FreeEntity Mesh
Next
time=MilliSecs()-time
Print "time "+time
printmeshinfo(mesh)
Print

brush=CreateBrush()
surfs=Ceil(N/Float 10)
Print "Transform & Combine sphere mesh to "+surfs+" Surfaces "+N+" times"
time=MilliSecs()
mesh=CreateMesh()
For i=1 To N
	ScaleMesh bodacious,1.1,1.1,1.1
	RotateMesh bodacious,i*45,i*45,0
	PositionMesh bodacious,2,2,2
	BrushColor brush,i Mod surfs,0,0
	PaintMesh bodacious,brush 
	AddMesh bodacious, mesh
Next
time=MilliSecs()-time
Print "time "+time
printmeshinfo(mesh)

WaitKey()
End

Function PrintMeshinfo(Mesh)
	verts=0 : tris=0 : surfs=CountSurfaces(mesh)
	For s=1 To surfs
		surf=GetSurface(mesh,s)
		verts=verts+CountVertices(surf)
		tris=tris+CountTriangles(surf)
	Next
	Print "Surfs "+Surfs
	Print "Verts "+Verts
	Print "Tris  "+tris
End Function


Function largemesh(verts,tris)
	Local mesh=CreateMesh()
	surf=CreateSurface(mesh)
	For v=1 To verts
		VertexNormal(surf,AddVertex(surf,v,v*2,v*3),0,0,1) 
	Next
	For t=1 To tris
		AddTriangle(surf,0,0,0)
	Next
	Return mesh
End Function

How many Gigs of memory do you have? I am not even going to attempt the 10000 test on my old croaker. It is starting to page away on the 1000 test. As for rendering, I am definately not even going there :)

Interesting stuff; I had to wonder if you weren't cheating it a bit by using vertex 0 all the time, but it doesn't appear to be the case. Thanks again for investigating; if manual poly creation is actually =faster= then I really can't explain my current results yet; I'll do some more experimenting and see if I can't narrow down why I'm getting the results I'm getting.

I made a pseudo Geo-Mod system, and found that creating each vertex and tri was much slower than having a ready made template and using AddMesh.