Grabimage and image buffers

What would be the best way to use GrabImage where you want to, for example, add a decal to a ship image?
Are there imagebuffers to which I can draw both before the grabimage or will I need to use backbuffer and clever use of cls?

For now it is backbuffer and clever use of CLS.

For images I have not been able to get the maskcolor to work yet, for pixmaps it was pretty simple though. Just use the grabpixmap and then setpixmapmaskcolor (command name there may be messed up). I tried doing the same thing for an image but the mask color didn't work (after locking the image and working on the returned pixmap that is).

I have not tried doing the writepixel command on it yet, ran out of time last night.

I've got writepixel and maskpixmap working OK but there's a lot of messing around.
You've probably seen the other post concerning Image masking...
http://www.blitzbasic.com/Community/posts.php?topic=42105
If there was an easy way to turn pixmap into a new normal image it might be OK but having to do the 'backbuffer shuffle' is a pain.
<edit> P.S. Because MaskPixmap returns another pixmap I don't think it will affect the initial (locked) image as Writepixel does.

Yup - I saw that after I was messing around a bit. I didn't read through all the logic to figure it out though. I think I'll create and extended image class that does some of this good stuff and post it later.

Oh dear.... i think i may have made a mistake buying max just yet :( quite a few things i need just arent there yet.. Oh well, i will wait and be patient, and hopefully they will arrive.

Something else..

ive tried finding where the DrawRect command is stored in the BMX modules.... i can't.. anyone know where they are? i may try to create a kludge!

Surely the best way to add a decal to ship image is to draw the ship then draw the alpha-blended decal on top?

<image>=LoadImage(p:TPixmap)

Hows about that :)

@shagwana
Blimey.. that's right.
Not sure how it helps me unless I can draw 2 images to the same pixmap preferably without using writepixel
@teamonkey
if I had a number of decals (bullet marks) on numerous ships wouldn't that involve a lot of drawing?

you wouldn't draw decals on any map you place meshes ...

drawing a bullet on one ship would place the bullet on all ships with that texture.

@tonyg
Yes, but your graphics card is optimised to draw lots and lots of textured polygons. Let it do what it's good at.

The amount of information you're sending to the graphics card, telling it to render a textured quad is not much greater than it is to tell it to render an individual pixel.

Sorry everybody but I'm not sure how people are using 'textures' in this situation.
If I have a 100*100 image (sprite, bob or whatever) and start adding burn marks I'd like the actual image to contain those marks rather than create a pixmap of the marks to draw over the top or drawing anumber of seperate images.
Maybe a better example is a composite sprite in a 2D RPG.
I'd like my base sprite in it's own imagebuffer and another imagebuffer where I superimpose another image (armor, weapons and the like). This imagebuffer is then written to the backbuffer.
I must be missing something but th ability to draw more than one image to a buffer and then save it as a seperate image is very useful.

Why not just draw it straight to the backbuffer?

Max2D isn't classic 2D. I think this is where people are running into problems. The "fast" techniques of the old pixel buffer world (think original BlitzBasic for the PC) aren't necessarily the best route to take with 3D-based 2D.

As teamonkey said, modern 3D graphics cards are optimised to draw lots of textured polygons; in the case of Max2D, these are equivalent to "sprites" but even that name is misleading.

In general, modifying textures at runtime is going to cost you in performance. On 3D hardware it's far more efficient to generate more polygons to gain the effect you're looking for, even if they're just "colored" to emulate what a pixel was in the 2D world.

For instance, "drawing" to the backbuffer isn't directly putting a color value for a pixel into a memory buffer. Here's what the Plot method does:

Method Plot( x#,y# )
    DisableTex
    glBegin GL_POINTS
    glVertex2f x+.5,y+.5
    glEnd
End Method

The end result is the same (usually) but there's a lot more happening here. This gives additional power but also carries some additional overhead and design considerations.

This is one of the problems with 2D in 3D. It's a partial abstraction. It's easier to think of it as 3D with a fixed camera in one dimension than to think of it in terms of classic 2D.

TPixmap (in DRAM) is the closest thing you have to the old world 2D. However, once a TPixmap is "promoted" to a TImage (VRAM) it's very costly to alter it.

A few people have said this so I'm guessing it isn't a problem to (psuedo)...
cls
draw base_image
draw comp_image
grabimage to pixmap
draw extra pixels
convert pixmap to image
cls
drawimage
What if I did this for 100 images?
I guess I'll have to try it and see.
@Xacto, hmmm, I can't pretend I understand everything you're saying. I'm 2D by heart and I have stuck with Blitz so I don't have to understand too much about programming.
If I have 2 png files which I want to 'amalgamate' then are you saying it's better to load and draw them indivudally?
Would it matter whether I drew 100 base images and then another 300 composite sprites on top along with tiles and background data? Wouldn't I be doing this every 'render'
rather than the 'old way' of drawing the 100 base images with the occasional draw to an imagebuffer?

@tonyg
If each composite sprite was to be identical, then you're right. But I would create the composite using TPixmaps (i.e. in system RAM) and then convert it to a TImage when ready.

If, on the other hand, you had loads of similar ships and want to decorate them individually (different colours, burn marks etc.) you'd be better off layering them. Tilemaps are good for layering - take a base ground texture and overlay craters, scrub, houses etc. on top. Better than having loads of different textures.

OK, a quick check shows drawimage in bmx is MUCH slower than B3D/B2D but maybe that's what people are trying to tell me. Using pixmaps as if they were B2D 'standard' images is something I'll need to check.
I still think I am missing something though.
What are 'standard' images used for in bmx if they are slower than B3D equivalent and not designed to be changed?
What other objects (rather than png, jpg etc) should I be using to get a decent draw speed.
For info here are the 2 tests...
BMX

Graphics 640,480
Incbin "kilt.png"
Incbin "man.png"
man = LoadImage("incbin::man.png")
kilt = LoadImage("incbin::kilt.png")
While Not KeyDown(key_escape)
  start_time=MilliSecs()
  For x = 10 To 600 Step 6 
     For y = 10 To 480 Step 6 
      DrawImage man,x,y
      DrawImage kilt,x,y+40
     Next 
  Next 
  End_time=MilliSecs()
  total_time = end_time - start_time
  DrawText "Total time : " + total_time,0,0
  Flip
  Cls
  FlushMem
Wend
End

Results : 625ms/610ms (debug/non-debug)
B3D

Graphics 640,480
SetBuffer BackBuffer()
man = LoadImage("man.png")
kilt = LoadImage("kilt.png")
While Not KeyHit(1)
  start_time=MilliSecs()
  For x = 10 To 600 Step 6 
     For y = 10 To 480 Step 6 
      DrawImage man,x,y
      DrawImage kilt,x,y+40
     Next 
  Next 
  End_time=MilliSecs()
  total_time = end_time - start_time
  Text 0,0,"Total time : " + total_time
  Flip
  Cls
Wend
End

Results : 280/260
bmx

Graphics 640,480
Incbin "kilt.png"
Incbin "man.png"
man = LoadImage("incbin::man.png")
kilt = LoadImage("incbin::kilt.png")
DrawImage man,0,0
DrawImage kilt,0,40
new_man=CreateImage(32,100)
GrabImage new_man,0,0
cls
While Not KeyDown(key_escape)
  start_time=MilliSecs()
  For x = 10 To 600 Step 6 
     For y = 10 To 480 Step 6 
      DrawImage new_man,x,y
'      DrawImage kilt,x,y+40
     Next 
  Next 
  End_time=MilliSecs()
  total_time = end_time - start_time
  DrawText "Total time : " + total_time,0,0
  Flip
  Cls
  flushmem
Wend
end

Results : 424/418
B3D

Graphics 640,480
SetBuffer BackBuffer()
man = LoadImage("man.png")
kilt = LoadImage("kilt.png")
DrawImage man,0,0
DrawImage kilt,0,40
new_man=CreateImage(32,100)
GrabImage new_man,0,0
Cls
While Not KeyHit(1)
  start_time=MilliSecs()
  For x = 10 To 600 Step 6 
     For y = 10 To 480 Step 6 
      DrawImage new_man,x,y
;      DrawImage kilt,x,y+40
     Next 
  Next 
  End_time=MilliSecs()
  total_time = end_time - start_time
  Text 0,0,"Total time : " + total_time
  Flip
  Cls
Wend
End

Results : 195/182 (same results using imagebuffers).
man = 32*100 stickman
kilt = 32*32 rectangle.
Maybe I'm not testing apples for apples.
What 'objects' should I be using in BMX to get similar results?
<edit> I realise these tests are simply measuring draw time
but, in the end, that's probably the issue.
<edit> Create the composite sprite from pixmaps. Trying to drawpixmap caused my system to sloooowwww (124610ms was the only measurement I saw for a loop).
Converting to an image worked but, obviously, produced similar results to the single drawimage test.

Those tests aren't measuring draw time, they're measuring how fast you can tell the graphics card to draw the image. glFinish waits until all the graphics have finished drawing.

Just as an experiment I stuck a glFinish just after FlushMem in the first test. My total time dropped from an average 260ms down to 60ms! Basically, the Cls is taking a lot of time and the screen hasn't finished clearing by the time the loop starts drawing the sprites again. You're timing some of the Cls in this test. I don't know why Cls is taking so long though.

Moral - to optimise, stick some game logic between the Cls and the actual drawing and some more between the drawing and Flip (which calls glFinish before it flips buffers).

Sorry to reply to my own post.
I found this thread...
http://www.blitzmax.com/Community/posts.php?topic=41644
which, at first, suggested grouping drawimages commands.
I tried this with the first test...

Graphics 640,480
Incbin "kilt.png"
Incbin "man.png"
man = LoadImage("incbin::man.png")
kilt = LoadImage("incbin::kilt.png")
While Not KeyDown(key_escape)
  start_time=MilliSecs()
  For x = 10 To 600 Step 6
    For y = 10 To 480 Step 6
      DrawImage man,x,y
'      DrawImage kilt,x,y+40
 '    sprite_drawn:+1
     Next 
  Next 
  For x = 10 To 600 Step 6
    For y = 10 To 480 Step 6
'      DrawImage man,x,y
      DrawImage kilt,x,y+40
 '    sprite_drawn:+1
     Next 
  Next 
 ' TileImage(man,0,0)
  End_time=MilliSecs()
  total_time = end_time - start_time
  DrawText "Total time : " + total_time,0,0
'  DrawText "Sprite     : " + sprite_drawn,0,10
  Flip
  FlushMem
Cls
Wend
End

Results : 520/500
Doing the same with B3D...

Graphics 640,480
SetBuffer BackBuffer()
;timer=CreateTimer(60)
man = LoadImage("man.png")
kilt = LoadImage("kilt.png")
While Not KeyHit(1)
  start_time=MilliSecs()
  For x = 10 To 600 Step 6
     For y = 10 To 480 Step 6 
      DrawImage man,x,y
;      DrawImage kilt,x,y+40
     Next 
  Next 
  For x = 10 To 600 Step 6
     For y = 10 To 480 Step 6 
;      DrawImage man,x,y
      DrawImage kilt,x,y+40
     Next 
  Next 
  End_time=MilliSecs()
  total_time = end_time - start_time
  Text 0,0,"Total time : " + total_time
  Flip
;  WaitTimer(timer)
  Cls
Wend
End

Results : 295/270
Obviously, the big downside is, in this situation, I don't get the same final display as the previous test.
I *MUST* be doing something wrong so please don't hesitate to let me know what it is.

Sorry Teamonkey, didn't see you reply.
This is how I, and probably 1000's of others, would code a game.
The 'cls' is the same in both products and, generally, I would keep all my drawing until just before the flip ; even more so if I am using backbuffer to draw/grab images earlier in the cycle.
I read a few articles talking about loadimage converting images to textured quads (or whatever) so they should be even quicker... shouldn't they?

<edit> I took cls out of both tests and similar results to before. I also added the 'cls' before the mainloop as there was a suggestion this 'readied' the card (???).
Drawing in 'groups' does seem quicker but doesn't produce the same screen output.

Hmm. So it looks as though what you're trying to do is:

a) Draw a basic tiled background.
b) Have some objects moving around over the background which, individually, need to be modified to show some kind of residue or burn mark or something

Yes Pixmaps are going to be slower to render than images unless you only have a software implementation of OpenGL on your computer, in which case they are much the same speed anyway. Hardware-wise, Images are a lot faster. So preferably you want to use images as much as you can.

Maybe you should think of video memory as kind of a fast cache of graphics data, like a CPU cache. You can move stuff to it that you want to use a lot, and it's fast, but because it's cached you forego being able to modify the original data. If you want to modify the original non-cached image, you have to do it in memory, you can't do it in the cache so well (at least not in 1.2 OpenGL?). Some people have mentioned that they added the ability to render to an image in cache (in video ram) in later versions, but at this time that isn't supported in BlitzMax from what I can tell.

So when it comes to modifying a cached image, you are more or less left with having to modify the version in main memory and then moving the updated copy to the cache again - ie into an image. And of course, drawing commands only work on the cached backbuffer memory so you can't really use them to do stuff to pixmaps. That's one of the only drawbacks I see to using OpenGL - that you can't easily modify the textures.

You say that you want to modify the image to show that it's been affected by certain things ... couldn't you approximate that by rendering several different versions of the image beforehand each showing a progressive amount of deterioration? Or does it need to be a more accurate representation?

Alternatively, if you are not rotating or scaling your sprite, you could perhaps do some rendering in the backbuffer or some other buffer and then use CopyPixels to render it - then you can draw on the image.

For me, any modifications to images are being done to a pixmap in ram and I guess I'll probably have to use some kind of custom low-level-as-possible (with pointers?) rendering routine to modify it, then move the modifications over to videoram when it's done.

Alternatively, you could render your background tiles to the backbuffer, then render your un-modified base objects, and then render the modifications in realtime to the images that have been drawn in the backbuffer - ie draw on the backbuffer. This of course means that you have to do the modifications every frame and they are non-permanent.

Ie..

1) Draw all the background tiles
2) Loop through each of the objects,
2a) Draw a base image
2b) Draw damage on top of it
3) Keep going with the next base image

Something like that? That way you wouldn't have to do anything with pixmaps, you can use fast images already in videoram to render the objects, and probably that alone is enough of a time saving compared to pixmaps that you can render the damage every frame and still be faster.

The Cls isn't the same in both products. In BlitzPlus/3D it will clear the buffer and return when it's done. In Max it will tell the graphics card to clear the buffer then return immediately without waiting for it to finish. You have to factor in that while your game code is executing, your graphics card might be doing its assigned list of tasks in the background.

If you have some game logic in between the Cls and your first drawing commands, like you say you normally would, then that's great because your game logic will execute while the screen is clearing. But that's not represented in these tests.

Decals will have to be accurate when I am creating composite sprites (i.e. the helmet needs to go on his head).
Pre-generating different set-ups of sprite in all anims with all directions will be a HUGE number.
Sorry to quote the manual at people but this is taken from the B3D Doc for Imagebuffer...

Now, for whatever reason, imagebuffers are not available in BMX. That means either a) there is another *simple* method to do the same or b) messing about with draw/grab etc.
Now, as BMX should be accessable to programmers of various skills (i.e. novices as well as pros), I would like to find option a.
On the other hand if it's option b) then the drawimage commands are a lot slower than B3D (from my tests which, I believe, is how MANY BMX buyers would code).

@teamonkey, I appreciate there's probably a difference.
However, the onus should not be put on the Game Creator (remember it won't necessarily be programmers who use bmx)
to add enough code to not get any slowdown.
BlitzMax should take responsibility for the 'cls' to function at a premium... surely?
Anyway, taking out the 'cls' shows no real speed-up/slow-down of both tests. Is this expected?

Clearscreens are pretty fast on a decent gfx card, it has almost no effect.

Also yes, it would be nice to be able to decide to what buffer an image is drawn, and be able to then render using that buffer, but at the moment it's a bit tricky because that's not how OpenGL works, at least not the version implemented. As someone mentioned, pbuffers were added later to do this.

Meanwhile, it's time to get creative to find a solution that works for you.

Anyway... you're saying that you want to be able to draw like different body parts to a buffer and then render that single buffer? Why not just draw all the body parts seperately in realtime? I still don't get that you really need to put it in a buffer first?

That's right **BUT** running the same programs on my 9800Pro PC (rather than S3 laptop) gives 20ms for BMX and 100 for B3D. Think most of the tests were irrelevant for drawimage timings.
Sorry about that.
With drawing being (a lot) quicker than I first thought then imagebuffers are NOT as missed as I first thought.
The idea is you have a single body with 'add-ons'. This might be a different type or colour of sword and/or armor and/or helmet. Each 'final' image might be made up of 5-6 seperate images. For an RPG/RTS there might be a large number of characters. If we had 100 characters, each made up of 5 'composites' they'd be 500 images drawn without tiles, backgrounds, additional decals, buildings etc.
Using image buffers (in b2D/B3D) you would draw each character plus composites to a seperate buffer at the start of the game.
The first 'game render' would then draw the tiles, buildings etc plus 100 images for the composite characters.
If characterA found bigger sword you would then redraw that particular image in it's imagebuffer with the new sword just the once. This means, the next time you draw the 'game render' you still only draw 100 characters.
Drawing, potentially, 5 times fewer images per cycle MUST make a difference.... surely.
Using the imagebuffer method (rather than drawing each composite) reduces the cycle time from 110ms to 50ms and that's for an image made from 2 composite images.
A composite image created from 4 images in B2D/B3D within an imagebuffer still displays at 50ms per cycle because the main render only draws 1 character.
Drawing the same 2 images each cycle within bmx takes 22ms
but increasing it to a 4 image composite takes 60ms to render.
I think the current answer is to use the backbuffer to create the composite images which will involve grabimage.
I hope all this makes sense and sorry again for the timing cock-ups.

Makes good sense, thanks for the clarity. So I guess you can still create your composite images at the start of the game and then when it comes time to change something, like, how often will that even happen? You can probably do at least a few changes-with-grabs per frame. You could stagger the changes accross several frames if there are a lot at once. Sometimes you just gotta test it out to see how it runs.

There are optimizations you might want to look into, for example. You could start to use a depth buffer in OpenGL. You could position your entities in the Z space in the correct drawing order and make your rendering affect the depth buffer - ie check the depth value for each pixel and if something is nearer it records the pixel of the closer image. And if you have an orthographic projection with no size change happening with further away objects, the system will end up only rendering the parts of images that don't overlap others. This cuts down on the amount that it needs to actual render to the backbuffer. I don't know any speed stats about using it but that is one thing you might want to consider. OpenGL can have it so that it only ends up rendering the pixels that are foremost, which with a lot of objects may help you considerably.

Ahh yes. Sorry, my bad. Cls isn't that slow at all. The problem was that I assumed that Flip was essentially just the same thing as bglSwapBuffers. It's not - there's also some timing code in there which, on my system at least, messes up the results. Watch out for that.

You can change the timing by saying glSetInterval 0 I think. By default it synchronizes to the vertical blank.