Compiler Bloat

Does anybody here think that Pure basic Produces "Bloated code"

Dunno... I've written ten commercial games in Blitz (more if you count the abandoned ones), and none have ever gone over about 1.6mb. You can calculate it approximately - 1.2Mb + however much code you have.

The problem (if it really can be called that), is that every Blitz3D command is contained in the same DLL, so there's no way of limiting whats included, like you can with BlitzMax. Therefore even a simple "Hello World!" will come out at over 1.2mb.

In these modern times of 512Mb+ machines, I don't consider this an issue to be losing sleep over.

Thats ok

You know that quote is going to come back to haunt you, don't you? XD

EDIT:

LOL, and so's that one. But a lot sooner XD

Isize 10.7 MB

I doubt that very much.

What was the original source code size in B3D?

adia 43 K at http://www.amiga-B3D

Yipe, pure basic needs a diet

You never answered my question. What was the original source code size in B3D?, NOT the executable.

564K

Meh. You couldn't write 10mb of sourcecode even if you bought Blitz3D on day 1 and coded non-stop ever since.

I'm curious to know what compiler your 'friend' used to compile Blitz3D code with...

what I mean is the ROM binary files where made into .bb files by a program I wrote (making it stand alone exe).

keep it coming

OK I'm WAY too drunk for this.

Nite.

OK I just totalled up the main source code (no data in here)

460 K of pure source code

Dunno how to account for the .exe size differences but here's a different way of looking at what your saying about speed differences:

10,956,800 bytes divided by 43,000 bytes equals a size difference of over 2,500 times. By your logic this would mean that the B3D equivalent is executing 2,500 times more instructions than the C version. Wouldn't this mean that the B3D version would run 2,500 times slower as a direct result of this overhead? Is that the case, or not? For sure B3D would be slower because it's BASIC, but I'm fairly certain it won't be 2,500 times slower at doing logic (i.e. For,Next,If,Then etc...) based stuff. Emulators tend to be logic intensive. So, does your emulator run 2,500 times slower under B3D?

That sounds ok but can you go into more detail

Blitz-3D runs way faster than needed to keep up with any commerical 3D game ,but that was not what I hoped to discuss;

This topic is about the number of bytes of pure machine code needed to translate say as an example: poke byte to memory instruction using constants (PokeByte memory,100,255)

Gav said:


Something does not add up in your calculations. 1.2 MB is (roughly) 1.2 millions of bytes and 1.6 MB is (again roughly) 1.6 millions of bytes. If we deduct one from another we'll get (yes, roughly) 400,000 bytes which at 3 bytes per instruction comes to (all together now) roughly 130,000 instructions. Where's that four million you are talking about?

Barney

460Kb of source code will in no way create a 10.7mb B3D executable.

If your end end end calculations are correct, then that does seem like a lot.

Are you absolutly totaly positive you were compiling in release (Even then I think 1.5K for an end is excessive)

But even worse, any excuse for a single end being 1.5k connot explain why four one after each other should be 6K. Unless there is no pass for optimisation

I also think blitz code (In fact code on any modern language), is bloated. But thats only because I started with 16K (Which was a lot). Which I imagine you did.

460K(main source) + 1.56Mb

So, 2/3rd's of the B3D version is taken up with data statements which are not present in the C version. It would have been nice if you could have mentioned this major difference earlier in the thread;)

I agree with your point about bloated individual instructions though. In particular, this

Even if End used 0.1K it's still way too much. Be nice to know what all the space is required for.

H&K

that was my first thought this is "evidence of no optimisation"

I will look very closely at this over the next few days and get back

Maybe the data is being stored as long floating point when it is in fact 0-255 fitting into a single byte ,standard BASIC of course has no statement to identify data type, this could be done by smart compiling.

Its hard to determine exactly

Upx is your friend.

Uh.... I don't care if a compiler is bloated, as it di the job it's OK....

I use Inno setup compiler ,makes very small single setup files

Gav is probably correct... about the size of the resultant B3D code being bigger...

... i think one BIG factor is being overlooked here.

all c code is not entirely independent... especially MS compiled code... it relies heavily on the various runtimes to accomplish whatever it needs to do...

taking this into account, i would think that any c or c++ code that produces a single binary of similar functionality, that Blitz3D implements, will be the same size, or similar...

try and run some c stuff without MSVCRTxxx.xxx... surely the sizes of those libs must also be factored in...

i would then have to conclude that this fact makes this whole thing about compiler bloat (whatever that actually is referring to) as being relatively... insignificant...

** note ** ** disclaimer **
the above response is not a product of any Blitz3D fanboy paranoia or associative mental disorders...
nor is it a prewritten template response, from a member of the Entity Guild...
my objectivity is a matter of public record as far as Blitz vs the World issues is concerned...

:)

--Mike

this is true, i recently looked at how much it took to store data entrys (in b3d), and if it is numeric ie. an integer, it is stored as 8 bytes, 4 bytes for the value and 4 more bytes for i'm not sure what. so if you store binary as numeric data ie. values only 0-255 then this is 8 times more bytes than necessary.

strings are better, they have one null byte separating each of them so i decided the smallest realistic size of storing binary as internal data was as fake hex-byte strings ie. two string chars, this is 3 bytes instead of 8 then, but has to be converted which may slow things down a bit.

ideally, if you have a lot of binary data it would be best to store it all in separate file(s) as actual binary.

Isnt it 4 bytes for the number, and four bytes pointing to where the first four bytes are?
(ie The number and a pointer?)

And if you only want 0-255, why not (as you sort of implied), just store them in string data?

And if you are realy worried about it, partition off a bit of memory, and accsess that.
Or make a 256 colour screen/picture (or just 2 colour), and just use that?
Or (Again I think you Implied), just xor 4 times.
Or if just Binary ROL and ROR, (But Imnot sure we have them)

The problem gav is talling about isnt how big the "Variables" are, but rather why some small commands make the program bigger. In particular why is

end
end
end
end

Bigger (by 4) 6K, than just End 1.5K

> Isnt it 4 bytes for the number, and four bytes pointing to where the first four bytes are?

i doubt that, the first 4 bytes appear to be a type id (integers are 1 and floats are 2) followed by 4 bytes for the value. data is stacked and then pointed to by labels.

also, i just thought that you could store binary data as 4-byte values and then specify the number of bytes to read, that would be better than hex-byte strings by one byte.

I disagree with this.

Sometimes you need to bloat the code to produce an instruction block that would otherwise be badly performant due to code alignment.

Also, certain compilers (though I can't speak for Blitz in this regard), will in-line function instructions or unroll loops to favor code that executes faster rather than worrying about executable size.

Fast code execution has far more to do with the CPU expense of instructions than their final executable size.

very well put RG...

--Mike

So has anyone put it to the ultimate test??? Make a simple primative game in B3D and compile it with ALL blitz compilers out there... Then we see if there is a real difference in Blitz compilers...

In C compilers I am sure... you only link what you need... I am assuming basically the same in BMax... but SOLELY as a discussion in Blitz3D.

"?

I disagree time to execute=number of instructions executed

http://en.wikipedia.org/wiki/Loop_unrolling

RZ

Testing Blitz on different machines will produce widely varying indeterminate results ,what you need to look at in order to benchmark any compiler is the actually assembler code generated by said compiler.

OK, maybe there could be a switch for turning off loop-unrolling, Although based on what I read BlitzMax doesnt Unroll, because the EXE doesnt get bigger in situations where the compiler could do loop-unrolling

OK then here is one you will have trouble with

I compiled two programs with Blitz-MAX

**program 1**

Framework BRL.System
a=6
b=2
c=1
d=3
e=5

**program 2**

Framework BRL.System
a=1

Both produced Identical Executables byte for byte except for couple of bytes in the PE header (the date/time stamp no doubt)

how come there is no difference in the executables???????????????????????????????????????????????????????????????????????

Purhaps the variables where never used so some optimization routine omitted them

after using the same framework command and compiling two executables
1.blank

2. for i = 1 to 10
b=b+1
next

and I found that the assember code was placed at location 30583 ($7778)

and around 120 bytes where changed or moved

We had the same thing with C when we were benchmarking it against B3D once... the code was set up to perform a number of mathematical operations something like ten million times, and see how long it took. It looked like C was phenomenally fast until we realised that, because the results of the calculations were never actually used for anything, the compiler was completely ignoring them and we were testing nothing!

bing-go

Perhaps, for the non UK Citizens out there someone could define "wing-ding prizle wizzle"?

i think it was more like American slang:
http://www.urbandictionary.com/define.php?term=wingding
http://www.urbandictionary.com/define.php?term=prizzle
http://www.urbandictionary.com/define.php?term=Wizzle

"I disagree time to execute=number of instructions executed "

Psst, your ignorance is showing.

Taking the maturity of most C compilers vs. that of blitz, that's probably not too surprising. I imagine this particular optimization might be a little more difficult to achieve under Blitz due to it's loosely typed nature. Though certainly not impossible if the compiler is keeping track of such logic bugs. This is really a case of the compiler compensating for sloppy programming rather than optimizing. Though I suppose one could argue that the compensation is an optimization itself.


In the course of my career, I've known a lot of people who know enough to be dangerous. The ones that actually get anywhere and become the experts are the ones that concede to the fact that there's always more to learn and there just might be someone out there that knows more than you. Of course, that's a religious position, regardless of how relevant it seems here.



Ok... here goes..

Code alignment is a processor related issue that affects the way data is retrieved to process.

When the processor reads data, it reads the block of instructions/data in multi-byte chunks. This differs from processer to processer and it's bus capacity.

If you store the "start" of your data at addresses that are not multiples of that chunk size, the processor must resolve the issue by performing multiple fetches across those borders and "glue" the instructions and/or data together to emulate the alignment it prefers.

Most modern processors can handle this emulation just fine. Though sometimes, the emulation will go out to lunch, and you'll end up with a bus error, and the program will abort, or crash, or cause terrible things to happen at the OS level. The undesired outcome is really dependant upon how the program is executed.

You can usually see the biggest impact when the placement of labels and loop points starts in the middle of an alignment point, or straddles two alignment segments.

I know this used to be an issue with the Blitz compiler back in the early days of Blitz2D. People would perform benchmark tests, and see performance increase if they added a "do-nothing" type of instruction just before the loop they were testing.

What was happening was that the "do-nothing" instruction was bumping the loop start to an address that the processor could fetch without doing that emulation. The result was performance in the magnitude of 10ms vs 1500ms.

At one point in time, I know this was an issue with the VB6 compiler as well. Though that might have been corrected in a service pack. Modern processors are typically fast enough now where this code can be written sloppily, and you can get away with it without noticable impact.

However, that doesn't excuse the fact that the emulation is not nearly as performant as aligning addresses the way the processor expects, or writing code that will destabilize into a bus error if the processor munges it up.


I guess we can agree to disagree. However, since I've already pointed out two issues that you hadn't even heard of, perhaps it wouldn't be a bad idea to re-evaluate your opinion.

Instructions carry various weights in terms of processing time, and as illustrated by the code alignment issue, they are not always apparent.

@Gav,

On the Motorola 68000, a "feature" of the CPU prevented a dataread on an odd address. If you attempted to jump to an odd address, or read data from an odd address, you would generate a bus exception and then it's game over. The later generation CPU's (68020, 030, 040, 060) fixed this probelm, but you had to keep it in mind.

This lack of an ability to read an odd address caused problems within data structures. Suppose you had a byte field. Storing it within a struct is no problem, but what of the next fied within the struct? If you put it right next to the byte, the rest of the struct would be out of alignment. The compiler would pack out the struct with extra bytes to ensure proper alignment.


Get an Intel spec book. It gives you precise timings of the various instructions.

Back in the old days, when writing a game loop, we would write out all the opcodes we intended to use, then work out the time it takes for the game loop to execute. Different combinations of instructions would take different times. On the Z80, for example, it had these wonderful instructions LDIR and LDDR. They were one-shot data movers (moving up to 64k of data with a single instruction!), and had a number of useful features. Under some circumstances, though, it was faster to manually move a few bytes with a regular loop (or regular move instructions) than with LDxR. There was the setup, as you needed the B/C, D/E and H/L registers, and the storage of information contained within those registers, so this had to be factored in.

Since LDxR was a atomic instuction, it had the interesting problem that interrupts were essentially disabled during execution. No problem for small data moves, a major issue for big moves. You were better off breaking the moves up into smaller moves, after first working out the timing.

A lot of this isn't as important today as it once was, but with the rise of pixel shaders and other fun things, taking into account the speed of execution is becoming important again.

Neil