Mark's new worklog

http://www.blitzbasic.com/logs/userlog.php?user=1&log=1043


QUOTE:
" Although max3d is C++, the API will be procedural. This makes it easier to use with a greater range of languages, and also allows me a 'barrier' of sorts between interface and implementation."

" There will indeed be a new native file format, very probably b3d like (ie: nested chunk/len blocks) but not b3d compatible."

wOOt!

It seems we are going the Blitz3D 2.0 way :)

Three worklogs in a few days! Amazing!!

regarding the file format, would you consider having your file format with a binary version for speed and ascii version for ease of support?

yay, more good neuze!

-"Blitz3D v2.0"
Dont you mean "BlitzMax3D"? (or whatever they end up calling it, mabe it will be called "b3d2", whatever, but perhaps not quite what you appear to think it means.)

This appears to look like MAX will the next 'blitz3D language' in conjunction with a universal b3d-like procedural API. Which is what most have wanted all along (and that also means all the anti-oop lobby dont need to worry either).

I hope there will still be support for old .b3d files though as well as the new one, would be a shame if there wasnt.

I wonder if physics stuff could be switched off. There are many games where real life physics is not needed.



Excellent! I have BMax but I would prefer to use the new 3D lib with other languages where procedural API is preferred solution.



As long as there's a conversion tool this should not be a problem.

Barney

Mark: If you are reading this, keep up the worklog updates! Little nuggets like these are great.

mark if you are reading this, dont keep up the worklog updates! (maybe that will push him to update more offen, we should tell him not to update, then we'll have update every week)

Yep, a conversion tool would have to be a must. Or an exporter for UU :o)

About the 3D file format, he didn't give a word about converters yet... :D

I don't get the appeal of convertors. You can't "convert" a full character rig with some command line convertor tool. You need native exporters for all the main 3D packages if you're going to insist on your own 3D file formats. As I recall, the B3D format had a milkshape exporter which didn't export all the necessary data, and that was about it apart from a list of the specs, go do it yourself. Everything else was up to users. In fact, even now, I think 3dsMax is the only package with a fully working exporter, although good user efforts have been made in Anm8r and Blender recently.

Ok, an exporter for Ultimate Unwrap would work for 3dsMax because the Ultimate Unwrap developer actually provides his own exporter for that, and I guess if someone's paid for 3dsMax, they're not likely to object to paying for UU, but if it's going to be taken seriously, this 3D engine needs real exporters this time around.

Exactly why I asked there be a Ascii version of the file format Grabiel, it makes community programmed exporters much easyer.

What would be lovely would be a binary format (like now) an ascii format and a command line 'compiler' to compile the ascii format into the faster loading binary one.

It would then be a lot easyer for people (even I could do it) to program xsi and maya exporters relativly easy in ascii format, then at the end of the mel/vbscipt script put the file through the compiler and you would have yourself a nice fast loading binary.

Actually, to have an ascii format would enable me to build my own models much easier ;)

oooooh please!!!!! give me a reason to write a new game!! pleeassee

It's hard to call Milkshape 3D a full fledged modeller. It wasn't from the beginning. People who don't understand why you can't compare Milkshape 3D with other modelling tools will find very difficult to understand the needs for a good conversion tool. It was also explained why it isn't needed to write a plethora of plugins, one for each modeller (fbx, plus one or two optional major ones are enough). Ultimate Unwrap isn't a good solution (it is a dedicated unwrapped with limited support for animation and other key features: you don't want to buy a product for limited exporting if you have a professional modeller at hand). If you have to do for yourself even mesh importing, you could as well switch to Visual Studio. Rapid development tools should at the very least cover such needs.

The problem is that most developers in the development community don't even know what the advanced features of modelling tools are or how to use them - you should be both expert in graphics and coding - so it isn't very realistic that there will be advanced exporters made by the community (and it is why it didn't happen with B3D); it is also a very complex topic. I sincerely hope that Mark Sibly understands that without proper converters he is cutting off a great share of the commercial development market.

I think that expert artists such as Mustang could agree with me on the subject, and maybe share some views.

Instead of repeating anything from the beginning, I suggest to read the discussion in this topic.

This should be interesting.

What Gabriel said. Also the 3D format needs to be in XML (ala. Collada), not some binary IFF format. In which case writing a "converter" could be as easy as making an XSLT from Collada to the custom format. This would go a long way towards addressing the exporter issue.

A base XML "definition" file format and then a BRL supplied compiler to convert that to an optimised protected binary .m3d format seems like a sensible mix to me...(although there is some argument as to whether the binary version is really needed).

Models are too complex to efficiently store as text. There isn't an argument here when it comes to game development, it's simply not realistic to store models as text for anything but conversion from one format to another.

It would go way longer if Mark uses his knowledge to write at least one (fbx) or more (max, lws, ma, xsi) fully functional official exporters too. I agree that a ascii file format is helpful for writing exporters for user created content though.

I agreed with Doiron...akso for me and I guess many other 3dsmax users, new next gen B3D will be useless without exporter for 3dsmax or one of the formats max support at least..

Hi,

File-format wise, my main priority here is to come up with a new fast, compact native format.

The design of this will be dictated more by Max3d object layout than any grand scheme, kind of a 'quasi-reflection' style format.

As for converters/loaders, there are a few things that I plan on doing that I think will help here:

* A SaveEntity function.

* Full access to object internals via 'Get' style commands.

This means writing an exporter would look like...

1) Create entity from source format data.
2) Call SaveEntity.

...and an importer...

1) Call LoadEntity.
2) Write entity data in dest format.

The final product will likely include b3d, 3ds etc loaders - ie: what's in b3d right now - but as for fbx, collada and other similar 'mega formats', I dunno. They appear to be pretty complex and I could easily see implementing them 'properly' chewing up a ton of time. So for now at least, my plan is to 'leave them up to the community'! Given that Max3d will be more of an SDK thing anyway, it should be easier to write importers/exporters for a native Max3d format anyway as you'll be able to call LoadEntity/SaveEntity from within the importer/exporter itself rather than having to manually code them in as you currently have to.

If you see what I mean...

Does anyone know if this will be cross-platform? I don't know about you guys, but the real thing I'm disappointed with in Blitz3D is the Windows-only aspect. Although, granted, I don't do many technical things with Blitz, it's the only thing that regularly springs to mind as a downside. Otherwise, one way or another, Blitz3D seems to do pretty much whatever I want (after much fiddling and complaining on the forums), so I don't know how important all of these new features are.

Cross-platform, anyone?

This answer is truly a shock to me.
It takes time because it is a very important subject.

Formats like 3DS are from an old generation which doesn't suppost almost anything but mesh and UV setting, which is totally unacceptable today. Current modelling format are max (3DS Max), lwo/lws (Newtek Lightwave), ma (maya) and XSI to name a few. FBX is just a way to save time instead of working on more formats and make everyone happy regardless of the pipeline.

There isn't a single next gen engine which doesn't have support for *at least* one contemporary format, and the reason is obvious. No game developer will even consider a next gen engine if it has no support at all for existing modellers. It's the first thing you look as a requisite, bells and whistles comes after (it also means that it widens your userbase to new heights).

And the problem is that if you don't provide support, no one will do it properly (as it already happened with B3D, creating tremendous headaches with everyone which was looking for a working conversion pipeline). It's almost irresponsible to leave such a burden on the community, since it is like saying that you don't care about such a fundamental aspect.

Community exporters are always been half-baked solutions, missing important features or which didn't work properly at all (there has been only one exception in more than five years).

Most of the work today is done on modellers, which can even became game editors when used to full extent. Without the ability to proper importing at least materials and animations you are left with a powerful engine and no content, thus making the engine almost useless.

It is like having a powerful video editing software without a camera for shooting.

In conclusion, take your time but do something about it, I beg you. :)

I can see how it is difficult to support so many different formats when they're all vying to be proprietary. Application X has its own format, application Y wants its own, then Z wants it's own as well. And then along comes Max3D and it has to support these other guys because they all think and play differently. If there is going to be support for some external non-blitz format then it should be either the most widely used or some kind of `open` format which is accessible to a number of applications.

Or maybe BRL can team up with someone who makes model format conversion software and have them make like a plugin or something to support Max3D files. Then it can act as middleware for file format conversion rather than expecting Max3D to specialize in it. I think I would be daunted too if I were, let's say, making a graphics app and had to write file loaders and savers for 20 different image and movie formats.

I like Mark's approach to making it fairly easy to write a converter within BlitzMax by using in-build compatibility commands like those described. Maybe it would be a good idea if necessary to hire someone to spend time creating converters for a few major or open formats, or formats that people can use more easily like ascii or xml or whatever.

But in addition to all that, Max3D still also has to have its own file format, in order to support everything Mark wants to do, and so that Max3D can have its OWN presence in the marketplace with its own file format. Having a file format sort of says that you think you're big enough. And if you actually are big enough then other people start to support your format. It doesn't have to all be one-way.

Hi,


Well, currently I don't. The production of a cool 3D engine is priority one for me at the moment. It certainly makes sense to do that *before* an importer! And then, once the engine is done, should I just sit on it while I grind out some 'mega format' importer on my own?


I think you do the community a disservice here - the 3ds pipeline thing is awesome, and I think better than I could ever have done on my own. No, it's not perfect, but either are *any* importers I've ever seen.

I do think the community will have an important role to play in producing importers, and I think my time (for now at least) is far better spent making that easier to do than trying to do *everything* myself.


It's not even really that - it's more just a case of needing *some* way to get stuff from disk to memory (and back again with SaveEntity) as efficiently as possible!

[edit]
Max3d will also support some form of 'texture name munging' to allow you to provide specular, emissive, normal etc maps for stuff loaded via b3d, 3ds etc. This works by looking for similar texture names with special extensionettes (a new word!) eg: if the primary texture name is 'wall_d.jpg', the system will also look for 'wall_s.jpg', 'wall_e.jpg' etc for specular, emissive etc maps. No, not particularly pretty, but very easy for me to do and very convenient for users.
[/edit]

there are b3d exporters for fragmotion and UU3D
so maybe when the new format is released the same people will write the new exporters for B3d2.0

When I wrote "only one exception" I expressely meant the Max pipeline of course, which I already mentioned in the past. It is exceptionally well done and supports almost everything you could need. However, it is the only one which is as good and not everyone uses 3DS Max.

What I am worried about is that, as you suggested, the matter won't be addressed officially as it already was the case with B3D, and you confirmed my worries. I agree that implementing the import of animation/materials/lights comes near the end of the development, but is always taken for granted and I find very unusual to disregard it since it is a must for a next-gen engine which must be addressed one way or the other, even if by hiring a developer for writing a library. Basically that's why I was being so repetitive about it lately... it's just too important of an issue to let it slip another time.

Being a time intensive task but essential nonetheless, maybe you could have someone working on it while you concentrate on more pressing matters.


One thing that I do not understand is how a 3D engine can be/look cool without 3D media inside of it (or just static 3D models). Or at least it looks cool until you have to do something artistically serious with it. Of course the help of the community is important (and I am willing to help anyway I can), but it is an aspect which must be addressed, since as I said earlier you can't do much with a 3D engine if you can't have properly animated meshes and well modelled scenes with various maps applied in it.

Anyway, interchangeable formats such as FBX and Collada can be overly complex, but not all currently used formats are. Also, though the FBX format is not simple you could disregard all the information you don't need to import into Max3D's native format.

Finally, judging from the aggressive answer, I'm sorry if I sounded arrogant or sarcastic. With the B3D format you already reached an high standard, and my only remark is that it needed so little to be fully exploited (as explained earlier) that I became frustrated while getting around it, and you can be sure I'm not alone here. I sincerely think that any work on this aspect would make the product far more profitable for everyone (it would also attract a lot of game developers and artists looking for rad tools).

Hi,


You didn't - and sorry if I sounded aggressive, I was just trying to explain where things are at.

Did you see my edit re: texture name munging? This alone will allow people to soup up 'old format' models dramatically, to the level of most modern games. I think this'll provide a pretty good 'bridging' technique until gruntier converters/importers evolve.

I'n not sure how to think about the procedural API as how does this look like compared to an object oriented one? Having your own proprietary fileformat is nice as long as you also support a widely spread fileformat (like fbx) but without that it's likely that it will be a pain to be used with existing modeller software and the workflow is broken at this point. One thing regarding physics that i would love to see would be softbodies but it sounds that it's more about rigid ones only. Anyway thanks for the update!

"The extensionettes". Name of my next band! For sure! :-)

I have worked with and around 3D cad file format issues for the past many years and I totally support Marks take one this. Belive you me: cad formats come and go and so will Collada and FBX.

As long as we get methods to get and set all object internals, there are several good takes on converting permanently and at loading time.

I've been doing these conversions with b3d and XSLT for years, and it does not need to be hard.

Look at this little Snipplettina(TM) for inspiration: http://blitzmax.com/codearcs/codearcs.php?code=665

Re: texture name munging...

A more elegant solution would be to store the extra texture names in a simple text file format and check that instead, then the text file could be named the same as the 3d file to associate it but with say a .tex extension ie. mytest.b3d and mytest.tex, it would also leave it easier to control for the user.

Regarding the involvement of the community in expanding/creating art pipeline tools. I think its a really solid idea.

Typically artists with a programming bent make the best importing/exporting tools, programmers make 'programmer art' capable tools.

Secondly other projects like wxMax prove that the community (in this case one hyper-commited member) take on projects that someone like Mark decided was 'too large to wrap' when he was looking for UI solutions. Remember Mark/BRL is one person and his labour output is finite.

Ditto the community ads the spit and polish to other BRL creations (such as MaxGUI)

Marks brain has always been best at language design and graphic engines. Any tools that BRL have created have always been minimilist and in most cases outdone by someone in the community who feels more passionate about that particular part of the process.

Note: I do not mean the above offensively just objectively.

By Mark making hooks that allow 3rd parties easier development for art pipeline tools you get.

a) Him bringing the part of the product that only he can do, to market quicker (which is what everyone seems to want)
b) a viable and vibrant commercial outlet for art pipeline tools.
c) Mark spending time doing what he loves namely languages and engines. You always do the best job of what you enjoy. e.g. Blitzmax the language vs Blitzmax - the docs.

Surely if you look at the history of the products this is the best outcome for all, and would seem to me that Mark has rationalised where his skills and passions are best spent. And been very upfront and honest about it, which I applaud.

Now.... if only someone in the community was passionate about documentation :)

I'm assuming that the procedural API is more for the benefit of providing a usable public interface in other languages, rather than the final interface. It could become a hassle to keep track of the many different global functions, especially since OOP provides some organization in a way, but I suppose what you could hope for is that the BMax wrapper of the engine, and perhaps the C++ interface itself, will have their own wrappers around the procedural API.

I'm not enthusiastic about the way this is being handled, but then I really don't know that much about how it's being done in the first place. I'd say it's best to wait and see in this case. It may be that coding will simply be better with the straight procedural interface if B3D's is anything to go by (which was completely procedural and still very easy to use). That's not to say I'd go back to using that sort of interface- Truevision 3D spoiled me quite a bit.

(Not talking to taumel now)
On the topic of formats/tools, I'm going with Mark on this one, although perhaps for slightly different reasons. The problem is that no one tool is going to meet your needs. This is an unfortunate part of game development and I think some of you haven't quite realized this yet because you're too busy complaining about the decisions Mark makes. Rather than try to make the be-all-end-all tool and end up with some gigantic, horrible behemoth (FBX and Collada are great examples of standardized formats backfiring so hard that they splattered everyone with blood and bits of poo) that never really does what you want. Write your own tools, come up with what you want in the tool, and stop complaining about how you won't get the wonderful tools you're imagining (because you won't, and that's not going to change unless you do it yourself). If you really can't handle that, do you think you're in any position to be making games?

Hi Mark,

I agree, I think all most would need to start with is the new engine and new b3d format. (a converter/impoort for the old one so at least we have some media to play with would be very useful too). A lot of the import/exporters we have all collected (-that are in my sticky thread in the b3d programming forum) could be polished, improved, updated and re-engineered using the new max code, saving you a ton of work. People will write additional ones quickly enough.

See for yourself:
http://www.blitzbasic.com/Manuals/_index_.php
or
http://irrlicht.sourceforge.net/docu/namespaces.html
The :: oop :: api :: while :: powerful :: can :: be :: a :: little :: bit-> longwinded().sometimes = true;

Both are good depending on your needs & experience, but which is 'nicer' and easier to use & learn?
Personally i am glad that MAX3D will be procedural as my programming habbits are more inclined that way.

I would have been happy either way. I just want some native 3d in max

I think it depends on how it's done in detail. If you name your objects and the methods properly then i would't say that oo is a lot worse and one advantage you have is that you always know were you're belonging to but i'm open to both.

As for file formats, if you have work to do then all this theoretical talk is kind of senseless as you simply just need a working format with which different people can exchange their work. Either you have defined pipelines or you work with a common format which does the job best for you.

I agree totally, however i only really want to worry about my own objects and methods, not have to dive into other namespaces and classes for interfacing with the engine itself. That way if i want my 3d world to be oop based it can still be, but completely on "my side of the fence" so to speak.

But as i said above, its either-or as far as im concerned. I would adapt either way.

Consideration about 'extensionettes'.
Suppose I have 2 color maps (Ie: color1.png, color3.png) but I want the SAME specular map (spec1.png)
With the extensionette concept the loader will check if it exists a file called color1_s.png and color2_s.png - failing.
If it seems to be 'logic' (and faster) I dont' believe this system is very 'elastic'. Even if it is possibile to load (after) a proper spec map this mean a lost of time.
And the solution to provide 2 files (color1_s and color2_2 ) that are THE SAME image is not very good (from memory saving point of view).
At this point it will be more logic to have a 'material file' that points to the images I want to load.

Just a simple consideration.

That helps a lot. However, keep in mind that even B3D except for the max pipeline lacks good importing tools. Why don't you have a chat with the guys behind the Max pipeline or Brucey (which other than being a great coder lately talked about seeing a fbx converter in action for BMax) and see if you can come up with some interesting ideas or arrangements?

I agree with Markcw and Degac on the indexing file for textures: it avoids texture duplicates, and since we are talking about big textures, it is a valid observation.
[edit: an index file would also permit to apply shaders in a easier way.]

About the original format, to be up to date a new one would just need more supported maps (dot3, specular, subsurface scattering, glow), and animation blending (merging of two or more animations).


Just out of curiosity, do you plan on supporting geometry instancing? that would be great!



Nonsense. There are working pipelines in the cg industry which fit almost all needs. You can't have a whole industry working without proper tools.

@Peter Scheutz: The FBX format has been used for a lot of years in the whole CG industry and I don't see why support should stop since it is the only interchangeable format which is widely supported and regularly updated by Autodesk and support is kept up to date from all major modelling authors. And I didn't see many formats from leading modellers come and go: every modeller uses the same format which was using years ago, albeit updated.

Since Mark's latest worklogs I now think that BMax3D is going to kick ass because of his experience in C++ and 3d engine architecture. The main thing to me is that BMax3D will be cross-platform, support for industry-standard 3d file formats comes a long way down the list for me, it's something you can worry about later.

You need the best standard industry formats to get good media into your engine. This is especially important with such a niche engine where tool support for high end scenegraph, mesh, animation and material features arent likely to be developed for specific tools.

Just look at engines like Irrlicht, or even B3D. B3d was a couple of years old when I first looked at it. I almost quit one week into aerial antics because the art pipeline was rubbish. Not even vertex color support in any tools let alone extra UV's and lightmaps (even though the file format supported them). It was only a personal plea to Brad from Ultimate unwrap and his hard work over the next couple of weeks (supporting B3D 100% withina month) that made me stick around.

I think a ascii format with a binary compiler like Ogre has would good. That would allow the format to grow with the engine rather than having to hack around a early developed format that doesn't support the engines later features 100%.

Ogre's format is always being tweaked, updated and optimized. Tools like Unwrap3D support the ascii format so you can use the binary compiler to convert to the latest version.

Gives you the option to support ascii or binary output from your tools. Oh theres a binary updater too, incase you have outdated binary media. Things change a lot in 3D graphics these days so its good to be flexible.

Collada already has Ascii and binary versions, and many tools support it. Also has its own collada .fx format for moving shaders around which is supported by FX composer 2 and mental mill which comes with FXc2.

That's what I am trying to explain. Most people here don't understand the importance of a 3D pipeline because never even reached this stage of development (there aren't many 3D artists around here, and nearly all the community is formed by lonewolf coders). This way you miss the biggest share of the development community, being almost everything graphically driven these days.

Even the idea of creating a game editor is mostly covered by 3d modelling packages today (in Burnout 3 the developers came to the point of using Lightwave for AI and traffic simulations directly in the layout editor). Mark is right, he doesn't have to do everything on his own... that's why he should support other existing tools for the job though! Most of the needs today are covered by existing tools, and I see Max3D as a glue for them. In the past I also wondered why going the world editor route with Maplet when modellers were taking place of these limited editors at the time (and now completely replaced them): with good exporters he could have said "get one of these modellers: here there are the instructions for importing media into B3D" getting the best out of it and truly allowing to add content to the engine.

Mark Sibly: I don't know if you are already confident on the subject, but if not you should take a grasp of the current possibilities offered by todays scene editors/modellers in order to see how further you could go toward the game editor goal with lesser headscratching and without having to write it for yourself, just by integrating existing systems.

To make an art pipeline easy, why not create one big importer/exporter tool like Autodesk have for fbx? You can update the format as Evak suggests as required, as well as what formats the converter app can deal with so you're not having to write a ton of plugins for various apps. People can export out of their app in fbx, x, obj... whatever they like, and then batch process the conversion into the 'new' format. If there are any issues, the tool can give feedback on that, as opposed to whatever limited feedback there might be from a specific art tool.

That's my awesum idea, anyway ;)

Ooo! That's me! Me! I know about pipelines!

(sorry, just basking in the glory for a moment there)

If you support collada there are existing libraries Fcollada and ColladaDOM open source C++ import and export libraries.

Fcollada supports the following:

http://www.feelingsoftware.com/content/view/74/91/lang,en/

Features  			FCollada
			 			ColladaMax
			 			ColladaMaya
			 			ColladaMotionBuilder
			 						 
Import COLLADA 1.4.1
Export COLLADA 1.4.1
 
Geometry           
  
Mesh
    Polygon 
    Polygon with holes
    Polygon list 
    Triangle list
    Triangle fans
    Triangle strips
    Normals
    Multiple color sets
    Multiple texture coord sets
    Multiple tangent sets
    Extra data sets
 
   Splines
   Linear interpolation
   Cubic Bezier interpolation
   NURBS interpolation
   Trimmed NURBS surfaces 
  
 Animation           
Skinning and morphing           
  
      Skinned meshes 
      Morphed meshes
     (aka blend shapes)
      Non-tesselated morph targets
      Skinned + morphed
  
General animation
           
     Animated rotate,
     translate, scale 
     Bezier interpolation
     Linear / step interpolation 
     TCB interpolation
 
 COLLADA Physics
  Import
  Export
  
  Simulation
  
   Rigid body
   Analytical shapes
   Rigid constraints
   Breakable constraints 
             
 COMMON materials
           
  Constant, Lambert, Blinn,
  Phong  
  Animated colors and coef.
  
  Environment mapping
           
    Spherical
    Cubic  
    Video Textures
    AVI (MS Windows)
    Image Sequences 
             
COLLADA FX
           
  Supported profiles
           
    NVIDIA Cg and CgFX
  
     Real-time shader edition
     Real-time parameter animation
   Import
   Export
   Real-time display
             
Asset Standardization
           
  Triangulate on export
   Up axis
   Units
   Negative scale removal
  
            
Other features
           
  File referencing 
   ID preservation
   <extra> preservation

            
Supported platforms            
  Windows XP, Vista
  Mac OS X
  Linux (Redhat, FC, Ubuntu)
  Playstation 3
 
Integration           
  ActiveX (IE)
  Firefox for Windows
  Firefox for Mac OS X
  COM (MS Office)  

Notes:

Supports AGEIA PhysX engine. Bullet engine support is also almost complete.

Collada DOM is open source on sourceforge. currently at v2.1

Wiki at: http://collada.org/mediawiki/index.php/Portal:COLLADA_DOM

I like the idea of having a "SaveEntity" command.

This way you can write your own converter or add additional features to your model easily(like setting additonal materials).

One of the problems of the current B3D ist, that you just can't get/set all of the parameters in code(like bones and texture flags).
SaveEntity is only useful if each and every aspect of the format can be accessed within the program. When building up a mesh in my program, I just want to call SaveEntity and all textures, blend modes, shader settings and maybe physic stuff are saved into the model.


Wasting time on exporters made by Mark doesn't make any sense.
The main problem of creating an exporter is not the file format (like ascii or XML or grouped in chunks or ...).
It's most about mapping the functionality of your modeller into functionality of that file format.
Therefore the programmer of an exporter needs to be an 'expert' in both, B3D AND the modelling tool.

Blitz Research should hire such experts for some modelling tools to create good exporter plugins, when the format is ready.
Until then converting via SaveEntity should do pretty well.

Welcome back Mark. ;-)

And welcome back me - just re-bought BlitzMax.

Supporting more than one format is a waste of time. For starters it means you need to have multiple points of truth in your engine, as not all formats behave in the same way. Practically every application that you care to use has some pretty decently supported Collada Exporter, and as mentioned earlier since Collada is XML, writing and maintaining a "converter" for it is a trivial matter (using XSLT) and providing it doesn't have to work with Operating Systems older than Windows XP, all target platforms have an XSLT processor built-in.

But that's the beauty of it all. You don't have to "implement it properly". All you have to do is provide an XML schema for your format, then anyone with a rudimentary understanding of XSLs could write and/or maintain a converter for it (or if they have biztalk or similar they can simply DRAW the converter). Even better, since Collada is a "gateway" format, you don't have to have a whole bunch of different tools to convert it to your own format, all of which have slightly different issues.

I don't think you understand the design goal of Collada. It was not designed to be a format as such. It was designed to retain as much information as possible about the model, working around the typical problem with art pipelines that data is sometimes "Lost in translation".

Writing a fully fledged 3D modeler really shouldn't be a necessary prerequisite to making a game - particularly if you already have a modeler that does what you want. Even worse if you already have an idea of what features you want to use in the modeler, and which you want to use in the engine, with a format like Collada, you don't need to write a tool. You just need to write an XSLT that takes the data from Collada and provides the subset of the data you want, in a format that your engine understands. It can be fully automated, and included as a step in your build procedure.

I think a fully featured SaveEntity command will go masses of a way to making convertors easier to write!

When I did the Lightwave and Prior to that (Truespace) convertors, I barely understood how the 3D info in them worked (compared to now).

But I had the job of reading them in correctly, munching it in memory, then convert that into .B3D and writing it out correctly. In fact for both the convertors, I read it in, wrote it out, then loaded the .B3D back in to get the preview. Just having to interpret the file properly then have all the saving done would make the whole process easier.

Also, the Model making packages all had so many features (surfacing options in LW are scary to behold, let along approximate in Blitz) that they just couldn't be squeezed into Blitz3D, so what usually happened, was that something you did in say, Lightwave would not move or look as it did in Layout, because you'd used a Null based IK Rig or something outlandish to set it up. It's not just the file format that's an issue, but the type of information stored in it.

Collada and filmbox are file interchange formats. You shouldn't use them directly in your engine. Flameduck explained it pretty well.

I didn't say you should. I said they're bad standardized formats.

If Collada was bad standardized format, why other 3d engines support?
1. OGRE:
http://www.ogre3d.org/wiki/index.php/Ogre_COLLADA_Plugin

2. irrlicht:
http://www.irrlicht3d.org/archives/archive_2006-m08.php


3. C4:
http://www.terathon.com/wiki/index.php?title=Collada_Plugins
(This list shows many 3d softwares could import/export Collada format)

Edit:
IMO, Mark could just concentrate on new B3d native format and allow some basic Collada conversion that could be modified by other users or as a plug-in.
With this, the current models could immediately import into the new 3d engine for playing once the engine release.

Wow, you can just draw something in biztalk to convert quaternion rotated planar mapping to per vertex UVs? Convert left hand coordinate NURBS to right hand coordinate triangle meshes by doodling? Why on earth have I been writing this 'code' stuff for the last 6 years?!

First, because it's a standardized format. Second, of the two example engines you've listed, only OGRE is any good. Third: there are examples of bad standards everywhere, try looking.

Next time, pick good engines.

Personally I think there's a compelling argument for someone with the appropriate set of skills and knowledge to develop a tool that's entirely devoted to converting 3D models between different formats. I think there would be a decent market for this type of tool, and there's no real competition for it that I'm aware of (although I could be wrong).

Polytrans and Deep Exploration are two examples of solid conversion programs. Unwrap3D often serves as a cheapie converter tool at times for many people as well, although its support for different formats is quite limited.

Sure. But using a $500 conversion tool, to get your models into a $100 engine like Blitz3D, seems like a moot point under even the best of circumstances.

Ultimate Unwrap3D??

As I said, the conversion tool should ideally be exclusively devoted to the purpose of converting 3D models between formats, without getting side-tracked by other uses.

And it should be sanely priced for the indie and hobbyist market, as FlameDuck indicated.

---

While the tool developer would probably provide most of the converter plugins, the tool should also accept user provided plugins.

Why create a tool to go between formats? Most modern 3D packages can import and export the common formats without a hassle, like obj, x and fbx. That's usually more than enough. Even $500 tools have restrictions on what can be converted to what.

I still think a tool should be created for the sole purpose of converting formats to a new blitz format. After all, if conversion is linked directly to the Blitz engine, that should be the main purpose of the tool. Perhaps even, it could be designed so people could easily write plugins for it, so the main tool itself wouldn't need to be updated all the time.

Expecially considered they're not good when you go past static meshes with UV's (Polytrans is better than Deep Exploration in that regard though, but it isn't as near as good as exporting to FBX and then importing back in another modeller for example).

That tool would have a relatively small market (the Blitz community), which would make it a fairly unatractive proposition for the potential userbase, and as such; for any commercial tool developer who was considering developing such a tool. If the tool was already going to be importing from a variety of formats, then t's (possibly) a relatively small step to write exporters for those formats also, and end up with a more widely usefull and marketable tool.

I was actually talking about it being free... built by Blitz Research. Isn't creating a pipeline for a new blitz format part of what this thread is about?

As far as building a model converter- it's best not to talk about it, but simply go and make it. It might not be as easy to create and maintain as you think it would be, but there's a market for it, for sure.

Nothing more to say than.... Keep your excellent job Mark, B3D V2 sound like a dream come true. =)

1) A format like FBX is a format which most modellers understand.
2) Collada a.o. issues is supported by less modellers.
3) A propritary format is nice to have for internal or basic stuff but nothing reasonable for most modellers and more complex stuff unless you're hoping for support by one of those scary sub $50 tools.
4) Conversion tools are an extra step, not available on all platforms, cost extra $, not always work the way you think they should and so are just another annoying error source.

=> 1 or 1+3 sounds like a good solution.

FBX is ok if you have a program with a good export as i said above in cinema4d the fbx export is rubbish.

I would get the people that make ultimate unwrap 3d to make a import export plugin. then for a very good price you could export and import from just about any model format. although for me this still doesnt include cinema4d but there software support FBX,Collada, maya,3ds and lots more. its also a handy program for tweaking UVs.

Plus alot of people her already own it.

Pete

As far as i know C4D is the only modeller out of the more reasonable ones which has problems with fbx but that's Maxxon's fault. Actually C4D not only has problems with this. Would be nice if their userbase would nag them hard enough in order to get proper support included. A format i found very good for exchanging static meshes is wavefront's obj.

I don't know but if you want to tweak UVs why don't you set them up properly in your modeller?

To me it seems that saveentity would also make it easier to make mesh editors and such like. I do think that making a command that saves in multiple formats a waste of time, just make it so it saves to .b3d this way we can still covert it over using other apps. Other save functions could be 'introduced' at a later date.

With a b3d only file format and no support for a fileformat known by modellers the usage of the engine will be pretty much limited.

I have been through this some times already. Anyone here who has to deal with Director's w3d for instance?!

Proper fileformat support is extremely important for a todays 3d engine due to that there are so many apps out there and a proper workflow when working with different apps/people. One thing users love about Unity is their support for quite some modeller fileformats without the need of converting your data or the need to pray for a hopefully working importer/exporter which maybe a user has written and then after some time looses interest in.

Take a look at what Unity supports:

Taumel,

I entirely agree with you - and Unity is sweeeet! But I think Marks point is let him get the engine ready first, then he will sort out the file formats. Rather like he did with B3d in the first place. I quite like the .b3d format - it was a GODSEND when it finally arrived, 3ds is aweful.

Keep up the fantastic work Mark - As Steve said above - "Welcome back!" - lol
Regards,

IPete2.

Does Unwrap3D support storing tangents, binormal and edgelist data amongst other things?

Don't get me wrong, I think Unwrap is great tool for basic engines but modern engines can benefit if they support custom features not found in generic older formats including B3D.

FBX is an autodesk format and only supported 100% in their products. There is a free SDK though. The reason I prefer collada is that artist friendly shader tools support it out of the box. The node based editor Mental Mill through FX composer (Free from Nvidia). And shaderFX for 3dsmax $250. Both the editors work well with FX composer 2. Both filmbox and Collada work with shaders, but only expensive tools seem to support FBX properly.

Heres a list of tools that support Collada. The first ones in bold are what indie devs commonly use for games.

3dsmax
maya
blender
xsi
milkshape 3D
ultimate unwrap
newton
bullet
ageia Physx
fxcomposer
sketchup
Render monkey
modo
DeleD

Full list

Why not having this 3d sourcefile format similar to povray? Where you could put loops and a program structure for building meshes? Povray's scripting is incredibly flexible, so much that it can create scenes that are absolutely lifelike.

http://en.wikipedia.org/wiki/POV-Ray

Of course povray's scripts are for complete scenes which include cameras and lighting and so forth. I dunno how useful it would be to have a file format that would include whole scenes. But my suggestion would be to at least be able to include some logical operator handling and loops as well as working variables. Also, possibly add tables and table lookup functions.

These are just suggestions, but I think this would really open up the format.

Why on earth would you write a scripting language when you're already using a full blown programming language? You can put loops in a Povray scene description file? You'd like to handle operators and variables, and possibly even lookup tables? Guess what, Blitz3D already does all of that, and BlitzMax does it even better and faster. Given that Mark has already said that you will be able to create/set every single attribute of both static and animated models in the API, why would you possibly want him to waste months adding some kind of script facility which - at it's very best - would let you do half of what you would already be able to do directly in code.

@Evak

These lists are fast generated but there is a big difference between just listing support like in your list and if the importer/exporter really works the way it should and how much of a given scene it actually imports/exports.

Let me just take Modo and Collada for instance, there only exists a somehow working exporter as a sourceforge project, no importer, the exporter isn't done by Luxology itself which means no testing from them for future versions, ... and that's the way Collada is implemented in quite some apps.

Secondly for the shaders, how do you know if the shaders you get out by those tools will also work together with the 3d engine? This and how much problems you'll face strongly depends on how the 3d engine works.

Wait a minute. It's ok for people who have the high end packages (legit or not). But what about the people who can't afford a 3d modeller that supports these formats? This is why i'm asking for a convertor for UU. I use that extensively, along with Gile[s], because it works with .b3d.

And i'll probably continue to use .b3d in the new engine, as i don't HAVE any tools to create these new thing that the new modellers can do. SO, think of the guy with no money please :o)

As for converters, the only gfx converter i can remember using with joy was the Kefren's IFF converter, yay! :O)

yeah, I'm hoping that marks engine supports a standard shader format that can be exported from FX composer rather than using his own blitz native one exclusively. Otherwise there will be the usual problem where artists can't code, and have no support for tools that enable them to be creative with their shader materials.

BRL doesn't have a good track record supporting artists needs. You can't even post on the forums without buying a programming language first. Something that limits the community to being 90% coders.

@Evak: I think you only have to create an account on the site, you don't need to buy any product to be part of the comunity, otherwise people using evaluation versions wouldn't be posting.

@Evak

Still it's very likely that the shaders won't work together just as a 1:1 copy, maybe for simple stuff or in a limited case but you can take the shaders as a base before adapting them.

A graphical shader editor is something which is very nice to have in the 3d dev tool unless your 3d engine perfectly works together with a modeller's shader editor. This btw once has been announced on the forum for a future version of unity.

collada and 3ds max support plz :)

@Ziggy, your right :). I should pay more attention. For the longest time I wasn't able to post on the forums at all. And you couldn't create an account without purchasing something. Being an artist I Originaly bought maplet. Anyway, looks like you can sign up without purchasing something now :)

Side question- why doesn't Unity support obj file textures?

What it does is that it saves the information about different materials and uvs. As long as the texture is also in the project folder it's automatically referenced. If it's not in there then you just have to drag&drop the texture(s) on the object's textureslot(s) in the material(s) after putting them into the project folder.

Looks like this:



Once this is done Unity keeps this information. If you alter the object (geometry, uvs) and save it again, the changed object is updated automatically, so that there is no need to do it twice.

UU does Collada. As long as the built-in format can be represented in XML (although not necessarily exclusively so), "converting" it is at worst, a trivial task that can be automated.

Also Collada was designed with video games (specifically video games for the PS3) in mind and is an open format.

Why don't you just use Blender?

-> http://www.blender.org/

It's free, crossplatfrom and a.o. comes with fbx export.

I'd be interested to know what version of fbx it's exporting...

Then head to their forum.

ASCII version of FBX as the binary version SDK tools and libraries are incompatible with the GPL under which Blender is released.

It's development was funded by the people behind the Unity Engine by reverse engineering ascii files. Works really well. Work continues on a .fbx importer. There is an additional branch of the exporter dedicated to users of XNA.

http://wiki.blender.org/index.php/Scripts/Manual/Export/autodesk_fbx

The link may be down a lot at the moment because of the public release of the open movie, Big Buck Bunny and all the production files etc (some 16Gb of data) on the 2 DVDs I received. So, obviously the Blender Foundation servers' are getting hammered with all the interest generated by BBB despite most downloads being pushed through torrents.

Back to the worklog... when Mark says it's going to be C++, does he mean the engine is going to be programmed in C++ (which I had already assumed), or that instead of a Blitz language, us Blitz users will be using C++?

And does anyone know if this is going to be cross-platform?

FBX is a tranfer format for 3d apps, not games and thats a big disadvantage if your working on realtime graphics. Collada was developed for transfering realtime graphics and is more feature rich in game specific data.

I wonder why it's working so well with Unity for instance and why the Collada exporters i've seen are such a pain to be used in comparison.

> does he mean the engine is going to be programmed in C++ (which I had already assumed), or that instead of a Blitz language,

Yes, we end users will still be using Max, or B3D, or something similar. No need for us to use C++

An earlier worklog said it would be cross-platform.

why does it always have to be cross, cant it be 'relaxed platform' ;)

That is shocking - go hang your head in shame! :D

Interestingly, the FBX SDK supports Collada Read/Write, amongst others.