My idea for a network library

Hi there,

I am writing this idea down here, because I don't have the time to actually develop this concept. Perhaps we can start a dialogue here: is this a good idea, or is there a better way?

The code is in pseudo Blitz3D, but ofcourse the target language would be BlitzMax.
----------------------------------------------------------

CURRENT STATE OF BLITZ NETWORK LIBRARIES

The programmer sends and receives network messages by using a library like Blitzplay. By checking the message ID and sending values, you know what is going on 'on the other side' (e.g message id 5 is: the player died) and then you act accordingly to that signal (delete mesh, remove player record). This is cool and should be in, but sometimes it is not very practical.

SHARED OBJECTS

The concept is this: the whole network shares objects like normal variables.

Let's say all computers share the object:

type human
  field head
  field intelligence
  field hair
  method deletePlayer()
    // delete this object
  end function
  method createPlayer
   // automaticly ran when creating record
end type

The "big" command that makes it a shared object:

setSharedNetworkObject(human)
// make 'human' shared.

This tells the network module: the 'human' object is going to be shared across the network. Whenever something happens with an object (created,trigger,variables being set), the request and data is sent to all other computers, and is set (variables) or executed (methods, functions) over there as well.

h.human = new human
h\head="red"
h\intelligence = 50
h\hair = "dark"

...If the human object is a shared object, it will contain the values above, on every connected computer!

The same would count for executing methods and functions in that object. E.g: human.shoot("shotgun");
or delete h (remove the instance on all computers).

----------------------------------------------------------

Now, I do not know if it's possible to do this in Blitzmax. Somehow you will need to write a module that somehow knows about Blitz' internal object creation/deletion mechanism.

Secondly, it would be nice to set exceptions for certain variables inside a shared object. You could "unshare" them.

Lastly, perhaps a network Player() object would be required in order to do things like:

answer=isPlayerLocal(playerID)
if answer=1 then do collisionDetection

It's called "remoting" in dotNET and "Remote Method Invokation" in Java, and it's usually not the best approach for real-time applications.

It is an excellent sollution to software using a "thin-client / terminal server" deployment configuration, but it usually requires a considerably larger ammount of internet bandwidth then doing it oldskool, particularly because of sending redundant information. Considering your idea uses "full-blown" objects, and traditional remoting solutions only use interfaces, this would probably increase this problem further, additionally it introduces a syncronization, and object state problems. If two player fire near-simultaneously, both killing each other, who decides "who shot first", and should that player still die, since the other player was "technically" dead at the time.

Not universally (that would require reflection), but you can do it locally by extending a "remoting object".

It's not a bad idea, really - it's just not very practical with the current state of most peoples hardware. With distributed systems in theory becomming mainstream appliances with the launch of the PS3, a system like this would become much more achievable - at least on the PS3.

Okay, thanks for the reply :-)

Your reply on object state problems; yes, that is something to worry about, but don't you have the same problem with normal messages?

In Dukatiers (UDP network game) we had diamonds, which could be picked up. Player A picks up a diamond, and the object is removed locally and remotely by sending a command.

Well, that wasn't smart of me, because it introduced the problems you described (syncronisation) when the object had to be added again. Players who connected later on could crash randomly because of different object states.

The answer was by hiding/showing the diamond entities, not removing/adding the objects while the game was running.
Players who joined during a game had to load all the information on diamond locations (X,Y,Z,hidden/shown).

What I'm trying to say here: what is the syncronisation difference between sending network messages (current way of doing) and remoting? It's basicly this: instead of ME handling the object mutation, the module does it FOR me :-)
When you look at the network design, it's almost the same.

BlitzPlay for example, converts the data to a string, and back from a string to variables. This is still happening internally, only with the difference that it would also take care of placing the data in the right objects for me.

---------------------------------------------------

Optimisation:

Perhaps, with optimization, the module could check which parts of the objects are mutated, and only send this change through the network. Also with "unsharing" parts of the objects this could be reduced.

In fact, the whole bandwidth issue is the programmers' responsibility. I would only use it for players (X,Y,Z, playerStatus (int), character_face_look and some other commands.

Here, it doesn't make a difference (syncronisation) which method I use.

I would "unshare" object data like references to entities or other unnessesary stuff.

--------------

Yeah alot of network libraries do this now, it will only send an object state that has changed, however.

An example of one that does this is Torque Network Library, by Garage Games.

Yep, Raknet has stuff builtin for syncing classes and blocks of memory. Very cool stuff really. In many network optimized games its not practical though. It works very well and easily for stuff that must be sent and be the same on all comps, but for stuff that might not matter to the player, it just slows it down. For instance, things totally outside the view or sound range of the player often times dont matter. Even in the cases that its in the sound range, you might onlyu have to tell it that a certain sound is playing at a certain X,Y,Z, and not have to give unecessary details.

For stuff like player names, game state/messages this stuff is great though.

As far as optimizations go, we're a bit limited by the fact that there are no properties (methods that act like variables). If we had them then it would be possible to easily keep track of modified variables. Ideally it would be possible to automatically begin each property with stuff for networking, but this would likely require a preprocessor, or some interesting OOP stuff not seen in any other langauges. Once enough variable changes had built up, or some short amount of time elapsed, it could send off a packet summarizing all the changes.

Another possibility is that when a method is called, send a packet that will call the same method with the same parameters in the other types.

There are problems with having links to other types in parameters or fields (you don't want to pass memory location pointers), however this could be fixed by having type IDs that are the same on all computers. Then on passing it will convert the pointer to an ID.

Bot Builder, Raknet sounds awesome! It sounds you did some research!


Is it possible in Blitzmax to do this?

myString$ = "jeroen"
myClass.callMethod(myString$)

or

myVar$= "health"
myValue = 50
myClass.setVar($myVar$)= myValue

?

What you want in action games is usually speed, this means sending action-states (keypresses) and you can also interpolate the movement afterwards.

So to update another player you would send what keys he is holding down (action game) or where he clicks (strategy). This means you only send one byte for 8 keys! Actually it is wrong to think about keys because it more that you send their action state.
1.MoveForward 2.Back 3.TurnLeft.. 7.Jump 8.Fire
Same goes for AI-players

So you decide what actions the "player" has and then you make sure to update them - not the actuall data.

That's kinda how I'm doing it in my library. But there you have to manually pick the "actions" to update and build your program after that system.

It would be confortable to simple pick an object and the the library takes care of the rest. It would be fun to test to send data of an object (the change only) each time it changes. in this way you can get away with pretty small messages. Especially if you put eveything into packs.

But I don't know how to program an interface like that. Flameduck you mentioned extending an remoteobject, how did you mean? How would the library do to be able to check if any field in this object has been changed?

yes Wave, that seems logical. It's how I did it with Dukatiers. But, in this "new concept" you can do exact the same if you prepare your objects right.

E.g in pseudo code:

if keypress(LEFT)
p\keypress=0
end if

...and the network library will only sync the "keypress" variable, which happens to be a byte in this case. So...in the end you have the same data traffic, not more, if you prepare your objects right.

I don't know how you'll be able to program this smart syncronisation. I don't think BlitzMax allows you to go THAT deep?

How did you mean for the interface of the network lib to be?

This is how I do it now in simplified pseudo code:

Your Local Input.
If KeyDown(KEY_LEFT) P\Left = 1 else P\Left = 0
SendKeys(P\Up,P\Down,P\Left,P\Right)

Your received data of other players.
Each Msg
P = MsgFrom
P\Up = GetKeys(1)
P\Down = GetKeys(2)
P\Left = GetKeys(3)
P\Right = GetKeys(4)
del msg
next

Do you think I could add another interface on top of that, so that it can be even more simple to implend multiplayer? I mean the "user" (of the Netlibrary) can use a more intuitive command?

I had this idea from your idea =)

SyncKey( PLAYER1_MOVE_LEFT_KEY ) ' Creates a sync
SendKeys() 'Sends all your keys to all others

UpdateNet() 'Get message/sync

Each Player
If NetKeydown(KEY_LEFT)'Has this client this keydown?
'If it is a local player then it calls the normal
KeyDown()
'if the key is not in sync?
'All clients must sync the same keys right?
Endif
next

The is there a problem when you have more than 1 player on each client connected or AI players?

Hmmm I prefer your "old" method.

Perhaps you could make use of a net_id (player ID on the net).

If you send to net_id 0, you send to ALL clients, if >0 you send to one specific client.

Why don't you just send/receive the raw user input over the network, ie pushing the joystick up, pressing the fire button. This is much simpler data than

a) receive the input
b) interpret the input and turn it into internal meaningfulness
c) encode meaningfulness into the packet
d) send it over the network
e) receive it from the network
f) decode it to something meaningful and interpret

It could just be

a) receive input
b) send input over network
c) receive input from network
d) process it the same as for local input

AngelDaniel, if I get you right that would work for a game with max one player on each computer and no AI players. As Ai-players don't have input directly from the keyboard. And two players on one computer has different keys for the same actions.

And Jeroen thats how it works right now =)
SendUDP( MsgType , DataString$ , OnlyToID=0 , NotToID=0 )
if OnlytoID = 0 (or not specified) send to all. NotToID is specified if you want to send to all but that ID.

I'm thinking about allowing for "local" clients. So that you can have several players on one computer. In that way you could match the client number with the AIplayers and RealPlayers on your computer and in that way specify the keysync of each. But I can't think af an easier way to do it. I actually want to make it even more simple but I don't know how (got any ideas?).

And even though you send keypresses (when changed) you need to send fullupdates once in a while because of latency.

This is how I'm thinking the interface will look when it's done:

--- THIS IS NOT A WORKING EXAMPLE ---


'In main loop
For C = eachin localClient
    C.UpdateSend( 5 , [X,Y,Dir,Energy,Armor])
    ' 5 = Five times per sec - or once every 200ms

    C.UpdateKeysOnly( 0 ) 
    ' 0 =Always send keys if they have changed
Next


'In player input method (do this for each key)
C = GetClientID( player.ID ) 'Match with your type

 If Keydown( ~~~ )  

      'SetKeyToSend( KeyNR - From 0 to 7 , True or False )
      C.SetKeyToSend( 0 , True )  'Pressed

 Else

      C.SetKeyToSend( 0 , False ) 'Not down

 Endif



'in mainloop
If UpdateReceived() 'Check if you got any msg sent by UpdateSend()

    For C = Eachin RemoteClient 

       If C.ID = Player.ID ' match with your type

           'In the same order you sent them..
           Player.X = C.GetUpdate( 0 )
           Player.Y = C.GetUpdate( 1 )
           Player.Dir = C.GetUpdate( 2 )
           Player.Energy = C.GetUpdate( 3 )
           Player.Armor = C.GetUpdate( 4 )

       Endif
    Next

endif

So when you create a player you need to create an client equalent.

'In the New() method of your player type
AddLocalClient( Name$ = "" , Info$ = "")

'And in Delete()
RemoveLocalClientwithID( Self.ID )

When the Client connects to host (or when he himself host a game) the local clients will be given IDs by the Host. If you AddLocalClient() After host() or Join() it will announce itself to all others. Same goes if you change the name/info of any client by using yourClient.SetName( newName$ )

I'll probably notice that this doesn't work after trying to code it for X weeks, but it's the guideline I'm working after. Ideas are welcome =)

hi Wave,

Woops, I didn't see that my id=0 suggestion was already in :-)

To get back to your suggestion to use a different method for sending/receiving variables, perhaps for now your current method is the best, after reading this discussion again.

But, you could make an exception for player movement though.
What I mean is network prediction. It is something I suggested for Blitzplay as well. In this concept, prediction is done automaticly by the network library.
pseudo code:

main loop
 for p.player = each player
   if syncposition(p\id)=false
     playerposition=calculatePrediction(p\id)
     positionentity myplayer, playerposition[0], playerposition[1], playerposition[2]
   else
     playerposition=getPosition(p\id)
     positionentity myplayer, playerposition[0], playerposition[1], playerposition[2]
   end if
 next

end main loop

What I mean with the above example is that sometimes the network library does not send/receive playerpositions, but calculates the position based on previous player positions. It predicts. The statement after the 'else' DOES receives the actual player coordinates.

How often the library predicts or not, can be then set using a constant:
PREDICT = 5 means the library predicts every pass, except pass 0. So changing this value into 10 means that the library predicts more, but syncs less.

Good idea. I have to admit I haven't tried to implend movement prediction yet. (right now I see no gain in it) but when it comes to bad connections (which I will test when I get the basics working) then I'll look into it. And the prediction could actually be done in the update function from the example above. Which means you simply set the predition type and or frecvency before you start the game. EnableQubicPredition( 10 )
But as I said I don't know how hard it will be to include or if it will actually reduce/improve visual lag.
I would also need a 3D action shooter to it test on =)

Cool!
Blitzplay source includes a Cubic Splines include which you can look into.

Is there a way for you to simulate a slow network connection?
If I need to help to test "on the other side", send me a mail!

Also a GNET kind of system will be nice. Blitzplay uses a DLL to avoid network hickups while doing a TCP request (sending "i am alive"), but this isn't cross platform. And, perhaps Blitzmax won't "hickup" doing this.

I hope we get GNet for BlitzMax, it would be great, perhaps it is easy to convert it. The hard thing with QuibicSplines are the implentation, which I think BlitzPlay doesn't do automatically (just a function for it, or?). The first version will only contain UDP, I guess I'll run into some problems when I add TCP, but I probably will because in some game it is of very good use.

check out Mark's worklog: you'll get some competition, perhaps :-P

Blitzplay delivers you the function but the user needs to implement it some sort of way. That's illogical I think; player coordinates are always based on two or three axes. So why not implement it in a higher level? (e.g my example I mentioned in this post)

I'll definetly go high-level for the user of the library. And that Mark is working on a network module probably only means more secure/stable networking with more features. Perhaps he is doing a [Blitz]MaxPlay Module?
He wrote in his log it was to be a "simple" module.
If he meant the module should be simple to use for the user, which I aim for, or simple to code for him I'm unsure of.