UDP Networking Basics (questions)

This may well be considered a frequently asked questions about UDP networking, because I think I have to ask just about every question under the sun about it. I'm trying to understand how to do the most basic things with UDP but I'm not quite getting it. It's partly because not much of it is clearly documented and partly because this just isn't seemingly discussed much. I'm sorry about the sheer length of this and the number of questions, but this is stuff that I and others really need to know.

I've tried looking at some networking libraries but for the most part that isn't entirely helping. I know that UDP is a bare-bones protocol with no `connections` that just send out packets hoping they will arrive while the other end waits to receive something from some unknown sender.

I want to put together or use really basic commands to just open a port for sending, open a port for receiving, then send or receive packets of byte data ie like between a server and a client, or two peers. Nothing fancy at this point, I can expand with features later. I don't want to turn to a third party network library, I want to understand how to do this myself.

From what I gather, either you need to use really low-level commands which are not documented and refer to C include files, like `send_`, `recv_` etc, or use TSocket which wraps them fairly lightly into more user-friendly methods such as Send() and Receive(). So this is largely going to be questions about using TSocket.

So I have some questions and I'm sure many other people want to know about this too.

1. How the h*** do you actually set up a port on each machine ready for sending/receiving data? What commands are needed and what is the format and what data do you have to pass to it? From looking at the code for the TSocket type, my guess is that you need to do something like:

'Client trying to connect to someone
Local Sock:TSocket=Sock.CreateUDP()
If Sock=Null then CantCreateUDPSocket() 'deal with failure
Local Success:Int=Sock.Bind(MyPortNumber) 'what port?
If Success=False Then CantBindThePort() 'deal with failure
Sock.Connect(RemoteIPNumber,RemotePortNumber)

Is this code even right?

2. What port number should you pass to Bind()? I read online about reserved ports and all that, seems like you have to use numbers from 50000 to 64000 or so, but I see other people using ports like 80 or 8080 or whatever? How do you choose a port number and why do you choose it?

3. Does the port number you choose to be bound to on the client machine have to match with the port number of the server machine or can they be different so long as they both know which port to send TO? e.g If the client sends and receives on port 50000 and the server on port 55000, it will work so long as the server receives on 55000 and sends to 50000, and vice versa?
Is there any reason to make them the same port number on both?

4. If trying to bind the port fails, should you just try another port number? How many should you try? How should you space them out? Are there safe or likely-to-work numbers you should use? How do you know which ones are available?

5. Is the RemoteIPNumber just an integer representation of the dotted IP? Ie something like 156.341.192.39 could be the address of the server machine that you want to connect to, so you need to convert this to an integer representation to use with the Connect() method? The question of where to get the server's IP address from is not important right now but I would like to know how to convert such an address into the appropriate value to use with Connect() ??? DottedIP() seems to convert from an IP integer to a dotted version but how do you convert the other way? Do you use HostIP() and if so what data do you pass to it? Like HostIP("156.341.192.39",0) ???

6. Do I actually need to call Connect() at all for use with UDP? e.g. to set the remote address and port that I'm sending to? I presume in order for the client machine to build a UDP datagram packet it has to know what destination IP and port number to send it to, to put into the packet header, so somewhere that has to be set, right? So does Connect() just set that information, but not actually try to send anything like TCP would? I know for a TCP connection it has to do a handshake but UDP does not. So do I just do Connect(RemoteIP,RemotePort) and it immediately returns, having just set that data ready for future sending and having not actually sent anything?

7. Is there anything else I need to do to set up the client for this readiness to send out UDP packets?

Assuming we've got this far and the port and addresses and all that are set up correctly and is just waiting to send stuff...

8. Do I just call Sock.Send(MyMemoryBufferBytePtr,NumberOfBytes,Flags)?

9. When I do call Sock.Send(), what are the possible values for the Flags and why might I use them?

10. Each time I call Sock.Send() will it actually send out the UDP packet right away or does it add it to some kind of accumulative buffer until a certain number of bytes are in the packet, or would I need to write that as a separate abstraction layer myself?

11. How many bytes should I send with each call to Send()? I have heard different numbers like no more than 1500, no more than 1200, no less than 64 due to it automatically padding it to at least 64 bytes, etc. Is there a way to detect what the maximum number of bytes is that would work without it flooding? Does it depend on the client's transmission hardware and/or the server's hardware? What size do you recommend that is widely supported, 1024 bytes? Do I need to play it safe or is there a way to actually determine the size?

12. Is there anything else I need to do to cause the system to send out the packet I've given to the Send() method?

Presuming this is all there is to it, we then move on to assuming that the packet was sent to the address provided in Connect() at the port provided, and that the packet actually does make it to that address and port (I know this is unlikely sometimes, but for the sake of proceeding)...

13. What is the server now doing to look for or anticipate this potentially incoming packet, having absolutely no prior knowledge of the computer it is being sent from or what is being sent in the packet? I presume at the very least the server will have set up a similar process whereby it says someting like:

'Server getting ready to receive client data
Local Sock:TSocket=Sock.CreateUDP()
If Sock=Null then CantCreateUDPSocket() 'deal with failure
Local Success:Int=Sock.Bind(MyPortNumber) 'what port?
If Success=False Then CantBindThePort() 'deal with failure

Is this correct?

14. Presumably you cannot use Connect() because you don't know who is going to be sending a packet to the server or where from, right, or is this a requirement at this point? Do I need to know the IP and port of the computer that is going to be sending packets to me, prior to them having ever sent any at all? Or is there a way to just listen for `any packet` that might be available? And if so, how?

15. I presume this is where I would use the Sock.Recv() method on the server, in some kind of loop which occasionally tries to receive data from the socket. From looking at the code in the Socket type it looks like it tries to read data from the socket into a byte buffer at some address and then tells you how many bytes it read, or what looks to be 0 if there were none (am I on the right track?). It also looks like you can tell it how many bytes to receive at a time, so that you don't overwhelm your own buffer space. So if I'm understanding this, does the server now just sit in a loop and call Sock.Recv(MyStorageBufferBytePtr,MyStorageBufferSize,Flags), keeping track of the value that is returned until that value >0? And when it is >0 does this mean that I've successfully received *something* from *someone* with a given number of bytes, up to the size of my storage space?

16. Does Recv() allow packet data to come in regardless of who sent it or from which port? Or is there something I have to do in order to allow it to receive data from a given sender? I noticed that for TCP there is Accept() and also ReadAvail(), but do these apply to UDP at all? I guess I can use ReadAvail() to tell me if bytes have arrived without actually transferring them to my buffer, for UDP as well as TCP? Do I need to do an Accept() for UDP? and does ReadAvail() tell you that bytes arrived even if you still don't know yet who sent them or from where?

17. Assuming that I would now be at the point where data is beginning to come in from somewhere, and assuming that I didn't need to do anything else to allow that data to arrive from a given client, does the data that gets transferred to my memory space include the UDP header information or is it purely the payload that the client gave to the Send() method, and only the amount of bytes that it gave?

18. If I get say 200 bytes of data from client A and 200 bytes from client B at the same time, presuming client A's data went into the buffer first followed by B's, so I then need to read my memory buffer in such a way that I know where client A's data ends and client B's data begins? Like, by sending a `length` value or using some kind of delimiter? ie does my input `stream` of byte data end up being just all the packet payloads concatenated after each other and I have to divide it up myself into actual `packets`?

19. Once I have some data received into my buffer and I've determined the size of the packet - and maybe it fits into my buffer or is the result of several calls to Recv() joined together, how do I determine what IP address and port it came from? From what I gather this information is in the UDP packet and I know some libraries like BNetEx are able to `sniff` the packet header to find out who sent it, is there a way that I have access to this data using only the TSocket methods/functions, or is that something you'd have to do at a lower level working with the o/s calls? Is that simple to do, while also using TSocket? And what is the alternative - just have the client send their IP address and Port number as a part of the byte data payload, ie as two integers?

20. Do I ever need to use the Listen() method with UDP? Seems to me listening sounds like waiting. Does it apply to UDP?

21. If I use Sock.Close() does that close the entire socket and all ports it is associated with? If so, can I then reopen another socket and bind it to the same ports? Can you have more than one socket bound to the same port at the same time?

22. Does the Sock.Connected() method work with UDP, since it does not really have connections?

23. How do I find out what the IP address is of the client's machine, presuming they don't know how to go looking for it manually? I will need to send this IP address to the server so what are the appropriate methods or functions to call to get this IP address either as an integer or as a dotted/human-readable version? I presume DottedIP$() returns the dotted version of the IP integer, but how do I get either from the client's machine in a reliable way that will definitely work? I've seen a few approaches here and there so I am confused which one works.

24. What is the 127.0.0.1 or whatever it is? What is it for? Why would you use it? How does it work?

25. Can I use RemoteIP() or RemotePort() to get the IP and port of the client after they send a packet to the server, or does this purely tell me what I already know - which IP and port I already said I was sending data TO?

26. Can multiple users access the same port on the server? Should I give each user their own port to avoid flooding the entire system? If so, can I just add 1 to each port number each time (within the allowed range?) I read that maybe you should receive `connection attempts` on one specific port and then give each user their own port?

27. Do I need to open multiple sockets for any reason whatsoever? ie if I want multiple ports, do I need one socket for each port or can a socket have multiple ports? (The code looks like it has one port per socket?)

28. If I close a socket on the server with Close(), will that mean that if the client still sends packets to that port at the server it just will be ignored by the TCP stack or whatever? And if they get absolutely no response from the server (e.g. downtime) would I have to code it into the client to interpret this to mean `server is not available`?

29. What does the UpdateLocalName() method do? It appears to be related to the IP address of the computer that the application is running on? Why would this likely change and why might I want to call this method directly?

30. Same applies to UpdateRemoteName(), what does it do and why would I use it?

31. Presuming that at this point I have set up the client and the server (or peers) with open ports ready to receive data and am receiving it correctly and sending stuff back and forth, is there anything else important that I need to know - any do's or don'ts?

I know that on top of these basics you could have a whole host of features like authentication, different topology models, streaming, dynamic bandwidth adjustment, logging-in/logging-out, sending files/chats, putting together game packets, etc. That stuff actually seems easier to understand and implement to me. It's this fundamental UDP stuff that I am not quite grasping. Any help is greatly appreciated.

24. 127.0.0.1 is a loopback network adapter. Its the IP behind "localhost" which means "yourself"

28. UDP has no connection so its up to the client to realize that the server does not react and make it conclusions basing on that or further tests

31. quite some stuff actually like "how to retrieve the systems MTU" and stuff like that which is very important to the network layer.

Before you even think about the stuff below your "socket layer" needs to work correctly following the protocol definitions etc. Otherwise you will run into bugs you will never be able to solve.

Ok. So MTU is the `maximum transfer unit`, relating to the maximum packet size that your computer can send out before it gets split up. I know that in my eventual abstraction layer I will need to decide on some kind of limit, keeping in mind that if the limit is breached the data will be split into more packets. Does this mean if I tried sending 64k of data with Send(), it would potentially break it up into like over 50 individual packets? It would still be sent out in full but with the risk that certain packets (pieces of the initial packet) might be lost?

If UDP is going to split up my larger packets into more than 1 smaller packet, and those packets could be received out of order or lost, how can I know what the size of the packet is, given that it might end up joined onto the end of some other packet data when I call Read() (boy, I still need these other questions answered)? Or do I not need to be concerned with that? If I set my maximum size fairly low (PPP is 576 bytes?) could I assume safely that I won't get this phenomenon? Is this what you were alluding to?

Or do you just mean that both the client and the server (or all users) have to agree to use the same maximum size?

If you send that large UDP packets, something went massively wrong anyway as UDP is only used for low latency - high performance networking. Not for "continous data streams", which is TCPs job.

Not that sure that the network end will split it up ... it could as well just die down as Goldstars problem a few threads below this shows and he was using the TCP Socket, not the UDP one.

You will have to handle all that kind of stuff including handshaking, message reordering, reliability etc which you most likely will need (to drop ordered packets that shall be dropped if a newer one is present and a few other things) yourself. The socket does not give you more than a possibility to talk to someone else through UDP.
But your app has to make sure that the "letters have a valid weight", are sent with "signature needed" etc to tell it in paper mail terms.

Yah, I was just reading some more about the MTU and Wikipedia is saying that the Ethernet limit is 1500 bytes and that most of the internet supports that. What I'm not sure of yet is whether I have to code the re-ordering of fragmented packet parts myself or whether the IP protocol does that beneath UDP.

[Edit] It appears IP handles this, and if part of the original packet which was fragmented gets lost then the entire packet is compromised and won't be delivered, requiring a resend.[/edit]

I recognize that I will need to do all those other things eventually, like handshaking and message ordering and reliability etc, but that's really a layer on top of the basic use of UDP. At this point I don't care so much what is sent or whether it is received or in order, I just want to know the finer points of how to get something sending and receiving at all.

"But your app has to make sure that the "letters have a valid weight", are sent with "signature needed" etc to tell it in paper mail terms."

I did not understand what you meant in that sentence at all. Are you just saying that if I want all packets to arrive reliably I will need to say in the packet data itself that I want an acknowledgment?

Basically yes.

I still don't understand why all who want to use UDP actually try to start with 0 instead of using either Pub.ENet which is a high performant UDP layer with support for ordered messages and reliable message but nothing more than that. So a working basic UDP layer.
Alternative there is BNetEx from Vertex which takes care that you don't exceed the MTU and other problems you will run into (like: how the heck do I handle it if I have more than 1 network device installed? how do I broadcast something to my local lan etc). BNetEx has no ordered packet delivery or reliable packets. Its "only" a TStream wrap up of TCP and UDP socket with usefull and nice extra functionality that you might potentially need in "realworld" scenarios.

I just have a different vision for how to go about my higher level stuff which isn't compatible with these other libraries. I need to get down to the basic direct access of using UDP.

Besides that I would just like to know what I'm working with here. BRL provided the TSocket type and it isn't very clear how to use it for UDP connections so that's why I have all these questions.

Thanks for your help. Maybe someone can start with question 1?

Hi ImaginaryHuman,

I did find the documentation on UDP a bit limited as well.
And BNetEx didn't compile on my Intel Mac.
A small example which works for me:

Strict

' one for reading
Local udpRead:tSocket = CreateUDPSocket()
BindSocket (udpRead, 49000)

' one for writing
Local udpWrite:tSocket = CreateUDPSocket()
ConnectSocket (udpWrite, HostIp ("127.0.0.1"), 49000)

' Send some data
Local b:Byte [] = [Byte(9), Byte(7), Byte(5), Byte(3), Byte(1)]
UdpWrite.send (Varptr b[0], b.length)

' create a loop which waits for some data
While True

If SocketReadAvail (UdpRead)
' a bank to store the input
Local tmpBank:TBank = CreateBank (1024)
' size of data-package
Local length:Int = UdpRead.recv (BankBuf (tmpBank), tmpBank.size() )
' print data
For Local i:Int = 0 To length - 1
Print tmpBank.PeekByte(i)
Next
' exit this loop
Exit
End If

Wend


CloseSocket (udpWrite)
CloseSocket (udpRead)
End

Ghislain
MacIntel 10.5 - BlitzMax v1.28

BNetEx is really good and fast! Ive done -as you may know- quite some reaseach myself. Thanks goes to Dreamora, hes really been a good help.
127.0.0.1 is your local IP. It can also be accessed by type localhost instead. Just like the ip 84.124.58.34 maybe linked with www.somesite.com.
To check your link tables for your local computer on windows check: C:\WINDOWS\system32\drivers\etc\ open the "hosts" file with notepad.

Well TSocket does not need to be documented.
Its an OS level interface so consult your OS documentation as it will work differently with different constraints on any OS.

Thanks for the example, Ghislain. It seems pretty straightforward and pretty much shows the basics which is good. Thanks.

I am going to have to figure out and answer my own questions I guess, as best I can. Please correct me if any of this is not right.

1. Based on your code Ghislain, I guess the following should work:

'Client trying to connect to someone
Const ClientPortNumber=49000
Const ServerPortNumber=50000
Local Sock:TSocket=TSocket.CreateUDP()
If Sock=Null then CantCreateUDPSocket() 'deal with failure
Local Success:Int=Sock.Bind(ClientPortNumber)
If Success=False Then CantBindThePort() 'deal with failure
Sock.Connect(HostIp("123.456.789.012",ServerPortNumber)

and then for the server...

'Server getting ready to receive the clients
Const ServerPortNumber=50000
Local Sock:TSocket=TSocket.CreateUDP()
If Sock=Null then CantCreateUDPSocket() 'deal with failure
Local Success:Int=Sock.Bind(ServerPortNumber)
If Success=False Then CantBindThePort() 'deal with failure

and then the server will sit in some kind of loop which will call Sock.ReadAvail() to see if there is something that can be read, and then Socket.Read() to get some bytes into a buffer.

2. It seems ports in the range 49152–65535 are safe to use as they are unassigned to any permanent application. Some applications seem to have registered port numbers so I suppose if you try a port and it is in use you should just try another. Ports 0 to 1023 are well known ports, ports 1024 to 49151 are registered ports (and you can register a port officially if you want to, but otherwise should not use these ports unless you want to try your luck not clashing with an application). Also ports 0 to 1023 seem to need privileges (normally admin/root) to get access. See http://en.wikipedia.org/wiki/List_of_TCP_and_UDP_port_numbers and the official list of registered numbers is here: http://www.iana.org/assignments/port-numbers

3. I don't think the port has to be the same, it's just that you have to know what port to send TO, and that port number must be the port that the recipient is receiving on. So if a client has port 50000 and a server has port 55000, the client sends to 55000 and the server sends to 50000, the client receives on 50000 and the server receives on 55000. So long as each knows what ports are involved it should work, or otherwise you could just make the ports the same on both - but it doesn't seem to be a requirement.

4. As per previous answers it seems if you go with the range 49152–65535 you should be fairly safe not to conflict with at least those applications that have registered port numbers. If the port is already allocated you can just use some scheme to find another port, such as incrementing it by 1 and trying them until you find one that works, or spacing out the attempts maybe adding 100 each time or something. Chances are you aren't going to have 65000 ports being used simultaneously on the same machine.

5. You use HostIp() to convert a dotted IP address from a string into an integer value. This works for the Internet Protocol version 4 which has 4 byte-values with dots between them, but would need 6 bytes for IPv6. If the server's address was "123.456.789.012" I'd use IP:Int=HostIP("123.456.789.012") to convert it to an integer. I am not sure what the remaining parameter of HostIP() is though. Please note that each of the values between the dots are byte values from 0 to 255 only, so you will never see 456 or 789. Various ranges of numbers are reserved for various purposes but we don't need to be concerned with that for just using whatever we're given ;-)

6. It appears that yes you need to call MySocket.Connect() in order to assign a destination IP address and port to the outgoing packets. If you don't do this the packets do not get sent and they do not arrive. I cannot tell from the TSocket code whether there is any handshaking or data sent when trying to `connect` a UDP socket, but I would think that there is not because UDP doesn't do those things for you. I am making an educated guess in saying that all it does it assign the IP address (in its integer form, converted from HostIP(), and the destination port address to the socket so that when you send a UDP packet it will know what to put into the header for where to send it to. It doesn't actually send anything to establish a connection but it does set it up ready to send stuff. This function also calls the UpdateLocalName() and UpdateRemoteName() functions mentioned later, so we might not need to call those methods directly. Also it appears that the Sock.Connected() method may not work with UDP since there is no connection. With or without Connect() being called, Connected() returns a 1. So we have to keep track of whether we have made a `connection` ourselves (ie whether we've applied a destination port/IP to the socket ready for use).

7. It doesn't appear that anything else needs to occur in order to be ready to send/receive data via UDP. All you need is to create a socket, bind a port to it, `Connect()` using the destination IP in integer-form and the destination port number, and then you are ready to send/receive on that port.

8. Yes to send something on that socket, once you have associated a destination IP and port number with it using Connect(), you just call Sock.Send(MyMemoryBufferBytePtr,NumberOfBytes,Flags) or you can omit the Flags parameter. Calling Send() sends the packet of bytes immediately.

9. There doesn't appear to be any meaningful use to most people for the Flags parameter of Send(). I came up with: "MSG_EOR Terminates a record (if supported by the protocol). MSG_OOB Sends out-of-band data on sockets that support out-of-band communications. The significance and semantics of out-of-band data are protocol-specific." I don't see that most people have a use for it, and I don't know what it means so we might as well let the Flags default to 0 which they do if you omit the parameter.

10. Yes when you call Sock.Send() it sends the packet immediately using only the bytes you've indicated to it in the Send() method's parameters. It does not accumulate previous byte data that you asked it to send and it does not wait to send it. Each call to Send() actually sends the data over the network, and only the data you specify. If you want it to accumulate you have to write an accumulating buffer of your own and then only call Send() with it when you actually want it to go out.

11. It seems that the question of how much data to send at one time is a bit uncertain. There are ICMP protocols defined which if used are supposed to be able to tell you whether a packet of a given size was able to get through to the destination, over a given route. But apparently this information itself can get blocked sometimes and it seems to be at the IP level not the UDP level so that's not an option. Also the route can change dynamically at any time. What it comes down to is that 576 bytes is the minimum packet size which MUST be supported by the entire internet running on IP version 4, which is the total bytes including any header data. So if you have say 30 bytes of UDP header data (I forgot how much exactly) then you subtract that from 576 to get like 546 or so. If you send packets no larger than this then they will not be fragmented into more packets. Basically if you send packets which are larger than this then you start to run the risk that certain pieces of hardware along the route may split your packet into multiple sub-packets and then there is the potential for a sub-packet to be lost, which means the entire packet does not reach the destination and you'd have to resend it if needed (talking about the MTU maximum transfer unit here). That said, it appears that the minimum for Ethernet is 1500 bytes (including headers). Ideally you'd want to set your maximum packet size to be as large as possible, given that the larger it is the longer it takes for slower modems etc to transfer that whole packet before some other packet can get through, increasing lag. Also it depends on the route your packet ends up taking, which you cannot predict, and which may change each time you send something, because if that route includes low-MTU hardware it might get split up, but who knows maybe you get a good lucky connection and can send 1450 or more bytes per packet. It seems some libraries monitor the rate of successful throughput and so on to try to estimate an ideal packet size. Also if your packet size varies you're going to have to tell the receiver what size it should be, or delimit it, unless you're doing some kind of stream of a larger number of bytes. If in doubt 576 total bytes per packet including headers is supposed to be safe but not as efficient as closer to 1500. Each packet has to have a header so if it gets split into sub-packets each sub packet gets a header which means more bloat and less throughput. Also the more packets you send the more overhead trying to figure out how to route it and that can result in more lag. The ideal would be to send big packets and get them through a fast connection with no splitting or extra overhead. I guess in the `real world` you might either try some kind of estimated `pretty good all round number` for your packet size (between 576-header and 1500-header), or otherwise do some kind of monitoring and adjust it dynamically. In terms of how small a packet can be, it can be even just 1 byte that you send, but I have heard that UDP will pad up to at least 64 bytes of data. So if possible you should send at least 64 bytes at a time even if that means joining together several pieces of information.

12. No it seems all you need to do is call Send() with some bytes. It is sent out right away without delay and returns immediately. It is then entirely uncertain whether the packet will arrive at the destination, whether it will be split into sub packets, what order it will arrive in, whether it will arrive with exactly the same data as it had when it was sent ie not hacked, etc). Also if you want to send data longer than the packet size limit you'll have to implement an abstraction layer which spools data off from a larger buffer, which is pretty easy since you can pass the Byte Ptr of the next chunk of data to the Send() function along with how many bytes to send. The other end then just needs to know what it's dealing with when it receives data and how to reassemble it into the original larger file or whatever.

13. Code as above, if the server is waiting to receive data on a port, the sender has to have sent it to the same port number that you're trying to read from and the packet has to have been successfully transmitted across the internet (which it may not be). Then you can test in a wildcard kind of way to see just `if there is any data to read` using Sock.ReadAvail() which returns the number of bytes waiting to be read from the socket's buffer, and then when you detect some you can use Sock.Recv() to read the bytes into your own buffer - a bank or whatever. Sock.Recv() also appears to tell you how many bytes were actually read, or 0 if none, so you could avoid using ReadAvail() if you just want to try reading straight away. Maybe ReadAvail() is more efficient, though, I don't know. Revc() does NOT appear to wait when there is no data to read, it just returns immediately with 0 as the bytes transferred.

14. You only need to call Connect() in order to send OUT data to a destination. If you were just the server sitting idly waiting for a connection attempt you would not be sending anything to the client yet and so would not need to Connect() with them (ie assign destination IP and port). You would not necessarily even know the IP and port to assign. You monitor packet availability with Sock.ReadAvail(), or Sock.Recv() to read data if available. Once you are reading data from the client you can presumably obtain from them in some way their IP address and port number. This data is in their UDP packet header which is not easily accessed, otherwise you can just have them transmit their IP and port as a part of the byte data they send. Then once you know the IP and port of the client, you call Sock.Connect(ClientIP,ClientPort) and from then on can now send TO the client on the port of their choice (which may or may not be the same as the port number you're receiving on or sending on). You can send *on* port 50000 and yet send *to* port 55000 if you want to.

15. Yes, you use Sock.Recv() to read bytes from the socket into a buffer. Presumably your buffer should be at least as large as the maximum packet size so that you get the entire packet into the buffer in one go, otherwise you would have to continue to make more calls to Recv() until you've got the entire data. If your buffer is smaller than the packet size then you'll need to do something with it or add abstraction to deal with the fact that there's possibly still more bytes to be received. Although calling Sock.Recv() brings in data from the socket's buffer to your own accessible buffer of whatever size, this does not tell you who sent the packet or where from. You may have to ask the sender to send you their IP address and the port number that they want to *receive* data on, in order to communicate with them, or otherwise find a way to obtain this data from the UDP packet header which is NOT available using Recv(). Recv() only gives you the packet payload - the data bytes which the sender wanted to send, minus the header. Also note that the socket itself has a buffer of who knows what amount and you have to be careful that your sending application doesn't send too much data too often as to make the socket's buffer become full. If it gets full it may completely ignore any further input until some of its space has been flushed out by calling Recv() (which removes the data from the socket's buffer). This is where `flow control` comes in, or making sure you have an estimate of how much bandwidth you really should be playing with or how often you should send how much data.

16. Accept() is not compatible with UDP, it is a part of the handshaking protocol of TCP. Since UDP does not actually have a notion of `yes you are connected to me or no you are not`, it doesn't care about who is sending or who is allowed to send. Recv() does not tell you anything about who sent the packet or from where. You could also be receiving multiple packets from different senders coming into the same port, so unless you've assigned a specific unique port number to each client you can potentially receive packets from many clients into the same port, and would have to therefore identify who sent what, either within the packet data or by `sniffing` the packet header using some low-level technique at the IP level below UDP. You might want to use one port for `new callers` so that all new clients send a message to that one port asking to `connect` and then when you have done your handshaking or whatever (or nothing) you assign them each their own dedicated port. This way multiple users sharing the same port with lots of traffic wont so easily flood the port's buffer.

17. Using Recv() you only get the maximum of either the number of bytes in ONE packet, or the number of bytes you've asked to receive. If the packet size is smaller than your buffer and you ask to read an entire buffer's worth of data you will only get a packet's worth of data back - and the value that Recv() returns to you will tell you how many bytes were transferred to your buffer. It is not likely to be the same as your buffer size unless you are working with some kind of totally fixed packet/buffer combination. If the packet is larger than your buffer you will only get the amount of bytes you asked for from that packet and not the entire packet. You will have to Recv() again to get additional bytes, for as many times as are needed until you get the whole packet. You could potentially ask for 1 byte at a time but that would be slow, you should try to read the whole packet at once if possible so your buffer should probably be at least 1500 bytes to cover all potential packet sizes. There are such things as jumbo-size packets like up to 9000 bytes, or even others that are up to 64k, but you probably won't get into that.
Note that you will only get ONE packet's worth of data per call to Recv() (or up to one packet's worth depending on your buffer size). You will NOT get the contents of two or more packets joined together in a single call. Each time you call Recv() you get only one packet.

18. From my tests it appears that if you make two calls to Send(), sending whatever small amount of data, and then after a significant enough delay to allow the packets to be transmitted and received at the socket you call Recv(), you only get the first packet transferred to your buffer. The next time you call Recv() you get the second packet. It doesn't appear to matter how much buffer space you have or the socket has, the Recv() seems to only let you see the contents of up to a single packet at a time, or in other words, Recv() stops transferring data to your buffer when it gets to the end of a packet. So even if you have a 64k buffer it is useless to be that big because if your packets are no bigger than say 1000 bytes you will never be able to get more than 1000 bytes from calling Recv(). Recv() does NOT join together the contents of multiple packets. You also cannot tell what order you will receive packets in, either from multiple senders or from the same sender. Each packet needs to identify itself. Now, if you did want to send data larger than the maximum packet size (say you want to send a big file) the you have to write your own code to do some kind of `stream`, ie send a chunk of the file with multiple calls to Send() and then receive the chunks with each call to Recv() to gradually rebuild the whole file. Also you have to then consider flow control and not just spitting out thousands of bytes as fast as your computer will go - it will totally flood the receiver. TCP has a nice sliding window protocol for dynamically adjusting the sender's send-rate based on how much of the received data has been acknowledged etc. Or you can do your own streaming protocol. You may need to write your own method of keeping track of which chunks of data have been successfully sent and received, like sending a chunk or sequence number as part of the data, which also allows you to then receive chunks out of order without having to wait for a linear receipt order. It doesn't appear to be necessary to delimit your packet at the end with any kind of flag or value or code or character, as Recv() seems to automatically know where the packet ends based on its access to the IP header data which includes a length value (the one you passed to Send()). But once you are designing your own structure for what data you send, ie you might want to send strings, then maybe you will need to use delimiters to tell yourself where each string ends, or send the length of each.

19. Here is some code which someone here wrote which may possibly tell you who sent a packet, but I don't know if this works.

'Get sender's IP and port#
Local bytesbefore:Int = socket.ReadAvail()
Local buffer:TBank = CreateBank(bytesbefore)
Local from_ip:Int,from_port:Int
recvfrom_(socket._socket,buffer.Buf(),bytesbefore,0,from_ip,from_port) 'Function recvfrom_( socket,buf:Byte Ptr,size,flags,sender_ip Var,sender_port Var)
Local bytesafter:Int = socket.ReadAvail()
Local bytesread:Int = bytesbefore-bytesafter
buffer.Resize(szread)
Local data:TBankStream = CreateBankStream(buffer)

It is using a function recvfrom_() which seems to be lower level than the TSocket, possibly an o/s call that BlitzMax already has incorporated, so this might be a way to get the actual sender's IP address and Port number from the header of the UDP packet itself - something which is otherwise inaccessible using just Recv(). It also seems to possibly be reading some bytes in from the packet at the same time, so perhaps is a substitute for Recv() that additionally tells you who send the packet. Perhaps someone can test this (I will at some point). Otherwise you may have to have your sender include their IP address and receiver-port as part of their data when they initially contact your computer. Later, perhaps after you've done some kind of logging-in protocol or whatever, you can then identify that user by some other method like a codeword or Integer value that you include in each packet. It adds a little overhead though but not much.

20. No, Listen() is a part of the TCP protocol and from what I can tell is basically telling Blitz to watch for or allow data to be received on a given port, prior to actually `accepting` an attempted `connection`. This does not apply to UDP and is not needed. You could write your own protocol to do similar things on top of UDP like setting a port to allow input or not, or to accept new clients on a port, whatever. But Listen() does nothing for UDP.

21. From what I gather, you can only Bind() a port to a socket - ie associate it with it, if that port is not already being used by some other socket. I don't think it would make sense to be able to join multiple sockets to the same port, regardless of whether those sockets are a part of your own application or a part of some other application. There are only so many ports shared amongst the whole operating system on a given computer. Once you Close() the socket it closes all of its associations with other ports. Looking at the code for TSocket it seems each TSocket only has support for ONE port per socket. If you want to use multiple ports you have to create a socket for each port and Bind() each one individually. If you Close() a socket the port it was bound to is now available for any other socket to bind with. This all applies only on a given computer - if a port is already used on your computer but not used on another person's computer you can still send to that port on another person's computer, you just can't bind it on your own computer in order to send FROM that port or to receive on that port. 1 port is bound to 1 socket on 1 computer. Multiple other computers can send to and receive from any single port, but each computer cannot associate multiple sockets with a given port on their own computer. Logically, a socket and a port really should be one single entity - a `socketport`, with a given port number. You either use them together or not at all.

22. No, I tried it and Sock.Connected() just returns a 1 regardless of whether you have called Connect() or not. It does not apply to UDP and cannot be used like a `get method` to determine whether you previously have called Connect() on a socket. Also UDP doesn't have connections of its own, you have to write that as your own layer/protocol if you want it. All it could really mean is that if the socket is `connected` then you previously assigned a destination IP and Port to it. You should probably implement your own wrapper for Connect() so that you can set a variable to say True if you did successfully call Connect(), so that later you can report whether it is `connected`, but if you are implementing your own actual connection protocol then you may want to do it some other way. Please note that since you can only set the destination IP and Port each time you call Connect(), and possibly it may not be `safe` to try to call Connect() again after it's already been called, you might want to give each client that you *send to* their own socket and port. Then you can just call Connect() once per client and then proceed to send(), rather than constantly switching the port's destination parameters every time you want to send to a different person. If the user changes their IP address you can just close the socket, make a new one on the same port number, bind it, and then Connect() with the new IP and Port. NB: From the test I performed it appears that each time you call Connect() it actually does let you change the destination IP and Port without any errors or problems, but you may still want to give each user their own port/socket for efficiency.

23. No, you can't use RemoteIP() or RemotePort() to get data FROM the sender. It will only return the data that you already provided to Connect(). That means you already knew the IP and Port and set it yourself, so now you are only reading the values that you set, like a `get` method. It does not tell you anything about who sent a packet to you because you can call them even when there are no packets and has nothing to do with any senders. It's to do with who will *receive* data from YOU. With TCP it would actually tell you, however, who the sender of the packets is, based on the TCP packet header, so that when you actually form a `connection` with TCP it will actually figure out the IP address and Port of who is sending to you and then when you call RemoteIP() or RemotePort() it will tell you what that is, rather than who you *thought* you were sending to. You could use TCP for the sole purpose of figuring out who is sending to you, but to do that you have to make a TCP connection, which seems to block the application and cause a delay, plus experiences some lag, so it is not ideal, and may not be necessary if we can get this data from the UDP packet as in the code above. Once you DO get the correct sender's IP and Port, you could call Connect() with it to set the remote IP and Port within your TSocket instance, or could directly write those values to its variables (not sure if that is safe or good practice), so that when you do call RemoteIP() and RemotePort() it would then tell you who the real sender was.

24. The address 127.0.0.1 is a special IP address which creates a `loop back` so that if you as the sender try to send data to that as a destination, it actually doesn't send it out, it just sends it back to yourself. This is useful if you wanted to test your networking code without having anyone to test it with. You could also be running a server and a client on your own machine and get your own server to talk to your own client for testing purposes. But you will still want to test things over the real internet, and not just over a LAN either, because the internet is quite unreliable and problematic.

25. No, when you Connect() you tell the socket who you will be sending packets to and this is all that you'll get back when you call RemoteIP() or RemotePort(). If you use TCP it will actually figure out and tell you what the remote IP and Port are based on the handshaking. But for UDP you are on your own to figure it out.

26. As mentioned already you can use one port for new connections and then give each unique client their own port, bound a separate socket each. Multiple clients can send to the same port which you can receive on, but you can't send to multiple clients from the same port without using a Broadcast, which is not available using TSocket. You'd have to go lower-level to set up a broadcast where, like email, you can send once and have it be received by multiple recipients. To do it yourself with UDP you'll have to call Connect() and then Send() for each user, so it's advisable to give each user their own port so as to not have to do this every time you send. Choose port numbers as mentioned earlier - start at 49152 and go up from there until you find a free port.

27. Yes, one socket is associated with one port only. If you want multiple ports to send from or receive on you need multiple sockets, one per port. Call TSocket.CreateUDP() or CreateUDPSocket(). It will return a new TSocket object so you don't need to do MySocket:TSocket=New TSocket first. Let it create it for you. If it returns Null the socket failed to be created in which case you've gotta decide what to do. I presume the only reason for a socket to fail to be created is that you don't have a network or something. You should probably give one port/socket to each `user` you're connecting with so that each socket has its own buffer to prevent everyone's combined transmissions causing a floor (ie not enough buffer space).

28. Yes if you Close() the socket the port that was Bind()ed to it is now free for being bound to some other socket but any application on your computer. If you do not send anything to the user to tell them you are going to `close` off the communication channel with them, they will not know what to do. The receiver/client will then have to make a decision about what this means to them. They might try resending packets that are not being acknowledged or they might be trying to ping the server to get a response. You might code it so that if there is no response within a time period then you consider the server unavailable. You might try another server if available or just assume that means the connection is closed. Alternatively, just send the client a message saying `you have been logged off` and make sure they receive it by waiting for an acknowledgment from them. Then close the socket.

29. and 30. These are called by other methods such as Connect() to update what IP addresses and Ports are associated with a socket. It appears that these methods (UpdateLocalName() and UpdateRemoteName()) speak to the o/s to find out and alter the local IP address and the remote IP address for a socket. The remote name seems to be read from the socket at a lower level, suggesting this is meant for reading the remote IP based on previous TCP communication/connection with that user. This means it may not do anything useful for UDP. I don't see any reason that you'd want to call either of these for a UDP networking system.

31. I guess this is where everything else begins. There is a lot of other stuff that can be discussed and considered, as others have pointed out. But I hope that the above is enough to clarify the basic use of the TSocket type for use with UDP connections. To summarize, the only methods you are really interested in for UDP are: TSocket.Create(), and then .Bind(), Connect(), Send(), ReadAvail(), Recv(), Close(). Most other methods are to do with TCP and are not relevant to UDP. You might also use the code above to try to find the remote IP and Port from a new packet sent by someone trying to communicate with you, followed by a Connect() to them. Everything else that you want your network to do like forming an agreed-to `connection`, authenticating, flow control, streaming, abstraction of larger data being sent, bandwidth monitoring, ping/lag/throughput etc, and in particular what data you decide to actually send is all up to you. Structure, encryption, checksums, custom headers, compression, etc, whatever you want to add is your choice.

Anyone feel free to add to or correct anything that's been said here as you see fit. I hope this is helpful.

Below is some code which Vertex posted in another thread which acts as a simple type for UDP sending and receiving. It uses some lower level o/s calls which I don't fully understand yet, but it does appear you can get the IP and Port of the remote sender based on the packet, using RecvFrom_(). This code is untested and also extends `socket streams` which may or may not be how you want it to operate, but at least the function calls are of interest.

Type TSocketStreamEx Extends TSocketStream
	Field RecvTimeout : Int
	Field SendTimeout : Int
	Field MessageIP   : Int
	Field MessagePort : Short

	Method SetTimeouts(Recv:Int, Send:Int)
		Self.RecvTimeout = Recv
		Self.SendTimeout = Send
	End Method

	Method Read:Int(Buffer:Byte Ptr, Count:Int)
		Local Read:Int, Size:Int, Result:Int, TempPort:Int

		If Self._socket._socket = -1 Then Return 0
		Read = Self._socket._socket
		If select_(1, Varptr(Read), 0, Null, 0, Null, Self.RecvTimeout) <> 1 Then Return 0
		If ioctl_(Self._socket._socket, FIONREAD, Varptr(Size)) = SOCKET_ERROR_ Then return 0
		if Size <= 0 Then Return 0
		
		Result = recvfrom_(Self._socket._socket, Buffer, Count, 0, Self.MessageIP, TempPort)
		If result = SOCKET_ERROR Or Result = 0 Then Return 0
		Self.MessagePort = Short(TempPort)

		Return Result
	End Method

	Method Write:Int(Buffer:Byte Ptr, Count:Int)
		Local Write:Int, Size:Int, Result:Int

		If Self._socket._socket = -1 Then Return 0
		Write = Self._socket._socket
		If select_(0, Null, 1, Varptr(Write), 0, Null, Self.SendTimeout) <> 1 Then Return 0
		
		Result = sendto_(Self._socket._socket, Buffer, Count, 0, Self._socket._remoteIp, Self._socket._remotePort)
		If result = SOCKET_ERROR Or Result = 0 Then Return 0

		Return Result
	End Method

	Method MessageIP:Int()
		Return Self.MessageIP
	End Method

	Method MessagePort:Short()
		Return Self.MessagePort
	End Method

	Function Create:TSocketStreamEx( socket:TSocket,autoClose=True )
		Local t:TSocketStreamEx=New TSocketStreamEx
		t._socket=socket
		t._autoClose=autoClose
		Return t
	End Function
End Type

Function CreateSocketStreamEx:TSocketStreamEx(socket:TSocket, autoClose=True)
	Return TSocketStreamEx.Create(socket, autoClose)
End Function

Function SocketStreamSocket:TSocket(stream:TSocketStreamEx)
	Return stream.Socket()
End Function

Also this code is said to work for finding the IP address and Port of the sender, while also reading an integer into a bank, written by Rck:

Local sra:Int = SocketReadAvail(servSocket)
If sra > 0 Then
	Local bytes:Int[1]
	Local rip:Int = 0
	Local rport:Int = 0
	recvfrom_(servSocket._socket, bytes, sra, 0, rip, rport)
	Print "from: " + DottedIP(rip) + ": " + rport
	Print "got int: " + bytes[0]
EndIf

Hi ImaginaryHuman,

You sure seem to know a lot about it already..
I adjusted my example a little bit.
I guess you can use one udp-socket to send and recieve. Seems obvious..
Not sure if the ports can be different or not.

Strict

' one udp
Local Udp:tSocket = CreateUDPSocket()
' to recieve
BindSocket (Udp, 49000)
' to send
ConnectSocket (Udp, HostIp ("127.0.0.1"), 49000)

' Send some data
Local b:Byte [] = [Byte(9), Byte(7), Byte(5), Byte(3), Byte(1)]
Udp.send (Varptr b[0], b.length)

' create a loop which waits for some data
While True

	If SocketReadAvail (Udp)
		' a bank to store the input
		Local tmpBank:TBank = CreateBank (1024)
		' size of data-package
		Local length:Int = Udp.recv (BankBuf (tmpBank), tmpBank.size() )
		
		' print data
		For Local i:Int = 0 To length - 1
			Print tmpBank.PeekByte(i)
		Next
		
		' exit this loop
		Exit
	End If

Wend

CloseSocket (Udp)
End

PS the while/wend exits when data is available..

Ghislain
MacIntel 10.5 - BlitzMax v1.28

Ok great, thanks. Yah it seems the port number doesn't matter so long as it's not being used by a registered application currently running and ideally the port numbers should be 49152 or above. You can have ports from 1024 upwards also but these are registered with the port-management people so certain applications may conflict with you. I guess either way you have to keep searching for a port number until you find an open one, and it's not likely that they'll all (or even many) be used.

So long as the computer you are sending to is reading the port number that you are sending to, the message should get there (assuming it doesn't get lost), so the ports for the sender and receiver don't have to be the same. The send-to port has to be the same as the receive-from port for a given computer, but I guess you could also have dedicated send and receive ports or however you want to do it.

You should add some reference to the official Internet Protocol RFCs; that would answer most of your question #2.

The choice of the port is mostly determined by the protocol you want to use or implement. IRC, for example, usually uses 6666, 6667 and 6668. SMTP is on port 25, POP3 on 110, NNTP on 119. HTTP is on 80, and 8080 is usually used for a local HTTP server (for testing purposes) or an HTTP proxy. However, all of these examples are TCP ports, and those are defined in the "Request For Comments" that describe commonly used protocols.

You only need administrator privileges (on Unix machines) to open listening servers on ports 1 - 1023. Clients can always try to open a connection through these ports (otherwise neither your email program nor your web browser would work without administrator privileges).

Once the connection is established, the sockets will eventually communicate through a different port anyway; that is managed by the operating system's sockets layer and completely transparent to the user/programmer.

There might be few reasons why one would want to use UDP rather than TCP, but usually you only want to use UDP when you don't really care for the (possible) responses from any peer computer. Reliable communication in real world applications is always handled via TCP. I guess only multiplayer games or chat clients and servers use UDP for certain (broadcasting) tasks.

UDP only makes sense when you intend to have messages where reliability does not mather as low latency is more important or when you know that the amount of connections would be troublesome (P2P like)

Thanks for the extra info, Winni. I guess it does depend what you intend to use your network for, as to which ports you choose, although mainly I guess I was focussing on just use for games. I'm not so sure about the lack of use of UDP for broadcasting, as many other people seem to be saying that UDP is the way to go for most games these days. I certainly wouldn't want to live with the potential halting of transmission that TCP induces when it stops getting acknowledgments, for example, although otherwise it seems pretty nice. I think mainly if you don't need acknowledgments then UDP is the better choice.

Here is a page documenting the RecvFrom_() function call, if you want to do direct receiving of data from a socket and also get the UDP sender's IP and Port number (I haven't checked yet if the parameters match up with those in Blitz but I presume they do):

http://www.opengroup.org/onlinepubs/000095399/functions/recvfrom.html
http://www.mkssoftware.com/docs/man3/recvfrom.3.asp
http://msdn2.microsoft.com/en-us/library/ms740120(VS.85).aspx?PHPSESSID=gmkmem7inohg2q677fst152mr6

The only part I'm not sure about is the last parameter which is referred to as a `length` and does not mention anything about a port number??

There is some mention of the Select_() function here:

http://support.sas.com/documentation/onlinedoc/sasc/doc750/html/lr2/select.htm
http://www.codersource.net/winsock_tutorial_server_select_model.html
which seems to have something to do with testing if data is available for reading, like ReadAvail()

And here is some info about ioctl_():
http://www.docs.hp.com/en/36430-90007/ch04s45.html