Leadwerks Netwerks

BlitzMax Forums/BlitzMax Programming/Leadwerks Netwerks

JoshK(Posted 2009) [#1]
Here's my take on a networking module. Based off of enet, this features UDP networking with optional reliable communication and packet sequencing. And it's really easy to use. Source included:
http://www.leadwerks.com/files/netwerks.zip

Example:format_code('Strict

Framework brl.eventqueue
Import leadwerks.netwerks
Import brl.standardio

Local host:THost
Local peer:TPeer
Local packet:TPacket
Local message:String="Hi, how are you?"

host=THost.Create()
peer=host.Connect("127.0.0.1",81)

Repeat
Local event:TEvent=Host.WaitEvent(1000)
If event
Select event.id

Case EVENT_CONNECT
peer=TPeer(event.source)
Print "Connected to "+HostName(peer.ip)

Case EVENT_DISCONNECT
peer=TPeer(event.source)
Print "Disconnected from "+HostName(peer.ip)

Case EVENT_PACKETRECEIVED
packet=TPacket(event.extra)
Print "Packet received"
Print packet.ReadLine()

EndSelect
EndIf

If peer
packet=New TPacket
packet.WriteLine(message)
host.sendpacket(peer,packet)
EndIf

Forever')


slenkar(Posted 2009) [#2]
it doesnt werk, only joking, good one


xlsior(Posted 2009) [#3]
Thanks for sharing!

Will take a look at this later.


spacerat(Posted 2009) [#4]
This looks really good, I've been looking for a blitz friendly ENet implimentation for ages. I can see myself using this a lot in the future.

Some constructive comments for if you're still adding to it: it could do with a few more convenience features, for example, Host.Broadcast(packet), a default packet flags global, and some Get/Set functions for the sake of encapsulation. If you're finished with the module though I'll just add them myself.


byo(Posted 2009) [#5]
It looks really easy to use from the source code you posted.
Thanks a lot for sharing.


JoshK(Posted 2009) [#6]
Broadcast sends a packet to all peers?

What do get/set functions do?


Wayne(Posted 2009) [#7]
One may also use enet_host_broadcast() to send a packet to all connected peers on a given host over a specified channel id, as with enet_peer_send().


Brucey(Posted 2009) [#8]
What do get/set functions do?

Really?
They get and set values :-)

It's one of those programming-concept type things some people like spacerat like to use.


plash(Posted 2009) [#9]
It's one of those programming-concept type things some people like spacerat like to use.
Not to mention Brucey, Nilium and the Plash.

Broadcast sends a packet to all peers?
Highly probable.


N(Posted 2009) [#10]
Not to mention Brucey, Nilium and the Plash.
No, I hate 'em. I'd much rather pass around offsets into a global block of memory and read/write as necessary. Also, I decided to save space by removing the null at the end of all my strings in C, and it hasn't broken anything yet. So glad strcpy magically knows how long my strings are.


spacerat(Posted 2009) [#11]
Well, I've added a few things as per my first post. I've not actually tested it but I see no reason for it not to work.

http://pastebin.com/d6ae4ef25


JoshK(Posted 2009) [#12]
Yeah, but what would you want to get and set in this code, besides ip and port, which you would only get?


spacerat(Posted 2009) [#13]
That's basically all you'd need to get. To be honest, it's simply a matter of different programming styles, for example I noticed you had Get functions separate from the types at the bottom, I would have put them in the types instead as methods. The idea of encapsulation like that is to ensure that, if you change the mechanics/back end of a module, the interface stays the same and you don't break anyone's code. And I personally think sticking to methods is neater than using global utility functions.

Anyway, that's not important. The main thing which needed adding was a broadcast method, which I've put in that pastebin. Take a look at it, and if you think it's good, feel free to add it to the module. I also added a little more documentation, e.g. on the structure of the events returned by WaitEvent().


Brucey(Posted 2009) [#14]
And I personally think sticking to methods is neater than using global utility functions.

Ahh, but that's not the procedural style that the official modules lean towards ;-)

And of course, many people who use Blitz are from (or still in) the procedural school of thought, and therefore find it easier to use global functions, than type/methods.


Sanctus(Posted 2009) [#15]
"And of course, many people who use Blitz are from (or still in) the procedural school of thought, and therefore find it easier to use global functions, than type/methods."
Yes but if you come to think of it. The procedural style that the official modules lean towards is looking like procedural while functioning behind as OOP. I still say it's the same thing. And it's nice to have both of them.
I for one write methods but I also write functions. And I use the functions that use the methods. This way I can always use the methods because they are there.


Chroma(Posted 2009) [#16]
Josh, this is really cool. I've decided to go with your code here as opposed to a netlib like RakNet or BNetEx. This is much more simple and faster imo.

But...does it come with sauerkraut? and mustard?

Anyways, do you think it's speed efficient to send strings and then parse them with blah[].split("|")? or is there a faster way?


_Skully(Posted 2009) [#17]
UDP is the only way to go and with reliable UDP thats sweet.. I hate TCP's pauses that happen while it waits for packet confirmation... no problem for turn by turn, but not for a live game.

I can see adding this to TileMax as the entire original intension was to have this networkable...scoop :)

Cheers


TaskMaster(Posted 2009) [#18]
Yes, there is a faster way. Pack your packets. In other words, have a set packet format.

Have the first byte be a header that tells what the packet is. Then the rest of the packet will be determined by the type of packet.

For instance, here is a simple idea...

Packet id 1 might be a status packet, packet id 2 might be a chat packet, etc...

Read the first byte, see what the packet id is. If it is id 1, then you know that the next two bytes are x position, the nnext two bytes are y position, the next two bytes are directiont he player is facing, etc...

If the packet is 2, then the next 2 bytes are a number that tells how many characters are in the chat message. The next two bytes are the id of who the chat message is to go to, and the next bytes are of length equal to the number of characters in the text for the chat...

Well, hopefully you get the general idea.


_Skully(Posted 2009) [#19]
In fact if you can determine the default packet size of the network you can squish multiple updates into one packet... if you didn't already know, sending a 10 byte packet will be padded to at least (I believe) 64 bytes anyway so queuing outbound packets with a default send queue timer will greatly speed up data exchange. Obviously, when an outbound queue is full it would be sent instantly to clear the queue. when I did my old network code for B3D it worked that way... each connection had a send queue.

Cheers


_Skully(Posted 2009) [#20]
Add..
From http://en.wikipedia.org/wiki/User_Datagram_Protocol

[quote]The practical limit for the data length which is imposed by the underlying IPv4 protocol is 65,507 bytes.[/code]


TaskMaster(Posted 2009) [#21]
That's true Skully, but a large packet will actually be broken down into smaller packets, determined by the OS's MTA setting (I think that is it). Anyway, the transmission size limit is usually around 1500 bytes on most OS's.


_Skully(Posted 2009) [#22]
Thats what I meant..

It looks like you are correct for eithernet devices... normally 1500 (which I thought was an odd size originally) but it makes sense...

Windows:
http://technet.microsoft.com/en-us/library/cc737093(WS.10).aspx
The default value is 1280 bytes.

without doing more thorough research, I think we could safely stick within 1280 - 8 for the header so 1272


Chroma(Posted 2009) [#23]
So I guess you're just write the bytes directly to the socket stream then right?


TaskMaster(Posted 2009) [#24]
Depends.

It looks like this LeadWerks Netwerks mod allows you to build the packet my appending to packet, likein the example above, packet.writeline(). Then when you have your packet ready to go, you call host.sendpacket() to send it.

That is essentially how you want to do it. Build your complete packet, then send it.


matibee(Posted 2009) [#25]
I can't believe I missed this thread for two months! I guess I assumed it was LeadWerks engine specific.

For anyone that missed it, I also assume this is being in used in conjunction with Helios' excellent sql/php script (which I followed an implemented in minutes!).

http://www.blitzbasic.com/Community/posts.php?topic=77140#908132

I'm so excited by this I'm going to buy you a new keyboard that has a functional space bar! :D

enetpacket=enet_packet_create(packet._bank.buf(),packet._bank.size(),flags)



Seriously tho, big thanks for sharing.


siread(Posted 2009) [#26]
Can anyone post an example of this working in a gamelike situation? The examples are a bit barebones and don't make much sense to my little brain. I can get a host set up and published to my sql database but how do I go about setting up connections between players and getting feedback?


Chroma(Posted 2009) [#27]
So what's the waitmessage(c,1000)? It will actually sit there and wait for a message for 1 second? I just want it to detect if a packet is there and if not, keep looping.


Mahan(Posted 2009) [#28]

Have the first byte be a header that tells what the packet is. Then the rest of the packet will be determined by the type of packet.



This is a good approach.

If you plan on more advanced networking (allowing higher loads, more players etc.) you can also have an even more elaborate packaging:

Lets say that you use two bytes (a word) at the start of the package then you can use 15 of the bits to determine "message struct type" (> 32000 different combos) and use the remaining bit to indicate if there is another structure following this one.

This allows you to fill up your packages (~1200 bytes as suggested above might be chosen as a default) with several "message structs". If you got a multiplayer game with much going on I think this strategy will pay off in efficiency as fewer packages are sent. This also implies that you have a package queue, where the application prepares messages for sending (and does not sent at once). Disclaimer: I have read about this and just tried it in home-projects briefly. I got no experience of using this in a real heavy networked app.


siread(Posted 2009) [#29]
So what the waitmessage(c,1000)? It will actually sit there and wait for a message for 1 second? I just want it to detect if a packet is there and if not, keep looping.


That's what I was wondering. Shouldn't there be an event queue?


Chroma(Posted 2009) [#30]
Packet id 1 might be a status packet, packet id 2 might be a chat packet, etc...

Read the first byte, see what the packet id is. If it is id 1, then you know that the next two bytes are x position, the nnext two bytes are y position, the next two bytes are directiont he player is facing, etc...

If the packet is 2, then the next 2 bytes are a number that tells how many characters are in the chat message. The next two bytes are the id of who the chat message is to go to, and the next bytes are of length equal to the number of characters in the text for the chat...

Well, hopefully you get the general idea.


Doh, just seen that post. Yeah you have to have a set packet format. I've already been doing it like that for years lol. I don't think there is any other way to do it. I was talking about how to write the data to the stream.

For instance, using the msg.ToCString() way or should I do something like a For/Loop and write each byte to the stream like how BlitzPlay does its? Sorry for not being clear but which prompted your misunderstanding of what I was asking Taskmaster.


JoshK(Posted 2009) [#31]
packet.WriteByte(5)
packet.WriteString("Hello")

So what's the waitmessage(c,1000)? It will actually sit there and wait for a message for 1 second? I just want it to detect if a packet is there and if not, keep looping.

Use 0 for the timeout value if you don't want it to wait.


Chroma(Posted 2009) [#32]
I did use 0 but then I was unable to read any messages. Or at least it appeared that way. Either way, using 0 produced the effect that nothing was being received.


siread(Posted 2009) [#33]
I'm getting my head around this now and it's very nice.

I've noticed though that when a host forces a disconnect with it's peer (or if one app is closed/crashes) the peer thinks that the connection is still open. I figure that if there is a regular ping between host and peer then it could time out the connection if nothing comes back. Any chance of this being included in the mod or should I just create my own reliable ping/pong packets?


siread(Posted 2009) [#34]
What are the benefits of using separate channels for sending packets?


_Skully(Posted 2009) [#35]
Leadwerks,

Are you using this module yourself?

I'm just curious how well it works... I'll be installing network into TileMax in about a month (probably)... I don't want to get too far and find out i messed something for network support... I'll likely run the network as a thread too.


siread(Posted 2009) [#36]
I starting some online testing over the net with this last night in New Star GP.

1 v 1 was smooth as silk but any more racers caused problems. There's lots of optimizing I can do though, so I'm confident I can get some good multiplayer gaming sorted. :)


JoshK(Posted 2009) [#37]
There's an update in the code archive. I started using all integer IPs.


Chroma(Posted 2010) [#38]
Josh, do you know if there's a way to set the UDP socket to non-blocking?

The C command to do that is below. Anyone able to convert?

format_code('
DWORD nonBlocking = 1;
if ( ioctlsocket( handle, FIONBIO, &nonBlocking ) != 0 )
{
printf( "failed to set non-blocking socket\n" );
return false;
}
')


JoshK(Posted 2010) [#39]
No idea.


*(Posted 2010) [#40]
man wish I found this ages ago before I wrote me own one :s


Galaxy613(Posted 2010) [#41]
Is this module free to use? It seems to be part of the Leadwerks Engine now and I don't seen any remarks about a license anywhere.


Dreamora(Posted 2010) [#42]
code is posted in the code archive -> public domain


Galaxy613(Posted 2010) [#43]
Ah, I was hoping so. I forgot he posted it in the code archives also. Thanks for clarifying. :D