-
-
Notifications
You must be signed in to change notification settings - Fork 11
Protobuf
A class wrapping Haxe implementation of Google's Protocol Buffers. It allows one to use this implementation with net sockets.
##Introduction
Being better than XML in some parametres, Protocol Buffer seems to be a good way for those who want to use some kind of lingua franca for data. Although there are just three languages (C++, Python and Java) providing by developers of Protocol Buffers themselves, there are also a lot of third-party add-ons (you can see them here). Here this implementation is used.
##About using Protocol Buffers
To use Protocol Buffers one needs to have two files, .proto file and .bin file. The .proto file is a file representing data formatted as plain text, the .bin file is a compiled binary file that directly used for serialization. The syntax of .proto file is C-like, so you might not face any troubles creating it. Let us consider a simple example:
package example;
message SomeMessage
{
required int32 int1 = 1;
required int32 int2 = 2;
}
Here we defined a package named "example" (meaning of package is just the same as Haxe's one) and also defined a message SomeMessage in it (message can be considered as a structure). The message has two integer fields called int1 and int2, they are annotated with reqiured modifier. Corresponding to Google's tutorial,
- required: a value for the field must be provided, otherwise the message will be considered "uninitialized". If libprotobuf is compiled in debug mode, serializing an uninitialized message will cause an assertion failure. In optimized builds, the check is skipped and the message will be written anyway. However, parsing an uninitialized message will always fail (by returning false from the parse method). Other than this, a required field behaves exactly like an optional field.
- optional: the field may or may not be set. If an optional field value isn't set, a default value is used. For simple types, you can specify your own default value, as we've done for the phone number type in the example. Otherwise, a system default is used: zero for numeric types, the empty string for strings, false for bools. For embedded messages, the default value is always the "default instance" or "prototype" of the message, which has none of its fields set. Calling the accessor to get the value of an optional (or required) field which has not been explicitly set always returns that field's default value.
- repeated: the field may be repeated any number of times (including zero). The order of the repeated values will be preserved in the protocol buffer. Think of repeated fields as dynamically sized arrays.
There is also an opinion that required modifier should not take place at all. As written in the tutorial,
You should be very careful about marking fields as required. If at some point you wish to stop writing or sending a required field, it will be problematic to change the field to an optional field – old readers will >consider messages without this field to be incomplete and may reject or drop them unintentionally.
The fields of the message is of int32 type. There are fifteen scalar types at all: double, float, int32, int64, uint32, uint64, sint32, sint64, fixed32, fixed64, sfixed32, sfixed64, bool, string, bytes.
Note, that number after field definition are not initial value but a unique numbered tag. To set an initial value one should type
required int32 int1 = 1 [default = 10];
Let us consider the example that shows how to use Protocol Buffer. The first of all, you should download protoc and protoc-gen-haxe. Then we create a .proto file in the following way:
package example;
message SomeMessage
{
required int32 int1 = 1;
required int32 int2 = 2;
}
Save it with name Sample.proto.
Next step is converting our .proto file to .bin one. To do it I created a file named CreateProto.cmd and typed in it
protoc Sample.proto --descriptor_set_out=example.proto.bin
Then I added this to pre-build section (I use FlashDevelop IDE, so the path looks like this: Project - Properties - Build - PreBuild Command Line). So, the last step is running it with macro
--macro com.dongxiguo.protobuf.commandLine.Importer.importDescroptorFileSet('example.proto.bin')
(In FlashDevelop the path is Project - Propeties - Compiler Options - Additional Compiler Options). If all is done right, you will see the file named example.proto.bin. In fact, we also have three files called SomeMessage_Builder.cs, SomeMessage_Merger.cs, SomeMessage_Writer.cs (I have .cs files because I set the platform to be C#). These files contain classes that allow to serialize and deserialize a data.
The time has come to see how Protocol Buffer can be used in net interaction. First we are not going to use a Protobuf class for better understanding. So, let us have two .proto files,
package example;
message SomeMessage
{
required int32 int1 = 1;
required int32 int2 = 2;
}
and
package example1;
message SomeMessage1
{
required int32 int1 = 1;
required int32 int2 = 2;
}
In main function we create a server and a client,
var server:SocketServer = new SocketServer(6001);
var client:SocketClient = new SocketClient(6001);
The using of Protocol is based on SomeMessage_Builder class, that represents the structure depicted in SomeMessage. Let us create instances of both classes. First import them and add writers and mergers by using,
import example.SomeMessage_Builder;
import example1.SomeMessage1_Builder;
using example.SomeMessage_Writer;
using example.SomeMessage_Merger;
using example1.SomeMessage1_Writer;
using example1.SomeMessage1_Merger;
and create,
var protoServer = new SomeMessage_Builder();
var protoClient = new SomeMessage1_Builder();
Next we set values in the following way:
protoClient.int1 = 12334;
protoClient.int2 = 345;
To serialize the message we use method writeTo,
var b_out:BytesOutput = new BytesOutput();
protoClient.writeTo(b_out);
Then we just send it as usual:
client.send(b_out);
To receive the data and deserialize the message we use mergeFrom method in onData callback,
server.onData << function(b_in:BytesInput)
{
var inp:LimitableBytesInput = new LimitableBytesInput(b_in.readAll());
protoServer.mergeFrom(inp);
trace(protoServer.int1);
trace(protoServer.int2);
}
Note, that we use here a LimitableBytesInput class so we need to import it,
import com.dongxiguo.protobuf.binaryFormat.LimitableBytesInput;
The output:
Main.hx:58: 12334
Main.hx:59: 345
All the code:
package ;
import example.SomeMessage_Builder;
using example.SomeMessage_Writer;
using example.SomeMessage_Merger;
import haxe.io.BytesInput;
import haxe.io.BytesOutput;
import com.dongxiguo.protobuf.binaryFormat.LimitableBytesInput;
import example1.SomeMessage1_Builder;
using example1.SomeMessage1_Writer;
using example1.SomeMessage1_Merger;
import pony.net.SocketClient;
import pony.net.SocketServer;
class Main
{
static function main()
{
var server:SocketServer = new SocketServer(6001);
var client:SocketClient = new SocketClient(6001);
var protoServer = new SomeMessage_Builder();
var protoClient = new SomeMessage1_Builder();
protoClient.int1 = 12334;
protoClient.int2 = 345;
var b_out:BytesOutput = new BytesOutput();
protoClient.writeTo(b_out);
client.send(b_out);
server.onData << function(b_in:BytesInput)
{
var inp:LimitableBytesInput = new LimitableBytesInput(b_in.readAll());
protoServer.mergeFrom(inp);
trace(protoServer.int1);
trace(protoServer.int2);
}
Sys.getChar(false);
}
}
Let us consider how Protobuf library helps to implement interaction. The example we are going to try to make out will be similar to previous one. We have two instances of Protobuf,
static var protoServer:Protobuf<SomeMessage_Builder, SomeMessage1_Builder>;
static var protoClient:Protobuf<SomeMessage1_Builder, SomeMessage_Builder>;
The messages being used here are the same.
Notice one important thing: the first type in declaring Protobuf is a message we will write to, the second - a message we will merge from.
We initialize them in the following way:
protoServer = new Protobuf<SomeMessage_Builder, SomeMessage1_Builder>(new SocketServer(60001), example.SomeMessage_Writer.writeTo, example1.SomeMessage1_Merger.mergeFrom);
protoClient = new Protobuf<SomeMessage1_Builder, SomeMessage_Builder>(new SocketClient(60001, 3000), example1.SomeMessage1_Writer.writeTo, example.SomeMessage_Merger.mergeFrom);
The constructor has three arguments:
- The first one is a socket, INet (INet is an interface implemented by SocketServer and SocketClient, so one can set the first argument to be either server or client);
- The second one is a writer function being in Writer class;
- The third one is a merger function being in Merger class.
Next we set an onData callback for server,
protoServer.onData.add(function(builder:SomeMessage1_Builder):Void
{
trace(builder.int1);
});
And send a data from client. Note, that sending is not similar to what it is in SocketClient class. We have to set a callback like this:
protoClient.send(function(builder:SomeMessage1_Builder)
{
builder.int1 = 10;
});
The last step, being seemingly the most unnatural, is
Timer.delay(1000/60, DeltaTime.fixedUpdate.dispatch.bind(1000/60));
Since DeltaTime.fixedUpdate is used in Protobuf implementation, we have to do it. Just take it as a matter of course. To use it, we, indeed, need to import Timer and DeltaTime,
import pony.time.DeltaTime;
import pony.time.Timer;
Note, that this is needed only when platform is C#.
The output is:
Main.hx:35: 10
All the code:
package ;
import example.SomeMessage_Builder;
import pony.time.DeltaTime;
import pony.time.Timer;
import example1.SomeMessage1_Builder;
import pony.net.Protobuf;
import pony.net.SocketClient;
import pony.net.SocketServer;
class Main
{
static var protoServer:Protobuf<SomeMessage_Builder, SomeMessage1_Builder>;
static var protoClient:Protobuf<SomeMessage1_Builder, SomeMessage_Builder>;
static function main()
{
protoServer = new Protobuf<SomeMessage_Builder, SomeMessage1_Builder>(new SocketServer(60001), example.SomeMessage_Writer.writeTo, example1.SomeMessage1_Merger.mergeFrom);
protoClient = new Protobuf<SomeMessage1_Builder, SomeMessage_Builder>(new SocketClient(60001, 3000), example1.SomeMessage1_Writer.writeTo, example.SomeMessage_Merger.mergeFrom);
protoServer.onData.add(function(builder:SomeMessage1_Builder):Void
{
trace(builder.int1);
});
protoClient.send(function(builder:SomeMessage1_Builder)
{
builder.int1 = 10;
});
DeltaTime.fixedDispatch();
Sys.getChar(false);
}
}
There are two functions with similar functionality, send and queue. The difference between them is that send builds "solid" structure and sends it whole then, while queue forms a queue of a data and sends each element of it independently.