[TIP] Generic Binary Format

I Love use Binary formats, instead of XML, and JSON. Here is my Generic Binary Format for data transmissions or serializations. Data is composed by three blocks. The first one 1byte that describe all information about the object, like “is a single Int object” or “is a list”, then tells you the second block length. The second block contains the size of the third block (The Data-Block) or the Number of Items in List.

Unified Notification Service: Avoid The Wheel Reinvention

Every programmer loves to reinvent the wheel, and reinventing the wheel is still my primary hobby. Sometimes you need to reimplement a Network protocol to use with your favorite language/library, sometimes is only for fun, but if you’re in the Business World maybe is “better” (faster) to use one of the thousand existing libraries.

In the most cases you need to reimplement a Protocol to embed it in your application, and sometimes you have to reimplement two/three protocols that does the same job like IM Protocols (XMPP, AIM, Yahoo…).

A better solution, that avoid you to reinvent the wheel is to use an existent library to handle the protocol(s) that you need, and build an Abstract Interface, with your data format, that allows you to use a generic way to communicate between various provider. Below you can find a graphical example of what I mean.

You can have many providers, written in different languages. These providers talk with the Notification Service providing an abstract interface for the Apps. In this way, the end App has just to say “Write a Mail To X”, “Download Todays Mail”, “Send an IM to X”… and you can intercept notification to displays as a popup on your desktop… or something similar.

This solution will be used in MokoTouch Project, to provide Core Services to the Apps. For more information send me a mail.

A Bit of Distributed Computation…

In the last months I’ve worked on various 2D rendering projects, that requires lots of row power to be executed in smallest time as possible.

The rendering result is an aggregation of components (or better, Group of Components) that can be rendered independently of each other in a  process because each components has its own input data and until the aggregation process starts there’re no dependencies between components. In a few words, foreach input data I’ve to call a Render method that returns the “computed” data, that at the end will be aggregated with all the computed values.

So, how speed-up the rendering? The standard answer is using Threads, to take advantage of multi-core system. But I need lots row power and lots of core to be fast enought. Another solution that doesn’t require to spent a lot of money for a faster computer it to distribute the computation across different machines. And this is my attempt…

The Master receive an array of elements as input and splits up it according to the number of the available Nodes (Machines), and foreach node assigns its sub-array and sends an “informative message” to the node.

The Node waits for an “informative message” and when it has received it, starts its computation. Foreach Data Item that fetch runs the Computation and sends back the result.

The Master decompose data into equal-size partitions, so each node has  an equal-size queue to process, but if someone finishes its job and there’re more data to process (in someone else queue) The Master dequeue couple of items from the slower node queue and add to the queue of the one that has finished its job. In this way you’ve done your computation faster then ever, and if one machine crashes (slow one case) its job will be taken by someone else that has finished its own.

And this is just a bit of General Theory, but the implementation is really simple. Maybe I’ll try to reimplement something more generic in the near future (October, November) when I’ll have a bit more of free time. If you’ve any question send me a mail!

Reliable Protocol on Unreliable Channel

When you work with Unreliable communication channel (like UDP), you need to implement your own “Reliable Mechanism” to be sure that packet that you’ve send was received correctly. Here there’s my attempt.

Reliable Transmitter:

The schema below shows the operations that reliable transmitter has to do. “Red” are “error or unexpected operations”, Green are “expected operations”, Blue are TX Operations.

Transmitter send a packet and waits for the ACK packet (sended by receiver). If ACK doesn’t arrive before the Tx Timeout packet is resended. When ACK arrive Tx send the CACK (Confirm ACK) packet and waits for a timeout, if another ACK Arrive TX resend the CACK, this because receiver doesn’t have the confirm of transmitter. When TX timeout all is done, and the packet is sended correctly.

Reliable Receiver:

The schema below shows the operations that reliable receiver has to do. “Red” are “error or unexpected operations”, Green are “expected operations”, Blue are Rx Operations.

When RX receive a Packet sends an ACK, and waits for the CACK, if CACK doesn’t arrive before timeout RX resend the ACK. When CACK arrive all is done. We’ve received the packet and the transmitter knows that we’ve received it.

OpenSSL: SSL Client/Server Example

I’m experimenting a bit with the new iPhone 3.0 SDK, Core Data, Game Kit, Push Notification Service and so on…

I’m using C, so today a little Example of SSL Client/Server written in C using OpenSSL, I’ve written a small wrapper for SSL Socket, and here is How to use it.

/* SERVER CODE
 * ==============================
 */
SFSocketGlobalInit();       /* Initialize SSL */

/* Alloc Socket, Initialize SSL and Listen */
SFSocket *socket = SFSocketAlloc();
SFSocketInit(socket, CA_FILE, DH_FILE, KEY_FILE, KEY_PASSWORD, NULL);
SFSocketListen(socket, INADDR_ANY, PORT);

do {
   SFSocket *clientSocket;
   char buffer[64];
   int rdSize;

   /* Accept Client Connection */
   if ((clientSocket = SFSocketAccept(socket)) == NULL)
       break;        

   /* Read Data from Client */
   if ((rdSize = SFSocketRead(clientSocket, buffer, 64 - 1)) > 0) {
       buffer[rdSize] = '\0';
       printf("Client: %s\n", buffer);
   }

   /* Write to Client */
   strcpy(buffer, "Hello Client!");
   SFSocketWrite(clientSocket, buffer, strlen(buffer));

   /* Disconnect Client */
   SFSocketRelease(clientSocket);
} while (1);

/* Close and Release Socket Resources */
SFSocketRelease(socket);

Above you’ve the simplified server code (without error check!) and below you’ve the client code. The client try to connects to server, send an “Hello” message and the server reply with other greetings.

/* CLIENT CODE
 * ==============================
 */
SFSocketGlobalInit();       /* Initialize SSL */

/* Alloc Socket, Initialize SSL */
SFSocket *socket = SFSocketAlloc();
SFSocketInit(socket, CA_FILE, NULL, KEY_FILE, KEY_PASSWORD, NULL);

/* Connect to Host */
SFSocketConnectToHost(socket, HOSTNAME, PORT);

/* Send Message to Server */
char buffer[64];
strcpy(buffer, "Hello from Client!");
SFSocketWrite(socket, buffer, strlen(buffer));

/* Read Message from Server */
if ((rdSize = SFSocketRead(socket, buffer, 64 - 1)) > 0) {
   buffer[rdSize] = '\0';
   printf("Server: %s\n", buffer);
}

/* Close and Release Socket Resources */
SFSocketRelease(socket);

Remember that you need to generate, at least, the Authority Certificate, Server Certificate and Clients Certificates. and here is How to do it.

- AUTHORITY Certificate:
 openssl genrsa -des3 -out ca.key 1024
 openssl req -new -x509 -key ca.key -out ca.crt

- SERVER Certificate
 openssl genrsa -des3 -out server.key 1024
 openssl req -new -key server.key -out server.csr
 openssl x509 -req -in server.csr -out server.crt -sha1 \
                   -CA ca.crt -CAkey ca.key -CAcreateserial 

- CLIENT Certificate
 openssl genrsa -des3 -out client.key 1024
 openssl req -new -key client.key -out client.csr
 openssl x509 -req -in client.csr -out client.crt -sha1 \
                   -CA ca.crt -CAkey ca.key -CAcreateserial

The Full Source Code is Available Here: SSL Client/Server Example Source Code.