fsync() is often more expensive than it absolutely needs to be. The easiest way to implement it is to force out every outstanding write to the file system, regardless of whether it is a journaling file system, a COW file system, or a file system with no crash recovery mechanism whatsoever. This is because it is very difficult to map backward from a given file to the dirty file system blocks needing to be written to disk in order to create a consistent file system containing those changes. For example, the block containing the bitmap for newly allocated file data blocks may also have been changed by a later allocation for a different file, which then requires that we also write out the indirect blocks pointing to the data for that second file, which changes another bitmap block… When you solve the problem of tracing specific dependencies of any particular write, you end up with the complexity of soft updates. No surprise then, that most file systems take the brute force approach, with the result that fsync() commonly takes time proportional to all outstanding writes to the file system.

Thinking for a while… Is not hard to implement fsync(), to flush just one file using Delayed Allocation (Allocate on Flush). We’ve all new data in memory, and old data stay on its block. So, modification in place means that you’ve just to flush blocks. Append means that you need to allocate something.
The image above, is a little bit old, but it’s the original idea of the RaleighFS in Memory Structure. There’re general information like super-block, bad blocks list and free blocks lis, current cache size and some other things. But today I’m focusing on Block Cache and Write Items Cache.

When you open a file, you load its metadata in memory then when you need file content you load it in the Block Cache. RaleighFS data block contains the File Key, so you can easily find blocks with specified key, but also you can easily find your file blocks using pointers.

So, why is easy to fsync() only the specified file with Delayed Allocation:

• Modification in place, requires just a scan of the block cache to find what blocks are to flush (and obviously metadata)
• Append to file, has all new data in memory and there’re no Modification to B*Tree(s) or Free blocks until you flush something.
• Remove is just a command, and when fsync() is called all the delete operation on B*Tree(s) and list will take places.

But remember, syncing just one file is not a good idea, Trust your File-System’s flush policy!

Following the ideas of Valerie Aurora of Repair-driven File System Design, I’ve decided to add for each block (B*Tree and Data blocks) an head that contains a Magic Number and a CRC Sum of the block. In this way you can easily identify what kind of block you’ve peeked without scanning all metadata. Another step is to add a back reference (or back pointer) to the data block, in this way you can easily jump back to it’s the extent block (and obviously to its OID) so you can easily understand what is the Object owner of this block and you can easily swap two blocks reading at most 4 blocks (2 Data and 2 Extends).

Another idea stolen from Valerie is to double the metadata blocks with a COW-like approach, as explained in this paper “Double the Metadata, Double the Fun: A COW-like Approach to FS Consistency“, really useful for personal file-systems but maybe less in a distributed file-system. I’m working on it adding only as an mkfs option.

When the source Code will be online? I don’t know.. I’ve less time to work on it. Maybe at the end of this year I’ll publish the File-System and the Distributed System (explained some posts ago).

struct blkcache_buf {
    struct blkcache_buf *  next;    /* Next Queue Item */
    struct blkcache_buf *  prev;    /* Prev Queue Item */
    struct blkcache_buf *  hash;    /* Next Item with the same hash */

    xuint16_t              count;   /* Retain count */
    xxx_block_t            block;   /* Cached Block */
};

typedef struct {
    struct blkcache_buf ** buf_hash;        /* Bufs Hashtable */
    xuint16_t              buf_hash_size;   /* Bufs Hashtable Size */
    xuint16_t              buf_used;        /* Bufs in use */

    struct blkcache_buf *  head;            /* Head of the Bufs Queue */
    struct blkcache_buf *  tail;            /* Tail of the Bufs Queue */

    xxx_device_t *         device;          /* Block Device used for I/O */
} xxx_blkcache_t;

Above, you can see the cache data structure and below the core of the cache Algorithm.

#define BUFHASH(cache, blocknr)     ((blocknr) % (cache)->buf_hash_size)

xxx_block_t *xxx_blkcache_read (xxx_blkcache_t *cache,
                                xxx_blkptr_t blocknr)
{
    struct blkcache_buf *buf;
    xuint16_t hash_index;

    /* Scan the hash chain for block */
    hash_index = BUFHASH(cache, blocknr);
    if ((buf = __blkcache_find(cache, blocknr, hash_index)) != NULL) {
        buf->count++;

        /* Move Buf far from head */
        __blkcache_buf_shift(cache, buf);

        return(&(buf->block));
    }

    /* Cache is Full, Remove one Item */
    if ((cache->buf_used + 1) > cache->buf_hash_size) {
        /* All buffers are in use */
        if (cache->head->count > 0)
            return(NULL);

        /* Remove Least-Frequently Used */
        buf = __blkcache_remove_lfu(cache);
        cache->buf_used--;
    }

    /* Desidered block is not on available chain, Read It! */
    if ((buf = __blkcache_buf_alloc(cache, buf, blocknr)) == NULL)
        return(NULL);

    /* Add One Use, Block cannot be removed */
    buf->count++;

    /* Go get the requested block unless searching or prefetching. */
    __blkcache_readwrite(cache, buf, RFALSE);

    /* Update Cache Hash */
    cache->buf_used++;
    buf->hash = cache->buf_hash[hash_index];
    cache->buf_hash[hash_index] = buf;

    /* Update Cache Queue */
    if (cache->head == NULL) {
        cache->head = cache->tail = buf;
    } else {
        buf->prev = cache->tail;
        cache->tail->next = buf;
        cache->tail = buf;
    }

    return(&(buf->block));
}

You can download a demo implementation here: lru-block-cache.c, but I’m waiting some ideas or suggestions to improve (or change radically) the Cache Algorithm. Thanks in advance!

In the most cases you need to reimplement a Protocol to embed it in your application, and sometimes you have to reimplement two/three protocols that does the same job like IM Protocols (XMPP, AIM, Yahoo…).

A better solution, that avoid you to reinvent the wheel is to use an existent library to handle the protocol(s) that you need, and build an Abstract Interface, with your data format, that allows you to use a generic way to communicate between various provider. Below you can find a graphical example of what I mean.

You can have many providers, written in different languages. These providers talk with the Notification Service providing an abstract interface for the Apps. In this way, the end App has just to say “Write a Mail To X”, “Download Todays Mail”, “Send an IM to X”… and you can intercept notification to displays as a popup on your desktop… or something similar.

This solution will be used in MokoTouch Project, to provide Core Services to the Apps. For more information send me a mail.

The rendering result is an aggregation of components (or better, Group of Components) that can be rendered independently of each other in a  process because each components has its own input data and until the aggregation process starts there’re no dependencies between components. In a few words, foreach input data I’ve to call a Render method that returns the “computed” data, that at the end will be aggregated with all the computed values.

So, how speed-up the rendering? The standard answer is using Threads, to take advantage of multi-core system. But I need lots row power and lots of core to be fast enought. Another solution that doesn’t require to spent a lot of money for a faster computer it to distribute the computation across different machines. And this is my attempt…

The Master receive an array of elements as input and splits up it according to the number of the available Nodes (Machines), and foreach node assigns its sub-array and sends an “informative message” to the node.

The Node waits for an “informative message” and when it has received it, starts its computation. Foreach Data Item that fetch runs the Computation and sends back the result.

The Master decompose data into equal-size partitions, so each node has  an equal-size queue to process, but if someone finishes its job and there’re more data to process (in someone else queue) The Master dequeue couple of items from the slower node queue and add to the queue of the one that has finished its job. In this way you’ve done your computation faster then ever, and if one machine crashes (slow one case) its job will be taken by someone else that has finished its own.

And this is just a bit of General Theory, but the implementation is really simple. Maybe I’ll try to reimplement something more generic in the near future (October, November) when I’ll have a bit more of free time. If you’ve any question send me a mail!

Reliable Transmitter:

The schema below shows the operations that reliable transmitter has to do. “Red” are “error or unexpected operations”, Green are “expected operations”, Blue are TX Operations.

Transmitter send a packet and waits for the ACK packet (sended by receiver). If ACK doesn’t arrive before the Tx Timeout packet is resended. When ACK arrive Tx send the CACK (Confirm ACK) packet and waits for a timeout, if another ACK Arrive TX resend the CACK, this because receiver doesn’t have the confirm of transmitter. When TX timeout all is done, and the packet is sended correctly.

Reliable Receiver:

The schema below shows the operations that reliable receiver has to do. “Red” are “error or unexpected operations”, Green are “expected operations”, Blue are Rx Operations.

When RX receive a Packet sends an ACK, and waits for the CACK, if CACK doesn’t arrive before timeout RX resend the ACK. When CACK arrive all is done. We’ve received the packet and the transmitter knows that we’ve received it.

I’m using C, so today a little Example of SSL Client/Server written in C using OpenSSL, I’ve written a small wrapper for SSL Socket, and here is How to use it.

/* SERVER CODE
 * ==============================
 */
SFSocketGlobalInit();       /* Initialize SSL */

/* Alloc Socket, Initialize SSL and Listen */
SFSocket *socket = SFSocketAlloc();
SFSocketInit(socket, CA_FILE, DH_FILE, KEY_FILE, KEY_PASSWORD, NULL);
SFSocketListen(socket, INADDR_ANY, PORT);

do {
   SFSocket *clientSocket;
   char buffer[64];
   int rdSize;

   /* Accept Client Connection */
   if ((clientSocket = SFSocketAccept(socket)) == NULL)
       break;        

   /* Read Data from Client */
   if ((rdSize = SFSocketRead(clientSocket, buffer, 64 - 1)) > 0) {
       buffer[rdSize] = '\0';
       printf("Client: %s\n", buffer);
   }

   /* Write to Client */
   strcpy(buffer, "Hello Client!");
   SFSocketWrite(clientSocket, buffer, strlen(buffer));

   /* Disconnect Client */
   SFSocketRelease(clientSocket);
} while (1);

/* Close and Release Socket Resources */
SFSocketRelease(socket);

Above you’ve the simplified server code (without error check!) and below you’ve the client code. The client try to connects to server, send an “Hello” message and the server reply with other greetings.

/* CLIENT CODE
 * ==============================
 */
SFSocketGlobalInit();       /* Initialize SSL */

/* Alloc Socket, Initialize SSL */
SFSocket *socket = SFSocketAlloc();
SFSocketInit(socket, CA_FILE, NULL, KEY_FILE, KEY_PASSWORD, NULL);

/* Connect to Host */
SFSocketConnectToHost(socket, HOSTNAME, PORT);

/* Send Message to Server */
char buffer[64];
strcpy(buffer, "Hello from Client!");
SFSocketWrite(socket, buffer, strlen(buffer));

/* Read Message from Server */
if ((rdSize = SFSocketRead(socket, buffer, 64 - 1)) > 0) {
   buffer[rdSize] = '\0';
   printf("Server: %s\n", buffer);
}

/* Close and Release Socket Resources */
SFSocketRelease(socket);

Remember that you need to generate, at least, the Authority Certificate, Server Certificate and Clients Certificates. and here is How to do it.

- AUTHORITY Certificate:
 openssl genrsa -des3 -out ca.key 1024
 openssl req -new -x509 -key ca.key -out ca.crt

- SERVER Certificate
 openssl genrsa -des3 -out server.key 1024
 openssl req -new -key server.key -out server.csr
 openssl x509 -req -in server.csr -out server.crt -sha1 \
                   -CA ca.crt -CAkey ca.key -CAcreateserial 

- CLIENT Certificate
 openssl genrsa -des3 -out client.key 1024
 openssl req -new -key client.key -out client.csr
 openssl x509 -req -in client.csr -out client.crt -sha1 \
                   -CA ca.crt -CAkey ca.key -CAcreateserial

The Full Source Code is Available Here: SSL Client/Server Example Source Code.