Synchronization Mechanisms and Yahoo!

From the title you must have guessed that this one is going to be a damn technical entry. To be fair enough, if you are not a computer geek you might have some difficulty in getting to the point which I am trying to make. But then you can always google up and learn things which you don't know.

Anyway I assume that the reader has some idea of synchronization mechanisms used by various operating systems (another comp jargon). I am listing the ones I know and use frequently:

• Events
• Semaphores
• Pipes and Message Queues
• Critical Sections
• Mutex

When I was in college, I didn't understand any of this hifi stuff except Pipes. Thanks to STMicroelectronics, I have understood all of them during my job here as a software engineer.

Hey what about the Yahoo! in the title. Just wait a little longer.

We were taught all these synchro mechanisms in Operating Systems course during our B. Tech. But the way of teaching there was highly orthodox. For example, we had Producers & Consumers associated with a message queue, and there was the highly infamous Dining Philosophers problem. I understood literally nothing. No practical code samples were shown about the application of these mechanisms. Luckily I knew something about Pipes from own Unix experience there. And I could manage to pass the course with a C grade.

Let's come to the Yahoo! part. During our final year (2001-2002), I and my friend Dhar started designing a Yahoo! Messenger client for Unix systems (actually Linux). There wasn't any good client for Linux at that time. There was GAIM, but it didn't support HTTP Proxy interface with GET and POST requests. There were other clients (one of them was the official beta from Yahoo! itself) which supported the HTTP Proxy, but they used to hang in the middle of some operation (damn frustrating!@#%$%).

Dhar started analyzing the messenger protocol by studying data dumps from the windows IM client (how he got those dumps is another story and it wont be revealed to the public for security reasons) and we began with a small console application, which would log on Yahoo!, list your buddies and logout. Then we added functionality to send a message to a buddy. Next we added the ability to add buddies. But the console application was not interactive. It was used to test whether we had decoded the protocol properly or not. So to test any feature we would code that part and then run the program. It would log on, execute the feature, logout and then end itself.

In a course of 3 months we had decoded enough of the YMSG protocol to be able to write a fairly good IM client. So we thought why not have a full fledged GUI application. Let me tell you that writing GUI stuff in Linux/Unix X11 environment was a hell lot difficult in those days. You didn't have the equivalent of Microsoft Visual C++ (Windows) in Linux. A lot of thanks to guys at Trolltech, who developed Qt, the best multi-platform GUI Toolkit in C++. We selected Qt and I started developing GUI stuff. Some of the fancy art work (icons, progress bar, visual UI) was designed by Dhar.

We had only one design goal to meet: Our app should not hang!! Of course, another goal was to have sufficient features so that people could actually use it. In order that the application does not hang, we had to ensure that all operation were non blocking in nature. This restraint meant that user interaction should not interfere with network operation. We had the following approach:

• We had one queue (normal queues implemented through linked lists, not message queues) to store the messages to be sent from the user to the network. When user sent a message, there were not delivered directly to the network (otherwise the GUI would hang for the time this message is being sent). Let's call this queue Send.
  
• Another queue (call it Receive) was used to store the message received from the network.
• We had a function, say SendToNetwork(), to get message from the Send queue and deliver it to network.
• Another function, say ReceiveFromNetwork(), used to put messages in the Receive queue.
  
• Hell, we had one more function, say GetMessage(), which used to get messages from the Receive queue and show it on the screen.

The strategy we had to use was to somehow keep these functions independent of one another, so that one did not interfere with another and they have to keep doing their work in a while loop unless user exits the program. (You must be thinking to put them in three different threads, but sorry! we didn't have any idea of multi-threading on Linux).

Again, Qt came to our help. It has a concept of timer, through which you can call a function when the timer expires. We used 3 timers for these three functions which ensured that these will be called at regular intervals of 10, 50 and 100 millisecs. So these functions were sort of running concurrently and doing their job without any need of multithreading.

Needless to say, the public was impressed by the nice GUI and the special feature of multiple logins through a single PC (actually it is not a big deal, the big deal is to prevent multiple logins). We developed the client to include a lot of features including multiple profiles for a single login, getting buddy names from Yahoo! addressbook, Yahoo! smileys in all their glamour, typing notification, and notably the most complex file transfer. And our design goal was met. The application never hung in the midst of any damn operation. The user could always give some input and get a response from the system. It was much better than any Yahoo! client available on Linux.

What's the connection between synchro mechanisms and Yahoo! Messenger as the title of the post suggests? Well, that will be continued in the next post to keep the length of this entry manageable.