Structuring Data

The developer I was pairing with today had an interesting question. We were working on a piece of code that reads a set of messages in increasing order of priority. This priority depends upon two variables. The date that the message was played last, the total number of times the message was ever played. Looking at this piece of code, the developer asked me the data structure that is used for implementation. My response was, “Err, list”.

Thinking back, the question is an interesting one for many reasons. One, we hardly come across any badass data structure programming when building enterprise applications (at least the kinds that I’m currently working on). So I thought to myself, hang on, whats the DS courses in colleges worth. I quickly understood that this was a stupid line of thought. As Stroustrup points out, “Craftsmen must understand their tools and not believe in magic”. Knowing fundamentals of data structures are important, that way one would have noticed that we were indeed able to use lists for message prioritisation  because we were already filtering out a huge number of messages at the database layer.  Had we not filtered out messages at the database layer, we would have had too much data to sort in memory, which would lead to all sorts of problems. The database in turn, uses B+ trees to make this process efficient.

On the other hand, Its my belief that the choice of data structure to use is not the first question that has to be asked when solving a problem. A data structure by definition is a construct that adds meaningful meta information to a data set so that the desired function could be written in terms of the meta data added by the data structure. For example, a binary search tree adds value by providing meta information which divides the current data set such that, a function to search a value will converge faster. Thus a data structure merely helps in faster convergence of a solution. Identifying the parameters that affect the outcome, and more importantly, isolating those that do not affect the result add much more value by helping us model the problem space.

In summary, data structures are considered to be divine answer to life, universe and everything, which they are not. A data structure, merely adds meta information to the solution space which helps us to solve the problem.

Quote for the day,

“The cause of fear is ignorance.” – Seneca.

Advertisements

2 comments

  1. If one dint model the traveling salesman problem using graphs, and used plain old lists instead, would the salesman get anywhere at all, i wonder..

    • Yes. The graphs in this case provide the very essential meta information about the relationship between nodes, which will be very difficult to describe using lists. Though I wonder if there is any other meta information out there which will reduce the complexity of the problem


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s