Innovation rarely happens in a vacuum. It usually starts with an individual brave enough to contribute an idea and a team inspired enough to make it great. This blog provides a forum for all Centrons to contribute ideas, make suggestions, ask questions and inspire others. There are no boundaries. To participate, all you need is the desire to build great products.

Understanding OLTP Distributed Data Storage Systems

Ever wonder why it takes so long to build a distributed system and why most people don’t get it right? You’re not alone. Why is implementing a distributed data management system especially for OLTP purposes so complex?

When building an OLTP system, the first item to consider is consistency. Consistency in this article talks about mutual consistency. Mutual consistency is consistency obtained across servers. Traditional single server applications do not have to worry about mutual consistency. These applications only have to consider consistency in the context of correctness mostly related to transactions. Deciding what consistency models the application wants to use is a tough job in itself. Most vendors offer options for eventual consistency and/or strong consistency.

First let’s talk about what each option means through examples and then I’ll explain how the requirement for consistency impacts concurrency and availability of the application.

Consistency

Case 1: Both Read and Write are Eventually Consistent

If a write is sent to the system, the system sends a command to replicate to the other servers, i.e., write sent to server 3, replicated to server 2 and server 1. However as soon as an ACK is received from the initial server, the client obtains an acknowledgement that the operation succeeded. However while the operation did not succeed on server 2 and server 3 yet, if a read operation is triggered, it can read the value from server 1 and get an older value.

Case 1

Case 2: Both Read and Write are Strongly Consistent

The write will return to the client only if all the replicas have the new value. For read, the value read will always be the value, which is committed across replicas. If a write operation is going on while read is triggered, then the read will wait for the write to finish.

Case 2

Case 3: Read is Eventually Consistent and Write is Strongly Consistent

The write will commit only if all the replicas have the same value. If a read is triggered while write is going on, then the read will deliver the last committed value.

Case 3

Case 4: Read is Strongly Consistent and Write is Eventually Consistent

The write will return as soon as one server gets the value. If read is triggered while write is updating that value, the read will wait until all the replicas get the value.

Case 4

Concurrency

The consistency decision in an application heavily impacts concurrency of the system. If the system is eventually consistent, there is no waiting for operations and hence improved concurrency. This leads to improved throughput. If the consistency in the system is strong, then a lot of time is spent waiting for other operations to finish, which reduces the concurrency of the system considerably. This leads to decline in throughput of the system.

Availability

The consistency decision also impacts availability of the system. CAP theorem proves that consistency, availability, and partition tolerance cannot all exist at the same time. Partition tolerance is assumed in a distributed system. That means a user has to choose between a completely consistent system and availability of the system. Having strong consistency can impact availability of the system by making the system unavailable to clients. Having eventual consistency configured in the system can allow accessibility to the system from the client even in the case of partition failure and allow for system availability even though the data in one partition is not consistent with data in another partition at a particular point in time.