Review for Paper: 18-Dynamo: Amazon's Highly Available Key-value Store

Review 1

Most commercial DBMSes are designed on the relational model and assume that users desire a high level of consistency, if the database is to be stored in a distributed way. Unfortunately, these DBMSes cannot provide high availability due to their strict consistency controls. In a distributed system with frequent network or node failures, it is not possible to provide high availability and consistency at once. For example, a large online retailer might want users to be able to update their shopping carts even if there is a partial network outage, which could lead to inconsistent versions of the shopping cart on different server nodes. A new DBMS model is needed to provide always-on services.

The authors of “Dynamo” present Amazon's solution, a NoSQL DBMS called Dynamo that is designed to be highly available, allowing clients to read and write data even under partial network failure. Dynamo is an eventually consistent database, which implements a key-value store as a distributed hash table. A key design goal is to allow heterogeneous nodes to be added to or removed from the cluster, without requiring manual reconfiguration. Dynamo accomplishes this goal with consistent hashing. In this scheme, each key in the database is hashed to value in a circular “ring” domain. Each node that joins the cluster is randomly assigned a set of virtual locations in the ring, and it becomes responsible for storing key-value pairs whose keys hash to a value between one of these locations and the next neighbor node's position.

The paper contributes a novel key-value store that had been in use by Amazon for about two years at the time of writing. The authors describe the implementation of the DBMS, focusing on the schemes for allocating regions of the hash space to servers, and on consistency control. For example, the paper discusses a novel system of “vector clocks” the authors use to check which of two tuples is the ancestor, or if they are in conflict.

One limitation of Dynamo is that the system used for anti-entropy in the routing tables does not scale well. Dynamo has been used in clusters of a few hundred servers, but the authors admit that its gossip-based routing table distribution would be too expensive if there were thousands of nodes. Currently, Dynamo nodes periodically forward the cluster's entire routing table to each other, to share knowledge about changes in connectivity.

Review 2

The paper presents the design and implementation of Dynamo, Amazon’s Highly Available Key-value storage system giving an “always-on” experience to the user. This system is being utilized by many of Amazon’s services which are willing to trade-off consistency for high availability. Amazon’s systems handle millions of customers at peak times and thousands of commodity machines are being used to cater to hundreds of services. The system was developed with the point in mind that even the slightest outage of Amazon’s systems has significant financial consequences and impacts customer trust. The concepts of high-availability, scalability, failure tolerance and manageability are addressed by the system.

Dynamo assumes a query model with simple read/write operations to data with unique IDs. Eventual (weaker) consistency is supplied by the system. It targets optimization for 99.9% of the queries using a service-oriented architecture. It uses decentralized systems to avoid single points of failure. It uses consistent hashing method to partition the data and creates a preference list of N nodes to aid in replication. Use of virtual nodes helps in recovery along with the use of data versioning to achieve eventual consistency. It handles failures with the help of Hinted handoffs and Merkle Trees (Hashing trees to recover from permanent failures).

The paper is successful in explaining the design and implementation of Dynamo. It shows incremental scalability and is able to handle routine failures efficiently. The evaluations provided by the authors are fruitful in exhibiting Dynamo’s high efficiency and fast recovery capabilities along with ease of archival.

The system provides an inflexible way to resolve conflicts among applications. It doesn’t address security issues and is only applicable for in-house development of a closed-network group.

Review 3

What is the problem addressed?
This paper show that design and implementation of Dynamo, a key-value storage system of Amazon. One big difference between Dynamo and previous work is that Dynamo put the availability at the top priority and sacrifices consistency under certain failure scenarios. Dynamo adopt an eventual consistency model and several techniques to handle different problem.
Why important?
For a world-wide e-commerce platform, there are strict operational requirements arises: performance, reliability efficiency and scalability. Their work is important because of its specific needs: a single purpose key-value pair storage system focusing fully on availability. However most of the existing methods are either too complicated and costly or focusing on strict consistency and sacrificing availability.
1-­‐2 main technical contributions? Describe.
Thought most of the algorithms are not original, they apply such algorithm to the real application, and I consider it to be one of the largest contributions. One important contribution is the partitioning algorithm, a variant of the consistent hashing. They use virtual node to partition load to have uniform load distribution in case of outages. Such separation of responsibility and execution makes it much easier to handle the node failure and soon come back problem. The paper also describes an interesting way of comparing two replicas namely Merkle trees, which requires only hash values and parts of the tree to be transferred to determine if there is an inconsistency thereby reducing network traffic.
1-­‐2 weaknesses or open questions? Describe and discuss
Dynamo is also developed for a very specific setting and hence does not consider security in its framework. The number of applications that might be able to use this implementation is limited by this.

Review 4

cannot upload...

cannot upload...

cannot upload...

Review 5


Review 6

This paper introduces a new design of Dynamo which is a highly available key-value storage system. It is designed inside Amazon, and some core services use it to provide an “always-on” experience. Dynamo is designed to be easily scale up and down, and it allows service owners to customize their storage system to get desired performance, durability and consistency Service Level Agreements (SLAs). This paper first provides related work and background, then it presents the system design and implementation of Dynamo. Following that is the experience and lessons learned during running Dynamo in production.

The problem here is that Amazon encounters a big challenge about how to ensure reliability at massive scale. Even a slight outage could bring a huge damage and impact user’s experience. Thus there are strict operation requirements on performance, reliability, efficiency and the ability to be hight scalable. Therefore, Amazon has developed several storage techniques such as Amazon Simple Storage Service (Amazon S3). Dynamo is the new developed design, and it provides a simple primary-key only interface for many services in Amazon’s platform which only need primary-key access to a data store.

The major contribution of the paper is that the new design Dynamo has succeeded in production , which shows that decentralized techniques can be combined into a single highly-available system. Also Dynamo sacrifices consistency under certain failure situations and introduced a new concept called eventually consistency. Here we summarize the key features of Dynamo:

1. store simple primary-key only interface
2. partitioning relies on consistent hashing to distribute load across multiple storage hosts
3. replication and data versioning
4. use Hinted Handoff to ensure R/W not failed due to temporary node or network failure
5. use anti-entropy protocol to keep the replicas synchronized

One interesting observation: this design is in general very innovative, however, the paper does not provide many performance evaluations on Dynamo in comparison with traditional store system. Also, it does not provide a detailed discussion about eventually consistency. We have no idea of how this would give the right output eventually and it will meet the requirements for certain workloads. It would be better if the paper mentioned more about this.

Review 7

Reliability at massive scale is one of the biggest challenges faced at Amazon which is one of the largest e-commerce operations in the world where slightest outage has significant financial consequences and impacts customer trust.
With relational databases, complex querying is difficult.Again, for large scale datasets, expensive hardware and highly skilled personnels are needed for various operations in RDBMS.So, it is not a good solution for scaling and partitioning schema for load balancing.
The development of Dynamo is based on motivating scaling, high availability and guaranteed SLA on large scale datasets keeping the design simple using key value pair.

Dynamo requires the query model to be simple read write operations to data items uniquely identified by a key along with ACID properties and non hostile, non security operating environment.The design considerations include sacrificing strong consistency for availability, conflict resolution through read instead of write, incremental scalability, symmetry, decentralization and heterogeneity
Dynamo solves the problem of partitioning using consistent hashing which also brings about incremental scalability with the help of virtual nodes.
High availability of writes are available using vector clocks with reconciliation reads and the version size is decoupled from update rates as the vector clocks capture causality between different versions of the same object
Sloppy quorums and hinted handoff are used for handling temporary failures. This provides high availability and durability guarantee when replicas are not available.Setting the number of read plus writes implements a quorum like system and in this system, the latency depends upon the slowest of the read/write replicas.
Similarly, anti-entropy using Merkle trees helps to recover from permanent failures as it synchronizes divergent replicas in the background.
Gossip-based membership protocol helps in failure detection which preserves symmetry and a decentralized registry for storing membership and liveness information
Dynamo is an implementation on JAVA where local persistence components allows user to use different storage engines

Conflict resolution is pushed to the application to resolve which might result in more complexity or higher software development cost. The security measures are not there due to an assumption for trusted environment along with a wastage in bandwidth.

Review 8

This paper discusses Amazon's Dynamo, another custom built "database like" system created by a web-scale company. Amazon designed this system for their own internal services at first, to serve as a highly available and configurable storage system. They had unique constraints, and could custom write applications to work with it, so they could remove any barriers related to having to maintain some consistent interface. They had a few constraints for their system. The first constraint for their system is that it must be "always-on", so users experience no down time. Reliability is also a huge concern - if an users adds something to their cart, that write must persist. Also, Amazon's ecosystem is composed of a large number of diverse applications, so the system designers also wanted to create something that users could tune to better suit their needs. Amazon also decided to give up the relational database, since most of their workload only needs a primary key lookup, and the overhead of a full RDMS limits scaling. The system is able to scale linearly with the number of machines added, a essential feature for scalability. Like Google's file system, this system assumes hardware failure is the norm, and the system must still perform - writes must persist, even if disks are failing. Amazon design this service to perform very well for all users - they don't simple look at average time, but at the time in which 99.9% (or more) of request can be answered.

The interesting idea of this paper is the concept of eventual consistency. Rather than attempt to use transactions, 2pl, ssi, or other methods we've seen in class to maintain the "C" in ACID, Amazon decided to loosen the requirements. They noticed, that for their workload, it wasn't really a problem - 99.4% of all requests on their workload were consistent with only one version. Because of this, Amazon can leave the problem of consistency for the application to deal with - in the paper shopping cart example, the application can simply merge the two inconsistent carts into a new version. I also really liked their discussion on how they distributed the workload between nodes. It reminded me of a few of the lectures towards the end of 482 - its cool to see stuff we learned in class used in the real world!

I really liked how well Amazon presented their ideas in this paper. They gave us a lot of information on their system - including architecture, business models and how they shaped the requirements, conflict resolution, scalability, hashing and nodes storage. Their was quite a bit covered in this paper.

That being said, that's also probably my only complaint. They're really trying to cover a lot, and it gets very condensed and some things are covered a bit too quickly.
They also have a few areas where they choose not to go into discussion because of trade secrets - understandable, but I'm curious.

Review 9

The problem put forward in the paper is that the continuous failures in the infrastructure should be handled properly since the reliability and scalability at massive scale is important for Any slight error can cause significant financial consequences and negative influence on customers. The traditional RDBMS has low efficiency and high cost with complex query model and a higher priority on consistency over availability. The solution proposed in the paper is building a highly available key-value storage system called Dynamo.

Dynamo simplifies some settings of RDBMS. It uses key-value instead of complex models to manage data. And it decreases the consistency level from always consistent to eventually-consistent. The system adopts Service Level Agreement to handle the communication between clients and services. To improve the customer experience, Dynamo is designed to be “always writable” by moving the conflict resolution from writes to reads. Also, it has other designs called incremental scalability which means Dynamo can scale out one node at a time, symmetry, decentralization and heterogeneity. And its partitions and replications of keys are hashed in a fixed circular space called Dynamo ring. And since the Dynamo is eventually-consistent, there may be multiple versions of data which keep distinct version sub-histories of the system to reconcile in the future. The solution is using vector clocks to capture causality between different versions. The designers of Dynamo tried several strategies to ensure the uniform load distribution, and finally adopts Q/S tokens per node strategy which takes equal-sized partitions. The system handles failures by hinted handoff.

The strength of the paper is that it proves that the eventually-consistent system can be adopted in the production. Though it provides limit guarantees, it provides high performance and better experience. Also, different from traditional database, it resolves the conflicts in reads instead of the writes to improve the customer experience since people prefer to be able to add items into cart immediately. That idea is very particular to a shopping website like amazon. It shows that Dynamo is very professional and highly customized.

The weakness is that after the paper I still wondered whether it will take too much time to reach the eventual consistency if a large quantity many writes come in. If the time is quite long, it is still possible to affect the customer experience.

Review 10

This paper presents Dynamo, the key-value store storage system used by Amazon. The motivation and design goals of Dynamo, along with high level details on its implementation, performance, and key observations in deploying the system are described in this paper.

The need to design a custom storage system such as Dynamo came from the fact that there aren’t any existing systems that provide good performance flexibility, while guaranteeing high availability. According to the paper, these are the requirements that needed to be met in the design of Dynamo:
- Query Model: simple read and write to a data that is uniquely identified by a key. no operations span multiple data items, so a relational schema is unnecessary.
- ACID properties: weak consistency is okay, as long as it guarantees high availability.
- Efficiency: the system need to operate on commodity hardware, while providing good consistent performance at 99.9 percentile.
- Security: authentication and authorization is unnecessary, because it runs on internal service.

Dynamo achieve these goals through the following:
- Partitioning Algorithm: Use consistent hashing where it use “virtual nodes” to map multiple points in the hash ring to a node. This allow more flexibility and scalability.
- Replication: By replicating data on multiple hosts, high availability and durability is achieved.
- Data Versioning: Support eventual consistency, where updates propagate to all replicas asynchronously. This allow multiple versions of the object to exist depending on the type of applications where this is allowed. Eventually, the versions get merged.
- Hinted Handoff: Use sloppy quorum where all read and write operations are performed on the first couple healthy nodes from the preference list. This helps handle failures in the network by bypassing the temporarily unavailable nodes.
- Replica Synchronization: To detect permanent failures, a Merkle tree is used as a way to find inconsistencies between replicas

The paper describes the requirements and design decisions of Dynamo in a clear and concise manner. One thing that feels missing are some limitations or problems in Dynamo, and any example cases of when the system would not be ideal. The idea of trading consistency for better availability and performance is interesting, but it seems like it depends heavily on the type of applications and workloads it is running on.

Review 11

This paper introduces the Amazon Dynamo storage system, which is a distributed key-value store with high availability, high scalability. Dynamo implements a combination of techniques to achieve this, such as eventual consistency protocol, gossip protocol, constant hashing with virtual node. Dynamo also provides certain flexibility to suit each application’s requirement on availability and durability.

One key contribution of Dynamo is that it shows combining several common techniques could create huge distributed systems with the desired behavior. Besides the techniques mentioned above, it uses data versioning to deal with conflict and vector clock to detect the causal dependency between versions and implements optimistic replication approach to provide eventual consistency with tunable feature by the configurable parameters R and W. All these methods are simple and straightforward to their purpose but carefully chosen to achieve the ultimate features of the system.

One of other contributions is that it implements a new standard: 99.9th percentile metric to enhance its overall performance. Dynamo improves the response time by the 99.9th percent metric. It requires at least 99.9% of read and write operations to be performed within a certain bound. From the amazon’s analysis, it achieves small average latencies as well as bounded latencies with reasonable scale even at the peak times. This metric not only gives a better user experience in practice but also provides predictable behavior inside the system.

Dynamo is a great peer-to-peer distributed systems. However, it may still suffer from several drawbacks:
1.Dynamo implements optimistic replication approach. But this eventual consistency protocol makes programmer do extra efforts to deal with possible conflicts. In addition, eventual consistency protocol can not support certain constrain, such as uniqueness. For example, if one need to create an amazon prime account, the new account can never be stored in the Dynamo.
2.Dynamo do not support well for many updates on a certain object and it might lose updates. The vector clock support only limited length and truncate the least recent updated entry when there is an excess in size. As a result, a picky user who frequently modify its cart might find out at last something unexpected is missing or shows up. More generally, an application may not fit to Dynamo if it need updates an object frequently in absence of a read.

Review 12


Many companies like Amazon require high availability and scalability in their applications, and transitively in their data storage solutions. However, relational databases are not ideal for many applications, because they usually emphasize consistency over availability in their replication schemes, which are needed for data durability at large scales. In addition, many applications only need key-value storage access, and do not need the relational functionality of RDBMS. To address this problem, Amazon created Dynamo, which is a key-value storage solution focused on high availability.

Dynamo runs on a decentralized cluster of nodes. Data is distributed across nodes by Consistent Hashing, where each node is assigned partitions of the hash space. Each node also keeps a replica of the data in the previous N partitions of the hash space (N can be tuned by the user) and these replicas help ensure durability.

Read or write requests can be serviced by any node that contains the requested data (although preferring certain nodes can increase performance). However, when servicing a write request, only W nodes (again, configured by user) are required to write the new data before returning successfully to the user . This means that different nodes may have different versions of the data, and Dynamo uses its Eventual Consistency scheme to manage this inconsistency. Data denote their version using a Vector Clock, which means it is possible to determine if two data objects are descended one from another or whether they have diverged. Ancestor data can be overwritten by descendant data; however, divergent data must be merged by the client application, and this happens during the next read of the data. After the merge, the data is once again consistent.

Information and membership and health of nodes is disseminated through the network using a gossip protocol, where each node broadcasts its known membership information to a random node every second. This allows the network to automatically adjust to nodes being added and removed, and node failures.

This paper has very strong rationale for its design decisions (key-value interface, merging consistencies during reads to increase performance for writes) that are motivated by Amazon’s application requirements.

The flow of the paper seemed a little scattered, because design questions were discussed in the System Architecture, Implementation, and Experiences/Lessons Learned sections. This lead to a bit of confusion as to what design was finally used in production.

Review 13

The paper presents a simple key-value storage system used at Amazon called Dynamo, which focuses on achieving a high availability while sacrificing its consistency. People at Amazon have identified that most of their workloads can be sufficiently serviced by a simple key/value interface. The paper states that traditional relational DBMSs are not suitable since their availability is not good enough and they are simply too complex for what applications require at Amazon. Dynamo is a good example of non-traditional, key/value storage systems that work in practice. The paper explains the design principles and techniques that have been applied to the system.

The paper explains the following aspects of Dynamo: partitioning, replication, versioning, membership, failure handling and scaling. For partitioning, it uses consistent hashing where the output of a hash function is treated as a ring and it introduces the notion of “virtual nodes” to enable each node to be responsible for multiple points in the ring. This can account for heterogeneity of the system, meaning that better performance nodes can be responsible for more keys, handling more workloads than others. However, this property is not experimented as they use nodes with homogeneous hardware configurations in the paper. It was unfortunate that the authors have chosen the configuration that cannot verify such claim.

For replication, Dynamo replicates its data on multiple hosts as usual for distributed storage systems. The data is replicated at N nodes and also there is a ‘preference list’, which is the list of nodes that is responsible for storing a particular key. An administrator of Dynamo can specify parameters R and W, which controls the number of nodes that should perform read or write operation persistently before return the result to a user respectively. The parameters N, R and W controls the trade-off between the performance, availability, durability and consistency of the system. I think that it is smart to have these knobs that give its users freedom to choose such trade-offs. One aspect of the paper itself that I do not like is that the relevant subsection does not seem to explain the relationship between N and a ‘preference list’ well in my opinion. When I first read the subsection, it had no concrete descriptions on how it constructs a ‘preference list’, which made me slightly confused and question how it actually works.

Dynamo uses vector clocks to handle data versioning and each node of Dynamo detects failure of other nodes locally. A gossip-style protocol is used to spread the detected failure to every node and also let each node aware of the data hosted by its peers. The paper points out that the gossip-style protocol can be problematic with scalability. The paper describes many more techniques in depth than things I mention in this review.

In conclusion, Dynamo is a highly available key-value storage system that sacrifices consistency for some degree and provides configurable parameters so that a user can choose appropriate values that suit his/her needs. There is a lot of technical details in the paper. Most of the details are well explained, but sometimes a desired explanation or experiment is missing or an explanation is confusing. For me, it is understandable since the paper discusses a real-world system, which is bound to be much more complex than just lab prototypes.

Review 14

Databases have traditionally been designed in maintaining four key properties: atomicity, consistency, integrity, and durability. However, support for each of these properties comes at the cost of other system considerations such as availability and performance. In this paper, Amazon discusses Dynamo, which is a key-value storage system that values data availability and low latency at the cost of database consistency.

Amazon, like many others, has realized that a relational database is not an ideal solution to support their business needs. In Amazon's case, a relational database supports many features that Amazon services do not need such as complex query processing. Furthermore, Amazon's goal primary goal is to provide their customers the illusion of always being on even though there may be system failure behind the scenes.

One of the key goals of Dynamo is to almost never reject a write request regardless of the system state, but still have some system durability. In order to do so, Dynamo uses an eventual consistency model in combination with data replication. In Dynamo, data is partitioned between nodes using a consistent hashing scheme so nodes are responsible for the data that lies in their hash range. The data is also replicated to the N-1 clockwise successors. When data is updated, the updates are eventually propagated to all the replicas, but the result is returned once W, a predefined number, nodes respond. Dynamo allows for multiple versions of an object to exist and keeps track of update versions with (node, counter pairs). If multiple versions of an object exist, the conflict is either resolved with a last update wins policy or with a policy defined by the application. To handle system failures, Dynamo allows for data traveling to a failed node to be forwarded to the next N healthy nodes with information indicating which node the data actually belongs to. If the node returns, the "hint" guides the backup nodes in re transmitting the data. To avoid permanent failures in which the backup nodes also fail, Dynamo uses Merkle tree's to synchronize replica inconsistencies.

One thing I like about Dynamo is the design of knobs for the three parameters N, W, and R. Although Dynamo is designed to be always writable, the tuning knobs allow for it to also support high read environments. In general, Dynamo still satisfies the original goal of high data availability, but can also work with other use cases. Whereas a relational database is a jack of all trades, Dynamo is one end of the spectrum in which it specializes in writes.

One aspect of Dynamo which was unclear to me was the handling of permanent failures within the system. If a node fails, Dynamo uses hinted handoffs to redirect the data to the other replicas. However, suppose the nodes which have the hinted handoffs also fail. I did not understand how Dynamo deals with this. It seems that Merkle trees are the proposed solution, but I envision an edge case in which Node A fails and shortly after the other N-1 nodes that get the replicate they also fail. It seems reasonable to assume that the data is lost since this is an eventual consistency model so the replicas are updated asynchronously. I fail to see how a Merkle tree synchronizes the replicas when eventual consistency is an asynchrous model.

Review 15

Dynamo is a highly available and scalable distributed data store used by several Amazon's services. For such a big e-commerce company, any data outage is not acceptable. Among the ACID properties, as a result, Dynamo sacrifices consistency (C) in order to achieve very high availability (A). It should be noted that, however, Dynamo is designed to be an eventually consistent data store.

In this paper, some important features of Dynamo in Amazon are mentioned. Many of them are designed particularly to meet Amazon services' needs:
1. Dynamo is used internally, thus security issues are not taken into account.
2. Dynamo focuses on "always writeable" applications that no updates are rejected.
3. Dynamo does not provide complex hierarchical structure or relational schema.
4. Since the applications at Amazon require very low latency for a high percentage of operations, Dynamo maintains enough routing information for each node locally to avoid routing request through multiple nodes.
5. Dynamo uses a variant of constant hashing and incorporate "virtual node" to make partitioning decisions and ensure uniform load distribution.
6. Dynamo creates a data version for each modification. This guarantees its eventual consistency even if it allows updates to be made before previous updates have been applied at all the replicas. Dynamo uses "vector clocks" to record the versions.

Weakness of this paper:
Dynamo uses a decentralized peer-to-peer scheme and maintains routing information in each node locally, which requires the nodes to actively communicate with each other. This makes the overhead of the design grows more than linearly, and thus makes the system highly non-scalable.

Dynamo is a highly available and scalable distributed data store used by several Amazon's services. For such a big e-commerce company, any data outage is not acceptable. Among the ACID properties, as a result, Dynamo sacrifices consistency (C) in order to achieve very high availability (A). It should be noted that, however, Dynamo is designed to be an eventually consistent data store.

In this paper, some important features of Dynamo in Amazon are mentioned. Many of them are designed particularly to meet Amazon services' needs:
1. Dynamo is used internally, thus security issues are not taken into account.
2. Dynamo focuses on "always writeable" applications that no updates are rejected.
3. Dynamo does not provide complex hierarchical structure or relational schema.
4. Since the applications at Amazon require very low latency for a high percentage of operations, Dynamo maintains enough routing information for each node locally to avoid routing request through multiple nodes.
5. Dynamo uses a variant of constant hashing and incorporate "virtual node" to make partitioning decisions and ensure uniform load distribution.
6. Dynamo creates a data version for each modification. This guarantees its eventual consistency even if it allows updates to be made before previous updates have been applied at all the replicas. Dynamo uses "vector clocks" to record the versions.

Weakness of this paper:
Dynamo uses a decentralized peer-to-peer scheme and maintains routing information in each node locally, which requires the nodes to actively communicate with each other. This makes the overhead of the design grows a lot more than linearly on the number of nodes, and thus makes the system highly non-scalable. Some further optimization might be needed to address this problem.

Review 16

This paper discusses the design and implementation of Amazon’s platform that has the ability to service tens of thousands of servers and networks in datacenters throughout the world. This is important because databases at the large of a scale are subject to component failures, so it is interesting to study the Amazon’s method of maintaining reliability and availability, at the cost of consistency, at such a large scale. This paper presents Dynamo, a highly available and scalable distributed data store, as a solution because it is flexible enough to provide application developers with the power to define their data store based on tradeoffs, effectively achieving high availability and performance.

What sets Dynamo apart from relational databases (whose key-value format would introduce scaling issues and inefficiencies, and limit availability) is that it provides a simple primary-key only interface to meet the requirements of services on Amazon’s platform that only need primary-key access. Dynamo achieves scalability and availability by partitioning data and replicating it with consistent hashing. Consistency is farther ensured with object versioning. Replicas are kept consistent with a decentralized replica synchronization protocol and techniques that resemble quorum. Gossip based distributed failure detection and membership protocol are used. Adding on or removing storage nodes is simple and does not require manual partitioning or redistribution, as the system is decentralized. Dynamo uses a service-oriented architecture that almost resembles MapReduce techniques; a service manages its state with different data stores, and some services act as aggregators and use other services to produce a composite response. The paper emphasizes a 99.9th percentile (which is chosen because a higher percentile would increase cost too much, making the cost-benefit not worth it) of distributions, which reflects Amazon’s value on performance for the customers’ perspective. The design is rooted in values of incremental scalability (scaling to one storage host at a time at little cost to the system and its operators), symmetry (each node has same responsibilities as peers), decentralization (peer-to-peer techniques over centralized control), and heterogeneity (work distributed proportionally to the capabilities of the server).

Dynamo addresses the problem of partitioning with consistent hashing because it provides incremental scalability. It solves the issue of high availability for writes with vector clocks that reconcile during reads, because this decouples version size from update rates. In facing the problem of temporary failures, Dynamo handles it with Quorum and hinted handoff because this provides high availability and guarantees durability when some replicas are not available. The recovery from permanent failures is handled with anti-entropy using Merkle trees, which were chosen because they synchronize diverged replicas in the background. Membership and failure detection are addressed with gossip-based membership protocol and failure detection, because this preserves symmetry and avoids having centralized registry for the storage of node liveness information and membership. The nodes of Dynamo handles data versioning almost metaphorically to Git; it provides eventual consistency, so updates can be propagated to all replicas asynchronously, and put() and get() are used by each doe to modify or read data. Request coordination (messages pipelines and split to multiple stages), membership and failure detection (ring membership, external discovery, and failure detection), and local persistence engine (chosen by applications based on their object size distribution) are the three essential software components to every Dynamo node.

Overall, this paper is very thorough and interesting. I thought that the paper did a good job of teaching the role and implementation of Dynamo nodes, and how they interact with each other to achieve availability and reliability. I though that a limitation of this paper was that it did provide a quantitative cost analysis of why performance can not be improved above 99.9, and how much efficiency would it really cost to raise the performance.

Review 17

This paper is about Dynamo, which is Amazon’s key store system it uses for applications which require high availability. This system makes assumptions about the kind of data used and the kinds of queries that the system will use. They assume that queries will use unique keys and that their simple query model does not require a relational schema. They implement ACID properties except with weaker consistency. They require high efficiency and allow programmers to trade performance, availability and durability through service level agreements. This paper then discusses related work and its influence on the architecture of the system and its implementation. The paper then has an extensive discussion of the experiences the authors had in the 2006-2007 year.

This paper’s strengths are in its design features and robustness. It is designed for very high availability at the cost of consistency. The paper has a great evaluation in that it was used at Amazon for a year and worked well under heavy workloads during the holiday season. Their use of consistent hashing, vector clocks and failure recovery are effective and well-motivated.

This is a pretty good paper and it does not seem that there are many drawbacks. The architecture is obviously very specialized for Amazons architecture so the portability to other architectures may be low. The paper sacrifices consistency for high availability, but there are still situations where you do not want inconsistent reads. If you are managing a shopping cart with less than 1% error, you still have many times over the course of millions of transactions where inconsistencies will occur. It is questionable as to whether or not it is reasonable to assume that the slight increase in latency is worth the sacrifice of consistency. Will users really not put up with a couple seconds slower responses? Additionally, I think that the lack of detail in statistics due to information withheld due to corporate concerns is frustrating. The parts of the paper that were explained in some detail were not enjoyable to read at such a low level.

Review 18

Part 1: Overview

This paper introduces the DynamoDB, a column based database with high availability, which is foundation for some “always-on” core services provided by Amazon. DynamoDB sacrifices some data consistency in trading for higher availability. Developers can utilize extensive object versioning and application assistance when using DynamoDB to provide services for customers.
Amazon is facing severe challenges as the Service Level Agreements guarantee that 300ms for 99.9% requests at 500 request per second peak. DynamoDB must be fast-responding.

Query model is NoSQL, no operation need to span multiple items and thus there is no need for a relational schema. ACID properties are satisfied however the consistency property may be weaker than the standard. SLA requires high performance and high computational efficiency of the whole distributed system. The key design is to enforce “always writable” data store. “Last write wins” policy is used for conflict resolution. In addition, other design criteria like incremental scalability, symmetry, decentralization, and heterogeneity are also embraced in DynamoDB design. Hinted Handoff scheme is used to dealing with recoverable conflicts, while permanent failures are settled by replica synchronization. An administrator uses some kind of command line tool or browser to add or delete node from a Dynamo ring membership.

Part 2: Contributions

High availability and good user experience are the top goals of DynamoDB. Dynamo provides “always writable” as well as “simple k-v interface” to their customers as well as developers. Developers can thus develop always-on applications on top of it. In addition, it also provides a configuration for users to set that which criteria matters more, performance or reliability.

DynamoDB is highly decentralized which at the same time minimizes the man labor. It does partitioning and redistribution without supervising.

DynamoDB combines different technologies to trade for higher availability. Consistent hashing facilitates the availability by object versioning. Quorum-like technique and decentralized synchronization protocol are used to maintain consistency across the distributed system.

Part 3: Possible Drawbacks

As DynamoDB sacrifices consistency for higher availability, some consistency focused applications may wait more time for the database to recover consistency problems. For example, sale transaction records should always be consistent across all distributed replica.

Review 19

The paper discusses a key-value storage system called Dynamo. The problem addressed by Dynamo is that in a system that provides large number of services, scalability and handling component failure scenarios is important. Specifically, this problem is important for Amazon as it provides services for tens of millions of customers. Dynamo provides a simple access to its key-value storage system only using a single key attribute as opposed to a complex interface in a RDBMS database system. It includes techniques which allows it to be a highly scalable and reliable system. One technique used by Dynamo to achieve scalability and availability is using consistent hashing for data partitioning and replication. Consistent hashing allows incremental scalability by distributing the database load dynamically across multiple storage nodes. While a node is added or removed, only the immediate neighbours are going to be affected as opposed to other hashing functions which causes nearly all keys to be remapped. This allows Dynamo to scale to large number of nodes. In addition, Dynamo automatically replicates each data items in a preconfigured number of nodes. This allows it to have a high availability. in face of node failures.

The other main technique is using object versioning to provide eventual consistency. In Dynamo, an update operation may return to its caller before the update has been applied at all the replicas, which can result in scenarios where a subsequent read operation may return an object that does not have the latest updates. In order to handle such issue, Dynamo treats each modification as a new and immutable version of the data. In addition, it uses a vector clocks to track dependency of each updates. Based on these informations, during request, nodes either use newer version and garbage collected older versions or in case a causal link can not be determined, the different versions will be returned to the client for semantic reconciliation. In addition, I like how the authors handles node failures. Their technique called “hinted handoff” allows them to perform read and write operations on the set of healthy nodes instead of being obstacles by a failure node in walking through the consistent hashing ring. In addition, Dynamo can handle failure of an entire data center by replicating each objects across multiple data centers.

The main strength of the paper is its insight in identifying the different requirements of a large scale web service provider like Amazon. It recognized that even a complex service provider like Amazon prefers a simple key-value based database instead of a complex RDBMS based system. In addition, I found their idea of guaranteeing availability of services during many component failures insightful. They also showed that strict transaction based consistency is not always a requirement even in a practical large scale implementation of database system and instead eventual consistency can be enough. I also found the concept of exploiting heterogeneity in the infrastructure as another strong idea as it allows them to add a new node with a higher capacity over the existing one and still fully exploit its potential.

The main drawback of the paper is its sacrifices consistency under certain failure scenarios. During this time it is possible that application might read inconsistent data objects. While they claims that such inconsistency could be tolerated by some applications in Amazon, this limits the applicability of Dynamo to a consistency demanding workloads. In addition, I found the assumption of an underlying non hostile environment rather weak. Even though Dynamo is used by Amazon’s internal services, there could be security breaches to the hosting infrastructure and compromises sensitive services. Furthermore, it only support simple query model and doesn’t support operation that spans multiple data items. Again, I believe this will limits the applicability of Dynamo to different area of application which can traditionally be handled by a RDBMS database.

Review 20

Key Contribution
The major contribution of this paper is that, they demonstrate how different technologies can be aggregated together to build a target-specific database system and how to tune those features to reach highest availability and scalability.

Purpose of the paper
This paper presents the design & implementation of Dynamo, a highly available key-value storage system in Amazon. The targets of the system are: 1. prevent outages as much as possible, so that the services that Amazon provides for their customers are always available. 2. meet the reliability and scalability needs. 3. ensure eventual consistency.
At Amazon, they have a large variety of applications. Some of them only involve primary key look up, which makes relational database useless because the queries are so simple. To improve availability, they must reduce system complexity and build the system specifically for those kind of requests. Taking shopping cart as an example, they want their customers always be able to modify their cart despite some underlining system outages such as server nodes breaking, one of the data center that containing the data down.

Dynamo uses a systems of well know techniques. They use consistent hashing for scalability and availability. User object versioning for consistency. And they use quorum-like technique and a decentralized replica synchronization protocol. The benefit of the system is that no manual administration is required for system execution and expansion. In the design, they tried to answer 2 key questions for eventual consistency. First, how eventual, meaning how should the data kept in different replica be eventually consistent with each other. And second, how is responsible to make resolve data conflicts. In the paper, depending on the data writing and reading pattern and also, most importantly, the customer requirements, they decide to push the responsible to the user application because it is more reasonable to resolve data conflicts with relational information presented, which the data base doesn’t have. The paper also mentioned their of error detection and replica synchronization scheme. They all earn user the benefit of high availability for writes.

Of course, as said in the paper, the system is used internally in Amazon. So they have no security consideration in the system. Tuning different techniques without worrying about security and shoot for high performance is problematic for latter development.

Review 21

This paper described the design and implementation of Dynamo, a highly available key-value storage system used in Amazon for some of their core services. Most successful example of Dynamo adoption is Amazon shopping Cart. Dynamo achieved high availability, scalability and efficiency using a eventual consistent model. Although some anomalies could occur, availability and performance are improved dramatically, and they are able to guarantee 99.9th percentile of their workload are served within 300ms.

To provide responses for high volume of workload, the key-values pairs are partitioned into a cluster of computer. And they partition and replicate data using a ring-like structure. Each virtual node is responsible for storing one segment on the ring and for replicating data of some other segments. Key is hashed into the ring. And request will usually be served by the first node meet when walking clock-wisely.

When request for writing a value is issued, one server will response by updating its local value and return. Then this update will propagate to other server along the ring for update. And eventually all server will have a consistent value for that key. This is why it is called eventual consistent model. Of course, anomalies could occur when writing appears simultaneously. Dynamo prevents this using 1) Vector clocks with reconciliation during reads and 2) a read/write request success only when k server returns.

This paper also talks about many other important implement issues such as scalability: adding and removing a node should be explicitly done by administrator, while detect temporary failure must be detected as quickly as possible automatically.
And performance and durability tradeoff is possible in Dynamo by adjusting R/W duplicas.

Technical contribution:
This paper provide architecture and certain level of details about a successful eventual consistent storage system. It provides important experience for further research of transaction processing systems without strict consistency.

At the beginning of this paper, it mentioned that to reconcile inconsistency, ‘when’ and ‘who’ should be pointed out. But I didn't see the ‘when’ part.

Review 22

The contribution of this paper is about how Dynamo: Amazon’s key value store is a highly available system built for high availability sacrificing consistency, to abide by the Service Level Agreements. This sytem was built recognizing that regular relational databases cannot handle the specific needs of Amazon’s decentralized architecture efficiently.

There were three factors that were specific being
1. Query model: No operation spans multiple schemas and do not need relational schema.
2. ACID properties: Since Amazon needed high availability, ACID properties would not be relevant in this particular database
3. Efficiency: The latency requirements of services are measured at the 99.9th percentile of the distribution which means for a high throughput, tradeoffs must be achieved
4. Security: Since the database is only used by the internal services, they have assumed that there are no security related requirements such as authentication and authorization.

One of the key contributions of this paper is that Dynamo is an “always writeable” data store since it is important to not lose an update on the customer’s end thereby moving the conflict resolution to the reads. They also replace the typical database by the key value store since they wish to store very small objects in a distributed system. Since they node failure as the norm rather than the exception, they have built in merkle trees to detect inconsistencies in which the nodes cooperate to ensure they are in sync in the background. The conflict resolution is taken care of using vector clocks that determine the time of a given write. There is the provision for it to be handled at the application level, which I think is a great win, however, even if the responsibility lies with the data store, the “last write wins” policy is used. They also have configurable R, W, N values, where R/Q is the minimum number of nodes that must participate in a successful read/write operation. The applications working with Dynamo can configure these values based on their requirements.

Their experiments definitely showed a great response in terms of consistent value or one value being visible overall to 99.94% of the requests. Also, the latency is lower when it is operating on a large load which again was surprising and yet a brilliant result.

I believe Dynamo is probably too specific, it may not be easy to use with multiple other applications without considerable modifications. The key value store essentially means writing a complex query spanning schemas might get difficult. The paper specifies that security is irrelevant since it is an internal system but there should at least be authentication/authorization for accessing specific critical nodes or for some highly critical operations. Also, the data blocks have the need to be small so huge stores of data cannot be stored even if required.

Overall, this paper shows a complete break from a typical database system built to address high availability.

Review 23

This paper introduces the Dynamo DB from Amazon as a highly available key-value storage system for large scale services provided by amazon.

The paper started revealing the problem that amazon was facing: serving millions of customers at peak times. Under such circumstances, Amazon’s stateful services needed to be supported by an efficient and reliable system. The example that is constantly mentioned in this article is the shopping cart, which is an exact fit to the Dynamo’s query model: Key-Value storage.

In terms of the system assumptions and requirements, Dynamo is facing very stringent latency requirements due to the SLAs’ measurement of 99th percentile of the distribution. As normal DBMS services who can provide strict ACID properties always suffer in performance(latency /throughput), the availability issue come as their bottleneck. Dynamo is not aiming at very strong consistency models, instead, it is a distributed replica system working on a weak consistency level: eventual consistency, as a tradeoff for high availability.

The Dynamo DB is implemented as a distributed system that uses consistent hashing to partition data pairs. And each record is replicated into the following N-1 nodes in the hash ring. Other than keeping a primary-consistent version of data like GFS, Dynamo is only guarantee the data to be eventually consistent. Eventual consistency is a new level of consistency that doesn’t providing any safety guarantee on the data record. The philosophy in the dynamo case is to allow different versions to exist simultaneously in different partitions, write operation writes to W virtual nodes and read operation read from R virtual nodes. Since the R+W is larger than N, with the help of vector clock, it forms a quorum like condition for determining the final version by causality. In this design, all the nodes are working with the same priority in the decentralized system. Request can be routed to any node that has the replica, and hence the write operation can be finished quickly after the servicing node finished to write to the W-1 replicas. The anti-entropy action is achieved in the read operations and background whisper, and after a few hundred milliseconds of write ops, all replicas have the same content.

To sum up, there are many strengths of the Dynamo’s design. Firstly, it used the new level of isolation, eventual consistency to avoid locking overhead in the traditional DBMS. Secondly, the W+R>N quorum-like system makes sure of the next read operation can always get the latest written version of data. Thirdly, the system latency is shown to be better than any other RDBMS. Lastly, this highly available system can automatically detect and repair system failure, which in turn, cut down the operational cost in DBA.

However, there is also some weak point in this paper:
1.the Dynamo DB can only support one record at a time accessing interface, no way to support range querys.

2.Besides causality decision, it will always need user specified reconciliation method to keep the multiversion system consistent, which would in turn, add to the work programmers who is using this service.

3.In the performance analysis, it only mentions that the chances to merge multiple versions are low, but it doesn’t specify what is chance of reconciliation failure and how much compensation would be associated to these failures.

Review 24

This paper talks about the design and implement of Dynamo which is a key-value storage system used by Amazon. Dynamo is a high available and reliable platform. It is “always writeable” and will succeed after W number of node participate in write operation. It is highly decentralized and data is partitioned using consistent hashing. Data is replicated among different node and uses quorum to achieve consistency.

It sacrifice the consistency to improve the availability and performance such as low latency. It allow multiply version of data and do consolation when read an item. By using Merkel trees, can detect the inconsistency and do consolation among conflict version.

Dynamo is a zero hop routing system that can have O(1) routing, as each has enough information to route the request to the right node. By using consistent hashing, each node will be assigned many points in a circle, thus it is easy to add or remove a node in the system and easy to handle failure in the system by Hinted Handoff. For load balancing, the circle is divided into many same size pieces and each node will be assigned same amount of hash range.

Dynamo is a highly scalable and available distributed storage system. It has veery good fault tolerance strategy and have inconsistency conciliation to achieve good performance of write. The feature of the system is a good fit for amazon(such as key-value store, no concern for security, etc.)

(1) Dynamo is good only for small object.
(2) No security protection when communicating.
(3) No support for complex query, even cannot do join in dynamo storage system
(4) Need high level algorithms to do the conciliation, so when version conflicts are frequent, the performance of read would be worse.
(5) If the workload of some node dramatically increases, seems no dynamically rebalancing strategy.

Review 25

The authors present Dynamo, a highly available, scalable, key-value store. The authors claim that many applications do not require strong consistency guarantees or ACID transactions, and in fact, do not require a table schema. They also claim that databases which do provide these feature will not scale to large datasets. Dynamo organizes its nodes into a ring structure and uses consistent hashing to partition the data. Keys are are first hashed using MD5 before the consistent hashing is used to place the data on node. Data is replicated (asynchronously) on the N-1 following (in a clockwise motion) nodes in the ring and is multi-versioned. Dynamo also uses several techniques to achieve durability: sloppy quorum, replica synchronization, and knobs to adjust the availability-durability tradeoff. Dynamo replicates data across data centers and can withstand outages in entire data centers. The evaluation shows that Dynamo is low-latency, achieving latencies in the 10s of milliseconds.

I thought that the ring structure of Dynamo was very interesting, as well as the fact that it works across data centers. I also liked how Dynamo is able to achieve high availability. However, the authors claimed that relational databases, and more specifically, transactions, can not scale to large databases. This claim is hotly contested and newer databases are able to scale and provide support for transactions (e.g. Spanner). These do give up some availability, but they are used and engineers are able to work around the issue.

Review 26

This paper proposes a distributed, highly available, key-value pair database system called Dynamo, which has been used by Amazon for their e-commerce business. Many of the relational databases ensure all four of the ACID properties, but because of that, the availability of the system is has to be compromised. Relational databases would sacrifice availability for consistency to make sure that every read and write corresponds to the latest, most consistent values in the database. Therefore, the authors argue that Dynamo does a better job of increasing availability.

Dynamo was created with the following considerations in mind: the database should be scalable so that when a node is added, there is a minimal impact on the whole system; every node should have the same responsibilities as every other node in the system; the whole system should be decentralized; and the work should be proportionate among the nodes. The authors explain that Dynamo ensures those considerations with the following. All nodes in the system are stored in a DHT or a distributed hash table, but each data item is stored in N nodes for better availability. Whenever the client wants to get or put values, R nodes or W nodes must respond for a successful read or write, where R and W are specified by the client. This is used to give the client control for the tradeoff between availability and consistency. Finally, the authors go over hinted handoff and replica synchronization to address durability.

Overall, the paper made a significant contribution to the database industry and explained the database system, Dynamo, very well. However, I believe there are still some weaknesses with the paper as follows:

1. What happens when there is no good way to resolve conflicts between concurrent writes? Are the authors saying that only when there is a way to resolve conflicts that Dynamo will work?
2. The authors state that N, R, and W are used to let database administrators control the tradeoff between availability and consistency, but they never did any experiments to show how the values of N, R, and W will affect the database. I would have liked to see what values Amazon uses for its implementation or at least several configurations with their plotted performance and fail rates.

This paper proposes a distributed, highly available, key-value pair database system called Dynamo, which has been used by Amazon for their e-commerce business. Many of the relational databases ensure all four of the ACID properties, but because of that, the availability of the system is has to be compromised. Relational databases would sacrifice availability for consistency to make sure that every read and write corresponds to the latest, most consistent values in the database. Therefore, the authors argue that Dynamo does a better job of increasing availability.

Dynamo was created with the following considerations in mind: the database should be scalable so that when a node is added, there is a minimal impact on the whole system; every node should have the same responsibilities as every other node in the system; the whole system should be decentralized; and the work should be proportionate among the nodes. The authors explain that Dynamo ensures those considerations with the following. All nodes in the system are stored in a DHT or a distributed hash table, but each data item is stored in N nodes for better availability. Whenever the client wants to get or put values, R nodes or W nodes must respond for a successful read or write, where R and W are specified by the client. This is used to give the client control for the tradeoff between availability and consistency. Finally, the authors go over hinted handoff and replica synchronization to address durability.

Overall, the paper made a significant contribution to the database industry and explained the database system, Dynamo, very well. However, I believe there are still some weaknesses with the paper as follows:

1. What happens when there is no good way to resolve conflicts between concurrent writes? Are the authors saying that only when there is a way to resolve conflicts that Dynamo will work?
2. The authors state that N, R, and W are used to let database administrators control the tradeoff between availability and consistency, but they never did any experiments to show how the values of N, R, and W will affect the database. I would have liked to see what values Amazon uses for its implementation or at least several configurations with their plotted performance and fail rates.

Review 27

This paper discusses the Amazon Dynamo key-store system. Amazon decided that it did not require a full RDBMS system for many of its services, so it developed a system that uses consistent hashing to store data that can be accessed by a single key. This system is designed to provide high availability at the cost of some consistency, and guarantees responses within a fixed interval of time in 99.9% of interactions.

Dynamo is an eventual consistency system, but it varies from other eventual consistency systems in several important ways. First, Amazon has decided that write requests should never be rejected. Many of their most important and most user-visible services, such as Add to Cart and Delete from Cart are write requests. Therefore, Dynamo gives precedence to write requests, performing conflict resolution between servers with different versions of data during read requests, rather than write requests. Second, Dynamo organizes the nodes in the distributed system into a ring, with each node being responsible for the key range in the region of the ring between it and its predecessor node on the ring. Dynamo allows nodes to be added to or removed from the ring as the administrator decides. Nodes contact a random node every second to gossip membership information so that all nodes know which nodes are in the network and which key ranges they are responsible for. This allows for a bounded number of network hops when distributing requests to the nodes that hold the data necessary to fulfill them. One of the most interesting features of Dynamo is that it allows administrators to tune the balance of performance, durability and availability by changing the number of nodes which must respond with confirmation of a durable read/write before the operation returns to the user and report success. Setting a low number of required acknowledgements improves performance, but may reduce availability and consistency.

Dynamo balances the load between servers by redistributing the key range when nodes are added to or removed from the network. Amazon’s scheme for rebalancing divides the hash space into Q equal partitions. Each node is then assigned Q/S tokens, where S is the number of nodes in the ring. When a node leaves the system it distributes its tokens randomly among the other nodes in order to preserve the load balance. Similarly, when a node is added, it takes tokens from the current nodes.

My chief complaint with the paper was that they never go into any detail about how the key-value system works. It seems to me as if some feature of an RDBMS would still be necessary for Amazon’s purposes. For example, there are millions of users registered with Amazon, each with a different customer ID, cart ID, etc. How then, does one generate a single key to lookup the products in a given customer’s cart for example? It seems to me that the key would either need to be some kind of concatenation or combination of different key values (e.g. customer ID + cart ID) or that multiple fields would need to be stored for a single object, as in an RDBMS.

Review 28

This paper described Dynamo, which is a highly available and scalable data storage system, used for storing number of core services of Amazon e-commerce platform. Amazon platform us implemented on an infrastructure of tens of thousands of servers and network components. Thus, reliability and scalability are two of the most important properties of the Amazon data storage system. This paper presented the design and implementation of Dynamo, handling tradeoffs between availability, consistency, cost-effectiveness and performance.

First, the paper talked about the system assumptions and requirement before describing the implementation of Dynamo. The storage system had to meet Query models, ACID properties and efficiency. In addition, the clients and services engaged in a Service Level Agreement (LSA). The important requirements of the storage systems are incremental scalability, symmetry, decentralization and heterogeneity.

Second, the paper talked more detail about the implementation of Dynamo to meet the requirements mentioned above. For incremental scalability, Dynamo is able to scale out one storage host with minimal impact on the system. Each node in Dynamo is symmetric, and has the same responsibilities as its peers. In addition, the design favored decentralization, and need to be able to deal with heterogeneity in the infrastructure since the work on the system is heterogeneous. Therefore, meeting these requirements, Dynamo is an important reliable, scalable storage system in e-commerce system.

The strength of this paper is that the flow of this paper is good for readers to study. It first described the background of the work needed in, and the assumptions and requirements. Then, following these motivations, it talked more about the implementation of Dynamo. Thus, it can make readers understand more about the motivation and implementation of the storage system.

The weakness of this paper is that it does not provide many details of the system due to company interest. For example, in the implementation section, it can only described little about the algorithm or mechanism of the system without details.

To sum up, this paper described Dynamo a highly available and scalable data storage system, used for supporting services of Amazon e-commerce platform.

Review 29

This paper was an introduction to Dynamo, a high availability high efficiency and highly scalable data store infrastructure. It was designed by Amazon and had a specific set of needs that apply to the problem that were relevant to Amazon.

There were some main assumptions and requirements that helped enable the development of Dynamo:
1.) The query model is simple read and write operations to a single data item, no operations span multiple data items and there is no need for a relational schema. This is important because it makes partitioning much easier.
2.) ACID properties they are willing to trade off weaker consistency for higher availability. They can not afford to ever go down so availability is huge and studies have shown that if you provide ACID guarantees availability is poor. Dynamo also does not provide any isolation guarantees.
3.) Efficiency is super important for Dynamo. They need to pass benchmarks where the 99.9th percentile of the distribution is better than the requirement they set out.
4.) They have to be capable of handling hardware failures. Similar to the GFS paper we read earlier they are dealing with systems of such scale that things will fail and Dynamo has to be able to handle that without slowup.
5.) The last big assumption is that Dynamo is only used for Amazon internally so it is assumed that there are no security related issues as everything is non-hostile. This allows for no authentication or authorization.

As you can imagine they were able to implement all of these requirements using some created techniques. My favorite technique that I will mention was to provide for high availability with writes they used vector clocks to make sure you can always write in O(1). The reconciliation of the clocks is done during reads so it does not affect writes at all. I thought this was a really cool idea.

There were a few downsides I saw to the paper.
1.) It spent too much time on background. Maybe some people thought this was a necessary part of the paper but it felt unimportant to me after they up front laid out their requirements and assumptions. When they did that it was clear to me that this was a unique system that would not have been designed before and they spent too much time discussion other systems in my opinion.
2.) I’m not convinced that the assumptions that were made for this paper are capable to be used by many other systems. I think the architecture is great for their specific purpose but I was not convinced that there would be many other companies that would need their own Dynamo. They designed a very effective system for a very specific case, which is great for them but I don’t know for how many other people.

Overall though I think this was a good paper. It was similar to the GFS paper in that it describes something they have implemented and have been using and explain how it was done. It is obviously a successful system as it is used in Amazon and with high success. The results also support this and show that it is very high performing. Overall, solid paper that was a good read and hopefully can be used by others.

Review 30

Amazon’s database scheming system, as Professor Mozafari frequently puts it, doesn’t require somebody using the website to input a SQL query to buy a pair of shoes or search for an item. In fact, most of the high concurrent experiences for Amazon are on the order of millions and billions, but they are often data retrieval or simple modification actions (search for an item, buy an item, etc). Thus, the motivation is to build a distributed storage system that is (A) simple, (B) scalable, (C) highly available so it is always on for use, and (D) predictable in its functionality, i.e. it can deliver results in bounded time.

“Customers should be able to view and add items to their shopping cart even if disks are failing, network routes are flapping, or data centers are being destroyed by tornados.”

Most of the optimizations come from knowing that we want availability more than we want consistency; the customer should always be able to do something (e.g. add to cart), and if there are any hiccups we can fix it later. The proposed solution is with the idea of “vector clocks,” which associates context use of an object that keep track of “sub-histories.” Thus one of the jobs is to reconcile multiple versions of an object (which ieads to problems like deleted items resurfacing, or an action appearing as though it wasn’t considered). Another interesting implementation I thought was cool was the “quorum” system by “voting” on the results of a read/write operation in order to tolerate failures, and reconciling the results should any of them conflict.

One of my questions is about how application-dependent this construction is, i.e. when maintaining states and contexts of object communication, how would the application maintain all this metadata? It is also discussed that determining the number of virtual nodes handled can decided based on its capacity, accounting for heterogeneity in the physical infrastructure, but I am not sure how this is implemented? Is this application level or dbms level?

Review 31

Dynamo was designed to provide flexibility in balancing performance, availability, fault tolerance and consistency by allowing the application to fine tune a set of parameters. It is described as being an always on key value system that maintains a weaker eventual consistency model that is resilient to failures and network partitions.

Dynamo is a hodgepodge of common techniques in distributed systems such as replication, consistent hashing, quorum systems, data partitioning and replication. Thus, the algorithms and techniques that are used to create Dynamo are not originated by the authors, but rather a collection of known techniques. The real contribution Dynamo makes is putting all of these algorithms and techniques all packaged into a real world production system. Dynamo also made some unique design decisions including: using a time stamp based system for applying an upper bound to the length of a vector clock. Providing an clients a simple to use interface for reconciling conflicting versions. Allowing users to define persistence storage specification.

As far as flaws go for Dynamo, its very hard to pin point specific weak points. Dynamo is a well designed disturbed key store system that doesn’t seem to have any obvious glaring flaws. While it probably has some flaws that appear when under heavy stress, it seems to be a pretty well designed system.

Review 32

This paper talks about the design and implementation of Dynamo: a highly available key-value storage system that some of Amazon’s core services use to provide “always-on” experience. Amazons business process demands a system with high reliability and scalability, which depends on how application state is managed. Since Amazon implements highly-decentralized architecture consisting of hundred of services in a distributed system, there needs to be storage technologies that ensure availability.

Dynamo is built based on Amazon workload. The system assumptions consists of query models (simple write and read identified by key), the choice of weaker consistency in ACID properties for better availability, the compatibility and configurability of Dynamo by Amazon’s services, and assumption that since Dynamo is used only by internal services, there is less security requirements. The paper also explains about Dynamo’s SLA (which is based on percentile) based on Amazon’s architecture; and design considerations, whose main problems are when and how to resolve conflicting changes. Basically, Dynamo implements “always writable” concept.

After talking a bit about related work regarding peer to peer system and distributed systems, the paper continues with Dynamo’s system architecture and the techniques used to meet the high availability requirement while try to (at least) maintain some degree of consistency. It explains the system interface, partitioning algorithm (using consistent hashing), replication (N parameter and preference list), data versioning (using vector clocks), and the execution of get () and put () operations. It also talks about failure handling and failure detection. The paper presents the implementation of dynamo and the performance measurements. The “lesson learned” section gives six topics (or questions) to be pondered about: balancing performance and durability, ensuring uniform load distribution, the management of divergent versions, client driven vs. server driven coordination (in which client-driven is suggested), and balancing background vs. foreground tasks.

The main contribution of this paper is it provides the evaluation of how different techniques can be combined to provide a single highly-available system – especially in production with demanding applications with distributed data storage – using an eventually-consistent storage system. One thing that stands out is that the writers embrace “system failure” as natural occurrence in the system, thus developing the replication technique and data versioning (with vector clock). Another contribution is it provides the insight to the tuning of these techniques.

However, since this paper is about a system built for Amazon by Amazon, things are a more to the practical side. Some possibilities of inefficiencies are left untouched, simply because the writers have not found them happen in Dynamo when executing Amazon’s workload. For example, in order to maintain the size of vector clock, Dynamo set a certain threshold so that when the number of node in vector clock has reached the threshold, it will truncate the oldest node. Yet, this paper does not explain further if there is any consistency check (i.e.: whether the deleted node is still used by another node). --

Review 33

The purpose of this paper is to present Amazon’s Dynamo system, which is different from many existing sytems because it has the goal of being highly reliably at a hugely massive scale. In this system, component failures are considered the norm rather than the exception, and this paper presents the eventual consistent and highly reliable system used by many of Amazon’s production services. This is important because it is one of the first papers to present eventual consistency in a production system!

The technical contributions of this paper are numerous. The authors present a system with a strict requirement that it be “always on” and highly available with an extremely low probability of system failure. To achieve this, the system is forced to sacrifice consistency of the system and thus must use extensive versioning and conflict resolution in order to revive the system to a state of consistency. The authors present a novel way to do this versioning to allow for the reconciliation of conflicting data across replicas in the case of a failure through the use of vector clocks). Another main contribution of this paper is a system that allows the client (not the users, but rather the internal developer client) to tune aspects of the database to best suit their application’s performance. This tunability is key in a system that is designed to be used by a company such as Amazon since many of their applications have a very diverse set of storage requirements. The other main contribution of this paper is the partitioning algorithm they present and refine. There is extensive analysis of the various strategies for carrying out this hashing before one is chosen and integrated into the final system.

I think this is a very strong paper. The design specifications and assumptions are presented in detail along with justification for decisions. In the results section, though the figures are on unusual pages relative to the placement of their discussion, the results and trends seen in the graphs are explained in detail. I also appreciated the fact that at many points in the paper, especially when discussing the different ways the system could be tuned for different applications, the authors reference example Amazon services that would use the system in that given way.

As far as weaknesses go, I thought there was a little bit of ambiguity regarding the functionality of Dynamo. From the introduction, I was under the impression that Dynamo was a storage system, but in the implementation section, the different kinds of storage engines that Dynamo runs on top of were presented. I think another weakness, that is through no fault of the authors, is that they could not present all of the technical details of the system due to their sensitive nature. It was nice that the authors acknowledged this lack of detail from the beginning, but I do think the presentation of even greater detail would have contributed to the strength of the paper.

Review 34

Review: Dynamo: Amazon’s Highly Available Key-value Store

Paper Summary:
This paper presents the design and implementation of Dynamo. Dynamo is a storage system that some of Amazon’s core services use providing a service that almost never suffers from outage. The primary feature of Dynamo is it’s high reliability as a key-value storage system. To achieve such reliability, Dynamo makes sacrifice on consistency. To solve the consequences of this trade-off, the inconsistency, it applies extensive use of object versioning and application-assisted conflict resolution algorithms.

Details and Reviews:
This paper focuses on the differences between the Dynamo and other systems while presenting the details and implementations of it. The major difference, as mentioned before, was the trade-off between reliability and consistency. This is done in multiple steps. First the data is replicated and stored on different hosts and accessed via a list of nodes and keys. A consequence of replication is the need of consistency control and conflict resolution system. This is done by versioning in Dynamo. Also, in case of failures, the Dynamo system also handles them differently from other systems. Instead of recovery, which would drastically damage reliability, Dynamo uses hinted handoff. Using hinted handoff, there can be some inconsistency in the order of execution. Dynamo then uses Merkle trees for inconsistency detection and repair.

I think the most inspiring point of this paper, or the Dynamo system, is that no system can be perfect in every aspect. A good/applicable system is welcomed because it makes reasonable trade-offs under the given constraints and demands.

A not-so-related question is, given that this is a real-world application paper, what is the estimated financial benefit for Amazon using this system compared to the baseline models mentioned in the paper. E.g., what’s the expected lost saved by implemented this highly reliable system of Dynamo.

Review 35

This paper proposed Dynamo, a highly available, dynamic partitioning, key-value storage system that overcomes constant failure in large-scale system. In order to archive a fault-tolerance high availability system, it sacrifice consistency: instead of maintain a strong consistent model, it use eventual consistency to both accommodate the requirement of high availability (High SLA) and the consistent level for Amazon services.

Dynamo uses a highly decentralized, loosely coupled services oriented architecture to solve the high availability problem, and provide the configuration to let its user choose between reliability and performance. It uses vector clock to keep tracking of the data versions and uses a quorum mechanism to ensure the consistency, and push the context to the client level for reconciliation. The design model of Dynamo can be divided into 3 layers: 1) request coordination layer, 2) membership and failure detection layer, and 3) local persistent engine. Request coordination component is built on top of an event-driven messaging substrate and it handles read and writes operation with state machine mechanism. Dynamo uses a gossip style protocol to maintain an eventual consistent ring membership, and it also uses local failure detection rather than gossip style decentralized failure detection to minimize the overhead. Moreover, it also uses virtual nodes and hinted handoff mechanism to handle the common node failure, and uses a Mekle tree data structure over nodes to handle the permanent failures and archive eventual consistency among replicas. The local persistent engine layer of the system is built to enable multiple storage engines like Mysql and BDB to plugin, provides more flexibility for its users.
Moreover, this paper also does a great review of related works, including previous works in distributed file system and in P2P system, and proves the novelty of Dynamo both conceptually and by real world experiments result.

1. This paper proposed Dynamo, a highly available, dynamic partitioning, auto-scale key-value store distributed system. It introduces the implementation details as well as the design philosophy behind the implementations, which inspire several followers (Facebook Cassandra, Riak, Linkedin Voldemort) in open source community. Together with Google BigTable, it starts the era of Nosql.
2. It introduces the design consideration along with the Amazon SLA for Dynamo, and also the services and background requirements for Dynamo, which gives its reader clearer idea about why the system is designed in a highly available, fault tolerance but eventual consistent fashion.
3. In many parts of the paper, it introduces the limitation of the system: i.e. in section 6.6 it states that since it makes each node of the system to store all membership information, it is hard for the system to scale to tens of thousands of nodes since membership coordination would cause lots of overhead. Moreover, it also gives the resolution of the issues it mentioned, for example, for the scale problem, it mentioned to use the design of O(1) DHT to overcome this issue. This idea has been adapted by its followers, for example Cassandra uses the finger table to only store neighbors and exponential distance nodes to keep enough information while still provide linear scalability for cluster size over than 1000 nodes.

1. In order to protect the business interests, many parts of the paper are not discussed in details. However, this is understandable.
2. The Dynamo system has great performance for large scale concurrent simple transactions, however, its eventual consistent model may fails for multistep complex transactions. This limits the fields it can be applied to.
3. The Dynamo system push the complex reconciliation work to its clients, which may causes the client side code hard to implement, and brings inconvenient to its users.

Review 36

This paper presents Amazon’s Dynamo, a distributed, highly available key-value store. The main design goal of Dynamo is to provide high availability, measured with the latency of 99.9% percentile of latency. The target workload is write-heavy. The low latency of both reads and writes are highly desired. To achieve this goal, Dynamo trade in consistency for high availability, namely only guaranteeing eventual consistency. A typical Dynamo instance consist of hundreds of machines. Data is partitioned and replicated using consistent hashing. Object versioning is applied to provide consistency. Consistency among replicas are maintained by quorum-like approach in updates and periodic anti-entropy jobs. Dynamo doesn’t guarantee strong consistency over writes. Instead, it defers the work of reconciliation inconsistent replicas to applications. It also provides a few knobs, N, R, W, to configure consistency. N is the number of replicas for each object. R and W are the number of nodes that must participate in a successful read or write respectively.

The main advantages of Amazon’s Dynamo include:
The performance, availability and durability are configurable. By tuning the N, R and W, users can set the performance, availability and durability to a desired level. For example, if the service has a strong desire of low write latency, with tolerance to low durability, it can set W=1 so that any write will response right after one node persistently stored it.
Measure the service latency with percentile rather than average latency. In the past work, request latency are commonly measured by average. However, we can see from the production data that the 99.9% percentile latency can be 2 magnitudes higher than the average. The percentile gives a better view of how service latency is perceived by most of users.
Dynamo exposes the data consistency and reconciliation logic to the application developers. The application’s business logic is taken advantage to tolerance the eventual consistency model.

One weakness of Dynamo is using full membership model. Storing the full routing table of the whole cluster is not scalable when the cluster size largely increases, as the overhead of keeping full routing table in each node is super-linear to the cluster size.