Review for Paper: 29-Bigtable: A Distributed Storage System for Structured Data

Review 1

Google has many projects that use very large data sets, which are stored in the proprietary Google File System. GFS provides high availability and fault tolerance for very large files in a distributed network. But for structured data that conforms to a standardized table layout, Google services need a distributed data structure that let the client specify how data should be stored. This can provide better performance for use cases like Google Earth and web search, which may need to enforce that data be stored compressed or not, or stored with certain attributes partitioned together.

Bigtable was created at Google as a distributed map data structure, built on top of GFS, which can be configured by the client to store data in a given tabular schema, with columns and rows grouped as specified, and certain values compressed or not. A Bigtable has three dimensions, including rows (stored in lexicographic order and grouped into “tablets”), columns (grouped into column families with their own access controls), and timestamps. Timestamps are used for versioning; either the last n timestamped versions, or versions created in a recent time window can be kept, with the rest being garbage collected.

Bigtable is built on top of an immutable, persistent table component called the SSTable. SSTables are immutable for performance and simplicity, as garbage collection is as simple as deleting old, unused SSTables. When data in a Bigtable instance is updated, a record of diffs is made to a “memtable” buffer, which is periodically written to an SSTable stored in GFS. Now and then these SSTables of diffs are merged to conserve storage space. To generate a version of the Bigtable, these diffs are applied to an older version.

Although the speed of reads and writes (random or sequential) in Bigtable scales fairly well in performance tests with the number of tablet servers, there appears to be significant overhead in the system on small random reads, due to transferring data from GFS in 64KB blocks. This makes the scaling much worse than linear in the number of tablet servers for small random reads. It might be better for Bigtable to support small reads without sending an entire SSTable block to the requesting tablet server.


Review 2

This paper focuses on Bigtable, a distributed storage system created by Google which manages structured data scaling to a large size. The motivation of creating the system lies majorly in scaling problem. A lots of data is being handled by millions of machines being operated by hundreds of millions of users on different projects. The storage requires a semi structured data to handle such a variety of task. No commercial system was big enough to handle this.

Bigtable is a distributed multi-level map which applies the concepts of fault-tolerance and persistence. It is scalable over distributed systems and is self-managing i.e. servers adjust to load imbalance. Its data model implements a sparse, distributed, persistent multi-dimensional sorted map. The rows contain arbitrary strings, accesses to which are atomic. Column keys are grouped into sets called column families, which form the basic unit of access control. The row range for a table is dynamically partitioned. Each row range is called a tablet, which is the unit for distribution and load balancing. The Bigdata API provides functions for creating and deleting tables and column families. It uses GFS to store its persistent data. MapReduce can also be used for data processing for read/write operations. It relies on a highly-available and persistent distributed lock service called Chubby. For implementation purposes, a single-master distributed system is used with three major components, a library that is linked to every client, one master server and many tablet servers. The implementation required a number of refinements to achieve the high performance, availability, and reliability required. This includes compression of values and maintaining locality groups which groups column families into a single group, and segregating column families that are not typically accessed together, enabling more efficient reads.

The paper is successful in providing a comprehensive study of Bigtable. Performance evaluation is provided based on basic operations projecting its high performance on a large amount of data. Real-world application provide insight into its relevant usage today along with performance evaluation. But the lessons provided while the authors worked on the system provide useful knowledge wrapping up the paper adequately.

However, Bigtable’s rebalancing algorithm does not work perfectly as can be seen in the evaluations as well. Sufficient solutions are not provided to mitigate this problem. Also, user adoption rate and acceptance of a new interface is low.



Review 3

What is the problem addressed?
Bigtable is a distributed storage system for managing structured data that is designed to scale to a very large size: petabytes of data across thousands of commodity servers.
Why important?
Many projects at Google store data in Bigtable, including web indexing, Google Earth, and Google Finance. These applications place very different demands on Bigtable, both in terms of data size (from URLs to web pages to satellite imagery) and latency requirements (from backend bulk processing to real-time data serving). Despite these varied demands, Bigtable has successfully provided a flexible, high-performance solution for all of these Google products. In this paper we describe the sim- ple data model provided by Bigtable, which gives clients dynamic control over data layout and format, and we describe the design and implementation of Bigtable.

1-­‐2 main technical contributions? Describe.
A Bigtable is a sparse, distributed, persistent multi- dimensional sorted map. The map is indexed by a row key, column key, and a timestamp. The Bigtable API provides functions for creating and deleting tables and column families. It also provides functions for changing cluster, table, and column family metadata, such as access control rights. The key contributions may be the decision of implementation. The Bigtable implementation has three major components: a library that is linked into every client, one master server, and many tablet servers.

Tablet servers can be dynamically added (or removed) from a cluster to accommodate changes in workloads. The master is responsible for assigning tablets to tablet servers, detecting the addition and expiration of tablet servers, balancing tablet-server load, and garbage collection of files in GFS. In addition, it handles schema changes such as table and column family creations. Each tablet server manages a set of tablets. The tablet server handles read and write requests to the tablets that it has loaded, and also splits tablets that have grown too large. They use a three-level hierarchy analogous to a B+- tree to store tablet location information. The first level is a file stored in Chubby that contains the location of the root tablet which will never split to make sure there are three levels.

1-­‐2 weaknesses or open questions? Describe and discuss
I would like GFS better which presents implementation better. It would be good if the paper can pose the current challenges and how they overcome them would give reader more clear view about why they need to do the jobs all over again.


Review 4

This paper presents a new design called Bigtable, which is a distributed storage system for managing structural data that needs to scale to a very large size. In short, Bigtable is a sparse, distributed, persistent multidimensional sorted map, which is different from traditional RDBMS. The map is indexed by a row key, column key, and a timestamp. Bigtable is developed inside Google and has been used by more than 60 Google products and projects including Google Analytics, Google Finance, and Google Earth. This paper first gives an overview about Bigtable, as well as client API. Then it describes the infrastructure and fundamental implementations. Following is a discussion on refinements and performance evaluation, including several examples. Finally, it presents some lessons learned and related work.

The problem here is that there is a need for storage system to scale to a very large size: petabytes of data across thousands of commodity servers. However, traditional methods might not provide good enough performance. There are several databases achieving scalability and high performance, such as parallel databases and main-memory databases, but Bigtable provides a different interface. Also, there are many applications in Google place high demands in terms of data size and latency requirements, so a good solution is needed to provide a flexible and high performance.

The major contribution of the paper is that it provides a detailed explanation of the new design Bigtable, including both design concerns and details (ideas, models, API, implementation). Also, it provides detailed performance evaluation with plenty of examples. Here we will summarize the key components of Bigtable below:

1. Index by a row key, column key, and a timestamp
2. maintain data in lexicographic order by row key (arbitrary strings up to 64kB)
3. row range is dynamically partitioned
4. column keys are grouped into column families (basic unit of access control)
5. access control, disk and memory accounting are performed at the column-family level
6. each cell contains multiple versions of same data (two different garbage-collect settings)

One interesting observation: this paper is very innovative and Bigtable has been successful which is used by a lot of applications inside Google. However, the examples shown in this paper vary a lot to many real applications. It might be better if the paper provides some comparison between Bigtable and other relational DBMS.


Review 5

The paper describes big table as a distributed storage system which is Google's columnar key-value store built on top of the Google File System(GFS). Most of the Google projects and applications store their data in Bigtable.Unlike the traditional relational databases where table schema is fixed, Bigtable is flexible to dynamically add and store data in the columns and it can be used with MapReduce for parallel computing.

TECHNICALITIES:
Data Model
The data model of Bigtable is a multidimensional sorted map indexed by row key, column key and a timestamp.The row keys are arbitrary strings.Reads and writes under a row key is atomic.The row range for a table(known as tablets) is dynamically partitioned.The columns are grouped together in columnar families.Multiple versions of the same data can be stored using unique timestamps.The Bigtable data model allows cells to be used as integer counters.Though this model supports row transactions, it does not support transactions across row keys.Again, access control and both disk and memory accounting are performed at the column family level and the 2 per column setting lets the data model to garbage collect old cell versions.

API:
The APIs of the Bigtable provides functions for creating functions for changing clusters, table and column family metadata.Client supplied scripts can run on servers with the help of Sawall. Though the scripts aren't able to write back to the table, it still allows manipulation of the data.

Table Architecture and Chubby Lock Service:
The SSTable file formats which are a sequence of blocks are used internally to store the data.Mapping is done from keys to values and iterations are performed over them.The Bigtable relies on a persistent distributed lock service called Chubby which consist of 5 replicas - one of them is a master which actively serves requests. Paxos algorithm is used for consistency.Chubby locks both at file and block levels.
The three major components of this structure are:
- Library that is linked into every client
- A master server which is responsible for managing the tablet servers and for garbage collection
- Many tablet servers which perform read and write operations in the table
Table can be split whenever needed and each tablet is assigned with one server by the master.Table locations are stored in hierarchical pattern.Chubby is responsible for having only one master at a time and killing tablet servers if they lose their exclusive locks even after attempts to recover them
Updates are committed to a commit log that store redo records in a sorted buffer called memtable.Older updates are stored in a sequence of SSTable.

STRENGTHS:
The paper describes compaction as a process which helps in the process of speeding tablet recovery.Along with this , the paper also defines commit log implementation which benefits performance during normal implementation which though complicated tablet recovery process.It clearly describes the performance of the system using the sequential read and write scans and scaling experiments,At the end, the vulnerabilities of the distributed system are mentioned along with the lessons they learnt while performing the experiments which include proper system level monitoring, value of simple design and delaying of addition of new features until clarity is obtained.

WEAKNESSES:
- Compactions can be expensive and the split and migration of tablets might cause can imbalance on the load system.
- The paper would be more complete if it can introduce some fault-tolerance in Bigtable. E.g., how Bigtable implementation handles the crashes of tablet servers.


Review 6

Bigtable was designed by Google to provide another layer of abstraction over the existing Google File System. GFS sees everything as large, contiguous blocks of data, and this system builds on top of that to allow efficient management of structured data. The system must be adaptable, as Google has a variety of uses cases for the system - some workloads will be latency sensitive, while others are batch-oriented where throughput is more important.

The data model for big table is unique. It uses two "keys" - a row key, and a column key. There is also a timestamp field in the record, and this triplet maps to the array of bytes to be stored. Bigtable maintains data in lexicographic order by row key, so it is advantageous to make sure elements that are often accessed together have a lexicographically close row key. This is done for URL's by inverting the domain - maps.google.com/index.html becomes com.google.maps/index.html. Bigtable also uses "column families" - these are columns of the same type, so that all the columns can be compressed together, saving large amounts of space. Bigtable allows several versions of each pair to be stored - the timestamp for each object is stored, and the different versions are sorted in order of decreasing timestamp.

This system has a good mix of traditional RDMS technologies, while relaxing some constraints to increase scalability. It uses a variety of existing technologies, such as GFS, Chubby, and SSTable. The architecture of the system is similar to GFS - it uses a mater that is responsible for assigning "tablets" to "tablet servers", and clients communicate directly with the tablet servers, much in the same way as GFS master/chunkservers. It is robust and adaptable - it works for services as different as Google Earth and Google Analytics, as well as the web crawler.

The contiguous array of bytes is probably not the best option for data storage - some kind of typing, or structure, could probably be beneficial to application developers. However, the biggest problem is the lack of generic transactions - making users reason about various updates is difficult. Row level transactions are simply not powerful enough.



Review 7

Problem and solution:
Google has the needs to handle the huge quantity of structured data. To solve the problem, the distributed storage system named Bigtable is proposed. It is used to manage data with petabytes size and thousands of servers. Besides high performance and high availability, Bigtable should have a wide range of configurations since its workloads have different demanding. Though Bigtable has the similar strategies to the databases, it still has a lot of difference. One is that it provides dynamic control of data model instead of a full relational data model. And it treats data as uninterpreted strings. Also, the clients can control the locality of the data dynamically by changing the schema parameters. Actually, Bigtable is an ordered map indexed by row key, column key and timestamp. Timestamps are used to track the multi versions of a cell. And Bigtable groups columns families, compresses data on common strings, caches previous readings and so on to improve the system performance.

Main contribution:
I think one of the main contribution is that it combines multiple Google products like Google File System, SSTable, Chubby. Comparing to learn and build with several different infrastructures, join them together is a good practice for the product developers.

Another interesting part is the compression. The method is never met in previous papers. It searches for common strings in the data and compress them together. In this way, it not only improves the performance, but also saves the space.

Weakness:
The main weakness of the Bigtable is that it must be able to support various types of workloads. To accomplish this aim, it needs to have a bunch of configurations, and make a compromise between performance and applicability. I think it would be better if the workloads can be classified into different categories and develop the specified storage system. For example, one for throughput-oriented batch-processing work and one for latency-sensitive work. In this way, the API and protocol of the storage system may be simpler and the system will be more efficient.


Review 8

This paper introduces Google’s Bigtable, a distributed storage system for managing structured data that is designed to scale to a very large size. Bigtable is a sparse, distributed, persistent multi-dimensional sorted map, which is indexed by a row key, column key, and a timestamp. It support atomic read and write to a row. Bigtable is partitioned by row range and each partition is called a tablet. The implementation of Bigtable is relatively simple: Data is stored persistently in GFS in SSTable file format; It uses a distributed lock service, Chubby to indicate state of master or tablet server and store schema information of Bigtable.

One main advantage of Bigtable is its support for massive data. Bigtable is highly scalable and the aggregate throughput increases dramatically as more tablet servers are added. As Bigtable is a sorted map and user can specify a data set to cache, random read is really fast, even when table is extraordinarily huge. In addition, SSTable is fast compressed in block level , which saves disk space and makes disk access more efficient. In Bigtable, data can be stored with access control: user can specify locality groups in each tablet. Locality groups separate column families that may not be accessed together, which makes read more efficient.

There are some drawbacks of Bigtable:
1.Though Bigtable provide availability in most of cases, yet Bigtable becomes unavailable when Chubby becomes unavailable for some time.
2.Bigtable does not efficiently support small random writes and some operators like join. The updates of Bigtable is handled by GFS. Hence, small random write is not optimized just as in GFS. In addition, Bigtable is NoSQL, so some operators in relational DBMS may not be well-supported.


Review 9

Problem/Summary:

Bigtable is a distributed storage system for structured data. Data is structured in tables, which behave like 3-dimensional maps. Each data value is stored as a string of bytes, and is uniquely identified by a row key, column key, and a timestamp.

When designing Bigtable, the authors realized that many clients only perform single row updates. Therefore, single row updates in Bigtable are atomic. The rows in a table are also sorted by the row key, which means that entries with similar names alphabetically will have good locality to one another. Because tables are horizontally partitioned, this means that clients accessing a small range of keys may only need to talk to one node.

Column keys have a column family component and a column name component. Columns in the same family are all the same type, and can be compressed together.

Timestamps are usually used to represent different versions of objects over time.

The implementation of Bigtable uses a master-slave system, which uses one master server to control many tablet servers. Tablet servers contain tablets, which are the aforementioned horizontal partitions of tables. Information on data location is stored in a three level hierarchy- Tablet locations are stored in special metadata tables, which are in turn pointed to by a root tablet.

Strengths:
The data model was explained clearly, and the authors show that the system is useful for real world applications by giving examples of how different Google projects make use of Bigtable.

Weaknesses/Open questions:
The biggest question I have is, how does Bigtable solve problems that are not solved by other systems mentioned (like parallel or distributed databases). Is it just that it is a key value store? Is the performance better?



Review 10

The paper describes a distributed storage system that is being used at Google, called Bigtable. Bigtable is aimed to provide a service that manages structured data that scales to a very large size, petabytes of data across thousands of commodity servers, which is not surprising considering the amount of data that Google handles throughout its applications.

Bigtable is basically a distributed multi-dimensional sorted map, where its keys consist of row and column string keys and a timestamp and its values are an uninterpreted array of bytes. The authors take this generic approach for their data structure in order to support a wide variety of applications that are serviced at Google.

There is nothing that greatly surprises us about Bigtable. Its structure reminds us of parallel and distributed database systems. The main strength of Bigtable is its utilization of other frameworks explicitly available at Google. Bigtable uses Google File System (GFS) to store its data and a distributed lock system called Chubby for its service management. Bigtable itself would be so much more difficult to implement without such frameworks in hand. Google has done a great job of combining multiple principles that they have learned from different distributed systems (e.g. GFS) and building Bigtable upon their existing infrastructures.

The problem with Bigtable is that it is users’ responsibility of figuring out how to use it efficiently. Bigtable is required to be flexible and has a such generic data structure in order to support a large number of applications at Google, but even the paper recognizes the problem and states that there has been a question on how difficult it has been for users to adapt Bigtable. The authors justify by saying that many Google products that use Bigtable work sucessfuly in practice, but this does not actually justify its usability and maybe it is the result of Google promoting and dictating the internal use of Bigtable and people adapting to it nonetheless.

In conclusion, Bigtable is a distributed storage system for a wide variety of applications at Google. Bigtable works in harmony with other frameworks developed previously at Google, such as Chubby and GFS. Looking at the number of applications that use Bigtable and work successfully, it is hard to doubt its capability of managing very large-scale data, but we should question its usability at least.



Review 11

This paper describes Google's Bigtable storage system. Bigtable is a distributed storage system that is an alternate to relational databases. Unlike parallel or distributed databases, Bigtable assumes data to be simple strings rather having a strict structure and it allows users to control data locality through clever schema choices. Bigtable stores data in a multidimensional map that is indexed by a row key, a column key, and a timestamp. The row key is also used to order the information. A range of row keys is the partitioning unit used by the system and is called a tablet.

The design of Bigtable is a single master distributed system, which means that the master is not involved in data reads or writes. All reads or writes are assigned to other servers called tablet servers. These servers maintain information about the tablets they are responsible for and the master periodically queries the servers to obtain tablet information. Bigtable also uses several other systems to maintain different parts of the system. The Google File System is used to store logs and data files and Chubby is used to provide a distributed, highly available lock service.

Bigtable uses several interesting optimizations to achieve high performance. For example, the designers realized that read operations had the potential to perform a large amount of disk accesses if the data they needed was not in memory. Since Bigtable is designed for large data storage, these accesses would be too much overhead. To lower the overhead, they allowed the use of Bloom filters in order to indicate whether an SSTable contained data based on the key in question. Bigtable also performs predicative fetching in that if a certain data object is requested, the system will fetch the entire block assuming that other parts of the block will be used. The system uses this mechanism as well as two different caches in order to provide higher read performance.

At the end of the paper, there is a single sentence questioning the ease of use of Bigtable. The paper simply states even though new users are unsure of how to use the interface, the success of Bigtable at Google makes that problem meaningless. However, I think this is an unfair point. Bigtable is developed at Google so it would make sense to me that their applications would use it regardless of how easy it is to use. The problem comes when other database administrators and users of the database need to understand the system. It is not trivial learning a new API that differs from traditional relational databases. Furthermore, this API is very specific to learn and only applies to big table, unlike the API for relational databases. I think C-Store for example, although using a column storage system, would be much easier to learn for users considering it is based on a familiar interface. I think Google's dismissal of the difficulty is only allowed since they designed it.


Review 12

This paper introduces Bigtable, the distributed storage system for managing structured data at Google. Bigtable is similar to database but does not support a full relational data model. Its data model is a sorted map indexed by a row key, column key, and a timestamp. Row ranges are used to horizontally partition the data for distribution and load balancing. Column families are sets of column keys that serve as unit of access control. Timestamps allow clients to keep multiple versions of the same data. Bigtable garbage-collects data versions with old timestamps automatically.

The underlying infrastructure used by Bigtable is also presented in this paper. It uses distributed GFS to store log and data files, SSTable file format to store Bigtable data, and Chubby as its distributed lock service.

As many other single-master distributed storage systems, Bigtable also assigns one master server and many other tablet servers. However, different from the GFS architecture we have seen before, the clients do not rely on the master for tablet location information. In contrast, clients communicate directly with tablet servers. The tablet location information is stored in a three-level hierarchy: first metadata tablet (root), other metadata tablets, and the tablets. The GFS keeps the persistent state of tablets. Bigtable keeps the most recently committed updates in "memtable" in memory. When memtable grows large enough, it is written to GFS in SSTable format.

This paper also provides some experimental statistics and some real applications using Bigtable. The experiments are conducted on several operations given different number of tablet servers. Unfortunately, it is shown that the performance (per-server throughput) does not scale linearly; the random read operation shows the worst scaling ability. At Google, Bigtable is used extensively in many applications or for development purposes on 24,500 tablet servers.

The major contribution of this paper includes:
1. It introduces the data model, system design, and implementation of Bigtable.
2. It discusses several refinements that can be made on top of Bigtable to improve its performance.
3. It shows the performance of Bigtable with experiment statistics on frequently used operations and scales up the number of tablet servers.

Some drawbacks of this paper are:
1. The per-server throughput does not scale linearly. (However, the paper does state that the users are satisfied with its performance.)
2. It would be better if they can describe in more detail for choosing the three-level hierarchy other than conventional B+-tree structure.



This paper introduces Bigtable, the distributed storage system for managing structured data at Google. Bigtable is similar to database but does not support a full relational data model. Its data model is a sorted map indexed by a row key, column key, and a timestamp. Row ranges are used to horizontally partition the data for distribution and load balancing. Column families are sets of column keys that serve as unit of access control. Timestamps allow clients to keep multiple versions of the same data. Bigtable garbage-collects data versions with old timestamps automatically.

The underlying infrastructure used by Bigtable is also presented in this paper. It uses distributed GFS to store log and data files, SSTable file format to store Bigtable data, and Chubby as its distributed lock service.

As many other single-master distributed storage systems, Bigtable also assigns one master server and many other tablet servers. However, different from the GFS architecture we have seen before, the clients do not rely on the master for tablet location information. In contrast, clients communicate directly with tablet servers. The tablet location information is stored in a three-level hierarchy: first metadata tablet (root), other metadata tablets, and the tablets. The GFS keeps the persistent state of tablets. Bigtable keeps the most recently committed updates in "memtable" in memory. When memtable grows large enough, it is written to GFS in SSTable format.

This paper also provides some experimental statistics and some real applications using Bigtable. The experiments are conducted on several operations given different number of tablet servers. Unfortunately, it is shown that the performance (per-server throughput) does not scale linearly; the random read operation shows the worst scaling ability. At Google, Bigtable is used extensively in many applications or for development purposes on 24,500 tablet servers.

The major contribution of this paper includes:
1. It introduces the data model, system design, and implementation of Bigtable.
2. It discusses several refinements that can be made on top of Bigtable to improve its performance.
3. It shows the performance of Bigtable with experiment statistics on frequently used operations and scales up the number of tablet servers.

Some drawbacks of this paper are:
1. The per-server throughput does not scale linearly. (However, the paper does state that the users are satisfied with its performance.)
2. It would be better if they can describe in more detail for choosing the three-level hierarchy other than conventional B+-tree structure.



Review 13

Bigtable is a distributed storage system for structured data that was created at Google around 2005. Bigtable provides high availability and performance and is easily extensible by adding more machines to their cluster. This system was pretty widely used at Google at the time of publication of this paper in 2006 and resembles a relational model. However, the data structure implemented is a sorted map and it gives users control over data locality and whether data is served from memory or disk. The paper describes how data is stored, how to interface with Bigtable, and how this system was implemented. The authors describe three real applications and describe the lessons learned in the year that this system was in use before publication.

Like other Google papers this paper received thorough testing before publication. This makes it harder to critique the paper, as section nine describes several initial flaws of the system. They mention that many distributed components can fail and that they have taken steps to resolve these issues. The rest of section 9 is related to software engineering and development issues which are less interesting in terms of database insight.

There are more strengths than drawbacks of this paper. Most notably the paper describes a system that has been used at Google on large scale systems before publication. The authors describe the real world applications of Google Analytics, Google Earth, and personalized search. This describes a variety of workloads and data. The paper has a performance evaluation section which describes run times of various operations on a 1786 machine cell. This gives us a good idea of retrieval times that one would expect using Bigtable and how the performance scales with number of servers. Overall this paper provides a useful method for data storage and a convincing description of the performance and implementation of this system.



Review 14

Part 1: Overview
This paper presents a distributed database system for large size data managing, called Bigtable. The target data size would be petabytes and would be across thousands of servers. Bigtable takes some mechanism designs from parallel databases as well as main memory databases. Bigtable provides users with a simple data model instead of a relational one thus users need to build their own data format. Bigtable is a sparse, distributed, persistent sorted map where records are indexed by row and column keys and also timestamps. Bigtable is built on other Google infrastructures including GFS. Records are stored as SSTable files which is often 64KB in size. Distributed locks are used to support Bigtable. Chubby locking service consists of 5 replicas and one of them performs as master machine. When time stamps expire the applications would automatically lose all locks.

The Bigtable service is provided by one master server along with several tablet servers where tablet servers can be dynamically added or removed. The master server would take care of assigning tablet servers. Flow control is also done by the master server where this server is responsible for balancing tablet server workloads and handling schema changes. Three-level B+ Tree is used to store tablet nodes. Refinements including locality groups, compression, read results caching, bloom filters and commit logs are also introduced. When moving from one tablet server to another tablet server the source server would first do a self compaction to help speed up the moving. All of the generated SSTables are immutable and thus can help avoiding synchronization control. Therefore garbage collection process would take care of permanently removing deleted data.

Part 2: Contributions
Bigtable is actually used in Google web indexing, Google earth, and Google finance. Applications requires different stuffs from Bigtable including low latency, different data types, and data size. More than 60 applications inside Google are using Bigtable now. Bigtable gained huge success in practice.

SSTable files can be fetched into memory and then whatever method, binary search or table scan can both be done without touching the disks.

Part 3: Drawbacks
Bigtable does not support full relational data model. Programmer’s labor is needed to dynamically design the data layout and format.



Review 15

The paper discussed about BigTable, a distributed storage system for managing (semi-)structured data. The main problem addressed by BigTable is that there are a lot of (semi-)structured data at Google such as urls and geographic locations which are large in size. In addition, millions of users are sending request in parallel. Big table addressed these issues by scaling its storage system up to petabytes of data across thousands of servers. The basic data model of BigTable is a sparse, distributed multi-dimensional sorted map. A (row key, columns, timestamp) is mapped to cell contents. This data structure is used by most Google applications. The row key is an arbitrary strings assigned per every read or write regardless of the number of different columns being read or written in the row. This makes it easier to reason about concurrent updates to the same row, as every read/write events are recorded using the new row key. The data is ordered by row key and partitioned into different range called “tablet”. The column part contains a family:qualifier pair. The family is the attribute value where as the qualifier stores what this attribute is referencing to. For example “language” could be family and the qualifier could be the name of the url it is referring to. The timestamp is used to store different versions of data in a cell. It allows easier management of concurrent operations. BigTable is designed using different building blocks. It uses GFS to store persistent data. It uses a scheduler to schedule jobs onto different machines. In addition, it uses a distributed lock service called “Chubby” to ensure that there is one active master at any time to actively serve requests. Furthermore, BigTable can be used by MapReduce as its input source and output target.

In addition, the paper discuss different components for implementation of BigTable. The first one is a library that is linked to every client which handle client request. The other one is a master server which is responsible for assigning tablets to tablet servers and does the overall management including garbage collection and handling schema changes. Each tablet server handles read and write requests to ten to a thousands tablets. Furthermore different refinements has been added to BigTable . One of the refinement is creating locality groups, which contains multiple column families. Segregating column families that are not typically accessed together into separate locality groups enables more efficient reads. For example by segregating content data and metadata into different locality groups, an application which need to access the metadata does not need to read through all of the page contents.

One of the main strength of BigTable is that it can scale to thousands of machines, which is important for serving millions of users with a lot of data by doing large scale structured data processing. I like the fact that BigTable uses existing building blocks such as the GFS and MapReduce instead of reinventing wheels. In addition, the authors have discussed a considerable number of real world applications including “Google Earth” in order to discuss the importance of scalability. This makes it easier to understand the various features of BigTable in a complex real-world applications. Furthermore, the paper points out that structured data is important at this time when more focus is being given to unstructured data.

One of the main limitation of the paper is that the quires are assumed to be simple and cannot handle complex queries. This is a problem if BigTable is adopted for complex database systems in addition to specific applications at Google. In addition, BigTable relies on users identification of which data should stay in memory and which should be on disk rather than dynamically identifying it. This might be a problem as users might not be expert enough to different data types. It would have been better if BigTable dynamically identify hot-data and cache them in-memory. Furthermore, BigTable does not consider using views and indexes which are found to be important aspect of RDBMSs. While not using views and indexes might simplify distributed large scale storage, it might impact the performance negatively for complex structured database systems.



Review 16

===overview===
In this paper, the distributed storage system across, again, commodity servers in Google, Bigtable, is introduced. It is designed to hold structured data that is designed to scale to a very large size. It is also a flexible solution for difference requirements on latency. In this paper, they provided an overview and discussion of the design and the implementation of Bigtable.

To summarize the properties of Bigtable:
1. flexible (wide applicability)
2. scalability
3. high performance
4. high availability

Clients are responsible to control the locality of their data by using the simple data model provided that supports dynamic control over data layout and format. A Bigtable is a sparse, distributed, persistent multidimensional sorted map, and entries are indexed by a row key, column key, and a timestamp. Besides, clients have the ability to dynamically control whether to serve the data out of memory or from disk.

The paper also talked about the implementation details. Bigtable is built on top of GFS and a lock manager, called Chubby. They applied a three-level hierarchy analogous to that of a B+ tree to store tablet location information. Multiple refinements are applied to the design to speedup the readings and increase reliability.

===strength===
As always, papers from Google have a lot of real world applications to support them. For this one we see that the system is widely used for multiple Google projects, like Google Earth. The way they implemented Bigtable was usage oriented. What matters most is not how complex and elegant a design is, rather, is how we put simple stuff together.

===weakness===
Actually, the usage of Bigtable is very limited. Using it for multiple big Google projects doesn't mean that it is flexible enough, for example, it doesn't solve locality problem for multi-dimensional indexed data sets. But undeniably, Bigtable serves some projects very well.



Review 17

This paper describes how Google’s distributed storage system for managing structured data, BigTable. Although it is used to store structured data, the structure of the stored data is simple. BigTable stores its data using key-value store. Data is treated as an uninterpreted array of bytes and can be looked up using three keys.
This map can be described as:
(row: string, column: string, time: int64) -> string
Rows are arbitrary strings and used for partition. Columns are grouped into logical column families and timestamps are used mainly for version control.

For implementation, Bigtable depends on other Google infrastructure. It relies on GFS for cluster management and Chubby for distributed lock service.
To store the data tablets described above, BigTable has a three-level hierarchy analogous to B+. The location of the Root tablet (level 1) is stored using Chubby. This root level in turn stores all other Metadata. And those metadata contains location of user tables. In this configuration, location of every table can be found very quickly.
Another problem is tablet assignment. BigTable has a master server to manage the addition and removal of tablet server automatically. When a tablet server is added, a chubby lock is acquired for it, and some table will be assigned to it. And when a tablet server is no longer reachable, table stored on it is reassigned.
To support reading and writing, two layer of table is used. The first layer is caller memtable. This table remains in memory and all new writes will be written into in. When a memtable reaches certain size. It will be converted to a SSTable and moved to disk. All these tables will be kept in chronological order, so read will need to scan tables until there is a match.

Contribution:
This paper provided a method to store massive amount of data using a simple interface. And it gives good performance. It created this new storage method using memtable and SSTable. This increases write performance dramatically. And this is also what I like most.

Weakness:
BigTable depends on to many infrastructures, and this makes it hard to distinguish features belongs to BigTable from features belongs to other projects.



Review 18

Bigtable: A Distributed Storage System for Structured Data:

In this paper, the new google data storage product called BigTable is introduced. The primary reason for developing this storage product is that the size of data set that google product is working on is very big and the not all of them can be simply managed as documents by Google file system.

Bigtable is a high performance distributed storage system that supports large scale structured data. Although In many ways Bigtable resembles a database, it does not support any relational data model. The author compare it to be a sparse and persistent multi-dimensional sorted map, which is indexed by a row key, column key, and a timestamp. The word ‘sparse’ means that not each row should contain all the columns and the empty space is not wasted. Row keys can be arbitrary strings (e.g. URL) and each row can support atomic read/write operation or transactions. Rows in the
Bigtable are lexicographically sorted and table is partitioned into row ranges (called tablets). Column keys are grouped together into column families which form the basis of access control. Each column family in bigtable can contain multiple columns and a qualifier can be used to access columns within a family. The timestamp are designed to indicate multiple versions of data within a certain cell.

BigTable is not a stand alone product in the data infrastructure in google, instead, it is a new storage system build on top of many mature google data infrastructures. For example, Google file system is used to keep logs and data files. SSTable is used to keep the tablet in a sorted order by the keys. The distributed lock service called Chubby is exploited to ensure consistent replication. The three major components implemented by BigTable are master server, tablet server and library linked to each customer.

BigTable is a novel internal data storage in google, and it carries so many traits: the first one is that, it supports large scale data, in peta bytes. The second one is just like shown in the performance test, that Bigtable can provide very good random read/write speed and the sequential access speed is even better. The third one is that it scales to larger data size by increasing the number of commodity machines when the data size grows. And the last one is kind of special within google, since so many rows of data from the web source(like crawling result) can be overload at different timestamp, a multiversion cell support is essential for writing efficient programms.

And there are also a few cons for big table:

1.Since BigTable is built upon many other google infrastructures, it’s stability may be affected but them. Say, if Chubby becomes unavailable for an extended period of time, Bigtable becomes unavailable.

2.it does not support a full relational data model. So the programmer still can’t use a high level DML. Also it does not support transactions over multiple rows

3. In the experiment section, not performing comparison against other systems. It would have been interesting to see how relational database/ other in memory DBMS performs under the same scenario.

4. Once an SSTable is written, it’s never changed. If the data in the tablet is changed, a new SSTable needs to be push to GFS. So reader may wonder: would small writes suffer?






Review 19

This paper talks about Bigtable which is a distributed storage system for managing structured data inside the Google. The data model provided by Bigtable gives clients dynamic control over data layout and format. The Bigtable has achieved several goals like scalability, availability.

The Bigtable is a sparse, distributed multi-dimension map, each data cell is indicated by row, column, timestamp triple. The table is sorted based on lexicographic order of rows. The row ranges for a table is dynamically partitioned. Each row range is called a tablet. Column keys are grouped into sets called column families. Each cell in Bigtable can contain multiple versions of data, each indexed by timestamp. Data is stored in decreasing timestamp order, so that most recent data is easily accessed.

Bigtable APIs provide functions for creating/deleting tables, column families and changing cluster, table and column family metadata such as access control rights. Bigtable uses GFS to store log and data files. SSTable is used internally to store data files. Bigtable relies on a highly-available and persistent distributed lock service call chubby, it stores the root tablet, schema information, access control lists and synchronize and detect tablet servers.

The Bigtable implementation has three major component: a library linked into every client, one master server and many tablet servers. The master is responsible for assigning tablets to tablet servers, detecting the addition and expiration of tablet servers and load balancing. So, the master node can simply monitor the related directory on chubby server to get a full view of the whole system. Each tablet server manage a set if tablets and handle read/write.

Strength:
This paper talks about Bigtable which has high performance, scalability and availability. It has been successfully deployed in real apps like GoogleMaps, Google earth etc.. It has great flexibility in designing data model, control over implementation and allow user to tune the configuration of storage.

Weakness:
(1) Very expensive for some sql operation like join.
(2) May save same data in different places thus expensive to update.


Review 20

This paper presents the BigTable database that uses a sparse, multi-dimensional key-value data model. BigTable was built to scale to large datasets and handle read- and write-heavy workloads on commodity hardware. The authors achieve this by layering BigTable above two other distributed storage systems.

BigTable stores data by storing new writes in a memtable then flushing these into an SSTable once full. The SSTables are sorted an immutable, which allows BigTable to provide fully-consistent views into the data on top of GFS which is eventually consistent. Because the SSTables are sorted, they can be searched very quickly and BigTable is not required to build indexes to look-up data.

Distributed, shared-nothing databases are important for scalability and avoiding single points of failure. BigTable's design has a number of advantages:
* BigTable is able to handle mixed read-write workloads (unlike C-Store which is optimized for read-heavy workloads)
* BigTable is able to compress data well by compressing columns in the same locality group
* By caching tablet locations in clients, BigTable reduces lookup latency

However, the paper also has some shortcomings:
* BigTable does not provide multi-row transactions (it does provide single-row transactions)
* BigTable provides a very simple API with few operations
* BigTable depends on Chubby which may lose availability under network paritions
* The paper had a minimal evaluation section




Review 21

This paper introduces a new distributed storage system called Bigtable, designed to hold very large scaled projects across thousands of commodity servers. The database was created by Google and is now used on many of the projects in Google including Google Earth, Google Finance and Google Analytics. Bigtable brings a widely applicable, scalable, highly available and fast performing database to the industry.

Even though Bigtable resembles a traditional database, its interface is very different. To index into a value in the database, we need to have a row key, a column key and a timestamp. Row keys in Bigtable are arbitrary strings that can be up to 64KB and is how data is distributed and load balanced. In other words, we will always have items with the same row key in the same partition or tablet. Column keys are grouped into column families with the syntax family:qualifier, where the qualifier can be any arbitrary string. Finally, because each cell can have multiple versions with the same row and column keys, the timestamp is used to differentiate between the different versions. The values of the timestamps must be generated by the application to avoid collisions.

Bigtable is implemented as three major components: a library, a master server and many tablet servers that store the cells. The master server is in charge of assigning each tablet to the correct tablet server. To find out which tablet server a tablet is located, we go through a three level B+-tree with the first two levels just being metadata.

Overall, the paper does a good job of outlining how Bigtable works and why it is more efficient. However, I still have some concerns about the paper:

1. The paper mentions that Bigtable cannot support traditional transactions, but that Google’s products have been successfully using Bigtable. How does one get around using traditional transactions?



Review 22

This paper discusses Bigtable, a distributed database optimized for efficient access to huge datasets ranging from a few terabytes to a few petabytes. Bigtable uses a key-value system rather than a relational model and supports client control over data layout and format. It also allows clients to choose whether data should be served from memory or from disk.

Values stored in Bigtable are identified by a row, column, and timestamp. Row keys are generally selected such that data in the same row is related underneath that key. For example, a row key might be a domain name, and the columns in that row may be the contents of a web page and a few anchors. Within a cell, multiple versions of a value can exist, each identified by a unique timestamp. These values are ordered from most recent to least recent, and Bigtable allows clients to specify that wither the N most recent copies should be saved, or any copy younger than time T should be saved.

Bigtable uses the Google File System for to store log files and records. One of the keys to its high performance is that records are stored in SSTables, which are immutable memory maps. Since these tables are readonly, concurrent access for reads requires no overhead for access management. Additionally, these tables can be mapped to main memory, so clients can serve data that is most important or most frequently accessed without incurring disk I/O. Write operations are written to the commit log and inserted into an in-memory table called the memtable. The memtable is periodically frozen and converted into an SSTable in order to manage the amount of main memory used by each server. Bigtable also manages space by periodically merging all SSTables into a single SSTable in what they call a “Major compaction”. This process allows Bigtable to reclaim space used to hold references to deleted data and ensures that garbage collection is efficient and timely.

My primary concern with this paper is that they offer no comparison of how Bigtable performs in relation to existing databases. In particular, I would have liked to see a comparison with another key-value store such as Amazon’s Dynamo system, but it would also have been beneficial to show a performance comparison with one of the major parallel DBMSs or a columnar database such as C-Store. While the experimental results that they provide are enlightening, it would be nice to see a direct comparison showing how Bigtable (theoretically) outperforms other systems on the huge datasets it is designed for.



Review 23

This paper introduces Bigtable, which is a distributed storage system for managing structured data in a scale to petabytes across thousands of commodity servers. Bigtable is designed to meet the need of many Google projects, such as Google Earth, and Google Finance. The applications have various demands, which range from throughput-oriented batch-processing jobs to latency-sensitive serving of data to end users. Thus, Bigtable plays a big role in Google to support the storage for their various projects.

First, the paper describes the data model and the APIs of the Bigtable. The Bigtable is a distributed, persistent multidimensional sorted map, and this map is indexed by a row key, column key, and a timestamp. The row keys in a table are arbitrary strings, and are stored in lexicographic order. In addition, The row range is dynamically partitioned, and each row range is called a tablet. Column keys are grouped into sets called column families, which form the basic unit of access control. Each cell in a Bigtable can contain multiple versions of same data, which are indexed by timestamp. For the APIs, client applications can write or delete values in Bigtable, look up values from individual rows, or iterate over a subset of the data.

Second, the paper talks about the implementation and the application of the Bigtable. It consists of a library that is linked into every client, one master server, and many tablet servers. The master is responsible for assigning tablets to tablet servers, and each tablet server manages a set of tablets. The tablet location information is stored in a three-level hierarchy. Each tablet is assigned to one tablet server at a time. The authors also mention some refinements for efficiency, including locality groups, compression, caching, bloom filters, and commit-log implementation. Bigtable has been used to support many Google projects, such as Google Analytics, Google Earth, and Personalized Search.

The strength of the paper is that it provides an overview of the Google Bigtable, including the motivation, data model, implementation, and some refinements. It is good for readers who want to know about Bigtable. In addition, it can also provide a design example for a huge-size, distributed database.
The weakness of the paper is that it does not provide details about how Bigtable is used for various Google projects. It only mentioned some projects that use Bigtable. However, I think it is better to describe more about how these projects use this data model, which can make readers more clear about the application of Bigtable.

To sum up, this paper introduces Bigtable, which is a distributed storage system for managing structured, large-sized data across thousands of commodity servers.



Review 24

This paper is an introduction to Google’s BigTable large data storage solution. BigTable is a multidimensional map, not a relational table, that was created because of Google’s need of high availability, high performance, massive amounts of data, large scale up, and many flexible types of data. It is unique in that every row can have a different set of columns and all the data is accessed in a map format.

The data model is each row has a unique row identifier, as many columns as it wants, and a timestamp for every entry. You are capable of storing the last k entries in terms of timestamps or all entries that happened in the last k amount of seconds. You are also able to create column families that are columns that all store the same type of data, which can help for storing by column family rather than by row.

BigTable is built on top of GFS and uses map reduce for computation. Similar to GFS (how it uses master->chunk servers->chunks) BigTable uses a master that controls tablet servers that serve tablets, which is where the data is stored. All information is persistent as it uses GFS for it’s storage. All writes are written to a single commit log on the tablet server and group commit is used to increase performance.

In terms of performance, as you could imagine BigTable performed extremely well and had a near linear scale up for throughput as you threw more machines at it. This is significant because it is designed using cheap machines so it is relatively easy to throw more machines at it.

One complaint I have about this paper is that it doesn’t do a good job in my opinion of using examples for the need for BigTable and why it had to be built rather than just using existing parallel DBMSs. The could have been a better narrative of that in the introduction to exemplify why it was built. Also I do not think it did a great job of explaining Chubby (the locking service it uses) and some other core implementation features. More examples, images and in depth explanation could have been done on that front for sure in my opinion.

Overall though, I think this was a solid paper and BigTable is very important and is used largely within Google now for a large amount and wide variety of projects. This is a very important paper and it was definitely worth the read.



Review 25

BigTable is yet another Google technology built for managing large amounts of structured data in response to the growing demands of highly available, highly scalable distributed storage systems. It is built on top of the Google File System and as such does not support fully relational DB scheming. However, the motivation was not to build a fully relational database, but a client-controlled storage system that allows flexibility of control over how exactly data is served as well as its distribution across multiple nodes.

Multi-dimensional data is stored in BigTable as a sparse map of cell data (i.e. row key, column, timestamp) which can contained versioned storage information. “Tablets,” another abstraction, store row data to be local to each other in order to facilitate transactional consistency. Row keys are stored as strings and are the unit of transactional consistency, but are stored in lexical order so most recent data is more easily accessed. BigTable also relies on several other services, primarily Chubby and SSTable; Chubby is a highly available and persistent distributed locking mechanism to ensure atomic transactions. This row-level atomicity is important for handling read and write request for tablets split between tablet-centric servers. SSTable is a key-value based storage system for ordered immutable data, optimized for GFS (with the option of in-memory caching). Block indexing is used to navigate the disk, requiring usually only one disk read to access a specified chunk of physical data.

The advantages of BigTable seem to be mainly that it doesn’t come with all of the overhead that traditionally comes with RDBMS systems, such as consistency checking and joins, etc. However, there is no ACID guarantee; it is another eventually consistent system that is good for usually relaxed scenarios. However the consistency seems to be for single row updates, not multi-row or cross-table updates, so applications that use BigTable would have to do application-side checks themselves.


Review 26

The paper talks about Bigtable, a distributed storage system for managing structured data that is designed to scale a very large size (petabytes). Developed by Google, the motivation behind it is that Google has many applications that vary in workloads (from throughput-oriented batch processing jobs to latency-sensitive serving of data) with a wide range of configurations.

The paper starts with Bigtable data model. Each row has row key that is an arbitrary strings, and ranges of row is called a tablet, which serves as the unit of distribution and load balancing. Every read or write under single row key is atomic. Column keys are grouped into column families (access control and both disk and memory accounting are performed at this level). Bigtable uses timestamp, where each cell contains multiple version of same data (limited to recent 3 versions). Bigtable provides API for creating and deleting tables and column families, as well as changing cluster, table, and column family metadata. Bigtable can also be used with MapReduce. The major component of Bigtable implementation: a library that is linked to every client, one master server, and many tablet servers. The master assigns tablets to tablet servers. The paper further explains the location structure of the tablets (from chubby file to root tablet to metadata tables to user table), tablet assignment, and tablet serving (how it is stored and called in the Google File System), and compactions. Bigtable stores log and data file in SSTable format in GFS. The SSTable is immutable and persistent. The paper continues with several refinements on Bigtable(locality groups, compression, bloom filters, caching, commit-log implementation, speeding-up recovery and exploiting immutability), performance evaluation, real application (taking example of Google Earth, Google Analytics, and Personalized Search), lesson learned, and related work.

The main contribution of this paper is explaining the works of Bigtable (its data model, API, building blocks, and refinements) to cater to different application requirements in a distributed system, how it is used by Google applications, and – importantly – lessons learned. The “lessons learned” part consists of challenges and implemented solutions in real-life system implementation. Lessons learned in implementing Bigtable are: (1) large distribution systems are vulnerable to many types of failure, (2) it is important to delay adding new features until it is clear how the new features will be used, (3) it is important to have proper system-level monitoring, and (4) the value of simple designs.

However, as is mentioned in lesson no. 3, Bigtable needs system-level monitoring. It is run in a shared pool of machine with other applications (after all, it is for this kind of environment that Bigtable is created), so the system management depends a lot on the machine. I think the paper should discuss more about the dynamics between Bigtable and the system/CPU, such as how Bigtable transaction competes with other transaction in the system, or how the system concurrency control affect the work of Bigtable.--



Review 27

The purpose of this paper is to introduce Bigtable, a high-performance, flexible, distributed data storage system that is designed to scale to very large datasets that contain structured data. Bigtable also provides a lot of flexibility to the user in the specification of data layout and format within the Bigtable system.

Though Bigtable is a storage system, the authors point out the similarities it shares to database systems as far as strategies in implementation go. It uses a much simpler data model than a relational database, but it stores data indexed in rows and columns, and the schema parameters give clients lots of flexibility regarding the locality and location of their data. I think one of the main contributions of this paper is its combination of many existing techniques in a new way. They also provide a framework so that Bigtable can interface with MapReduce either as an input or output data structure.

One major strength of this paper is its strong grounding in existing Google features. The fact that it utilizes existing, well-tested features such as Google File System for the underlying file system and SSTable to map from keys to values lends a large about of legitimacy to the system. They also use an existing distributed lock service called Chubby. Not developing all of these components on their own from scratch in theory makes their system more reliable. I also thing the very specific real-world examples they give toward the end of the paper also lend to the strength. They show that this system is actually used in production systems.

I think one weakness of this paper is the amount of implementation that is left to the user. We discussed this in class as a downside of MapReduce too. By forcing the user to implement and tune many components of the system, this puts more pressure on the system administrator to do things properly rather than relying on optimizations in the system to handle things. s One specific example of this is the fact that all data entries are stored as an uninterpreted array of bytes. This requires more work on user-end to keep a standardized format and translate/untranslate from this byte array.


Review 28

Review: Bigtable: A Distributed Storage System for Structured Data

Paper Summary:
This paper presents a data storage named Bigtable. Bigtable is a solution to the increasing demand of applications on structured data storage with respect to data size and latency. The most significant achievements of Bigtable are its applicability, scalability, high performance, and high availability. Comparing to a database system, Bigtable does not support a full relational data model but instead provides clients with a simple data model supporting dynamic control over data layout and format, and allows clients to reason about the locality properties of the data.

Paper Review:
One significant contribution of the technique presented in this paper, Bigtable is its applicability, which is advocated in the experiment section. In the paper it demonstrates that the Bigtable model has been adapted by many applications from Google. Google Earth is a typical example for this model to be applied to, where large amount of structured data (images) are constantly read from the storage. However it would be more convincing in demonstrating the advantages of this model if comparisons were done to compare the performance of the Bigtable and a traditional relational database system. It would be nice to see how faster Bigtable can perform in real time in those applications than the other relational database systems.

Another thing I really like about this paper is that it first lists out the goals of the work and then talk about how each of them are achieved in the work. It helps the reader to keep in mind the motivation and the intuition of why things are done this way. It would be even better if in the explanation of the model’s implementation the paper can focus more on comparing the implementation differences between the proposed model and the baseline models.



Review 29

This paper present Bigtable, a distributed database system built on top of Google File system. Bigtable is a columnar key-value store system. The schema of BigTable is dynamic so it is more flexible and can be queried with MapReduce.
The data model of Bigtable is a distributed persistent multi-dimensional sorted map, called SSTable, which has index on row key, column key and timestamp. The values in a Bigtable is considered as an array of Bytes, each operation on a row key is atomic. Table in Bigtable is a set of column called column families, and the access control is at this level. The timestamp in the SSTable is used to maintain multiple version of Bigtable cells, by which Bigtable archive dynamic control on data layout and format.
The structure of Bigtable can be viewed as a Master-Slave-Client structure, where a single master was selected using Google Chubby, and the master server is responsible for assigning tablets, load balancing and also garbage collection. The Slave layer compose of many tablets server each is responsible for the corresponding write and read requests. The client layer is a API library that user can use to communicate with Master and conduct operations on data objects.

Strengths:
1. This paper as the 3 famous system paper published by Google (the other two are GFS and MapReduce) provide an impressive solution of large scale distributed database system. Together with GFS and MapReduce, it started the era of Big Data.
2. Unlike GFS, it utilize Chubby to act to handle failure on Master, and to communicate with tablets servers, which handles partition failure perfectly.
Weakness:
1. To process data in BigTable, users need to write a MapReduce program instead of using DML like SQL to conduct set operations, which is less efficient and waste of labor.
2. Unlike SparkSQL and DataFram API, it is really hard to manipulate data between Bigtable and other data system, which is hard to use for its users.



Review 30

This paper introduced Google’s Bigtable, a distributed storage system for structured data. Bigtable achieved wide applicability, scalability, high performance and availability. The data model can be viewed as a sparse, distributed, persistent multi-dimensional sorted map. The index consist of three dimensional: (row, column, time). Bigtable supports single-row transactions, which can be used to perform atomic read-modify-write sequences on data stored under a single row key. It can also be used with MapReduce. Bigtable is implemented on top of Google File System(GFS). Underlying Bigtable relies on a distributed lock service Chubby to coordinate replicas and make them consistent. A large table is partitioned into multiple tablets. It uses a three-level tree hierarchy to store tablet location information. A single BigTable tablet server handles all the reads and writes for a tablet. There is a single master server that assigns tablets to tablet servers and dynamically adds tablet servers whenever needed. Like GFS, clients talks to master for metadata and exchanged data directly with tablet servers.

A main advantage of Bigtable is that it proposes a relaxed version of data schema. The simple data model of multi-dimensional map is so flexible such that Bigtable is applicable to various applications. It is also interesting that masters will shut down themselves if the Chubby lock becomes unavailable. This mechanism prevents the inconsistency of partitioned data. The design of single master is simple and effective, making the complexity of the whole system controllable.

One weakness is that the availability of Bigtable highly depends on the Chubby lock service. Though providing simple data manipulation APIs, The Bigtable doesn’t support SQL. Users with experience with SQL still need time to get adapted to the special APIs. It is also huge amount of work to migrate the applications implemented in SQL.