Review for Paper: 15-OLTP Through the Looking Glass, and What We Found There

Review 1

When today's DBMSs were designed, it was assumed that data would be too large to fit in memory, and would be kept on disk. It was also assumed that some transactions might be long-running, so multithreaded execution was needed. Today, most OLTP systems have data that can fit in memory, and transactions in OLTP systems typically execute quickly. Long-running transactions and ad hoc queries occur in OLAP data warehouses, not in most commercial OLTP systems. Therefore, it should be possible to make an OLTP database produce higher throughput by removing high-overhead components that allow the system to work with data on disk in a multithreaded way.

In “OLTP Through the Looking Glass, and What We Found There,” the authors present a study of the DBMS called Shore, in which they systematically removed several components of the system and measured the change in performance on an OLTP benchmark. This analysis allowed the authors to show how much time a modern DBMS spends in various code systems, versus how much time is spent doing useful work. These results indicate how much faster a newly designed DBMS for OLTP could be, if it removed the overhead of unneeded components.

The authors present findings about parts of the DBMS that can be removed if data resides in memory and has no long-running transactions, without leading to incorrect OLTP performance. The authors also show the relative throughput of the DBMS after various pieces have been removed. Key results are that the buffer pool manager and the logging system are the largest bottlenecks in throughput. Logging and latching also use significant CPU time. After removing these components of Shore, the authors produced a 20x speedup relative to baseline Shore. Their optimal solution, a hand-coded B-tree system designed just to run the benchmark, was another three times faster, due in part to smaller call stack depth than the Shore kernel.

One limitation is that the authors' reduced version of Shore is designed only to run a restricted benchmark suite of transactions, and is not a complete DBMS. As such, the results may be less believable than they would be if they were part of a complete working system, as in the Hekaton or PostgreSQL projects. There may be additional sources of overhead that appear when integrating a system like the reduced Shore “kernel” with other DBMS features, which would lead to smaller speedup than reported.


Review 2

The paper discusses feature-rich Online Transaction Processing (OLTP) databases. These databases were optimized to work with computer technology in the late 1970s and haven’t evolved much to keep up the hardware advancements over the years. This and a request from outside the Database community for alternative DB architecture motivated the authors to look for variants of conventional database systems which make use of current hardware specs.

The trends in OLTP include cluster computing (clusters of computers executing instructions on a shared nothing ideology), Memory-resident databases (databases stored in main-memory instead of secondary owing to the fact that the sizes of most OTLP systems are not growing as dramatically as RAM capacity, Single threading (owing to memory-resident DBs and short transactions), High Availability vs. Logging (recovery by copying missing states from other database replicas avoiding logging) and transaction variants (relaxed consistency like snapshot isolation, leading to no logging). A Scalable Heterogeneous Object Repository (SHORE) system was developed which was a hybrid object system sharing features of a file system and object-oriented database technologies. It was observed that after removing all the Shore’s components like Log manager, lock manager, buffer manager and Latch gives a 20 times improvement in throughput.

The paper is successful in explaining the cost implications in modern database systems in various tasks. It was found that most significant overhead contributors are buffer management and locking operation, followed by logging and latching. The authors provide suggestions for future OLTP engine with regards to the performance tests made on SHORE in the end thereby quantifying the theory.

The suggestions made by the authors fall short on account of speculation on some features. The authors fail to define a protocol best suited for concurrency control since many DBMS applications do not support single threaded transactions. The system trades on consistency of data which is a required feature for some of the critical applications.



Review 3

What is the problem addressed?
For Online Transaction Processing Database, a number of other variants are possible; for example: legless database, single thread database, transaction-less database. How well these different configurations would perform if they were actually built is the question they try to solve. They performed a performance study of a database, Shore.

Why important?
Advances in modern processors, memories, and networks mean that today’s computers are vastly different from before, and many OLTP databases design constrain is resolved such that database would fit in main memory, and most OLTP transactions can be processed in milliseconds or less. Yet database architecture has changed little. So it’s time to rethink about what should a modern OLTP databases look like.

1-­‐2 main technical contributions? Describe.
We then modified a OLTP database by removing different features from the engine one at a time, producing new benchmarks each step of the way, until we were left with a tiny kernel of query processing code. This kernel is a single-threaded, lock-free, main memory database system without recovery. During this decomposition, they identified four major components whose removal substantially improved the throughput of the system:

Logging. Assembling log records and tracking down all changes in database structures slows performance. Logging may not be necessary if recoverability is not a requirement or if recoverabil- ity is provided through other means (e.g., other sites on the net- work).

Locking. Traditional two-phase locking poses a sizeable over- head since all accesses to database structures are governed by a separate entity, the Lock Manager.

Latching. In a multi-threaded database, many data structures have to be latched before they can be accessed. Removing this feature and going to a single-threaded approach has a noticeable performance impact.

Buffer management. A main memory database system does not need to access pages through a buffer pool, eliminating a level of indirection on every record access.

1-­‐2 weaknesses or open questions? Describe and discuss
It is lack of validation. The paper only runs some experiment on a particular database. The complexity of the system shouldn’t be linear. That is the cost of two components not necessary equal to the sum of each cost.


Review 4

This paper presents a modification on a modern online transactional processing (OLTP) database system (Shore). To be more precise, they provide a detailed instruction-level breakdown of the major components in Shore (stripping out components and evaluate the performance) on a TPC-C workloads. The optimized system runs 20 times faster than original system on the test benchmark. In general, they progressively modified Shore and evaluate the overheads/performance after every feature removal or optimization, including removing logging, locking/latching, buffer manager, optimizing B-tree key evaluation logic, accelerating directory lookups, increasing page size and removing transactional sessions. Also they provide serval observations and future recommendations.

The general problem here is that traditional OLTP databases include some features designed or optimized for computer technology 30 years ago, which might not meet the trends in today’s technology. One important difference is that we can have OLTP databases fit int main memory now, which might not need some optimizations developed before. Another example is that a log-free database system might provider better performance since it does not need recovery. Therefore, the paper presents several experiments on stripping out some features in Shore and evaluate how performance varies.

The major contribution of the paper is that they provide a detailed performance evaluation for each feature stripping out from the original Shore system. They compare the difference in a way that we can easily see how performance increases, for example with lots of graphs. Another contribution I think is the idea of breaking down components. It is natural but often ignored by people. The technology has changed, so the old optimization or design might not fit now. So taking some components off or doing new optimizations might improve the performance. Also, the paper organized the paper very well, presenting different feature in order with the evaluation, which makes it easy to follow.

One interesting observation: the idea behind this paper is very innovative, and the detailed evaluation also helps to explain the performance difference. I found it very interesting that without technology changes, there is always a tradeoff between a fully implemented system and a not fully implements system (even fully stripped down system). On the one hand, people want system with 100% reliable, consistent, etc. On the other hand, people want to trade the extra design cost or reliability cost for a better performance. I think it also depends on the environment that database system applies to.


Review 5

The paper revisited the design of a transactional database management system. Given the significant improvement in the hardware available now such that many transactional databases can fit in the the main memory, there are many assumptions that we have made when we designed the database systems in the way it is. By actually implementing a database system under certain design decisions, the authors are able to improve the performance of the system significantly (up to 20x). And they also attribute the improvement to each design decision whether we should build certain feature or not. By evaluating against a well-established benchmark suite, the paper demonstrates the big opportunity to redesign the database systems with better knowledge of the underlying hardware specifications, and points out multiple aspects that contribute to the performance gain.

Strengths:

- I think this is a very interesting topic to revisit given the hardware has changed so much in the last 40 year that many assumptions we made when we first designed the database systems are no longer true. This is very similar to the previous paper we read about the operating system support for database system written by Prof. Stonebraker, but with more up-to-date hardware background.
- The paper demonstrates the potential opportunity by actually implementing certain features on a open source database system, which is definitely way more convincing than simply arguing about it.
- It is great that the authors attribute the performance improvement to different design decisions, so that the paper can give us insights that how much each component counts and where we should be focusing on given Amdahl's law.

Weaknesses:

- I don’t think the point that the paper made with single-thread and multi-thread is very promising, given that most of the modern processors already have multiple / many cores built in. And it seems that the number of processors that is available on each processor still keeps increasing now.
- In the evaluation, the paper shows certain improvement can be achieved by doing such optimizations. However, it is easier to demonstrate something will work by implementing it, comparing to demonstrate something would not work because it might be the case that you are doing it in the wrong way.



Review 6

In this paper, the authors take and existing database system, Shore, and gradually remove components that they don't find necessary, creating a faster system. The idea is that existing systems were built for old, slow, expensive computers in the 70's and 80's, for purely business processing data. Now, machines are fast, small, and cheap, and their are more users and more varied needs - some people want things that are database "like". For example, whydo you need a log if you have other machines, do the exact same thing? You can just rebuild from those. Also, if the database is in memory, than multi-threading and concurrency control isn't really needed anymore - small jobs should finish quickly, and then you move onto the next one - there is no wasted time waiting for disk. Also, many large scale distributed systems favor eventual consistency instead of transactions.

The authors then went about removing logging, latching, locking, the buffer manager, and made a few other various improvements. They discuss what each component is, and what was involved in removing it. They then move on to the performance study. In short, their system is much faster without all of the components than the stock system. They then raise questions about the future of concurrency control(locking), multi-core support, and dealing with replication management.

The authors did a great job of showing a system could be stripped down (or better yet, rebuilt! See the next paper!) to dramatically increase performance, and they do a good job of explaining what they removed, why, and how they did it, as well as showing the performance impact.

However, I think they may have removed a bit too much. Moving to a single-threaded model, while simpler, does limit performance - if you have all of those cores available, why not used them? I can see executing one transaction at a time, for simplicity, but then multiple cores should be used for each query. Better yet, where do we go from here? Do the authors want us to strip more components away, or build on the skeleton that they have created?



Review 7

The problem put forward in the paper is that the traditional architecture of the OLTP database is designer more than 20 years ago and is obsolete. With the development of hardware, the processors are fast and the main memory is huge enough to accommodate the entire OLTP database. The needs of the market are also changed. People requires the online database-like service instead of the entire suite of features with the development of internet.

The solution is to place the database into main memory and optimize it by redesigning several features. Proposed by the paper, the features can be optimized is the logging, locking, latching and buffer management. The approach adopted by the paper is modifying the Shore database by removing the components one by one and test its performance to benchmarks. First, load the database into the main memory because the growth of database size is much slower without meeting Moore’s law. Second, use single thread strategy since the short transactions can be executed very fast in the memory, while the long transactions will be executed in the warehouse system instead of OLTP. It means the locking and latching can be removed. Third, remove loggings because high availability is becoming a more general recovery method. Lastly, they optimize the algorithms like those about B-tree. From the test result, we know the optimization improves the performance greatly.

The strength of the paper is that it described the modification performed to the Shore database step by step. The behavior allows the author to define the range of actions done in every components removal. Also, it enables other researchers to replay the modification and use the design in future work.

The drawback of the paper is that the proposed database architecture is based on a cluster of machines over the internet, while in the paper the modification and tests are performed only on a single machine. This behavior overlooks the latency may be caused by the nodes and the network communication. Also, it does not mention how to solve the recovery issue after removing the logging in the Shore database, which may cause the database unreliable. Thus, though the improvement for removing unnecessary components are great, it may not be the same in practice.


Review 8

This paper analyzes how a traditional Online Transaction Processing (OLTP) database executes under a subset of modern TPC-C workload. The paper thoroughly dissect the instruction breakdown of various database resources that are used by the TPC-C benchmarks, what are the major areas of optimizations and bottleneck, and how different techniques are likely to affect the performance.

The OLTP architecture the paper use for analysis is the Shore open source database system. The paper breaks down the database system component into four major parts that are significant portion of the database’s execution: logging, locking, latching, and buffer management. From the baseline Shore architecture, the paper progressively optimized and stripped the major components that affected performance. First they optimized B-tree key evaluations, then removed logging, locking, latching and buffer manager functionality. This striped the Shore database to only contain the necessary functionality for running the target TPC-C benchmark, and sped up the system to be about 20 times faster than the original. Each reduction in component had only a small impact on improving performance, but the combined gains of all the optimizations combined was quite significant.

The paper’s approach in stripping the traditional DBMS and iteratively optimizing the system to reach the final system is interesting, and gives a good perspective of how much impact each component has on the performance. The paper shows that while individual optimization efforts are costly and might not be worth the significant investment required for the effort, the combined system result in significant gains in performance. One thing that is a bit lacking is look at the runtime performance of the system. While looking at instructions executed and cycle count can be a good estimate of performance, it doesn’t necessarily correlate to the actual runtime of the system as there could be memory stalls or halts that affect the execution.



Review 9

This paper measures the performance of the future promising OLTP databases by truncating one or several components in the current relational databases(Shore in this paper) with transaction management, lock manager and buffer management, etc. This experiment is conducted on the benchmark TPC-C. The result of this experiment reflects that each components are significant in Shore but the performance can only be dramatically enhanced unless all components are removed. This result implies a potential evolution in future OLTP engines.

The key advantages of this paper is that it decomposes the current databases into components, and measures the performance on each of them. This approach gives us some evidence to analysis on where becomes the main draw back in the databases. The experiment result shows that buffer manager, latching and locking, logging consumes high portion of CPU cycles respectively and the buffer management takes the highest CPU cycles.

This paper also shows some insight in the method we may implement for new generation databases. When running on a “clean" version of db kernel, the boost of the performance is 20 times fast of the original one. To enhance the performance of the current DB, the paper emphasizes that some new strategy should address problems in concurrency control, replicas, consistency and etc. And it implies that log-less, transaction-less, single threaded database might be a solution.

However, the paper workload in the test is simple that may reflect only some part of the features of the actual workload. With other transactions and more tables involved, the contention of each table may become relatively high and some consistency problems may show up. In addition, one of the key assumption of this paper is that the OLTP database can fit in main memory and the system may be distributed in many sites. But the experiment does not take multi-core or multi-site system into consideration. The potential performance upgrade of databases over distributed sites remains to be tested.


Review 10

Problem:

Computer hardware has advanced greatly since 1970, but many databases still have features that were developed based on the hardware limitations of that era. For example, back then RAM was not big enough to store entire databases, and so OLTP operations were optimized to minimize disk I/O. However, now it is possible for RAM to store an entire database. Transactions and concurrency control were introduced so that transactions could execute interleaved, to improve response time. However, since CPU’s have become drastically more performant, executing transactions consecutively could still produce acceptable response time.

The authors of the paper simulate the performance of this new type of OLTP- single threaded, completely in-memory, and without logging. They start with a standard OLTP system Shore, and progressively strip out logging, locking, latching, and buffer management, and compare the performance of each version. The end result performed at 20x the speed of the original database- buffer management accounted for ~30% of the saved time, locking ~20%, Latching and logging ~10% each.

The authors make the point that while removing one component resulted in only modest speedup, removing many components resulted in very significant speedup. They suggest that removing many components seems to be the way forward for new OLTP’s. They also mention new problems in concurrency control (if needed), multi-core support, replication management, and consistency that these new OLTP’s will have to face.


Strengths:

This paper’s strength is in the authors’ ability to look ahead and see where OLTP’s are headed. They provide plausible quantifiable evidence that the trend towards databases with different guarantees of reliability and consistency can indeed lead to better performance.


Weaknesses/Open questions:

While the 20x speedup that the authors achieved is impressive, this probably should be taken with some salt. The authors have some suggestions for dealing with new questions raised by this new type of OLTP, but their suggestions (optimistic concurrency control, latching/virtualization for multi-core support) surely have some overhead of their own.



Review 11

The paper questions the necessity of a standard suite of features that is included in modern OLTP database systems, most of which came more than 30 years ago. These features include heap files, B-trees, support for concurrent queries via locking-based concurrency control, log-based recovery and an efficient buffer manager. The paper argues that many non-conventional, recent modern applications may not need these features with the advancement in computer technology. The authors demonstrate that the significant performance gain is achievable by removing those standard features for conventional OLTP systems and suggests that such approaches that can remove many of features of the traditional OLTP system may become attractive.

Two main reasons that encourage the development of non-traditional, database-like applications are that first, modern processors are very fast and it is not uncommon that the entire database fits in memory and second, the variety of data-intensive applications is used with the rise of Internet results in a huge interest in database-like applications without the full-suite of standard database features. In such applications, transactional support may not be needed and eventual consistency may be sufficient.

The paper takes an open-source database system, called Shore, and strips its major components that comprises the standard database features one-by-one to see the effect of each component to the performance of database system. The authors basically remove logging, locking, latching and buffer management from the original Shore system gradually and demonstrate the performance of database with respect to the removal of each component. The readers would not be surprising to see that the performance (i.e. throughput) improves as each of these components gets removed. The notable finding in their experiment is that the performance improvement is very significant (i.e. about 20x) when every single standard feature has been removed from the DBMS. This also suggests that the performance improvement becomes insignificant as soon as you introduce any of these standard features.

The performance measure in this paper is surely not a ground-truth that would be applicable to every other case as the experiment only tests a single workload in a single DBMS implementation, but it does provide some degree of insight into the feasibility of the non-traditional, database-like applications. The paper argues that the performance gain is not that attractive to change the traditional DBMS unless you lift every standard feature of the traditional DBMS, but once you do so, the performance gain will be very significant.

As I mentioned earlier, the most notable finding is that the degree of the performance improvement achievable by removing the standard features is very significant. While the authors say that you have to remove every standard feature to see much better performance, I do not think that this limits the development of other domain-specific database-like applications as long as it has its own advantages that are not mentioned in the paper. We also need to note that the discussion in the paper is limited to a single-node in-memory database and may not be applicable to multi-node distributed database systems or parallel database systems which are more prevalent today.



Review 12

In the past, databases were designed under the assumption that the database would not fit all in RAM, so it was expected that parts of the database needed to be written out and retrieved from disk. According to this paper, main memory technology is improving at a faster rate than the amount of information required for an OLTP database. If this is the case, then the old design decisions are outdated since databases can entirely fit in main memory, thus cutting the cost of disk I/O.

This paper explores different database architectures with the assumption that the database can fit entirely in main memory and compares those architectures with traditional design based on performance. The paper discusses logless databases, single threaded databases, and transactionless databases and the different applications of each. In order to simulate each of the architectures, an open source database, Shore, was used and modified. The modification removed and altered different components within the database such a getting rid of locking protocols and removing the buffer manager.

The paper, then, measured the performance of the database as each change was made using a modified TPC-C benchmark on a desktop computer. The performance results showed impressive improvements, eventually resulting in a 20 times improvement over a traditional architecture.

This paper provides a well explained, easy to understand motivation as to why to change traditional database architecture given that main memory is so large now. Whereas before most of the overhead of a database came from disk I/O, the new source of overhead is components inside the database. Removing them, as was done in this paper, is one possible solution and can result in large performance gains. Although some database applications may need certain guarantees, such as recoverability, those can be supported outside of the database in order to improve database performance.

I thought the setting of the parameters and the actual system used did not seem reflective of an actual OLTP setting. According to the paper, only 5 warehouses with an item list of 10 were used. These tests were done on a desktop computer with 1 GB of RAM. Although this does simulate a setting in which the database fits entirely in RAM, it isn't clear to me than their results would scale linearly as the system parameters changed. Supposed instead of 500MB, it was 500GB. Would the results still hold true? Would the performance gains still be there? The paper seems to assume that the results scale linearly. To make the paper more complete, it would have been illustrative to also provides a few more results where the parameters were increased. It then may be possible to assume linear scaling of the results.


Review 13

Classic OLTP databases used to support a lot of features, which result in significant overheads; however, recent hardware improvements (for example, the whole database might now fit into RAM) and some other usage changes have made the desired feature set very different from before.

However, how much does it cost for supporting a specific feature? How do we decide which feature should be removed for performance issue and which feature is worth supporting? This paper investigates into such questions and provides us some insight.

The main contributions of this work include:

1. They breaks down the number of CPU instructions in OLTP database and show how many instructions are spent on performing each of the features other than actually processing the query.

2. This paper uses Shore database system as an example and introduces the supported features in Shore such as thread, locking, logging, and buffer manager.

3. They remove the previous components one by one from Shore database system and present the performance in terms of the number of instruction reduction, and conclude that the improvement is not good enough unless multiple optimizations are applied.

It is an interesting idea to strip down the features from DBMS and closely look into the actual cost for each component. This provides a good reference for future DBMS architecture designs when considering whether to include some features.


Review 14

The paper discusses Online Transaction Processing (OLTP), a database that supports disk-resident B-trees and heap files, locking-based concurrency control, and support for multi-threading. As the paper argues, OLTP needs to be updated to take advantage of advances in hardware such as fast modern processors, large main memory and networks, and internet demands. The purpose of this paper is to suggest how hardware trends can be exploited to minimize time spent in logging, latching, locking, b-tree, and buffer management operations.

One hardware improvements and its OLTP implications is that thanks to fast modern processors, OLTP transactions can now be completed in microseconds. Also, thanks to large main memory and networks, the OLTP can now be stored in RAM. In addition, the internet has many applications that demand database-like applications that use a subset of database features. Thus, the paper suggests alternative DBMS architectures to optimize OLTP with these current trends, and uses the Shore (Scalable Heterogeneous Object Repository) as an example DBMS system to exemplify the effects of these changes. One way is the logless database. Logging slows performance due to assembling log records and tracking down changes in the DB structure. A logless database is useful is there is no need for recovery or if the recovery can be performed in other sites in the cluster. Also, high-availability facilitates recovery, so instead of REDO, we can copy the state from DB replicas. Another aspect of optimization is single threading. Multi-threading used to be desirable because of latency from slow disk writes, but now that the OLTP DB can fit in memory, there are no disk writes. Also, multi-threading was used before because long-running transactions blocked short ones, but now there are simple OLTP workloads. The current multiple core architecture can be utilized as multiple processing nodes on a shared cluster. We can thus get rid of locking and latching that was used in multi-threading. The next optimization comes from modifying transaction processes. The paper proposes a transaction-less database because many DBMS do not need transactions. Also, with the desire for availability over consistency, internet applications require eventual consistency over transactional consistency. There are proposed models for relaxed models for strict consistency, in which there are non-transactional snapshot isolation. Thus, we can remove buffer management calls. Additional optimizations included streamlining B-tree key evaluation logic, accelerating directory lookups, increasing the page size, and consolidating transactions into a single session.

In conclusion, all of these components (adjusting logging, latching, locking, b-tree, buffer management, and operations) were needed for there to be significant improvement in the overall system. The improvements were 20x better in performance than the original Shore code. Future considerations included dynamic locking to support concurrency when needed, reconsidering multi-threading since the overhead of latching is not significantly high, implementing active-active replication management because the current active-passive scheme has remote copies that are not consistent with the primary, identifying the existence of workloads with eventual consistency, and using indexing structures instead of B-trees.

Overall, this was a very thorough and well-written paper to read. I appreciated that it talks clearly runs through the specific effects of hardware advances on architecture, and how we can change OLTP to take advantage of the improvements. I think a limitation of this paper is that it does not discuss how these improvements have been used in database systems today, and I would've liked to see the paper quantify that effect on internet applications.



Review 15

This paper is on OLTP transactions and how data base technologies today can take advantage of current hardware, which has changed a lot in the past 20 years, and can be used to speed up queries in RDBMS systems. This paper looks at removing B-tree keys, logging, locking, latching, and buffer management in order to isolate the causes of overhead in the Shore system which was used in the 1990s. This paper has the main insight that small improvements can be made by removing any of these parts individually but the biggest performance contribution comes from making all of these improvements simultaneously.

This paper takes a real database system, Shore, and removes its parts to isolate the causes of overhead. The paper cites a large amount of previous work that looks at performance bottlenecks in these systems but states that no previous work attempts to isolate these factors. This is a strong point of the paper. Its insight is that causes of overhead and the collective improvement that can be made by removing things that might not be needed anymore is a significant portion of the processing time for the many queries in TPC-C that they used to test their system. This is a powerful insight and as we saw in the Stonebraker paper for this week it is part of the motivation for completely reinventing RDBMS systems.

Despite the upsides, this paper has some drawbacks. This paper was published in 2008 and it is using a database system that they know wasn’t even supported after the 1990s. This means it might not be representative of current RDBMS systems. Also, they note on several occasions that they are restricted in what they can modify within Shore. They state that they have to try to keep Shore from breaking and thus cannot remove or change certain parts. Most notably, they state that they can increase the page size from 8K to 32K but that they cannot make the page larger. The authors state that this improvement led to a significant decrease in instruction count of 14%. It would be more representative of current possibilities for OLTP systems if the authors had written their own system that could operate with or without each of the components that they wanted to isolate and experiment on individually.



Review 16

Part 1: Overview

This paper explores some of the modern hardware features and try to exploit them to make Online Transaction Processing (OLTP) databases faster. Some online transaction processing databases can now fit in memory and queries can be processed much faster than before. Other factors such as CPU development as well as new disk technology introduce some opportunity for online transaction databases to get faster. Real online transaction database system (Shore) is used for testing. The key idea is to run the OLTP in main memory when the whole database could fit in RAM. From there they proposed three possible alternatives for database architecture, logless database, single threaded database and transaction-less database, respectively.

They used TPC-C benchmark ranged from 640 transaction per second to 12700 transactions per second and they gathered overhead information of logging, locking, latching, as well as buffer management. For the experiment, they cut out some components of Scalable Heterogeneous Object Repository (SHORE) database such as logging, locking, latching, and buffer managing. Optimization like B-tree accelerating with hand coded searches, dictionary speeding up by cache look ups, increasing page size to avoid reallocation of memory. Transaction termination overhead could also be eliminated as they are using the same session.

Part 2: Contributions

The major contributions includes finding out where the time overhead comes from and use the results to shape alternative database architecture such as log free databases or transaction less databases. In addition to cost estimation, they actually disassembled a database system, SHORE, to test a concise version in memory online transaction processing database. The performance could lead to new architecture in memory database design.

Part 3: Possible Drawbacks

RAM is still limited as the database is growing bigger and bigger. People tend to store more data even in smart devices.

Single thread lead to no concurrency control however at the same time may need higher frequency super CPU. In the modern CPU world, power and heat transfer limitation are becoming major concerns. We still need multi-threaded concurrent transactions as we will have more cores with lower frequency. Also cloud computing is becoming a new wave in the industry, we still need concurrent databases.



Review 17

The paper discusses about performance overhead and optimizations of conventional online transaction processing (OLTP) databases. Most features added to OLTP was developed a long time ago without consideration of the current facts including: larger main memory size, high speed CPU, and emergence of database-like applications. These features includes database logging, latching, locking and buffer pool management. However, such features create a considerable performance overhead. In order to improve performance of OLTP database, the paper proposes removing these features as they are either not important because of the emergence of memory resident database or they can be handled by a different mechanism.

The paper suggests to remove traditional logging feature as it is possible to achieve recoverability through a guaranteed availability in multiple redundant hardwares. In addition, the paper claims that multithreading can be removed since with a memory resident database, there will never be a disk wait which was the primary motivation for this feature. Furthermore, buffer pool management will no more be needed as a main memory resident database doesn’t need to access database pages through a buffer pool. The paper point out that unless all of these features are stripped out, the performance gain is unlikely to be much better than a conventional OLTP database where most the data fit into main memory.

The main strength of the paper is breaking down the performance overhead of various features and pointing out that addressing a single bottleneck will not lead to a better performance and rather all overheads have to be addressed wholistically. Furthermore, their approach of step-by-step stripping down and measuring performance improvement was beneficial for understanding overhead contribution of each features.

Although insightful in motivating the need to redesign OLTP database system, the authors fails to consider approaches other than removing features. Even with memory resident database, while it is true that there will never be a disk wait, there will be main memory wait. The speed gap between the processor and main memory is still large and only expected to increases. As a result, there is a need of hiding memory read latency through mechanism like multithreading. Instead of simply switching into single threaded approach, the authors could have investigated multithreaded database optimized for in-memory database. For example, a finer granularity locking, like at a cache-block level, instead of database structure level locking, might avoid many of the bottleneck associated with locking and latching in the original implementation. In general, they fail to investigate or compare against with a direct optimization of the bottlenecks for in-memory database. This approach could have given a good performance while still maintaining important features of OLTP database system.


Review 18

Paper focus and motivation
The purpose of this paper is to 1. look at some interesting variants of conventional database systems that one might build that exploit recent hardware treads, and 2. Identify overheads and optimizations that explain a total difference of about a factor of 20x. Since hardware is faster and better today, maybe we can design some set of features that were not offered by standard OLTP engines, which was invented when all good things we have now are not available that time.

How would different configurations perform in today's databases?
1. Do we still need logging mechanism? maybe we can have multiple machine in the cluster. Machines are cheaper now. But performance is still the key.
2. Single threaded database?
Today, there is no need to hide latency because most transactions are running so fast.
3. Transaction-less database

Experiment overview
This is an analysis paper, and they tried their proposed database configurations.
Platform
1. User Shore, an open source database system.
2. TPC-C benchmark
Result
desktop capacity: 640 TPS (transaction per second) -> improved to 12700 TPS
Modified kernel
1. single threaded,
2. lock-free
3. main memory database system without recovery
Given the experiment result, they concluded that removal of the following is good.
1. logging
no recovery or other method available, it is not needed
2. locking
not needed for single thread
3. latch
same as above
4. buffer management
get rid of this can eliminate a level of indirection on every record access
When we step back to single-threaded configuration, we get a lot of good things.

Weakness and limitations
The paper has one big assumption that may not be true latter on. Is OL transactions stay small and easy to handle? Will we come back to logging and locking again? What is the real trend after all?


Review 19

This paper analyzed OLTP database components, including a set of features that defines a transaction processing system such as logging and locking. Authors of this paper points out that many of these features gives bad performance and might not be necessary in todays situation. They then give a performance analysis on how much computation needed to support each of these components and discussed performance of each variations of possible database supporting different scenario.

This paper used Shore for their analysis. So it first begins with a introduction on Shore’s Architecture and features. Then Shore’s components are removed one by one. Performance of the remaining subset of Shore is measured. They found that four components: 1) buffer manager, 2) latching, 3) locking and 4) logging used major part of machine cycles. They used TPC-C schema and a combination of different OLTP workload to do performance experiments.

As they pointed out, this paper has three contributions.
1) Gave a dissect on a modern database system,
2) Measured the performance of various stripped down variants of a modern DB.
3) Provided a speculation on the performance of different DBMS.

I think this paper is very creative. It identifies the need for different OLTP database systems. And it used a smart method to provide evidence of existence of a better OLTP database system: to strip an existing DB and measure performance of that stripped system.

For the weakness, I think it will be better if this paper could first give a quick overview of what OLTP is and provide some key difference between OLTP and traditional database. This helps reader to understand the following discussion.



Review 20

The purpose of this paper is to identify the various modules of a database in terms of an OLTP transaction and how each module contributes to the reduction/overhead involved in processing.

The authors have taken the Shore database engine in order to provide with the results of relative CPU consumption. The authors changed the code of the given database engine in order to reduce it to only processing the queries provided to it in an incremental manner.
The main components of the engine that on being removed provided an substantial improvement are
1. Logging
2. Locking
3. Latching
4. Buffer management

Besides disabling these components, the authors also performed a few more steps to improve the performance further. This included increasing the page size to reduce page lookups, removing transactional sessions and accelerating directory lookups.

Of these four components, the authors describe the buffer management having consumed a significant amount in terms of CPU instructions. In case of a memory-resident database, this overhead can be taken care of since the database system would be optimized accordingly.

What was impressive was that by just removing these components, a 20x improvement was observed in terms of resource utilization.

In the example they have given with respect to payment and new order, it was not very clear how they are handling concurrency issues without using locks or latches in case of a remote transaction.

The authors also point out that just removing the remaining components before they removed the buffer manager only used a factor of two or three fewer instructions. Since the results were produced by replacing the calls to the various modules with if blocks, I am not sure if it would be identical to an actual database engine that does not have these capabilities in the first place. A real-world example or database query run on this system would have given us a better idea as to how it would work in such a scenario.

Being able to take advantage of multiple cores by running multiple single-threaded processes would require some form of latching which would bring up the same issues the authors have tried to avoid by performing this experiment. But despite that, the results clearly give us an idea about the kind of performance improvement we can see on removing these modules and therefore, the overhead involved in each of these steps. It helps us understand how we need to shape the database according to typical needs to avoid unnecessary overhead.



Review 21

Optimization of Sequence Queries in Database Systems

This paper mainly reviewed the experience of exploring the feature of the online transaction process and studying the where the time is spent in modern DBMS in terms of OLTP. Based on the ideas learned from the study, researchers measured the performance gain of some recently proposed alternative database architecture.

OLTP, which refers to online transaction processing, is the processing of high frequency and low concurrency short queries. To be processed on traditional disk based DBMS that focuses on achieving high concurrency by minimizing I/O overhead using multi-threading terminology, most of processing time would be accounted towards the DBMS structures like logging, locking and buffer managing.

With the development in larger ram capacity, high processing rate CPU and cheaper clustering computing center, new architectures for OLTP based DBMS are proposed. The basic features of the new architectures represented by Shore extension are:

1.Memory resident database: fitting all database data into main memory allows faster record/index accessing and changeable page size.

2.single threading: since all the data are fitted into memory, there is no need to have one task to be preempted for I/O waiting and run the next. The transaction at a time processing method reduces overhead for running lock manager and Threading Library code

3.High availability without logging: solving recovery issue via copying from replica would reduce the overhead of logging unnecessary information.

According to the test done on the Shore extension with the TPC-C benchmark, the most of cpu time are spent on buffer managing(~30%), the logging, latching, locking and Btree-keys accounts for 10%-15% of the system time. By eliminating any one of the above features from traditional DBMS wouldn’t create significant improvement for the whole system, however, the new architecture that avoid all these structures can create 20X faster throughput or in the performance result, making cpu executing 15x faster.

But there are still some interesting point worth considering:

1.concurrency failure cost: this paper states that in the Shore system, the eventual concurrency overweighs transaction concurrency. However, there is no statistics provided to show to what extent the failure in transactional concurrency would affect the final result, or how much would it cost.

2.system cost(ram): Also, the memory resident database would mainly use memory to support fast execution of single transaction, but how much would it actually cost compared to the traditional disk arrays remains to be seen.

3.implementation in multi-core infra, and virtualization: since most current design of cpus are multi-core, how to exploit that to create more single nodes in this system is not clearly stated, and more issues in terms of virtualization should be addressed.



Review 22

As the substantial time is spent in logging, latching, locking, B-tree and buffer management in OLTP and possibility to have a very large memory to fit the database in RAM, there rises a kind of alternative database to OLTP which has different features.

The main overhead in OLTP are logging, locking, latching and buffer management. So the optimize for main memory OLPT has such features :
(1) Logless as no need for recovery or recovery from other sources.
(2) Single threaded database: As the memory is much faster than the disk, it is not so important to have transactions to utilize the disk wait time.
(3) Transaction consistence is not necessary and only need eventual consistence.

Then the author use the experiment result to show where the time goes inside the modern database and measure how the performance changes when disable some features of DBMS. The author did the experiment on Shore and talks about the component inside the Shore and how they remove the components in some order.

The reports in the paper is based on CPU instruction counts. The result shows that the removing of such components largely reduce the cost of transactions and shows how much the reduce according to the different components. It concludes that only tripping out one component cannot largely benefits the overall performance and optimizing all is preferable.

In the end, the author talks about some implications for the result. First, it will benefit a lot if turn off the concurrency control when some situations like the transaction-at-a-time is sufficient. Second, as the single thread OLTP has better performance, it should first deal with the multi-core. Third is when using replicas to do recovery, the author introduce a active-active strategy that all replicas do the transaction synchronously.

Strength:
The paper first introduce the motivation and the availability to have a “simple” DBMS and then use experiment to show that what the future work should be done to optimize. The author clearly introduce the experiment environment and the note points in the experiment like the ordering to remove different parts.

Weakness:
In the experiment, the author modify a existed DBMS named Shore. But the remove is not perfectly done as the author use if-else and comment-out to modify the DBMS. So the DBMS the experiment use didn’t exactly show the benefit of these optimize. And it should be shown that how the DBMS is when build such DBMS on purpose.


Review 23

The authors of this paper argue that modern Online Transaction Processing databases face different challenges that conventional database systems try to solve. Main memory has grown substantially while many datasets have not, allowing many workloads to fit entirely in memory. This suggests that perhaps new architectures should be used which avoid the limitations of relational, disk-based databases. The authors claim that consistency requirements should be relaxed. They then look at the design of a database and examine its performance characteristics, identifying ways that performance can be improved, and showing that there is no single performance bottleneck in the system.

The diagrams in this paper made it easier to understand the content. I also liked how the authors looked at individual components (locking, logging, etc) and looked and their contribution to runtime. Given the emphasis on modern systems, I was hoping to see discussion on databases that operate fully in memory that achieve persistence through replication (or erasure coding), but I did not see anything on this.


Review 24

This paper argues that a fully stripped down database management system can perform up to twenty times faster than the unstripped version. With modern hardware and processing speed, many databases are able to be stored completely in RAM and many transactions can be completed in the span of microseconds. Therefore, there are many features of database management systems that were originally used to combat the issues of poor hardware that we can remove to speed up modern DBMS.

The main overheads of a DBMS are logging, locking, latching and managing the buffer. We can then remove all of these overheads the following ways. Since systems are already using more hardware to take care or availability issues and failures, we can use the extra copies to perform the same operations that logging provides. To remove locking, we can run the DBMS in one single thread. This will ensure serializability and recoverability. To recover from an abort, we can just make a copy of the database before running the transaction and recovering to that copy if an abort occurs. Finally, if the entire database is in memory, there is no need for a buffer manager because nothing will be written back to disk.

The paper then explains how these optimizations were implemented in SHORE and the results compared to SHORE without the optimizations. After all of the optimizations, the two test workloads averaged out to about 80 microseconds per transaction compared to 588 microseconds per transaction. The two test workloads that they experimented with were the payment transactions and the new order transactions.

Even though this paper points out some reasons why we can remove all of this overhead from DBMS, there are several oversights that I want to point out:

1. The authors tested the optimized version of SHORE on single workloads with a single type of transaction. In a real-world workload, there will be different types of transactions and volume peaks and valleys. Would either the different types of workloads or the peak times affect the performance of the single threaded DBMS?
2. When there is a complete power failure, in-memory databases will be completely destroyed. To prevent this, another copy will have to be kept in a remote location or some power backup system will have to be in place. How does the overhead of maintaining these systems and keeping everything in sync affect performance?



Review 25

This paper discusses the need to reevaluate the principles or relational database design given advances in technology over the last several decades. Many of the assumptions that went into the design of traditional databases are not true anymore. Specifically, main memory is often large enough to hold an entire OLTP database, methods of recovery other than logging have been developed, and many companies do not require transactional consistency, claiming that “eventual consistency” is good enough for their purposes. The authors of this paper analyze the overhead of several major pieces of OLTP databases by beginning with an out-of-the-box version of Shore, then removing components systematically and measuring the subsequent performance increase.

The authors make five major changes to Shore:

1) Hand-coded optimizations to increase page size, accelerate directory lookups, and optimize B-Trees for common cases.
2) Remove logging
3) Switch to a single-threaded, one transaction at a time approach to execution that gets rid of locking and latching
4) Remove transactions
5) Remove the buffer manager, as no disk I/O is needed for an in-memory database

The authors used a workload of 40,000 transactions made up of New Order and Payment transactions from the TPC-C benchmark. They found that each piece described above contributed between 10% and 35% of the total instructions associated with a query. By removing all of these components, they observed a 20-fold increase in transaction throughput. They note that the most drastic improvement was seen when they removed the last piece. With the buffer manager intact, they observed a roughly 3-fold improvement in performance, but, once the final piece was pulled, enormous increases were seen. The authors’ experiments demonstrate that considerable progress can be made by rethinking what was once taken for granted. They stress, however, that the development effort must be dedicated, as the largest performance increases will only be seen by removing all of the core pieces that they identify.

My chief complaint with this paper is that they ignore several important cases. They acknowledge scenarios such as system crashes, but do not offer solutions to how to handle these cases after removing the core of the database code. Additionally, I was curious to see how transaction response times are affected by the authors’ modifications. If transactions are run in a single-threaded fashion, short transactions could become stuck behind long transactions in some situations. While the average throughput may have increased 20-fold, this may still introduce an unacceptable margin of error for some small time-sensitive transactions that rely on the interleaved execution of multi-threaded database structures for reliable response times.



Review 26

This paper speculated the performance of modified database systems that are stripper down variant features, such as logging, locking, latching, and buffer manager. Online Transaction Processing (OLTP) databases include many features. However, today’s computers are much different than those when OLTP databases are proposed. For example, many OLTP databases can now fit in main memory, and thus a buffer manager is not essential in DBMS design. This paper obtained experimental results of performance measurement when the database systems are stripped off one or more various features.

First, the paper identified four major components whose removal substantially improved the throughput of the database systems, including logging, locking, latching, and buffer management. Then, the authors used Shore (Scalable Heterogeneous Object Repository) to perform some experiments of stripping some of these features, removing logging, locking, latching, and buffer manager calls. The objective was to reduce theses features that contribute substantial overhead to database systems.

Second, the paper discussed their experimental results. These results implied that the benefit of stripping out any one of the components has a small benefit on overall performance. The most significant benefit is gained when multiple optimizations are applied. Therefore, when designing modified DBMS, multiple features that bring overhead should be stripped off.

The strength of this paper is the organization. It first identified the problem it was dealing with, and provided some statistics in the introduction section, which is not usually seen in other paper, to verify the importance of the problem. Then, it started proposing its methods. The statistics related to the problem introduction can help readers to get impressed with the main problem in the paper.

The weakness of this paper is that it has little explanation about the issues about the concurrency control, replication management when the DBMS is stripped of the important features for the performance reason. It mainly talked about the improved performance. However, it rarely explained how to tackle the issues the stripped issues were tackling with in database systems.

To sum up, this paper studied the performance of database systems stripped off some important features that contributes overhead, given the different situation of modern computers from old computers.



Review 27

This paper is an analysis of what takes up time in current online transaction processing (OLTP). It talks about the initial needs of transaction management and how things are changing, and uses this to reevaluate whether some improvements can be made. It analyzes a system named Shore and toggles off and on certain features to see how many instruction cycles can be saved with these things.

The main things it looked into optimizing away were logging, locking, latching, and buffer management. They were able to reduce the instruction count by 93.2% when they toggled off all of those systems. This is an obvious performance improvement but you can imagine there are downsides. The downside of eliminating logging is eliminating recovery in the event of a crash (or setting up a separate way to recover such as having copies of the data). Eliminating locking and latching makes the system single threaded, which in the past would have been a performance struggle as potential disk wait times would have been an issue, but now with systems that can have gigabytes of memory (how the times have changed since this paper) on them whole databases can be stored in memory so there is no need to wait for disk time so the multi threaded becomes less necessary. The biggest improvement however is done by removing buffer management, which can reduce the instruction count by 34.6%.

It urges DBA’s to think about what they need and shows how removing certain features can improve performance.

One weakness of the paper in my mind was the organization of the paper. It first did an introduction, and then results before I really knew what the paper was about (whether it would be a specific system or just OLTP as a whole). Because of this after the discussion of Shore the results at the end just felt like a repeat of the beginning results. I felt as if they should have been removed and only the ending results should have been kept in the paper. Another problem I had was any time you make the argument to remove multithreading to improve performance I have a hard time believing that. They made that argument in this paper (or saying it wouldn’t hurt performance much) but I have a hard time being convinced that making something single threaded is ever the solution. And I don’t think they did a good job of convincing me. Lastly, I don’t think they spoke enough about the downsides of removing all of these things. I feel as if they brushed off the fact that removing logging negatively affects recoverability, or removing locking hurts consistency or any other downsides they seemed to gloss over.

Overall though I think the results were impressive (though maybe infeasible) and it was a good paper. I would be curious to see what came of it and if any of these things were implemented in practice after this paper.



Review 28

This paper, with Stonebraker featured as an author, focuses on analyzing a different aspect of DBMS management. The motivation of this paper was to uncover different forms of overhead that arise from processing schemes in relational DBMS’s. By optimizing on such overheads (logging, latching, index management, locking, buffer management, etc) and running most processes in-memory, the performance improvements would be substantial.

Attempting to progressively strip down components of the Shore ecosystem seemed to provide significant performance improvements, behind the justification that the overhead incurred with each mechanism mentioned is not worth the benefits, as simply having a bare-bones system creates a speedup orders of magnitude greater than the original. Of all of the findings in the paper, they seemed to perceive the largest performance speedup when optimizing over the buffer manager and the lock management system. What makes this paper special is that other recent works tend to focus holistically on DBMS systems, but rarely do a component-wise analysis and attempt to focus on each section.

Overall, what was most confusing was the structure of the paper; it was difficult to even understand what the paper’s goal was exactly until after the introduction (whether it was simply an analysis or an attempt at improvement in DBMS performance). In addition, the graphs used for instruction breakdown are hard to read, and it was difficult to decipher what exactly each axis represented. On the other hand, however, the content of the paper was interesting regarding how stripping the system of many components can yield such a benefit. However, is there any sacrifice to desired output, or are these strippings worthwhile in any scenario? If so, in which cases would it be better to keep in these industrial components?


Review 29

This paper gives an overview of current OLTP architectures and examines various in which optimizations can be made for new age hardware systems. The paper dissects where current databases spend much of their time, and proposes various differing stripped down versions of these models to encourage performant systems.

One novel insight offered in the paper is that of single threaded execution in OLTP systems. All modern databases provide extending support for multithreading. The motivation for multithreading in OLTP systems is predicated on the idea that there will be long running transactions that are blocking short transactions from completing. However, the authors of this paper argue that this notion no longer holds true in current systems. The argument is that most databases can reside in memory, thus long disk i/o is no longer an issue. Second, OLTP workloads usually consist of simple index lookups/updates that occur in milliseconds. The solution proposed is to instead treat multi core systems as multi node none sharing clusters.

Another idea presented is utilizing redundant hardware in high availability systems for recovery. The authors claim that most high availability systems use two ore more times the amount of hardware to guarantee availability in the event of a failure. They propose that this redundant hardware can be used to aid recovery by copying missing state from the database replicas, thus rendering the recovery code in databases useless.

While this paper seems to present very applicable and simple optimizations, many of the predicates used to reach their conclusions seem heavily flawed. In particular, many of the arguments in the paper are based on the idea that most OLTP systems fit in RAM, however this claim isn’t quantified in general seems flawed. And even if this claim were completely true, the techniques presented would not be the most optimal solution. There are database systems that are designed only for in memory workloads (such as Redis) that employed for workloads of small scales.


Review 30

This paper basically says that many of the existing OLTP technologies are still based on that that was developed during the 1970s despite the fact hardware capacities have improved significantly. So, the writers decided to look at some interesting variant of conventional databases that might be built by exploiting recent hardware trends and speculate on their performance. For the experiment, it uses Shore database system and runs it on a subset of TPC-C benchmark. The paper proposes alternative database system – such as logless databases, single threaded databases, and transaction-less databases – and sees how they perform.

The paper goes with identifying the four major (potential) components to remove in order to improve performance: logging, locking, latching, and buffer management. Then it discusses the trends in OLTP, such as cluster computing, memory resident databases, single threading, high-availability vs. logging, and transaction variants. Then it explains about the architecture of Shore database system – especially regarding the four components mentioned previously – before finally removes the components, plus does several hand-coded optimizations. Then paper does performance study after components removal. The experiment focuses on two type of transactions: New Order and Payment. The result yields a factor of 20 improvements in overall throughput. For both transactions, the buffer management removal causes the biggest improvement. The comparison of Instructions vs. Cycles for New Order transaction shows that the removal of each component plays different part in performance improvement. The last part discusses the implication for future OLTP engines, especially regarding the concurrency control, multi-core support, replication management, and weak consistency.

There are three major contributions of this paper. First, it dissects where the time goes inside of a modern database system. Second, it carefully measures the performances of various stripped down variants of a modern database system. In other words, it shows the first by explaining the second (the four major component). Third, it uses these measurements to speculate on the performance of different data management systems that could be built. The experiment helps explaining the first and second contribution.

However, there is the fact that we must remove all four major components in order to gain the performance benefit. What happens when we are only able to remove certain components? I think removing the components one by one and compare the result would help determining which component is the most significant for certain types of database. And what about the performance after removal for distributed system? In distributed system where data is partitioned, how would it go?



Review 31


The purpose of this paper is challenge why database architectures haven’t changed much in the last 30 years, and following the other Stonebraker article that we read for this week, it looks to assign performance overheads to specific parts of a legacy DBMS so that these specific bottlenecks and categorical problems can be addressed. To do this and prove that the elimination of specific DBMS components does provide performance increases, they implement their suggested improvements on top of the SHORE system.

The technical contributions of this paper involve their presentation of results that detail the breakdown of overhead in traditional DBMSs for use in online transaction processing systems (OLTPs), and how they can exploit the specific properties of these kind of systems to make improvements to existing DBMSs to handle more diverse and more “current” workloads. The discuss trends in OLTP that lead them to their decisions about optimizations as well as providing empirical motivations for many of their decisions. They perform these optimizations on Shore’s Shore Storage Manager, an existing database system. A lot of these modification involve inserting workarounds and if-statements to bypass code blocks into the existing infrastructure of Shore. In the end, they present a logless, single-threaded system that is main-memory resident and performs one transaction at a time.

I think the strengths of this paper are in the end product it produces as well as the empirical evaluation. It was a very important study to quantify the overheads incurred based on the various components of typical DBMSs that they belonged to as the databases community looked forward to the future and updating systems to handle more modern workloads.

I think one of the weaknesses in this paper is in its general ambiguity. Especially in reference to their “optimal” system that they compare the other results too. They even refer to their minimal-overhead kernel as “optimal” in scare quotes, and that makes me even more confused. I think the authors should have produced a proof of optimality or at least some sort of informal argument, rather than expecting the reader to accept optimality without justification. I was, on the whole, disappointed with the detail provided about this minimal-kernel system and justification for why they wouldn’t try to implement something more like this as opposed to Shore.


Review 32

Summary:
This paper first points out the phenomenon that current Online Transaction Processing DBMS(OLTP) are heavily designed to accommodate the old hardware as well as for old work loads back to 1970s. Then it gives a 20X faster approach by shrinking the components of OLTP while maintaining the functionality to accommodate the current workload for a specific OLTP system Shore. It archive the approach by removing components from Shore, and provides the result based on doing experiments with TPC-C benchmarks. With the experiments, it also come up with the conclusion that each of these components plays a important role slowing down the OLTP, only removing one of them cannot archive promising speedup.
Moreover, It identifies the following components are needless in modern OLTP system: 1) Log, since crash recovery are either not required or can be archived by replicas; 2) multi-thread, since for in memory system, no slow disk write need to overcome; 3) transaction, since for nowadays OLTP jobs, eventual consistency is enough. By identifying these properties, it removing the following components from Shore in order: logging, locking, latching, buffer managements. It also add some general optimizations to the system, including increasing page size, streamline B-tree key evaluation logic, accelerate directory lookups and remove transaction sessions. These modification involves simply if-return hack or more complicated function rewrites. After this, the paper does a series of experiments measurement on each time it removing the components, and finally come up with the conclusion in paragraph 1.
Besides, this paper also reviews the past physical design of OLTP system, and summarizes the future trends based on the review.

Strengths:
1. This paper clearly indicates its contribution in section 1.4: it identifies the needless part for modern OLTP system and then dissect the time consumption of each components by removing them, and give suggestions based on the measurements of their experiments.
2. This paper utilizes the open source project shore and conduct experiments based on removing each needless components of it, which is both objective and fast to archive.
3. It gives great suggestion and reasons of redesigning the OLTP system. Moreover, it also identifies that instead of disks, network would be the bottleneck for future OLTP system.
4. It clearly clarifies the background of the experiments. It also gives concise explanations and graphs to show their results, which helps its reader to understand the result clearly.

Weakness:
1. Although it is a great approach to use open sourced Shore to illustrate the needless part are actually needless in nowadays OLTP system, it would be more objective if the paper can conduct some research by comparing the light version Shore with some other currently using OLTP system in addition, since there exists the possibility that shore itself has some problems implementing the needless parts.
2. Although this paper did a great experiments by using TCP-C benchmarks, it would be more interesting to see the difference when deploying the original shore and light version shore to real world problems, since some key features of shore may be designed for real world problems that cannot be capture by benchmarks.
3. Some terminologies used in this paper lack a full explanations, which may lead its reader into confusion. For example, in section 5.2 it used "infra-query parallelism" but does not explain it. It would be great if the paper can gives full definition of terminology as "infra-query parallelism " to help its reader to understand.



Review 33

This paper speculates the potential performance gain of main-memory optimized database compared to traditional database that have most data fit into memory. The hardware speed and price are quite different from 1970s, when the traditional DBMS were first developed. This change motivates people to rethink the assumptions made with traditional databases, such as data cannot fit in memory. Moreover, the new workload in modern OLTP make alternative database system feasible. The authors break down a transactional database system(Shore) into a few components, including logging, locking, latching and buffer management. By progressively removing the components from this system and count the instructions, they analyze the percentage of time spend in each component. The result shows each component accounts for 10% ~ 30% of the total runtime. This observation suggest that a main-memory optimized database system with one component dropped is achieving significant performance gain. However, a fully stripped down system(a single-threaded database system with only query processing and recovers with copied nodes) is with orders of magnitude better performance compared to in-memory traditional database system.

The main advantage of this paper is the way it speculate the performance gain of alternative database systems, namely modularizing a traditional database system and count the number of instructions while removing the modules one at a time. This is a clever method to show how much benefit we can get by dropping a particular component in the traditional system.

However, the fact that they evaluated on Shore may be not convincing. Seems the Shore is research project without deployment in production. The conclusion from such a system is hardly broadened to popular production systems like Oracle or even MySQL and PostgreSQL. Those production system may be much more complex than Shore, thus leading to different observations. It would be more feasible if the authors can do similar analysis on systems with a much larger user base.