|
This paper introduces RemusDB, a highly available database management system, whose technique can be applied to any DBMS with little or no customization. RemusDB is reliable and cost-effective. It is a active-standby HA solution based on running DBMS in a virtual machine, with three main advantages: (1) transparent to DBMS (2) runs on commodity hardware (3) low performance overhead The system is based on Remus, a VM checkpointing system, and develop the following ways to reduce overhead: (1) RemusDB include optimizations to memory checkpointing. Approaches include sending compressed data to backup, and protecting less memory space (ignoring some memory pages during checkpointing). (2) use the commit protection approach to reduce network buffering overhead. This is done by dynamically switch a client network connection between protected and deprotected modes. I think both optimizations are very good examples of how we can optimize a system where loose rules are used for checkpointing information. Particularly, I like the memory optimization approach. The idea of running XOR operation between old and new data to generate zeros and enable run-length encoding is very elegant and can provide good performance boost, and I really like this idea. I think the idea of protecting less memory is aggressive (requires a lot of extra coding to take care of things), but also seems reasonable. Another thing I like about this paper is, it succeeds in introducing the backgrounds step-by-step. It first introduces the problem, and then the original Remus system's approach and cons, then propose their own optimizations, finally give experiment results. I think the flow of this paper is very clear, making it very easy to follow and understand. |
|
Every DBMS needs some mechanisms to provide availability against possible hardware failure. Many systems have to serve users with uninterrupted service even if the hardware fails. Expensive code and extra hardware are used in order to maintain high availability in DBMS. Therefore, the paper proposes a reliable and cost-efficient solution on commodity hardware. The technique is based on Remus, a high availability solution uses asynchronous virtual machine state replication to provide transparent HA and failover capabilities. Some of the strengths and contributions of this paper are: 1. The paper pointed out the performance overhead of checkpointing systems, which are the large amount of state that needs to be backup during a checkpoint and the sensitivity of database workloads due to network latency. 2. The author implemented a LRU-based cache of frequently changing pages from previous checkpoints which can reduce the amount of data replication. 3. The paper conducts thorough experiment and analysis on the behaviors of RemusDB during failure. Some of the drawbacks of this paper are: 1. Current RemusDB implementation on commit protection has drawbacks that unbuffered packets advance TCP sequence counters the protecting situation are not reserved, which may lead to connection stalling until timeout. 2. RemusDB is not completely transparent to the DBMS. |
|
High Availability is necessary for any DBMS, and many database systems are designed to continue serving user requests with little or no disruption even when hardware fails.However, the main challenges of HA are the high complexity in code and setup and the need of extra specialized hardware. Therefore, this paper proposed a RemusDB, a reliable, cost-effective HA solution that is transparent to the DBMS, runs on commodity hardware, and incurs a low performance overhead. RemusDB provides HA guarantees by running the DBMS in a virtual machine and pushing much of complexity associated with HA out of the DBMS to the virtualization layer. The main contribution of RemusDB is it reduced the overhead of VM replication used by Remus and implemented the DBMS-aware VM checkpointing. Specifically, to reduce the number of states that needs to be transferred, RemusDB uses memory optimization including asynchronous checkpoint compress and disk read tracking. Next, to reduce the latency added to the communication path, RemusDB uses network optimization including commit protection. The main advantages of RemusDB are as follows: 1. RemusDB is an efficient HA solution implemented at the virtualization layer which offers HA as a service, relies on whole VM checkpointing and runs on commodity hardware. 2. RemusDB can make any DBMS highly available with little or no modification while imposing very little performance overhead. 3. Remus system shows comparable performance to other systems and has a very fast recovery time. 4. The idea to of disk read tracking is very novel and clever. RemusDB interposes on disk read requests from the virtual machine and tracks the set of memory pages into which the reads will be placed, and the associated disk addresses from which those pages were read. This approach saves sending the read disk over to stand-by virtual machine. This optimization largely reduces the memory replication overhead. 5. RemusDB is automatic and fully transparent failover to a warmed up system. The main disadvantages of RemusDB are follows: 1. RemusDB solved the hardware failure problem by using only one stand-by server, but if the backup server also fails, the whole DBMS could lose the connection with the clients and even lose data. 2. RemusDB did not fully explore the workload awareness. It would be a much better solution if RemusDB were able to recognize workload type immediately before deciding which optimization technique to use. 3. The Commit Protection of RemusDB requires code changes to the client communication layer of a DBMS, which is not trivial to implement. |
|
Problem & Motivation Maintaining availability in the face of hardware failures is an important goal for any database management system. However, the high availability (HA) comes at a high cost in terms of complex code in the DBMS. A solution to handle this problem is to use virtual machine replication. By adding the “virtualization layer”, the developer of the VM can push much of the complexity associated with HA out of the DBMS. The DBMS can run on the virtualization layer without or with few modifications. Yet, the traditional way like Remus imposes a significant performance overhead on database systems. Contributions: The authors revisit the architecture of the Remus. The key insight behind the Remus is to communicate with remote backups for each epoch and let the backup handle the query when the primary crashes. Then the authors point several optimizations, mainly from two aspects. The first aspect is the memory intensively. The authors argue that the amount of state that needs to be transferred from the primary VM to the backup VM is too large and some of them are unnecessary. 1. If the new checkpoint changes few contents compared to the previous checkpoint, we can encode the changes rather than sending the whole uncompressed pages. This can be achieved by xor the page within the memory and domain 0 - cache. 2. Many memory pages don’t need to be transferred if they can be reconstructed from the existing knowledge. For example, the clean page – the disk content is transferred as well, we can reconstruct the clean page from the disk. Also for the dirty page if we can reconstruct it from the commit log. The second aspect is reducing network overhead caused by buffering network packets for controlling output commit. If the DBMS wants to adopt this optimization, they need to do modifications to supersede the Remus network transaction protocol and decide which packets sent to the client should be protected. Drawbacks: It seems the outcome of this method doesn’t offer much better performance than other methods after the failure of the primary. In other words, the overhead of the system is still very large. |
|
As cloud services continue to become increasingly popular, large enterprises and organizations have begun to shift operations that were formerly done in-house over to service providers, in order to realize increased efficiency and reduce costs. Data management is no exception, but with the growing scale comes the challenge of being to provide high-availability to these clients. Providing high-availability often incurs additional complexity and/or cost to the system, however. This paper presents a technique for achieving high-availability for a large-scale DBMS that is flexible enough to be adapted easily to any DBMS, while requiring only a reasonable amount of performance overhead. It is based on virtual machine replication and leverages the capabilities of the underlying virtualization layer. The virtualization layer captures changes to the state of the whole VM (including the DBMS), which is propagated to the standby host and applied to a backup VM. Additionally, it detects failure and manages failover in a manner transparent to the DBMS while maintaining ACID properties and client connections. The system proposed in the paper is based on Remus, a VM checkpointing system that is a part of the Xen hypervisor, and provides strong availability guarantees and failure recovery, at the cost of significant performance overhead. This is due to the large amounts of memory used by database systems (which have to be copied over), as well as the sensitivity of database workloads to network latency. According to the authors, they provide ways to reduce this performance overhead, resulting in a system they call RemusDB. RemusDB is comprised of units of two servers, one being the active VM, which handles all requests during normal operation, and a standby VM, that constantly receives checkpoints from the active VM and takes over in the event of a failure in the active VM. By using the VM checkpointing, the entire state, including disk, memory, CPU, and network device state, is captured which makes it possible to perform fast failover while only incurring a small performance overhead. This requires some optimizations to reduce the size of a given Remus checkpoint, since database systems uses a lot of memory, as well as reducing latency added by network buffering. The first optimization is to send less data by using a LRU-based cache of frequently changing pages from previous checkpoints. The second is to protect less memory, i.e. selectively ignoring changes to certain parts of memory. Clean pages in the buffer pool do not need to be sent to checkpoints, and in some cases dirty buffer pool pages can be omitted if the DBMS can recover changes to these pages from the transaction log. Finally, the authors decided to use a less conservative approach to sending packets, since they saw that the latencies of sending packets were increased by orders of magnitudes of 2 to 3 due to Remus’s network buffering. Termed commit protection, the communication channel between the database server and client provides abstractions to dynamically enable or disable buffering of network messages. This allows them to guarantee that a committed transaction will survive a failure. Alongside this, they added a new setsockopt() option to Linux. The main strength of this paper is that it addresses a critical issue in designing and implementing large-scale database systems: maintaining high-availability at a reasonable overhead. By using an existing system, Remus, they are able to leverage the many strengths, particularly, VM checkpointing. As a result, they only had to focus on ways to reduce the performance and latency penalties incurred. As the results show, they were able to realize gains of around 30% over the existing Remus system, while still being able to maintain the desirable properties of transparent, quick failover. Additionally, this was achieved with a much lower complexity (in terms of lines of code) compared with similar approaches like Binlog for MySQL. One weakness of this paper is that it only compares its results against Remus, which was not originally designed to work directly with DBMSs in the first place, which would make performance gains and overhead reductions almost a foregone conclusion. If the authors could have been able to compare RemusDB with even more high availability solutions beyond Binlog, it would have been valuable in seeing just how well it stacks up with the competition. |
|
This paper outlines an approach to developing RemusDB, a high availability database management system (DBMS). The author’s approach is based on Remus, which is a previous high availability solution built on a virtualization layer, which allowed for high availability and enabled failover capabilities, but at the cost of performance overheads. The solution in this paper however provides fast failover and low performance overhead. The significance of this contribution is the need for highly available DBMSs, which typically comes at a high cost of code complexity and sometimes hardware needed. This paper proposes an active-standby method for sending updates to the DBMS from an active copy to a standby copy. This approach is based on the virtualization layer which detects changes in the active host DBMS and sends them to a backup virtual machine host. In the case of a failure the system must recover by using the standby host, while preserving the ACID properties we have seen before. The solution in this paper is a “DBMS-aware VM checkpointing system” based on Remus that attempts to reduce some of the performance overhead that Remus faced. The main contributions of this paper are reducing the amount of the system state that needs to be propagated at checkpoints and also reducing latency on the client-server communication line. The system works by taking frequent checkpoints which behave as heartbeats from the active server to the standby server. These checkpoints are installed on the standby server and in the event of a failure, the DBMS is run off the standby server which has the last version of the system state that was checkpointed. This failover is triggered when a heartbeat has not been received in some designated amount of time, and the standby server’s hypervisor ensures that all traffic is rerouted from the dead active server to the live standby server, which shares the same IP address as the active server. Some optimizations that the author took to improve performance from Remus were compressing the data sent in checkpoints and ignoring parts of memory that were not changed since the last checkpoint. I did not like how long it took for the authors to introduce their contributions. It was almost confusing how they mentioned Remus but did not distinguish it from RemusDB until the end of the introduction, well after the abstract and most of the introducion. I did like however how well the system seemed to scale as they increased their buffer pool size as well as when they increased the database size. |
|
RemusDB: Transparent High Availability for Database Systems This paper proposed RemusDB which provides transparent high availability for database system. The underlying situation is the need for high availability which means that database management system should remain accessible to users in the face of hardware failures. And users expect all time availability even for simple database application and high availability requirement is no longer limited to mission critical applications. However, the existing high availability solution are both complex and expensive. If you want to minimize the impact of high availability on performance, this will induce complexity in code structure and setup especially for specialized hardware. For instance, active and standby replication, though the copy of the database is stored at two servers, clients still lose connections on failure and sometimes the transactions might be lost. RemusDB can provide high availability as a service from the underlying virtualization infrastructure. There are several benefits of doing these. First, it has very low performance overhead. Second, it pushes the complexity out of the database management system. Third, it implements active/standby replication at the virtual machine layer. Forth, any database management can be make high availability with little or no modification. To sum up, RemusDB is a reliable, cost-effective, active.standby high availability solution implemented at the virtualization layer propagates all changes in virtual machine sate from primary to backup. There will be no lost in connection, and no lost in transactions. And both database management system state and the database are replicated not just the database. RemusDB is based on Remus which is part of the Xen hypervisor, so that it imposes significant performance overhead on databases. It has the properties of maintaining replica of a running virtue machine on a separate physical machine, and it extends live migration to do efficient virtue machine replication, and it provides transparent failover with only seconds of downtime. The contribution of RemusDB is that it maintains availability in the face of hardware failures which is an important aspect of any database management system. Then, RemusDB is an efficient high availability solution implemented at the virtualization layer, and it offers high availability as a service and relies on whole virtue machine chieck-pointing and it runs on commodity hardware. Finally, the RemusDB can make any DBMS highly available with little modification while imposing very little performance overhead. The advantages of this paper is: first, the RemusDB is completeness which means that it never loses transactions. Second, it is transparent, it never loses client connections. Third, it has lower implementation complexity. The drawback of the this paper is that it fails to compare the performance with other databases already offering high availability. Second, this paper does not consider the overhead of running databases inside virtual machine. |
|
This paper mainly presents a solution for building a high-availability database management system, which is named RemusDB. One of the key challenges in providing high availability lies in maintaining database consistency in the face of failures and minimizing the impact of high availability on performance. Current system involve complex code within the DBMS, and this code is often tightly integrated into the rest of the database. An alternative is RemusDB, a system that use Virtual Machines and replication to maintain high availabiliry, instead of complex DBMS code. RemusDB is a Remus-based virtual machine checkpoint system. Remus is a general-purpose VM checkpoint system with high performance overhead on database systems that supports DBMS high availability. The system makes frequent incremental checks on the complete state of virtual machines on active servers. These checkpoints will be moved to the standby virtual machine where they are installed. If the standby database does not receive these checkpoints, it is assumed that the primary server is closed and that it will become an active system. This article describes in detail how to minimize disk access and how to reduce the amount of data transferred in checkpoints. Both can lead to significant performance improvements. To reduce the size of checkpoints synchronized from active nodes, they use asynchronous checkpoint compression, where the entire original page content is not sent to the checkpoint, because most modifications between successive checkpoints only change part of the page. To avoid checking pages that can be read from disk, they also track disk read operations and put metadata into checkpoint data, which can be used by standby servers to rebuild these memory pages. The main contribution of this article is to propose the idea of RemusDBwhich support high availability though virtual machine replication. The contribution also lies on that the system provide a low overhead and can be run on commodity hardware and provide high availability. The system also overcome the weakpoint of the original Remus system that the overhead is huge. It implements multiple memory optimizations to reduce performance overhead in traditional VM strategies. It implements LRU cache tracking to frequently change pages to avoid unnecessary messages and compression checkpoints. The weak point that during the experiment, the comparation is only with the different configuration of RemusDB, but fail to perform a compare with other existing databases. |
|
RemusDB is a system that provides high availability (HA) for DBMSs. It takes advantage of VM checkpointing to keep a backup server up-to-date. There is an existing system, Remus, that uses VM checkpointing for HA, but RemusDB provides some DBMS specific optimizations. One optimization is that less data is sent by using compression - when data is going to be sent, pages are XORed, leading to a page of mostly zeros, as usually there are not many changes. Then, run-length encoding is used to compress the data. Another is the idea of commit transaction, which essentially says that Remus’s expensive network transaction protocol is not necessary as the DBMS already provides a robust transaction processing system, which RemusDB adds to. These optimizations and more are implemented, and in the results section, the system is tested with various sets of optimizations. To me the most valuable contribution of this paper is the simplicity of the solution. Instead of implementing a many-thousand line HA system, the authors piggyback on something that already exists in the form of VM checkpointing. Sometimes innovation does not need to be complex, it can just be clever. These kinds of solutions can be less error-prone and more easily generalizable to other systems, as RemusDB is. In the results section, the system is evaluated with both MySQL and Postgres. It is mentioned that Postgres currently has no integrated HA support, but MySQL does with binlog replication. The downsides of binlog replication as a HA solution are addressed clearly, and some are quite legitimate - binlog replication may lose up to one transaction, client side recovery is more complex, and there is a lot of complexity in the implementation leading to bugs. Ultimately, though, I think that DBAs are interested in sticking with what they know has worked in the past for many people, which in this case is binlog replication. This makes me wonder if Remus can get adoption. Perhaps one way to do it is a direct integration with MySQL or cloud providers offering it. I also wondered about the feasibility of running in a VM, as I believe the previous paper mentioned that it used a lot of resources. RemusDB is a system that provides high availability (HA) for DBMSs. It takes advantage of VM checkpointing to keep a backup server up-to-date. There is an existing system, Remus, that uses VM checkpointing for HA, but RemusDB provides some DBMS specific optimizations. One optimization is that less data is sent by using compression - when data is going to be sent, pages are XORed, leading to a page of mostly zeros, as usually there are not many changes. Then, run-length encoding is used to compress the data. Another is the idea of commit transaction, which essentially says that Remus’s expensive network transaction protocol is not necessary as the DBMS already provides a robust transaction processing system, which RemusDB adds to. These optimizations and more are implemented, and in the results section, the system is tested with various sets of optimizations. To me the most valuable contribution of this paper is the simplicity of the solution. Instead of implementing a many-thousand line HA system, the authors piggyback on something that already exists in the form of VM checkpointing. Sometimes innovation does not need to be complex, it can just be clever. These kinds of solutions can be less error-prone and more easily generalizable to other systems, as RemusDB is. In the results section, the system is evaluated with both MySQL and Postgres. It is mentioned that Postgres currently has no integrated HA support, but MySQL does with binlog replication. The downsides of binlog replication as a HA solution are addressed clearly, and some are quite legitimate - binlog replication may lose up to one transaction, client side recovery is more complex, and there is a lot of complexity in the implementation leading to bugs. Ultimately, though, I think that DBAs are interested in sticking with what they know has worked in the past for many people, which in this case is binlog replication. This makes me wonder if Remus can get adoption. Perhaps one way to do it is a direct integration with MySQL or cloud providers offering it. I also wondered about the feasibility of running in a VM, as I believe the previous paper mentioned that it used a lot of resources. |
|
High-availability is always desired in any database management systems. However, it’s quite hard to actually implement a system that has high-availability. The code is complex, the setup is complex and sometimes even extra hardware is needed. As a solution to these problems, this paper purposed RemusDB, which is based on virtual machine replication and leverages the capabilities of the underlying virtualization layer to provide high-availability. This method pushes much of the complexity of high-availability out of the DBMS, so any DBMS running on a virtualized infrastructure can use this method with very little or even no code changes. The key idea here is that the virtualization layer will capture all the changes in the virtual machine including those for the DBMS and propagate them to the backup VM. The virtualization layer can also detect failures and manages the failover from the active host to the backup. This idea by itself is not new, however, this paper made several improvements which greatly reduces overhead introduced by this method. The overhead has two sources. The first one is memory checkpointing. A naive approach will copy all the pages of memory that change from the active host and transmit them over the network to the backup. During the process, the commit message should be buffered and can’t be sent to the client. The buffering is controlled by Remus. However, this also means the larger the checkpoint, the longer a commit message should be buffered. To relief this problem, RemusDB utilize an LRU-based cache of frequently changing pages. When sending pages to the backup, the system will first check if the previous version of this pages exists in the cache. If so, the pages are XORed and resulting in a page that contains mostly zeros. The result is then run-length encoded for transmission. Another approach RemusDB takes is that it ignores pages whose contents can be regenerated or can be safely thrown. The second source of overhead is network buffering. RemusDB implemented an approach to commit protection to solve the problem. Using this approach, DBMS can decide which message should be protected and which, thus reduce the overhead. I think this paper is quite easy to follow and to understand. It basically treats the database as a black box and copies the whole DBMS state to the backup. One downside I found is that the performance is not clear when the buffer pool size of the database is very large. The experiment only shows the result for up to 1000MB buffer pool size. But in reality, people may use a size of 10GB. |
|
In the paper "RemusDB: Transparent High Availability for Database Systems", Umar Farooq Minhas and Co. discuss RemusDB, a system that is used to describe a technique for building a high-availability database management system (HADBMS). Much like Amazon's motto for high-availability, high-availability systems are becoming much more popular due to the user's expectations for 24/7 service for even non-critical applications. To add onto this, businesses can even suffer some monetary losses and unexpected disruptions from hardware failure that are beyond their control. However, implementing high availability is not as simple as it seems - it usually requires very complex code, complex setup for a database administrator, and even complex hardware. Amazingly, RemusDB's technique can be applied to any DBMS with very little customization at the cost of reasonable performance overhead. Naturally, it seems that this is a trade-off that one must consider when choosing the schema for a database. This done through a virtualization layer that captures changes in the state at the active host and propagates them to the standby host. This layer also preserves all ACID properties making the transition from a failure both transparent and seamless. Since this technique has some desirable traits for some companies, it is an interesting and important problem to explore. The paper is divided into several sections: 1) System overview: In a single DBMS, two servers are used to provide high-availability; one hosts the active VM while the other handles all client requests in normal operation. While the active VM is running, it is being constantly monitored by a standby VM on a second physical server. This is being monitored through a heart-beat system. In the case of failure, the active VM falls over to the standby VM in order to preserve all the ACID properties. Specifically, the failover should persist the effect of transactions that commit and should not compromise transactional atomicity. Check-pointing captures the entire state of the active VM, which includes disk, memory, CPU, and network device state. This can potentially be really large (in terms of memory overhead) to store, but there are several optimizations that come into play to reduce this. 2) Memory optimizations: Even though the concept of check-pointing is desirable for high-availability, there are major consequences in terms of memory overhead. One optimization that can be made is sending less data - there is usually lots of redundancy between checkpoints. RemusDB uses a LRU-based cache of frequently changing pages from previous checkpoints in order to compress data to make it a manageable size. Another optimization comes from protecting less memory - RemusDB ignores certain parts of memory and frees up memory to use for other tasks such as lock tables, query plan cache, working memory for query operators, and connection states. 3) Commit protection: One requirement for a high-available DBMS is that it should be able to protect transactional control messages (acknowledgments to commit and abort as well as sever generated abort messages). These messages that compromise the contents of the transaction can remain unprotected - they dub this under "commit protection". In order to follow through with this, several steps are taken: if there is a commit or abort, the client/server communication mode is switched to private mode. Afterwards, all the necessary steps are taken that are involved in a commit or abort and written to logs. An acknowledgment message is sent and the channel is turned back into the deprotected mode. Thus, there is no risk for leakage of confidential transactional information. Much like other papers, this paper has several drawbacks. The first drawback that I noticed was in the related works section. I was surprised to see that there was no mention of Amazon on it's eventually consistent scheme. Since Amazon's infrastructure revolves around 99.99% availability at the cost of misplaced information, it seems like a good model to judge their design on (I'm fully aware that Amazon probably will not consent to comparing system performance since it is not open-source). Another drawback that I noticed was the experimental verification of the availability of their system. I felt that it was a bit far-fetched to say that failure recovery is quick (~3secs) with only one test - there needs to be multiple tests and these trials need to have summary statistics for each. This includes both tests for MySQL and PostgreSQL. The last drawback I noticed was the lack of discussion for future work. In the appendix, there is mention of memory deprotection techniques - these could potentially be used to improve performance and availability. Since this is on the cloud, the need to save on computing resources is always something to make a big deal about. |
|
This paper describes RemusDB, which converts the Remus Highly Available system to one that works well for DMBSs specifically. Remus is a Highly Available system, so it keeps itself available to the user in the event of crashes. It does this by maintaining two virtual machines. One of the machines is the one that the user directly interacts with, and one is a backup. In the standard Remus system, the active virtual machine constantly sends checkpoints to the backup machine. Each checkpoint is very simple, and contains a copy of every page in the machine’s memory that has changed since the last checkpoint. If the active machine fails, then the user seamlessly switches to using the backup machine, which starts from the last checkpoint. In order to preserve durability, the active machine doesn’t send any commit acknowledgements until the results of that commit have been checkpointed on the backup machine. However, the size of the checkpoints and their network usage can cause issues for running a DBMS. Since DBMSs change small parts of many memory pages, each checkpoint tends to be quite large. RemusDB has two ways of handling this. First, it keeps a cache of the most recently used memory pages. Any page in this cache isn’t copied over in its entirety; instead, the system just calculates the changes made on that page, compresses them, and sends that. This is referred to as memory deprotection. Second, RemusDB implements disk read tracking. It does this by keeping track of which pages are read from disk. If these pages are unchanged, or if the changes can be reconstructed from the log, then they won’t be copied over in a checkpoint. The other issue that Remus has is network latency. In order to keep durability, as mentioned before, it won’t send messages back to the client until a checkpoint happens. However, this can introduce unacceptable latency when using a DBMS. However, this can be fixed with a more relaxed implementation. There are several kinds of DBMS messages that don’t need to be buffered; as such, these can be sent without waiting for a checkpoint. Only commit-crucial messages need to be buffered, so the latency between when commits happen and when the client receives confirmation is minimized. The positive aspects of this paper are that it’s able to show how to convert an existing system into one that is more friendly to running DBMSs. It’s able to dive into which properties a DBMS actually needs, and which ones it can relax. The negative aspect would be that it doesn’t show off very large changes; in general, it’s still just using most of the original Remus system, so the high availability that the DBMS provides is mostly provided by something else. |
|
This paper presents a technique for building a high-availability database management system. The approach is based on Remus, which uses asynchronous virtual machine state replication to provide transparent high-availability and failover capabilities. This approach protects a DBMS and incurs a performance overhead of up to 32% as compared to an unprotected DBMS. Therefore, optimization is needed to mitigate the problem. HA is needed because maintaining availability in the face of hardware failure is important. However, the high availability comes with a high cost. Therefore, a cost-effective HA is needed. The paper first talks optimizations to mitigate the performance overhead. Remus takes a deliberately simple approach to memory checkpointing: at every checkpoint, it copies all the pages of memory that change from the active host and transmits them over the network to the backup host. This provides high availability with an acceptable degree of overhead. The first stage of the optimization is compressing checkpoints, which is beneficial when the amount of data to be replicated is large and the data contains redundancy. An LRU-based cache of frequently changing pages from previous is implemented to support the optimization. The advantage of compressing checkpoints is that it moves the implementation into an asynchronous stage and allowing the VM to resume execution in parallel with compression and replication. The second part of the optimization is protecting less memory by selectively ignoring changes to certain parts of memory so that the performance overhead is reduced. For example, memory pages whose contents can be regenerated, or alternatively can be safely thrown away may be ignored during checkpointing. The two checkpointing optimizations developed are disk read tracking and memory deprotection. For commit protection, several protocols are used to implement commit protection inside a DBMS: if the message received from the client is either COMMIT or ABORT, or if the server decides to abort an active transaction, switch the client/server communication channel to protected mode; perform all actions necessary to commit or abort the transaction, up to writing the COMMIT or ABORT log record to disk; send the corresponding acknowledgement to the client through the channel after successfully committing or aborting the transaction; lastly, switch the channel back to deprotected mode. This protocol ensures that the client only receives the acknowledgement of the commit or abort after state changes made by this transaction. Lastly, the paper set up experiments to evaluate the system. The advantage of this paper is that its experiment is very thorough and examines many aspects of the system. Therefore, the experimental results are convincing. The weakness of this paper is that there could be more graphs illustrating the flow of the optimizations, which could be more clear for understanding. |
|
This paper proposed RemusDB, which is a high-availability database management system. The highlight of the paper is that the proposed RemusDB can be applied to any DBMS with little or no customization with a reasonable performance overhead. The proposal is based on a high availability solution implemented in the virtualization layer, using asynchronous virtual machine state replication to provide transparent high availability and failover capability. The paper discussed that high availability comes at high cost in terms of complex code in DBMS, complex setup for the database administrator, and sometimes extra specialized hardware. So the proposal of this paper is very promising for end users. The paper showed that the idea of providing high availability by replicating machine state at the virtualization layer is new but there is a significant performance overhead when such a technique is applied on database systems. So they proposed several ways t reduce the overhead and finally come up with the RemusDB.To deal with the high overhead caused by the situation that database systems consume a lot of memory and database workloads can be sensitive to network latency, this paper extends Remus with optimizations for reducing checkpoint size and for reducing latency added by network buffering. Two memory optimization methods were developed. One is sending less data, based on the observation that the database workload involves a large set of frequently changing pages of memory and the memory writes often change only a small part of the pages. Another is protecting less memory, by ignoring changes to certain parts of memory. I think the strong part of the paper is exactly proposing a transparent high availability service outside the DBMS and can be used by almost all DBMSs without additional effort. And the performance of the proposed technique is really promising. The overhead of the high availability service is worthwhile. The shortcoming of the paper to me is that it doesn't consider the overhead of virtual machine. In the last paper I read, relational cloud. It was mentioned that virtual machine can be the bottleneck of the performance. So I think this paper should also take it into consideration. |
|
In this paper, the authors present a technique for building a high-availability (HA) database management system. The proposed technique can be applied to any DBMS with little or no customization, and with a reasonable performance overhead. The problem they are going to solve is to design a new system which can provide high-availability DBMS service in an underlying infrastructure. This issue is definitely an important issue for the DBMS nowadays because for internet companies, making their service always-on takes first priority, the high-availability property is always preferable. Also, it is important if one can easily improve the availability of DBMS without change the DBMS itself, this will reduce the extra-overhead for data migration and parsing to different DBMS. Based on these demands, RemusDB is introduced which is based on Remus, a commodity HA solution implemented in the virtualization layer, that uses asynchronous virtual machine state replication to provide transparent HA and failover capabilities. Next, I will summarize the crux of RemusDB with my understanding. In this paper, they develop ways to reduce this overhead of virtual machine replication and implement them in a DBMS-aware VM checkpointing system called RemusDB. The main way they are using to reduce the overhead for VM replication includes checkpoint compression, tracking disk reads. For checkpoint compression, RemusDB utilizes an LRU cache to maintain all the frequent changing pages and only sends the different part between two versions when the page resides in the cache. It reduces the amount of data sends to the backup VM. They track the disk reads by using a strategy like copy-on-write. The replica will not read a page from disk as primary do until the page is modified by the primary. In RemusDB, they also provide an interface for DBMS to decide which network packet to be buffered until next checkpoint. In this way, the memory space is saved by avoiding store the unprotected packet. As a result, in their experiment, they show that RemusDB can provide higher availability by only sacrificing a little bit of performance. I think this is a nice paper with great insights. The most important technical contribution of this paper is the introduction of RemusDB, a framework that provides high availability for existing DBMS platforms. Another important contribution is that this paper raises a discussion about whether the complexity of HA is best implemented within the DBMS, or as a service by the infrastructure below it. There are several advantages of this paper, first of all, the RemusDB is built at a lower level so that this method can be easily extended to other systems. Second, the idea behind the RemusDB is simple, one can easily adapt it. Also, it provides good insight for DBaaS problems, rather can incorporate distributed layer with DBMS, we can push down the problem to low-level, this idea is very nice. I think this solution is good for some startup company who do not have a lot of much to build a strong infrastructure, they can easily build up their backend storage system with RemusDB. There are also some downsides of this paper. I think it likes some tradeoff between performance and adaptivity. The solution proposed by RemusDB utilizing the virtualization layer can be a good solution, however, it is still true that the overhead of this solution is also high. I think this framework can only be applied to some scenario where the performance requirement is not very high, for a large-scale web application, it is better to modify the DBMS itself to achieve high availability. Besides, from the system itself, I think RemusDB is not always available, since maintaining availability in the face of hardware failures is an important goal for any DBMS, it only uses one standby server, if that standby server also fails at the same time, the whole service will be down and may also lose data. Another issue is the experiments, in this paper, they compare different optimizations with different configurations, while it does not compare the performance with other DBMS that supports HA. I wonder the performance of RemusDB compared to other HA databases. |
|
This paper introduces RemusDB, which is a High Availability (HA) system whose main goal is to provide extremely high availability for a DBMS system. Its main strategy for doing this is by running the DBMS inside a virtual machine and leveraging the capabilities of the virtualization layer to take care of many issues that the DBMS would normally have to do, such as. If a machine fails, frequent incremental checkpointing via asynchronous VM replication (which is taken care of by Remus) is used to handle the failure and is much more efficient than normal replication, which helps RemusDB achieve its high availability goal. This process is called managing a failover from an active VM to a standby VM. Several optimizations are also given, such as compressing checkpoints, advanced disk read tracking, and protecting commits by eliminating redundant transactions that the original Remus system and instead implementing them at the DBMS level. I think one major strength of RemusDB is the fact that little to no customization is required in order to implement this technique. Its low overhead is even more impressive considering this, as low overhead is usually associated with high levels of customization and environment-specific optimizations. One weakness is that some of their best performance benefits only worked under INSERT WORKLOAD HERE, so the “one-size-fits-all” benefit isn’t fully true. Finally, the introduction of this paper confused me, as it introduced HA systems as “Businesses can suffer costly and embarrassing disruptions when hardware fails” and “27x7 availability” which is supposed to imply that HA systems are necessary in these cases, but hasn’t this been true for many of the non-HA systems we’ve seen throughout the semester as well? Especially when we’ve been talking about Amazon & Google’s systems, high availability was important—so I’m not totally sure as to what systems we’ve seen have been HA and which haven’t, based on this paper’s description of a HA system. |