Quadric Software

SQLite WAL in Alike 4.0

Performance is one of those things that makes everything else possible. If performance isn’t good enough, all the other bells and whistles start to matter a lot less. Because Alike has global data deduplication, it relies on databases to track crucial metainformation and storage data. The performance of these databases is critical since it affects how quickly data can be processed at the end of a backup when Alike begins consolidating the new backup data and later, during the validation phase, when Alike ensures data is cryptographically identical to what acquired from the hypervisor or machine.

Alike relies on SQLite as its database engine, and SQLite is the most commonly deployed database in the world, being used in everything from commercial accounting packages to automobile computers to even XenServer itself. SQLite is a very proven and reliable database, noted for its small footprint and excellent throughput.

Several years ago, SQLite introduced write-ahead logging, or WAL, as an option for handling database transactions. Quadric Software sat on the sidelines and kicked the tires on WAL, but didn’t want to put customers on the bleeding edge. Instead, we waited for the industry to vet this with us before folding it into our architecture.

Alike 4.0 presented the perfect opportunity to move forward. WAL changes SQLite’s journaling behavior, eliminating the creation of a rollback journal by instead writing uncommitted data to a new log and simply marking it as committed when the transaction is complete. Thus new data is logged “write-ahead” instead of the usual means of “write-behind”. This approach is significantly faster for applications that are write-intensive since the number of I/O operations are cut in half for the commit phase and readers spend less time being locked out of the database.

We’ve found in our internal operations that a 20% improvement in database performance is the floor for improvement, but the ceiling is significantly higher. This is because Alike scales better with WAL and thus can better handle faster disks with more capability for concurrent I/O load. Customers who have tested WAL in early access report installations running WAL are faster and more stable overall compared to their installs without out.

Alike 4.0 has some other I/O and database tweaks, such as defaulting to a better page size for SQLite and an auto-tune feature that selects the optimum threading configuration for an Alike install based on performance tests. Taken together, we expect great things for Alike performance, allowing you to really enjoy the bells and whistles.