How do you guarantee fast writes to the database?

Choose a database with a storage engine that follows the LSM Tree approach to store and retrieve data.

An LSM Tree, which stands for Log-Structured Merge-Tree, is a data structure designed to store and retrieve key-value pairs efficiently. It is specifically optimized for high write throughput.

Storage engines based on this data structure are called LSM storage engines. For example, LevelDB and RocksDB are a couple of key-value storage engine libraries that use LSM trees. Similar storage engines are also used in databases like Cassandra and HBase.

4 Steps to LSM Storage Engine

Here’s how a typical LSM Storage Engine works:

• STEP 1: When a write request for a key-value pair arrives, it is added to an in-memory structure known as memtable.
• STEP 2: When the memtable gets bigger than some threshold, it is flushed to disk as an immutable file called an SSTable (Sorted String Table). This is handled asynchronously so that the new requests are not blocked.
• STEP 3: To serve a read request, first try to find the key in the memtable, then in the most recent SSTable, then in the next older SSTable, and so on.
• STEP 4: To keep the number of SSTable files in check, run a periodic merging and compaction process in the background to combine them and discard overwritten or deleted values.

You can play around with the diagram on Eraser.io

Components of an LSM Tree

What is an LSM Tree made up of?

It has a few key components:

• Memtable
• SSTable
• Compaction Process

The diagram below shows the overall design of LSM Trees

You can play around with the diagram on Eraser.io