Tidehunter: Sui’s Next-Generation Database Optimized For Low Latency And Reduced Write Amplification

by
Alisa Davidson

Printed: February 03, 2026 at 2:10 am Up to date: February 03, 2026 at 2:10 am

by Victor Dey

Edited and fact-checked:
February 03, 2026 at 2:10 am

To enhance your local-language expertise, typically we make use of an auto-translation plugin. Please word auto-translation will not be correct, so learn authentic article for exact info.

Sui, a Layer 1 blockchain community, has launched Tidehunter, a brand new storage engine engineered to align with the efficiency calls for, information entry traits, and operational constraints generally present in modern blockchain infrastructures. 

The system is positioned as a possible successor to the prevailing database layer utilized by each validators and full nodes, reflecting a broader effort to modernize core infrastructure in response to the evolving scale and workload profiles of manufacturing blockchain environments.

Sui initially relied on RocksDB as its major key–worth storage layer, a extensively adopted and mature resolution that enabled fast protocol improvement. Because the platform expanded and operational calls for elevated, basic limitations of general-purpose LSM-tree databases turned more and more obvious in production-like environments. 

In depth tuning and deep inner experience couldn’t totally tackle structural inefficiencies that conflicted with the entry patterns typical of blockchain methods. This led to a strategic shift towards designing a storage engine optimized particularly for blockchain workloads, ensuing within the improvement of Tidehunter.

A central issue behind this choice was persistent write amplification. Measurements underneath sensible Sui workloads confirmed amplification ranges of roughly ten to 12 instances, that means that comparatively small volumes of software information generated disproportionately giant quantities of disk site visitors. Whereas such habits is widespread in LSM-based methods, it reduces efficient storage bandwidth and intensifies rivalry between background compaction and skim operations. In write-intensive or balanced read-write environments, this overhead turns into more and more restrictive as throughput scales. 

Load testing on high-performance clusters confirmed the affect, with disk utilization nearing saturation regardless of reasonable software write charges, highlighting the rising mismatch between typical storage architectures and fashionable blockchain efficiency necessities.

Tidehunter Structure: A Storage Engine Optimized For Blockchain Entry Patterns And Sustained Excessive-Throughput Workloads

Storage habits in Sui and comparable blockchain platforms is dominated by a small set of recurring information entry patterns, and Tidehunter is architected particularly round these traits. A big portion of state is addressed utilizing cryptographic hash keys which are evenly distributed and sometimes map to comparatively giant data, which removes locality however simplifies consistency and correctness. 

On the similar time, blockchains rely closely on append-oriented buildings, akin to consensus logs and checkpoints, the place information is written so as and later retrieved utilizing monotonically growing identifiers. These environments are additionally inherently write-heavy, whereas nonetheless requiring quick entry on latency-critical learn paths, making extreme write amplification a direct menace to each throughput and responsiveness.

On the heart of Tidehunter is a high-concurrency write pipeline constructed to take advantage of the parallel capabilities of recent solid-state storage. Incoming writes are funneled by a lock-free write-ahead log able to sustaining extraordinarily excessive operation charges, with rivalry restricted to a minimal allocation step. 

Information copying proceeds in parallel, and the system avoids per-operation system calls by utilizing writable memory-mapped information, whereas sturdiness is dealt with asynchronously by background companies. This design produces a predictable and extremely parallel write path that may saturate disk bandwidth with out changing into constrained by CPU overhead.

Decreasing write amplification is handled as a major architectural goal quite than an optimization step. As a substitute of utilizing the log as a short lived staging space, Tidehunter shops information completely in log segments and builds indexes that reference offsets immediately, eliminating repeated rewrites of values. 

Indexes are closely sharded to maintain write amplification low and to extend parallelism, eradicating the necessity for conventional LSM-tree buildings. For append-dominated datasets, akin to checkpoints and consensus data, specialised sharding methods preserve current information tightly grouped in order that write overhead stays steady whilst historic information grows.

For tables addressed by uniformly distributed hash keys, Tidehunter introduces a uniform lookup index optimized for predictable, low-latency entry. Moderately than issuing a number of small and random reads, the index reads a barely bigger contiguous area that statistically incorporates the specified entry, permitting most lookups to finish in a single disk spherical journey. 

This method intentionally trades some learn throughput for decrease and extra steady latency, a tradeoff that turns into sensible as a result of diminished write amplification frees substantial disk bandwidth for learn site visitors. The result’s extra constant efficiency on latency-sensitive operations akin to transaction execution and state validation.

To additional management tail latency at scale, Tidehunter combines direct I/O with application-managed caching. Massive historic reads bypass the working system’s web page cache to forestall cache air pollution, whereas current and continuously accessed information is retained in user-space caches knowledgeable by application-level entry patterns. Together with its indexing format, this reduces pointless disk spherical journeys and improves predictability underneath sustained load.

Information lifecycle administration can also be simplified. As a result of data are saved immediately in log segments, eradicating out of date historic information may be carried out by deleting complete log information as soon as they fall outdoors the retention window. This avoids the complicated and I/O-intensive compaction mechanisms required by LSM-based databases and permits quicker, extra predictable pruning whilst datasets increase.

Throughout workloads designed to replicate actual Sui utilization, Tidehunter demonstrates larger throughput and decrease latency than RocksDB whereas consuming considerably much less disk write bandwidth. Probably the most seen enchancment comes from the close to elimination of write amplification, which permits disk exercise to extra carefully match application-level writes and preserves I/O capability for reads. These results are noticed each in managed benchmarks and in full validator deployments, indicating that the beneficial properties lengthen past artificial testing.

Analysis is carried out utilizing a database-agnostic benchmark framework that fashions sensible mixes of inserts, deletions, level lookups, and iteration workloads. Assessments are parameterized to replicate Sui-like key distributions, worth sizes, and read-write ratios, and are executed on {hardware} aligned with beneficial validator specs. Beneath these circumstances, Tidehunter persistently sustains larger throughput and decrease latency than RocksDB, with the most important benefits showing in write-heavy and balanced eventualities.

Validator-level benchmarks additional affirm the outcomes. When built-in immediately into Sui and subjected to sustained transaction load, methods utilizing Tidehunter preserve steady throughput and decrease latency at working factors the place RocksDB-backed deployments start to undergo from rising disk utilization and efficiency degradation. Measurements present diminished disk strain, steadier CPU utilization, and improved finality latency, highlighting a transparent divergence in habits underneath comparable load.

Tidehunter represents a sensible response to the operational calls for of long-running, high-throughput blockchain methods. As blockchains transfer towards sustained quite than burst-driven workloads, storage effectivity turns into a foundational requirement for protocol efficiency. The design of Tidehunter displays a shift towards infrastructure constructed explicitly for that subsequent stage of scale, with additional technical element and deployment plans anticipated to comply with.

Alisa, a devoted journalist on the MPost, makes a speciality of cryptocurrency, zero-knowledge proofs, investments, and the expansive realm of Web3. With a eager eye for rising tendencies and applied sciences, she delivers complete protection to tell and have interaction readers within the ever-evolving panorama of digital finance.

Extra articles