What’s New in Dremio,  Setting New Standards in Query Stability and Durability

The Version 25 release underscores Dremio's commitment to providing an SQL query engine that excels in performance and sets new benchmarks in stability and durability.

A New Standard in Query Performance

Dremio's bold assertion that it offers market-leading stability and durability isn't just a claim—it's a commitment backed by substantial advancements in technology. The platform is designed to handle highly complex workloads with unmatched stability, ensuring that no query fails, regardless of the workload's complexity or size.

Enhanced Query Durability

At the core of Dremio's new release is improved query durability. This feature guarantees consistent and reliable performance across large and complex workloads, living up to the vision that no query will fail. Such robustness is crucial for businesses that depend on timely and accurate data analytics for their decision-making processes.

Innovations in Product Features

Dremio version 25 introduces several key features aimed at bolstering the engine's durability and performance, particularly for substantial and intricate workloads:

Advanced Memory Management

One of the standout enhancements is the improved memory management capabilities. This advancement eliminates the risk of out-of-memory issues, a common challenge in data analytics, ensuring uninterrupted query execution. Dremio's memory management system meticulously monitors the dynamic memory usage of high-risk operators, allocating memory as necessary to prevent failures.

When a complex query or process is about to exhaust available memory, Dremio smartly diverts the process to disk, reducing memory consumption and guaranteeing continuity. This optimizes memory usage and enhances performance, scalability, and stability.

Spillable Hash Joins

Introducing spillable hash joins marks a significant upgrade in Dremio's join processes. When memory capacity is surpassed, the system temporarily shifts data to disk, allowing the join operation to continue without hiccups. This approach significantly boosts scalability, enabling Dremio to process datasets larger than the available memory, thus facilitating seamless queries on vast data volumes.

Memory Arbiter

The new memory arbiter feature further fortifies Dremio's defense against out-of-memory errors. It empowers executors to closely monitor and allocate memory in real time, enhancing the system's responsiveness and reliability during memory-intensive operations.

Conclusion

Dremio's version 25 is not just an incremental update; it's a transformative release that redefines stability and durability in SQL analytics. By introducing sophisticated memory management, spillable hash joins, and a proactive memory arbiter, Dremio ensures businesses can rely on its engine for their most critical and complex data workloads. These enhancements reinforce Dremio's position as a leader in the data analytics space, offering an engine that is powerful but also resilient and dependable, catering to the evolving needs of modern enterprises.

Get Started with Dremio Today