The Dremio Blog

Modern organizations are increasingly reliant on data to drive innovation, optimize operations, and gain a competitive edge. However, extracting meaningful insights from the ever-growing volume of data presents a significant challenge. Despite substantial investments in data infrastructure and specialized teams, many organizations struggle to make their data readily accessible and actionable for decision-making. The traditional centralized approach to data management, while offering control and standardization, often leads to bottlenecks, delays, and frustrated data consumers. This, in turn, can hinder agility, stifle innovation, and ultimately impact the bottom line.

Organizations are constantly seeking ways to optimize data management and analytics. The Dremio and NetApp Hybrid Iceberg Lakehouse Reference Architecture brings together Dremio’s Unified Lakehouse Platform and NetApp’s advanced data storage solutions to create a high-performance, scalable, and cost-efficient data lakehouse platform. With this solution combining NetApp’s advanced storage technologies with Dremio’s high-performance lakehouse platform, […]

Apache Iceberg has an expansive ecosystem, and this article provides an overview of eight powerful tools that can facilitate data ingestion into Apache Iceberg and offers resources to help you get started. Whether leveraging Dremio's comprehensive lakehouse platform, using open-source solutions like Apache Spark or Kafka Connect, or integrating with managed services like Upsolver and Fivetran, these tools offer the flexibility and scalability needed to build and maintain an efficient and effective data lakehouse environment.

The evolution of data storage—from the simplicity of CSV and JSON to the efficiency of Parquet and the advanced capabilities of Apache Iceberg—reflects the growing complexity and scale of modern data needs. As organizations progress through this journey, the Dremio Lakehouse Platform emerges as a crucial ally, offering seamless query capabilities across all these formats and ensuring that your data infrastructure remains flexible, scalable, and future-proof. Whether you're just starting with small datasets or managing a vast data lakehouse, Dremio enables you to unlock the full potential of your data, empowering you to derive insights and drive innovation at every stage of your data journey.

A data flow design for modern data analytics. The medallion architecture empowers data analysts to access trusted data, collaborate with colleagues, and uncover invaluable insights quickly and efficiently. Analysts can unlock the full potential of their organization's data and drive informed decision-making by understanding the distinct layers of the data lakehouse and its role in unifying data analytics.

Platforms like Dremio facilitate this hybrid approach by connecting to various data sources and utilizing the Apache Iceberg format, ensuring that your data is always accessible and performant, regardless of where it resides. Whether you are looking to optimize costs, enhance performance, or achieve greater agility, a hybrid data lakehouse could be the perfect solution for your data needs.

With the latest release of Dremio, 25.0 we are helping accelerate the adoption and benefits of Apache Iceberg, while  bringing  your users closer to the data with lakehouse flexibility, scalability and performance at a fraction of the cost.  We are excited to announce some of the new features that improve scalability, manageability, ease of use […]

Dremio enables directly serving BI dashboards from MongoDB or leveraging Apache Iceberg tables in your data lake. This post will explore how Dremio's data lakehouse platform simplifies your data delivery for business intelligence by doing a prototype version that can run on your laptop.

Tabular and Dremio have received approval from the Apache Iceberg Project Management Committee to organize the inaugural Iceberg Summit, a free-to-attend virtual event to be held May 14 - 15, 2024. Iceberg Summit is an Apache Software Foundation (ASF) sanctioned event. Those wishing to attend can register here. Your information will only be used for […]

Dremio offers a robust platform for creating data products by simplifying data integration, providing a semantic layer for data curation, and enabling secure data sharing. Whether you're curating data for a single product or managing multiple data products, Dremio's features can streamline the process and enhance collaboration among data professionals, ultimately leading to the successful creation of valuable data products.

Learn how to optimize the data files in your Apache Iceberg Table using compaction and its different strategies including z-order.

A short introduction to the new file format called Puffin in Apache Iceberg that helps with additional table statistics

What happens under the hood with Apache Iceberg when you run a read query.

Time travel capabilities and privacy laws like GDPR and CCPA are at odds with each other. Here’s how to make sure you’re GDPR/CCPA compliant while using time travel in Apache Iceberg.

Learn how to ingest streaming data from AWS Kinesis into Apache Iceberg Tables using AWS Glue, and then query it with Dremio.