Alex Merced

Organizations face a common challenge: ensuring consistent and reliable data insights across multiple departments, tools, and teams. As data becomes increasingly central to decision-making, the need for a unified view—one everyone in the organization can rely on—has never been more critical. This is where a universal semantic layer comes into play. By creating a standardized […]

Modern enterprises are increasingly adopting data mesh architecture to keep up with demand for accessible, consistent data. Unlike traditional, centralized data models, data mesh prioritizes a decentralized approach, allowing individual teams to own and manage their own data domains. This structure enables organizations to achieve greater agility, faster access to data, and enhanced scalability. For […]

Organizations need rapid access to insights from their data to stay competitive. However, the complexity of managing data from diverse sources often slows down this process. Traditional methods like ETL (Extract, Transform, Load) are effective but can create delays due to data replication and movement. To overcome these challenges, data virtualization tools provide a robust […]

An Iceberg Data Lakehouse—a unified system that combines the scalability of data lakes with the analytical power of data warehouses—has emerged as a powerful solution to modern data requirements for performance, accessibility and costs. However, what makes this architecture effective is the strategic use of metadata to optimize performance, ensure data consistency, and enhance governance. […]

The demand for quick, actionable insights is higher than ever. Businesses are moving beyond traditional data warehouses to adopt lakehouses and other flexible data architectures that better support real-time analytics, BI, and AI applications. Dremio is at the forefront of this shift, providing a robust, high-performance hybrid lakehouse platform that enables fast, scalable analytics in […]

The rise of data lakehouses is transforming the way organizations manage, analyze, and leverage their data. Lakehouse architecture offers a flexible, scalable solution that bridges the gap between traditional data warehouses and data lakes. Apache Iceberg, an open table format designed to deliver reliable, high-performance analytics on large datasets, is at the heart of this […]

As organizations increasingly adopt hybrid data architectures, they often face challenges in accessing and analyzing data stored across cloud and on-premises environments. Databricks’ Unity Catalog offers a unified metastore that centralizes data management for cloud-based Delta Lake tables, enabling streamlined access to cloud data. At the same time, many companies retain valuable data in on-premises […]

Organizations often have a blend of cloud and on-premises data sources, creating a need for tools that can seamlessly bridge these environments. Dremio has introduced a new connector for Polaris Catalogs managed by Snowflake’s “Open Catalog” service, designed for Iceberg tables, and provides an open-source catalog solution for flexible data access and interoperability across cloud […]

Dremio has just rolled out version 25.2, and it’s bringing a feature many users have been eagerly waiting for – full dark mode across the entire platform. Whether you’re working late into the night or simply prefer the aesthetics and reduced eye strain that dark mode offers, this update brings a refreshing new way to […]

Organizations often have data distributed across cloud and on-premises environments, which poses significant integration challenges. Cloud-based platforms like Snowflake offer scalable, high-performance data warehousing capabilities, while on-premises systems like HDFS and Hive often store large volumes of legacy or sensitive data. Traditionally, analyzing data from these environments together would require complex data movement and transformation […]

Organizations are striving to build architectures that manage massive volumes of data and maximize the insights drawn from it. Traditional data architectures, however, often fail to handle the scale and complexity required. The modern answer lies in the data lakehouse, a hybrid approach combining the best aspects of data lakes and data warehouses. This blog […]

Flexibility and simplicity in managing metadata catalogs and storage solutions are key to efficient data platform management. Nessie’s REST Catalog Implementation brings this flexibility by centralizing table management across multiple environments in the cloud and on-prem, while PyIceberg provides an accessible Python implementation for interacting with Iceberg tables. In this blog, we’ll walk through setting […]

Version control is a key aspect of modern data management, ensuring the smooth and reliable evolution of both your data and the code that generates insights from it. While code versioning has long been a staple in software development, the advent of catalog versioning brings a powerful new tool to data teams. In this post, […]

The Apache Polaris (incubating) lakehouse catalog is the next step in the world of open lakehouses built on top of open community-run standards. While many other lakehouse catalogs are vendor-controlled or don’t enable full read-and-write support for Iceberg lakehouses, Polaris takes it a step further by being a community-run project integrating seamlessly with Apache Iceberg […]

The ability to transform raw data into actionable insights is critical. As organizations scale, they need efficient ways to standardize, organize, and govern data transformations. This is where dbt (data build tool) shines. What is dbt? dbt is an open-source tool that allows data analysts and engineers to transform raw data into models using SQL. […]

Operational analytic capabilities are foundational to delivering the personalized experiences that customers expect. While first and third-party market data are often the natural starting point, organizations are increasingly discovering that second-party data—the information exchanged securely and confidentially through partnerships and other strategic vendor relationships — are the differentiator that elevates the customer experience to new […]

Maintaining a well-structured and version-controlled semantic layer is crucial for ensuring consistent and reliable data models. With Dremio’s robust semantic layer, organizations can achieve unified, self-service access to data, making analytics more agile and responsive. However, as teams grow and models evolve, maintaining this layer can become increasingly complex. This is where automation and version […]

Efficiency and reliability are paramount when dealing with the orchestration of data pipelines. Whether you’re managing simple tasks or complex workflows across multiple systems, orchestration tools can make or break your data strategy. This blog will explore how orchestration plays a crucial role in automating and managing data processes, using tools like CRON and Apache […]

Introduction to the Hybrid Data Lakehouse Organizations are increasingly challenged to manage, store, and analyze vast data. While effective in the past, more than traditional data architectures is needed to meet the demands of modern data workloads, which require flexibility, scalability, and performance. This is where the concept of the hybrid data lakehouse comes into […]

In this tutorial, you’ll learn how to use Dremio’s Reflections to accelerate query performance. We’ll walk through the process of setting up a Dremio environment using Docker, connecting to sample datasets, running a complex query, and then using Reflections to significantly improve the query’s performance. Further Reading on Reflections Step 1: Spin Up a Dremio […]

Designing an optimal partitioning strategy for your data is often one of the most challenging aspects of building a scalable data platform. In traditional systems, data engineers frequently partition data by multiple columns, such as date and another frequently queried field. However, this can result in too many small files or partitions, which ultimately leads […]

Change Data Capture (CDC) is a design pattern used in databases and data processing to track and capture data changes—such as insertions, updates, and deletions—in real-time. Instead of periodically extracting entire datasets, CDC focuses on capturing only the data that has changed since the last update. This approach is crucial in modern data architectures, where […]

The recent support for the Iceberg REST Catalog Spec in Project Nessie marks a significant improvement in how we interact with the Iceberg catalog across multiple environments and languages. Historically, using a catalog required the client side be implemented for each language (Java, Python, Rust, Go), resulting in disparities of catalog support across different tools […]

The scale, speed, and variety of data are growing exponentially. Organizations are inundated with vast amounts of information from an ever-increasing number of sources, ranging from traditional databases to cloud-based systems and real-time data streams. This data deluge presents significant challenges for traditional data architectures, which often rely on extensive data pipelines and centralized storage […]

Organizations are inundated with vast amounts of data generated from diverse sources. Managing, processing, and extracting meaningful insights from this data is a significant challenge. The Data Lakehouse architecture has become the next evolution in overcoming these challenges, combining the best features of data lakes and data warehouses to deliver a unified platform for both […]

Businesses often face the daunting task of unifying data scattered across multiple platforms. Whether it’s transactional data stored in PostgreSQL, customer preferences housed in MongoDB, or analytics data in a data warehouse like Snowflake, integrating these disparate sources into a cohesive dataset is a complex challenge. This fragmentation not only complicates data analytics but also […]

Apache Iceberg and the Data Lakehouse architecture have garnered significant attention in the data landscape. Technologies such as Dremio, Nessie, and Minio play a vital role in enabling the Lakehouse paradigm, offering powerful tools for data management and analytics. In this blog, we’ll explore the concept of the Lakehouse, introduce the key technologies that make […]