JDBC Driver for Arrow Flight SQL: An Open Source Universal Driver

Today we’re excited to announce the release of the JDBC driver for Apache Arrow Flight SQL, an open source driver that specializes in moving large amounts of data very quickly. This driver benefits Dremio users in terms of performance, efficiency, and time to insight. It allows you to use any JDBC client and connect to your Dremio datasets. You can also connect to any data source that supports Arrow Flight SQL or directly through Python. This new driver works with Dremio Version 21.0+.

Benefits of JDBC Driver for Arrow Flight SQL

Currently, most data analysts and data scientists must use one proprietary driver per database when connecting to their data. This requires them to download and keep each driver up to date, which is cumbersome, especially for less technical users.

The Arrow Flight SQL JDBC driver streamlines the connection experience by creating a universal driver and bringing the performance of Arrow Flight to every JDBC client. As more Arrow Flight SQL endpoints are added to databases, JDBC clients can connect to those databases. Flight SQL is a standardized protocol, so the same driver can talk to any system that supports this protocol. The Apache Arrow Flight protocol saves time and network resources by not having to serialize and then deserialize the data. Using this modern protocol allows you to move large amounts of data over the network quickly, which reduces time to insight and enables users to ask more questions and work with larger datasets.

Dremio’s Contribution to the Apache Arrow Community

Dremio has a long history of working with and contributing to the Apache Arrow community. In that spirit, this driver is now available to the Apache Arrow community. We will collaborate with the broader community to extend the driver to meet the growing community and industry needs.

Get Started with the JDBC Driver for Dremio Cloud and Software

If you’re using Dremio Cloud, the new driver works with Dremio Cloud with no additional effort. If you’re using Dremio Software, simply upgrade to version 21, which includes the Arrow Flight SQL endpoint that enables you to use the new JDBC driver. To get started with the new driver, download and install it on your client.

We are working with various JDBC-based BI tools to get the new Arrow Flight SQL JDBC driver natively embedded so you won't have to install or manage the driver — it will automatically be part of your choice of tools.

How to Use JDBC Driver with DBeaver Client

You can leverage the new Arrow Flight JDBC driver and start using it with your Dremio cluster today. For example, here’s an overview of how  you would use the new driver with the DBeaver client.

1. First, you need to create a new driver by following these instructions for the DBeaver client in Dremio documentation.

2. Once your driver is set up, you will see it in the Driver Manager.

3. Next, you will create a new connection from the JDBC URL. Then you will see the option of the new driver.

4. And success, you are connected to your data in Dremio.

5. You can now query Dremio from an existing client tool, but under the covers, leverage the performance benefits of Arrow Flight. Give it a try!

Learn More

Use the following resources to learn more about Apache Arrow Flight and the new JDBC driver:

• Understanding Apache Arrow Flight
• An Introduction to Apache Arrow Flight SQL
• Arrow Flight SQL: A Universal JDBC Driver