Skip to content

ADBC support? #5814

@pimvh

Description

@pimvh

Hi! Thanks for creating this project!

I am wondering, have you considered supporting ADBC as a connecting interface towards the different databases? ADBC is relatively new, but read for production imo. I think introducing ADBC would:

  1. simplify the codebase, and ease the maintenance of the several engine adapters, as they share the similar dbapi interface and have less non-shared dependencies.
  2. improve the preformance of fetchdf when moving large amounts of data.
  3. allow a BYOD (bring your own driver) approach.

Some pointers:
https://arrow.apache.org/adbc/current/python/recipe/postgresql.html#using-pandas-and-adbc
https://docs.adbc-drivers.org/drivers/bigquery/index.html

The difference between for example BigQuery, Databricks and DuckDB Quack and Trino then becomes:

from adbc_driver_manager import dbapi

dbapi.connect(
    driver="bigquery" # driver="databricks" or driver="quack" or driver="trino"
    db_kwargs={
       # these can be passable via the config and vary per database and user requirements
       # "uri": "quack://localhost:9494/?token=quack-secret"
    },
)

I think it would simplify managing models in multiple databases/datalakes, and enhances/leverage the existing power of SQLGlot even more.

I'd have to dive deeper into the code to figure out the overall impact on the codebase though.

Metadata

Metadata

Assignees

No one assigned

    Labels

    ImprovementImproves existing functionality

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions