Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BigQuery adbc_ingest Does Not work #2172

Open
WillAyd opened this issue Sep 19, 2024 · 1 comment
Open

BigQuery adbc_ingest Does Not work #2172

WillAyd opened this issue Sep 19, 2024 · 1 comment
Labels
Type: bug Something isn't working

Comments

@WillAyd
Copy link
Contributor

WillAyd commented Sep 19, 2024

What happened?

When trying to ingest data into BigQuery I get a ProgrammingError: INVALID_ARGUMENT: unknown statement string type option adbc.ingest.target_table error

Stack Trace

Cell In[29], line 14
     11 tbl = pa.Table.from_pydict({"col": [0, 1, 2]})
     13 with adbc_driver_bigquery.dbapi.connect(db_kwargs) as conn, conn.cursor() as cur:
---> 14     cur.adbc_ingest(table_name="foo", data=tbl)

File ~/clones/arrow-adbc/python/adbc_driver_manager/adbc_driver_manager/dbapi.py:895, in Cursor.adbc_ingest(self, table_name, data, mode, catalog_name, db_schema_name, temporary)
    891 if db_schema_name is not None:
    892     options[
    893         adbc_driver_manager.StatementOptions.INGEST_TARGET_DB_SCHEMA.value
    894     ] = db_schema_name
--> 895 self._stmt.set_options(**options)
    897 if temporary:
    898     self._stmt.set_options(
    899         **{
    900             adbc_driver_manager.StatementOptions.INGEST_TEMPORARY.value: "true",
    901         }
    902     )

File ~/clones/arrow-adbc/python/adbc_driver_manager/adbc_driver_manager/_lib.pyx:1482, in adbc_driver_manager._lib.AdbcStatement.set_options()

File ~/clones/arrow-adbc/python/adbc_driver_manager/adbc_driver_manager/_lib.pyx:260, in adbc_driver_manager._lib.check_error()

ProgrammingError: INVALID_ARGUMENT: unknown statement string type option `adbc.ingest.target_table`
> /home/demo/code/adbc14/adbc_driver_manager/_lib.pyx(260)adbc_driver_manager._lib.check_error()

How can we reproduce the bug?

import adbc_driver_bigquery.dbapi
from adbc_driver_bigquery import DatabaseOptions
import pyarrow as pa

db_kwargs = {
    DatabaseOptions.PROJECT_ID.value: "some-demo-project-1234",
    DatabaseOptions.DATASET_ID.value: "demo_dataset",
    DatabaseOptions.TABLE_ID.value: "foo",
}

tbl = pa.Table.from_pydict({"col": [0, 1, 2]})

with adbc_driver_bigquery.dbapi.connect(db_kwargs) as conn, conn.cursor() as cur:
    cur.adbc_ingest(table_name="foo", data=tbl)

Environment/Setup

No response

@WillAyd WillAyd added the Type: bug Something isn't working label Sep 19, 2024
@WillAyd WillAyd changed the title BigQuery Ingest Does Not work BigQuery adbc_ingest Does Not work Sep 19, 2024
@joellubi
Copy link
Member

Hi @WillAyd. The bigquery driver is still not feature-complete, and bulk ingestion is among the features that still need to be implemented. I thought we already had an issue tracking the remaining work but I can't find it now.

I'll open an issue today to help track the features that still need to be added.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Type: bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants