Summary
The Tinybird DynamoDB connector is generally available through the Tinybird CLI (tb connection create dynamodb, TYPE dynamodb in .connection files, IMPORT_TABLE_ARN / IMPORT_EXPORT_BUCKET in .datasource files), but the Python SDK currently only exposes helpers for Kafka, S3, and GCS.
This forces SDK users to keep a mixed project (Python definitions + raw .connection / .datasource files) just for DynamoDB.
Docs page: https://www.tinybird.co/docs/forward/ingest-data/connectors/dynamodb
Proposal
Add a define_dynamodb_connection helper and a dynamodb block on define_datasource, mirroring the Kafka / S3 / GCS shape.
Connection
from tinybird_sdk import define_dynamodb_connection, secret
my_ddb = define_dynamodb_connection("my_ddb", {
"arn": secret("dynamodb_role_arn_my_ddb"), # DYNAMODB_ARN
"region": "us-east-1", # DYNAMODB_REGION
})
Datasource
from tinybird_sdk import define_datasource, engine, t
from .connections import my_ddb
orders = define_datasource("orders", {
"schema": {
"pk": t.string(),
"sk": t.string(),
"_record": t.string(),
"_old_record": t.string().nullable(),
"_timestamp": t.date_time64(3),
"_event_name": t.string().low_cardinality(),
"_is_deleted": t.uint8(),
},
"engine": engine.replacing_merge_tree({
"sorting_key": ["pk", "sk"],
"ver": "_timestamp",
"is_deleted": "_is_deleted",
}),
"dynamodb": {
"connection": my_ddb,
"table_arn": "arn:aws:dynamodb:us-east-1:123456789012:table/orders", # IMPORT_TABLE_ARN
"export_bucket": "my-orders-exports", # IMPORT_EXPORT_BUCKET
},
})
Notes:
- The DynamoDB datasource must use
ReplacingMergeTree; consider validating this in the SDK.
- The columns
_record, _old_record, _timestamp, _event_name, _is_deleted are mandatory system columns. Consider exposing them as a helper / preset so users don't have to redeclare them every time.
- Once
engine.replacing_merge_tree and the dynamodb block exist, the docs at https://www.tinybird.co/docs/forward/ingest-data/connectors/dynamodb can add Python SDK tabs alongside the Tinybird CLI ones.
Mirror of the matching request on the TypeScript SDK: tinybirdco/tinybird-sdk-typescript#164
Why
Today docs need an explicit callout that the DynamoDB connector is CLI-datafile-only, which breaks the "define everything in Python" workflow the SDK is built around.
Summary
The Tinybird DynamoDB connector is generally available through the Tinybird CLI (
tb connection create dynamodb,TYPE dynamodbin.connectionfiles,IMPORT_TABLE_ARN/IMPORT_EXPORT_BUCKETin.datasourcefiles), but the Python SDK currently only exposes helpers for Kafka, S3, and GCS.This forces SDK users to keep a mixed project (Python definitions + raw
.connection/.datasourcefiles) just for DynamoDB.Docs page: https://www.tinybird.co/docs/forward/ingest-data/connectors/dynamodb
Proposal
Add a
define_dynamodb_connectionhelper and adynamodbblock ondefine_datasource, mirroring the Kafka / S3 / GCS shape.Connection
Datasource
Notes:
ReplacingMergeTree; consider validating this in the SDK._record,_old_record,_timestamp,_event_name,_is_deletedare mandatory system columns. Consider exposing them as a helper / preset so users don't have to redeclare them every time.engine.replacing_merge_treeand thedynamodbblock exist, the docs at https://www.tinybird.co/docs/forward/ingest-data/connectors/dynamodb can add Python SDK tabs alongside the Tinybird CLI ones.Mirror of the matching request on the TypeScript SDK: tinybirdco/tinybird-sdk-typescript#164
Why
Today docs need an explicit callout that the DynamoDB connector is CLI-datafile-only, which breaks the "define everything in Python" workflow the SDK is built around.