Skip to content

fix(clickhouse): combine physical_properties into one SETTINGS clause#5817

Merged
StuffbyYuki merged 1 commit into
SQLMesh:mainfrom
mokashang:fix/clickhouse-physical-properties-settings
May 29, 2026
Merged

fix(clickhouse): combine physical_properties into one SETTINGS clause#5817
StuffbyYuki merged 1 commit into
SQLMesh:mainfrom
mokashang:fix/clickhouse-physical-properties-settings

Conversation

@mokashang
Copy link
Copy Markdown
Contributor

Description

Fixes #5803.

A ClickHouse model that declared more than one physical_properties entry produced SQL with repeated SETTINGS keywords, e.g.

SETTINGS min_age_to_force_merge_seconds = 3600 SETTINGS min_age_to_force_merge_on_partition_only = 1

ClickHouse rejects this with a syntax error. The valid form is a single comma-separated clause:

SETTINGS min_age_to_force_merge_seconds = 3600, min_age_to_force_merge_on_partition_only = 1

Root cause

In ClickhouseEngineAdapter._build_table_properties_exp (and the analogous _build_view_properties_exp), every leftover table_properties_copy / view_properties_copy entry was wrapped in its own exp.SettingsProperty and added to the property list. sqlglot renders each SettingsProperty as a separate SETTINGS … keyword, so N settings became N independent clauses.

Fix

Refactor _build_settings_property to accept the full settings mapping and emit a single SettingsProperty whose expressions list holds one exp.EQ(...) per key=value pair. Both the table and view paths now call it once with the entire dict and append the resulting property, producing the expected SETTINGS k1 = v1, k2 = v2, … output.

The behavior for a single setting is unchanged — a one-entry SettingsProperty still renders as SETTINGS k = v.

Test Plan

  • Updated the existing test_model_properties assertion that pinned the broken SETTINGS prop1 = 1 SETTINGS prop2 = '2' output; it now expects the combined form.
  • Added a three-entry regression test inside test_model_properties that mirrors the model from the issue report (min_age_to_force_merge_seconds, min_age_to_force_merge_on_partition_only, index_granularity) to guard against the failure mode beyond N=2.
  • Added a new test_view_properties_combine_settings that exercises _build_view_properties_exp directly, since the same bug existed there and had no unit-test coverage.
  • Verified the rendered SQL with sqlglot matches ClickHouse's accepted syntax.
  • pytest tests/core/engine_adapter/ — 517 passed, 1 skipped, no regressions.
  • ruff check and mypy clean on the modified files.

Checklist

  • I have run make style and fixed any issues
  • I have added tests for my changes (if applicable)
  • All existing tests pass (make fast-test)
  • My commits are signed off (git commit -s) per the DCO

When a ClickHouse model declared more than one physical_property,
_build_table_properties_exp produced a separate exp.SettingsProperty
per key, which sqlglot renders as repeated 'SETTINGS k = v' keywords.
ClickHouse rejects that with a syntax error at execution time.

Refactor _build_settings_property to take the full settings mapping
and emit a single SettingsProperty with one EQ expression per entry,
yielding the expected 'SETTINGS k1 = v1, k2 = v2' form. The same fix
applies to view properties, which used the identical pattern.

Update the existing assertion that pinned the broken output, add a
three-entry regression test mirroring the issue reporter's model, and
cover the view-properties path that was silently affected.

Fixes SQLMesh#5803

Signed-off-by: mokashang <shangmengjiajiajia@gmail.com>
@StuffbyYuki StuffbyYuki force-pushed the fix/clickhouse-physical-properties-settings branch from af767e8 to ac6df37 Compare May 29, 2026 14:51
@StuffbyYuki StuffbyYuki merged commit c5d008b into SQLMesh:main May 29, 2026
31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: Multiple physical_properties generates duplicate SETTINGS clauses for ClickHouse

2 participants