Skip to content

Commit b240c8f

Browse files
authored
Expand LookML adapter surface area (#111)
* Expand LookML adapter surface area Wire up previously unsupported LookML features to existing core model capabilities: view extends/inheritance resolution, refinement merging, label/value_format/drill_fields on dimensions and measures, hidden/ group_label/tags via meta dict, explore from:/description/label/ sql_always_where/always_filter, join type (left_outer/inner/full_outer/ cross), percentile and list measure types. * Fix three LookML adapter issues from PR review 1. Refinement scalar preservation: build Model kwargs conditionally so unset scalars (table, description, primary_key) stay out of model_fields_set and don't overwrite the base view during merge. 2. Explore alias in sql_on: when an explore uses from: aliasing, _parse_join now accepts both the view name and the explore alias when matching references in sql_on, preventing silent join drops. 3. Always filter view qualifier: strip view.field qualifiers (e.g. fact_orders.created_date -> created_date) before generating SQL, avoiding double-qualified {model}.view.col expressions. * Fix three more LookML adapter issues from PR review 1. sql_always_where ref translation: convert ${view.field} references to {model}.field so generated SQL is valid, not raw LookML syntax. 2. Resilient inheritance resolution: pre-filter to models whose full extends chain is present so one missing/circular parent doesn't block valid chains from resolving. 3. Percentile guard: skip percentile measures with no SQL instead of constructing a Metric(type="derived", sql=None) that fails validation.
1 parent 0ee407d commit b240c8f

3 files changed

Lines changed: 859 additions & 71 deletions

File tree

docs/compatibility/lookml.md

Lines changed: 42 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -16,11 +16,14 @@ Features are marked **supported**, **partial support**, or **unsupported**. Part
1616
| Multi-view files | Supported |
1717
| Directory parsing (recursive `.lkml` discovery) | Supported |
1818
| Empty/minimal views (zero dimensions) | Supported |
19-
| `view: +name {}` (refinements) | Partial support: parsed as a separate model named `"+name"`. First refinement kept, duplicates skipped. Fields are not merged into the base view. |
20-
| `extension: required` (abstract views) | Partial support: view parses normally but the flag is not stored. No validation prevents direct use. |
21-
| `extends: [base_view]` | Partial support: extending view is parsed with only its own fields. Inherited fields from the base view are not merged. Each view in an extends chain is independent. |
19+
| `view: +name {}` (refinements) | Supported (merged into the base view; multiple refinements are applied in order) |
20+
| `extension: required` (abstract views) | Supported (stored in `model.meta["extension_required"]`) |
21+
| `extends: [base_view]` | Supported (inheritance resolved using `core/inheritance.py`; child inherits all parent fields with child values taking precedence) |
22+
| `label` | Supported (stored in `model.meta["label"]`) |
23+
| `hidden` | Supported (stored in `model.meta["hidden"]`) |
24+
| `tags` | Supported (stored in `model.meta["tags"]`) |
2225

23-
Not mapped: `hidden`, `label`, `tags`.
26+
Multi-extends (`extends: [a, b]`) uses only the first parent. This is rare in practice.
2427

2528
---
2629

@@ -42,9 +45,17 @@ Not mapped: `hidden`, `label`, `tags`.
4245
| `${dimension_name}` references | Supported (resolved recursively, max depth 10; circular references degrade gracefully) |
4346
| `${other_view.field}` cross-view refs | Supported (preserved as-is in SQL) |
4447
| `description` | Supported |
48+
| `label` | Supported (stored in `Dimension.label`) |
49+
| `value_format_name` | Supported (stored in `Dimension.value_format_name`) |
50+
| `value_format` | Supported (stored in `Dimension.format`) |
4551
| `case: { when: {} }` (case dimension) | Supported |
52+
| `hidden` | Supported (stored in `Dimension.meta["hidden"]`) |
53+
| `group_label` | Supported (stored in `Dimension.meta["group_label"]`) |
54+
| `tags` | Supported (stored in `Dimension.meta["tags"]`) |
55+
| `order_by_field` | Supported (stored in `Dimension.meta["order_by_field"]`) |
56+
| `can_filter` | Supported (stored in `Dimension.meta["can_filter"]`) |
4657

47-
Not mapped: `hidden`, `label`, `group_label`, `value_format`, `value_format_name`, `order_by_field`, `drill_fields`, `tags`, `can_filter`, `suggest_dimension`, `suggest_explore`, `map_layer_name`, `alpha_sort`, `html:`, `link:`, `action:`.
58+
Not mapped: `drill_fields`, `suggest_dimension`, `suggest_explore`, `map_layer_name`, `alpha_sort`, `html:`, `link:`, `action:`.
4859

4960
---
5061

@@ -57,6 +68,8 @@ Not mapped: `hidden`, `label`, `group_label`, `value_format`, `value_format_name
5768
| `dimension_group type: time` | Supported (one `Dimension(type="time")` per timeframe) |
5869
| `timeframes: [...]` | Supported (`raw` explicitly skipped) |
5970
| Default timeframe | Supported (defaults to `["date"]` if unspecified) |
71+
| `label` | Supported (propagated to each generated dimension) |
72+
| `description` | Supported (propagated to each generated dimension) |
6073

6174
Naming convention: `{group_name}_{timeframe}`.
6275

@@ -98,15 +111,22 @@ Not mapped: `convert_tz`, `datatype`.
98111
| `type: string` (derived) | Supported |
99112
| `type: yesno` (boolean measure) | Supported (maps to derived) |
100113
| `type: period_over_period` | Supported (maps to `time_comparison` with `based_on`, `comparison_type`, `calculation`) |
114+
| `type: percentile` | Supported (generates `PERCENTILE_CONT` SQL as a derived metric; `percentile:` parameter value is preserved) |
115+
| `type: list` | Supported (generates `STRING_AGG(DISTINCT ...)` SQL as a derived metric; measures without SQL are skipped) |
101116
| `sql: ${dimension_ref}` | Supported (resolved to dimension SQL) |
102117
| `sql: ${measure_ref}` in type:number | Supported (converted to plain names for dependency resolution) |
103118
| No explicit type (sql present) | Supported (treated as derived) |
104119
| `description` | Supported |
105-
| `type: percentile` | Partial support: parses without error but the `percentile:` parameter value is lost. Becomes `agg_type=None`. |
106-
| `type: list` | Partial support: parses but has no native aggregation mapping. |
120+
| `label` | Supported (stored in `Metric.label`) |
121+
| `value_format_name` | Supported (stored in `Metric.value_format_name`) |
122+
| `value_format` | Supported (stored in `Metric.format`) |
123+
| `drill_fields` | Supported (stored in `Metric.drill_fields`) |
124+
| `hidden` | Supported (stored in `Metric.meta["hidden"]`) |
125+
| `group_label` | Supported (stored in `Metric.meta["group_label"]`) |
126+
| `tags` | Supported (stored in `Metric.meta["tags"]`) |
107127
| `type: date` | Partial support: becomes derived rather than a date-aware aggregation. |
108128

109-
Not mapped: `hidden`, `value_format`, `value_format_name`, `drill_fields`, `link:`, `tags`.
129+
Not mapped: `link:`.
110130

111131
---
112132

@@ -171,10 +191,17 @@ Not mapped: `persist_for`, `datagroup_trigger`, `sql_trigger_value`, `materializ
171191
| `sql_on: ${a.col} = ${b.col}` | Supported (foreign key extracted from `${model.column}` pattern) |
172192
| `relationship:` | Supported (all four: `many_to_one`, `one_to_one`, `one_to_many`, `many_to_many`) |
173193
| Multi-hop join detection | Supported (transitive joins skipped; adjacency graph computes path) |
194+
| `from:` (explore-level) | Supported (resolves to the actual view for model lookup) |
195+
| `from:` (join-level) | Supported (relationship points to actual view name, not the join alias) |
196+
| `description` | Supported (set on base model if model has no description) |
197+
| `label`, `group_label` | Supported (stored in `model.meta["explore_label"]` / `model.meta["explore_group_label"]`) |
198+
| `sql_always_where` | Supported (converted to a Segment on the base model) |
199+
| `always_filter` | Supported (each filter converted to a Segment on the base model) |
174200
| Explore-level `extends:` | Unsupported |
175-
| `from:` (aliased views) | Partial support: parsed by lkml but not resolved to a different view name. |
176201

177-
Not mapped: join `type` (`left_outer`, `inner`, etc.), `fields`, `sql_always_where`, `sql_always_having`, `always_filter`, `access_filter`, `required_joins`, `cancel_grouping_fields`, explore `label`, `description`, `group_label`.
202+
| join `type` (`left_outer`, `inner`, `full_outer`, `cross`) | Supported (stored in `Relationship.metadata["join_type"]`) |
203+
204+
Not mapped: `fields`, `sql_always_having`, `access_filter`, `required_joins`, `cancel_grouping_fields`, `conditionally_filter`.
178205

179206
---
180207

@@ -231,6 +258,9 @@ Sidemantic can export its semantic model back to LookML.
231258
| Filtered measures | Supported (`filters__all` format) |
232259
| Segments | Supported (exported as `filter:` blocks) |
233260
| Primary key | Supported |
261+
| `label` | Supported (roundtrips on dimensions and measures) |
262+
| `value_format_name` | Supported (roundtrips on dimensions and measures) |
263+
| `value_format` | Supported (roundtrips on dimensions and measures) |
264+
| `drill_fields` | Supported (roundtrips on measures) |
265+
| `hidden`, `group_label`, `tags` | Supported (roundtrips via `meta` dict) |
234266
| Roundtrip fidelity | Supported (LookML -> parse -> export -> re-parse produces semantically equivalent graphs) |
235-
236-

0 commit comments

Comments
 (0)