Skip to content

Commit 0ee407d

Browse files
nicosuavenicosuave
authored
Add compatibility docs for LookML, Cube, OSI, and Malloy adapters (#109)
Comprehensive feature-by-feature documentation of what each adapter supports, partially supports, or does not support when importing from these semantic layer formats.
1 parent 63c6222 commit 0ee407d

4 files changed

Lines changed: 1051 additions & 0 deletions

File tree

‎docs/compatibility/cube.md‎

Lines changed: 288 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,288 @@
1+
# Cube Compatibility
2+
3+
Sidemantic's Cube adapter parses YAML schema files (`.yml` / `.yaml`) and maps Cube concepts to Sidemantic's semantic model (Model, Dimension, Metric, Segment, Relationship, PreAggregation). It also supports exporting back to Cube YAML for roundtrip workflows.
4+
5+
Features are marked **supported**, **partial support**, or **unsupported**. Partial support entries include notes explaining the limitation. Properties that parse without error but have no Sidemantic equivalent (UI hints, caching config, access control) are grouped together per section rather than listed individually.
6+
7+
---
8+
9+
## Schema Format
10+
11+
| Feature | Status |
12+
|---------|--------|
13+
| YAML schema (`.yml`, `.yaml`) | Supported |
14+
| Directory parsing (recursive YAML discovery) | Supported |
15+
| Multiple cubes in one file | Supported |
16+
| Empty YAML files | Supported (silently skipped) |
17+
| JavaScript schema (`.js`) | Unsupported |
18+
19+
The adapter only reads YAML. Cube's original JavaScript schema format (`cube.js` files with `cube()` function calls) is not parsed. Projects using JS schemas must convert to YAML first (Cube supports both natively since v0.31).
20+
21+
---
22+
23+
## Cubes
24+
25+
| Feature | Status |
26+
|---------|--------|
27+
| `name` | Supported |
28+
| `sql_table` | Supported (stored as `Model.table`) |
29+
| `sql` (inline query) | Supported (stored as `Model.sql`, `Model.table = None`) |
30+
| `description` | Supported |
31+
| `extends` | Partial support: the extending cube is parsed with only its own fields. Inherited fields from the base cube are not merged. Each cube in an extends chain is independent. |
32+
| `public` / `shown` | Unsupported |
33+
| `data_source` | Unsupported |
34+
| `sql_alias` | Unsupported |
35+
| `refresh_key` (cube-level) | Unsupported |
36+
| `calendar: true` | Partial support: parses without error but the flag is not stored. The cube is treated as a regular model. |
37+
38+
Not mapped: `title`, `rewrite_queries`, `context_members`.
39+
40+
---
41+
42+
## Dimensions
43+
44+
| Feature | Status |
45+
|---------|--------|
46+
| `name` | Supported |
47+
| `type: string` | Supported (maps to `categorical`) |
48+
| `type: number` | Supported (maps to `numeric`) |
49+
| `type: time` | Supported (maps to `time`, default granularity `day`) |
50+
| `type: boolean` | Supported (maps to `categorical`) |
51+
| `type: geo` | Partial support: parses without error but latitude/longitude sub-fields are not stored. Maps to `categorical`. |
52+
| `type: switch` | Partial support: parses without error. The `values` list is not stored. Maps to `categorical`. |
53+
| `sql: ${CUBE}.column` | Supported (`${CUBE}` replaced with `{model}` placeholder) |
54+
| `sql: {CUBE}.column` (no dollar sign) | Supported (also replaced with `{model}`) |
55+
| `sql: ${cube_name}.column` | Supported (cube-name references replaced with `{model}`) |
56+
| `primary_key: true` | Supported |
57+
| `description` | Supported |
58+
| `format` | Supported (stored on `Dimension.format`) |
59+
| Cross-cube dimension references (`${other_cube.field}`) | Supported (preserved as-is in SQL) |
60+
| Path-qualified references (`{b.d.id}`) | Supported (preserved as-is in SQL) |
61+
| `case: { when: [...] }` (case dimensions) | Partial support: the `case` block is parsed by YAML without error but the case/when/else structure is not evaluated or stored. The dimension has no SQL expression. |
62+
| `sub_query: true` | Unsupported (flag ignored, dimension SQL preserved verbatim) |
63+
| Custom `granularities` on time dimensions | Unsupported (parsed by YAML but not stored) |
64+
| `meta` | Unsupported |
65+
66+
Not mapped: `shown`, `title`, `propagate_filters_to_sub_query`, `case` labels.
67+
68+
---
69+
70+
## Measures
71+
72+
| Feature | Status |
73+
|---------|--------|
74+
| `type: count` | Supported |
75+
| `type: count_distinct` | Supported |
76+
| `type: count_distinct_approx` | Supported (maps to `count_distinct`, approximation semantics lost) |
77+
| `type: sum` | Supported |
78+
| `type: avg` | Supported |
79+
| `type: min` | Supported |
80+
| `type: max` | Supported |
81+
| `type: number` (calculated/derived) | Supported (see Derived Measures below) |
82+
| `sql: ${CUBE}.column` | Supported (normalized to `{model}` placeholder) |
83+
| `sql: ${dimension_ref}` (referencing a dimension) | Supported (preserved for resolution) |
84+
| `description` | Supported |
85+
| `format` | Supported (stored on `Metric.format`) |
86+
| `type: rank` | Partial support: parses without error but rank semantics (`order_by`, `reduce_by`) are not stored. Becomes a regular measure with `agg_type=count` (default fallback). |
87+
| `type: string` | Unsupported (no mapping; falls back to `count`) |
88+
| `type: boolean` | Unsupported (no mapping; falls back to `count`) |
89+
| `type: running_total` | Unsupported (no mapping) |
90+
91+
Not mapped: `shown`, `title`, `drill_members` (on import; used on export), `meta`, `drill_filters`.
92+
93+
---
94+
95+
## Derived Measures (`type: number`)
96+
97+
Derived measures are detected by `type: number` and handled with several strategies depending on the SQL pattern.
98+
99+
| Pattern | Status |
100+
|---------|--------|
101+
| Simple ratio: `${measure1} / NULLIF(${measure2}, 0)` | Supported (converted to `ratio` metric type with `numerator`/`denominator`) |
102+
| Ratio with type cast: `${measure1}::float / NULLIF(${measure2}, 0)` | Supported |
103+
| Complex derived: `${measure1} - ${measure2} + ${measure3}` | Supported (measure references converted to `cube_name.measure_name` format) |
104+
| Inline aggregation: `COUNT(CASE WHEN ... THEN 1 END)::float / NULLIF(COUNT(*), 0)` | Supported (detected as SQL expression metric, `agg_type=None`) |
105+
| Scalar multiplication: `${mrr} * 12` | Supported (treated as derived with measure reference replacement) |
106+
107+
---
108+
109+
## Measure Filters
110+
111+
| Feature | Status |
112+
|---------|--------|
113+
| `filters: [{ sql: "..." }]` | Supported (SQL expressions normalized and stored in `Metric.filters`) |
114+
| Multiple filters | Supported (all filters stored; combined as AND at query time) |
115+
| `${CUBE}` / `{CUBE}` in filter SQL | Supported (normalized to `{model}`) |
116+
| `${cube_name}` in filter SQL | Supported (normalized to `{model}`) |
117+
118+
---
119+
120+
## Rolling Window / Cumulative Measures
121+
122+
| Feature | Status |
123+
|---------|--------|
124+
| `rolling_window: { trailing: unbounded }` | Supported (metric type set to `cumulative`, window stored) |
125+
| `rolling_window: { trailing: "1 month" }` | Supported (trailing value stored in `Metric.window`) |
126+
| `rolling_window: { offset: end }` | Partial support: the `offset` value is not stored. Only `trailing` is captured. |
127+
| `rolling_window: { leading: "-1 month" }` | Partial support: the `leading` value is not stored. |
128+
| `rolling_window: { type: to_date, granularity: year }` | Partial support: treated as cumulative. The `type` and `granularity` sub-fields are not stored. |
129+
130+
---
131+
132+
## Segments
133+
134+
| Feature | Status |
135+
|---------|--------|
136+
| `segments: [{ name, sql }]` | Supported (mapped to `Segment`) |
137+
| `${CUBE}` / `{CUBE}` replacement in segment SQL | Supported |
138+
| `${cube_name}` replacement in segment SQL | Supported |
139+
| `description` | Supported |
140+
| Query-time segment application | Supported |
141+
| Segments without `sql:` | Supported (correctly skipped, not added) |
142+
143+
---
144+
145+
## Joins
146+
147+
| Feature | Status |
148+
|---------|--------|
149+
| `joins: [{ name, sql, relationship }]` | Supported (creates `Relationship` on the base model) |
150+
| `relationship: many_to_one` | Supported |
151+
| `relationship: one_to_many` | Supported |
152+
| `relationship: one_to_one` | Supported |
153+
| `relationship: many_to_many` | Supported (stored as-is) |
154+
| Foreign key extraction from `${CUBE}.column = ${target.id}` | Supported (regex-based extraction for both `many_to_one` and `one_to_many`) |
155+
| FK fallback | Supported (falls back to `{join_name}_id` convention if regex parse fails) |
156+
| Diamond join patterns (A -> B -> D, A -> C -> D) | Supported (each cube's joins parsed independently) |
157+
| Multi-hop transitive joins | Supported (graph traversal computes paths at query time) |
158+
| Nullable foreign keys | Supported (join SQL preserved; LEFT JOIN semantics at query time) |
159+
160+
Not mapped: join `type` (not the same as `relationship`; Cube does not use this concept).
161+
162+
---
163+
164+
## Pre-Aggregations
165+
166+
Pre-aggregations are fully mapped to Sidemantic's `PreAggregation` model, including refresh configuration, partitioning, and indexes.
167+
168+
| Feature | Status |
169+
|---------|--------|
170+
| `type: rollup` | Supported |
171+
| `type: rollupJoin` / `rollup_join` | Supported (type normalized to `rollup_join`) |
172+
| `type: rollupLambda` / `lambda` | Supported (type normalized to `lambda`) |
173+
| `type: original_sql` | Supported |
174+
| `measures` (list of measure references) | Supported (`CUBE.` prefix stripped) |
175+
| `dimensions` (list of dimension references) | Supported (`CUBE.` prefix stripped) |
176+
| `time_dimension` | Supported (`CUBE.` prefix stripped) |
177+
| `granularity` (hour, day, week, month, quarter, year) | Supported |
178+
| `partition_granularity` | Supported |
179+
| `refresh_key: { every }` | Supported |
180+
| `refresh_key: { sql }` | Supported |
181+
| `refresh_key: { incremental }` | Supported |
182+
| `refresh_key: { update_window }` | Supported |
183+
| `scheduled_refresh` | Supported (defaults to `true`) |
184+
| `indexes: [{ name, columns, type }]` | Supported |
185+
| `build_range_start` / `build_range_end` | Supported (SQL expression extracted) |
186+
| Cross-cube dimension references in pre-aggs (e.g., `visitors.source`) | Partial support: parsed as dimension name string; the cross-cube prefix is not stripped. |
187+
| `rollups` (list of rollup references for rollupJoin/rollupLambda) | Unsupported (not stored) |
188+
| Empty pre-aggregation sections (YAML null) | Supported (treated as empty list) |
189+
190+
---
191+
192+
## Views
193+
194+
Unsupported. The `views:` top-level section in Cube YAML is silently ignored during parsing. Files that contain only views (no `cubes:` section) parse without error and produce an empty graph.
195+
196+
Cube views are a composition layer that project and rename members from cubes via `join_path`, `includes`, `excludes`, `prefix`, and `alias`. None of these concepts are mapped:
197+
198+
- `join_path` traversal
199+
- `includes: "*"` wildcard and selective includes
200+
- `excludes` list
201+
- `prefix: true` namespacing
202+
- `alias` renaming
203+
- View-level `access_policy`
204+
- `folders` (grouping members in views)
205+
- `extends` on views
206+
207+
---
208+
209+
## Hierarchies
210+
211+
Unsupported. `hierarchies:` blocks on cubes are parsed by YAML without error but not stored. Hierarchies define drill-down level ordering (e.g., year -> quarter -> month) and cross-cube level references. They have no Sidemantic equivalent.
212+
213+
---
214+
215+
## Access Control (`access_policy` / `accessPolicy`)
216+
217+
Unsupported. Access policy blocks on cubes and views are parsed by YAML without error but not stored. This includes:
218+
219+
- `role` / `group` definitions
220+
- `row_level` filters with `member`, `operator`, `values`
221+
- `row_level: { allow_all: true }`
222+
- `member_level: { includes, excludes }`
223+
- `conditions` with security context expressions (`{ security_context.* }`)
224+
225+
---
226+
227+
## Multi-Stage Calculations
228+
229+
Unsupported. The `multi_stage: true` flag on measures is parsed by YAML without error but not stored. Multi-stage calculations in Cube enable measures that reference other measures as inputs, run in separate query stages, and support features like `group_by` (for percent-of-total) and `time_shift` (for period comparisons). The adapter parses the measure's `sql` and `type` normally, so the measure still appears in the graph, but multi-stage execution semantics are not reproduced.
230+
231+
Related unsupported sub-features:
232+
- `time_shift: [{ time_dimension, interval, type }]` on measures
233+
- `group_by` (percent-of-total grouping)
234+
- `order_by` / `reduce_by` (ranking)
235+
- `case` / `switch` / `when` / `else` on measures
236+
237+
---
238+
239+
## Custom Calendars and Granularities
240+
241+
Unsupported. Cube supports custom calendar cubes (`calendar: true`) with custom granularity definitions on time dimensions (`granularities: [{ name, sql, interval, origin }]`) and dimension-level `time_shift` definitions. The adapter parses these structures without error but does not store them. Time dimensions always default to `granularity: day`.
242+
243+
---
244+
245+
## SQL Syntax Normalization
246+
247+
The adapter normalizes three Cube SQL reference patterns to Sidemantic's `{model}` placeholder:
248+
249+
| Cube Pattern | Sidemantic Output |
250+
|-------------|-------------------|
251+
| `${CUBE}` | `{model}` |
252+
| `{CUBE}` (no dollar sign) | `{model}` |
253+
| `${cube_name}` (e.g., `${orders}`) | `{model}` |
254+
| `{cube_name}` (no dollar sign) | `{model}` |
255+
256+
This normalization applies to dimension SQL, measure SQL, filter SQL, and segment SQL.
257+
258+
Measure-to-measure references (`${measure_name}`) in derived measures are converted to `cube_name.measure_name` format for Sidemantic's dependency resolution.
259+
260+
---
261+
262+
## Cube Export (Roundtrip)
263+
264+
Sidemantic can export its semantic model back to Cube YAML format.
265+
266+
| Feature | Status |
267+
|---------|--------|
268+
| Cubes with `sql_table` | Supported |
269+
| Cubes with `sql` (inline query) | Supported |
270+
| `description` | Supported |
271+
| Dimensions (string, number, time, boolean) | Supported (`{model}` mapped back to Cube types) |
272+
| `primary_key: true` on dimensions | Supported |
273+
| `format` on dimensions | Supported |
274+
| Standard measures (count, count_distinct, sum, avg, min, max) | Supported |
275+
| Derived measures (`type: number`) | Supported |
276+
| Ratio metrics | Supported (exported as `type: number` with `${numerator}::float / NULLIF(${denominator}, 0)`) |
277+
| Cumulative metrics | Supported (exported with `rolling_window: { trailing }`) |
278+
| Time comparison metrics | Partial support: exported as `type: number` with a description annotation; no `time_shift` block generated. |
279+
| Measures with filters | Supported (exported as `filters: [{ sql }]`) |
280+
| `format` on measures | Supported |
281+
| `drill_members` on measures | Supported (exported from hierarchy dimensions when available) |
282+
| Segments | Supported (`{model}` replaced back with `${CUBE}`) |
283+
| Joins (many_to_one) | Supported (generates `sql` join expression from foreign key and primary key) |
284+
| Joins (one_to_many, one_to_one) | Partial support: only `many_to_one` relationships are exported as joins. Other relationship types are omitted. |
285+
| Pre-aggregations | Unsupported (not exported) |
286+
| Model inheritance resolution | Supported (inheritance resolved before export) |
287+
| Roundtrip fidelity (Cube -> parse -> export -> re-parse) | Supported for dimensions, metrics, and segments. Relationships are not fully round-tripped due to the export limitation above. |
288+

0 commit comments

Comments
 (0)