feat: support nested offsets#9899
Conversation
Allow xOffset and yOffset to define ordered arrays of discrete offset levels with an optional continuous quantitative leaf. Compile indexed scales and recursive Vega offset references, including ranged baselines, sizing, transforms, invalid handling, and stack grouping.
Compute descendant-aware steps for discrete offset scales instead of deriving child ranges from a parent bandwidth in step-sized layouts. This removes the implicit width and height dataflow cycle while preserving numeric and explicit-step sizing.
Respect explicit scale: null definitions when creating indexed scale components so direct color and fill encodings do not gain default palettes or legends. Add scale-level and end-to-end regressions for disabled color scales.
Deploying vega-lite with
|
| Latest commit: |
4eba8f4
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://8b963215.vega-lite.pages.dev |
| Branch Preview URL: | https://feat-nested-offsets.vega-lite.pages.dev |
Prevent x2 and y2 domain contributions from supplying their own default sort to the shared primary scale. This restores explicit sort: null behavior and the original waterfall and nonlinear histogram layouts while leaving nested offset sorting unchanged.
joelostblom
left a comment
joelostblom
left a comment
There was a problem hiding this comment.
I resolved a couple of regressions in the examples and added a few line comments
| export const SCALE_CHANNELS = keys(SCALE_CHANNEL_INDEX); | ||
| export type ScaleChannel = (typeof SCALE_CHANNELS)[number]; | ||
|
|
||
| export type OffsetScaleKey = OffsetScaleChannel | `${OffsetScaleChannel}_${number}`; |
There was a problem hiding this comment.
Indexed offset scale keys are compiler-only. Level zero retains the original channel name so scalar offsets continue producing identical Vega
| } | ||
|
|
||
| if ( | ||
| if (isXorYOffset(channel) && isArray(channelDef)) { |
There was a problem hiding this comment.
Offset arrays are ordered outermost to innermost. Every level that contains another level must provide a discrete scale with bandwidth.
| bandPosition, | ||
| }); | ||
| const defs = getOffsetDefs(encoding, channel); | ||
| const ref = defs.reduceRight<VgValueRef>((child, def, index) => { |
There was a problem hiding this comment.
Vega offset references are recursively nested. Build them inside-out so each outer reference can attach the previously constructed child reference.
| const isFinal = index === defs.length - 1; | ||
| const current = midPoint({ | ||
| channel, | ||
| channelDef: isFinal && finalOffsetDatum !== undefined ? {datum: finalOffsetDatum} : def, |
There was a problem hiding this comment.
Ranged marks replace only the quantitative leaf with zero while preserving all categorical ancestors in the baseline reference.
| } | ||
|
|
||
| function getOffsetStep(step: Step, offsetScaleType: ScaleType) { | ||
| function getOffsetStep(step: Step, offsetScaleType: ScaleType, key: ScaleKey, model: UnitModel) { |
There was a problem hiding this comment.
The base band must contain the complete bandspace of every discrete offset level. Account for intermediate padding so each parent's bandwidth, rather than its step, equals the span required by its children.
| } | ||
|
|
||
| const paddingInner = mergedScaleCmpt.get('paddingInner') ?? mergedScaleCmpt.get('padding'); | ||
| const discreteOffsetKeys = offsetDefs.filter((key) => hasDiscreteDomain(model.getScaleType(key))); |
There was a problem hiding this comment.
The base band must contain the complete bandspace of every discrete offset level. Account for intermediate padding so each parent's bandwidth, rather than its step, equals the span required by its children.
| let specifiedScale = fieldOrDatumDef && (fieldOrDatumDef as any).scale; | ||
| if (fieldOrDatumDef && specifiedScale !== null && specifiedScale !== false) { | ||
| let specifiedScale = fieldOrDatumDef && model.specifiedScale(key); | ||
| if (fieldOrDatumDef && (fieldOrDatumDef as {scale?: unknown}).scale !== null && specifiedScale !== null) { |
There was a problem hiding this comment.
initScales normalizes an unspecified scale to {}, so consult the encoding definition as well to preserve an explicit scale: null.
|
|
||
| const dimensionChannel: 'x' | 'y' | 'theta' | 'radius' = getDimensionChannel(fieldChannel); | ||
| const groupbyChannels: StackProperties['groupbyChannels'] = []; | ||
| const groupbyFieldDefs: FieldDef<string>[] = []; |
There was a problem hiding this comment.
A channel name identifies only the offset family, so retain the individual field definitions to group stacks by every nested offset level.
|
This is super cool! I like the designs this enables. I like this better than the Also I'm wondering about the discrete → discrete → ... → optional quantitative pattern. The use of an array suggests that all the elements should have the same type. So this suggests to me that maybe the innermost offset should use a different channel than the outer ones. Also feels like the outer ones are much closer to facets (although they don't treat the coordinate spaces the same way as facets). Another restriction I notice here is that all the offsets must be in the same direction so you can't mix x and y offsets in a chain. I'm wondering if you had a reason for that restriction? Or if it might be possible or desirable to lift it? I'm not sure what the syntax would look like in that case. |
|
By the way, not sure if this is the right PR to mention this, but I did notice that we can already do basic violin plots just by setting "stack": "center" on the existing faceted density plot example. 
Open the Chart in the Vega Editor {
"$schema": "https://vega.github.io/schema/vega-lite/v6.json",
"title": "Distribution of Body Mass of Penguins",
"width": 400,
"height": 80,
"data": {"url": "data/penguins.json"},
"mark": "area",
"transform": [
{"density": "Body Mass (g)", "groupby": ["Species"], "extent": [2500, 6500]}
],
"encoding": {
"x": {"field": "value", "type": "quantitative", "title": "Body Mass (g)"},
"y": {"field": "density", "type": "quantitative", "stack": "center"},
"row": {"field": "Species"}
}
}However, stacking is ignored on area charts with aggregation, so introducing an "aggregate": "sum" to the spec breaks it even though it is effectively a no-op. 
Open the Chart in the Vega Editor {
"$schema": "https://vega.github.io/schema/vega-lite/v6.json",
"title": "Distribution of Body Mass of Penguins",
"width": 400,
"height": 80,
"data": {
"url": "data/penguins.json"
},
"mark": "area",
"transform": [
{
"density": "Body Mass (g)",
"groupby": ["Species"],
"extent": [2500, 6500]
}
],
"encoding": {
"x": {"field": "value", "type": "quantitative", "title": "Body Mass (g)"},
"y": {"field": "density", "aggregate": "sum", "type": "quantitative", "stack": "center"},
"row": {"field": "Species"}
}
} |
I'm glad you like it =) I also prefer this route to the
The array is intended as ordered composition, not as a collection of independent encodings. Every element is an OffsetDef that maps into its parent interval. Categorical levels provide bandwidth and may therefore contain another level; a continuous level maps to a point and must terminate the chain. I considered a separate quantitative-leaf channel, but that would tie the API to a fixed depth and distinguish scale types at the channel level, ie the last element in the offset array is still an offset channel/encoding, it just allows for a quantitative type of this encoding. I think we could do something like this with a recursive object, but I personally prefer the syntax of the current ordered array: {
"yOffset": {
"field": "Sex",
"type": "nominal",
"offset": {
"field": "jitter",
"type": "quantitative"
}
}
}
That restriction is intentional because nesting is based on bandwidth. An Since it is possible to specify both {
"xOffset": [
{"field": "columnGroup", "type": "nominal"},
{"field": "xJitter", "type": "quantitative"}
],
"yOffset": [
{"field": "rowGroup", "type": "nominal"},
{"field": "yJitter", "type": "quantitative"}
]
}
Yes, we actually have an example using that approach in the altair gallery, but it becomes tricky that it doesn't compose the same way as other marks such as boxplots etc. That stacking approach was part of the inspiration for #9897 though! |
|
Thanks for the detailed replies. I built this branch locally and ran test specs against it and against the current release to check my intuitions. I tried to come up with an example where I'd need an ordering across x and y, but I couldn't do it, so I'm gonna drop that.
I wasn't interpreting it as an unordered collection but as an ordered collection. I also agree that the nested object notation is very hard to read. However, I tend to think of arrays as always holding an arbitrary number of (usually ordered) things of the same type, while objects/tuples hold a fixed collection of (usually unordered) things of different types. So when I saw Just pattern matching on the existing spec I'd almost want to write {
"data": {"url": "data/penguins.json"},
"transform": [
{"filter": {"field": "Sex", "oneOf": ["MALE", "FEMALE"]}},
{"calculate": "random()", "as": "jitter"}
],
"mark": {"type": "point", "filled": true, "opacity": 0.7},
"encoding": {
"row": [{"field": "Species"}, {"field": "Sex"}],
"x": {"field": "Body Mass (g)", "type": "quantitative"},
"y": {"field": "jitter", "type": "quantitative", "axis": null},
"color": {"field": "Sex", "type": "nominal"}
}
}Vega already has nested facet support, and you can get two levels today in VL with an outer "facet" and inner "row" field. I think it still makes sense to have the offset channels, but I'm not sure if they need the array or not. I think this design covers all the same cases? One tradeoff of this approach is that you don't get the more compressed look from these PRs automatically right now. I wonder if we could set defaults and/or expose customizations that would make it straightforward to achieve that look. |
|
Thanks for taking the time to test this out locally and provide more comments!
I see facets as complementary to nested offsets, rather than being able to fill the same purpose. They are of course related in that they allow for groupings according to categorical variables, but I think a facet is a much more distinct grouping that partition data into separate coordinate spaces that e.g. cannot be connected, have machinery around axis decorations, and could have independent scales. Since separating into facets makes the groups visually further apart, I think it can make comparisons either harder or easier depending on the complexity of the chart and what is being compared. To me, offsets are more akin to what other libraries call "dodge" in ggplot/seaborn and (I think) "groupX/Y" in observable; ie a less formal way of placing marks right next to each other to facilitate direct comparisons. I do agree that using row/col would avoid ambiguity in the sense that a facet is always a categorical grouping and an offset is always categorical or quantitative, but I think more is lost than gained in this case and I wonder if we can clarify in other ways that the last position of a nested offset encoding is the only one that can be quantitative (see below for suggestions)
One immediate example that comes to mind is the recent issue #9870 where a line mark needed to connect points across an ordinal domain (whereas connected categorical points with a line often is ill-advised, I think an ordinal domain is a good example of where it can make sense). This would not be possible if marks are separated across facets. Another issue is that the spec distance between two similar visual outputs would be larger than expected. Let's say I initially create with a chart that like this where points where offset by sex: After making the chart, I realize that it is hard to see all the individual observations and therefore I want to add jitter to the points to create this chart: It would be unexpected to me to find out that this chart is impossible to create and that I instead need to rework my spec to add row faceting and arrive at something like this (which would need a fair amount of styling to remove redundant labels, axis lines etc, as you noted): I think this issue is similar to how we currently could create violin plots via center stacking in separate facets, but it does not feel intuitive to do so. I think part of the enjoyment I get from using Vega-Lite is that transitions between similar charts often feel intuitive and that when I expect something to work, it often does. To me, the nested offset channels preserves that feeling more than requiring switching to a faceted spec. If offset channels were never introduced in the first place, I think there could be a stronger argument for staying with facets (although I still like the distinction from the fist point above), but currently offset channels work well for some types of marks and charts but not for others, which I think takes away from the intuitiveness of the grammar. As an alternative to nesting offsets, I did initially consider a different mechanism such as using the
All good points, I wonder if it would be sufficient to clarify this in the code and docs by referring to it as an ordered heterogeneous tuple/array? We could reflect this in the TypeScript naming and documentation, and validate that all non-final definitions are discrete. Something like this conceptually: I think this is one option that could give us the structural distinction you are describing while preserving the incremental syntax evolution when going from one to two levels of offsets, which I think is intuitive to write and aligned with how the {
"yOffset": {
"field": "Sex",
"type": "nominal"
}
}{
"yOffset": [
{
"field": "Sex",
"type": "nominal"
},
{
"field": "jitter",
"type": "quantitative"
}
]
} |
|
Thanks for your detailed rebuttal. You've convinced me that the yOffset array is a better approach than the row array! I agree with you about the spec distance consideration and connecting marks across facets. (Though I do think that connecting categorical points is sometimes useful for, e.g., parallel coordinate plots.) What if we call it |
2 participants
Offset channels currently accept only a single field, so they can either subdivide a band categorically or position values quantitatively, but not both. This means that they can only be used to group marks that don't require to use the offset channel for a quantitative position, e.g. a jitter/stripchart in an offset since the the offset channel is occupied by the random jitter. This restriction also applies to any other mark that relies on the offset encoding for a quantitative range/position, such as areas/densities/violins, bars, lines, etc.
I think the functionality to group these marks would be useful, so this PR introduces arbitrary depth/nesting to offsets, which can be specified as an ordered array of fields to an
xOffset/yOffsetencoding (similar to how thedetailencoding can take an array of fields). Nested offsets make it possible to subdivide a positional band categorically and then apply another categorical or quantitative offset within that subdivision.{ "y": {"field": "Species", "type": "nominal"}, "yOffset": [ {"field": "Sex", "type": "nominal"}, {"field": "jitter", "type": "quantitative"} ] }This allows to apply a categorical offset grouping to all marks, including jittered points:
Open the Chart in the Vega Editor
A few notes
Violin plots
This PR, together with the other improvements to ranged marks in offset channels (particularly #9897) offers a more generalized mechanism that also covers the specific case of creating violin plots in a way that's compatible with categorical offset grouping. Due to its more general form and support for other marks, it might be a preferable alternative to #9883.
Details
discrete → discrete → ... → optional quantitative
{ "xOffset": {"field": "group", "type": "nominal"} }{ "xOffset": [ {"field": "region", "type": "nominal"}, {"field": "sex", "type": "nominal"}, {"field": "treatment", "type": "nominal"}, {"field": "jitter", "type": "quantitative"} ] }Compiler Changes
yOffset,yOffset_1,yOffset_2, etcPath Marks