feat(reasoning): optional System 2 causal head behind a typed SceneContext#81
Merged
m-zain-khawaja merged 5 commits intoJun 22, 2026
Conversation
Adds Model/model_components/causal_reasoning.py (CausalReasoningModule: reasoning_latent [B,H] + decision_logits [B,5] over a closed set intersection/pedestrian/traffic_light/obstacle/clear; reason() exposes the latent to condition the planner in a perception->reasoning->trajectory cascade with gradients flowing to the trunk, no torch.no_grad). Includes causal_consistency_loss (cross-entropy on VLM/KITScenes pseudo-labels). Optional and independent: AutoE2E's 3-tuple contract is untouched. Relates to autowarefoundation#17. Signed-off-by: GABRIELA CORDOVA <100548769@alumnos.uc3m.es>
…s for noisy pseudo-labels Signed-off-by: GABRIELA CORDOVA <100548769@alumnos.uc3m.es>
…l producer Signed-off-by: GABRIELA CORDOVA <100548769@alumnos.uc3m.es>
…anner untouched Per maintainer feedback (autowarefoundation#56): keep the causal head as an optional producer of a typed SceneContext, but do NOT modify the shared BasePlanner interface. Revert trajectory_planning/base.py to upstream (drop the scene_context parameter): planners already accept **kwargs, so any planner that wants to consume the context reads it from kwargs — no abstract-signature change, no forced update on GRU/FlowMatching. Fix an infer-mode assertion in the integration test (future is None in infer, per the AutoE2E 3-tuple contract). Signed-off-by: GABRIELA CORDOVA <100548769@alumnos.uc3m.es>
mypy (no_implicit_optional=True) rejected implicit-Optional defaults: - CausalReasoningModule.__init__: latent_dim: int = None -> int | None - causal_consistency_loss: class_weights: torch.Tensor = None -> torch.Tensor | None Signed-off-by: GABRIELA CORDOVA <100548769@alumnos.uc3m.es>
gcordova10
force-pushed
the
feat/system2-causal-head
branch
from
6282611 to
fd1091a
Compare
gcordova10
changed the title
m-zain-khawaja
approved these changes
Jun 22, 2026
m-zain-khawaja
left a comment
m-zain-khawaja
left a comment
Member
There was a problem hiding this comment.
Approved, thanks @gcordova10
gcordova10
added a commit
to gcordova10/auto_fsd
that referenced
this pull request
Jun 24, 2026
… head (autowarefoundation#17/autowarefoundation#3) The merged causal head (autowarefoundation#81) trains via causal_consistency_loss(decision_logits, labels) but the labels — 'VLM pseudo-labels over KITScenes LongTail' per its docstring — did not exist. This adds that supervision side, following NVIDIA's Chain-of-Causation autolabeler (NVlabs/alpamayo-coc-autolabeler; Alpamayo-R1 arXiv:2511.00088): - VLMTeacher interface (real Qwen3-VL/GPT backends plug in by subclassing) + a deterministic KeywordVLMTeacher for offline use & CI; - text_to_class_index: maps free-form effect_on_ego_behavior / captions to the 5 CAUSAL_CLASSES by priority (most safety-critical first); - parse_coc_yaml: reads the autolabeler's cot_*.yaml schema into a class index; - build_causal_labels: builds the [B] label tensor (with an abstain index). Taxonomy imported from the head (single source of truth). Additive, new package under data_parsing; 14 tests incl. end-to-end driving the merged causal_consistency_loss; mypy/ruff clean. Note: full-suite CI gated by autowarefoundation#88 (broken shared test infra), unrelated. Signed-off-by: GABRIELA CORDOVA <100548769@alumnos.uc3m.es>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements #17 / the typed-context reframing @ryotayamada asked for in #56: typed SceneContext (confidence+provenance), causal head as an OPTIONAL producer, passed to planners via **kwargs — BasePlanner interface untouched. Labels via KIT longtail (pending). Relates to #56.