feat(stack): add stack catalog generator and serve artifact#167
feat(stack): add stack catalog generator and serve artifact#167catrielmuller wants to merge 2 commits into
Conversation
Introduce `stack/catalog.yaml` as the curated source of truth mapping data-engineering technologies to marketplace skills and MCPs. Add `bin/generate-stack-catalog.ts`, which validates the catalog against the mcps/ and skills/ inventories, enforces the trust policy (only stable official skills and MCPs may be enabled by default), auto-fills association metadata from the resources registry, stamps `curated:true`, and writes the serve artifact to `stack/marketplace.json`. Wire the generator into the marketplace regeneration workflow so the stack artifact is produced alongside the existing marketplace.yaml files and committed by the bot on regeneration.
![kilo-code-bot[bot]](https://avatars.githubusercontent.com/in/2193792?s=60&v=4){kind=small}
Code Review SummaryStatus: No Issues Found | Recommendation: Merge The previously flagged trust-policy WARNING ( Files Reviewed (3 files in incremental diff)
Previous Review Summary (commit 237bab4)Current summary above is authoritative. Previous snapshots are kept for context only. Previous review (commit 237bab4)Status: 1 Issue Found | Recommendation: Address before merge Overview
Issue Details (click to expand)WARNING
Files Reviewed (4 files)
Reviewed by claude-sonnet-5 · Input: 60 · Output: 9.9K · Cached: 1.4M Review guidance: REVIEW.md from base branch |
markijbema
left a comment
markijbema
left a comment
There was a problem hiding this comment.
I think it would be better to integrate it more tightly with the existing skills etc; I think it's slightly confusing to have two skill/mcp etc repositories in this repo. That being said, I think this is a fine said and am not opposed to merging. My bot found some feedback which i present as is, but none of it is blocking imho
| // Check existence in marketplace inventories | ||
| if (res.kind === "mcp" && !mcpIds.has(res.id)) { | ||
| const msg = `resources.${res.ref}: MCP id "${res.id}" is not in mcps/` | ||
| if (STRICT) throw new Error(msg) |
There was a problem hiding this comment.
[LLM-generated comment]
F1 — KILO_MARKETPLACE_STRICT is never enabled in CI, so referential drift won't fail builds. In non-strict mode a missing MCP/skill ID only produces a WARN log line, and the workflow runs the generator without --check and without STRICT. GitHub Actions log warnings do not fail a job, so if a skill/MCP is later deleted or renamed while still referenced in catalog.yaml, CI will silently regenerate and commit an artifact pointing at a non-existent resource. Consider running with KILO_MARKETPLACE_STRICT=1 (and/or --check) in a validation step.
| run: pnpm exec tsx generate-mcps-marketplace.ts | ||
| working-directory: bin | ||
|
|
||
| - name: Generate stack catalog artifact |
There was a problem hiding this comment.
[LLM-generated comment]
F2 — No --check/validation gate on PRs that touch the stack files. This workflow self-heals by regenerating and committing, but there is no job that fails when catalog.yaml is malformed on a fork PR (where the bot cannot push). The throw inside the generate step is the only guard. A dedicated validation job (like validate-skills.yml) scoped to stack/**, running the generator with --check (and ideally KILO_MARKETPLACE_STRICT=1), would make failures explicit rather than depending on the auto-commit path.
| id: agent-platform-mcp | ||
| kind: mcp | ||
| name: Agent Platform MCP | ||
| trust: official |
There was a problem hiding this comment.
[LLM-generated comment]
F3 — Registry trust/maturity/source/name are hand-maintained and never cross-checked against the actual MCP.yaml/SKILL.md. The entire trust-policy enforcement trusts this resources: block; nothing verifies these values against source. Concrete drift already exists, e.g. registry name: AWS MCP Server vs mcps/aws/MCP.yaml name: AWS Managed MCP Server. IDs are validated, but trust/maturity/name/source are not. Consider documenting that this registry is the sole authority (accepting the drift risk) or adding a name/source cross-check against the resource files.
| } | ||
|
|
||
| // Auto-fill metadata from the resources registry if not provided in the source. | ||
| const trust = typeof a.trust === "string" ? a.trust : resource.trust |
There was a problem hiding this comment.
[LLM-generated comment]
F4 — Association-level trust/maturity/source/warnings overrides are unguarded and can bypass the trust invariant. The trust-policy check above uses the registry values (resource.trust/resource.maturity), but these lines emit the association override when present. So an entry with default: true plus trust: community would pass the check (registry is official) yet serialize as trust: community, default: true, contradicting the "only stable official may be default" invariant consumers may rely on. These overrides are also never validated against VALID_TRUST/VALID_MATURITY/HTTPS. Since the documented source schema is only ref + default (+ optional rationale/deprecated/replacement), consider dropping the override paths or validating them.
| inVertical.add(tech.id) | ||
| const assocRefs = new Set<string>() | ||
| for (let i = 0; i < tech.resources.length; i++) { | ||
| const a = validateAssociation(tech.resources[i], `verticals.${v.id}.technologies.${tech.id}.resources.${tech.resources[i].ref}`, registered) |
There was a problem hiding this comment.
[LLM-generated comment]
F5 — Malformed entries crash with a raw TypeError instead of a validation message. This loop reads tech.resources[i].ref to build the error path before validateAssociation confirms the entry is an object, and v.technologies/tech.resources/v.categories are never asserted to be arrays. A null/missing entry throws "Cannot read properties of null" rather than the nice ${path}: expected object. Low impact for a curated file, but inconsistent with the otherwise-careful validation.
| # | ||
| # Revision format: YYYY-MM-DD.N where N starts at 1 and increments each same-day update. | ||
|
|
||
| revision: "2026-06-25.2" |
There was a problem hiding this comment.
[LLM-generated comment]
F6 — revision was not bumped despite semantic changes. The header comment mandates YYYY-MM-DD.N incrementing on each same-day update, yet the second commit ("refactor: address comments") flipped many default: true → false while revision stayed 2026-06-25.2. Nothing enforces this, so the served revision no longer reflects the content.
| * unknown IDs are logged as warnings only. | ||
| */ | ||
|
|
||
| import * as fs from "fs" |
There was a problem hiding this comment.
[LLM-generated comment]
F7 — Style nit: no semicolons. This file omits semicolons while every sibling in bin/ (generate-*-marketplace.ts, marketplace-generator-utils.ts) uses them. There is no linter/formatter config in the repo to enforce either way, so this is purely cosmetic consistency.
| @@ -0,0 +1,6026 @@ | |||
| { | |||
There was a problem hiding this comment.
[LLM-generated comment]
F8 — Large generated artifact committed to the repo (non-blocking). This 6k-line file is fully derived from catalog.yaml and kept in sync by the bot. It matches the existing marketplace.yaml convention, so this is acceptable — noting only for completeness that it introduces a sync obligation.
2 participants
Summary
Introduces the Kilo Stack Catalog: a curated mapping of data-engineering technologies to marketplace skills and MCPs, plus a generator that validates it and produces a serve artifact.
stack/catalog.yaml— curated source of truth associating technologies (e.g. Apache Airflow, Dagster, Azure Data Factory) with qualified resource refs (skill:id/mcp:id). Associations only needref+default; everything else is auto-filled.bin/generate-stack-catalog.ts— validates the catalog against themcps/andskills/inventories, enforces the trust policy (only stable official skills and MCPs may bedefault: true), auto-fills association metadata (trust/maturity/source/warnings/rationale) from the resources registry, stampscurated: true, and writesstack/marketplace.json. Supports--check(CI staleness mode) andKILO_MARKETPLACE_STRICT(reject unknown IDs).stack/marketplace.json— generated serve artifact..github/workflows/generate-marketplace.yml— runs the stack catalog generator and includesstack/marketplace.jsonin the bot-committed regeneration artifacts.Validation
npx tsx bin/generate-stack-catalog.ts --check→stack/marketplace.json is up to date.