Introduce package metadata resolution state tracking - #6794
Conversation
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
|
@sahibamittal FYI. Please have a look when you get the time. |
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | 17 |
🟢 Coverage 98.13% diff coverage · +0.06% coverage variation
Metric Results Coverage variation ✅ +0.06% coverage variation (-1.00%) Diff coverage ✅ 98.13% diff coverage (70.00%) Coverage variation details
Coverable lines Covered lines Coverage Common ancestor commit (c910194) 43434 37778 86.98% Head commit (391371b) 43563 (+129) 37917 (+139) 87.04% (+0.06%) Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch:
<coverage of head commit> - <coverage of common ancestor commit>Diff coverage details
Coverable lines Covered lines Diff coverage Pull request (#6794) 160 157 98.13% Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified:
<covered lines added or modified>/<coverable lines added or modified> * 100%
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
* Adds a new `PACKAGE_METADATA_RESOLUTION` table that tracks when a given PURL was last resolved, and with what outcome. * Drives the resolution candidate query from the new table instead of the full `COMPONENT` table, significantly reducing query cost. This also allows permanently skipping resolution for PURLs that are unresolvable, e.g. because they're malformed. * Uses a cursor within a single `resolve-package-metadata` workflow run to avoid having to re-scan the full `PACKAGE_METADATA_RESOLUTION` table from the beginning for every batch. * Enables HOT updates on the `PACKAGE_METADATA` and `PACKAGE_ARTIFACT_METADATA` tables by dropping the now-unused indexes on `RESOLVED_AT` columns. See the included ADR for more details, and alternative options that were considered. Co-authored-by: Sahiba Mittal <sahiba.mittal@citi.com> Signed-off-by: nscuro <nscuro@protonmail.com>
5cf84a6 to
391371b
Compare
2 participants
Description
Introduces package metadata resolution state tracking:
PACKAGE_METADATA_RESOLUTIONtable that tracks when a given PURL was last resolved, and with what outcome.COMPONENTtable, significantly reducing query cost. This also allows permanently skipping resolution for PURLs that are unresolvable, e.g. because they're malformed.resolve-package-metadataworkflow run to avoid having to re-scan the fullPACKAGE_METADATA_RESOLUTIONtable from the beginning for every batch.PACKAGE_METADATAandPACKAGE_ARTIFACT_METADATAtables by dropping the now-unused indexes onRESOLVED_ATcolumns.Addressed Issue
Fixes #6654
Relates to #6771
Additional Details
See the included ADR for more details, and alternative options that were considered.
Checklist
This PR introduces new or alters existing behavior, and I have updated the documentation accordinglydocs/adr/