Skip to content

Add ability to assert metadata properties on input dataset parameters - #15825

Merged
mvdbeek merged 4 commits into
galaxyproject:devfrom
bernt-matthias:validation_metadata
Jul 29, 2023
Merged

Add ability to assert metadata properties on input dataset parameters#15825
mvdbeek merged 4 commits into
galaxyproject:devfrom
bernt-matthias:validation_metadata

Conversation

@bernt-matthias

@bernt-matthias bernt-matthias commented Mar 19, 2023

Copy link
Copy Markdown
Contributor

revive #3159 from @jmchilton (anno 2016 .. just a little bit more than crazy 10K PRs/issues in between :))

Changed the name for the validator .. dataset prefix might be good for comparability.

Not sure if one could / should generalize / reuse dataset_metadata_in_range (which considers only numbers at the moment).

ping @bgruening

Side note: I would find this super userful to check for column names, but tabular does not set them .. still downstream tools could and et voila it can be used :).

How to test the changes?

(Select all options that apply)

  • I've included appropriate automated tests.
  • This is a refactoring of components with existing test coverage.
  • Instructions for manual testing are as follows:
    1. [add testing steps and prerequisites here if you didn't write automated tests covering all your changes]

License

  • I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

@github-actions github-actions Bot added this to the 23.1 milestone Mar 19, 2023
@bernt-matthias
bernt-matthias force-pushed the validation_metadata branch 2 times, most recently from cb03665 to b866ed3 Compare March 20, 2023 09:07
in_range=InRangeValidator,
length=LengthValidator,
metadata=MetadataValidator,
dataset_metadata_equal=MetadataEqualsValidator,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe metadata_equal for consistency with the above type?

What about stronger typing. I feel like I've been heading this way with tool stuff... if value is used treat it as a string and use a value_json attribute to json.load the value for a typed comparison.

I would make these two changes but you write more tools - if you prefer the way you have it here that is totally fine with me.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe metadata_equal for consistency with the above type?

Hmm. We also have dataset_metadata_... maybe we can simplify/unify all this by adding a few aliases and name all of them metadata_...?

What about stronger typing. I feel like I've been heading this way with tool stuff... if value is used treat it as a string and use a value_json attribute to json.load the value for a typed comparison.

value_json is a great idea. I vaguely remembered that I have seen something like this but was not sure. I will add it.

I would make these two changes but you write more tools - if you prefer the way you have it here that is totally fine with me.

Thanks for the feedback. Give me a moment :)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While thinking about this I would suggest to stick with dataset_metadata_equal: All the user (ie tool developer) facing validators are prefixed with dataset_metadata_. The metadata validator is unlikely to be used by tool developers (its automatically added to each data parameter). So maybe this is an acceptable level of inconsistency...

Comment thread test/functional/tools/metadata_check_eq.xml Outdated
Comment thread lib/galaxy/tool_util/linters/inputs.py Outdated
@mvdbeek mvdbeek modified the milestones: 23.1, 23.2 Jun 21, 2023
@mvdbeek

mvdbeek commented Jun 21, 2023

Copy link
Copy Markdown
Member

This needs a pass of black.

@bernt-matthias

Copy link
Copy Markdown
Contributor Author

Good to go?

@mvdbeek
mvdbeek merged commit fa64fc6 into galaxyproject:dev Jul 29, 2023
@github-actions

Copy link
Copy Markdown

This PR was merged without a "kind/" label, please correct.

@bernt-matthias
bernt-matthias deleted the validation_metadata branch July 29, 2023 10:01
super().validate(isinstance(value, model.DatasetInstance) and not missing, value_to_show=missing)


class MetadataEqualsValidator(Validator):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm late, but maybe MetadataEqualValidator instead of MetadataEqualsValidator for consistency?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not to late #16489 :)

Thanks!

bernt-matthias added a commit to bernt-matthias/galaxy that referenced this pull request Jul 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants