Cargo workspace support

[LebedevRI]

Currently, cargo-fuzz implements manual parser for Cargo.toml,
which means that only a very specific project structure is supported.
Namely, this only supports either one of:

1. Explicitly specified project directory
2. A mono-project in "<workspace root>"/fuzz
   ... where "" is found arbitrairly.

My biggest problem with this, is that if i want to have
a workspace-style layout for fuzz project,
i.e. /fuzz/Cargo.toml isn't a project,
but /fuzz/foo/Cargo.toml and /fuzz/bar/Cargo.toml are,
current cargo-fuzz simply doesn't support this.

The overall solution is to stop parsing Cargo.toml manually,
and rely on cargo metadata, via existing cargo-metadata dep.

I've tried very hard to preserve existing functionality,
but i'm not sure how good the test coverage is.

Now, let's mention fuzz artifact dirs location (corpus, etc).
Currently, they will still all be located in <workspace root>,
but with a caveat: if the <workspace root>/fuzz
is part of <workspace root> project,
then they will no longer be stored in fuzz subdirectory.

Additonally, i'm wondering if this is the right default anyway,
should they be stored in, <fuzz project root> of each target?
Incindentally, this would restore the previous default.

Thoughts?

Fixes #430.

[LebedevRI]

Now with basic tests. The feedback i would love to hear is:

1. Is there any interest in this at all?
2. Where should the fuzz artifact dirs be located, in each project's dir, or <workspace root>? After thinking about this, personally i'd think they should be per-project.

[fitzgen]

My gut reaction is that if you want project/fuzz/foo/Cargo.toml and project/fuzz/bar/Cargo.toml to be two different fuzz workspaces, then either

• you should cd into the relevant directory before running cargo fuzz ... (and cargo fuzz should determine that the current dir is a fuzz crate via the metadata entry in the current dir's Cargo.toml, if it doesn't already)
• you should use cargo fuzz ... --fuzz-dir project/fuzz/{foo,bar} ... to tell cargo fuzz which fuzz project you intend to run

I don't think cargo fuzz should automatically search for and operate upon multiple fuzz projects at the same time.

[LebedevRI]

Thank you for taking a look!

My gut reaction is that if you want project/fuzz/foo/Cargo.toml and project/fuzz/bar/Cargo.toml to be two different fuzz workspaces, then either

I'm sorry, i'm not quite following. By fuzz workspace, what exactly do you mean? The terms are a bit overloaded.
I have a root Cargo workspace with multiple sub-projects that could theoretically be a single project,
but are not because i feel like more granular projects is best.

And now i want to add fuzzing for each one of them.
Right now i would be forced to have a single gigantic fuzz project
that pulls in each crate, which defeats their granularity.

• you should cd into the relevant directory before running cargo fuzz ... (and cargo fuzz should determine that the current dir is a fuzz crate via the metadata entry in the current dir's Cargo.toml, if it doesn't already)
• you should use cargo fuzz ... --fuzz-dir project/fuzz/{foo,bar} ... to tell cargo fuzz which fuzz project you intend to run

How do i build all fuzz targets in all projects in this workflow?
I'm sorry, this really doesn't work for me. cargo fuzz is a tool,
it should not dictate how one lays out the cargo projects.

I don't think cargo fuzz should automatically search for and operate upon multiple fuzz projects at the same time.

Err, this feedback is a bit unactionable. I don't quite understand why this is unacceptable behaviour.

[LebedevRI]

(just to be explicit: this is not about different workspaces, this is about supporting different projects within a single cargo workspace. Whether or not the <root workspace>/fuzz is it's own workspace is a different matter, that i don't think changes the equasion much...)

[LebedevRI]

Hi. Is there perhaps a better place to have a disscussion about this?

I don't claim the diff as is to be the final correct approach,
but i hope that we at least agree that cargo fuzz build
should actually build everything when run from the root of the cargo workspace?

This change didn't come out of boredom,
i actually hope that cargo-fuzz-proper would support cargo workspaces.

[LebedevRI]

I've spent some more thought on this, and ended up completely ripping out
automatic --workspace/--package insertion logic and deferring that to the user.
This is actually closer to what i initially had in mind.

Does this look more reasonable? :) Any other suggestions?

[LebedevRI]

@fitzgen kindly, poke
I'm mainly wondering if cargo workspace support is simply intentionally WONTFIX,
or there's typical review bandwidth limitations and i should just wait.
Thank you!

[fitzgen]

IMO workspaces are supported just fine today (you can put multiple fuzz crates in a workspace or have them in their own workspace if you prefer) and I think making the search for the fuzz crate to operate upon more complicated isn't really worth it (today's algorithm of either taking an explicit path or walking up towards the root from the current dir is in a pretty sweet spot where it is both simple and effective).

[LebedevRI]

Thanks for taking a look!

IMO workspaces are supported just fine today (you can put multiple fuzz crates in a workspace or have them in their own workspace if you prefer)

Perhaps there is a miscommunication going on here.
Do you understand the problem i'm trying to solve here?
With current support level, how do i make cargo-fuzz build all fuzz crates in a workspace,
given that it only operates on a single fuzz crate at once?

and I think making the search for the fuzz crate to operate upon more complicated isn't really worth it (today's algorithm of either taking an explicit path or walking up towards the root from the current dir is in a pretty sweet spot where it is both simple and effective).

[LebedevRI]

@rust-fuzz @fitzgen @Manishearth who "owns" this project?
Is there some better place to clear the ongoing miscommunication?

I think making the search for the fuzz crate to operate upon more complicated isn't really worth it (today's algorithm of either taking an explicit path or walking up towards the root from the current dir is in a pretty sweet spot where it is both simple and effective).

(I think this clearly states that cargo workspaces aren't supported by the current design...)

[LebedevRI]

Ping. This isn't really inventing anything new, this is roughtly how it's done by other cargo subcommand transplants.

[LebedevRI]

Gentle poke, thank you.

[Manishearth]

I'm really struggling to understand the problems you are trying to solve here. I think more concrete examples and saying precisely what you expect to work might help.

I have spent ten minutes looking at this thread and I do not understand what you have been trying to say, and I do not really have the time or motivation to engage in depth here.

but i hope that we at least agree that cargo fuzz build
should actually build everything when run from the root of the cargo workspace?

I don't actually think this is a property we particularly care about?