Exclude volatile per-request fields from Jobs crash blacklist#2673
Draft
MelvinBot wants to merge 4 commits into
Draft
Exclude volatile per-request fields from Jobs crash blacklist#2673MelvinBot wants to merge 4 commits into
MelvinBot wants to merge 4 commits into
Conversation
The Jobs plugin populated crashIdentifyingValues with every request field, including requestID (unique per request). Since _wouldCrash requires an exact match on every stored key/value, the blacklist entry could never match a future command, making the poison-pill protection a no-op for Jobs commands. Exclude volatile per-request fields (requestID, ID, lastIP, _source) so the blacklist keys on semantically meaningful fields (e.g. job name and data). Co-authored-by: Carlos Alvarez <cead22@users.noreply.github.com>
cead22
requested changes
Jul 8, 2026
Extract the crashIdentifyingValues population into a testable helper (BedrockJobsCommand::populateCrashIdentifyingValues) and add CrashIdentifyingValuesTest covering that volatile per-request fields (requestID, ID, lastIP, _source) are excluded while semantically meaningful fields are retained. Co-authored-by: Carlos Alvarez <cead22@users.noreply.github.com>
Switch crashIdentifyingValues from a blacklist of volatile fields to an explicit whitelist of the semantically identifying fields (name, data). This makes BedrockServer::_wouldCrash's "more than one identifying set -> block everyone" safeguard behave correctly: the same command crashing twice now stays on a single blacklist entry (blocking only that command), while two genuinely different commands crashing trips the block-everyone path (we've already lost two nodes). A blacklist that left in non- identifying fields like jobID/priority would have wrongly tripped that safeguard on repeats of the same command. Co-authored-by: Carlos Alvarez <cead22@users.noreply.github.com>
tylerkaraszewski
requested changes
Jul 14, 2026
| for (const auto& name : request.nameValueMap) { | ||
| crashIdentifyingValues.insert(name.first); | ||
| } | ||
| populateCrashIdentifyingValues(); |
Contributor
There was a problem hiding this comment.
Instead of doing the AI thing where it adds 50 lines to make a 3 line change, I would simply change the loop removed here with:
crashIdentifyingValues.insert("name");
crashIdentifyingValues.insert("data");
No changes to the .h file, -3/+2 here.
| @@ -0,0 +1,74 @@ | |||
| #include <libstuff/SData.h> | |||
Contributor
There was a problem hiding this comment.
I find this entire test quite useless. It inserts two items in a map and then spends 74 lines asserting that the map contains those two items. We already know that maps work without this test class.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Details
The Jobs plugin's crash-command blacklist ("poison-pill" protection) was effectively broken. In
BedrockJobsCommand::peek,crashIdentifyingValueswas populated with every request field:That set included
requestID, which is unique for every request. SinceBedrockServer::_wouldCrashrequires an exact match on every stored key/value (skipping onlyContent-Length), a single blacklist entry could never match a future command — no two requests share arequestID.This change keys
crashIdentifyingValueson an explicit whitelist of the fields that semantically identify a Jobs command —nameanddata— instead of a blacklist of volatile fields. This is the correct approach given howBedrockServer::_wouldCrashhandles multiple crashes of the samemethodLine:_wouldCrashassumes we've already lost more than one node and blocks the command for everyone.name/data, the same command crashing twice records the same identifying set (one entry → only that command is blocked), while two genuinely different commands crashing records two sets → the block-everyone safeguard fires as intended.jobID,priority) would have recorded a different set on each crash of the same command and wrongly tripped the block-everyone path.Fixed Issues
https://github.com/Expensify/Expensify/issues/656214
Tests
test/tests/jobs/CrashIdentifyingValuesTest.cppto assert onlynameanddata(and their values) are kept and every other field is dropped, including present-but-non-identifying fields likejobID/priority.just docker rebuild-cpp— Bedrock + Auth compile cleanly with-Wall -Werror../test -only JobsCrashIdentifyingValues→ Passed: 2, Failed: 0../test -only CreateJob(server-based, exercises thepeek()path) → Passed: 17, Failed: 0.// TODO: The human co-author must fill out the tests you ran before marking this PR as "ready for review".
Internal Testing Reminder: when changing bedrock, please compile auth against your new changes