"Bad Status Server returned 404" when searching in https://search.nixos.org/(options/packages) with unstable channel

[kagurazaka-ayano]

Reproducible url: https://search.nixos.org/packages?channel=unstable&include_modular_service_options=1&include_nixos_options=1&query=users

Screenshot:

With 25.11 channel there are plenty:

I observed there is a 404 post request:

Request payload:

{"from":0,"size":50,"sort":[{"_score":"desc","option_name":"asc"}],"aggs":{"all":{"global":{},"aggregations":{}}},"query":{"bool":{"filter":[{"terms":{"type":["option","service"],"_name":"filter_option_service"}}],"must_not":[],"must":[{"dis_max":{"tie_breaker":0.7,"queries":[{"multi_match":{"type":"cross_fields","query":"users","analyzer":"whitespace","auto_generate_synonyms_phrase_query":false,"operator":"and","_name":"multi_match_users","fields":["option_name^6","option_name.*^3.5999999999999996","option_description^1","option_description.*^0.6","flake_name^0.5","flake_name.*^0.3","service_package^3","service_package.*^1.7999999999999998","service_packages^3","service_packages.*^1.7999999999999998"]}},{"wildcard":{"option_name":{"value":"*users*","case_insensitive":true}}}]}}]}}}

[TheRealGramdalf]

Also happening for me with https://search.nixos.org/packages?channel=unstable&include_modular_service_options=1&include_nixos_options=1&query=kanidm

{"from":0,"size":50,"sort":[{"_score":"desc","package_attr_name":"asc","package_pversion":"asc"}],"aggs":{"package_attr_set":{"terms":{"field":"package_attr_set","size":20}},"package_license_set":{"terms":{"field":"package_license_set","size":20}},"package_maintainers_set":{"terms":{"field":"package_maintainers_set","size":20}},"package_teams_set":{"terms":{"field":"package_teams_set","size":20}},"package_platforms":{"terms":{"field":"package_platforms","size":20,"include":["x86_64-linux","aarch64-linux","i686-linux","x86_64-darwin","aarch64-darwin"]}},"all":{"global":{},"aggregations":{"package_attr_set":{"terms":{"field":"package_attr_set","size":20}},"package_license_set":{"terms":{"field":"package_license_set","size":20}},"package_maintainers_set":{"terms":{"field":"package_maintainers_set","size":20}},"package_teams_set":{"terms":{"field":"package_teams_set","size":20}},"package_platforms":{"terms":{"field":"package_platforms","size":20,"include":["x86_64-linux","aarch64-linux","i686-linux","x86_64-darwin","aarch64-darwin"]}}}}},"query":{"bool":{"filter":[{"term":{"type":{"value":"package","_name":"filter_packages"}}},{"bool":{"must":[{"bool":{"should":[]}},{"bool":{"should":[]}},{"bool":{"should":[]}},{"bool":{"should":[]}},{"bool":{"should":[]}}]}}],"must_not":[],"must":[{"dis_max":{"tie_breaker":0.7,"queries":[{"multi_match":{"type":"cross_fields","query":"kanidm","analyzer":"whitespace","auto_generate_synonyms_phrase_query":false,"operator":"and","_name":"multi_match_kanidm","fields":["package_attr_name^9","package_attr_name.*^5.3999999999999995","package_programs^9","package_programs.*^5.3999999999999995","package_pname^6","package_pname.*^3.5999999999999996","package_description^1.3","package_description.*^0.78","package_longDescription^1","package_longDescription.*^0.6","flake_name^0.5","flake_name.*^0.3"]}},{"wildcard":{"package_attr_name":{"value":"*kanidm*","case_insensitive":true}}}]}}]}}}

Not just an isolated issue. Firefox 149.0.2 in case it's relevant, the search query can be anything but empty - if so, no packages/options load and no POST requests are sent:

[raboof]

Yes, this was reported in #1189 (comment) as well. #1196 should be a workaround for now.

The cause of the issue was that we didn't notice there weren't any successful imports of the latest-45-nixos-unstable index yet. The imports work as follows: there is an alias, latest-45-nixos-unstable, that points to the latest successful import. The import job looks if a git-hash-specific import (e.g. nixos-45-unstable-4c1018dae018162ec878d42fec712642d214fdfa) already exists. If so, it stops. If not, it creates it, populates it, and then points the alias to it.

In this case, the initial import of nixos-45-unstable-4c1018dae018162ec878d42fec712642d214fdfa had failed, and subsequent jobs didn't even attempt the import, and we didn't notice in manual testing - apparently we tested only with the 25.11 index.

The plan for fixing this is:

• fix: revert frontend schema version to 44 #1196
• manually remove nixos-45-unstable-4c1018dae018162ec878d42fec712642d214fdfa
• see if the next import succeeds (it did)
• Switch back to index version 45 #1200
• Fail import if warmup fails #1220 so we don't miss this so easily in the future
• automatically remove nixos-45-unstable-4c1018dae018162ec878d42fec712642d214fdfa again when its population fails, so it's automatically retried the next time.