@@ -31,8 +31,12 @@ and optionally apply fixes.
3131 accepts both ` project_uid ` and ` committee_uid ` can only send one at a time.
3232- ` handleSearchPastMeetingParticipants ` and ` handleSearchPastMeetingSummaries `
3333 are dedicated handlers — each owns its own filter logic independently.
34- - When sampling documents, use ` "size": 3 ` to keep output small. Use
35- ` _source ` filtering to request only ` tags ` and ` parent_refs ` fields.
34+ - ** Prefer aggregation counts over random sampling.** A handful of random
35+ documents proves nothing — you can get lucky and see the right fields while
36+ 90% of the corpus has them missing. Always run ` "size": 0 ` prefix count
37+ queries first. Only pull sample documents (` "size": 3 ` ) as a secondary
38+ debugging aid when a count is zero or surprising (e.g. to understand what
39+ fields are actually present on that resource type).
3640
3741## Step 1 — Discover infrastructure
3842
@@ -64,19 +68,32 @@ Substitute the kubectl context as needed to target dev vs. prod.
6468
6569## Step 2 — Enumerate search tools and their filter mappings
6670
67- Read every ` *Args ` struct in ` internal/tools/ ` and record how each filter
71+ ** Do not rely solely on the reference table below — always grep the codebase
72+ first** to find every file that calls ` QueryResources ` . The table may be out of
73+ date if new tools have been added since it was last updated.
74+
75+ ``` bash
76+ grep -rn " QueryResources\|QueryResourcesPayload" internal/tools/ | grep -v " _test.go"
77+ ```
78+
79+ For each file that appears, read the handler and record how each filter
6880parameter is sent to the query service. The mechanisms are:
6981
7082| Mechanism | Query service field | Index field |
7183| ---| ---| ---|
7284| ` payload.Parent = "<type>:<uid>" ` | ` Parent ` | ` parent_refs ` |
7385| ` payload.Tags = ["<key>:<value>"] ` | ` Tags ` | ` tags ` |
7486| ` payload.Filters = ["<field>:<value>"] ` | ` Filters ` | top-level doc fields |
87+ | ` payload.FiltersAll = ["<field>:<value>"] ` | ` FiltersAll ` | top-level doc fields (AND semantics) |
7588| ` payload.Name = "<value>" ` | ` Name ` | ` name ` (text search) |
7689| ` payload.DateField ` / ` DateFrom ` / ` DateTo ` | date range | date fields |
7790
78- Current tools using the query service SDK and their structured filter parameters
79- (update this list if new tools are added):
91+ Only ` Parent ` , ` Tags ` , ` Filters ` , and ` FiltersAll ` are structural filters that
92+ map to indexed fields — these are the ones to validate. ` Name ` and date fields
93+ are query-time text/range operations and do not need index field verification.
94+
95+ Reference table of known tools and their structured filter parameters (verify
96+ against the grep output above before trusting this):
8097
8198| Tool | Resource type | Parameter | Mechanism | Sent as |
8299| ---| ---| ---| ---| ---|
@@ -98,8 +115,13 @@ Current tools using the query service SDK and their structured filter parameters
98115| ` search_past_meeting_participants ` | ` v1_past_meeting_participant ` | ` project_uid ` | Parent (fallback) | ` project:<uid> ` |
99116| ` search_past_meeting_summaries ` | ` v1_past_meeting_summary ` | ` past_meeting_id ` | Parent (preferred) | ` past_meeting:<meeting_and_occurrence_id> ` |
100117| ` search_past_meeting_summaries ` | ` v1_past_meeting_summary ` | ` project_uid ` | Parent (fallback) | ` project:<uid> ` |
101-
102- Re-read the handler code to verify this table is current before proceeding.
118+ | ` search_members ` | ` project_membership ` | ` project_uid ` | FiltersAll | ` project_uid:<uid> ` |
119+ | ` search_members ` | ` project_membership ` | ` b2b_org_uid ` | FiltersAll | ` b2b_org_uid:<uid> ` |
120+ | ` search_members ` | ` project_membership ` | ` tier_uid ` | FiltersAll | ` tier_uid:<uid> ` |
121+ | ` search_members ` | ` project_membership ` | ` tier_name ` | FiltersAll | ` tier_name:<name> ` |
122+ | ` search_members ` | ` project_membership ` | ` status ` | FiltersAll | ` status:Active ` (hardcoded default) |
123+ | ` get_membership_key_contacts ` | ` key_contact ` | ` membership_uid ` | FiltersAll | ` membership_uid:<uid> ` |
124+ | ` search_b2b_orgs ` | ` b2b_org ` | * (none — Name only)* | — | — |
103125
104126## Step 3 — Fetch indexer contracts
105127
@@ -126,82 +148,81 @@ for each resource type. Record which tag keys and parent_ref prefixes the
126148contract defines. If a URL 404s or has no contract doc, note it and continue —
127149treat those filters as "no contract definition" in the report.
128150
129- ## Step 4 — Sample the live index
151+ ## Step 4 — Count hits in the live index
130152
131- For each resource type, run the following queries via the NATS box. Use the
132- ` $NATS_POD ` and ` $OPENSEARCH_BASEURL ` variables set in Step 1.
153+ For each filter parameter, run an ** aggregation count query** (` "size": 0 ` )
154+ via the NATS box. This is the primary evidence step. Use the ` $NATS_POD ` and
155+ ` $OPENSEARCH_BASEURL ` variables set in Step 1.
133156
134- ** Sample tags and parent_refs from 3 recently indexed documents (last 45 days):**
157+ ** Count documents where a specific tag key has non-empty values (last 45 days):**
135158
136159``` bash
137160kubectl exec -n lfx " $NATS_POD " -- \
138161 curl -s --max-time 15 -X GET " $OPENSEARCH_BASEURL /_search" \
139162 -H ' Content-Type: application/json' \
140163 -d ' {
141- "size": 3,
142- "_source": ["tags", "parent_refs", "object_type"],
164+ "size": 0,
143165 "query": {
144166 "bool": {
145167 "must": [
146168 { "term": { "object_type": "<RESOURCE_TYPE>" } },
169+ { "prefix": { "tags": "<TAG_KEY>:" } },
147170 { "range": { "updated_at": { "gte": "now-45d" } } }
171+ ],
172+ "must_not": [
173+ { "term": { "tags": "<TAG_KEY>:" } }
148174 ]
149175 }
150176 }
151177 }'
152178```
153179
154- ** Check whether a specific tag key has any non-empty values (last 45 days):**
180+ ** Count documents where a specific parent_ref prefix exists (last 45 days):**
155181
156182``` bash
157183kubectl exec -n lfx " $NATS_POD " -- \
158184 curl -s --max-time 15 -X GET " $OPENSEARCH_BASEURL /_search" \
159185 -H ' Content-Type: application/json' \
160186 -d ' {
161- "size": 1,
162- "_source": ["tags"],
187+ "size": 0,
163188 "query": {
164189 "bool": {
165190 "must": [
166191 { "term": { "object_type": "<RESOURCE_TYPE>" } },
167- { "prefix": { "tags ": "<TAG_KEY >:" } },
192+ { "prefix": { "parent_refs ": "<PREFIX >:" } },
168193 { "range": { "updated_at": { "gte": "now-45d" } } }
169- ],
170- "must_not": [
171- { "term": { "tags": "<TAG_KEY>:" } }
172194 ]
173195 }
174196 }
175197 }'
176198```
177199
178- ** Check whether a specific parent_ref prefix exists (last 45 days):**
200+ Record the ` total.value ` from each response. A non-zero count confirms the
201+ key/prefix is present in recently indexed data. If the count is zero but the
202+ resource type has older data, note it as "not seen in last 45 days" rather
203+ than immediately marking it broken.
204+
205+ ** Only when a count is zero or surprising** , pull a small sample to understand
206+ what fields are actually present on that resource type:
179207
180208``` bash
181209kubectl exec -n lfx " $NATS_POD " -- \
182210 curl -s --max-time 15 -X GET " $OPENSEARCH_BASEURL /_search" \
183211 -H ' Content-Type: application/json' \
184212 -d ' {
185- "size": 1 ,
186- "_source": ["parent_refs"],
213+ "size": 3 ,
214+ "_source": ["tags", " parent_refs", "object_type "],
187215 "query": {
188216 "bool": {
189217 "must": [
190218 { "term": { "object_type": "<RESOURCE_TYPE>" } },
191- { "prefix": { "parent_refs": "<PREFIX>:" } },
192219 { "range": { "updated_at": { "gte": "now-45d" } } }
193220 ]
194221 }
195222 }
196223 }'
197224```
198225
199- Record the ` total.value ` from each response. A non-zero value confirms the
200- key/prefix is present in recently indexed data. If the count is zero but the
201- resource type has older data, note it as "not seen in last 45 days" rather
202- than immediately marking it broken — check the sample query to understand
203- overall coverage before rendering a verdict.
204-
205226## Step 5 — Build the truth table
206227
207228Cross-reference: tool parameter → mechanism → contract definition → index
@@ -256,6 +277,6 @@ After applying fixes, run `make build` to confirm compilation succeeds.
256277
257278## Step 8 — Verify fixes
258279
259- Re-run the targeted ` curl ` queries from Step 4 against the corrected
280+ Re-run the aggregation count queries from Step 4 against the corrected
260281mechanism to confirm non-zero results. Report before/after hit counts for
261282each fixed filter.
0 commit comments