Commit a959654
Refactor base logger handlers for readability and safer flattening (#3782)
This PR refactors `ignite/handlers/base_logger.py` to improve
readability and maintainability, while preserving behavior and keeping
risk extremely low. All changes are localized to this module and are
fully covered by the existing
`tests/ignite/handlers/test_base_logger.py` (23/23 tests passing).
### What changed
#### BaseWeightsHandler
- Simplified `whitelist` handling:
- If `whitelist` is `None`, we still log all `model.named_parameters()`.
- If `whitelist` is callable, we now use a clear comprehension:
```python
weights = {
name: param
for name, param in model.named_parameters()
if whitelist(name, param)
}
```
- If `whitelist` is a list of prefixes, we build `prefixes =
tuple(whitelist)` and filter with `name.startswith(prefixes)`, which is
more idiomatic and readable than manual prefix checks.
- Store `self.weights` as:
```python
self.weights = tuple(weights.items())
```
making it explicit that `weights` is a stable snapshot of `(name,
param)` pairs, which is easier to reason about than a live view.
#### BaseOptimizerParamsHandler
- Kept the existing semantic check, but made the validation more
readable:
```python
if not (
isinstance(optimizer, Optimizer)
or (hasattr(optimizer, "param_groups") and
isinstance(optimizer.param_groups, Sequence))
):
raise TypeError(
"Argument optimizer should be torch.optim.Optimizer or has attribute
'param_groups' as list/tuple, "
f"but given {type(optimizer)}"
)
```
- No behavioral change: non‑torch optimizers with `param_groups` still
work (covered by `test_opt_params_handler_on_non_torch_optimizers`).
#### BaseOutputHandler
- Validation logic is unchanged in meaning but clearer in structure:
- `metric_names` must be a list or `"all"`.
- `output_transform` and `global_step_transform` must be callable if
provided.
- At least one of `metric_names`, `output_transform`, or
`state_attributes` must be set.
- Ensured a sensible default `global_step_transform` is injected when
`None`, with a small inlined helper:
```python
if global_step_transform is None:
def global_step_transform(engine: Engine, event_name: str | Events) ->
int:
return engine.state.get_event_attrib_value(event_name)
```
- `_setup_output_metrics_state_attrs`:
- Keeps existing behavior for:
- `metric_names` list vs `"all"`,
- `output_transform` returning dict or non‑dict,
- `state_attributes` only, and combinations of metrics + attributes +
output.
- Factors key building into two small helpers:
```python
def key_tuple_fn(parent_key, *args): ...
def key_str_fn(parent_key, *args): ...
key_fn = key_tuple_fn if key_tuple else key_str_fn
```
- Flattens `handle_value_fn` structure for clarity without changing
logic:
```python
def handle_value_fn(value):
if isinstance(value, numbers.Number):
return value
if isinstance(value, torch.Tensor) and value.ndimension() == 0:
return value.item()
if isinstance(value, str) and log_text:
return value
warnings.warn(f"Logger output_handler can not log metrics value type
{type(value)}")
return None
```
- This makes the value-handling branch easier to follow (no nested
`else`), while still doing exactly what the tests expect (numbers and
scalar tensors kept, strings optional via `log_text`, others warned and
dropped).
#### `_flatten_dict`
- Added a concise but informative docstring in the same style as other
docstrings:
```python
"""Recursively flatten a nested mapping.
Args:
in_dict: Mapping to flatten.
key_fn: Function used to build the flattened key from the parent key and
current key.
value_fn: Function used to convert leaf values to loggable values.
parent_key: Parent key prefix for nested values.
Returns:
Flattened dictionary of processed key-value pairs.
"""
```
- Fixed and clarified recursion over complex structures:
- Nested mappings:
```python
if isinstance(value, Mapping):
items.update(_flatten_dict(value, key_fn, value_fn, new_key))
continue
```
- Namedtuples (tuple with `_fields` attribute):
```python
if isinstance(value, tuple) and hasattr(value, "_fields"):
for i, item in enumerate(value):
items.update(_flatten_dict({str(i): item}, key_fn, value_fn, new_key))
continue
```
- Non‑string sequences:
```python
if not isinstance(value, str) and isinstance(value, Sequence):
for i, item in enumerate(value):
items.update(_flatten_dict({str(i): item}, key_fn, value_fn, new_key))
continue
```
- 1D tensors:
```python
if isinstance(value, torch.Tensor) and value.ndimension() == 1:
for i, item in enumerate(value):
items.update(_flatten_dict({str(i): item.item()}, key_fn, value_fn,
new_key))
continue
```
- Leaf values:
```python
new_value = value_fn(value)
if new_value is not None:
items[new_key] = new_value
```
- The use of `continue` keeps each case visually separated and makes the
final “leaf” branch easy to read.
- Behavior is verified by the existing tests that cover nested dicts,
lists, tuples, namedtuple-like structures, and tensors.
#### BaseWeightsScalarHandler
- Kept behavior intact but clarified the structure:
- Validate `reduction` is callable.
- Test `reduction` on `torch.ones(4, 2)` and ensure the output is either
a `numbers.Number` or a 0D tensor (checked via `_is_0d_tensor`).
- Assign `self.reduction` only after validation.
- Simplified docstring to a one-liner to match other classes.
### Why this is more readable
- Complex logic is split into small, clearly named helpers
(`key_tuple_fn`, `key_str_fn`, `handle_value_fn`).
- `_flatten_dict` now has:
- A clear docstring explaining the contract.
- Well-separated branches with minimal nesting and explicit `continue`s.
- Validation logic uses clear boolean conditions rather than deeply
nested `if/elif` chains.
- Docstrings follow a consistent pattern, making the module easier to
navigate for new contributors.
### Why it’s extremely low-risk
- No public API signatures were changed.
- Error messages are preserved (tests assert them via `match=...`).
- Existing behavior of:
- metrics/output/state-attributes flattening,
- optimizer param logging (including custom optimizers),
- reduction function validation,
- and the various composite metrics/state cases
is fully covered by `tests/ignite/handlers/test_base_logger.py`, which
all pass (23/23).
- The only functional fix is in `_flatten_dict`’s namedtuple/sequence
handling, and that path is already tested by the composite-metrics test
cases, which still pass after the refactor.
---------
Co-authored-by: vfdev <vfdev.5@gmail.com>1 parent 2d8c83a commit a959654
1 file changed
Lines changed: 27 additions & 40 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
5 | | - | |
| 5 | + | |
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
| |||
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
18 | | - | |
| 18 | + | |
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
22 | | - | |
23 | | - | |
| 22 | + | |
24 | 23 | | |
25 | 24 | | |
26 | 25 | | |
| |||
34 | 33 | | |
35 | 34 | | |
36 | 35 | | |
37 | | - | |
| 36 | + | |
38 | 37 | | |
39 | 38 | | |
40 | 39 | | |
41 | 40 | | |
42 | 41 | | |
43 | | - | |
44 | 42 | | |
45 | 43 | | |
46 | 44 | | |
47 | | - | |
48 | | - | |
49 | | - | |
| 45 | + | |
50 | 46 | | |
51 | | - | |
52 | | - | |
53 | | - | |
54 | | - | |
| 47 | + | |
| 48 | + | |
55 | 49 | | |
56 | | - | |
| 50 | + | |
57 | 51 | | |
58 | 52 | | |
59 | 53 | | |
| |||
90 | 84 | | |
91 | 85 | | |
92 | 86 | | |
93 | | - | |
| 87 | + | |
94 | 88 | | |
95 | 89 | | |
96 | 90 | | |
| |||
144 | 138 | | |
145 | 139 | | |
146 | 140 | | |
147 | | - | |
148 | | - | |
149 | 141 | | |
150 | 142 | | |
151 | 143 | | |
| |||
164 | 156 | | |
165 | 157 | | |
166 | 158 | | |
167 | | - | |
| 159 | + | |
| 160 | + | |
168 | 161 | | |
169 | | - | |
170 | | - | |
171 | | - | |
172 | | - | |
173 | | - | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
174 | 167 | | |
175 | 168 | | |
176 | 169 | | |
| |||
207 | 200 | | |
208 | 201 | | |
209 | 202 | | |
210 | | - | |
211 | | - | |
212 | | - | |
| 203 | + | |
213 | 204 | | |
214 | 205 | | |
215 | 206 | | |
| |||
223 | 214 | | |
224 | 215 | | |
225 | 216 | | |
226 | | - | |
| 217 | + | |
227 | 218 | | |
228 | 219 | | |
229 | 220 | | |
230 | 221 | | |
231 | | - | |
| 222 | + | |
232 | 223 | | |
233 | 224 | | |
234 | 225 | | |
235 | 226 | | |
236 | 227 | | |
237 | | - | |
| 228 | + | |
238 | 229 | | |
239 | 230 | | |
240 | 231 | | |
| |||
257 | 248 | | |
258 | 249 | | |
259 | 250 | | |
260 | | - | |
| 251 | + | |
261 | 252 | | |
262 | 253 | | |
263 | 254 | | |
| |||
270 | 261 | | |
271 | 262 | | |
272 | 263 | | |
273 | | - | |
274 | | - | |
275 | | - | |
| 264 | + | |
| 265 | + | |
276 | 266 | | |
277 | | - | |
| 267 | + | |
278 | 268 | | |
279 | 269 | | |
280 | 270 | | |
| |||
314 | 304 | | |
315 | 305 | | |
316 | 306 | | |
317 | | - | |
318 | | - | |
| 307 | + | |
319 | 308 | | |
320 | 309 | | |
321 | | - | |
322 | | - | |
| 310 | + | |
323 | 311 | | |
324 | 312 | | |
325 | 313 | | |
326 | 314 | | |
327 | 315 | | |
328 | 316 | | |
329 | 317 | | |
330 | | - | |
331 | | - | |
| 318 | + | |
0 commit comments