FIX: Don't revert the entirety of `sys.modules` after running `kernprof.main` by TTsangSC · Pull Request #437 · pyutils/line_profiler

TTsangSC · 2026-07-18T07:27:42Z

Closes #436.

(EDIT: typos; missing summaries for kernprof.y::main() and line_profiler/autoprofile/autoprofile.py::run())

Code changes

line_profiler/line_profiler_utils.py::restore:
- Migrated from kernprof.py::_restore, so that other parts of the codebase can use it for cleanup.
- The constructor method .mapping() (resp. .instance_dict()) now takes an extra argument keys: Collection[K] (resp. attrs: Collection[str]) to limit the scope of reversion.
- The context manager is now reentrant.
- Minor backward incompatibility: instances no longer expose the attributes .getter, .setter, and .old. (But we don't really use it anywhere other than as anonymous decorators and context managers anyway, so nothing changed.)
kernprof.py::main():
On exit:
- Instead of freezing the entire sys.modules, we now only revert sys.modules['__main__'] or the lack thereof.
- Instead of freezing the entire line_profiler._diagnostics namespace, we now only revert line_profiler._diagnostics.log which is overridden.
line_profiler/autoprofile/autoprofile.py::run():
Instead of freezing the entire sys.modules when running the supplied code, we now only revert sys.modules['__main__'] or the lack thereof.

Test changes

line_profiler/line_profiler_utils.py::restore.sequence()'s doctest extended to demonstrate/check for reentrant behavior.
tests/test_kernprof.py:
- Factored out common pattern of creating a tempdir and cd-ing therein into a helper context manager (chdir_temp).
- Minor fix: test_kernprof_m_import_resolution() and test_kernprof_eager_preimport_bad_module() now use os.pathsep instead of a hard-coded ':' to join ${PYTHONPATH} components.
- New test test_ppe_pickling formalizes the failing case in Reverting sys.modules causes pickling issues with concurrent.futures.ProcessPoolWorker #436, checking that repeated invocation of kernprof.main() running concurrent.futures code doesn't fail due to consistency issues with sys.modules.

tests/test_kernprof.py chdir_temp New helper context manager for creating a new tempdir and `cd`-ing into it, factored out from common pattern in tests test_*() Updated to use the above test_kernprof_m_import_resolution() test_kernprof_eager_preimport_bad_module() Replaced hard-coded ':' in the `${PYTHONPATH}` value with `os.pathsep` test_ppe_pickling() New test which uses `concurrent.futures.ProcessPoolExecutor` in code run by `kernprof.main()`; the case where PPE hasn't been imported prior to `kernprof.main()` being called twice currently fails

kernprof.py _restore Migrated to `line_profiler.line_profiler_utils.restore` main() Instead of reverting the entire module namespace of `line_profiler.diagnostics`, now only reverting `line_profiler.diagnostics.log` (which is replaced in `_parse_arguments()`) _main_profile() Instead of reverting the entire `sys.modules`, now only reverting `sys.modules['__main__']` line_profiler/autoprofile/autoprofile.py::run() Replaced internal context manager with a `restore` instance line_profiler/line_profiler_utils.py::restore - Migrated from `kernprof.py::_restore` - Added type annotations - Added optional argument `keys` (resp. `attrs`) to `.mapping()` (resp. `.instance_dict()`) so that users can selectively revert certain key-value pairs (resp. attributes) - Now reentrant - Note: the attributes `.setter`, `.getter`, and `.obj` are removed

codecov · 2026-07-18T07:58:30Z

Codecov Report

❌ Patch coverage is 98.24561% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 82.76%. Comparing base (4940bca) to head (5845332).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
line_profiler/line_profiler_utils.py	98.18%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #437      +/-   ##
==========================================
+ Coverage   82.40%   82.76%   +0.36%     
==========================================
  Files          20       20              
  Lines        2256     2298      +42     
  Branches      359      361       +2     
==========================================
+ Hits         1859     1902      +43     
  Misses        300      300              
+ Partials       97       96       -1

Files with missing lines	Coverage Δ
line_profiler/autoprofile/autoprofile.py	`90.90% <100.00%> (ø)`
line_profiler/line_profiler_utils.py	`96.05% <98.18%> (+9.09%)`	⬆️

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f8e40f6...5845332. Read the comment docs.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Erotemic

This all LGTM. Minor comments. You can choose what to do with them. Say the word and I'll merge.

Erotemic · 2026-07-18T15:01:57Z

+
+
+    def main() -> None:
+        with ProcessPoolExecutor(mp_context=get_context('spawn')) as ex:


Suggested change

with ProcessPoolExecutor(mp_context=get_context('spawn')) as ex:

with ProcessPoolExecutor(max_workers=2, mp_context=get_context('spawn')) as ex:

Not sure if we want to use all CPUs on a test. On one hand it could expose more edge cases on the other, setting it is more stable.

Erotemic · 2026-07-18T15:03:07Z

    short_string_path,
 )
+from line_profiler.line_profiler_utils import (
+    restore as _restore,  # Compatibility


Compatibility for a private name isn't required. It doesn't bother me though. This is mostly a nitpick.

Fair enough, since we're technically already changing restore's API (it no longer has .getter, .setter, and .old – but then again those members aren't used anywhere by the code outside of the class itself). Might as well just import under the new name.

Erotemic · 2026-07-18T15:05:05Z

+        python_path = str(temp_dpath)
        if 'PYTHONPATH' in os.environ:
-            python_path += ':' + os.environ['PYTHONPATH']
+            python_path += os.pathsep + os.environ['PYTHONPATH']


kernprof.py No longer importing `line_profiler.line_profiler_utils.restore` under the name `_restore` since it isn't fully compatible to the old `_restore` anyway tests/test_kernprof.py::test_ppe_pickling() Now explicitly passing `max_workers` to PPE to avoid spinning up too many processes

TTsangSC · 2026-07-18T15:28:34Z

Thanks for the quick review, should be alright now (other than the ty lint – but that's a known issue, and #434 would patch pyproject.toml and fix that).

Erotemic · 2026-07-18T16:23:56Z

Yeah I ignored the ty error for that reason, let's land #434 next

TTsangSC added 2 commits July 18, 2026 08:59

Erotemic approved these changes Jul 18, 2026

View reviewed changes

Erotemic mentioned this pull request Jul 18, 2026

FIX: handling multi-target (from-)import statements #434

Open

Erotemic merged commit 7c1e41f into pyutils:main Jul 18, 2026
44 of 45 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

FIX: Don't revert the entirety of `sys.modules` after running `kernprof.main`#437

FIX: Don't revert the entirety of `sys.modules` after running `kernprof.main`#437
Erotemic merged 3 commits into
pyutils:mainfrom
TTsangSC:436-fix

TTsangSC commented Jul 18, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Jul 18, 2026 •

edited

Loading

Uh oh!

Erotemic left a comment

Uh oh!

Erotemic Jul 18, 2026

Uh oh!

Erotemic Jul 18, 2026

Uh oh!

TTsangSC Jul 18, 2026

Uh oh!

Erotemic Jul 18, 2026

Uh oh!

TTsangSC commented Jul 18, 2026

Uh oh!

Uh oh!

Erotemic commented Jul 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants



		def main() -> None:
		with ProcessPoolExecutor(mp_context=get_context('spawn')) as ex:

	with ProcessPoolExecutor(mp_context=get_context('spawn')) as ex:
	with ProcessPoolExecutor(max_workers=2, mp_context=get_context('spawn')) as ex:

Uh oh!

Conversation

TTsangSC commented Jul 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code changes

Test changes

Uh oh!

codecov Bot commented Jul 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Erotemic left a comment

Choose a reason for hiding this comment

Uh oh!

Erotemic Jul 18, 2026

Choose a reason for hiding this comment

Uh oh!

Erotemic Jul 18, 2026

Choose a reason for hiding this comment

Uh oh!

TTsangSC Jul 18, 2026

Choose a reason for hiding this comment

Uh oh!

Erotemic Jul 18, 2026

Choose a reason for hiding this comment

Uh oh!

TTsangSC commented Jul 18, 2026

Uh oh!

Uh oh!

Erotemic commented Jul 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

TTsangSC commented Jul 18, 2026 •

edited

Loading

codecov Bot commented Jul 18, 2026 •

edited

Loading