VarTracer

VarTracer is a Python-based tool designed for dynamic code execution tracing and dependency analysis of Python code. It helps developers understand the execution flow, variable dependencies, and function call stacks in their code.

Features

• Dynamic Tracing: Tracks code execution uutilizing sys.settrace.
• Dependency Analysis: Analyzes variable dependencies and assignments using an AST-based parser.
• Execution Stack Export:
  • JSON format (VTrace_exec_stack.json)
  • Text format (VTrace_exec_stack.txt)
  • Raw trace output (VTrace_raw_output.txt)
• Module Filtering: Supports filtering of standard library and custom modules.
• Recursive Call Analysis: Handles nested function calls and dependencies.

File Structure

• VarTracer_Core.py: Core implementation of the tracing functionality.
• ASTParser.py: Dependency analysis using Python's ast module.
• Utilities.py: Helper functions for the tool.
• test_code/: Automated unittest test suite.
• trace_output/: Directory for trace output files.

Quick Start

1. Run the Example: Modify and execute the following code in VarTracer_Core.py:

   if __name__ == "__main__":
       from pathlib import Path
       output_dir = os.path.join(Path(__file__).resolve().parent, "trace_output")
       
       vt = VarTracer(clean_stdlib=True)
       vt.start()
   
       a = 1
       b = a + 2
   
       vt.stop()
       vt.raw_result(output_dir=output_dir)
       vt.exec_stack_txt(output_path=output_dir)
       vt.exec_stack_json(output_path=output_dir)

2. View Results:

   • trace_output/VTrace_exec_stack.json: Nested JSON execution stack.
   • trace_output/VTrace_exec_stack.txt: Human-readable execution stack.
   • trace_output/VTrace_raw_output.txt: Raw trace output.

Core Classes and Methods

1. VarTracer

   • start(): Start tracing.
   • stop(): Stop tracing.
   • raw_result(output_dir): Export raw trace results.
   • exec_stack_txt(output_path, output_name): Export execution stack in text format.
   • exec_stack_json(output_path, output_name): Export execution stack in JSON format.

2. ASTParser

   • LineDependencyAnalyzer: Analyzes dependencies and assignments in a single line of code.
   • DependencyTree: Parses the call stack to extract variable dependencies.

Example Output

JSON format

{
    "trace_started_at": "2025-05-13 12:00:00",
    "execution_stack": [
        {
            "module": "test_code.playground",
            "file_path": "/path/to/playground.py",
            "func": "main",
            "line_no": 10,
            "line_content": "x = y + z",
            "dependencies": ["y", "z"],
            "assigned_vars": ["x"]
        }
    ]
}

Text format

Trace started at 2025-05-13 12:00:00
CALL → MODULE 'test_code.playground' | FILE '/path/to/playground.py' | FUNC 'main'()
    LINE - MODULE 'test_code.playground' | FILE '/path/to/playground.py' | FUNC 'main'() | LINE 10
        x = y + z
        dependencies: ['y', 'z']
        assigned_vars: ['x']
RETURN ← MODULE 'test_code.playground' | FILE '/path/to/playground.py' | FUNC 'main'()

Testing

Why the tests are structured this way

The test suite is organized around the two main pipelines of the project:

• Data dependency analysis: verifies that assignments and variable uses are extracted correctly from Python syntax.
• Execution trace generation: verifies that runtime events are captured correctly and exported in the expected formats.

The design goal is to make failures easy to localize:

• Unit tests focus on the smallest analysis component, LineDependencyAnalyzer.
• Integration tests execute short temporary Python scripts through VarTracer, then inspect the generated execution stack and dependency tree.
• Interface tests cover extension_interface, because it launches a subprocess and has different failure modes from the direct API.
• Regression tests cover cases that previously failed and were repaired, such as multiline assignments, augmented assignments, walrus expressions, comprehension scoping, and internal tracer-frame leakage.

Design ideas behind the recent fixes

The current tests encode the following design decisions:

• Statement-level analysis instead of single-line parsing: dependency analysis is now tied to the full AST statement that covers the current trace line. This is what allows multiline assignments and walrus expressions in control-flow headers to work correctly.
• Read/write semantics for augmented assignment: statements like a += 1 must treat a as both a dependency and an assignment target.
• Local scope shadowing inside expressions: lambda parameters and comprehension targets should not be mistaken for outer-scope data dependencies.
• Clean execution traces: exported traces should describe the target program, not VarTracer's own internal implementation frames.

How to run the tests

Run the full suite from the repository root:

/Users/zhangmengqi/miniforge3/envs/vartracer/bin/python -m unittest discover -s VarTracer/test_code -p "test_*.py" -v

Run only the data dependency tests:

/Users/zhangmengqi/miniforge3/envs/vartracer/bin/python -m unittest \
  VarTracer.test_code.test_line_dependency_analyzer \
  VarTracer.test_code.test_vartracer_flow \
  VarTracer.test_code.test_known_limitations -v

Run only the execution trace tests:

/Users/zhangmengqi/miniforge3/envs/vartracer/bin/python -m unittest \
  VarTracer.test_code.test_execution_trace \
  VarTracer.test_code.test_execution_trace_limitations \
  VarTracer.test_code.test_extension_interface -v

What each test file is responsible for

• test_line_dependency_analyzer.py: unit tests for individual syntax forms.
• test_vartracer_flow.py: end-to-end dependency-tree checks on small scripts.
• test_known_limitations.py: regression coverage for cases that were previously broken.
• test_execution_trace.py: execution-stack structure, exception events, and trace exports.
• test_execution_trace_limitations.py: regression coverage for internal-frame filtering.
• test_extension_interface.py: subprocess-based extension entrypoint behavior.

How to interpret results

• ok: the tested behavior matches the current contract.
• FAIL: the code ran, but a result is wrong.
• ERROR: the test could not complete, often due to an exception, import problem, or file/output issue.

License

VarTracer is licensed under the MIT License.