pyccolo

Start here

  • Installation
    • Supported Python versions
    • Working from a source checkout
    • Next steps
  • Your first tracer
    • The tracer class
    • The class is the context manager
    • What is up with pyc.exec(...)?
    • Handlers can change behavior, not just observe it

Guides

  • Observe and override values
    • Override a value by returning it
    • Returning None means “don’t override”
    • The four sentinels
    • Fire a handler only sometimes
    • Static vs. dynamic predicates
    • Register by AST node type
    • Intercept an expression before it runs
  • Compose multiple tracers
    • Nesting threads values through
    • Activate several at once
    • Multiple handlers on one tracer
    • Tracers that only rewrite syntax
    • Re-entrant handlers
  • Guards and performance
    • Region guards: run once, then step aside
    • Local guards: run once per node
    • Per-call state with TraceStack
    • Class-level switches
  • Tracing real programs
    • Instrument imported modules
    • Instrument a single function
    • sys.settrace-style events
    • Run a whole script under the CLI
  • Source-to-source transforms
    • Desugar and resugar
    • Analysis-only (pure) transforms
    • Source maps
  • Add new surface syntax
    • Declare a rewrite with AugmentationSpec
    • A complete example: a pipe operator
    • The augmentation vocabulary
    • Paired delimiters: brace blocks
    • Context-sensitive rewrites
  • IPython and Jupyter
    • Load the extension and register a tracer
    • The four ways to name a tracer
    • Registration order is canonical
    • Composing with ipyflow
    • How it works
    • Two things worth knowing when you write a tracer for the notebook
    • JupyterLite

Tutorials

  • Tutorial: build a coverage tracer
    • Step 1: record every statement
    • Step 2: choose which files to cover
    • Step 3: ignore sandbox frames
    • Step 4: report at the end
    • Step 5: run it
    • Where to next
  • Tutorial: build optional chaining (?.)
    • Step 1: declare the new syntax
    • Step 2: handle the tagged access
    • Step 3: turn the sentinel back into None
    • Step 4: run it
    • Step 5: see the rewrite as source
    • Where to next
  • Tutorial: build a pipe operator (|>)
    • Step 1: declare the operator token
    • Step 2: redefine what the operator does
    • Step 3: chain it
    • Step 4: add an assigning pipe |>>
    • Where to next
  • Tutorial: quick lambdas (f[_ + _])
    • The idea
    • The key handler
    • Trying it out
    • Where to next
  • Tutorial: make imports lazy
    • The plan
    • Step 1: a placeholder for an unresolved import
    • Step 2: intercept the import statement
    • Step 3: resolve on first use
    • Putting it to work
    • Where to next
  • Tutorial: concolic execution
    • The one thing operator overloading can’t do
    • Capturing branch conditions
    • The search loop
    • Exploring a function
    • Where to next
  • Tutorial: a variable watchpoint debugger
    • Step 1: catch every assignment
    • Step 2: set a watch and run
    • Step 3: watch by condition
    • Where to next

How it works

  • The model: events and handlers
    • Events
    • Handlers
    • Observe, or override
    • Pay-as-you-go instrumentation
  • From source to running program
    • 1. Syntax augmentation (optional)
    • 2. Parse
    • 3. Rewrite: weave in event emission
    • 4. Bookkeeping
    • 5. Run: emit events, dispatch handlers
    • 6. Guards: turn instrumentation back off
    • What about sys events?
    • Getting the source instead of running it
  • Why composing tracers works
    • The usual pain
    • The Pyccolo approach
    • How return values compose
    • Transformational tracers
  • Why pyc.exec, and how scoping works
    • The AST is fixed at compile time
    • pyc.exec returns a namespace
    • When you don’t need pyc.exec
    • Reaching into the caller’s scope

Reference

  • Event taxonomy
    • How to read the table
    • The events
      • TraceEvent
  • Handlers
    • The signature
    • Keyword arguments
    • Return values
      • pyccolo.Null
      • pyccolo.Skip
      • pyccolo.SkipAll
      • pyccolo.Pass
    • The thunk convention (before-expression events)
    • Example
  • Registering handlers
    • The event decorator
    • register_handler
      • register_handler()
    • Gating a handler with when
      • What you can pass
      • Static vs. dynamic evaluation
      • Composing predicates
    • register_raw_handler
      • register_raw_handler()
    • skip_when_tracing_disabled
      • skip_when_tracing_disabled()
  • BaseTracer
    • BaseTracer
    • Activation & singleton
      • BaseTracer.tracing_context()
      • BaseTracer.tracing_enabled()
      • BaseTracer.tracing_disabled()
      • BaseTracer.tracing_non_context()
    • Running & compiling code
      • BaseTracer.exec()
      • BaseTracer.eval()
      • BaseTracer.execute()
      • BaseTracer.parse()
      • BaseTracer.transform()
      • BaseTracer.untransform()
      • BaseTracer.instrumented()
      • BaseTracer.trace_lambda()
      • BaseTracer.parse_fragment()
      • BaseTracer.exec_fragment()
    • Overridable hooks
      • BaseTracer.should_instrument_file()
      • BaseTracer.file_passes_filter_for_event()
      • BaseTracer.enter_tracing_hook()
      • BaseTracer.exit_tracing_hook()
      • BaseTracer.static_init_module()
      • BaseTracer.should_propagate_handler_exception()
    • Guards
      • BaseTracer.activate_guard()
      • BaseTracer.deactivate_guard()
      • BaseTracer.register_local_guard()
    • Augmentation helpers
      • BaseTracer.syntax_augmentation_specs()
      • BaseTracer.get_augmentations()
      • BaseTracer.make_ast_rewriter()
      • BaseTracer.make_syntax_augmenter()
      • BaseTracer.make_sandbox_fname()
    • AST ancestry
      • BaseTracer.is_outer_stmt()
      • BaseTracer.is_initial_frame_stmt()
      • BaseTracer.stmt_only_has_ancestor_types()
    • State
      • BaseTracer.make_stack()
      • BaseTracer.reset()
    • NoopTracer
      • NoopTracer
  • Configuration attributes
  • Predicates
    • Predicate
    • Composing predicates
      • CompositePredicate
  • Syntax augmentation
    • AugmentationSpec
      • AugmentationSpec
    • AugmentationType
      • AugmentationType
    • CustomRewrite
      • CustomRewrite
    • Positions
      • Position
    • The rewriter
      • AstRewriter
  • Source-to-source API
    • transform()
    • untransform()
    • parse()
    • Pure (analysis-only) transforms
      • is_pure_transform()
    • Sandbox filenames
      • pyccolo.SANDBOX_FNAME
      • pyccolo.SANDBOX_FNAME_PREFIX
      • pyccolo.TRACEBACK_VISIBLE_SANDBOX_FILES
  • Command-line interface
    • Options
      • Positional Arguments
      • Named Arguments
  • Utilities & globals
    • Tracer stack & contexts
      • tracer()
      • instance()
      • resolve_tracer()
      • multi_context()
      • tracing_context()
      • tracing_enabled()
      • tracing_disabled()
      • instrumented()
      • allow_reentrant_event_handling()
    • Frames & guards
      • set_frame_local()
      • make_guard_name()
      • pyccolo.PYCCOLO_BUILTIN_PREFIX
    • Traceback visibility
      • mark_traceback_visible()
      • is_traceback_visible()
    • Sentinels

More

  • Example gallery
    • In the wild
  • History
    • 0.0.94 (2026-07-10)
    • 0.0.92 (2026-07-05)
    • 0.0.87 (2026-06-24)
    • 0.0.86 (2026-06-12)
    • 0.0.85 (2026-01-20)
    • 0.0.84 (2026-01-19)
    • 0.0.83 (2026-01-15)
    • 0.0.82 (2026-01-09)
    • 0.0.81 (2026-01-08)
    • 0.0.80 (2026-01-06)
    • 0.0.78 (2026-01-03)
    • 0.0.77 (2025-12-26)
    • 0.0.76 (2025-12-25)
    • 0.0.75 (2025-12-22)
    • 0.0.73 (2025-11-21)
    • 0.0.72 (2025-04-10)
    • 0.0.71 (2025-03-16)
    • 0.0.70 (2025-01-04)
    • 0.0.69 (2024-12-19)
    • 0.0.68 (2024-12-12)
    • 0.0.67 (2024-10-13)
    • 0.0.66 (2024-10-06)
    • 0.0.65 (2024-08-25)
    • 0.0.64 (2024-07-21)
    • 0.0.63 (2024-07-19)
    • 0.0.58 (2024-07-12)
    • 0.0.55 (2024-06-24)
    • 0.0.54 (2024-02-29)
    • 0.0.53 (2024-02-24)
    • 0.0.52 (2023-12-21)
    • 0.0.51 (2023-12-19)
    • 0.0.50 (2023-12-07)
    • 0.0.49 (2023-08-19)
    • 0.0.48 (2023-07-15)
    • 0.0.45 (2023-01-02)
    • 0.0.44 (2022-12-14)
    • 0.0.43 (2022-12-06)
    • 0.0.41 (2022-12-04)
    • 0.0.39 (2022-11-19)
    • 0.0.38 (2022-10-30)
    • 0.0.37 (2022-10-26)
    • 0.0.36 (2022-10-24)
    • 0.0.35 (2022-08-06)
    • 0.0.34 (2022-07-18)
    • 0.0.33 (2022-07-12)
    • 0.0.32 (2022-07-02)
    • 0.0.31 (2022-07-02)
    • 0.0.30 (2022-07-02)
    • 0.0.28 (2022-05-30)
    • 0.0.27 (2022-05-30)
    • 0.0.26 (2022-05-21)
    • 0.0.25 (2022-04-18)
    • 0.0.24 (2022-04-18)
    • 0.0.23 (2022-03-18)
    • 0.0.22 (2022-03-17)
    • 0.0.21 (2022-03-02)
    • 0.0.20 (2022-02-14)
    • 0.0.19 (2022-02-14)
    • 0.0.17 (2022-02-03)
    • 0.0.14 (2022-02-02)
    • 0.0.13 (2022-01-31)
    • 0.0.12 (2022-01-30)
    • 0.0.11 (2022-01-30)
    • 0.0.10 (2022-01-26)
    • 0.0.9 (2022-01-24)
    • 0.0.8 (2022-01-23)
    • 0.0.7 (2022-01-06)
    • 0.0.6 (2022-01-06)
    • 0.0.5 (2021-12-29)
    • 0.0.4 (2021-12-26)
    • 0.0.3 (2021-12-23)
    • 0.0.2 (2021-12-22)
    • 0.0.1 (2020-10-25)
pyccolo
  • Search

© Copyright 2020, Stephen Macke.

Built with Sphinx using a theme provided by Read the Docs.