Syntax augmentation

These are the building blocks for defining new surface syntax — see Add new surface syntax for the how-to and Tutorial: build optional chaining (?.) for a full worked example. Syntax augmentation requires Python ≥ 3.8.

AugmentationSpec

class pyccolo.AugmentationSpec(aug_type, token, replacement, close_token, close_replacement, name_pattern, body_func_wrapper, custom)

aug_type: AugmentationType

Alias for field number 0

token: str

Alias for field number 1

replacement: str

Alias for field number 2

close_token: str | None

Alias for field number 3

close_replacement: str | None

Alias for field number 4

name_pattern: str | None

Alias for field number 5

body_func_wrapper: str | None

Alias for field number 6

custom: CustomRewrite | None

Alias for field number 7

A spec declares a source-level rewrite from an (otherwise illegal) token to a legal one; Pyccolo records where the rewrite happened so a handler can attach to the resulting node via get_augmentations(). For paired-delimiter (brace-block) syntax, also set close_token / close_replacement; is_paired and is_custom report the kind.

optional_chaining_spec = pyc.AugmentationSpec(
    aug_type=pyc.AugmentationType.dot_suffix, token="?.", replacement="."
)

AugmentationType

class pyccolo.AugmentationType(*values)

prefix = 'prefix'

suffix = 'suffix'

dot_prefix = 'dot_prefix'

dot_suffix = 'dot_suffix'

binop = 'binop'

boolop = 'boolop'

call = 'call'

subscript = 'subscript'

custom = 'custom'

The kind of token position a spec rewrites — e.g. dot_suffix (?.), binop (|>), boolop (??), or custom.

CustomRewrite

class pyccolo.CustomRewrite

Drives a AugmentationType.custom spec. A cooperating tracer that needs a context-sensitive surface rewrite (one the static token/paired passes can’t express – e.g. a token whose meaning depends on the preceding token, or a per-occurrence choice between two lowerings) implements these three methods so the rewrite participates in pyccolo’s position-remapping and untransform machinery uniformly, instead of bolting on a parallel resugaring path.

Subclassing is optional (the methods are duck-typed); it just documents intent.

rewrite(code: str, register: Callable[[int, int], None]) → Tuple[str, List[Tuple[int, int, int]]]

Forward-rewrite code -> (new_code, edits).

edits are (start, end, new_len) triples in code’s input absolute offsets – sorted ascending and disjoint, the same shape replace_tokens_and_get_augmented_positions() builds – so the caller can remap tracked positions through them with remap_through_edits(). For each rewritten occurrence whose resulting AST node should be reverse-handled by untransform, call register(line, col) with the 1-indexed line / 0-indexed col of the node’s anchor in ``new_code`` (the location range_for() will re-derive on the parsed node).

range_for(node: AST) → Range | None

Anchor Range of a rewritten node (mirrors the built-in _get_<aug_type>_range_for helpers), or None if node is not one this rewrite produced. Used both to bind the registered position forward and to locate the node during untransform.

reverse(node: AST, spec: AugmentationSpec, aug_range: Range, code: str, line_starts: List[int]) → Tuple[int, int, str] | None

Reverse (resugar) splice (start, end, new_text) on valid code, or None to leave node untouched. A single whole-span edit suffices; the caller applies edits right-to-left.

The extension point for context-sensitive rewrites that a single token→token substitution can’t express. Subclass it and reference it from a spec with aug_type=AugmentationType.custom.

Positions

class pyccolo.Position(line, col)

line: int

Alias for field number 0

col: int

Alias for field number 1

pyc.transform / pyc.untransform accept and return Position\ s so tooling can build source maps across the rewrite.

The rewriter

class pyccolo.AstRewriter(tracers: List[BaseTracer], path: str, module_id: int | None = None)

visit(node: AST, instrument: bool = True)

Visit a node.