Abstract Syntax Trees (ASTs)¶
By default, an AST is either:
a value, for simple elements such as token, pattern, or constant
a
tuple
, for closures, gatherings, and the right-hand-side of rules with more than one element but without named elementsa
dict
-derived object (AST
) that contains one item for every named element in the grammar rule, with items can be accessed through the standarddict
syntax (ast['key']
), or as attributes (ast.key
).
AST entries are single values if only one item was associated with a
name, or tuple
if more than one item was matched. There’s a provision in
the grammar syntax (the +:
operator) to force an AST entry to be
a tuple
even if only one element was matched. The value for named
elements that were not found during the parse (perhaps because they are
optional) is None
.
When the parseinfo=True
keyword argument has been passed to the
Parser
constructor or enabled with the @@parseinfo
directive, a parseinfo
item is added to AST nodes
that are dict-like. The item contains a collections.namedtuple
with the parse information for the node:
ParseInfo = namedtuple(
'ParseInfo',
[
'tokenizer',
'rule',
'pos',
'endpos',
'line',
'endline',
]
)
With the help of the Tokenizer.line_info()
method, it is possible to
recover the line, column, and original text parsed for the node. Note
that when ParseInfo
is generated, the Tokenizer
used during parsing
is kept in memory for the lifetime of the AST.
Generation of parseinfo
can also be controlled using the
@@parseinfo :: True
grammar directive.