This module contains the interface to the compiler's abstract syntax tree (AST). Macros operate on this tree.

See also:

• macros tutorial
• macros section in Nim manual

The AST in Nim

This section describes how the AST is modelled with Nim's type system. The AST consists of nodes (NimNode) with a variable number of children. Each node has a field named kind which describes what the node contains:

typeNimNodeKind=enum## kind of a node; only explanatorynnkNone,## invalid node kindnnkEmpty,## empty nodennkIdent,## node contains an identifiernnkIntLit,## node contains an int literal (example: 10)nnkStrLit,## node contains a string literal (example: "abc")nnkNilLit,## node contains a nil literal (example: nil)nnkCaseStmt,## node represents a case statement...## many moreNimNode=refNimNodeObjNimNodeObj=objectcasekind:NimNodeKind## the node's kindofnnkNone,nnkEmpty,nnkNilLit:discard## node contains no additional fieldsofnnkCharLit..nnkUInt64Lit:intVal:BiggestInt## the int literalofnnkFloatLit..nnkFloat64Lit:floatVal:BiggestFloat## the float literalofnnkStrLit..nnkTripleStrLit,nnkCommentStmt,nnkIdent,nnkSym:strVal:string## the string literalelse:sons:seq[NimNode]## the node's sons (or children)

For the NimNode type, the [] operator has been overloaded: n[i] is n's i-th child.

To specify the AST for the different Nim constructs, the notation nodekind(son1, son2, ...) or nodekind(value) or nodekind(field=value) is used.

Some child may be missing. A missing child is a node of kind nnkEmpty; a child can never be nil.

Leaf nodes/Atoms

A leaf of the AST often corresponds to a terminal symbol in the concrete syntax. Note that the default float in Nim maps to float64 such that the default AST for a float is nnkFloat64Lit as below.

Identifiers are nnkIdent nodes. After the name lookup pass these nodes get transferred into nnkSym nodes.

Calls/expressions

Command call

Concrete syntax:

echo"abc","xyz"

AST:

nnkCommand(nnkIdent("echo"),nnkStrLit("abc"),nnkStrLit("xyz"))

Call with ()

Concrete syntax:

echo("abc","xyz")

AST:

nnkCall(nnkIdent("echo"),nnkStrLit("abc"),nnkStrLit("xyz"))

Infix operator call

Concrete syntax:

"abc"&"xyz"

AST:

nnkInfix(nnkIdent("&"),nnkStrLit("abc"),nnkStrLit("xyz"))

Note that with multiple infix operators, the command is parsed by operator precedence.

Concrete syntax:

5+3*4

AST:

nnkInfix(nnkIdent("+"),nnkIntLit(5),nnkInfix(nnkIdent("*"),nnkIntLit(3),nnkIntLit(4)))

As a side note, if you choose to use infix operators in a prefix form, the AST behaves as a parenthetical function call with nnkAccQuoted, as follows:

Concrete syntax:

`+`(3,4)

AST:

nnkCall(nnkAccQuoted(nnkIdent("+")),nnkIntLit(3),nnkIntLit(4))

Prefix operator call

Concrete syntax:

?"xyz"

AST:

nnkPrefix(nnkIdent("?"),nnkStrLit("abc"))

Postfix operator call

Note: There are no postfix operators in Nim. However, the nnkPostfix node is used for the asterisk export marker*:

Concrete syntax:

identifier*

AST:

nnkPostfix(nnkIdent("*"),nnkIdent("identifier"))

Call with named arguments

Concrete syntax:

writeLine(file=stdout,"hallo")

AST:

nnkCall(nnkIdent("writeLine"),nnkExprEqExpr(nnkIdent("file"),nnkIdent("stdout")),nnkStrLit("hallo"))

Call with raw string literal

This is used, for example, in the bindSym examples here and with re"some regexp" in the regular expression module.

Concrete syntax:

echo"abc"

AST:

nnkCallStrLit(nnkIdent("echo"),nnkRStrLit("hello"))

Dereference operator []

Concrete syntax:

x[]

AST:

nnkDerefExpr(nnkIdent("x"))

Addr operator

Concrete syntax:

addr(x)

AST:

nnkAddr(nnkIdent("x"))

Cast operator

Concrete syntax:

cast[T](x)

AST:

nnkCast(nnkIdent("T"),nnkIdent("x"))

Object access operator .

Concrete syntax:

x.y

AST:

nnkDotExpr(nnkIdent("x"),nnkIdent("y"))

If you use Nim's flexible calling syntax (as in x.len()), the result is the same as above but wrapped in an nnkCall.

Array access operator []

Concrete syntax:

x[y]

AST:

nnkBracketExpr(nnkIdent("x"),nnkIdent("y"))

Parentheses

Parentheses for affecting operator precedence use the nnkPar node.

Concrete syntax:

(a+b)*c

AST:

nnkInfix(nnkIdent("*"),nnkPar(nnkInfix(nnkIdent("+"),nnkIdent("a"),nnkIdent("b"))),nnkIdent("c"))

Tuple Constructors

Nodes for tuple construction are built with the nnkTupleConstr node.

Concrete syntax:

(1,2,3)(a:1,b:2,c:3)()

AST:

nnkTupleConstr(nnkIntLit(1),nnkIntLit(2),nnkIntLit(3))nnkTupleConstr(nnkExprColonExpr(nnkIdent("a"),nnkIntLit(1)),nnkExprColonExpr(nnkIdent("b"),nnkIntLit(2)),nnkExprColonExpr(nnkIdent("c"),nnkIntLit(3)))nnkTupleConstr()

Since the one tuple would be syntactically identical to parentheses with an expression in them, the parser expects a trailing comma for them. For tuple constructors with field names, this is not necessary.

(1,)(a:1)

AST:

nnkTupleConstr(nnkIntLit(1))nnkTupleConstr(nnkExprColonExpr(nnkIdent("a"),nnkIntLit(1)))

Curly braces

Curly braces are used as the set constructor.

Concrete syntax:

{1,2,3}

AST:

nnkCurly(nnkIntLit(1),nnkIntLit(2),nnkIntLit(3))

When used as a table constructor, the syntax is different.

Concrete syntax:

{a:3,b:5}

AST:

nnkTableConstr(nnkExprColonExpr(nnkIdent("a"),nnkIntLit(3)),nnkExprColonExpr(nnkIdent("b"),nnkIntLit(5)))

Brackets

Brackets are used as the array constructor.

Concrete syntax:

[1,2,3]

AST:

nnkBracket(nnkIntLit(1),nnkIntLit(2),nnkIntLit(3))

Ranges

Ranges occur in set constructors, case statement branches, or array slices. Internally, the node kind nnkRange is used, but when constructing the AST, construction with .. as an infix operator should be used instead.

Concrete syntax:

1..3

AST:

nnkInfix(nnkIdent(".."),nnkIntLit(1),nnkIntLit(3))

Example code:

macrogenRepeatEcho()=result=newNimNode(nnkStmtList)varforStmt=newNimNode(nnkForStmt)# generate a for statementforStmt.add(ident("i"))# use the variable `i` for iterationvarrangeDef=newNimNode(nnkInfix).add(ident("..")).add(newIntLitNode(3),newIntLitNode(5))# iterate over the range 3..5forStmt.add(rangeDef)forStmt.add(newCall(ident("echo"),newIntLitNode(3)))# meat of the loopresult.add(forStmt)genRepeatEcho()# gives:# 3# 3# 3

If expression

The representation of the if expression is subtle, but easy to traverse.

Concrete syntax:

ifcond1:expr1elifcond2:expr2else:expr3

AST:

nnkIfExpr(nnkElifExpr(cond1,expr1),nnkElifExpr(cond2,expr2),nnkElseExpr(expr3))

Documentation Comments

Double-hash (##) comments in the code actually have their own format, using strVal to get and set the comment text. Single-hash (#) comments are ignored.

Concrete syntax:

## This is a comment## This is part of the first commentstmt1## Yet another

AST:

nnkCommentStmt()# only appears once for the first two lines!stmt1nnkCommentStmt()# another nnkCommentStmt because there is another comment# (separate from the first)

Pragmas

One of Nim's cool features is pragmas, which allow fine-tuning of various aspects of the language. They come in all types, such as adorning procs and objects, but the standalone emit pragma shows the basics with the AST.

Concrete syntax:

{.emit:"#include <stdio.h>".}

AST:

nnkPragma(nnkExprColonExpr(nnkIdent("emit"),nnkStrLit("#include <stdio.h>")# the "argument"))

As many nnkIdent appear as there are pragmas between {..}. Note that the declaration of new pragmas is essentially the same:

Concrete syntax:

{.pragma:cdeclRename,cdecl.}

AST:

nnkPragma(nnkExprColonExpr(nnkIdent("pragma"),# this is always first when declaring a new pragmannkIdent("cdeclRename")# the name of the pragma),nnkIdent("cdecl"))

Statements

If statement

The representation of the if statement is subtle, but easy to traverse. If there is no else branch, no nnkElse child exists.

Concrete syntax:

ifcond1:stmt1elifcond2:stmt2elifcond3:stmt3else:stmt4

AST:

nnkIfStmt(nnkElifBranch(cond1,stmt1),nnkElifBranch(cond2,stmt2),nnkElifBranch(cond3,stmt3),nnkElse(stmt4))

When statement

Like the if statement, but the root has the kind nnkWhenStmt.

Assignment

Concrete syntax:

x=42

AST:

nnkAsgn(nnkIdent("x"),nnkIntLit(42))

This is not the syntax for assignment when combined with var, let, or const.

Statement list

Concrete syntax:

stmt1stmt2stmt3

AST:

nnkStmtList(stmt1,stmt2,stmt3)

Case statement

Concrete syntax:

caseexpr1ofexpr2,expr3..expr4:stmt1ofexpr5:stmt2elifcond1:stmt3else:stmt4

AST:

nnkCaseStmt(expr1,nnkOfBranch(expr2,nnkRange(expr3,expr4),stmt1),nnkOfBranch(expr5,stmt2),nnkElifBranch(cond1,stmt3),nnkElse(stmt4))

The nnkElifBranch and nnkElse parts may be missing.

While statement

Concrete syntax:

whileexpr1:stmt1

AST:

nnkWhileStmt(expr1,stmt1)

For statement

Concrete syntax:

forident1,ident2inexpr1:stmt1

AST:

nnkForStmt(ident1,ident2,expr1,stmt1)

Try statement

Concrete syntax:

try:stmt1excepte1,e2:stmt2excepte3:stmt3except:stmt4finally:stmt5

AST:

nnkTryStmt(stmt1,nnkExceptBranch(e1,e2,stmt2),nnkExceptBranch(e3,stmt3),nnkExceptBranch(stmt4),nnkFinally(stmt5))

Return statement

Concrete syntax:

returnexpr1

AST:

nnkReturnStmt(expr1)

Yield statement

Like return, but with nnkYieldStmt kind.

nnkYieldStmt(expr1)

Discard statement

Like return, but with nnkDiscardStmt kind.

nnkDiscardStmt(expr1)

Continue statement

Concrete syntax:

continue

AST:

nnkContinueStmt()

Break statement

Concrete syntax:

breakotherLocation

AST:

nnkBreakStmt(nnkIdent("otherLocation"))

If break is used without a jump-to location, nnkEmpty replaces nnkIdent.

Block statement

Concrete syntax:

blockname:

AST:

nnkBlockStmt(nnkIdent("name"),nnkStmtList(...))

A block doesn't need an name, in which case nnkEmpty is used.

Asm statement

Concrete syntax:

asm""" some asm """

AST:

nnkAsmStmt(nnkEmpty(),# for pragmasnnkTripleStrLit("some asm"),)

Import section

Nim's import statement actually takes different variations depending on what keywords are present. Let's start with the simplest form.

Concrete syntax:

importstd/math

AST:

nnkImportStmt(nnkIdent("math"))

With except, we get nnkImportExceptStmt.

Concrete syntax:

importstd/mathexceptpow

AST:

nnkImportExceptStmt(nnkIdent("math"),nnkIdent("pow"))

Note that import std/math as m does not use a different node; rather, we use nnkImportStmt with as as an infix operator.

Concrete syntax:

importstd/strutilsassu

AST:

nnkImportStmt(nnkInfix(nnkIdent("as"),nnkIdent("strutils"),nnkIdent("su")))

From statement

If we use from ... import, the result is different, too.

Concrete syntax:

fromstd/mathimportpow

AST:

nnkFromStmt(nnkIdent("math"),nnkIdent("pow"))

Using from std/math as m import pow works identically to the as modifier with the import statement, but wrapped in nnkFromStmt.

Export statement

When you are making an imported module accessible by modules that import yours, the export syntax is pretty straightforward.

Concrete syntax:

exportunsigned

AST:

nnkExportStmt(nnkIdent("unsigned"))

Similar to the import statement, the AST is different for export ... except.

Concrete syntax:

exportmathexceptpow# we're going to implement our own exponentiation

AST:

nnkExportExceptStmt(nnkIdent("math"),nnkIdent("pow"))

Include statement

Like a plain import statement but with nnkIncludeStmt.

Concrete syntax:

includeblocks

AST:

nnkIncludeStmt(nnkIdent("blocks"))

Var section

Concrete syntax:

vara=3

AST:

nnkVarSection(nnkIdentDefs(nnkIdent("a"),nnkEmpty(),# or nnkIdent(...) if the variable declares the typennkIntLit(3),))

Note that either the second or third (or both) parameters above must exist, as the compiler needs to know the type somehow (which it can infer from the given assignment).

This is not the same AST for all uses of var. See Procedure declaration for details.

Let section

This is equivalent to var, but with nnkLetSection rather than nnkVarSection.

Concrete syntax:

leta=3

AST:

nnkLetSection(nnkIdentDefs(nnkIdent("a"),nnkEmpty(),# or nnkIdent(...) for the typennkIntLit(3),))

Const section

Concrete syntax:

consta=3

AST:

nnkConstSection(nnkConstDef(# not nnkConstDefs!nnkIdent("a"),nnkEmpty(),# or nnkIdent(...) if the variable declares the typennkIntLit(3),# required in a const declaration!))

Type section

Starting with the simplest case, a type section appears much like var and const.

Concrete syntax:

typeA=int

AST:

nnkTypeSection(nnkTypeDef(nnkIdent("A"),nnkEmpty(),nnkIdent("int")))

Declaring distinct types is similar, with the last nnkIdent wrapped in nnkDistinctTy.

Concrete syntax:

typeMyInt=distinctint

AST:

# ...nnkTypeDef(nnkIdent("MyInt"),nnkEmpty(),nnkDistinctTy(nnkIdent("int")))

If a type section uses generic parameters, they are treated here:

Concrete syntax:

typeA[T]=expr1

AST:

nnkTypeSection(nnkTypeDef(nnkIdent("A"),nnkGenericParams(nnkIdentDefs(nnkIdent("T"),nnkEmpty(),# if the type is declared with options, like# ``[T: SomeInteger]``, they are given herennkEmpty(),))expr1,))

Note that not all nnkTypeDef utilize nnkIdent as their parameter. One of the most common uses of type declarations is to work with objects.

Concrete syntax:

typeIO=objectofRootObj

AST:

# ...nnkTypeDef(nnkIdent("IO"),nnkEmpty(),nnkObjectTy(nnkEmpty(),# no pragmas herennkOfInherit(nnkIdent("RootObj")# inherits from RootObj),nnkEmpty()))

Nim's object syntax is rich. Let's take a look at an involved example in its entirety to see some of the complexities.

Concrete syntax:

typeObj[T]{.inheritable.}=objectname:stringcaseisFat:booloftrue:m:array[100_000,T]offalse:m:array[10,T]

AST:

# ...nnkPragmaExpr(nnkIdent("Obj"),nnkPragma(nnkIdent("inheritable"))),nnkGenericParams(nnkIdentDefs(nnkIdent("T"),nnkEmpty(),nnkEmpty())),nnkObjectTy(nnkEmpty(),nnkEmpty(),nnkRecList(# list of object parametersnnkIdentDefs(nnkIdent("name"),nnkIdent("string"),nnkEmpty()),nnkRecCase(# case statement within object (not nnkCaseStmt)nnkIdentDefs(nnkIdent("isFat"),nnkIdent("bool"),nnkEmpty()),nnkOfBranch(nnkIdent("true"),nnkRecList(# again, a list of object parametersnnkIdentDefs(nnkIdent("m"),nnkBracketExpr(nnkIdent("array"),nnkIntLit(100000),nnkIdent("T")),nnkEmpty())),nnkOfBranch(nnkIdent("false"),nnkRecList(nnkIdentDefs(nnkIdent("m"),nnkBracketExpr(nnkIdent("array"),nnkIntLit(10),nnkIdent("T")),nnkEmpty()))))))

Using an enum is similar to using an object.

Concrete syntax:

typeX=enumFirst

AST:

# ...nnkEnumTy(nnkEmpty(),nnkIdent("First")# you need at least one nnkIdent or the compiler complains)

The usage of concept (experimental) is similar to objects.

Concrete syntax:

typeCon=conceptx,y,z(x&y&z)isstring

AST:

# ...nnkTypeClassTy(# note this isn't nnkConceptTy!nnkArgList(# ... idents for x, y, z)# ...)

Static types, like static[int], use nnkIdent wrapped in nnkStaticTy.

Concrete syntax:

typeA[T:static[int]]=object

AST:

# ... within nnkGenericParamsnnkIdentDefs(nnkIdent("T"),nnkStaticTy(nnkIdent("int")),nnkEmpty())# ...

In general, declaring types mirrors this syntax (i.e., nnkStaticTy for static, etc.). Examples follow (exceptions marked by *):

Take special care when declaring types as proc. The behavior is similar to Procedure declaration, below, but does not treat nnkGenericParams. Generic parameters are treated in the type, not the proc itself.

Concrete syntax:

typeMyProc[T]=proc(x:T){.nimcall.}

AST:

# ...nnkTypeDef(nnkIdent("MyProc"),nnkGenericParams(# here, not with the proc# ...)nnkProcTy(# behaves like a procedure declaration from here onnnkFormalParams(# ...),nnkPragma(nnkIdent("nimcall"))))

The same syntax applies to iterator (with nnkIteratorTy), but does not apply to converter or template.

Type class versions of these nodes generally share the same node kind but without any child nodes. The tuple type class is represented by nnkTupleClassTy, while a proc or iterator type class with pragmas has an nnkEmpty node in place of the nnkFormalParams node of a concrete proc or iterator type node.

typeTypeClass=proc{.nimcall.}|ref|tuple

AST:

nnkTypeDef(nnkIdent("TypeClass"),nnkEmpty(),nnkInfix(nnkIdent("|"),nnkProcTy(nnkEmpty(),nnkPragma(nnkIdent("nimcall"))),nnkInfix(nnkIdent("|"),nnkRefTy(),nnkTupleClassTy())))

Mixin statement

Concrete syntax:

mixinx

AST:

nnkMixinStmt(nnkIdent("x"))

Bind statement

Concrete syntax:

bindx

AST:

nnkBindStmt(nnkIdent("x"))

Procedure declaration

Let's take a look at a procedure with a lot of interesting aspects to get a feel for how procedure calls are broken down.

Concrete syntax:

prochello*[T:SomeInteger](x:int=3,y:float32):int{.inline.}=discard

AST:

nnkProcDef(nnkPostfix(nnkIdent("*"),nnkIdent("hello")),# the exported proc namennkEmpty(),# patterns for term rewriting in templates and macros (not procs)nnkGenericParams(# generic type parameters, like with type declarationnnkIdentDefs(nnkIdent("T"),nnkIdent("SomeInteger"),nnkEmpty())),nnkFormalParams(nnkIdent("int"),# the first FormalParam is the return type. nnkEmpty() if there is nonennkIdentDefs(nnkIdent("x"),nnkIdent("int"),# type type (required for procs, not for templates)nnkIntLit(3)# a default value),nnkIdentDefs(nnkIdent("y"),nnkIdent("float32"),nnkEmpty())),nnkPragma(nnkIdent("inline")),nnkEmpty(),# reserved slot for future usennkStmtList(nnkDiscardStmt(nnkEmpty()))# the meat of the proc)

There is another consideration. Nim has flexible type identification for its procs. Even though proc(a: int, b: int) and proc(a, b: int) are equivalent in the code, the AST is a little different for the latter.

Concrete syntax:

proc(a,b:int)

AST:

# ...AST as above...nnkFormalParams(nnkEmpty(),# no return herennkIdentDefs(nnkIdent("a"),# the first parameternnkIdent("b"),# directly to the second parameternnkIdent("int"),# their shared type identifiernnkEmpty(),# default value would go here)),# ...

When a procedure uses the special var type return variable, the result is different from that of a var section.

Concrete syntax:

prochello():varint

AST:

# ...nnkFormalParams(nnkVarTy(nnkIdent("int")))

Iterator declaration

The syntax for iterators is similar to procs, but with nnkIteratorDef replacing nnkProcDef.

Concrete syntax:

iteratornonsense[T](x:seq[T]):float{.closure.}=...

AST:

nnkIteratorDef(nnkIdent("nonsense"),nnkEmpty(),...)

Converter declaration

A converter is similar to a proc.

Concrete syntax:

convertertoBool(x:float):bool

AST:

nnkConverterDef(nnkIdent("toBool"),# ...)

Template declaration

Templates (as well as macros, as we'll see) have a slightly expanded AST when compared to procs and iterators. The reason for this is term-rewriting macros. Notice the nnkEmpty() as the second argument to nnkProcDef and nnkIteratorDef above? That's where the term-rewriting macros go.

Concrete syntax:

templateoptOpt{expr1}(a:int):int

AST:

nnkTemplateDef(nnkIdent("optOpt"),nnkStmtList(# instead of nnkEmpty()expr1),# follows like a proc or iterator)

If the template does not have types for its parameters, the type identifiers inside nnkFormalParams just becomes nnkEmpty.

Macro declaration

Macros behave like templates, but nnkTemplateDef is replaced with nnkMacroDef.

Hidden Standard Conversion

varf:float=1

The type of "f" is float but the type of "1" is actually int. Inserting int into a float is a type error. Nim inserts the nnkHiddenStdConv node around the nnkIntLit node so that the new node has the correct type of float. This works for any auto converted nodes and makes the conversion explicit.

Special node kinds

There are several node kinds that are used for semantic checking or code generation. These are accessible from this module, but should not be used. Other node kinds are especially designed to make AST manipulations easier. These are explained here.

To be written.