User manual

mikoPascal PRO for PIC32

MikroElektronika

Tokens

Token is the smallest element of a mikroPascal PRO for PIC32 program, meaningful to the compiler. The parser

separates tokens from the input stream by creating the longest token possible using the input characters in a left–to–

right scan.

mikroPascal PRO for PIC32 recognizes the following kinds of tokens:

- keywords

- identiers

- constants

- operators

- punctuators (also known as separators)

Token Extraction Example

Here is an example of token extraction. Take a look at the following example code sequence:

end_ag := 0;

First, note that end_ag would be parsed as a single identier, rather than as the keyword end followed by the identier

_ag.

The compiler would parse it as the following four tokens:

end_ag // variable identier

:= // assignment operator

0 // literal

; // statement terminator

Note that := parses as one token (the longest token possible), not as token : followed by token =.