Datasheet

Tokens
Token is the smallest element of the Pascal program that compiler can recognize.
The parser separates tokens from the input stream by creating the longest token
possible using the input characters in a left–to–right scan.
mikroPascal PRO for AVR recognizes the following kinds of tokens:
- keywords
- identifiers
- constants
- operators
- punctuators (also known as separators)
Token Extraction Example
Here is an example of token extraction. Take a look at the following example code
sequence:
end_flag := 0;
First, note that end_flag would be parsed as a single identifier, rather than as the
keyword end followed by the identifier _flag.
The compiler would parse it as the following four tokens:
end_flag // variable identifier
:= // assignment operator
0 // literal
; // statement terminator
Note that := parses as one token (the longest token possible), not as token : fol-
lowed by token =.
122
MIKROELEKTRONIKA - SOFTWARE AND HARDWARE SOLUTIONS FOR EMBEDDED WORLD
Language Reference
mikroPASCAL PRO for AVR
CHAPTER 5