Datasheet

TOKENS
Token is the smallest element of a C program that compiler can recognize. The
parser separates tokens from the input stream by creating the longest token possi-
ble using the input characters in a left–to–right scan.
The mikroC PRO for AVR recognizes the following kinds of tokens:
- keywords
- identifiers
- constants
- operators
- punctuators (also known as separators)
Tokens can be concatenated (pasted) by means of the preprocessor operator
##.
See the Preprocessor Operators for details.
Token Extraction Example
Here is an example of token extraction. Take a look at the following example code
sequence:
inter = a+++b;
First, note that inter would be parsed as a single identifier, rather than as the key-
word
int followed by the identifier er.
The programmer who has written the code might have intended to write inter = a
+ (++b)
, but it wouldn’t work that way. The compiler would parse it into the seven
following tokens:
inter // variable identifier
= // assignment operator
a // variable identifier
++ // postincrement operator
+ // addition operator
b // variable identifier
; // statement terminator
Note that +++ parses as ++ (the longest token possible) followed by +.
According to the operator precedence rules, our code sequence is actually:
inter (a++)+b;
125
MIKROELEKTRONIKA - SOFTWARE AND HARDWARE SOLUTIONS FOR EMBEDDED WORLD
Language Reference
mikroC PRO for AVR
CHAPTER 5