Token

This type is used when the token is invalid or its type cannot be determined because of the ambiguous context. Further analysis might be required to detect its type.

SQL specific keywords: SELECT, UPDATE, INSERT, etc.

Any type of legal operator.

Arithmetic operators: +, -, *, /, etc. Logical operators: ===, <>, !==, etc. Bitwise operators: &, |, ^, etc. Assignment operators: =, +=, -=, etc. SQL specific operators: . (e.g. .. WHERE database.table ..),

• (e.g. SELECT * FROM ..)

Spaces, tabs, new lines, etc.

Any type of legal comment.

Bash (#), C (/ \/) or SQL (--) comments:

 -- SQL-comment

 #Bash-like comment

 /*C-like comment*\/

or:

 /*C-like
   comment*\/

Backslashes were added to respect PHP's comments syntax.

Boolean values: true or false.

Numbers: 4, 0x8, 15.16, 23e42, etc.

Literal strings: 'string', "test".

Some of these strings are actually symbols.

Database, table names, variables, etc.

For example: SELECT `foo`, `bar` FROM `database`.`table`;.

Delimits an unknown string.

For example: SELECT * FROM test;, test is a delimiter.

Labels in LOOP statement, ITERATE statement etc.

For example (only for begin label): begin_label: BEGIN [statement_list] END [end_label] begin_label: LOOP [statement_list] END LOOP [end_label] begin_label: REPEAT [statement_list] ... END REPEAT [end_label] begin_label: WHILE ... DO [statement_list] END WHILE [end_label].

The token it its raw string representation.

The value this token contains (i.e. token after some evaluation).

The keyword value this token contains, always uppercase.

The type of this token.

The flags of this token.

The position in the initial string where this token started.