class Token

A structure representing a lexeme that explicitly indicates its categorization for the purpose of parsing.

Constants

TYPE_NONE

This type is used when the token is invalid or its type cannot be determined because of the ambiguous context. Further analysis might be required to detect its type.

TYPE_KEYWORD

SQL specific keywords: SELECT, UPDATE, INSERT, etc.

TYPE_OPERATOR

Any type of legal operator.

Arithmetic operators: +, -, *, /, etc. Logical operators: ===, <>, !==, etc. Bitwise operators: &, |, ^, etc. Assignment operators: =, +=, -=, etc. SQL specific operators: . (e.g. .. WHERE database.table ..), * (e.g. SELECT * FROM ..)

TYPE_WHITESPACE

Spaces, tabs, new lines, etc.

TYPE_COMMENT

Any type of legal comment.

Bash (#), C (/* *\/) or SQL (--) comments:

 -- SQL-comment

 #Bash-like comment

 /*C-like comment*\/

or:

 /*C-like
   comment*\/

Backslashes were added to respect PHP's comments syntax.

TYPE_BOOL

Boolean values: true or false.

TYPE_NUMBER

Numbers: 4, 0x8, 15.16, 23e42, etc.

TYPE_STRING

Literal strings: 'string', "test".

Some of these strings are actually symbols.

TYPE_SYMBOL

Database, table names, variables, etc.

For example: SELECT `foo`, `bar` FROM `database`.`table`;.

TYPE_DELIMITER

Delimits an unknown string.

For example: SELECT * FROM test;, test is a delimiter.

TYPE_LABEL

Labels in LOOP statement, ITERATE statement etc.

For example (only for begin label): begin_label: BEGIN [statement_list] END [end_label] begin_label: LOOP [statement_list] END LOOP [end_label] begin_label: REPEAT [statement_list] ... END REPEAT [end_label] begin_label: WHILE ... DO [statement_list] END WHILE [end_label].

FLAG_KEYWORD_RESERVED

FLAG_KEYWORD_COMPOSED

FLAG_KEYWORD_DATA_TYPE

FLAG_KEYWORD_KEY

FLAG_KEYWORD_FUNCTION

FLAG_NUMBER_HEX

FLAG_NUMBER_FLOAT

FLAG_NUMBER_APPROXIMATE

FLAG_NUMBER_NEGATIVE

FLAG_NUMBER_BINARY

FLAG_STRING_SINGLE_QUOTES

FLAG_STRING_DOUBLE_QUOTES

FLAG_COMMENT_BASH

FLAG_COMMENT_C

FLAG_COMMENT_SQL

FLAG_COMMENT_MYSQL_CMD

FLAG_OPERATOR_ARITHMETIC

FLAG_OPERATOR_LOGICAL

FLAG_OPERATOR_BITWISE

FLAG_OPERATOR_ASSIGNMENT

FLAG_OPERATOR_SQL

FLAG_SYMBOL_VARIABLE

FLAG_SYMBOL_BACKTICK

FLAG_SYMBOL_USER

FLAG_SYMBOL_SYSTEM

FLAG_SYMBOL_PARAMETER

Properties

string $token The token it its raw string representation.
mixed $value The value this token contains (i.e. token after some evaluation).
mixed $keyword The keyword value this token contains, always uppercase.
int $type The type of this token.
int $flags The flags of this token.
int $position The position in the initial string where this token started.

Methods

__construct(string $token, int $type = 0, int $flags = 0)

Constructor.

mixed
extract()

Does little processing to the token to extract a value.

string
getInlineToken()

Converts the token into an inline token by replacing tabs and new lines.

Details

at line 224
__construct(string $token, int $type = 0, int $flags = 0)

Constructor.

Parameters

string $token the value of the token
int $type the type of the token
int $flags the flags of the token

at line 240
mixed extract()

Does little processing to the token to extract a value.

If no processing can be done it will return the initial string.

Return Value

mixed

at line 330
string getInlineToken()

Converts the token into an inline token by replacing tabs and new lines.

Return Value

string