Class Tokenizer

Tokenizer, which split the code into single tokens

Inheritance
System.Object
Tokenizer
Inherited Members
System.Object.ToString()
System.Object.Equals(System.Object)
System.Object.Equals(System.Object, System.Object)
System.Object.ReferenceEquals(System.Object, System.Object)
System.Object.GetHashCode()
System.Object.GetType()
System.Object.MemberwiseClone()
Namespace: Simplic.EPL.Lexer
Assembly: Simplic.EPL.dll
Syntax
public class Tokenizer

Constructors

| Improve this Doc View Source

Tokenizer(IEPLLexerConstants, IErrorListener)

Create

Declaration
public Tokenizer(IEPLLexerConstants lexerConstants, IErrorListener errorListener)
Parameters
Type Name Description
IEPLLexerConstants lexerConstants

Lexer constants / config
IErrorListener errorListener

error listener for error capturing

Properties

| Improve this Doc View Source

Tokens

Tokens from the current tokenizer

Declaration
public Dequeue<RawToken> Tokens { get; set; }
Property Value
Type Description
Dequeue<RawToken>

Methods

| Improve this Doc View Source

Parse(String)

Parse script into tokens

Declaration
public void Parse(string Code)
Parameters
Type Name Description
System.String Code