lexer_builder 0.1.3
SDKdart
Platformwindowslinuxmacos
Generate lexer code from regular expression rules. Arbitrary custom tokens are supported with generics.
lexer_builder
A lexer generator for dart.
Features
- Match tokens using RegExp syntax.
- Rules dependent on lexer state.
- Generates lexer code automatically.
Caveats
- Generated lexers using regex are likely slower than handwritten ones.
Getting started
Include lexer_builder and build_runner as dev_dependencies and lexer_builder_runtime in your pubspec.yml.
Usage
Annotate a class with Lexer() and methods in it with Rule() to define a lexer.
See the example for detailed instructions.
TODO
- Eventually generate custom code for the rules instead of using RegExp internally.
- Support async lexing with that by accurately measuring partly-matched subgroups, dispatching a rule if the next character doesn't make any match longer.
Additional information
For more information about lexers in general, see the flex lexer generator and its documentation.