Very Unofficial Parser

Fan-made, handmade, recursive-descent parser for the Dart programming language.

Although this parser strives to parse the language according to the language specification, no guarantees can be made that it is perfect. Filing an issue, sending us valid code that should parse (but doesn't), and opening pull requests is all greatly appreciated — if you're into this sort of thing.

👷 Note: Implementing a parser for the Dart programming language is a large undertaking. As such, this project is still under development. Many of the basic tokens in the lexer have already been implemented, and the basic skeleton for parsing expressions with respect for operator precedence has also been constructed. For a full list of everything that has been accomplished, see below.

If you want development to finish faster, please help!

Features

A checklist of the implemented features is provided below.

Lexer

Comments

• Single line comments
• Documentation comments
• Multiline comments

Numbers

• Hexadecimal numbers
• Decimal numbers (w/ exponents, etc)

Strings

• Raw single line strings.
• Raw multiline strings.
• Single line strings with simple string interpolation.
• Single line strings with complex string interpolation.
• Multiline strings with simple string interpolation.
• Multiline strings with complex string interpolation.

Misc

• Whitespace
• Newlines
• Operators
• Braces, brackets, parenthesis, etc.

Parser

Expressions

• Operator precedence parsing

Primitives

• Numbers
• Booleans
• Raw strings, single and multiline simple interpolated strings
• Single and multiline strings with complex interpolation (in progress)

Misc

• Special handling for trivia tokens

Dart Analyzer and Historical Issues

Dart's analyzer API's are tied to certain platforms and not technically intended for public use (although that never stopped anyone).

The analyzer package README says the following:

The APIs in this package were originally machine generated by a translator and were based on an earlier Java implementation. Several of the API's still look like their Java predecessors rather than clean Dart APIs.

In addition, there is currently no clean distinction between public and internal APIs. We plan to address this issue but doing so will, unfortunately, require a large number of breaking changes. We will try to minimize the pain this causes for our clients, but some pain is inevitable.

No one wants to use a dependency that promises "pain is inevitable." If we liked pain, we'd write our own parsers. Er, okay — but that's not the point. Either way, it'd be nice if there was a lightweight alternative to the analyzer for situations where the analyzer is too cumbersome (or overkill).

Currently, if you need to programmatically analyze Dart source code (with or without fully resolved types), you have no choice but to use the analyzer packages. In my experience, the analyzer is most often encountered when creating source generators for build_runner (which is notoriously slow and heavy) or creating custom lint rules by working with the analyzer plugin system directly (via unofficial practices). To make matters worse, the analyzer system completely obliterates the computer's memory usage when loading custom plugins, which makes using custom lint rules nearly impossible in any substantial project.

If the analyzer is improved substantially within our lifetime, this project probably won't be needed.

Potential Uses

Wondering what could be done with this once it's finished? Allow me to offer a few ideas, roughly ranked from "barely reasonable" to "rather implausible":

• Create automatic code migration tools.
• Create a lightweight source generator or linting system that provides just an AST (no fully resolved types) for the sake of simplicity and performance.
• Create a transpiler — i.e., compile Dart code to another language.
• Create an interpreter so you can run Dart inside Dart.
• Create a compiler, in case you don't like the official one.
• etc.

Contributing

We would love your help! If you like pain love making programming languages, you might just enjoy converting the Dart language specification into code. Please strive for readability over conciseness, and don't forget the tests!

Source code for recursive descent parsers are really easy to read and maintain. Handwriting the parser and node descriptions helps ensure they are ergonomic and easy to work with.

Credits

You're probably wondering who would do a thing like this. Me, apparently—but I blame Bob. In fact, if Bob's book Crafting Interpreters hadn't made it so easy to understand how to build a programming language, I might have gotten a lot more sleep.

Programming Language Theory

In case you're wondering what this is all about, here's a list of some helpful books and guides on some programming language topics.

• Crafting Interpreters by Bob Nystrom
• Pratt Parsing: Expression Parsing Made Easy by Bob Nystrom
• Writing an Interpreter in Go by Thorsten Ball
• Writing a Compiler in Go by Thorsten Ball
• Compilers: Principles, Techniques, and Tools by Alfred V. Aho, Ravi Sethi, and Jeffrey D. Ullman (The Dragon Book — both editions are great).
• Modern Compiler Implementation in Java by Andrew W. Appel

Made with blood, sweat, and tears by 🐴 Very Unofficial Ventures.