How to recover from parser error?

[durianwaffle]

In our grammar, if there is parser error such as a random string without semicolon, the validator will fail for the current document.
Any solution on that?

Can I define a rule to parse the random strings in the grammar? If so, how should define this rule?

[msujew]

Hey @durianwaffle,

without diving to deep into parsing theory: There are some kinds of parser errors that are hard to recover from, due to the way how recursive descent parsers work. You're encountering one because the parser fails to perform its lookahead function, which in turn prevents the parser from identifying which rule to parse.

Usually, the parser then attempts to skip tokens to recover from this error, but this doesn't always work.

Can I define a rule to parse the random strings in the grammar? If so, how should define this rule?

I don't think that's possible, at least not to my knowledge. I would assume that even if you were able to define such a rule, you would likely run into unexpected side effects.

[durianwaffle]

Hi Mark, why the parser fails to perform its lookahead function?

And do you think it's possible that I parse the documents before parser error and after parser error individually to validator?

[durianwaffle]

I found that after this error, which is hard to recover from, Langium fails to parse the code below.
And the validator will not execute at all on that document.

Do you know what're the functions that I can use to validate the document that doesn't contain those errors?

[durianwaffle]

For example, in Xtext, if there are errors that can't recover from, the validator will show red squiggle lines for all the code below that error by default. But the tokenizer and parser will fail to parse the code after that error same as in Langium.

Can langium also implement that?

[msujew]

why the parser fails to perform its lookahead function?

Because the lookahead is instantly aborted once a token is identified that doesn't fit any of the given alternatives. See also here.

And do you think it's possible that I parse the documents before parser error and after parser error individually to validator?

No, because the entry point of the grammar already appeared, and so it will just result in more errors for all non-trivial grammars.

Do you know what're the functions that I can use to validate the document that doesn't contain those errors?

I guess you would need to look at the errors in the document? I'm not entirely sure.

Can langium also implement that?

Maybe? I need to take a look at what Chevrotain provides us as error messages to figure out whether parsing succeeded until the end of the document. If something like that isn't available, maybe we figure it out by looking at the last parsed token in relation to the input text.