Learning the syntax

Robert-Egan

A very fundamental question. I want to parse a SQL statement that starts with "CREATE SCHEMA", and obviously I am doing something fundamentally wrong. Here is the smallest possible use case I can make to demonstrated the issue.

TOKEN:
    <SqlCreate: "CREATE">
    <SqlSchema: "SCHEMA"> |
;

// Accept "CREATE SCHEMA" ???

SchemaDefinition:
    <SqlCreate> <SqlSchema>
;

And the test code


public class SchemaTest {

    public static void main(String [] args) throws Exception {
        new SchemaParser("CREATE SCHEMA").SchemaDefinition();
    }
}

And the error message

Exception in thread "main" schemaparser.ParseException:
Encountered an error at (or somewhere around) input:1:7
Was expecting one of the following:
SqlSchema
Found string " " of type INVALID
        at schemaparser.SchemaParser.handleUnexpectedTokenType(SchemaParser.java:262)
        at schemaparser.SchemaParser.consumeToken(SchemaParser.java:249)
        at schemaparser.SchemaParser.SchemaDefinition(SchemaParser.java:183)
        at schemaparser.SchemaTest.main(SchemaTest.java:6)

vsajip

Try this:

SKIP : <WHITESPACE : (" "| "\t"| "\n"| "\r")+>;

TOKEN:
    <SqlCreate: "CREATE"> |
    <SqlSchema: "SCHEMA">
;

SchemaDefinition:
    <SqlCreate> <SqlSchema>
;

Without the SKIP, it won't know what to do with the space (as indicated in the error message).

Robert-Egan

vsajip That makes sense, thanks.

Robert-Egan

Addendum. At first it did not work, but that's because I had WHITESPACE defined as a private (#). Seems odd that I did not get an error message during parser generation, because I have gotten it in other places. Perhaps because SKIP: follows it's own logic? Anyway, not a show stopper, more of a housekeeping thing I'm sure.

revusky

Robert-Egan Something being a private regexp, i.e. prefixed with # actually means that it is only used as a macro basically, to build up more complex regexp's.

Actually, it would probably be a good idea if there was a warning when you had such a "private" regexp that is not used anywhere, i.e. as a building block in another regexp. But currently there is no such warning.

As for what SKIP means, that just means that it matches the pattern but does not roll up a Token object from it. The input that matches the pattern is skipped basically. UNPARSED means that it does roll up a token object but it is ignored in terms of the syntactic grammar. In practice, SKIP is usually used for ignorable whitespace, while UNPARSED is used for comments. Typically anyway...

Anyway, don't be shy about asking questions.