What I mean by "reaching end of line" is that the feature set is frozen. In the last few weeks, I announced some new features, some of them rather trivial. For example, you can use single-quotes in regexp definitions. As of yesterday, you can redefine a token definition, which is not really so trivial!
So, what I mean to announce is that, as of now, there won't be any more such new features or fix-ups. Or, IOW, whatever limitations or warts the current JavaCC 21 has will only be addressed in CongoCC. This will probably take some self-discipline on my part, because some things are so easy to do, like the single-quotes thing was, that it is tempting to just do it.
There are some glaring glitches in how certain things work actually. For example, here is a flaw in JavaCC 21 that I have been aware of for a while. You can write, for example:
ASSERT {someCondition}
and the assertion is checked when parsing, but not in lookahead. You can write:
ASSERT {someCondition}#
or:
SCAN {someCondition}# => SomeExpansion
to indicate that the condition is checked in parsing and in lookahead. However, I didn't provide a way of specifying that a lookahead or assertion is used in lookahead but not in parsing. The result of this is mostly that there are surely plenty of cases where things are checked more than once unnecessarily. I guess one solution would be to use a double hash to indicate that the condition is checked in lookahead but not in parsing.
Well, there actually is a way of specifying that the condition (expressed in Java code) only applies in lookahead. You can write:
SCAN {currentLookaheadToken == null || someCondition}# => SomeExpansion
So the condition always returns true if you are not in a lookahead. But that is not so great because it requires some knowledge of internals that should be unnecessary and there is the question of having to drop into Java code, when in principle, CongoCC generates Python and CSharp as well. And then there are other things that you cannot specify currently.
But I've decided that I'm only going to turn my attention to this after the Congo rebranding.
There is also this issue of a very verbose pattern, where you write:
ACTIVATE_TOKENS YIELD (<YIELD>)
for contextual keywords. This is really crying out for some abbreviated syntax. But I'm not going to address this until the Congo rebranding.
I am pretty sure that this problem I mentioned before of a SCAN without any numerial or syntactic lookahead being taken to be an unlimited scanahead, that this is basically a bug -- or not quite a bug, but not really desirable, I don't think. But I'll fix it in Congo.
SCAN {someCondition} => SomeExpansion
I think will mean that we check the condition AND one token ahead. NOT scanning ahead through the whole SomeExpansion. I anticipate that I'll add a setting kind of similar to LEGACY_GLITCHY_LOOKAHEAD that turns on the legacy behavior.
But anyway, I think the intention now is just to freeze the current feature set (and bug set!) and anything to be addressed will be addressed after the Congo renaming. I really think the Congo renaming should take place in the next month or so. Right now, I've got the Github repository https://github.com/congo-cc/congo-parser-generator but it is currently not publicly visible. I hope to be able to unveil it in a month maybe. Or not much more than that. And again, the intention is to present CongoCC as something new. That it is the result of a rebranding of JavaCC 21, which is, in turn, was a total rewrite of the old JavaCC, that will not be a secret or anything, but more like a historical footnote. I would be perfectly happy if, in a year or two, the vast majority of end users (many of them Python and CSharp people) don't even know the history of the thing.