bnaudts As my Java modification analyzer ‘maddi’ (https://github.com/CodeLaser/maddi) is approaching production-level quality, I’d like to explain how CongoCC’s Java parser is now embedded in maddi’s front-end.
Hi Bart. Thanks for the note. It's good to hear back from people about their usage of CongoCC. I wish more people would give us some feedback like this!
bnaudts I used to have a home-grown type and method resolver that ran on a syntax tree parsed by CongoCC, but given the complexity of Java and time-pressure from my side, I had to decommission it in favor of the “guaranteed to do the correct thing” compiler itself.
Well, the Java parser generated by CongoCC is just a parser finally. It doesn't do anything beyond parsing the source code into an AST. So it won't check whether a variable is defined or is of the appropriate type in the context and so on. So, if you need that functionality, it is somewhere (must be!) in the jdk.compiler module (javac basically) and our Java parser is just a parser and simply doesn't do that. I would say that if all you want to do is parse Java source code, there is no particular reason to think that our Java parser is any less reliable than even javac itself. It is a pretty faithful implementation of the Java language spec. But, of course, anything beyond just parsing is really beyond the scope of what it does.
One comment I'd make about the situation is that the javac internals can be used by third parties but it isn't like they are really encouraging it. But, aside from that, I would say that, in terms of a multi-language project, which yours pretty obviously is, using CongoCC to parse has (it seems to me) one fairly big practical advantage over using javac or K2 (for Kotlin) etcetera. That is that the CongoCC generated parsers generate parsers with a common Node/Token API. In fact, you might not be aware of the ROOT_API_PACKAGE setting, which allows you to have multiple grammars but generate an AST with a common root Node. And you can use a common Node.Visitor API to walk the tree of all of your various source code AST's. I assume that if you use javac and K2 and other things, they have their own API for traversing the tree, but in each case it is different.
Well, generally speaking, you seem to be under-utilizing the functionality in CongoCC a fair bit. You don't seem to be using the Node.Visitor but perhaps more importantly, you're taking the Java parser that is generated by default and not leveraging the feature that you can inject code into the various generated Node subclasses. Actually, it might even be possible or useful to inject code such that the various nodes implement interfaces defined in com.sun.tools.javac.tree.
bnaudts The module-info.java parsing case is also related to refactoring: because of its strictness when compiling with modules, many an intermediate refactoring state must be compiled with the module system deactivated. But then, module-info.java isn’t parsed either, hence we can’t see it… so we syntactically parse it with CongoCC so that it is visible among the real types, and we can edit it.
Well, yes, the CongoCC generated parsers (at least, by default) are quite systematic in not throwing away any information. You can traverse the tree and all the location info is there, even for comments or delimiters like commas or delimiters, that a compiler might throw away.
Well, generally, my sense of things is that if all somebody wants to do is parse Java source code, then they would likely be better off using the CongoCC generated Java parser. I don't say that just because I'm the author. There is the fact that you have a common API for node traversal etc for all the various parsers for different languages, but also, I think that if you need a feature and show up here and it is well motivated, we'll be pretty responsive. If you go to the JDK people or the Jetbrains people or whoever, well.... I'm not so sure, but I think it might be hard to get the time of day from them! Of course, it's also true that a lot of people (the vast majority seemingly) just want to grab something and use it and don't want to have any interaction with the developers behind it. But that could be a consideration.
bnaudts Lastly, I’m planning to experiment a bit with your new Kotlin parser as a tool for doing not-too-complex Gradle build file refactorings.
Well, I would be quite happy if you do that. Do be aware that the most up-to-date version of the Kotlin grammar/parser is at https://github.com/congo-cc/congo-grammars.git
I sent some people an email about it (that I copied you on) and it pointed to another location, but the right one is the one above.
Thanks,
Jon