Hello,
Please don't take my questions and problems with the design of Trex as an offense. As we will have to decide about millions of hours of work, we have to know what paths are available for us.
The questions / problems regarding the memory usage, and tree structure are one of the most important issues in my view, because they will serve as the basis for later developements.
Kristof
Hello Kristof,
Kristóf Szabados (IJ/ETH) wrote:
No I was thingink on creating the nodes of the tree by hand. I think it will become a problem later on if we will have use getchild().getchild() and if that's not working than it's siblings kind of programming. In a long term it should be better to program like get_the_base_template().iterate_over_the_formal_parameters_until(). Well this is not a good example (long name, wrong structure), but it demonstrates how much easyer it is to maintain a code written that way, compared to the other general tree handling. (This is also a working method proven by TITAN, it is somewhere between the fully general parsing of ANTLR and the hand crafted parsers of gcc, JDT, CDT and other commercial compilers).
Yes, a class based data model for the source would certainly be nice and easier to use. Implementing this later on is indeed not that simple. However, i can imagine that it is possible to introducing a data-model facade based on the LocationAST that is doing something like that. That way, new code could use this facade and old code can be rewritten to use it as time permits. However, i also believe that the current AST approach can make some things easier. For example, i would imagine that the reference finder algorithm could not be realized in such a generic way that easily. But these things always depend on details in the implementation and design.
There is also an other problem of Trex's huge memory requirement. For the SIP protocol's files (which I assume you have) TITAN's peak memory usage only reaches 23 MB in the 3 seconds of parsing, semantic checkin, code generation, and runnable generation, while Trex uses more than 100 MBs for quite a few seconds to make the parsing, and metric analyzis (0.5.2 and the memory usage is constant because of the nature of the problem).
The metric analysis is definately expensive performance wise. The latest builds offer the possibility to disable it through the project property page. The parser speed and memory footprint can probably still be tuned a lot - that is correct, but it has not been the focus of our work in Göttingen. For a really good solution, i would expect it makes sense to go the JDT route and have a lightweight AST for IDE stuff and a non-lightweight AST for possible compilation and so on. Also, note that we currently analyze a project very naively, i.e. we don't analyze dependencies between TTCN-3 modules, but we load all TTCN-3 files in a project. Hence, loading times are high the first time on startup and unnecessary files may get loaded.
There is also an other problem that works for Trex's huge memory usage, if a file was edited, than a new tree is built while keeping the old tree . I understand that on this way you can support content assistant on non-correct files, but this doubles the memory needed temporarily, it might be better to re-parse only a sub tree (but I also understand that that would be quite a lot of work).
Also true. It is probably not as hard as you think. At least i have some rough ideas how i would try to approach it and this would involve maybe a few days to realize.
We basically knew about all these problems, but you should keep in mind that the TRex is the result of few persons who have worked on it or are still working on it only for their bachelor and master's thesises. Also it may currently seem like we are working on TRex full-time, but actually we aren't ;-) The things you mention are problematic for productive industrial use, but they are not as important for research. Hence, these things are somewhere on our list, but they have no immediate priority. We would expect that those things get addressed by those who need them improved and we would certainly assist if there are any questions. Otherwise, they will be addressed when our time permits, but this may not be very soon. You should not forget that we wrote everything from scratch in maybe a year or less with mostly only one or two persons implementing at a time. Therefore, it shouldn't be a surprise that we cannot offer a product which is ready for commercial use.
trex-devel@informatik.uni-goettingen.de