Output parsing tables in language-agnostic format (e.g. json)

[mewmew]

Gocc has always been intended to support a variety of output language, as evident by the separation of Go related gen packages, from the generic aspects of the code.

• lexer/gen/golang
• parser/gen/golang
• token/gen/golang
• util/gen/golang

This issue tracks the development of supporting Scala as an output language of Gocc, the requirements of which has been reported in llir/llvm#2 (comment)

cc: @reedkotler If you decided to investigate adding Scala support to Gocc, you may use this issue to discuss any issues you may run into, or any questions that may arise. @goccmack and @awalterschulze have good knowledge of the Gocc internals, so they should be able to help.

[mewmew]

@reedkotler Do you have an example Scala table that may be used as a reference for what the output should look like?

[reedkotler]

I don't know if you already have this, but it would be nice to have a universal language independent format in theform of ascii tables that could be read in. I have used LR parser generators back in the day that had such a structure. There were several even that had this option. Date: Sat, 10 Dec 2016 12:41:45 -0800 From: [email protected] To: [email protected] CC: [email protected]; [email protected] Subject: Re: [goccmack/gocc] Scala output language (#39) @reedkotler Do you have an example Scala table that may be used as a reference for what the output should look like? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[reedkotler]

I don't know if you already have this, but it would be nice to have a universal language independent format in theform of ascii tables that could be read in.I have used LR parser generators back in the day that had such a structure. There were several even that had this option. Date: Sat, 10 Dec 2016 12:41:45 -0800 From: [email protected] To: [email protected] CC: [email protected]; [email protected] Subject: Re: [goccmack/gocc] Scala output language (#39) @reedkotler Do you have an example Scala table that may be used as a reference for what the output should look like? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[awalterschulze]

Makes sense so you would want gocc to parse the bnf spec and generate the content of the tables.
But then instead of generated the tables as code in some target language, it should rather serialize the tables in a format which can be deserialized by a code generator. This code generator can then simply deserialize the table and generate the code. The generator would then typically be written in the same language as the target language. This way people maintaining the code generator can be experts in a single language.
Am I understanding you correctly?

[mewmew]

Actually this approach could also help alleviate the pain caused by huge tables, that #28 sought to address.

[awalterschulze]

I had the same thought ☺️

…

On Sat, 10 Dec 2016, 23:07 Robin Eklind, ***@***.***> wrote: Actually this approach could also help alleviate the pain caused by huge tables, that #28 <#28> sought to address. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#39 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABvsLedNl7MSuSSukU15bzmF4M4EZjCxks5rGyK-gaJpZM4LJwI7> .

[reedkotler]

Yes, it's all serialized and it's easy for any new language to either:1) read the tables in at runtime and configure the arrays then2) write a tool to generate exactly the table form they want statically. I've seen tools that do both, back in the day.I even wrote my own lr parser generator back in the day...I don't have it anymore. You may need a little creativity for mapping stack entries to $1 $2 and such. People will have to translate your skeleton to their language and hopefully will contribute what they did back to you. Then you might decide to productize some of the individual version. Date: Sat, 10 Dec 2016 13:58:05 -0800 From: [email protected] To: [email protected] CC: [email protected]; [email protected] Subject: Re: [goccmack/gocc] Scala output language (#39) Makes sense so you would want gocc to parse the bnf spec and generate the content of the tables. But then instead of generated the tables as code in some target language, it should rather serialize the tables in a format which can be deserialized by a code generator. This code generator can then simply deserialize the table and generate the code. The generator would then typically be written in the same language as the target language. This way people maintaining the code generator can be experts in a single language. Am I understanding you correctly? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[awalterschulze]

I am for this feature, but I don't have time to build it. But I will try my best to support anyone who is up for making this. It sounds really cool and opens this project up to a greater community.

[awalterschulze]

The sdt rules can be a problem to translate. In my newest parsers I write with gocc, my ast is a protobuf message. This is a cross language serialization format, which means I only need one parser implementation. But thats not the point I want to make here. What protobuf also does, is it generates structures or classes or whatever is appropriate for that language, given the proto spec. This means that given a bnf and proto spec it is technically possible to generate consistent cross language parsers. And I don't think that is something I have seen before. This is just an idea and serialization of the tables is a nice step in that direction, if we ever want to do that.

[awalterschulze]

correction : if we ever want to create consistent cross language parsers

[mewmew]

I am for this feature, but I don't have time to build it. But I will try my best to support anyone who is up for making this. It sounds really cool and opens this project up to a greater community.

I'm also for investigating the benefits and drawbacks of serializing the tables in a language-agnostic format.

Anyone interested in this development may give it a go, and we can all discuss different approaches in this issue thread. Marking this issue with the "Help wanted" tag for now, so it is clear that anyone may step up and play around with it.

[mewmew]

Also updated the title of the issue to make it clear that this is not specifically about adding Scala support, but to change the output format of parsing tables from Go code (as generated today by Gocc), to a more language-agnostic format (e.g. json, protobuf, or other file format that makes sense). Which specific format to choose may be discussed in this issue thread.