Doclet script is broken with Processing 4 (update to `jdk.javadoc.doclet.StandardDoclet` needed)

[SableRaf]

With the release of Processing 4, we upgraded from JDK 11 to JDK 17 and as a result, our script that parses the JavaDoc comments in the Processing source and creates JSON files in the website repo is broken.

The class that we're using, com.sun.tools.doclets.standard.Standard has been deprecated and replaced with jdk.javadoc.doclet.StandardDoclet which has a whole different API.

We are in need of someone who can evaluate the amount of work required to update the script to use the new class and API and/or make the changes if they are manageable. We understand that this might be a substantial amount of work and we will consider setting a bounty for this task if that is the case.

Please note that this issue is blocking updates to the reference (some examples are linked below) and we would greatly appreciate any help in resolving it, on behalf of the whole Processing community.

Raphaël de Courville
Processing Community Lead Fellow

Edit (20.01.2023): rewrote for clarity and added request for help

[yogitheboss]

Can I help with this issue? But will require a bit of time to understand the api well.

[SableRaf]

Hi @yogitheboss and thanks for your offer to help us solve this important issue! The Doclet script is currently being researched by @fdoflorenzano. I recommend getting in touch with them and ask if they need a hand.

[fdoflorenzano]

Hey there! As @SableRaf mentioned, I'm also currently working on this, although not full-time, and I do have a couple of busy weeks ahead myself, so having another person working on this is greatly appreciated.

I'm also not very far (currently) with the translation. I've mostly spent time analyzing the source code, and figuring out how to go about it. I can share with you some of my appreciations and findings:

• The ProcessingWeblet and writers.Shared classes are the main entryway to the doclet script. ProcessingWeblet by itself is the one that actually gets run, that uses a singleton instance of writers.Shared (as all other classes) for utility methods access. The use of the old API in both is shallow, so it's a good place to start translating. Tho ProcessingWeblet also calls for the other writer classes, and that's where it gets more heavy on the API, so I would recommend ignoring those sections initially.
• After those are settled, the next step would be translating the other writer classes. They are all very similar, as they all extend the BaseWritter class and use the TemplateWritter class. Most of them just override the write function and have a similar flow, so the main translation work here is figuring out how to translate BaseWritter and TemplateWritter, but really BaseWritter as it's the base of all of it.
• I haven't had much luck finding nice docs for the translation between APIs. The main sources that I've been using are:
  • Using the old Doclet API article
  • Using the new Doclet API article
  • Package Summary for jdk.javadoc.doclet
  • In the previous page, the class Migration Guide
  • Specific class documentation pages.

The hard part of the translation thus far for me, has been trying to understand how to obtain the same data the old API was able to directly provide. Even though there is a literal migration guide, this is more of a how to fit specific object types into the new API, and most of the time there isn't a one-to-one correspondence between methods. The new API is more general, so there's probably maybe multiple ways of getting the stuff the script use to. And I haven't been able to find clearer step by step migration for that. I guess that's what we'll do!

So yeah. Let me know @yogitheboss if you want to try it out. I hope this is helpful. Also feel free to post your progress and ask stuff. I'm no expert, but as mentioned I've been playing a little bit with the new API already.

Also my pronouns are them/them btw :)

[fdoflorenzano]

Little update! I made good progress over the weekend with the translation. Turned out going through one of the writer classes basically helped re-writing most of the others. There's still some sections that are pending and I'll continue when I have some time this week. But it's getting there :)