Draft JEP for upcoming work on snippets

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Draft JEP for upcoming work on snippets

Jonathan Gibbons
FYI,

JDK-8201533: Enhanced javadoc support for code samples (snippets)

This is a draft JEP that describes a proposal for a new doc comment tag
to replace the use of `<pre>{@code ...}</pre>` to include code samples
(also known as "snippets") in API documentation comments.

Feedback, comments welcome.

-- Jon, on behalf of the JavaDoc team

https://bugs.openjdk.java.net/browse/JDK-8201533


Reply | Threaded
Open this post in threaded view
|

Re: Draft JEP for upcoming work on snippets

roger riggs
Hi,

A very useful proposal.

A few comments, I expect these will be addressed as the development proceeds.
  • In the @snippet tag, can the "class=" indicate a module name in addition to the class name?
  • Does the "lang=" tag need to be a closed set; if it were open, it would give more flexibility in the checking tools
  • Indentation: The handling of whitespace may need a bit more care. Is the snippet text treated as a single string so whitespace is stripped only from the first line, or every line individually.
    Perhaps some of the logic from text-blocks can be used to keep the lines aligned even while discarding common leading whitespace. For example, supposing an external file with nested classes and fairly deep indentation in the snippet to be shown.
  • Can the markup tags be nested, for example a region within a region? 
    (@start: A...@start: B...@end: B...@end: A)
  • Markup: Can "text" include html or other javadoc markup, otherwise an error?
  • Is "@replace" needed to show text that would otherwise be invalid for the language?
    For example, '@replace "var a = null;" "var a = <your data here>;"'
Thanks, Roger


On 1/22/21 7:50 PM, Jonathan Gibbons wrote:
FYI,

JDK-8201533: Enhanced javadoc support for code samples (snippets)

This is a draft JEP that describes a proposal for a new doc comment tag to replace the use of `<pre>{@code ...}</pre>` to include code samples (also known as "snippets") in API documentation comments.

Feedback, comments welcome.

-- Jon, on behalf of the JavaDoc team

https://bugs.openjdk.java.net/browse/JDK-8201533



Reply | Threaded
Open this post in threaded view
|

Re: Draft JEP for upcoming work on snippets

Jonathan Gibbons


On 1/26/21 7:47 AM, Roger Riggs wrote:
Hi,

A very useful proposal.

A few comments, I expect these will be addressed as the development proceeds.
  • In the @snippet tag, can the "class=" indicate a module name in addition to the class name?

We'll need to address that. Ideas are to either allow an addition `module=` or use the `module/package.Class` convention.


  • Does the "lang=" tag need to be a closed set; if it were open, it would give more flexibility in the checking tools

It should be open.


  • Indentation: The handling of whitespace may need a bit more care. Is the snippet text treated as a single string so whitespace is stripped only from the first line, or every line individually.
    Perhaps some of the logic from text-blocks can be used to keep the lines aligned even while discarding common leading whitespace. For example, supposing an external file with nested classes and fairly deep indentation in the snippet to be shown.
Yes. The current plan is to leverage ideas from text blocks, like `stripIndent`

  • Can the markup tags be nested, for example a region within a region? 
    (@start: A...@start: B...@end: B...@end: A)

Current idea is no restrictions, so they can nest or overlap.



  • Markup: Can "text" include html or other javadoc markup, otherwise an error?
I'm not sure the exact context you mean for "text".  You may mean the `@insert` markup tag.

If so, the current answer is "don't know", but "yes" would be nice.


  • Is "@replace" needed to show text that would otherwise be invalid for the language?
    For example, '@replace "var a = null;" "var a = <your data here>;"'
yes, `@replace` is needed to display "invalid" text when you also expect other analysis to
be done on the underlying text. For example, if you want to be able to validate the source code as
compilable, or if you want more than just regex highlighting, then the underlying text
has to be "valid".    If you're OK for the text to just be a blob of plain text, then `@replace`
is obviously not required.


Thanks, Roger


On 1/22/21 7:50 PM, Jonathan Gibbons wrote:
FYI,

JDK-8201533: Enhanced javadoc support for code samples (snippets)

This is a draft JEP that describes a proposal for a new doc comment tag to replace the use of `<pre>{@code ...}</pre>` to include code samples (also known as "snippets") in API documentation comments.

Feedback, comments welcome.

-- Jon, on behalf of the JavaDoc team

https://bugs.openjdk.java.net/browse/JDK-8201533



Reply | Threaded
Open this post in threaded view
|

Re: Draft JEP for upcoming work on snippets

Jonathan Gibbons
In reply to this post by roger riggs


On 1/26/21 7:47 AM, Roger Riggs wrote:

  • In the @snippet tag, can the "class=" indicate a module name in addition to the class name?

A related question is whether and when it makes sense to specify a module name.

The current thinking is that external snippets are searched for in 3 locations: 1) the local `snippet-files` directory, 2) the enclosing class hierarchy, 3) a new search path.

Each location has an inherent notion of whether it is "package-oriented" or "module-oriented".

So far, all consideration of the local `snippet-files` directory has been that it is package-oriented, and so presumably would not be searched when looking for a named module.

The enclosing class hierarchy may be package-oriented (for a typical API library) or module-oriented (e.g. for JDK). If no module name is specified and the hierarchy is module-oriented, that would presumably imply the current module (containing the reference.)

For a new search path, search paths are either package-oriented or module-oriented, but not both. At a minimum, we should provide a new package-oriented search path (e.g. --snippet-path); if there is sufficient interest/demand, we could provide a new module-oriented search path as well (e.g. --snippet-module-path).

-- Jon

Reply | Threaded
Open this post in threaded view
|

Re: Draft JEP for upcoming work on snippets

Pavel Rappo
In reply to this post by Jonathan Gibbons
Roger,

I'd add to what Jon has already said about indentation. Early on in our prototype we recognized the need for clear rules and convenient control for indentation. We knew that having those would allow "Snippets" to improve on the solution provided by the "pre-code" compound:

    <pre>{@code <line 0>
        <line 1>
        <line 2>
        ...
        <line n>
    }</pre>

We realized that the problem of indentation and whitespace in multiline blocks was not by any means new and had been successfully solved in Text Blocks (JEPs: 355, 368, 378); and to some extent we reused that solution in the current prototype of "Snippets". Currently the prototype utilizes text-blocks-like solution in two constructs: (i) the {@snippet} tag and (ii) the region marked by `@start/@end`. If we depict this using the notation of JEP 378, we will get this:

i.

    {@snippet name1=value1 name2=value2
                   name3=value3 :
........      <html>
........          <body>
........              <p>Hello, world</p>
........          </body>
........      </html>
........}

ii.

    // @start name
........      <html>
........          <body>
........              <p>Hello, world</p>
........          </body>
........      </html>
........// @end name

The current prototype distinguishes between incidental and essential whitespace using the following pairs of tokens: (i) `:` and `}`, and (ii) `@start name` and `@end name`; this is similar to Text Blocks. That said, the prototype does NOT recognize escape sequences that Text Blocks recognize.

I'd stress that this is not set in stone and that the design space for indentation in snippets has not been fully explored. We are still exploring it and will update the JEP draft accordingly.

-Pavel

> On 26 Jan 2021, at 16:06, Jonathan Gibbons <[hidden email]> wrote:
>
>
>
> On 1/26/21 7:47 AM, Roger Riggs wrote:
>> Hi,
>>
>> A very useful proposal.
>>
>> A few comments, I expect these will be addressed as the development proceeds.
>> • In the @snippet tag, can the "class=" indicate a module name in addition to the class name?
> We'll need to address that. Ideas are to either allow an addition `module=` or use the `module/package.Class` convention.
>
>
>
>> • Does the "lang=" tag need to be a closed set; if it were open, it would give more flexibility in the checking tools
> It should be open.
>
>
>
>> • Indentation: The handling of whitespace may need a bit more care. Is the snippet text treated as a single string so whitespace is stripped only from the first line, or every line individually.
>> Perhaps some of the logic from text-blocks can be used to keep the lines aligned even while discarding common leading whitespace. For example, supposing an external file with nested classes and fairly deep indentation in the snippet to be shown.
> Yes. The current plan is to leverage ideas from text blocks, like `stripIndent`
>> •
>> • Can the markup tags be nested, for example a region within a region?  
>> (@start: A...@start: B...@end: B...@end: A)
> Current idea is no restrictions, so they can nest or overlap.
>
>
>
>> •
>> • Markup: Can "text" include html or other javadoc markup, otherwise an error?
> I'm not sure the exact context you mean for "text".  You may mean the `@insert` markup tag.
> If so, the current answer is "don't know", but "yes" would be nice.
>
>
>
>> • Is "@replace" needed to show text that would otherwise be invalid for the language?
>> For example, '@replace "var a = null;" "var a = <your data here>;"'
> yes, `@replace` is needed to display "invalid" text when you also expect other analysis to
> be done on the underlying text. For example, if you want to be able to validate the source code as
> compilable, or if you want more than just regex highlighting, then the underlying text
> has to be "valid".    If you're OK for the text to just be a blob of plain text, then `@replace`
> is obviously not required.
>
>
>> Thanks, Roger
>>
>>
>> On 1/22/21 7:50 PM, Jonathan Gibbons wrote:
>>> FYI,
>>>
>>> JDK-8201533: Enhanced javadoc support for code samples (snippets)
>>>
>>> This is a draft JEP that describes a proposal for a new doc comment tag to replace the use of `<pre>{@code ...}</pre>` to include code samples (also known as "snippets") in API documentation comments.
>>>
>>> Feedback, comments welcome.
>>>
>>> -- Jon, on behalf of the JavaDoc team
>>>
>>> https://bugs.openjdk.java.net/browse/JDK-8201533 
>>>
>>>
>>

Reply | Threaded
Open this post in threaded view
|

Re: Draft JEP for upcoming work on snippets

Dmitry Timofeev-2
In reply to this post by Jonathan Gibbons
Hi everyone,

Happy to see a JEP improving support for code snippets! I’ve got some feedback from the user perspective, and from some prototyping to support Javadoc snippet compilation:

> hide = regex —

Would it be possible to have a simpler mechanism? For example, Rust code snippets use # at the beginning of the line to hide it [1]. From the user perspective, it is very easy to use and maintain — you don’t have to design a regex, you won’t get any surprises from its evaluation, or read/maintain some over-complicated pattern that slipped through code review.

[1]: https://doc.rust-lang.org/rustdoc/documentation-tests.html#hiding-portions-of-the-example

> lang=name — […] Valid names are java, properties and text

Would it be possible to specify any language, as the goal seems to be to pass this information to any rendering tools? Some converter libraries might use snippets with configs (xml, json, …), some FFI libs — in C++/Rust, some libs might provide examples for some JVM-based languages.

> region=name —

Would it make sense to provide some kind of selectors for Java constructs? E.g., block:<method_name, or class_name> selecting the corresponding method or class? This plugin for Mkdocs does that, but it operates on text, not on AST, therefore, some unexpected curly braces break it (e.g., in string literals) [2]. However, even simple text-based selection works well. Does the Javadoc have the luxury of having ready access to the AST? Or is it too complex to implement/maintain?

[2]: https://github.com/rnorth/mkdocs-codeinclude-plugin#usage

> For inline snippets, especially those that are not a full compilation unit, it will be up to the test infrastructure to "wrap" the code fragment in a full compilation unit, such that it can be compiled and possibly executed.

Would it be possible to keep the non-goal of not providing a standard tool to test them (to limit the scope of the JEP), but suggest a format/restrictions/expansion procedure on the snippets to support such test infra well (or enable adding it to JDK in the future)? I agree that external snippets work well (especially for large-ish fragments), but for some small things it is an overkill (you have to somehow configure the build system to build these files, but exclude from the final artifact, etc.), yet as a user I’d love to check their correctness. Would it make sense to:
- Specify a standard way how a snippet is expanded (e.g., wrapped inside Callable#call, or some standard template — Java language designers are in a perfect position to pick a great one). [3]
- Specify a set of tags that define the expectations of the expanded snippet behaviour (not sure it can be pulled into the scope, but the tools will require that information): [4]
    - Compiles/fails compilation
    - Runs successfully/Throws exception
    - Is totally ignored.

In Rust doctests, for example, all these things are supported:
[3] https://doc.rust-lang.org/rustdoc/documentation-tests.html#pre-processing-examples (adds main unless you do that)
[4] https://doc.rust-lang.org/rustdoc/documentation-tests.html#attributes

Finally, you might be interested in the previous attempts to implement such a tool for Java: https://github.com/jakewins/javadoctest
Beware the docs aren’t updated, its current version does not require writing test manually, and finds and extracts snippets on itself, e.g.:
https://github.com/jakewins/javadoctest/blob/master/junit-platform-engine/src/test/java/javadoctest/engine/fixture/FixtureDocTestSimple.java#L18-L22

--
Best regards,
Dmitry Timofeev
Reply | Threaded
Open this post in threaded view
|

Re: Draft JEP for upcoming work on snippets

Pavel Rappo
Dmitry,

8201533 is a Draft JEP for a feature that is currently under active development. As such, that draft is prone to inconsistencies, inaccuracies, typos, etc. As we further develop the feature and receive feedback on it, that draft will improve and eventually reach the high bar of JEP Candidate.

Thank you for the feedback; detailed replies are inline.

On 29 Jan 2021, at 06:52, Dmitry Timofeev <[hidden email]> wrote:

Hi everyone, 

Happy to see a JEP improving support for code snippets! I’ve got some feedback from the user perspective, and from some prototyping to support Javadoc snippet compilation:

hide = regex —

Would it be possible to have a simpler mechanism? For example, Rust code snippets use # at the beginning of the line to hide it [1]. From the user perspective, it is very easy to use and maintain — you don’t have to design a regex, you won’t get any surprises from its evaluation, or read/maintain some over-complicated pattern that slipped through code review.

[1]: https://doc.rust-lang.org/rustdoc/documentation-tests.html#hiding-portions-of-the-example

We are open to amending the initial set of markup constructs. We could consider adding the parameterless trailing "@hide" construct which hides the line on which it stands; you won't need regex to use such a construct. Do you see much use for that particular construct or you provided it as an illustration for alternative, regex-less markup?

As for #, I don't think prepending a line with # to hide that line would be apt for snippets in JavaDoc. This is because it breaks invariance, one of the major qualities we sought in markup. Markup is invariant if it doesn't depend on whether it is used in an inline or external snippet. Although using leading # to hide a line seems to work fine with inline snippets, it won't work with external snippets. Indeed, in Java source a line starting with # results in a syntax error, whereas in properties file the leading # starts a comment.

lang=name — […] Valid names are java, properties and text

Would it be possible to specify any language, as the goal seems to be to pass this information to any rendering tools? Some converter libraries might use snippets with configs (xml, json, …), some FFI libs — in C++/Rust, some libs might provide examples for some JVM-based languages.

Currently, we see no reasons for not allowing an arbitrary language. That said, the JEP draft needs to clarify that. The draft needs to convey that while we are not restricting the set of languages, only the specified languages are guaranteed to be recognized by the standard doclet. What "recognized" translates to must also be explained.

region=name — 

Would it make sense to provide some kind of selectors for Java constructs? E.g., block:<method_name, or class_name> selecting the corresponding method or class? This plugin for Mkdocs does that, but it operates on text, not on AST, therefore, some unexpected curly braces break it (e.g., in string literals) [2]. However, even simple text-based selection works well. Does the Javadoc have the luxury of having ready access to the AST? Or is it too complex to implement/maintain?

[2]: https://github.com/rnorth/mkdocs-codeinclude-plugin#usage

We are considering allowing to specify method bodies as complete snippet source or a region thereof.

For inline snippets, especially those that are not a full compilation unit, it will be up to the test infrastructure to "wrap" the code fragment in a full compilation unit, such that it can be compiled and possibly executed.

Would it be possible to keep the non-goal of not providing a standard tool to test them (to limit the scope of the JEP), but suggest a format/restrictions/expansion procedure on the snippets to support such test infra well (or enable adding it to JDK in the future)? I agree that external snippets work well (especially for large-ish fragments), but for some small things it is an overkill (you have to somehow configure the build system to build these files, but exclude from the final artifact, etc.), yet as a user I’d love to check their correctness. Would it make sense to:
- Specify a standard way how a snippet is expanded (e.g., wrapped inside Callable#call, or some standard template — Java language designers are in a perfect position to pick a great one). [3]
- Specify a set of tags that define the expectations of the expanded snippet behaviour (not sure it can be pulled into the scope, but the tools will require that information): [4]
   - Compiles/fails compilation
   - Runs successfully/Throws exception
   - Is totally ignored.

In Rust doctests, for example, all these things are supported:
[3] https://doc.rust-lang.org/rustdoc/documentation-tests.html#pre-processing-examples (adds main unless you do that)
[4] https://doc.rust-lang.org/rustdoc/documentation-tests.html#attributes

The "Snippets" feature's first and foremost use case is JDK itself. Snippets found in "<pre>{@code" compounds in JDK are so different from each other that it doesn't seem possible to provide a solution sufficiently general to test those snippets automatically. Although the concept of "doctest", which you are referring to, offers a highly attractive end-to-end testing, that concept applies cleanly to snippets that were written with that concept in mind, such as snippets structured as tests or executable as-is. The thing is, we don't have many of those in JDK.

Since we cannot generally ensure that every snippet is correct in the resulting documentation, we aim for the next best thing: enabling authors to ensure that the snippets are produced from correct sources. Authors don't have to build these sources, exclude them from the build artifacts, or use external snippets altogether; it's an option.

While doctests are not immediately useful in JDK, we recognize that they might benefit other projects. That's why we are exposing snippets in JavaDoc API. That API may be used to test snippets externally or in JavaDoc. For example, using attributes of the {@snippet} tag, snippet markup and a custom taglet, one should be able to teach JavaDoc doctesting.

Finally, you might be interested in the previous attempts to implement such a tool for Java: https://github.com/jakewins/javadoctest 
Beware the docs aren’t updated, its current version does not require writing test manually, and finds and extracts snippets on itself, e.g.:
https://github.com/jakewins/javadoctest/blob/master/junit-platform-engine/src/test/java/javadoctest/engine/fixture/FixtureDocTestSimple.java#L18-L22
-- 
Best regards,
Dmitry Timofeev

Thanks for providing the links to mkdocs and javadoctest.

-Pavel

Reply | Threaded
Open this post in threaded view
|

Re: Draft JEP for upcoming work on snippets

Pavel Rappo
In reply to this post by Dmitry Timofeev-2
Resending my recent email in plain text; apologies for sending it previously in rich text.

***

Dmitry,

8201533 is a Draft JEP for a feature that is currently under active development. As such, that draft is prone to inconsistencies, inaccuracies, typos, etc. As we further develop the feature and receive feedback on it, that draft will improve and eventually reach the high bar of JEP Candidate.

Thank you for the feedback; detailed replies are inline.

> On 29 Jan 2021, at 06:52, Dmitry Timofeev <[hidden email]> wrote:
>
> Hi everyone,
>
> Happy to see a JEP improving support for code snippets! I’ve got some feedback from the user perspective, and from some prototyping to support Javadoc snippet compilation:
>
>> hide = regex —
>
> Would it be possible to have a simpler mechanism? For example, Rust code snippets use # at the beginning of the line to hide it [1]. From the user perspective, it is very easy to use and maintain — you don’t have to design a regex, you won’t get any surprises from its evaluation, or read/maintain some over-complicated pattern that slipped through code review.
>
> [1]: https://doc.rust-lang.org/rustdoc/documentation-tests.html#hiding-portions-of-the-example

We are open to amending the initial set of markup constructs. We could consider adding the parameterless trailing "@hide" construct which hides the line on which it stands; you won't need regex to use such a construct. Do you see much use for that particular construct or you provided it as an illustration for alternative, regex-less markup?

As for #, I don't think prepending a line with # to hide that line would be apt for snippets in JavaDoc. This is because it breaks invariance, one of the major qualities we sought in markup. Markup is invariant if it doesn't depend on whether it is used in an inline or external snippet. Although using leading # to hide a line seems to work fine with inline snippets, it won't work with external snippets. Indeed, in Java source a line starting with # results in a syntax error, whereas in properties file the leading # starts a comment.

>> lang=name — […] Valid names are java, properties and text
>
> Would it be possible to specify any language, as the goal seems to be to pass this information to any rendering tools? Some converter libraries might use snippets with configs (xml, json, …), some FFI libs — in C++/Rust, some libs might provide examples for some JVM-based languages.

Currently, we see no reasons for not allowing an arbitrary language. That said, the JEP draft needs to clarify that. The draft needs to convey that while we are not restricting the set of languages, only the specified languages are guaranteed to be recognized by the standard doclet. What "recognized" translates to must also be explained.

>> region=name —
>
> Would it make sense to provide some kind of selectors for Java constructs? E.g., block:<method_name, or class_name> selecting the corresponding method or class? This plugin for Mkdocs does that, but it operates on text, not on AST, therefore, some unexpected curly braces break it (e.g., in string literals) [2]. However, even simple text-based selection works well. Does the Javadoc have the luxury of having ready access to the AST? Or is it too complex to implement/maintain?
>
> [2]: https://github.com/rnorth/mkdocs-codeinclude-plugin#usage

We are considering allowing to specify method bodies as complete snippet source or a region thereof.

>> For inline snippets, especially those that are not a full compilation unit, it will be up to the test infrastructure to "wrap" the code fragment in a full compilation unit, such that it can be compiled and possibly executed.
>
> Would it be possible to keep the non-goal of not providing a standard tool to test them (to limit the scope of the JEP), but suggest a format/restrictions/expansion procedure on the snippets to support such test infra well (or enable adding it to JDK in the future)? I agree that external snippets work well (especially for large-ish fragments), but for some small things it is an overkill (you have to somehow configure the build system to build these files, but exclude from the final artifact, etc.), yet as a user I’d love to check their correctness. Would it make sense to:
> - Specify a standard way how a snippet is expanded (e.g., wrapped inside Callable#call, or some standard template — Java language designers are in a perfect position to pick a great one). [3]
> - Specify a set of tags that define the expectations of the expanded snippet behaviour (not sure it can be pulled into the scope, but the tools will require that information): [4]
>    - Compiles/fails compilation
>    - Runs successfully/Throws exception
>    - Is totally ignored.
>
> In Rust doctests, for example, all these things are supported:
> [3] https://doc.rust-lang.org/rustdoc/documentation-tests.html#pre-processing-examples (adds main unless you do that)
> [4] https://doc.rust-lang.org/rustdoc/documentation-tests.html#attributes

The "Snippets" feature's first and foremost use case is JDK itself. Snippets found in "<pre>{@code" compounds in JDK are so different from each other that it doesn't seem possible to provide a solution sufficiently general to test those snippets automatically. Although the concept of "doctest", which you are referring to, offers a highly attractive end-to-end testing, that concept applies cleanly to snippets that were written with that concept in mind, such as snippets structured as tests or executable as-is. The thing is, we don't have many of those in JDK.

Since we cannot generally ensure that every snippet is correct in the resulting documentation, we aim for the next best thing: enabling authors to ensure that the snippets are produced from correct sources. Authors don't have to build these sources, exclude them from the build artifacts, or use external snippets altogether; it's an option.

While doctests are not immediately useful in JDK, we recognize that they might benefit other projects. That's why we are exposing snippets in JavaDoc API. That API may be used to test snippets externally or in JavaDoc. For example, using attributes of the {@snippet} tag, snippet markup and a custom taglet, one should be able to teach JavaDoc doctesting.

> Finally, you might be interested in the previous attempts to implement such a tool for Java: https://github.com/jakewins/javadoctest 
> Beware the docs aren’t updated, its current version does not require writing test manually, and finds and extracts snippets on itself, e.g.:
> https://github.com/jakewins/javadoctest/blob/master/junit-platform-engine/src/test/java/javadoctest/engine/fixture/FixtureDocTestSimple.java#L18-L22
> --
> Best regards,
> Dmitry Timofeev

Thanks for providing the links to mkdocs and javadoctest.

-Pavel

Reply | Threaded
Open this post in threaded view
|

Re: Draft JEP for upcoming work on snippets

Dmitry Timofeev-2
Hi Pavel,

Thanks for your response! Inline:


On Tue, 2 Feb 2021 at 20:18, Pavel Rappo <[hidden email]> wrote:
>
> Resending my recent email in plain text; apologies for sending it previously in rich text.
>
> ***
>
> Dmitry,
>
> 8201533 is a Draft JEP for a feature that is currently under active development. As such, that draft is prone to inconsistencies, inaccuracies, typos, etc. As we further develop the feature and receive feedback on it, that draft will improve and eventually reach the high bar of JEP Candidate.

Thank you for the clarification! I discovered it on r/java, and saw it
was in a very good shape already. BTW, big thanks to your team members
who reach out to users through less formal channels than mailing lists
— otherwise, many aren't likely to even know of most in progress
features.

>
> Thank you for the feedback; detailed replies are inline.
>
> > On 29 Jan 2021, at 06:52, Dmitry Timofeev <[hidden email]> wrote:
> >
> > Hi everyone,
> >
> > Happy to see a JEP improving support for code snippets! I’ve got some feedback from the user perspective, and from some prototyping to support Javadoc snippet compilation:
> >
> >> hide = regex —
> >
> > Would it be possible to have a simpler mechanism? For example, Rust code snippets use # at the beginning of the line to hide it [1]. From the user perspective, it is very easy to use and maintain — you don’t have to design a regex, you won’t get any surprises from its evaluation, or read/maintain some over-complicated pattern that slipped through code review.
> >
> > [1]: https://doc.rust-lang.org/rustdoc/documentation-tests.html#hiding-portions-of-the-example
>
> We are open to amending the initial set of markup constructs. We could consider adding the parameterless trailing "@hide" construct which hides the line on which it stands; you won't need regex to use such a construct. Do you see much use for that particular construct or you provided it as an illustration for alternative, regex-less markup?

The latter — as an illustration of a construct that worked well for
this purpose in a different language.

>
> As for #, I don't think prepending a line with # to hide that line would be apt for snippets in JavaDoc. This is because it breaks invariance, one of the major qualities we sought in markup. Markup is invariant if it doesn't depend on whether it is used in an inline or external snippet. Although using leading # to hide a line seems to work fine with inline snippets, it won't work with external snippets. Indeed, in Java source a line starting with # results in a syntax error, whereas in properties file the leading # starts a comment.

I see, I wasn't aware of such a requirement for the snippets. With
both inline and external snippets it looks reasonable. Rust, AFAIK,
does not support external snippets. However, good support for internal
works fine: in jni-rs we literally launch a JVM in a documentation
test :-)
https://github.com/jni-rs/jni-rs/blob/242c9545890ba39246ddac53608a9928d1300fda/src/wrapper/java_vm/vm.rs#L67-L106

By allowing only valid Java code in snippets, do you mean only valid
fragments of Java _source files_, or would you also allow valid jshell
snippets? For example, would you allow using (and hiding) import
statements, mixed with other statements:

import static java.util.stream.Collectors.toList; // @hide
import java.util.stream.Stream; // @hide (or hide=^import.+ in a
pattern based variant)
Stream.of("foo", "baz", "").filter(s -> !s.isEmpty()).collect(toList());

>
> >> lang=name — […] Valid names are java, properties and text
> >
> > Would it be possible to specify any language, as the goal seems to be to pass this information to any rendering tools? Some converter libraries might use snippets with configs (xml, json, …), some FFI libs — in C++/Rust, some libs might provide examples for some JVM-based languages.
>
> Currently, we see no reasons for not allowing an arbitrary language. That said, the JEP draft needs to clarify that. The draft needs to convey that while we are not restricting the set of languages, only the specified languages are guaranteed to be recognized by the standard doclet. What "recognized" translates to must also be explained.
>
> >> region=name —
> >
> > Would it make sense to provide some kind of selectors for Java constructs? E.g., block:<method_name, or class_name> selecting the corresponding method or class? This plugin for Mkdocs does that, but it operates on text, not on AST, therefore, some unexpected curly braces break it (e.g., in string literals) [2]. However, even simple text-based selection works well. Does the Javadoc have the luxury of having ready access to the AST? Or is it too complex to implement/maintain?
> >
> > [2]: https://github.com/rnorth/mkdocs-codeinclude-plugin#usage
>
> We are considering allowing to specify method bodies as complete snippet source or a region thereof.
>
> >> For inline snippets, especially those that are not a full compilation unit, it will be up to the test infrastructure to "wrap" the code fragment in a full compilation unit, such that it can be compiled and possibly executed.
> >
> > Would it be possible to keep the non-goal of not providing a standard tool to test them (to limit the scope of the JEP), but suggest a format/restrictions/expansion procedure on the snippets to support such test infra well (or enable adding it to JDK in the future)? I agree that external snippets work well (especially for large-ish fragments), but for some small things it is an overkill (you have to somehow configure the build system to build these files, but exclude from the final artifact, etc.), yet as a user I’d love to check their correctness. Would it make sense to:
> > - Specify a standard way how a snippet is expanded (e.g., wrapped inside Callable#call, or some standard template — Java language designers are in a perfect position to pick a great one). [3]
> > - Specify a set of tags that define the expectations of the expanded snippet behaviour (not sure it can be pulled into the scope, but the tools will require that information): [4]
> >    - Compiles/fails compilation
> >    - Runs successfully/Throws exception
> >    - Is totally ignored.
> >
> > In Rust doctests, for example, all these things are supported:
> > [3] https://doc.rust-lang.org/rustdoc/documentation-tests.html#pre-processing-examples (adds main unless you do that)
> > [4] https://doc.rust-lang.org/rustdoc/documentation-tests.html#attributes
>
> The "Snippets" feature's first and foremost use case is JDK itself. Snippets found in "<pre>{@code" compounds in JDK are so different from each other that it doesn't seem possible to provide a solution sufficiently general to test those snippets automatically. Although the concept of "doctest", which you are referring to, offers a highly attractive end-to-end testing, that concept applies cleanly to snippets that were written with that concept in mind, such as snippets structured as tests or executable as-is. The thing is, we don't have many of those in JDK.
>
> Since we cannot generally ensure that every snippet is correct in the resulting documentation, we aim for the next best thing: enabling authors to ensure that the snippets are produced from correct sources. Authors don't have to build these sources, exclude them from the build artifacts, or use external snippets altogether; it's an option.
>
> While doctests are not immediately useful in JDK, we recognize that they might benefit other projects. That's why we are exposing snippets in JavaDoc API. That API may be used to test snippets externally or in JavaDoc. For example, using attributes of the {@snippet} tag, snippet markup and a custom taglet, one should be able to teach JavaDoc doctesting.

Totally agree that an expansion scheme that works for arbitrary legacy
code is an impossible task. I rather meant specifying a good enough
expansion scheme for most snippets that are written with it in mind.
For legacy ones (that, on top of being written without any
restrictions, might reference non-existent symbols, or have an
ellipsis in place of an initializing expression) there could be an
option to skip executing, or even compiling such a snippet. Rustdoc,
for instance, supports both. Any legacy snippets then could be
migrated automatically with an option to not test them, and adjusted
later.

I agree that producing snippets from correct, separate sources is a
useful feature. However, as it is simpler to write them inline and
immediately see inside the non-rendered Javadoc, could you possibly
consider specifying the expansion scheme, so that tools share the same
specification (IDEs, a doctest tool)? For example, as Rust does
specify that, these inline snippets are understood by both the doc
test tool, and an IntelliJ plugin, which supports syntax highlighting,
inspections and refactorings.

Also, I think it may help the renderers: they could provide an option
to copy a complete snippet, or run it in jshell (with all the hidden
code, and, possibly, all the required imports inherited from the
containing class).

If there is already a way to add metadata to the snippet tag that
could help communicate the intent to a doc testing tool, that's
awesome.

> While doctests are not immediately useful in JDK, we recognize that they might benefit other projects.

Absolutely, I've certainly used them in the API specs in my projects,
and benefitted from the examples in popular Java libraries: Mockito
has 250+; Guava — over 500; and AssertJ — 2.5K (with some
false-positives):
- https://sourcegraph.com/search?q=%5E%5Cs*%5C*.%2B%3Cpre+lang:java+repo:mockito/mockito+count:1000&patternType=regexp
- https://sourcegraph.com/search?q=%5E%5Cs*%5C*.%2B%3Cpre%3E+lang%3Ajava+repo%3Agoogle%2Fguava+count%3A1000&patternType=regexp
- https://sourcegraph.com/search?q=%5E%5Cs*%5C*.%2B%3Cpre+lang:java+repo:assertj/assertj-core+count:1000&patternType=regexp


Also, one more thing: there is a variety of present options for
embedding code snippets with various restrictions, and I often had to
reopen this comparison to choose the right one:
https://reflectoring.io/howto-format-code-snippets-in-javadoc/#code-markup-features-at-a-glance
Will @snippets require any escaping for "@"?

>
> > Finally, you might be interested in the previous attempts to implement such a tool for Java: https://github.com/jakewins/javadoctest
> > Beware the docs aren’t updated, its current version does not require writing test manually, and finds and extracts snippets on itself, e.g.:
> > https://github.com/jakewins/javadoctest/blob/master/junit-platform-engine/src/test/java/javadoctest/engine/fixture/FixtureDocTestSimple.java#L18-L22
> > --
> > Best regards,
> > Dmitry Timofeev
>
> Thanks for providing the links to mkdocs and javadoctest.
>
> -Pavel
>


--
Best,
Dmitry
Reply | Threaded
Open this post in threaded view
|

Re: Draft JEP for upcoming work on snippets

Jonathan Gibbons
Dmitry,

Some answers inline.

-- Jon

On 2/6/21 7:56 AM, Dmitry Timofeev wrote:

> Hi Pavel,
>
> Thanks for your response! Inline:
>
>
> On Tue, 2 Feb 2021 at 20:18, Pavel Rappo <[hidden email]> wrote:
>> Resending my recent email in plain text; apologies for sending it previously in rich text.
>>
>> ***
>>
>> Dmitry,
>>
>> 8201533 is a Draft JEP for a feature that is currently under active development. As such, that draft is prone to inconsistencies, inaccuracies, typos, etc. As we further develop the feature and receive feedback on it, that draft will improve and eventually reach the high bar of JEP Candidate.
> Thank you for the clarification! I discovered it on r/java, and saw it
> was in a very good shape already. BTW, big thanks to your team members
> who reach out to users through less formal channels than mailing lists
> — otherwise, many aren't likely to even know of most in progress
> features.
>
>> Thank you for the feedback; detailed replies are inline.
>>
>>> On 29 Jan 2021, at 06:52, Dmitry Timofeev <[hidden email]> wrote:
>>>
>>> Hi everyone,
>>>
>>> Happy to see a JEP improving support for code snippets! I’ve got some feedback from the user perspective, and from some prototyping to support Javadoc snippet compilation:
>>>
>>>> hide = regex —
>>> Would it be possible to have a simpler mechanism? For example, Rust code snippets use # at the beginning of the line to hide it [1]. From the user perspective, it is very easy to use and maintain — you don’t have to design a regex, you won’t get any surprises from its evaluation, or read/maintain some over-complicated pattern that slipped through code review.
>>>
>>> [1]: https://doc.rust-lang.org/rustdoc/documentation-tests.html#hiding-portions-of-the-example
>> We are open to amending the initial set of markup constructs. We could consider adding the parameterless trailing "@hide" construct which hides the line on which it stands; you won't need regex to use such a construct. Do you see much use for that particular construct or you provided it as an illustration for alternative, regex-less markup?
> The latter — as an illustration of a construct that worked well for
> this purpose in a different language.
>
>> As for #, I don't think prepending a line with # to hide that line would be apt for snippets in JavaDoc. This is because it breaks invariance, one of the major qualities we sought in markup. Markup is invariant if it doesn't depend on whether it is used in an inline or external snippet. Although using leading # to hide a line seems to work fine with inline snippets, it won't work with external snippets. Indeed, in Java source a line starting with # results in a syntax error, whereas in properties file the leading # starts a comment.
> I see, I wasn't aware of such a requirement for the snippets. With
> both inline and external snippets it looks reasonable. Rust, AFAIK,
> does not support external snippets. However, good support for internal
> works fine: in jni-rs we literally launch a JVM in a documentation
> test :-)
> https://github.com/jni-rs/jni-rs/blob/242c9545890ba39246ddac53608a9928d1300fda/src/wrapper/java_vm/vm.rs#L67-L106
>
> By allowing only valid Java code in snippets, do you mean only valid
> fragments of Java _source files_, or would you also allow valid jshell
> snippets? For example, would you allow using (and hiding) import
> statements, mixed with other statements:
>
> import static java.util.stream.Collectors.toList; // @hide
> import java.util.stream.Stream; // @hide (or hide=^import.+ in a
> pattern based variant)
> Stream.of("foo", "baz", "").filter(s -> !s.isEmpty()).collect(toList());

The input language for JShell is not Java (it's very similar, but obviously
has additional constructs). As such, I would expect JShell to be handled
as a different language, in the same way that we would handle properties
files and plain text files, by using the "lang" attribute.


>
>>>> lang=name — […] Valid names are java, properties and text
>>> Would it be possible to specify any language, as the goal seems to be to pass this information to any rendering tools? Some converter libraries might use snippets with configs (xml, json, …), some FFI libs — in C++/Rust, some libs might provide examples for some JVM-based languages.
>> Currently, we see no reasons for not allowing an arbitrary language. That said, the JEP draft needs to clarify that. The draft needs to convey that while we are not restricting the set of languages, only the specified languages are guaranteed to be recognized by the standard doclet. What "recognized" translates to must also be explained.
>>
>>>> region=name —
>>> Would it make sense to provide some kind of selectors for Java constructs? E.g., block:<method_name, or class_name> selecting the corresponding method or class? This plugin for Mkdocs does that, but it operates on text, not on AST, therefore, some unexpected curly braces break it (e.g., in string literals) [2]. However, even simple text-based selection works well. Does the Javadoc have the luxury of having ready access to the AST? Or is it too complex to implement/maintain?
>>>
>>> [2]: https://github.com/rnorth/mkdocs-codeinclude-plugin#usage
>> We are considering allowing to specify method bodies as complete snippet source or a region thereof.
>>
>>>> For inline snippets, especially those that are not a full compilation unit, it will be up to the test infrastructure to "wrap" the code fragment in a full compilation unit, such that it can be compiled and possibly executed.
>>> Would it be possible to keep the non-goal of not providing a standard tool to test them (to limit the scope of the JEP), but suggest a format/restrictions/expansion procedure on the snippets to support such test infra well (or enable adding it to JDK in the future)? I agree that external snippets work well (especially for large-ish fragments), but for some small things it is an overkill (you have to somehow configure the build system to build these files, but exclude from the final artifact, etc.), yet as a user I’d love to check their correctness. Would it make sense to:
>>> - Specify a standard way how a snippet is expanded (e.g., wrapped inside Callable#call, or some standard template — Java language designers are in a perfect position to pick a great one). [3]
>>> - Specify a set of tags that define the expectations of the expanded snippet behaviour (not sure it can be pulled into the scope, but the tools will require that information): [4]
>>>     - Compiles/fails compilation
>>>     - Runs successfully/Throws exception
>>>     - Is totally ignored.

What you describe is all part of the non-goal, but that does not mean it
will not be addressed at some point down the road. The primary test
harness for JDK itself is jtreg, and we will certainly be looking at how
to analyze snippets in the context of that system, and that may inform
users of other test frameworks. It's not clear that we need to impose
any guidelines in the spec of the tag itself: I think that guidelines
will arise from the context of the code used to analyze the snippets.


>>>
>>> In Rust doctests, for example, all these things are supported:
>>> [3] https://doc.rust-lang.org/rustdoc/documentation-tests.html#pre-processing-examples (adds main unless you do that)
>>> [4] https://doc.rust-lang.org/rustdoc/documentation-tests.html#attributes
>> The "Snippets" feature's first and foremost use case is JDK itself. Snippets found in "<pre>{@code" compounds in JDK are so different from each other that it doesn't seem possible to provide a solution sufficiently general to test those snippets automatically. Although the concept of "doctest", which you are referring to, offers a highly attractive end-to-end testing, that concept applies cleanly to snippets that were written with that concept in mind, such as snippets structured as tests or executable as-is. The thing is, we don't have many of those in JDK.
>>
>> Since we cannot generally ensure that every snippet is correct in the resulting documentation, we aim for the next best thing: enabling authors to ensure that the snippets are produced from correct sources. Authors don't have to build these sources, exclude them from the build artifacts, or use external snippets altogether; it's an option.
>>
>> While doctests are not immediately useful in JDK, we recognize that they might benefit other projects. That's why we are exposing snippets in JavaDoc API. That API may be used to test snippets externally or in JavaDoc. For example, using attributes of the {@snippet} tag, snippet markup and a custom taglet, one should be able to teach JavaDoc doctesting.
> Totally agree that an expansion scheme that works for arbitrary legacy
> code is an impossible task. I rather meant specifying a good enough
> expansion scheme for most snippets that are written with it in mind.
> For legacy ones (that, on top of being written without any
> restrictions, might reference non-existent symbols, or have an
> ellipsis in place of an initializing expression) there could be an
> option to skip executing, or even compiling such a snippet. Rustdoc,
> for instance, supports both. Any legacy snippets then could be
> migrated automatically with an option to not test them, and adjusted
> later.
>
> I agree that producing snippets from correct, separate sources is a
> useful feature. However, as it is simpler to write them inline and
> immediately see inside the non-rendered Javadoc, could you possibly
> consider specifying the expansion scheme, so that tools share the same
> specification (IDEs, a doctest tool)? For example, as Rust does
> specify that, these inline snippets are understood by both the doc
> test tool, and an IntelliJ plugin, which supports syntax highlighting,
> inspections and refactorings.
>
> Also, I think it may help the renderers: they could provide an option
> to copy a complete snippet, or run it in jshell (with all the hidden
> code, and, possibly, all the required imports inherited from the
> containing class).
>
> If there is already a way to add metadata to the snippet tag that
> could help communicate the intent to a doc testing tool, that's
> awesome.
>
>> While doctests are not immediately useful in JDK, we recognize that they might benefit other projects.
> Absolutely, I've certainly used them in the API specs in my projects,
> and benefitted from the examples in popular Java libraries: Mockito
> has 250+; Guava — over 500; and AssertJ — 2.5K (with some
> false-positives):
> - https://sourcegraph.com/search?q=%5E%5Cs*%5C*.%2B%3Cpre+lang:java+repo:mockito/mockito+count:1000&patternType=regexp
> - https://sourcegraph.com/search?q=%5E%5Cs*%5C*.%2B%3Cpre%3E+lang%3Ajava+repo%3Agoogle%2Fguava+count%3A1000&patternType=regexp
> - https://sourcegraph.com/search?q=%5E%5Cs*%5C*.%2B%3Cpre+lang:java+repo:assertj/assertj-core+count:1000&patternType=regexp
>
>
> Also, one more thing: there is a variety of present options for
> embedding code snippets with various restrictions, and I often had to
> reopen this comparison to choose the right one:
> https://reflectoring.io/howto-format-code-snippets-in-javadoc/#code-markup-features-at-a-glance
> Will @snippets require any escaping for "@"?
>
>>> Finally, you might be interested in the previous attempts to implement such a tool for Java: https://github.com/jakewins/javadoctest
>>> Beware the docs aren’t updated, its current version does not require writing test manually, and finds and extracts snippets on itself, e.g.:
>>> https://github.com/jakewins/javadoctest/blob/master/junit-platform-engine/src/test/java/javadoctest/engine/fixture/FixtureDocTestSimple.java#L18-L22
>>> --
>>> Best regards,
>>> Dmitry Timofeev
>> Thanks for providing the links to mkdocs and javadoctest.
>>
>> -Pavel
>>
>
> --
> Best,
> Dmitry