RFR: JDK-8002152: javadoc could write out files on a worker thread

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

RFR: JDK-8002152: javadoc could write out files on a worker thread

Jonathan Gibbons-2
The standard doclet works by creating `HtmlDocument` objects and then writing them out. Writing them can be safely done on a separate thread, and doing so results in about an 8% speedup, by overlapping the write/IO time with the time to generate the next page to be written.

The overall impact on the codebase is deliberately small, coming down to a critical `if` statement in `HtmlDocletWriter` to "write now" or "write later". As such, it is easy to disable the feature if necessary, although no issues have been found in practice. The background writer uses an `ExecutorService`, which provides a lot of flexibility, but experiments show that it is sufficient to use a single background thread and to block if new tasks come available while writing a file.

The feature is "default on", with a hidden option to disable it if necessary.

-------------

Commit messages:
 - JDK-8002152: javadoc could write out files on a worker thread

Changes: https://git.openjdk.java.net/jdk/pull/2665/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=2665&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8002152
  Stats: 268 lines in 9 files changed: 265 ins; 0 del; 3 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2665.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2665/head:pull/2665

PR: https://git.openjdk.java.net/jdk/pull/2665
Reply | Threaded
Open this post in threaded view
|

Re: RFR: JDK-8002152: javadoc could write out files on a worker thread

Pavel Rappo-3
On Sun, 21 Feb 2021 18:11:06 GMT, Jonathan Gibbons <[hidden email]> wrote:

> The standard doclet works by creating `HtmlDocument` objects and then writing them out. Writing them can be safely done on a separate thread, and doing so results in about an 8% speedup, by overlapping the write/IO time with the time to generate the next page to be written.
>
> The overall impact on the codebase is deliberately small, coming down to a critical `if` statement in `HtmlDocletWriter` to "write now" or "write later". As such, it is easy to disable the feature if necessary, although no issues have been found in practice. The background writer uses an `ExecutorService`, which provides a lot of flexibility, but experiments show that it is sufficient to use a single background thread and to block if new tasks come available while writing a file.
>
> The feature is "default on", with a hidden option to disable it if necessary.

This change is on the right track.

1. Although I'm not sure how "utilization" translates to a similar measurement obtained through a profiling tool, I do think we can use "utilization" as a rough estimate.

FWIW, on my machine the base "utilization" for JDK codebase is 13%. After I have moved the initialization of the `start` timestamp from the creation of the writer closer to the submission of the first task, "utilization" increased to 17%.

I'd recommend computing elapsed times using `java.lang.System.nanoTime`. Using `nanoTime` avoids some hazards of using `currentTimeMillis` for such purposes.

2. The `--background-writer` option is processed **after** the writer has been created. To see the "utilization" numbers I had to tweak the change and rebuild the project.

src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/BackgroundWriter.java line 40:

> 38:
> 39: /**
> 40:  * A class to write {@link HtmlDocument} objects on a background thread,

I'd simply say "A class to write HtmlDocument objects in tasks of an ExecutorService" or some such. This is because `ExecutorService` is an abstraction of a level higher than that of `Thread`.

src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/BackgroundWriter.java line 79:

> 77:     /**
> 78:      * The main executor for the background tasks.
> 79:      * It uses a fixed thread pool of {@value #BACKGROUND_THREADS} threads.

What purpose does the adjective "main" have here?

src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/BackgroundWriter.java line 94:

> 92:      * The time at which the writer is initialized.
> 93:      */
> 94:     private long start;

I suggest making `start` final.

src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/BackgroundWriter.java line 105:

> 103:      * background thread.
> 104:      */
> 105:     private boolean verbose;

`verbose` should be final too.

src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/BackgroundWriter.java line 113:

> 111:      * The writer uses a {@link Executors#newFixedThreadPool fixed thread pool} of
> 112:      * {@value #BACKGROUND_THREADS} background threads, and a blocking queue of up to
> 113:      * {@value #QUEUED_TASKS} queued tasks.

The writer uses the pool but not the queue; the queue is encapsulated in the pool. Could you rephrase that?

src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/BackgroundWriter.java line 165:

> 163:         try {
> 164:             executor.shutdown();
> 165:             executor.awaitTermination(5, TimeUnit.MINUTES);

The writer can safely read the `taskBusy` field if `awaitTermination` returns `true`. Please check that boolean status.

src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/HtmlOptions.java line 718:

> 716:      * Set by command-lione option {@code --background-writer}.
> 717:      */
> 718:     public boolean isBackgroundWriterVerbose() {

Repeated typo "command-lione".

src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/BackgroundWriter.java line 2:

> 1: /*
> 2:  * Copyright (c) 2017, Oracle and/or its affiliates. All rights reserved.

Is this just an outdated copyright header or this class is really that old?

src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/BackgroundWriter.java line 97:

> 95:
> 96:     /**
> 97:      * The cumulative time spent writing documents

Missing the trailing period.

-------------

Changes requested by prappo (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/2665
Reply | Threaded
Open this post in threaded view
|

Re: RFR: JDK-8002152: javadoc could write out files on a worker thread

Pavel Rappo-3
On Mon, 22 Feb 2021 13:03:04 GMT, Pavel Rappo <[hidden email]> wrote:

>> The standard doclet works by creating `HtmlDocument` objects and then writing them out. Writing them can be safely done on a separate thread, and doing so results in about an 8% speedup, by overlapping the write/IO time with the time to generate the next page to be written.
>>
>> The overall impact on the codebase is deliberately small, coming down to a critical `if` statement in `HtmlDocletWriter` to "write now" or "write later". As such, it is easy to disable the feature if necessary, although no issues have been found in practice. The background writer uses an `ExecutorService`, which provides a lot of flexibility, but experiments show that it is sufficient to use a single background thread and to block if new tasks come available while writing a file.
>>
>> The feature is "default on", with a hidden option to disable it if necessary.
>
> This change is on the right track.
>
> 1. Although I'm not sure how "utilization" translates to a similar measurement obtained through a profiling tool, I do think we can use "utilization" as a rough estimate.
>
> FWIW, on my machine the base "utilization" for JDK codebase is 13%. After I have moved the initialization of the `start` timestamp from the creation of the writer closer to the submission of the first task, "utilization" increased to 17%.
>
> I'd recommend computing elapsed times using `java.lang.System.nanoTime`. Using `nanoTime` avoids some hazards of using `currentTimeMillis` for such purposes.
>
> 2. The `--background-writer` option is processed **after** the writer has been created. To see the "utilization" numbers I had to tweak the change and rebuild the project.

To add to my earlier comment https://github.com/openjdk/jdk/pull/2665#discussion_r580185985.

I reread the `ExecutorService` API to refresh my memory. It looks like there is **NO** happens-before edge between "actions taken by a task" and "awaitTermination returned true" . So, to read that `taskBusy` value safely we have to do something else.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2665
Reply | Threaded
Open this post in threaded view
|

Re: RFR: JDK-8002152: javadoc could write out files on a worker thread

Pavel Rappo-3
On Mon, 22 Feb 2021 13:36:00 GMT, Pavel Rappo <[hidden email]> wrote:

>> This change is on the right track.
>>
>> 1. Although I'm not sure how "utilization" translates to a similar measurement obtained through a profiling tool, I do think we can use "utilization" as a rough estimate.
>>
>> FWIW, on my machine the base "utilization" for JDK codebase is 13%. After I have moved the initialization of the `start` timestamp from the creation of the writer closer to the submission of the first task, "utilization" increased to 17%.
>>
>> I'd recommend computing elapsed times using `java.lang.System.nanoTime`. Using `nanoTime` avoids some hazards of using `currentTimeMillis` for such purposes.
>>
>> 2. The `--background-writer` option is processed **after** the writer has been created. To see the "utilization" numbers I had to tweak the change and rebuild the project.
>
> To add to my earlier comment https://github.com/openjdk/jdk/pull/2665#discussion_r580185985.
>
> I reread the `ExecutorService` API to refresh my memory. It looks like there is **NO** happens-before edge between "actions taken by a task" and "awaitTermination returned true" . So, to read that `taskBusy` value safely we have to do something else.

Here's some more thoughts. I tweaked this change to see when the thread that submits tasks cannot immediately acquire a permit. Using that tweaked change I built documentation for JDK for various number of queued tasks (QUEUED_TASKS). That allowed me to collect the following statistics:


| queue, n | attempts, n | failed, n | utilization, % |
| -------- | ----------- | --------- | -------------- |
|        0 |       14789 |      3346 |             19 |
|        5 |           " |       348 |             20 |
|       10 |           " |       120 |             20 |
|       15 |           " |        30 |             20 |
|       20 |           " |  10 |             19 |
|      100 |           " |         0 |             19 |

1. Either I'm misinterpreting these results or the original premises (1) "we currently consume (execute) tasks faster than we produce them" and (2) "A semaphore is used to throttle the number of requests, although in practice this rarely takes effect, because it is quicker to write documents than to generate them" do not hold on my machine. In particular, when QUEUED_TASKS == 0, the task-submitting thread fails to immediately acquire a permit in a quarter of attempts.
2. Note that "utilization" doesn't change even though the `ExecutorService` always has tasks to execute. That raises a question of how to interpret "utilization".

-------------

PR: https://git.openjdk.java.net/jdk/pull/2665
Reply | Threaded
Open this post in threaded view
|

Re: RFR: JDK-8002152: javadoc could write out files on a worker thread

Jonathan Gibbons-2
In reply to this post by Pavel Rappo-3
On Mon, 22 Feb 2021 11:48:10 GMT, Pavel Rappo <[hidden email]> wrote:

>> The standard doclet works by creating `HtmlDocument` objects and then writing them out. Writing them can be safely done on a separate thread, and doing so results in about an 8% speedup, by overlapping the write/IO time with the time to generate the next page to be written.
>>
>> The overall impact on the codebase is deliberately small, coming down to a critical `if` statement in `HtmlDocletWriter` to "write now" or "write later". As such, it is easy to disable the feature if necessary, although no issues have been found in practice. The background writer uses an `ExecutorService`, which provides a lot of flexibility, but experiments show that it is sufficient to use a single background thread and to block if new tasks come available while writing a file.
>>
>> The feature is "default on", with a hidden option to disable it if necessary.
>
> src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/BackgroundWriter.java line 2:
>
>> 1: /*
>> 2:  * Copyright (c) 2017, Oracle and/or its affiliates. All rights reserved.
>
> Is this just an outdated copyright header or this class is really that old?

Yes, the code really is that old.  At the time, we were disappointed that the speedup was not bigger (!) and that we could not easily make more of the generation be concurrent. But, given that the footprint of this fix on the code is low, and that we are elsewhere fixing perf regressions of under 5%, I think it is worth incorporating this code.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2665
Reply | Threaded
Open this post in threaded view
|

Re: RFR: JDK-8002152: javadoc could write out files on a worker thread

Jonathan Gibbons-2
In reply to this post by Pavel Rappo-3
On Mon, 22 Feb 2021 11:44:13 GMT, Pavel Rappo <[hidden email]> wrote:

>> The standard doclet works by creating `HtmlDocument` objects and then writing them out. Writing them can be safely done on a separate thread, and doing so results in about an 8% speedup, by overlapping the write/IO time with the time to generate the next page to be written.
>>
>> The overall impact on the codebase is deliberately small, coming down to a critical `if` statement in `HtmlDocletWriter` to "write now" or "write later". As such, it is easy to disable the feature if necessary, although no issues have been found in practice. The background writer uses an `ExecutorService`, which provides a lot of flexibility, but experiments show that it is sufficient to use a single background thread and to block if new tasks come available while writing a file.
>>
>> The feature is "default on", with a hidden option to disable it if necessary.
>
> src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/BackgroundWriter.java line 165:
>
>> 163:         try {
>> 164:             executor.shutdown();
>> 165:             executor.awaitTermination(5, TimeUnit.MINUTES);
>
> The writer can safely read the `taskBusy` field if `awaitTermination` returns `true`. Please check that boolean status.

I don't understand why you think it is important to check `taskBusy`. If nothing else, this is only in the verbose/utilization code.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2665
Reply | Threaded
Open this post in threaded view
|

Re: RFR: JDK-8002152: javadoc could write out files on a worker thread

Jonathan Gibbons-2
In reply to this post by Pavel Rappo-3
On Mon, 22 Feb 2021 16:35:43 GMT, Pavel Rappo <[hidden email]> wrote:

>> To add to my earlier comment https://github.com/openjdk/jdk/pull/2665#discussion_r580185985.
>>
>> I reread the `ExecutorService` API to refresh my memory. It looks like there is **NO** happens-before edge between "actions taken by a task" and "awaitTermination returned true" . So, to read that `taskBusy` value safely we have to do something else.
>
> Here's some more thoughts. I tweaked this change to see when the thread that submits tasks cannot immediately acquire a permit. Using that tweaked change I built documentation for JDK for various number of queued tasks (QUEUED_TASKS). That allowed me to collect the following statistics:
>
>
> | queue, n | attempts, n | failed, n | utilization, % |
> | -------- | ----------- | --------- | -------------- |
> |        0 |       14789 |      3346 |             19 |
> |        5 |           " |       348 |             20 |
> |       10 |           " |       120 |             20 |
> |       15 |           " |        30 |             20 |
> |       20 |           " |  10 |             19 |
> |      100 |           " |         0 |             19 |
>
> 1. Either I'm misinterpreting these results or the original premises (1) "we currently consume (execute) tasks faster than we produce them" and (2) "A semaphore is used to throttle the number of requests, although in practice this rarely takes effect, because it is quicker to write documents than to generate them" do not hold on my machine. In particular, when QUEUED_TASKS == 0, the task-submitting thread fails to immediately acquire a permit in a quarter of attempts.
> 2. Note that "utilization" doesn't change even though the `ExecutorService` always has tasks to execute. That raises a question of how to interpret "utilization".

Thanks for the detailed comments.

As regards your experiments for different queue sizes, it would be interesting to monitor the average size of the queue, to help pick a reasonable value.  But that being said, there is very little change in the utilization of the background writer as a function of the question size, which suggests it's not not important to raise the value ... and higher values may lead to higher memory usage.

Instead of increasing the queue size, it might be interesting to repeat the experiments to increase the number of threads.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2665
Reply | Threaded
Open this post in threaded view
|

Re: RFR: JDK-8002152: javadoc could write out files on a worker thread

Jonathan Gibbons-2
On Mon, 22 Feb 2021 17:56:25 GMT, Jonathan Gibbons <[hidden email]> wrote:

>> Here's some more thoughts. I tweaked this change to see when the thread that submits tasks cannot immediately acquire a permit. Using that tweaked change I built documentation for JDK for various number of queued tasks (QUEUED_TASKS). That allowed me to collect the following statistics:
>>
>>
>> | queue, n | attempts, n | failed, n | utilization, % |
>> | -------- | ----------- | --------- | -------------- |
>> |        0 |       14789 |      3346 |             19 |
>> |        5 |           " |       348 |             20 |
>> |       10 |           " |       120 |             20 |
>> |       15 |           " |        30 |             20 |
>> |       20 |           " |  10 |             19 |
>> |      100 |           " |         0 |             19 |
>>
>> 1. Either I'm misinterpreting these results or the original premises (1) "we currently consume (execute) tasks faster than we produce them" and (2) "A semaphore is used to throttle the number of requests, although in practice this rarely takes effect, because it is quicker to write documents than to generate them" do not hold on my machine. In particular, when QUEUED_TASKS == 0, the task-submitting thread fails to immediately acquire a permit in a quarter of attempts.
>> 2. Note that "utilization" doesn't change even though the `ExecutorService` always has tasks to execute. That raises a question of how to interpret "utilization".
>
> Thanks for the detailed comments.
>
> As regards your experiments for different queue sizes, it would be interesting to monitor the average size of the queue, to help pick a reasonable value.  But that being said, there is very little change in the utilization of the background writer as a function of the question size, which suggests it's not not important to raise the value ... and higher values may lead to higher memory usage.
>
> Instead of increasing the queue size, it might be interesting to repeat the experiments to increase the number of threads.

Over the weekend, I tried a few times to move option deciding into `BackgroundWriter`. In the end I decided it is not worth it for just the two values currently supported.  But, we could extend the option decoding to allow the number of threads and queue size to be specified, in which case I would add a static nested `Options` class into `BackgroundWriter` and delegate to that from `HtmlOptions`.

But equally, while these experiments are fun, I'm n to sure there's much more to be gained once we decide on some initial default values.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2665
Reply | Threaded
Open this post in threaded view
|

Re: RFR: JDK-8002152: javadoc could write out files on a worker thread

Pavel Rappo-3
In reply to this post by Jonathan Gibbons-2
On Mon, 22 Feb 2021 17:48:37 GMT, Jonathan Gibbons <[hidden email]> wrote:

>> src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/BackgroundWriter.java line 165:
>>
>>> 163:         try {
>>> 164:             executor.shutdown();
>>> 165:             executor.awaitTermination(5, TimeUnit.MINUTES);
>>
>> The writer can safely read the `taskBusy` field if `awaitTermination` returns `true`. Please check that boolean status.
>
> I don't understand why you think it is important to check `taskBusy`. If nothing else, this is only in the verbose/utilization code.

`taskBusy` is accessed from multiple threads. Since those accesses are conflicting they should be correctly synchronized, which currently they are not. Also see: https://github.com/openjdk/jdk/pull/2665#issuecomment-783378558

-------------

PR: https://git.openjdk.java.net/jdk/pull/2665
Reply | Threaded
Open this post in threaded view
|

Re: RFR: JDK-8002152: javadoc could write out files on a worker thread

Pavel Rappo-3
In reply to this post by Jonathan Gibbons-2
On Mon, 22 Feb 2021 17:56:25 GMT, Jonathan Gibbons <[hidden email]> wrote:

> there is very little change in the utilization of the background writer as a function of the question size

Did you mean "queue size"? If so, then I also noticed that in my observation no. 2 under the results table here: https://github.com/openjdk/jdk/pull/2665#issuecomment-783503529 For me, utilization roughly stayed at the same 20% mark regardless of the number of queued tasks.

To me, this indicates conflicting readings. On the one hand, we have 20% of "utilization". On the other hand, 1/4 of attempts to acquire a permit to hand over a task to the executor fail. Put differently, in 1/4 of the cases the producer waits for the consumer who works at 20% capacity. Why?

I think we should use suitable tools (e.g. profilers) and/or ask experts in performance. If nothing else, this will help us diagnose such issues in the future.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2665
Reply | Threaded
Open this post in threaded view
|

Re: RFR: JDK-8002152: javadoc could write out files on a worker thread

Jonathan Gibbons-2
In reply to this post by Pavel Rappo-3
On Mon, 22 Feb 2021 20:31:01 GMT, Pavel Rappo <[hidden email]> wrote:

>> I don't understand why you think it is important to check `taskBusy`. If nothing else, this is only in the verbose/utilization code.
>
> `taskBusy` is accessed from multiple threads. Since those accesses are conflicting they should be correctly synchronized, which currently they are not. Also see: https://github.com/openjdk/jdk/pull/2665#issuecomment-783378558

Understood; I'm changing it to `AtomicLong` and using `AtomicLong.addAndget`

-------------

PR: https://git.openjdk.java.net/jdk/pull/2665
Reply | Threaded
Open this post in threaded view
|

Re: RFR: JDK-8002152: javadoc could write out files on a worker thread

Jonathan Gibbons-2
In reply to this post by Pavel Rappo-3
On Mon, 22 Feb 2021 11:45:21 GMT, Pavel Rappo <[hidden email]> wrote:

>> The standard doclet works by creating `HtmlDocument` objects and then writing them out. Writing them can be safely done on a separate thread, and doing so results in about an 8% speedup, by overlapping the write/IO time with the time to generate the next page to be written.
>>
>> The overall impact on the codebase is deliberately small, coming down to a critical `if` statement in `HtmlDocletWriter` to "write now" or "write later". As such, it is easy to disable the feature if necessary, although no issues have been found in practice. The background writer uses an `ExecutorService`, which provides a lot of flexibility, but experiments show that it is sufficient to use a single background thread and to block if new tasks come available while writing a file.
>>
>> The feature is "default on", with a hidden option to disable it if necessary.
>
> src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/HtmlOptions.java line 718:
>
>> 716:      * Set by command-lione option {@code --background-writer}.
>> 717:      */
>> 718:     public boolean isBackgroundWriterVerbose() {
>
> Repeated typo "command-lione".

I've moved option handling into `BackgroundWriter`.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2665
Reply | Threaded
Open this post in threaded view
|

Re: RFR: JDK-8002152: javadoc could write out files on a worker thread [v2]

Jonathan Gibbons-2
In reply to this post by Jonathan Gibbons-2
> The standard doclet works by creating `HtmlDocument` objects and then writing them out. Writing them can be safely done on a separate thread, and doing so results in about an 8% speedup, by overlapping the write/IO time with the time to generate the next page to be written.
>
> The overall impact on the codebase is deliberately small, coming down to a critical `if` statement in `HtmlDocletWriter` to "write now" or "write later". As such, it is easy to disable the feature if necessary, although no issues have been found in practice. The background writer uses an `ExecutorService`, which provides a lot of flexibility, but experiments show that it is sufficient to use a single background thread and to block if new tasks come available while writing a file.
>
> The feature is "default on", with a hidden option to disable it if necessary.

Jonathan Gibbons has updated the pull request incrementally with one additional commit since the last revision:

  Address review comments
  Improve option handling.

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/2665/files
  - new: https://git.openjdk.java.net/jdk/pull/2665/files/e9c4b11d..68702597

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2665&range=01
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2665&range=00-01

  Stats: 170 lines in 5 files changed: 105 ins; 29 del; 36 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2665.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2665/head:pull/2665

PR: https://git.openjdk.java.net/jdk/pull/2665
Reply | Threaded
Open this post in threaded view
|

Re: RFR: JDK-8002152: javadoc could write out files on a worker thread [v2]

Jonathan Gibbons-2
In reply to this post by Pavel Rappo-3
On Mon, 22 Feb 2021 20:55:08 GMT, Pavel Rappo <[hidden email]> wrote:

> I think we should use suitable tools (e.g. profilers) and/or ask experts in performance. If nothing else, this will help us diagnose such issues in the future.

Yes, but only a little bit yes. I think the "utilization" of minimal/questionable value. The bottom line is a real-time reduction of 10% or more, and, in my opinion, that is enough to declare success.

The latest commit provides more fine-grain control over the number of background threads and queue size, and increasing the number of threads or the queue size actually starts to degrade overall performance, as measured in real time over several runs of building the JDK docs.

I think there are diminishing returns beyond this point, given that we cannot easily introduce concurrency into the page generation (sigh) because of the underlying javac architecture.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2665
Reply | Threaded
Open this post in threaded view
|

Re: RFR: JDK-8002152: javadoc could write out files on a worker thread [v3]

Jonathan Gibbons-2
In reply to this post by Jonathan Gibbons-2
> The standard doclet works by creating `HtmlDocument` objects and then writing them out. Writing them can be safely done on a separate thread, and doing so results in about an 8% speedup, by overlapping the write/IO time with the time to generate the next page to be written.
>
> The overall impact on the codebase is deliberately small, coming down to a critical `if` statement in `HtmlDocletWriter` to "write now" or "write later". As such, it is easy to disable the feature if necessary, although no issues have been found in practice. The background writer uses an `ExecutorService`, which provides a lot of flexibility, but experiments show that it is sufficient to use a single background thread and to block if new tasks come available while writing a file.
>
> The feature is "default on", with a hidden option to disable it if necessary.

Jonathan Gibbons has updated the pull request incrementally with one additional commit since the last revision:

  Downgrade BackgroundWriter config values from hidden options to "internal" system properties.

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/2665/files
  - new: https://git.openjdk.java.net/jdk/pull/2665/files/68702597..939cc080

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2665&range=02
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2665&range=01-02

  Stats: 146 lines in 6 files changed: 24 ins; 106 del; 16 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2665.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2665/head:pull/2665

PR: https://git.openjdk.java.net/jdk/pull/2665