Re: RFR: 8252842: Extend jmap to support parallel heap dump [v10]

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: RFR: 8252842: Extend jmap to support parallel heap dump [v10]

Lin Zang-2
On Tue, 23 Feb 2021 08:23:49 GMT, Lin Zang <[hidden email]> wrote:

>> Hi @linzang,
>>
>> I've done more benchmarking using different numbers of threads for parallel heap iteration and have found values which give at least a factor of 2 speedup (for gzipped dumps) or 1.6 (for unzipped dumps). For my scenario using gzip compression about 10 percent of the available CPUs for parallel iteration gave the best speedup, for the uncompressed one it was about 7 percent.
>>
>> Note that the baseline I compared against was not the parallel=1 case, but the old code. The parallel=1 case was always 10 to 20 percent slower than the old code.
>>
>> Best regards,
>> Ralf
>
> Dear @ralf,
> Really Thanks for benchmarking it!
> It is a little surprise to me that "parallel=1" is 10~20 percent slower than before. I believe this can be avoid with some revise in code. And I also found a potential memory leak in the implementation, WIP in fixing it.
>
>> I've done more benchmarking using different numbers of threads for parallel heap iteration and have found values which give at least a factor of 2 speedup (for gzipped dumps) or 1.6 (for unzipped dumps). For my scenario using gzip compression about 10 percent of the available CPUs for parallel iteration gave the best speedup, for the uncompressed one it was about 7 percent.
>
> This data are really interest to me, it seems using gzipped dump is faster than unzipped dump, is the because of disk writing or something else? I would investigate more about it~
> Thanks a lot!
>
> BRs,
> Lin

Dear @plummercj,

I have reconsidered the solution of “dumpheapext”, IMHO maybe the name is too specific, for example, if in future there is more arguments for `-histo`, we have to made another command called "inspectheapext".
 
How about create a new command named (e.g.) "jmapext", and make the "dumpheapext" work as the subcommand and be passed to JVM as the 1st argument of "jmapext".  Then in future we don't bothering adding new command, just subcommand (or argument) like "inspectheapext". And hence there may be more easier to extend other commands?

What do you think?

BRs,
Lin

-------------

PR: https://git.openjdk.java.net/jdk/pull/2261
Reply | Threaded
Open this post in threaded view
|

Re: RFR: 8252842: Extend jmap to support parallel heap dump [v10]

Ralf Schmelter
On Tue, 23 Feb 2021 08:37:34 GMT, Lin Zang <[hidden email]> wrote:

>> Dear @ralf,
>> Really Thanks for benchmarking it!
>> It is a little surprise to me that "parallel=1" is 10~20 percent slower than before. I believe this can be avoid with some revise in code. And I also found a potential memory leak in the implementation, WIP in fixing it.
>>
>>> I've done more benchmarking using different numbers of threads for parallel heap iteration and have found values which give at least a factor of 2 speedup (for gzipped dumps) or 1.6 (for unzipped dumps). For my scenario using gzip compression about 10 percent of the available CPUs for parallel iteration gave the best speedup, for the uncompressed one it was about 7 percent.
>>
>> This data are really interest to me, it seems using gzipped dump is faster than unzipped dump, is the because of disk writing or something else? I would investigate more about it~
>> Thanks a lot!
>>
>> BRs,
>> Lin
>
> Dear @plummercj,
>
> I have reconsidered the solution of “dumpheapext”, IMHO maybe the name is too specific, for example, if in future there is more arguments for `-histo`, we have to made another command called "inspectheapext".
>  
> How about create a new command named (e.g.) "jmapext", and make the "dumpheapext" work as the subcommand and be passed to JVM as the 1st argument of "jmapext".  Then in future we don't bothering adding new command, just subcommand (or argument) like "inspectheapext". And hence there may be more easier to extend other commands?
>
> What do you think?
>
> BRs,
> Lin

Hi @linzang


> This data are really interest to me, it seems using gzipped dump is faster than unzipped dump, is the because of disk writing or something else? I would investigate more about it~

I would guess that is the case. In the compressed case we write about 800 MB per second, that is as fast as my SSD goes.

Here are some actual results I've gotten (uncompressed):

    jmap.exe -dump:file=ser.hprof,all 30600
    Dumping heap to ser.hprof ...
    Heap dump file created [32009882362 bytes in 59.303 secs]

    jmap.exe -dump:file=par1.hprof,parallel=1,all 30600
    Dumping heap to par1.hprof ...
    Heap dump file created [32009885809 bytes in 72.719 secs]

    jmap.exe -dump:file=par2.hprof,parallel=2,all 30600
    Dumping heap to par2.hprof ...
    Heap dump file created [32009881876 bytes in 57.546 secs]

    jmap.exe -dump:file=par4.hprof,parallel=4,all 30600
    Dumping heap to par4.hprof ...
    Heap dump file created [32009882956 bytes in 44.301 secs]

    jmap.exe -dump:file=par5.hprof,parallel=5,all 30600
    Dumping heap to par5.hprof ...
    Heap dump file created [32009882164 bytes in 40.282 secs]

    jmap.exe -dump:file=par6.hprof,parallel=6,all 30600
    Dumping heap to par6.hprof ...
    Heap dump file created [32009881156 bytes in 45.988 secs]

And here for the compressed case:

    jmap.exe -dump:file=par1.hprof.gz,parallel=1,all,gz=1 52372
    Dumping heap to par1.hprof.gz ...
    Heap dump file created [8076994216 bytes in 54.057 secs]

    jmap.exe -dump:file=par2.hprof.gz,parallel=2,all,gz=1 52372
    Dumping heap to par2.hprof.gz ...
    Heap dump file created [8075859421 bytes in 43.442 secs]

    jmap.exe -dump:file=par4.hprof.gz,parallel=4,all,gz=1 52372
    Dumping heap to par4.hprof.gz ...
    Heap dump file created [8075886152 bytes in 28.710 secs]

    jmap.exe -dump:file=par6.hprof.gz,parallel=6,all,gz=1 52372
    Dumping heap to par6.hprof.gz ...
    Heap dump file created [8075758374 bytes in 25.730 secs]

    jmap.exe -dump:file=par8.hprof.gz,parallel=8,all,gz=1 52372
    Dumping heap to par8.hprof.gz ...
    Heap dump file created [8075652558 bytes in 26.039 secs]

    jmap.exe -dump:file=par16.hprof.gz,parallel=16,all,gz=1 52372
    Dumping heap to par16.hprof.gz ...
    Heap dump file created [8075644423 bytes in 31.977 secs]

    jmap.exe -dump:file=par24.hprof.gz,parallel=24,all,gz=1 52372
    Dumping heap to par24.hprof.gz ...
    Heap dump file created [8075579546 bytes in 41.094 secs]

Best regards,
Ralf

-------------

PR: https://git.openjdk.java.net/jdk/pull/2261