RFD: Add new jcmd internal command - dumpheapext to solve the attach API's argument number limitation

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

RFD: Add new jcmd internal command - dumpheapext to solve the attach API's argument number limitation

linzang(臧琳)

Dear All,

            I want to get your suggestion about adding a new internal command to solve the argument passing problem of jmap alike tools.

 

             We recently encounter a problem when adding the 4th option (“parallel”) to jmap -dump, it failed because there is limitation of max number of arguments passed to JVM in attach API. As limited by arguments length at https://github.com/openjdk/jdk/blob/95d73129ce5074d3510710e7e238761a9af9ef3a/src/jdk.attach/aix/classes/sun/tools/attach/VirtualMachineImpl.java#L144  and arg_count_max at https://github.com/openjdk/jdk/blob/95d73129ce5074d3510710e7e238761a9af9ef3a/src/hotspot/share/services/attachListener.hpp#L138.

 

             And simply enlarge this  limitation is not acceptable because it cause old version of jmap command hang when working with new JVM, which has been encountered and disscussed before at https://mail.openjdk.java.net/pipermail/serviceability-dev/2019-February/027240.html, However the problem was not fix at that time because the limitation is modified unnecessarily and simply revert the change is ok.

 

            As mentioned, now we are facing the problem when developing the parallel heap dump, which introduced the 4th argument “parallel=<N>” into jmap -dump command. We (Chris, Serguei, Hamlin and I) recently discussed it and Chris has suggested a solution to fix this kind of issue by introducing a new internal command “dumpheapext”. When “jmap -dump” command work with “parallel” option, it issues “dumpheapext” command instead of “dumpheap” command, and this dumpheapext has same argument passing protocal with JVM – e.g. “all arguments passed is combined into one string, passing the whole string to jvm, and the jvm parse it to get correct ones”.

      

            With this change, old jvm will print error like “Operation dumpheapext not recognized!” when work with new “jmap -dump:parallel=<n>” command.  IMO this is not perfact as error message like “unsupported argument parallel=<n>”, but the command behaves correctly and in this way there is no severe compatibility issue introduced.

 

            Here are some of the discussions before, FYI:

  1. https://github.com/openjdk/jdk/pull/2261 - Parallel heap dump PR, with latest commit of dumpheapext command prototype. Also some history of the story included in this PR.
  2. https://github.com/openjdk/jdk/pull/2379 -  improve jcmd GC.class_histogram to support parallel, we also discussed serveral solution here.
  3. https://mail.openjdk.java.net/pipermail/serviceability-dev/2019-February/027240.html The problem encountered proviously, and also discussions related.

           

 

 

BRs,

Lin

Reply | Threaded
Open this post in threaded view
|

Re: [External] : RFD: Add new jcmd internal command - dumpheapext to solve the attach API's argument number limitation

Chris Plummer
I just want to clarify that what is being proposed is a new attach API command (not a jcmd). We want to extend the following to include "dumpheapext":

static AttachOperationFunctionInfo funcs[] = {
  { "agentProperties",  get_agent_properties },
  { "datadump",         data_dump },
  { "dumpheap",         dump_heap },
  { "load",             load_agent },
  { "properties",       get_system_properties },
  { "threaddump",       thread_dump },
  { "inspectheap",      heap_inspection },
  { "setflag",          set_flag },
  { "printflag",        print_flag },
  { "jcmd",             jcmd },
  { NULL,               NULL }
};

And the name "dumpheapext" is just a suggestion ("ext" for "extended"). Suggestions for a better name are welcome.

I also want to point out that dumpheapext would pass arguments in such a way that in the future more arguments could be added without having to add another attach API command (eg. dumpheapextv2).

thanks,

Chris

On 2/12/21 8:18 PM, linzang(臧琳) wrote:

Dear All,

            I want to get your suggestion about adding a new internal command to solve the argument passing problem of jmap alike tools.

 

             We recently encounter a problem when adding the 4th option (“parallel”) to jmap -dump, it failed because there is limitation of max number of arguments passed to JVM in attach API. As limited by arguments length at https://github.com/openjdk/jdk/blob/95d73129ce5074d3510710e7e238761a9af9ef3a/src/jdk.attach/aix/classes/sun/tools/attach/VirtualMachineImpl.java#L144  and arg_count_max at https://github.com/openjdk/jdk/blob/95d73129ce5074d3510710e7e238761a9af9ef3a/src/hotspot/share/services/attachListener.hpp#L138.

 

             And simply enlarge this  limitation is not acceptable because it cause old version of jmap command hang when working with new JVM, which has been encountered and disscussed before at https://mail.openjdk.java.net/pipermail/serviceability-dev/2019-February/027240.html, However the problem was not fix at that time because the limitation is modified unnecessarily and simply revert the change is ok.

 

            As mentioned, now we are facing the problem when developing the parallel heap dump, which introduced the 4th argument “parallel=<N>” into jmap -dump command. We (Chris, Serguei, Hamlin and I) recently discussed it and Chris has suggested a solution to fix this kind of issue by introducing a new internal command “dumpheapext”. When “jmap -dump” command work with “parallel” option, it issues “dumpheapext” command instead of “dumpheap” command, and this dumpheapext has same argument passing protocal with JVM – e.g. “all arguments passed is combined into one string, passing the whole string to jvm, and the jvm parse it to get correct ones”.

      

            With this change, old jvm will print error like “Operation dumpheapext not recognized!” when work with new “jmap -dump:parallel=<n>” command.  IMO this is not perfact as error message like “unsupported argument parallel=<n>”, but the command behaves correctly and in this way there is no severe compatibility issue introduced.

 

            Here are some of the discussions before, FYI:

  1. https://github.com/openjdk/jdk/pull/2261 - Parallel heap dump PR, with latest commit of dumpheapext command prototype. Also some history of the story included in this PR.
  2. https://github.com/openjdk/jdk/pull/2379 -  improve jcmd GC.class_histogram to support parallel, we also discussed serveral solution here.
  3. https://mail.openjdk.java.net/pipermail/serviceability-dev/2019-February/027240.html The problem encountered proviously, and also discussions related.

           

 

 

BRs,

Lin