Re: RFR: JDK-8260926: Trace resource exhausted events unconditionally [v2]

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: RFR: JDK-8260926: Trace resource exhausted events unconditionally [v2]

Thomas Stuefe
> Analyzing out-of-resource situations in cloud scenarios is no fun. With CloudFoundry, a JVMTI agent (jvmkill) is hooked up intercepting the jvmti "resource exhausted" event, then attempts to write up a heap report. That may fail, e.g. due to bugs in the agent [1], but also because that report runs java code and may suffer from the same resource exhaustion. Successful or not, it unceremoniously kills the VM when done, often leaving us with no information about the actual resource.
>
> It would be very helpful if we had unconditional tracing here. We do have tracing, but it requires a non-product build and is triggered with TraceJVMTI. Also, it traces at trace level which is way to fine granular.
>
> I'd like to introduce another, unconditional trace line here. Arguably, resource exhausted is fatal enough that it justifies unconditional tracing.
>
> This is a bit of a coin toss. Tracing unconditionally would help in most scenarios, where it would be either difficult or even impossible to specify a trace command line switch. OTOH it may trip up scripts parsing the VM output, or some of our tests (which can be fixed).
>
> Thoughts?
>
> ..Thomas
>
> [1] https://github.com/cloudfoundry/jvmkill/issues/18

Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision:

  Feedback David

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/2350/files
  - new: https://git.openjdk.java.net/jdk/pull/2350/files/abe2bf60..40e3af87

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2350&range=01
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2350&range=00-01

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2350.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2350/head:pull/2350

PR: https://git.openjdk.java.net/jdk/pull/2350
Reply | Threaded
Open this post in threaded view
|

Re: RFR: JDK-8260926: Trace resource exhausted events unconditionally [v2]

Coleen Phillimore-3
On Wed, 3 Feb 2021 06:39:01 GMT, Thomas Stuefe <[hidden email]> wrote:

>> Analyzing out-of-resource situations in cloud scenarios is no fun. With CloudFoundry, a JVMTI agent (jvmkill) is hooked up intercepting the jvmti "resource exhausted" event, then attempts to write up a heap report. That may fail, e.g. due to bugs in the agent [1], but also because that report runs java code and may suffer from the same resource exhaustion. Successful or not, it unceremoniously kills the VM when done, often leaving us with no information about the actual resource.
>>
>> It would be very helpful if we had unconditional tracing here. We do have tracing, but it requires a non-product build and is triggered with TraceJVMTI. Also, it traces at trace level which is way to fine granular.
>>
>> I'd like to introduce another, unconditional trace line here. Arguably, resource exhausted is fatal enough that it justifies unconditional tracing.
>>
>> This is a bit of a coin toss. Tracing unconditionally would help in most scenarios, where it would be either difficult or even impossible to specify a trace command line switch. OTOH it may trip up scripts parsing the VM output, or some of our tests (which can be fixed).
>>
>> Thoughts?
>>
>> ..Thomas
>>
>> [1] https://github.com/cloudfoundry/jvmkill/issues/18
>
> Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision:
>
>   Feedback David

This looks good!

-------------

Marked as reviewed by coleenp (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/2350
Reply | Threaded
Open this post in threaded view
|

Re: RFR: JDK-8260926: Trace resource exhausted events unconditionally [v2]

David Holmes-2
In reply to this post by Thomas Stuefe
On Wed, 3 Feb 2021 06:39:01 GMT, Thomas Stuefe <[hidden email]> wrote:

>> Analyzing out-of-resource situations in cloud scenarios is no fun. With CloudFoundry, a JVMTI agent (jvmkill) is hooked up intercepting the jvmti "resource exhausted" event, then attempts to write up a heap report. That may fail, e.g. due to bugs in the agent [1], but also because that report runs java code and may suffer from the same resource exhaustion. Successful or not, it unceremoniously kills the VM when done, often leaving us with no information about the actual resource.
>>
>> It would be very helpful if we had unconditional tracing here. We do have tracing, but it requires a non-product build and is triggered with TraceJVMTI. Also, it traces at trace level which is way to fine granular.
>>
>> I'd like to introduce another, unconditional trace line here. Arguably, resource exhausted is fatal enough that it justifies unconditional tracing.
>>
>> This is a bit of a coin toss. Tracing unconditionally would help in most scenarios, where it would be either difficult or even impossible to specify a trace command line switch. OTOH it may trip up scripts parsing the VM output, or some of our tests (which can be fixed).
>>
>> Thoughts?
>>
>> ..Thomas
>>
>> [1] https://github.com/cloudfoundry/jvmkill/issues/18
>
> Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision:
>
>   Feedback David

I would have formatted the message differently as per my first comment. But ok.

-------------

Marked as reviewed by dholmes (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/2350