Question regarding "native-compiled frame"

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Question regarding "native-compiled frame"

Felix Yang-2
Hi,

    Can some help explain what is a so called "native-compiled frame" in
hotspot please?
    This is mentioned in the frame::sender function which is defined in
file frame_x86.cpp:

----------------------------------------------------------------------------------------------------------
  frame frame::sender(RegisterMap* map) const {
      // Default is we done have to follow them. The sender_for_xxx will
      // update it accordingly
      map->set_include_argument_oops(false);

      if (is_entry_frame())       return sender_for_entry_frame(map);
      if (is_interpreted_frame()) return sender_for_interpreter_frame(map);
      assert(_cb == CodeCache::find_blob(pc()),"Must be the same");

      if (_cb != NULL) {
        return sender_for_compiled_frame(map);
      }

      // Must be native-compiled frame, i.e. the marshaling code for native
      // methods that exists in the core system.
=>  return frame(sender_sp(), link(), sender_pc());
  }

----------------------------------------------------------------------------------------------------------
    I did some experiments and found that the last "return" statement of
the function never gets executed when I do a specjbb2005 & specjbb2015 test.
    From the comments in the code, this "return" statement handles the
"native-compiled frame" case.
    I am curious about the condition of generating a native-compile frame
in hotspot.

Thanks for your help,
Felix
Reply | Threaded
Open this post in threaded view
|

Re: Question regarding "native-compiled frame"

Andrew Haley
On 06/09/17 14:55, Felix Yang wrote:
>     Can some help explain what is a so called "native-compiled frame" in
> hotspot please?

Compiled by C++.

--
Andrew Haley
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671
Reply | Threaded
Open this post in threaded view
|

Re: Question regarding "native-compiled frame"

Felix Yang-2
Hi,

  Thanks for the reply.
  Then when will the last return statement of frame::sender got a chance to
be executed?
  As I see it, when JVM does something in safepoint state and need to
traverse Java thread stack, we never calculate the sender of a
native-compiled frame.
  Except for running Specjbb, I also performed a full jtreg test in order
to catch one such case, but still the last return statement not hit.
  So it's strange to me that frame::sender needs to handle such a frame.
  Maybe I missed something.  Could you elaborate more on this please?

Thanks for your help,
Felix

On 6 September 2017 at 22:10, Andrew Haley <[hidden email]> wrote:

> On 06/09/17 14:55, Felix Yang wrote:
> >     Can some help explain what is a so called "native-compiled frame" in
> > hotspot please?
>
> Compiled by C++.
>
> --
> Andrew Haley
> Java Platform Lead Engineer
> Red Hat UK Ltd. <https://www.redhat.com>
> EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671
>
Reply | Threaded
Open this post in threaded view
|

Re: Question regarding "native-compiled frame"

Andrew Dinn
On 07/09/17 11:35, Felix Yang wrote:
>   Thanks for the reply.
>   Then when will the last return statement of frame::sender got a chance to
> be executed?
>   As I see it, when JVM does something in safepoint state and need to
> traverse Java thread stack, we never calculate the sender of a
> native-compiled frame.

It is possible for Java to call out into the JVM and then for the JVM to
call back into Java. For example, when a class is loaded the JVM calls
into Java to run the class initializer. This re-entry may happen
multiple times.

In that case a stack walk under the re-entry may find a Java start fame
and it's parent frame will be the native frame where Java entered the VM.

Note that the native frame will always be returned by the call to
sender_for_entry_frame(map). That method skips all the C frames between
the Java entry frame and the native frame which exited Java.

regards,


Andrew Dinn
-----------
Senior Principal Software Engineer
Red Hat UK Ltd
Registered in England and Wales under Company Registration No. 03798903
Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander
Reply | Threaded
Open this post in threaded view
|

Re: Question regarding "native-compiled frame"

Felix Yang-2
On 7 September 2017 at 19:04, Andrew Dinn <[hidden email]> wrote:

> On 07/09/17 11:35, Felix Yang wrote:
> >   Thanks for the reply.
> >   Then when will the last return statement of frame::sender got a chance
> to
> > be executed?
> >   As I see it, when JVM does something in safepoint state and need to
> > traverse Java thread stack, we never calculate the sender of a
> > native-compiled frame.
>
> It is possible for Java to call out into the JVM and then for the JVM to
> call back into Java. For example, when a class is loaded the JVM calls
> into Java to run the class initializer. This re-entry may happen
> multiple times.
>
> In that case a stack walk under the re-entry may find a Java start fame
> and it's parent frame will be the native frame where Java entered the VM.
>

Yes, that's the frame structure.


> Note that the native frame will always be returned by the call to
> sender_for_entry_frame(map). That method skips all the C frames between
> the Java entry frame and the native frame which exited Java.
>

True. And this is handled by the three IF statements of frame::sender
function.
It seems to me that the last return statement is not involved in the stack
walking process, isn't it?

Thanks for your help,
Felix
Reply | Threaded
Open this post in threaded view
|

Re: Question regarding "native-compiled frame"

Chris Plummer
On 9/13/17 6:10 AM, Felix Yang wrote:

> On 7 September 2017 at 19:04, Andrew Dinn <[hidden email]> wrote:
>
>> On 07/09/17 11:35, Felix Yang wrote:
>>>    Thanks for the reply.
>>>    Then when will the last return statement of frame::sender got a chance
>> to
>>> be executed?
>>>    As I see it, when JVM does something in safepoint state and need to
>>> traverse Java thread stack, we never calculate the sender of a
>>> native-compiled frame.
>> It is possible for Java to call out into the JVM and then for the JVM to
>> call back into Java. For example, when a class is loaded the JVM calls
>> into Java to run the class initializer. This re-entry may happen
>> multiple times.
>>
>> In that case a stack walk under the re-entry may find a Java start fame
>> and it's parent frame will be the native frame where Java entered the VM.
>>
> Yes, that's the frame structure.
>
>
>> Note that the native frame will always be returned by the call to
>> sender_for_entry_frame(map). That method skips all the C frames between
>> the Java entry frame and the native frame which exited Java.
>>
> True. And this is handled by the three IF statements of frame::sender
> function.
> It seems to me that the last return statement is not involved in the stack
> walking process, isn't it?
Hi Felix,

Are you talking about the following:

   // Must be native-compiled frame, i.e. the marshaling code for native
   // methods that exists in the core system.
   return frame(sender_sp(), link(), sender_pc());

If so, the Oracle arm port used to have this same code. However, 2-3
years ago I was doing quite a few fixes to arm stack/frame walking, and
suspected this code was never executed. In 9 I changed to:

   assert(false, "should not be called for a C frame");
   return frame();

We've never hit this assert in all our testing since. Possibly this
change could also be made to x86. However, the details of why I thought
this change was ok has escaped me.

cheers,

Chris
>
> Thanks for your help,
> Felix



Reply | Threaded
Open this post in threaded view
|

Re: Question regarding "native-compiled frame"

Felix Yang-2
On 16 September 2017 at 02:15, Chris Plummer <[hidden email]>
wrote:

> On 9/13/17 6:10 AM, Felix Yang wrote:
>
>> On 7 September 2017 at 19:04, Andrew Dinn <[hidden email]> wrote:
>>
>> On 07/09/17 11:35, Felix Yang wrote:
>>>
>>>>    Thanks for the reply.
>>>>    Then when will the last return statement of frame::sender got a
>>>> chance
>>>>
>>> to
>>>
>>>> be executed?
>>>>    As I see it, when JVM does something in safepoint state and need to
>>>> traverse Java thread stack, we never calculate the sender of a
>>>> native-compiled frame.
>>>>
>>> It is possible for Java to call out into the JVM and then for the JVM to
>>> call back into Java. For example, when a class is loaded the JVM calls
>>> into Java to run the class initializer. This re-entry may happen
>>> multiple times.
>>>
>>> In that case a stack walk under the re-entry may find a Java start fame
>>> and it's parent frame will be the native frame where Java entered the VM.
>>>
>>> Yes, that's the frame structure.
>>
>>
>> Note that the native frame will always be returned by the call to
>>> sender_for_entry_frame(map). That method skips all the C frames between
>>> the Java entry frame and the native frame which exited Java.
>>>
>>> True. And this is handled by the three IF statements of frame::sender
>> function.
>> It seems to me that the last return statement is not involved in the stack
>> walking process, isn't it?
>>
> Hi Felix,
>
> Are you talking about the following:
>
>   // Must be native-compiled frame, i.e. the marshaling code for native
>   // methods that exists in the core system.
>   return frame(sender_sp(), link(), sender_pc());
>

Yes, I want to know which kind of frame we are handling here.


>
> If so, the Oracle arm port used to have this same code. However, 2-3 years
> ago I was doing quite a few fixes to arm stack/frame walking, and suspected
> this code was never executed. In 9 I changed to:
>
>   assert(false, "should not be called for a C frame");
>   return frame();
>
> We've never hit this assert in all our testing since. Possibly this change
> could also be made to x86. However, the details of why I thought this
> change was ok has escaped me.


Thanks for this helpful information.
I performed JTreg and specjbb test and I find this code never hit on x86
platform.
So I also think of this return statement for x86 as dead code. Don't know
why the code is there.

Thanks for your help,
Felix
Reply | Threaded
Open this post in threaded view
|

Re: Question regarding "native-compiled frame"

Andrew Haley
On 16/09/17 10:43, Felix Yang wrote:
> Thanks for this helpful information.
> I performed JTreg and specjbb test and I find this code never hit on x86
> platform.
> So I also think of this return statement for x86 as dead code. Don't know
> why the code is there.

We'll probably never know.  We should take the x86 patch and apply it to AArch64
for JDK 10.

--
Andrew Haley
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671
Reply | Threaded
Open this post in threaded view
|

Re: Question regarding "native-compiled frame"

Felix Yang-2
Yes, the AArch64 port is doing the same thing as the x86 port.

I noticed this issue when I was trying to analyze a recent JVM crash issue
which triggers on the x86-64 platform with an official Oracle jdk8u121
release.

It turns out that this bug happens very rarely (triggered only once for my
Java workload and I am unable to reproduce it).

And I think it is the same as:
https://bugs.openjdk.java.net/browse/JDK-8146224 which is still pending
there.

Looking at the assembly instructions in the hs error log file, I find the
SIGSEGV is triggered by the last return statement of frame::sender
function.

Currently, I disabled the FindDeadLocks VM_Operation in my Java application
trying to lower the possibility of hitting the bug. But I am afraid that it
is something of a ticking bomb for us.

Thanks for your help,
Felix


On 18 September 2017 at 16:24, Andrew Haley <[hidden email]> wrote:

> On 16/09/17 10:43, Felix Yang wrote:
> > Thanks for this helpful information.
> > I performed JTreg and specjbb test and I find this code never hit on x86
> > platform.
> > So I also think of this return statement for x86 as dead code. Don't know
> > why the code is there.
>
> We'll probably never know.  We should take the x86 patch and apply it to
> AArch64
> for JDK 10.
>
> --
> Andrew Haley
> Java Platform Lead Engineer
> Red Hat UK Ltd. <https://www.redhat.com>
> EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671
>
Reply | Threaded
Open this post in threaded view
|

Re: Question regarding "native-compiled frame"

Andrew Haley
On 18/09/17 15:01, Felix Yang wrote:
> Currently, I disabled the FindDeadLocks VM_Operation in my Java application
> trying to lower the possibility of hitting the bug. But I am afraid that it
> is something of a ticking bomb for us.

It's clearly stack corruption of some kind.  It might even be a bug in
the host C++ compiler.  We'll never know until we can reproduce it.

--
Andrew Haley
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671
Reply | Threaded
Open this post in threaded view
|

Re: Question regarding "native-compiled frame"

Felix Yang-2
In reply to this post by Felix Yang-2
Hi,

    Sorry for the late reply. I missed this email.
    Unfortunately, I wasn't able to catch a core dump file of the JVM.
    The hs error log is the same as the one reported on:
https://bugs.openjdk.java.net/browse/JDK-8146224

Thanks.
Felix


On 19 September 2017 at 02:24, Chris Plummer <[hidden email]>
wrote:

> On 9/18/17 7:01 AM, Felix Yang wrote:
>
> Yes, the AArch64 port is doing the same thing as the x86 port.
>
> I noticed this issue when I was trying to analyze a recent JVM crash issue
> which triggers on the x86-64 platform with an official Oracle jdk8u121
> release.
>
> It turns out that this bug happens very rarely (triggered only once for my
> Java workload and I am unable to reproduce it).
>
> And I think it is the same as: https://bugs.openjdk.java.net/
> browse/JDK-8146224 which is still pending there.
>
> Looking at the assembly instructions in the hs error log file, I find the
> SIGSEGV is triggered by the last return statement of frame::sender
> function.
>
> frame::safe_for_sender() is suppose to prevent us from stepping into a
> frame that could result in a JVM crash, even if the frame is corrupted. Do
> you have any additional debugging info, like a gdb stack trace?
>