alternate pns() debug function

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

alternate pns() debug function

Chris Plummer
Hi,

I'd like to add an alternate version of pns() to debug.cpp, and would
like some initial comments first before filing a bug and sending out an
RFR. pns() is used to print out the native stack:

extern "C" void pns(void* sp, void* fp, void* pc) { // print native stack
   Command c("pns");
   static char buf[O_BUFLEN];
   Thread* t = Thread::current_or_null();
   // Call generic frame constructor (certain arguments may be ignored)
   frame fr(sp, fp, pc);
   VMError::print_native_stack(tty, fr, t, buf, sizeof(buf));
}

And here's the help output for it:

   pns(void* sp, void* fp, void* pc)  - print native (i.e. mixed) stack
trace. E.g.");
                    pns($sp, $rbp, $pc) on Linux/amd64 and Solaris/amd64
or");
                    pns($sp, $ebp, $pc) on Linux/x86 or");
                    pns($sp, 0, $pc)    on Linux/ppc64 or");
                    pns($sp + 0x7ff, 0, $pc) on Solaris/SPARC");
                  - in gdb do 'set overload-resolution off' before
calling pns()");
                  - in dbx do 'frame 1' before calling pns()");

The intent is to call it from gdb, but it is potentially useful to call
it from within hotspot when debugging. I'd like to propose the following
alternative, since it does away with needing to pass in register values
(not easy to do from C):

extern "C" void pns2() { // print native stack
   Command c("pns2");
   static char buf[O_BUFLEN];
   Thread* t = Thread::current_or_null();
   frame fr = os::current_frame();
   VMError::print_native_stack(tty, fr, t, buf, sizeof(buf));
}

I actually used this quite a bit with some recent debugging. There were
places in hotspot where I wanted to know what the native stack looked
like when called, and it was a lot easier to just add a call to pns2()
than to setup the test to run in gdb. This seems to work on Linux/x64,
macosx/64, and solaris/sparc. For some reason it crashes on Windows/x64,
but I don't see any indication from the above help for pns() that it
works on Windows/x64 anyway.

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x0000003bcfae1466,
pid=9872, tid=12212
#
# JRE version: Java(TM) SE Runtime Environment (10.0) (fastdebug build
10-internal+0-2017-11-07-0256309.chris.plummer.jdk10-hs)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (fastdebug
10-internal+0-2017-11-07-0256309.chris.plummer.jdk10-hs, mixed mode,
tiered, compressed oops, g1 gc, windows-amd64)
# Problematic frame:
# v  ~StubRoutines::get_previous_fp
#

I'm starting to suspect that os::current_frame() normally is never even
called on Windows (there are calls to it in the VmError code, but I
suspect they are never triggered on Windows) and that
StubRoutines::get_previous_fp() is not working on Windows, but I could
be wrong.

Let me know what you think.

thanks,

Chris

Reply | Threaded
Open this post in threaded view
|

Re: alternate pns() debug function

Ioi Lam


On 11/6/17 9:05 PM, Chris Plummer wrote:

> Hi,
>
> I'd like to add an alternate version of pns() to debug.cpp, and would
> like some initial comments first before filing a bug and sending out
> an RFR. pns() is used to print out the native stack:
>
> extern "C" void pns(void* sp, void* fp, void* pc) { // print native stack
>   Command c("pns");
>   static char buf[O_BUFLEN];
>   Thread* t = Thread::current_or_null();
>   // Call generic frame constructor (certain arguments may be ignored)
>   frame fr(sp, fp, pc);
>   VMError::print_native_stack(tty, fr, t, buf, sizeof(buf));
> }
>
> And here's the help output for it:
>
>   pns(void* sp, void* fp, void* pc)  - print native (i.e. mixed) stack
> trace. E.g.");
>                    pns($sp, $rbp, $pc) on Linux/amd64 and
> Solaris/amd64 or");
>                    pns($sp, $ebp, $pc) on Linux/x86 or");
>                    pns($sp, 0, $pc)    on Linux/ppc64 or");
>                    pns($sp + 0x7ff, 0, $pc) on Solaris/SPARC");
>                  - in gdb do 'set overload-resolution off' before
> calling pns()");
>                  - in dbx do 'frame 1' before calling pns()");
>
> The intent is to call it from gdb, but it is potentially useful to
> call it from within hotspot when debugging. I'd like to propose the
> following alternative, since it does away with needing to pass in
> register values (not easy to do from C):
>
> extern "C" void pns2() { // print native stack
>   Command c("pns2");
>   static char buf[O_BUFLEN];
>   Thread* t = Thread::current_or_null();
>   frame fr = os::current_frame();
>   VMError::print_native_stack(tty, fr, t, buf, sizeof(buf));
> }
>
> I actually used this quite a bit with some recent debugging. There
> were places in hotspot where I wanted to know what the native stack
> looked like when called, and it was a lot easier to just add a call to
> pns2() than to setup the test to run in gdb. This seems to work on
> Linux/x64, macosx/64, and solaris/sparc. For some reason it crashes on
> Windows/x64, but I don't see any indication from the above help for
> pns() that it works on Windows/x64 anyway.
>
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x0000003bcfae1466,
> pid=9872, tid=12212
> #
> # JRE version: Java(TM) SE Runtime Environment (10.0) (fastdebug build
> 10-internal+0-2017-11-07-0256309.chris.plummer.jdk10-hs)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (fastdebug
> 10-internal+0-2017-11-07-0256309.chris.plummer.jdk10-hs, mixed mode,
> tiered, compressed oops, g1 gc, windows-amd64)
> # Problematic frame:
> # v  ~StubRoutines::get_previous_fp
> #
>
> I'm starting to suspect that os::current_frame() normally is never
> even called on Windows (there are calls to it in the VmError code, but
> I suspect they are never triggered on Windows) and that
> StubRoutines::get_previous_fp() is not working on Windows, but I could
> be wrong.
>
Hi Chris,

I like this new pns2() function.

To make this work on Windows, you probably need to follow this pattern
in vmError.cpp.


      if (os::platform_print_native_stack(st, _context, buf, sizeof(buf))) {
        // We have printed the native stack in platform-specific code
        // Windows/x64 needs special handling.
      } else {
        frame fr = _context ? os::fetch_frame_from_context(_context)
                            : os::current_frame();

        print_native_stack(st, fr, _thread, buf, sizeof(buf));
      }

Thanks
- Ioi

> Let me know what you think.
>
> thanks,
>
> Chris
>

Reply | Threaded
Open this post in threaded view
|

Re: alternate pns() debug function

Chris Plummer
On 11/6/17 11:20 PM, Ioi Lam wrote:

>
>
> On 11/6/17 9:05 PM, Chris Plummer wrote:
>> Hi,
>>
>> I'd like to add an alternate version of pns() to debug.cpp, and would
>> like some initial comments first before filing a bug and sending out
>> an RFR. pns() is used to print out the native stack:
>>
>> extern "C" void pns(void* sp, void* fp, void* pc) { // print native
>> stack
>>   Command c("pns");
>>   static char buf[O_BUFLEN];
>>   Thread* t = Thread::current_or_null();
>>   // Call generic frame constructor (certain arguments may be ignored)
>>   frame fr(sp, fp, pc);
>>   VMError::print_native_stack(tty, fr, t, buf, sizeof(buf));
>> }
>>
>> And here's the help output for it:
>>
>>   pns(void* sp, void* fp, void* pc)  - print native (i.e. mixed)
>> stack trace. E.g.");
>>                    pns($sp, $rbp, $pc) on Linux/amd64 and
>> Solaris/amd64 or");
>>                    pns($sp, $ebp, $pc) on Linux/x86 or");
>>                    pns($sp, 0, $pc)    on Linux/ppc64 or");
>>                    pns($sp + 0x7ff, 0, $pc) on Solaris/SPARC");
>>                  - in gdb do 'set overload-resolution off' before
>> calling pns()");
>>                  - in dbx do 'frame 1' before calling pns()");
>>
>> The intent is to call it from gdb, but it is potentially useful to
>> call it from within hotspot when debugging. I'd like to propose the
>> following alternative, since it does away with needing to pass in
>> register values (not easy to do from C):
>>
>> extern "C" void pns2() { // print native stack
>>   Command c("pns2");
>>   static char buf[O_BUFLEN];
>>   Thread* t = Thread::current_or_null();
>>   frame fr = os::current_frame();
>>   VMError::print_native_stack(tty, fr, t, buf, sizeof(buf));
>> }
>>
>> I actually used this quite a bit with some recent debugging. There
>> were places in hotspot where I wanted to know what the native stack
>> looked like when called, and it was a lot easier to just add a call
>> to pns2() than to setup the test to run in gdb. This seems to work on
>> Linux/x64, macosx/64, and solaris/sparc. For some reason it crashes
>> on Windows/x64, but I don't see any indication from the above help
>> for pns() that it works on Windows/x64 anyway.
>>
>> #
>> # A fatal error has been detected by the Java Runtime Environment:
>> #
>> #  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x0000003bcfae1466,
>> pid=9872, tid=12212
>> #
>> # JRE version: Java(TM) SE Runtime Environment (10.0) (fastdebug
>> build 10-internal+0-2017-11-07-0256309.chris.plummer.jdk10-hs)
>> # Java VM: Java HotSpot(TM) 64-Bit Server VM (fastdebug
>> 10-internal+0-2017-11-07-0256309.chris.plummer.jdk10-hs, mixed mode,
>> tiered, compressed oops, g1 gc, windows-amd64)
>> # Problematic frame:
>> # v  ~StubRoutines::get_previous_fp
>> #
>>
>> I'm starting to suspect that os::current_frame() normally is never
>> even called on Windows (there are calls to it in the VmError code,
>> but I suspect they are never triggered on Windows) and that
>> StubRoutines::get_previous_fp() is not working on Windows, but I
>> could be wrong.
>>
> Hi Chris,
>
> I like this new pns2() function.
>
> To make this work on Windows, you probably need to follow this pattern
> in vmError.cpp.
>
>
>      if (os::platform_print_native_stack(st, _context, buf,
> sizeof(buf))) {
>        // We have printed the native stack in platform-specific code
>        // Windows/x64 needs special handling.
>      } else {
>        frame fr = _context ? os::fetch_frame_from_context(_context)
>                            : os::current_frame();
>
>        print_native_stack(st, fr, _thread, buf, sizeof(buf));
>      }
Thanks for the tip. I'll give that a try. Looks like it's needed for AIX
also.

Chris

>
> Thanks
> - Ioi
>
>> Let me know what you think.
>>
>> thanks,
>>
>> Chris
>>
>


Reply | Threaded
Open this post in threaded view
|

Re: alternate pns() debug function

Thomas Stüfe-2
In reply to this post by Ioi Lam
Hi Chris, Ioi,

On Tue, Nov 7, 2017 at 8:20 AM, Ioi Lam <[hidden email]> wrote:

>
>
> On 11/6/17 9:05 PM, Chris Plummer wrote:
>
>> Hi,
>>
>> I'd like to add an alternate version of pns() to debug.cpp, and would
>> like some initial comments first before filing a bug and sending out an
>> RFR. pns() is used to print out the native stack:
>>
>> extern "C" void pns(void* sp, void* fp, void* pc) { // print native stack
>>   Command c("pns");
>>   static char buf[O_BUFLEN];
>>   Thread* t = Thread::current_or_null();
>>   // Call generic frame constructor (certain arguments may be ignored)
>>   frame fr(sp, fp, pc);
>>   VMError::print_native_stack(tty, fr, t, buf, sizeof(buf));
>> }
>>
>> And here's the help output for it:
>>
>>   pns(void* sp, void* fp, void* pc)  - print native (i.e. mixed) stack
>> trace. E.g.");
>>                    pns($sp, $rbp, $pc) on Linux/amd64 and Solaris/amd64
>> or");
>>                    pns($sp, $ebp, $pc) on Linux/x86 or");
>>                    pns($sp, 0, $pc)    on Linux/ppc64 or");
>>                    pns($sp + 0x7ff, 0, $pc) on Solaris/SPARC");
>>                  - in gdb do 'set overload-resolution off' before calling
>> pns()");
>>                  - in dbx do 'frame 1' before calling pns()");
>>
>> The intent is to call it from gdb, but it is potentially useful to call
>> it from within hotspot when debugging. I'd like to propose the following
>> alternative, since it does away with needing to pass in register values
>> (not easy to do from C):
>>
>> extern "C" void pns2() { // print native stack
>>   Command c("pns2");
>>   static char buf[O_BUFLEN];
>>   Thread* t = Thread::current_or_null();
>>   frame fr = os::current_frame();
>>   VMError::print_native_stack(tty, fr, t, buf, sizeof(buf));
>> }
>>
>> I actually used this quite a bit with some recent debugging. There were
>> places in hotspot where I wanted to know what the native stack looked like
>> when called, and it was a lot easier to just add a call to pns2() than to
>> setup the test to run in gdb. This seems to work on Linux/x64, macosx/64,
>> and solaris/sparc. For some reason it crashes on Windows/x64, but I don't
>> see any indication from the above help for pns() that it works on
>> Windows/x64 anyway.
>>
>> #
>> # A fatal error has been detected by the Java Runtime Environment:
>> #
>> #  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x0000003bcfae1466,
>> pid=9872, tid=12212
>> #
>> # JRE version: Java(TM) SE Runtime Environment (10.0) (fastdebug build
>> 10-internal+0-2017-11-07-0256309.chris.plummer.jdk10-hs)
>> # Java VM: Java HotSpot(TM) 64-Bit Server VM (fastdebug
>> 10-internal+0-2017-11-07-0256309.chris.plummer.jdk10-hs, mixed mode,
>> tiered, compressed oops, g1 gc, windows-amd64)
>> # Problematic frame:
>> # v  ~StubRoutines::get_previous_fp
>> #
>>
>> I'm starting to suspect that os::current_frame() normally is never even
>> called on Windows (there are calls to it in the VmError code, but I suspect
>> they are never triggered on Windows) and that StubRoutines::get_previous_fp()
>> is not working on Windows, but I could be wrong.
>>
>> Hi Chris,
>
> I like this new pns2() function.
>
> To make this work on Windows, you probably need to follow this pattern in
> vmError.cpp.
>
>
>      if (os::platform_print_native_stack(st, _context, buf, sizeof(buf)))
> {
>        // We have printed the native stack in platform-specific code
>        // Windows/x64 needs special handling.
>      } else {
>        frame fr = _context ? os::fetch_frame_from_context(_context)
>                            : os::current_frame();
>
>        print_native_stack(st, fr, _thread, buf, sizeof(buf));
>      }
>
> Thanks
> - Ioi
>
>
Note that os::platform_print_native_stack() on Windows and AIX only print
native C/C++ frames. So, calling this from the debugger may not be that
useful. On AIX, it probably will not even be possible to invoke a function
in the debugger - never tried, but given that not much else works in that
thing, I am pessimistic.

But pns() may still be useful as a tracing option, though.

On Windows x64, we do not have frame pointers, so we cannot walk the frame
VM-Style, so os::platform_print_native_stack() is the only way to print a
stack. On AIX, both ways work - VMError::print_native_stack() and
os::platform_print_native_stack() - the former gives a mixed stack, the
latter the pure C/C++ stack, but with many more information.

In the end, for consistency, I'd suggest the same as Ioi, to follow the
pattern in VMError(). And we could probably factor that out to an own
function?

Kind Regards, Thomas



>
> Let me know what you think.
>>
>> thanks,
>>
>> Chris
>>
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: alternate pns() debug function

Thomas Stüfe-2
Hi Chris,


On Wed 8. Nov 2017 at 19:46, Chris Plummer <[hidden email]> wrote:

> Hi Thomas,
>
>
> On 11/8/17 7:33 AM, Thomas Stüfe wrote:
>
> Hi Chris, Ioi,
>
> On Tue, Nov 7, 2017 at 8:20 AM, Ioi Lam <[hidden email]> wrote:
>
>>
>>
>> On 11/6/17 9:05 PM, Chris Plummer wrote:
>>
>>> Hi,
>>>
>>> I'd like to add an alternate version of pns() to debug.cpp, and would
>>> like some initial comments first before filing a bug and sending out an
>>> RFR. pns() is used to print out the native stack:
>>>
>>> extern "C" void pns(void* sp, void* fp, void* pc) { // print native stack
>>>   Command c("pns");
>>>   static char buf[O_BUFLEN];
>>>   Thread* t = Thread::current_or_null();
>>>   // Call generic frame constructor (certain arguments may be ignored)
>>>   frame fr(sp, fp, pc);
>>>   VMError::print_native_stack(tty, fr, t, buf, sizeof(buf));
>>> }
>>>
>>> And here's the help output for it:
>>>
>>>   pns(void* sp, void* fp, void* pc)  - print native (i.e. mixed) stack
>>> trace. E.g.");
>>>                    pns($sp, $rbp, $pc) on Linux/amd64 and Solaris/amd64
>>> or");
>>>                    pns($sp, $ebp, $pc) on Linux/x86 or");
>>>                    pns($sp, 0, $pc)    on Linux/ppc64 or");
>>>                    pns($sp + 0x7ff, 0, $pc) on Solaris/SPARC");
>>>                  - in gdb do 'set overload-resolution off' before
>>> calling pns()");
>>>                  - in dbx do 'frame 1' before calling pns()");
>>>
>>> The intent is to call it from gdb, but it is potentially useful to call
>>> it from within hotspot when debugging. I'd like to propose the following
>>> alternative, since it does away with needing to pass in register values
>>> (not easy to do from C):
>>>
>>> extern "C" void pns2() { // print native stack
>>>   Command c("pns2");
>>>   static char buf[O_BUFLEN];
>>>   Thread* t = Thread::current_or_null();
>>>   frame fr = os::current_frame();
>>>   VMError::print_native_stack(tty, fr, t, buf, sizeof(buf));
>>> }
>>>
>>> I actually used this quite a bit with some recent debugging. There were
>>> places in hotspot where I wanted to know what the native stack looked like
>>> when called, and it was a lot easier to just add a call to pns2() than to
>>> setup the test to run in gdb. This seems to work on Linux/x64, macosx/64,
>>> and solaris/sparc. For some reason it crashes on Windows/x64, but I don't
>>> see any indication from the above help for pns() that it works on
>>> Windows/x64 anyway.
>>>
>>> #
>>> # A fatal error has been detected by the Java Runtime Environment:
>>> #
>>> #  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x0000003bcfae1466,
>>> pid=9872, tid=12212
>>> #
>>> # JRE version: Java(TM) SE Runtime Environment (10.0) (fastdebug build
>>> 10-internal+0-2017-11-07-0256309.chris.plummer.jdk10-hs)
>>> # Java VM: Java HotSpot(TM) 64-Bit Server VM (fastdebug
>>> 10-internal+0-2017-11-07-0256309.chris.plummer.jdk10-hs, mixed mode,
>>> tiered, compressed oops, g1 gc, windows-amd64)
>>> # Problematic frame:
>>> # v  ~StubRoutines::get_previous_fp
>>> #
>>>
>>> I'm starting to suspect that os::current_frame() normally is never even
>>> called on Windows (there are calls to it in the VmError code, but I suspect
>>> they are never triggered on Windows) and that
>>> StubRoutines::get_previous_fp() is not working on Windows, but I could be
>>> wrong.
>>>
>>> Hi Chris,
>>
>> I like this new pns2() function.
>>
>> To make this work on Windows, you probably need to follow this pattern in
>> vmError.cpp.
>>
>>
>>      if (os::platform_print_native_stack(st, _context, buf, sizeof(buf)))
>> {
>>        // We have printed the native stack in platform-specific code
>>        // Windows/x64 needs special handling.
>>      } else {
>>        frame fr = _context ? os::fetch_frame_from_context(_context)
>>                            : os::current_frame();
>>
>>        print_native_stack(st, fr, _thread, buf, sizeof(buf));
>>      }
>>
>> Thanks
>> - Ioi
>>
>>
> Note that os::platform_print_native_stack() on Windows and AIX only print
> native C/C++ frames. So, calling this from the debugger may not be that
> useful.
>
> With the recent debug session I had, it was useful even with just the VM
> part of the backtrace.
>

Interesting. What did you see what the debugger would not show you?


> On AIX, it probably will not even be possible to invoke a function in the
> debugger - never tried, but given that not much else works in that thing, I
> am pessimistic.
>
> I was also pessimistic with gdb, so I tried it yesterday, and it didn't
> work. It's not in the proper context for os::current_frame() to provide the
> right information. So this will only be useful when inserted in hotspot
> code for debugging purposes.
>
>
> But pns() may still be useful as a tracing option, though.
>
> Not sure if you are talking about the original or my version. Both have
> their use: pns() from the debugger and pns2() from within hotspot code.
>
>
> On Windows x64, we do not have frame pointers, so we cannot walk the frame
> VM-Style, so os::platform_print_native_stack() is the only way to print a
> stack.
>
> Yes, and I got this to work Windows/x64, but unfortunately no symbols are
> printed.
>

When you let the Vm crash with e.g. -XX:ErrorHandlerTest=14, does the
resulting hs-err file show a correct callstack with symbols? If not, what
does the dbghelp.dll loader say (the one line below the loaded libraries
section)?


> On AIX, both ways work - VMError::print_native_stack() and
> os::platform_print_native_stack() - the former gives a mixed stack, the
> latter the pure C/C++ stack, but with many more information.
>
> In the end, for consistency, I'd suggest the same as Ioi, to follow the
> pattern in VMError().
>
> I've made that change already.
>
> And we could probably factor that out to an own function?
>
> I see your point, but given the simplicity of the change (the addition of
> one function with no disruption to existing code), I'm inclined to choose
> simplicity over elegance.
>
>
Sure! But having the generic os::current_frame function assert on windows
x64 is a bit of a trap. Maybe we should fix this at some point.

thanks,
>
> Chris
>
>
Best regards, Thomas

>
> Kind Regards, Thomas
>
>
>
>>
>> Let me know what you think.
>>>
>>> thanks,
>>>
>>> Chris
>>>
>>>
>>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: alternate pns() debug function

Thomas Stüfe-2
Hi Chris,

I went accidentally off-list with my last response yesterday. I continue
on-list.

On Thu, Nov 9, 2017 at 2:27 AM, Chris Plummer <[hidden email]>
wrote:

> Hi Thomas,
>
> <snip>

>
>> Interesting. What did you see what the debugger would not show you?
>>
>> (gdb) call pns2()
>>
>> "Executing pns2"
>> Native frames: (J=compiled Java code, A=aot compiled Java code,
>> j=interpreted, Vv=VM code, C=native code)
>> C  0x00007ffff5d5e7df
>> C  0xcccccccccccccccc
>> C  0xcccccccccccccccc
>>
>> (gdb) bt
>> #0  0x00007fffd79eef5c in ?? ()
>> #1  0x00007ffff73d79a5 in SafeFetch32 (errValue=2748,
>> adr=0xabc0000000000abc) at /local/ws/jdk10/jdk10-hs.
>> clean/open/src/hotspot/share/runtime/stubRoutines.hpp:462
>> #2  test_safefetch32 () at /local/ws/jdk10/jdk10-hs.
>> clean/open/src/hotspot/share/runtime/stubRoutines.cpp:244
>> #3  StubRoutines::initialize2 () at /local/ws/jdk10/jdk10-hs.
>> clean/open/src/hotspot/share/runtime/stubRoutines.cpp:365
>> #4  0x00007ffff73d80ba in stubRoutines_init2 () at
>> /local/ws/jdk10/jdk10-hs.clean/open/src/hotspot/share/
>> runtime/stubRoutines.cpp:374
>> #5  0x00007ffff6bc2fa3 in init_globals () at /local/ws/jdk10/jdk10-hs.
>> clean/open/src/hotspot/share/runtime/init.cpp:150
>> #6  0x00007ffff7474453 in Threads::create_vm (args=args@entry=0x7ffff5d5ee60,
>> canTryAgain=canTryAgain@entry=0x7ffff5d5ed8f) at
>> /local/ws/jdk10/jdk10-hs.clean/open/src/hotspot/share/
>> runtime/thread.cpp:3616
>> #7  0x00007ffff6d19d0b in JNI_CreateJavaVM_inner (args=0x7ffff5d5ee60,
>> penv=0x7ffff5d5ee58, vm=0x7ffff5d5ee50) at /local/ws/jdk10/jdk10-hs.
>> clean/open/src/hotspot/share/prims/jni.cpp:3938
>> #8  JNI_CreateJavaVM (vm=0x7ffff5d5ee50, penv=0x7ffff5d5ee58,
>> args=0x7ffff5d5ee60) at /local/ws/jdk10/jdk10-hs.
>> clean/open/src/hotspot/share/prims/jni.cpp:4033
>> #9  0x00007ffff7dd89d1 in InitializeJVM () at /local/ws/jdk10/jdk10-hs.
>> clean/open/src/java.base/share/native/libjli/java.c:1478
>> #10 JavaMain () at /local/ws/jdk10/jdk10-hs.clean/open/src/java.base/
>> share/native/libjli/java.c:411
>> #11 0x00000034f8a07a51 in start_thread () from /lib64/libpthread.so.0
>> #12 0x00000034f86e893d in clone () from /lib64/libc.so.6
>>
>> Actually would have been more useful to compare pns2() with pns() instead
> of a gdb backtrace:
>
> (gdb) call pns($sp, $rbp, $pc)
>
> "Executing pns"
> Native frames: (J=compiled Java code, A=aot compiled Java code,
> j=interpreted, Vv=VM code, C=native code)
> V  [libjvm.so+0x1677a07]  StubRoutines::initialize2()+0x737
> V  [libjvm.so+0xe62fa3]  init_globals()+0xf3
> V  [libjvm.so+0x1714453]  Threads::create_vm(JavaVMInitArgs*, bool*)+0x2e3
> V  [libjvm.so+0xfb9d0b]  JNI_CreateJavaVM+0x9b
> C  [libjli.so+0x39d1]  JavaMain+0x81
> C  [libpthread.so.0+0x7a51]
>
> I think gdb is making use of inlining info from the .debuginfo file since
> it has a lot of extra frames.
>
>
I still do not understand - here, the debugger shows me more than both
pns() and pns2(). Especially, in this case, the real crash point, including
file/line number. Is that not what you want? So, why do you need to invoke
pns/pns2() ? Note that this is just curiosity, I still think pns/pns2() can
be valuable for dumping stacks within the hotspot.

<snip>

>
>>> On Windows x64, we do not have frame pointers, so we cannot walk the
>>> frame VM-Style, so os::platform_print_native_stack() is the only way to
>>> print a stack.
>>>
>>> Yes, and I got this to work Windows/x64, but unfortunately no symbols
>>> are printed.
>>>
>>
>> When you let the Vm crash with e.g. -XX:ErrorHandlerTest=14, does the
>> resulting hs-err file show a correct callstack with symbols? If not, what
>> does the dbghelp.dll loader say (the one line below the loaded libraries
>> section)?
>>
>> I'll need to give this a try, but I suspect it will not include symbols.
>> I recall that you need to run windows stack traces through some special
>> tool that converts addresses to symbols.
>>
>
> I do not think this is true, at least not to my knowledge. Dbghelp.dll
> must be installed on the system - it usually is - and you need the pdb
> files laying beside the binaries. The latter depends on how you build.
> -with-native-debug-symbols=external will place the pdb files beside the
> binaries, but =zipped (or so, I am away from my machine atm) will zip them.
> In zipped form they are unusable and they need to be unzipped before they
> can be used. At least jvm.pdb, which must be located beside jvm.dll.
>
> This might be the issue. I'm using our build+test infrastructure to do
> this testing, so I'm not in control of how it's built or deployed. I've
> downloaded the binary bundle used for the testing so I could have a look,
> and I see that it does not include the symbols. However, there is a
> separate bundle with just the symbols. I'm unsure if it's being installed
> for the testing. I'll ask around.
>
>
> I am curious about this because I recently changed the coding responsible
> for the symbol resolution on windows. I‘d like to understand why you do not
> see symbols, because in our tests everything works.
>
> The is from the hs_err file when running TestOnError.java, which uses
> -XX:ErrorHandlerTest=12:
>
> Stack: [0x00000038a0f40000,0x00000038a1040000],  sp=0x00000038a103f740,
> free space=1021k
> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native
> code)
> V  [jvm.dll+0xc3255a]
> V  [jvm.dll+0x7a6198]
> V  [jvm.dll+0x7a8e8f]
> C  [java.exe+0x3609]
> C  [java.exe+0xe83f]
> C  [java.exe+0xe9e6]
> C  [KERNEL32.DLL+0x13d2]
> C  [ntdll.dll+0x154f4]
>
> thanks,
>
> Chris
>

Yes, this may be a case of missing infos. I had a discussion with Coleen
some weeks ago offlist in the context of testing the changes for
https://bugs.openjdk.java.net/browse/JDK-8185712. I had to jtreg tests
disable tests which ran nicely here at SAP but would not yield callstacks
at Oracle. At that time I was wondering if there was something wrong with
my code but now I am quite certain you guys either remove the debug info
after building or you build with zipped debug info (which is the default on
all platforms but AIX). Or both. I'll ask on build.dev.

Thanks, Thomas
Reply | Threaded
Open this post in threaded view
|

Re: alternate pns() debug function

Thomas Stüfe-2
On Thu, Nov 9, 2017 at 7:52 AM, Chris Plummer <[hidden email]>
wrote:

> On 11/8/17 10:24 PM, Thomas Stüfe wrote:
>
> Hi Chris,
>
> I went accidentally off-list with my last response yesterday. I continue
> on-list.
>
> On Thu, Nov 9, 2017 at 2:27 AM, Chris Plummer <[hidden email]>
> wrote:
>
>> Hi Thomas,
>>
>> <snip>
>
>>
>>> Interesting. What did you see what the debugger would not show you?
>>>
>>> (gdb) call pns2()
>>>
>>> "Executing pns2"
>>> Native frames: (J=compiled Java code, A=aot compiled Java code,
>>> j=interpreted, Vv=VM code, C=native code)
>>> C  0x00007ffff5d5e7df
>>> C  0xcccccccccccccccc
>>> C  0xcccccccccccccccc
>>>
>>> (gdb) bt
>>> #0  0x00007fffd79eef5c in ?? ()
>>> #1  0x00007ffff73d79a5 in SafeFetch32 (errValue=2748,
>>> adr=0xabc0000000000abc) at /local/ws/jdk10/jdk10-hs.clean
>>> /open/src/hotspot/share/runtime/stubRoutines.hpp:462
>>> #2  test_safefetch32 () at /local/ws/jdk10/jdk10-hs.clean
>>> /open/src/hotspot/share/runtime/stubRoutines.cpp:244
>>> #3  StubRoutines::initialize2 () at /local/ws/jdk10/jdk10-hs.clean
>>> /open/src/hotspot/share/runtime/stubRoutines.cpp:365
>>> #4  0x00007ffff73d80ba in stubRoutines_init2 () at
>>> /local/ws/jdk10/jdk10-hs.clean/open/src/hotspot/share/runtim
>>> e/stubRoutines.cpp:374
>>> #5  0x00007ffff6bc2fa3 in init_globals () at
>>> /local/ws/jdk10/jdk10-hs.clean/open/src/hotspot/share/runtim
>>> e/init.cpp:150
>>> #6  0x00007ffff7474453 in Threads::create_vm (args=args@entry
>>> =0x7ffff5d5ee60, canTryAgain=canTryAgain@entry=0x7ffff5d5ed8f) at
>>> /local/ws/jdk10/jdk10-hs.clean/open/src/hotspot/share/runtim
>>> e/thread.cpp:3616
>>> #7  0x00007ffff6d19d0b in JNI_CreateJavaVM_inner (args=0x7ffff5d5ee60,
>>> penv=0x7ffff5d5ee58, vm=0x7ffff5d5ee50) at /local/ws/jdk10/jdk10-hs.clean
>>> /open/src/hotspot/share/prims/jni.cpp:3938
>>> #8  JNI_CreateJavaVM (vm=0x7ffff5d5ee50, penv=0x7ffff5d5ee58,
>>> args=0x7ffff5d5ee60) at /local/ws/jdk10/jdk10-hs.clean
>>> /open/src/hotspot/share/prims/jni.cpp:4033
>>> #9  0x00007ffff7dd89d1 in InitializeJVM () at
>>> /local/ws/jdk10/jdk10-hs.clean/open/src/java.base/share/
>>> native/libjli/java.c:1478
>>> #10 JavaMain () at /local/ws/jdk10/jdk10-hs.clean
>>> /open/src/java.base/share/native/libjli/java.c:411
>>> #11 0x00000034f8a07a51 in start_thread () from /lib64/libpthread.so.0
>>> #12 0x00000034f86e893d in clone () from /lib64/libc.so.6
>>>
>>> Actually would have been more useful to compare pns2() with pns()
>> instead of a gdb backtrace:
>>
>> (gdb) call pns($sp, $rbp, $pc)
>>
>> "Executing pns"
>> Native frames: (J=compiled Java code, A=aot compiled Java code,
>> j=interpreted, Vv=VM code, C=native code)
>> V  [libjvm.so+0x1677a07]  StubRoutines::initialize2()+0x737
>> V  [libjvm.so+0xe62fa3]  init_globals()+0xf3
>> V  [libjvm.so+0x1714453]  Threads::create_vm(JavaVMInitArgs*,
>> bool*)+0x2e3
>> V  [libjvm.so+0xfb9d0b]  JNI_CreateJavaVM+0x9b
>> C  [libjli.so+0x39d1]  JavaMain+0x81
>> C  [libpthread.so.0+0x7a51]
>>
>> I think gdb is making use of inlining info from the .debuginfo file since
>> it has a lot of extra frames.
>>
>>
> I still do not understand - here, the debugger shows me more than both
> pns() and pns2(). Especially, in this case, the real crash point, including
> file/line number. Is that not what you want? So, why do you need to invoke
> pns/pns2() ? Note that this is just curiosity, I still think pns/pns2() can
> be valuable for dumping stacks within the hotspot.
>
> Well pns2() is not even an option within the debugger. I believe the
> advantage of pns() is that it includes java frames.
>
>
> <snip>
>
>>
>>>> On Windows x64, we do not have frame pointers, so we cannot walk the
>>>> frame VM-Style, so os::platform_print_native_stack() is the only way
>>>> to print a stack.
>>>>
>>>> Yes, and I got this to work Windows/x64, but unfortunately no symbols
>>>> are printed.
>>>>
>>>
>>> When you let the Vm crash with e.g. -XX:ErrorHandlerTest=14, does the
>>> resulting hs-err file show a correct callstack with symbols? If not, what
>>> does the dbghelp.dll loader say (the one line below the loaded libraries
>>> section)?
>>>
>>> I'll need to give this a try, but I suspect it will not include symbols.
>>> I recall that you need to run windows stack traces through some special
>>> tool that converts addresses to symbols.
>>>
>>
>> I do not think this is true, at least not to my knowledge. Dbghelp.dll
>> must be installed on the system - it usually is - and you need the pdb
>> files laying beside the binaries. The latter depends on how you build.
>> -with-native-debug-symbols=external will place the pdb files beside the
>> binaries, but =zipped (or so, I am away from my machine atm) will zip them.
>> In zipped form they are unusable and they need to be unzipped before they
>> can be used. At least jvm.pdb, which must be located beside jvm.dll.
>>
>> This might be the issue. I'm using our build+test infrastructure to do
>> this testing, so I'm not in control of how it's built or deployed. I've
>> downloaded the binary bundle used for the testing so I could have a look,
>> and I see that it does not include the symbols. However, there is a
>> separate bundle with just the symbols. I'm unsure if it's being installed
>> for the testing. I'll ask around.
>>
>
>> I am curious about this because I recently changed the coding responsible
>> for the symbol resolution on windows. I‘d like to understand why you do not
>> see symbols, because in our tests everything works.
>>
>> The is from the hs_err file when running TestOnError.java, which uses
>> -XX:ErrorHandlerTest=12:
>>
>> Stack: [0x00000038a0f40000,0x00000038a1040000],  sp=0x00000038a103f740,
>> free space=1021k
>> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native
>> code)
>> V  [jvm.dll+0xc3255a]
>> V  [jvm.dll+0x7a6198]
>> V  [jvm.dll+0x7a8e8f]
>> C  [java.exe+0x3609]
>> C  [java.exe+0xe83f]
>> C  [java.exe+0xe9e6]
>> C  [KERNEL32.DLL+0x13d2]
>> C  [ntdll.dll+0x154f4]
>>
>> thanks,
>>
>> Chris
>>
>
> Yes, this may be a case of missing infos. I had a discussion with Coleen
> some weeks ago offlist in the context of testing the changes for
> https://bugs.openjdk.java.net/browse/JDK-8185712. I had to jtreg tests
> disable tests which ran nicely here at SAP but would not yield callstacks
> at Oracle. At that time I was wondering if there was something wrong with
> my code but now I am quite certain you guys either remove the debug info
> after building or you build with zipped debug info (which is the default on
> all platforms but AIX). Or both. I'll ask on build.dev.
>
> I've confirmed that it is a known issue that debug symbols are not
> included in our testing (or at least that is the case on Windows). pns2()
> is showing symbols on linux, but that might just be the result of having
> pretty much all the symbols exported, which allows lookup with dladdr().
>
> thanks,
>
> Chris
>

Thanks for confirming that!

..Thomas


>
> Thanks, Thomas
>
>
>