Quantcast

gc log analysis tools

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

gc log analysis tools

Ben Cheng-2
Hi,

I was trying to get a sense of total GC time spent in a Java app using 1.6 JDK. The gc logs (w/ and wo/ parallel gc) seem to break the ancient tool gc_analyze.pl found on the web.

Is there an update to date tool to produce the desired information for new JDKs?

Thanks in advance,
-Ben
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: gc log analysis tools

NeoJia
Ben,

What is your opion? I had one but not sure if it would work for you or not (

I have not worked on JDK for a while ...

Thanks,
Neo

On 10/25/07, Ben Cheng <[hidden email]> wrote:

> Hi,
>
> I was trying to get a sense of total GC time spent in a Java app using 1.6
> JDK. The gc logs (w/ and wo/ parallel gc) seem to break the ancient tool
> gc_analyze.pl found on the web.
>
> Is there an update to date tool to produce the desired information for new
> JDKs?
>
> Thanks in advance,
> -Ben
>


--
I would remember that if researchers were not ambitious
probably today we haven't the technology we are using!
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: gc log analysis tools

Ben Cheng-2
The reason I am interested in this information is because I have a benchmark whose score is X using 32-bit VM and 2G of heap. When I switched to 64-bit build of the same VM, the score dropped to 0.77X with the same amount of heap. If I increase the heap size to 3.5G (on a machine with 4GB of RAM), the score went to to 1.40X.

What I need is pretty basic - it sums up all the gc time from the verbose log during a benchmark run, as I want to verify if the change in gc time correlates to the change in performance. I can write a similar script to do that, but I just want to check to see if there is a working tool out there already. Apparently the verbose gc log format changed from 1.5 and beyond and the old script is not compatible.

Thanks,
-Ben

On 10/25/07, Neo Jia <[hidden email]> wrote:
Ben,

What is your opion? I had one but not sure if it would work for you or not (

I have not worked on JDK for a while ...

Thanks,
Neo

On 10/25/07, Ben Cheng <[hidden email]> wrote:
> Hi,
>
> I was trying to get a sense of total GC time spent in a Java app using 1.6
> JDK. The gc logs (w/ and wo/ parallel gc) seem to break the ancient tool
> gc_analyze.pl found on the web.
>
> Is there an update to date tool to produce the desired information for new
> JDKs?
>
> Thanks in advance,
> -Ben
>


--
I would remember that if researchers were not ambitious
probably today we haven't the technology we are using!

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: gc log analysis tools

NeoJia
Ben,

Which one should be better? lower score or higher score? I assume it
should be higher based on what you said here.

And, what is the relation between score and time? Will the execution
time longer with a low score?

And how many minor GC and full GC in each of your execution? Just grep
and wc them.

For 32-bit and 64-bit machine, at least the default heap layout is
different. Are you trying to use same amount of nursery space and
mature space.

And for 64-bit machine, the Jitter is much more better than 32-bit
machine, a lot of optimization is added.

So ...

Thanks,
Neo

On 10/25/07, Ben Cheng <[hidden email]> wrote:

> The reason I am interested in this information is because I have a benchmark
> whose score is X using 32-bit VM and 2G of heap. When I switched to 64-bit
> build of the same VM, the score dropped to 0.77X with the same amount of
> heap. If I increase the heap size to 3.5G (on a machine with 4GB of RAM),
> the score went to to 1.40X.
>
> What I need is pretty basic - it sums up all the gc time from the verbose
> log during a benchmark run, as I want to verify if the change in gc time
> correlates to the change in performance. I can write a similar script to do
> that, but I just want to check to see if there is a working tool out there
> already. Apparently the verbose gc log format changed from 1.5 and beyond
> and the old script is not compatible.
>
> Thanks,
> -Ben
>
>
> On 10/25/07, Neo Jia <[hidden email]> wrote:
> > Ben,
> >
> > What is your opion? I had one but not sure if it would work for you or not
> (
> >
> > I have not worked on JDK for a while ...
> >
> > Thanks,
> > Neo
> >
> > On 10/25/07, Ben Cheng < [hidden email]> wrote:
> > > Hi,
> > >
> > > I was trying to get a sense of total GC time spent in a Java app using
> 1.6
> > > JDK. The gc logs (w/ and wo/ parallel gc) seem to break the ancient tool
> > > gc_analyze.pl found on the web.
> > >
> > > Is there an update to date tool to produce the desired information for
> new
> > > JDKs?
> > >
> > > Thanks in advance,
> > > -Ben
> > >
> >
> >
> > --
> > I would remember that if researchers were not ambitious
> > probably today we haven't the technology we are using!
> >
>
>


--
I would remember that if researchers were not ambitious
probably today we haven't the technology we are using!
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: gc log analysis tools

Ben Cheng-2


On 10/25/07, Neo Jia <[hidden email]> wrote:
Ben,

Which one should be better? lower score or higher score? I assume it
should be higher based on what you said here.
 
Oops, I forgot to clarify that. Yes, the higher the better.

And, what is the relation between score and time? Will the execution
time longer with a low score?
 
 
The benchmark is doing fixed workload, so the score is simply calculated as work/s.

And how many minor GC and full GC in each of your execution? Just grep
and wc them.
 
 
Almost all of them are minor GCs.

For 32-bit and 64-bit machine, at least the default heap layout is
different. Are you trying to use same amount of nursery space and
mature space.
 
 
I think I am just using the default ration. I only specify -Xmx and -Xms to the same amount without using -Xmn.
 
And for 64-bit machine, the Jitter is much more better than 32-bit
machine, a lot of optimization is added.
 
I believe so, and that's why the 64-bit VM eventually provides 40% speedup with larger heap is given. I think under the same amount of heap space there are more GC activities in 64-bit VM due to larger object sizes. I will grep/wc the total gc time then.
 
Thanks,
-Ben

So ...

Thanks,
Neo

On 10/25/07, Ben Cheng <[hidden email]> wrote:

> The reason I am interested in this information is because I have a benchmark
> whose score is X using 32-bit VM and 2G of heap. When I switched to 64-bit
> build of the same VM, the score dropped to 0.77X with the same amount of
> heap. If I increase the heap size to 3.5G (on a machine with 4GB of RAM),
> the score went to to 1.40X.
>
> What I need is pretty basic - it sums up all the gc time from the verbose
> log during a benchmark run, as I want to verify if the change in gc time
> correlates to the change in performance. I can write a similar script to do
> that, but I just want to check to see if there is a working tool out there
> already. Apparently the verbose gc log format changed from 1.5 and beyond
> and the old script is not compatible.
>
> Thanks,
> -Ben
>
>
> On 10/25/07, Neo Jia <[hidden email]> wrote:
> > Ben,
> >
> > What is your opion? I had one but not sure if it would work for you or not
> (
> >
> > I have not worked on JDK for a while ...
> >
> > Thanks,
> > Neo
> >
> > On 10/25/07, Ben Cheng < [hidden email]> wrote:
> > > Hi,
> > >
> > > I was trying to get a sense of total GC time spent in a Java app using
> 1.6
> > > JDK. The gc logs (w/ and wo/ parallel gc) seem to break the ancient tool
> > > gc_analyze.pl found on the web.
> > >
> > > Is there an update to date tool to produce the desired information for
> new
> > > JDKs?
> > >
> > > Thanks in advance,
> > > -Ben
> > >
> >
> >
> > --
> > I would remember that if researchers were not ambitious
> > probably today we haven't the technology we are using!
> >
>
>


--
I would remember that if researchers were not ambitious
probably today we haven't the technology we are using!

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: gc log analysis tools

NeoJia
On 10/25/07, Ben Cheng <[hidden email]> wrote:

>
>
>
> On 10/25/07, Neo Jia <[hidden email]> wrote:
> > Ben,
> >
> > Which one should be better? lower score or higher score? I assume it
> > should be higher based on what you said here.
>
>
> Oops, I forgot to clarify that. Yes, the higher the better.
>
> > And, what is the relation between score and time? Will the execution
> > time longer with a low score?
>
>
>
> The benchmark is doing fixed workload, so the score is simply calculated as
> work/s.
>
> > And how many minor GC and full GC in each of your execution? Just grep
> > and wc them.
>
>
>
> Almost all of them are minor GCs.

You may try the same ratio for these two configurations and check if
there is any difference.

Neo

>
> > For 32-bit and 64-bit machine, at least the default heap layout is
> > different. Are you trying to use same amount of nursery space and
> > mature space.
>
>
>
> I think I am just using the default ration. I only specify -Xmx and -Xms to
> the same amount without using -Xmn.
>
> > And for 64-bit machine, the Jitter is much more better than 32-bit
> > machine, a lot of optimization is added.
>
>
> I believe so, and that's why the 64-bit VM eventually provides 40% speedup
> with larger heap is given. I think under the same amount of heap space there
> are more GC activities in 64-bit VM due to larger object sizes. I will
> grep/wc the total gc time then.
>
> Thanks,
> -Ben
>
>
> > So ...
> >
> > Thanks,
> > Neo
> >
> > On 10/25/07, Ben Cheng <[hidden email] > wrote:
> > > The reason I am interested in this information is because I have a
> benchmark
> > > whose score is X using 32-bit VM and 2G of heap. When I switched to
> 64-bit
> > > build of the same VM, the score dropped to 0.77X with the same amount of
> > > heap. If I increase the heap size to 3.5G (on a machine with 4GB of
> RAM),
> > > the score went to to 1.40X.
> > >
> > > What I need is pretty basic - it sums up all the gc time from the
> verbose
> > > log during a benchmark run, as I want to verify if the change in gc time
> > > correlates to the change in performance. I can write a similar script to
> do
> > > that, but I just want to check to see if there is a working tool out
> there
> > > already. Apparently the verbose gc log format changed from 1.5 and
> beyond
> > > and the old script is not compatible.
> > >
> > > Thanks,
> > > -Ben
> > >
> > >
> > > On 10/25/07, Neo Jia < [hidden email]> wrote:
> > > > Ben,
> > > >
> > > > What is your opion? I had one but not sure if it would work for you or
> not
> > > (
> > > >
> > > > I have not worked on JDK for a while ...
> > > >
> > > > Thanks,
> > > > Neo
> > > >
> > > > On 10/25/07, Ben Cheng < [hidden email]> wrote:
> > > > > Hi,
> > > > >
> > > > > I was trying to get a sense of total GC time spent in a Java app
> using
> > > 1.6
> > > > > JDK. The gc logs (w/ and wo/ parallel gc) seem to break the ancient
> tool
> > > > > gc_analyze.pl found on the web.
> > > > >
> > > > > Is there an update to date tool to produce the desired information
> for
> > > new
> > > > > JDKs?
> > > > >
> > > > > Thanks in advance,
> > > > > -Ben
> > > > >
> > > >
> > > >
> > > > --
> > > > I would remember that if researchers were not ambitious
> > > > probably today we haven't the technology we are using!
> > > >
> > >
> > >
> >
> >
> > --
> > I would remember that if researchers were not ambitious
> > probably today we haven't the technology we are using!
> >
>
>


--
I would remember that if researchers were not ambitious
probably today we haven't the technology we are using!
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: gc log analysis tools

Jon Masamitsu
In reply to this post by Ben Cheng-2
Ben,

It's typical to see lower performance with the 64bit VM.  A drop of 23%
seems a little high
but is in the ball park.  The drop in performance comes from the
increased memory traffic
due to the larger objects.  

Hope that helps.

Jon

PS.  I don't think the output from -verbosegc changed at all between jdk
1.4.2 and jdk 5.
If it did, I think the change was very minor.  One difference you might
see with jdk 5 is that
you could be getting a different default garbage collector.   That might
be the source of
differences in the logs you're trying to analyze.

Ben Cheng wrote:

> The reason I am interested in this information is because I have a
> benchmark whose score is X using 32-bit VM and 2G of heap. When I
> switched to 64-bit build of the same VM, the score dropped to 0.77X
> with the same amount of heap. If I increase the heap size to 3.5G (on
> a machine with 4GB of RAM), the score went to to 1.40X.
>
> What I need is pretty basic - it sums up all the gc time from the
> verbose log during a benchmark run, as I want to verify if the change
> in gc time correlates to the change in performance. I can write a
> similar script to do that, but I just want to check to see if there is
> a working tool out there already. Apparently the verbose gc log format
> changed from 1.5 and beyond and the old script is not compatible.
>
> Thanks,
> -Ben
>
> On 10/25/07, * Neo Jia* <[hidden email] <mailto:[hidden email]>>
> wrote:
>
>     Ben,
>
>     What is your opion? I had one but not sure if it would work for
>     you or not (
>
>     I have not worked on JDK for a while ...
>
>     Thanks,
>     Neo
>
>     On 10/25/07, Ben Cheng < [hidden email]
>     <mailto:[hidden email]>> wrote:
>     > Hi,
>     >
>     > I was trying to get a sense of total GC time spent in a Java app
>     using 1.6
>     > JDK. The gc logs (w/ and wo/ parallel gc) seem to break the
>     ancient tool
>     > gc_analyze.pl found on the web.
>     >
>     > Is there an update to date tool to produce the desired
>     information for new
>     > JDKs?
>     >
>     > Thanks in advance,
>     > -Ben
>     >
>
>
>     --
>     I would remember that if researchers were not ambitious
>     probably today we haven't the technology we are using!
>
>


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: gc log analysis tools

Peter B. Kessler
In reply to this post by Ben Cheng-2
Ben Cheng wrote:

> Hi,
>
> I was trying to get a sense of total GC time spent in a Java app using
> 1.6 JDK. The gc logs (w/ and wo/ parallel gc) seem to break the ancient
> tool gc_analyze.pl found on the web.
>
> Is there an update to date tool to produce the desired information for
> new JDKs?
>
> Thanks in advance,
> -Ben


I'll see if we can get an updated version of PrintGCStats out to
replace the ancient one available at

     http://java.sun.com/developer/technicalArticles/Programming/turbo/

That should give you the kinds of statistics you are asking for.

                        ... peter
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: gc log analysis tools

Tony Printezis
Hi Ben,

Additionally, we are trying to open source a graphical tool we have
developed, called GChisto, that can show you a lot of stats about your
log, as well as other interesting graphs. We have been using it to
analyze customer logs and it has been very helpful. Once we have the OK
to open source it, we'll announce it on this list.

Regards,

Tony

Peter B. Kessler wrote:

> Ben Cheng wrote:
>
>> Hi,
>>
>> I was trying to get a sense of total GC time spent in a Java app
>> using 1.6 JDK. The gc logs (w/ and wo/ parallel gc) seem to break the
>> ancient tool gc_analyze.pl found on the web.
>>
>> Is there an update to date tool to produce the desired information
>> for new JDKs?
>>
>> Thanks in advance,
>> -Ben
>
>
> I'll see if we can get an updated version of PrintGCStats out to
> replace the ancient one available at
>
>     http://java.sun.com/developer/technicalArticles/Programming/turbo/
>
> That should give you the kinds of statistics you are asking for.
>
>             ... peter

--
----------------------------------------------------------------------
| Tony Printezis, Staff Engineer    | Sun Microsystems Inc.          |
|                                   | MS BUR02-311                   |
| e-mail: [hidden email]    | 35 Network Drive               |
| office: +1 781 442 0998 (x20998)  | Burlington, MA01803-0902, USA  |
----------------------------------------------------------------------
e-mail client: Thunderbird (Solaris)


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: gc log analysis tools

Ben Cheng-2
In reply to this post by Peter B. Kessler
Thank you Peter. I just visisted the turbo... page and the PrintGCStats script there is working perfectly with the gc log I collected the other day.

-Ben

On 10/26/07, Peter B. Kessler <[hidden email]> wrote:
Ben Cheng wrote:

> Hi,
>
> I was trying to get a sense of total GC time spent in a Java app using
> 1.6 JDK. The gc logs (w/ and wo/ parallel gc) seem to break the ancient
> tool gc_analyze.pl found on the web.
>
> Is there an update to date tool to produce the desired information for
> new JDKs?
>
> Thanks in advance,
> -Ben


I'll see if we can get an updated version of PrintGCStats out to
replace the ancient one available at

     http://java.sun.com/developer/technicalArticles/Programming/turbo/

That should give you the kinds of statistics you are asking for.

                        ... peter

Loading...