Can GC implementations provide a cheap estimation of live set size?

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Can GC implementations provide a cheap estimation of live set size?

Jaroslav Bachorík
Hello,

In connection with https://bugs.openjdk.java.net/browse/JDK-8258431 I
am trying to figure out whether providing a cheap estimation of live
set size is something actually achievable across various GC
implementations.

What I am looking at is piggy-backing on a concurrent mark task to get
the summary size of live objects - using the 'straight-forward'
heap-inspection like approach is prohibitively expensive.

Thanks and regards,

-JB-
Reply | Threaded
Open this post in threaded view
|

Re: Can GC implementations provide a cheap estimation of live set size?

Roman Kennke-6
Hello Jaroslav,

> In connection with https://bugs.openjdk.java.net/browse/JDK-8258431 I
> am trying to figure out whether providing a cheap estimation of live
> set size is something actually achievable across various GC
> implementations.
>
> What I am looking at is piggy-backing on a concurrent mark task to get
> the summary size of live objects - using the 'straight-forward'
> heap-inspection like approach is prohibitively expensive.

In Shenandoah, this information is already collected during concurrent
marking. We currently don't print it directly, but we could certainly do
that. I'll look into implementing it. I'll also look into exposing
liveness info via JMX.

I'm not quite sure about G1: that information would only be collected
during mixed or full collections. I am not sure if G1 prints it, though.

ZGC prints this under -Xlog:gc+heap:

[6,502s][info][gc,heap     ] GC(0)                Mark Start
Mark End        Relocate Start      Relocate End           High
       Low
[6,502s][info][gc,heap     ] GC(0)  Capacity:      834M (10%)
1076M (13%)        1092M (14%)        1092M (14%)        1092M (14%)
      834M (10%)
[6,502s][info][gc,heap     ] GC(0)      Free:     7154M (90%)
6912M (87%)        6916M (87%)        7388M (92%)        7388M (92%)
     6896M (86%)
[6,502s][info][gc,heap     ] GC(0)      Used:      834M (10%)
1076M (13%)        1072M (13%)         600M (8%)         1092M (14%)
      600M (8%)
[6,502s][info][gc,heap     ] GC(0)      Live:         -
195M (2%)          195M (2%)          195M (2%)             -
        -
[6,502s][info][gc,heap     ] GC(0) Allocated:         -
242M (3%)          270M (3%)          380M (5%)             -
        -
[6,502s][info][gc,heap     ] GC(0)   Garbage:         -
638M (8%)          606M (8%)           24M (0%)             -
        -
[6,502s][info][gc,heap     ] GC(0) Reclaimed:         -
  -                32M (0%)          614M (8%)             -
       -

I hope that is useful?

Thanks,
Roman

Reply | Threaded
Open this post in threaded view
|

Re: Can GC implementations provide a cheap estimation of live set size?

Jaroslav Bachorík
Hi Roman,

Thanks for your response. I checked ZGC implementation and, indeed, it
is very easy to get the liveness information just by extending
`ZStatHeap` class to report the last valid value of
`_at_mark_end.live`.

I am also able to get this info from Shenandoah, although my first
attempt still involves a safepointing VM operation since I need to
iterate over regions to get the liveness info for each of them and sum
it up. I think it is still an acceptable trade-off, though.

The next one in the queue is the Serial GC. My assumptions, based on
reading the code, are that for young gen 'live = used' at the end of
DefNewGeneration::collect() method and for old gen 'live = used -
slack' (slack is the cumulative size of objects considered to be alive
for the purpose of compaction although they are really dead - see
CompactibleSpace::scan_and_forward()). Does this sound reasonable?

I will post my findings for Parallel GC and G1 GC later.

Cheers,

-JB-

On Wed, Feb 10, 2021 at 11:34 AM Roman Kennke <[hidden email]> wrote:

>
> Hello Jaroslav,
>
> > In connection with https://bugs.openjdk.java.net/browse/JDK-8258431 I
> > am trying to figure out whether providing a cheap estimation of live
> > set size is something actually achievable across various GC
> > implementations.
> >
> > What I am looking at is piggy-backing on a concurrent mark task to get
> > the summary size of live objects - using the 'straight-forward'
> > heap-inspection like approach is prohibitively expensive.
>
> In Shenandoah, this information is already collected during concurrent
> marking. We currently don't print it directly, but we could certainly do
> that. I'll look into implementing it. I'll also look into exposing
> liveness info via JMX.
>
> I'm not quite sure about G1: that information would only be collected
> during mixed or full collections. I am not sure if G1 prints it, though.
>
> ZGC prints this under -Xlog:gc+heap:
>
> [6,502s][info][gc,heap     ] GC(0)                Mark Start
> Mark End        Relocate Start      Relocate End           High
>        Low
> [6,502s][info][gc,heap     ] GC(0)  Capacity:      834M (10%)
> 1076M (13%)        1092M (14%)        1092M (14%)        1092M (14%)
>       834M (10%)
> [6,502s][info][gc,heap     ] GC(0)      Free:     7154M (90%)
> 6912M (87%)        6916M (87%)        7388M (92%)        7388M (92%)
>      6896M (86%)
> [6,502s][info][gc,heap     ] GC(0)      Used:      834M (10%)
> 1076M (13%)        1072M (13%)         600M (8%)         1092M (14%)
>       600M (8%)
> [6,502s][info][gc,heap     ] GC(0)      Live:         -
> 195M (2%)          195M (2%)          195M (2%)             -
>         -
> [6,502s][info][gc,heap     ] GC(0) Allocated:         -
> 242M (3%)          270M (3%)          380M (5%)             -
>         -
> [6,502s][info][gc,heap     ] GC(0)   Garbage:         -
> 638M (8%)          606M (8%)           24M (0%)             -
>         -
> [6,502s][info][gc,heap     ] GC(0) Reclaimed:         -
>   -                32M (0%)          614M (8%)             -
>        -
>
> I hope that is useful?
>
> Thanks,
> Roman
>
Reply | Threaded
Open this post in threaded view
|

Re: Can GC implementations provide a cheap estimation of live set size?

Roman Kennke-6
Notice that liveness information is only somewhat reliable right after
marking. In Shenandoah, this is in the final-mark pause, and then the
program is at a safepoint already. This is where you'd want to emit a
JMX event or something similar. You can't simply query a counter and
assume it represents current liveness in the middle or outside of GC
cycle. This should be true for all GCs.

For Serial and Parallel I am not sure at all that you can do this.
AFAIK, they don't count liveness at all.

Roman

> Hi Roman,
>
> Thanks for your response. I checked ZGC implementation and, indeed, it
> is very easy to get the liveness information just by extending
> `ZStatHeap` class to report the last valid value of
> `_at_mark_end.live`.
>
> I am also able to get this info from Shenandoah, although my first
> attempt still involves a safepointing VM operation since I need to
> iterate over regions to get the liveness info for each of them and sum
> it up. I think it is still an acceptable trade-off, though.
>
> The next one in the queue is the Serial GC. My assumptions, based on
> reading the code, are that for young gen 'live = used' at the end of
> DefNewGeneration::collect() method and for old gen 'live = used -
> slack' (slack is the cumulative size of objects considered to be alive
> for the purpose of compaction although they are really dead - see
> CompactibleSpace::scan_and_forward()). Does this sound reasonable?
>
> I will post my findings for Parallel GC and G1 GC later.
>
> Cheers,
>
> -JB-
>
> On Wed, Feb 10, 2021 at 11:34 AM Roman Kennke <[hidden email]> wrote:
>>
>> Hello Jaroslav,
>>
>>> In connection with https://bugs.openjdk.java.net/browse/JDK-8258431 I
>>> am trying to figure out whether providing a cheap estimation of live
>>> set size is something actually achievable across various GC
>>> implementations.
>>>
>>> What I am looking at is piggy-backing on a concurrent mark task to get
>>> the summary size of live objects - using the 'straight-forward'
>>> heap-inspection like approach is prohibitively expensive.
>>
>> In Shenandoah, this information is already collected during concurrent
>> marking. We currently don't print it directly, but we could certainly do
>> that. I'll look into implementing it. I'll also look into exposing
>> liveness info via JMX.
>>
>> I'm not quite sure about G1: that information would only be collected
>> during mixed or full collections. I am not sure if G1 prints it, though.
>>
>> ZGC prints this under -Xlog:gc+heap:
>>
>> [6,502s][info][gc,heap     ] GC(0)                Mark Start
>> Mark End        Relocate Start      Relocate End           High
>>         Low
>> [6,502s][info][gc,heap     ] GC(0)  Capacity:      834M (10%)
>> 1076M (13%)        1092M (14%)        1092M (14%)        1092M (14%)
>>        834M (10%)
>> [6,502s][info][gc,heap     ] GC(0)      Free:     7154M (90%)
>> 6912M (87%)        6916M (87%)        7388M (92%)        7388M (92%)
>>       6896M (86%)
>> [6,502s][info][gc,heap     ] GC(0)      Used:      834M (10%)
>> 1076M (13%)        1072M (13%)         600M (8%)         1092M (14%)
>>        600M (8%)
>> [6,502s][info][gc,heap     ] GC(0)      Live:         -
>> 195M (2%)          195M (2%)          195M (2%)             -
>>          -
>> [6,502s][info][gc,heap     ] GC(0) Allocated:         -
>> 242M (3%)          270M (3%)          380M (5%)             -
>>          -
>> [6,502s][info][gc,heap     ] GC(0)   Garbage:         -
>> 638M (8%)          606M (8%)           24M (0%)             -
>>          -
>> [6,502s][info][gc,heap     ] GC(0) Reclaimed:         -
>>    -                32M (0%)          614M (8%)             -
>>         -
>>
>> I hope that is useful?
>>
>> Thanks,
>> Roman
>>
>

Reply | Threaded
Open this post in threaded view
|

Re: Can GC implementations provide a cheap estimation of live set size?

Jaroslav Bachorík
On Thu, Feb 11, 2021 at 6:55 PM Roman Kennke <[hidden email]> wrote:
>
> Notice that liveness information is only somewhat reliable right after
> marking. In Shenandoah, this is in the final-mark pause, and then the

Yes, I understand this. What I am looking at is to have something like
'last known liveness' value - captured at a well defined point and
providing an estimate within the bounds of GC implementation.

> program is at a safepoint already. This is where you'd want to emit a
> JMX event or something similar. You can't simply query a counter and
> assume it represents current liveness in the middle or outside of GC
> cycle. This should be true for all GCs.
>
> For Serial and Parallel I am not sure at all that you can do this.
> AFAIK, they don't count liveness at all.
>
> Roman
>
> > Hi Roman,
> >
> > Thanks for your response. I checked ZGC implementation and, indeed, it
> > is very easy to get the liveness information just by extending
> > `ZStatHeap` class to report the last valid value of
> > `_at_mark_end.live`.
> >
> > I am also able to get this info from Shenandoah, although my first
> > attempt still involves a safepointing VM operation since I need to
> > iterate over regions to get the liveness info for each of them and sum
> > it up. I think it is still an acceptable trade-off, though.
> >
> > The next one in the queue is the Serial GC. My assumptions, based on
> > reading the code, are that for young gen 'live = used' at the end of
> > DefNewGeneration::collect() method and for old gen 'live = used -
> > slack' (slack is the cumulative size of objects considered to be alive
> > for the purpose of compaction although they are really dead - see
> > CompactibleSpace::scan_and_forward()). Does this sound reasonable?
> >
> > I will post my findings for Parallel GC and G1 GC later.
> >
> > Cheers,
> >
> > -JB-
> >
> > On Wed, Feb 10, 2021 at 11:34 AM Roman Kennke <[hidden email]> wrote:
> >>
> >> Hello Jaroslav,
> >>
> >>> In connection with https://bugs.openjdk.java.net/browse/JDK-8258431 I
> >>> am trying to figure out whether providing a cheap estimation of live
> >>> set size is something actually achievable across various GC
> >>> implementations.
> >>>
> >>> What I am looking at is piggy-backing on a concurrent mark task to get
> >>> the summary size of live objects - using the 'straight-forward'
> >>> heap-inspection like approach is prohibitively expensive.
> >>
> >> In Shenandoah, this information is already collected during concurrent
> >> marking. We currently don't print it directly, but we could certainly do
> >> that. I'll look into implementing it. I'll also look into exposing
> >> liveness info via JMX.
> >>
> >> I'm not quite sure about G1: that information would only be collected
> >> during mixed or full collections. I am not sure if G1 prints it, though.
> >>
> >> ZGC prints this under -Xlog:gc+heap:
> >>
> >> [6,502s][info][gc,heap     ] GC(0)                Mark Start
> >> Mark End        Relocate Start      Relocate End           High
> >>         Low
> >> [6,502s][info][gc,heap     ] GC(0)  Capacity:      834M (10%)
> >> 1076M (13%)        1092M (14%)        1092M (14%)        1092M (14%)
> >>        834M (10%)
> >> [6,502s][info][gc,heap     ] GC(0)      Free:     7154M (90%)
> >> 6912M (87%)        6916M (87%)        7388M (92%)        7388M (92%)
> >>       6896M (86%)
> >> [6,502s][info][gc,heap     ] GC(0)      Used:      834M (10%)
> >> 1076M (13%)        1072M (13%)         600M (8%)         1092M (14%)
> >>        600M (8%)
> >> [6,502s][info][gc,heap     ] GC(0)      Live:         -
> >> 195M (2%)          195M (2%)          195M (2%)             -
> >>          -
> >> [6,502s][info][gc,heap     ] GC(0) Allocated:         -
> >> 242M (3%)          270M (3%)          380M (5%)             -
> >>          -
> >> [6,502s][info][gc,heap     ] GC(0)   Garbage:         -
> >> 638M (8%)          606M (8%)           24M (0%)             -
> >>          -
> >> [6,502s][info][gc,heap     ] GC(0) Reclaimed:         -
> >>    -                32M (0%)          614M (8%)             -
> >>         -
> >>
> >> I hope that is useful?
> >>
> >> Thanks,
> >> Roman
> >>
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Can GC implementations provide a cheap estimation of live set size?

Jaroslav Bachorík
Hi again,

I continued experimenting with Shenandoah and ZGC which already are
tracking liveness. I am emitting a (partially filled) GCHeapSummary
JFR event to capture used/live sizes.
For Shenandoah the event is emitted at the very end of the
`ShenandoahConcurrentGC::op_final_mark()` method and for ZGC it is the
`ZMark::end()` method. The exact changes can be checked via branch
comparison (https://github.com/openjdk/jdk/compare/master...DataDog:jb/live_set_1)
but bear in mind that this is just an experimental code with no
intention being checked in in its current form.

Unfortunately, when I run an application on such modified JVM and
collect a JFR recording the live set size numbers seem a bit 'low' -
eg. on both ZGC and Shenandoah (using an already available liveness
info) the reported liveness is ~50% of the reported usage. Is there a
good explanation for this?

Thanks!

-JB-

On Thu, Feb 11, 2021 at 7:09 PM Jaroslav Bachorík
<[hidden email]> wrote:

>
> On Thu, Feb 11, 2021 at 6:55 PM Roman Kennke <[hidden email]> wrote:
> >
> > Notice that liveness information is only somewhat reliable right after
> > marking. In Shenandoah, this is in the final-mark pause, and then the
>
> Yes, I understand this. What I am looking at is to have something like
> 'last known liveness' value - captured at a well defined point and
> providing an estimate within the bounds of GC implementation.
>
> > program is at a safepoint already. This is where you'd want to emit a
> > JMX event or something similar. You can't simply query a counter and
> > assume it represents current liveness in the middle or outside of GC
> > cycle. This should be true for all GCs.
> >
> > For Serial and Parallel I am not sure at all that you can do this.
> > AFAIK, they don't count liveness at all.
> >
> > Roman
> >
> > > Hi Roman,
> > >
> > > Thanks for your response. I checked ZGC implementation and, indeed, it
> > > is very easy to get the liveness information just by extending
> > > `ZStatHeap` class to report the last valid value of
> > > `_at_mark_end.live`.
> > >
> > > I am also able to get this info from Shenandoah, although my first
> > > attempt still involves a safepointing VM operation since I need to
> > > iterate over regions to get the liveness info for each of them and sum
> > > it up. I think it is still an acceptable trade-off, though.
> > >
> > > The next one in the queue is the Serial GC. My assumptions, based on
> > > reading the code, are that for young gen 'live = used' at the end of
> > > DefNewGeneration::collect() method and for old gen 'live = used -
> > > slack' (slack is the cumulative size of objects considered to be alive
> > > for the purpose of compaction although they are really dead - see
> > > CompactibleSpace::scan_and_forward()). Does this sound reasonable?
> > >
> > > I will post my findings for Parallel GC and G1 GC later.
> > >
> > > Cheers,
> > >
> > > -JB-
> > >
> > > On Wed, Feb 10, 2021 at 11:34 AM Roman Kennke <[hidden email]> wrote:
> > >>
> > >> Hello Jaroslav,
> > >>
> > >>> In connection with https://bugs.openjdk.java.net/browse/JDK-8258431 I
> > >>> am trying to figure out whether providing a cheap estimation of live
> > >>> set size is something actually achievable across various GC
> > >>> implementations.
> > >>>
> > >>> What I am looking at is piggy-backing on a concurrent mark task to get
> > >>> the summary size of live objects - using the 'straight-forward'
> > >>> heap-inspection like approach is prohibitively expensive.
> > >>
> > >> In Shenandoah, this information is already collected during concurrent
> > >> marking. We currently don't print it directly, but we could certainly do
> > >> that. I'll look into implementing it. I'll also look into exposing
> > >> liveness info via JMX.
> > >>
> > >> I'm not quite sure about G1: that information would only be collected
> > >> during mixed or full collections. I am not sure if G1 prints it, though.
> > >>
> > >> ZGC prints this under -Xlog:gc+heap:
> > >>
> > >> [6,502s][info][gc,heap     ] GC(0)                Mark Start
> > >> Mark End        Relocate Start      Relocate End           High
> > >>         Low
> > >> [6,502s][info][gc,heap     ] GC(0)  Capacity:      834M (10%)
> > >> 1076M (13%)        1092M (14%)        1092M (14%)        1092M (14%)
> > >>        834M (10%)
> > >> [6,502s][info][gc,heap     ] GC(0)      Free:     7154M (90%)
> > >> 6912M (87%)        6916M (87%)        7388M (92%)        7388M (92%)
> > >>       6896M (86%)
> > >> [6,502s][info][gc,heap     ] GC(0)      Used:      834M (10%)
> > >> 1076M (13%)        1072M (13%)         600M (8%)         1092M (14%)
> > >>        600M (8%)
> > >> [6,502s][info][gc,heap     ] GC(0)      Live:         -
> > >> 195M (2%)          195M (2%)          195M (2%)             -
> > >>          -
> > >> [6,502s][info][gc,heap     ] GC(0) Allocated:         -
> > >> 242M (3%)          270M (3%)          380M (5%)             -
> > >>          -
> > >> [6,502s][info][gc,heap     ] GC(0)   Garbage:         -
> > >> 638M (8%)          606M (8%)           24M (0%)             -
> > >>          -
> > >> [6,502s][info][gc,heap     ] GC(0) Reclaimed:         -
> > >>    -                32M (0%)          614M (8%)             -
> > >>         -
> > >>
> > >> I hope that is useful?
> > >>
> > >> Thanks,
> > >> Roman
> > >>
> > >
> >
Reply | Threaded
Open this post in threaded view
|

Re: Can GC implementations provide a cheap estimation of live set size?

Per Liden
Hi,

On 2/15/21 10:44 AM, Jaroslav Bachorík wrote:

> Hi again,
>
> I continued experimenting with Shenandoah and ZGC which already are
> tracking liveness. I am emitting a (partially filled) GCHeapSummary
> JFR event to capture used/live sizes.
> For Shenandoah the event is emitted at the very end of the
> `ShenandoahConcurrentGC::op_final_mark()` method and for ZGC it is the
> `ZMark::end()` method. The exact changes can be checked via branch
> comparison (https://github.com/openjdk/jdk/compare/master...DataDog:jb/live_set_1)
> but bear in mind that this is just an experimental code with no
> intention being checked in in its current form.
>
> Unfortunately, when I run an application on such modified JVM and
> collect a JFR recording the live set size numbers seem a bit 'low' -
> eg. on both ZGC and Shenandoah (using an already available liveness
> info) the reported liveness is ~50% of the reported usage. Is there a
> good explanation for this?

When you create the GCHeapSummary, the "live" value reflects what was
live after marking, while the "used" value reflects the usage when the
GC cycle ended. So, after marking ended, some amount of garbage was
likely reclaimed, but then new objects were also allocated. For ZGC
(don't know if Shenandoah shows this), you can see details of how much
was reclaimed and how much was allocated in the GC log.

/Per

>
> Thanks!
>
> -JB-
>
> On Thu, Feb 11, 2021 at 7:09 PM Jaroslav Bachorík
> <[hidden email]> wrote:
>>
>> On Thu, Feb 11, 2021 at 6:55 PM Roman Kennke <[hidden email]> wrote:
>>>
>>> Notice that liveness information is only somewhat reliable right after
>>> marking. In Shenandoah, this is in the final-mark pause, and then the
>>
>> Yes, I understand this. What I am looking at is to have something like
>> 'last known liveness' value - captured at a well defined point and
>> providing an estimate within the bounds of GC implementation.
>>
>>> program is at a safepoint already. This is where you'd want to emit a
>>> JMX event or something similar. You can't simply query a counter and
>>> assume it represents current liveness in the middle or outside of GC
>>> cycle. This should be true for all GCs.
>>>
>>> For Serial and Parallel I am not sure at all that you can do this.
>>> AFAIK, they don't count liveness at all.
>>>
>>> Roman
>>>
>>>> Hi Roman,
>>>>
>>>> Thanks for your response. I checked ZGC implementation and, indeed, it
>>>> is very easy to get the liveness information just by extending
>>>> `ZStatHeap` class to report the last valid value of
>>>> `_at_mark_end.live`.
>>>>
>>>> I am also able to get this info from Shenandoah, although my first
>>>> attempt still involves a safepointing VM operation since I need to
>>>> iterate over regions to get the liveness info for each of them and sum
>>>> it up. I think it is still an acceptable trade-off, though.
>>>>
>>>> The next one in the queue is the Serial GC. My assumptions, based on
>>>> reading the code, are that for young gen 'live = used' at the end of
>>>> DefNewGeneration::collect() method and for old gen 'live = used -
>>>> slack' (slack is the cumulative size of objects considered to be alive
>>>> for the purpose of compaction although they are really dead - see
>>>> CompactibleSpace::scan_and_forward()). Does this sound reasonable?
>>>>
>>>> I will post my findings for Parallel GC and G1 GC later.
>>>>
>>>> Cheers,
>>>>
>>>> -JB-
>>>>
>>>> On Wed, Feb 10, 2021 at 11:34 AM Roman Kennke <[hidden email]> wrote:
>>>>>
>>>>> Hello Jaroslav,
>>>>>
>>>>>> In connection with https://bugs.openjdk.java.net/browse/JDK-8258431 I
>>>>>> am trying to figure out whether providing a cheap estimation of live
>>>>>> set size is something actually achievable across various GC
>>>>>> implementations.
>>>>>>
>>>>>> What I am looking at is piggy-backing on a concurrent mark task to get
>>>>>> the summary size of live objects - using the 'straight-forward'
>>>>>> heap-inspection like approach is prohibitively expensive.
>>>>>
>>>>> In Shenandoah, this information is already collected during concurrent
>>>>> marking. We currently don't print it directly, but we could certainly do
>>>>> that. I'll look into implementing it. I'll also look into exposing
>>>>> liveness info via JMX.
>>>>>
>>>>> I'm not quite sure about G1: that information would only be collected
>>>>> during mixed or full collections. I am not sure if G1 prints it, though.
>>>>>
>>>>> ZGC prints this under -Xlog:gc+heap:
>>>>>
>>>>> [6,502s][info][gc,heap     ] GC(0)                Mark Start
>>>>> Mark End        Relocate Start      Relocate End           High
>>>>>          Low
>>>>> [6,502s][info][gc,heap     ] GC(0)  Capacity:      834M (10%)
>>>>> 1076M (13%)        1092M (14%)        1092M (14%)        1092M (14%)
>>>>>         834M (10%)
>>>>> [6,502s][info][gc,heap     ] GC(0)      Free:     7154M (90%)
>>>>> 6912M (87%)        6916M (87%)        7388M (92%)        7388M (92%)
>>>>>        6896M (86%)
>>>>> [6,502s][info][gc,heap     ] GC(0)      Used:      834M (10%)
>>>>> 1076M (13%)        1072M (13%)         600M (8%)         1092M (14%)
>>>>>         600M (8%)
>>>>> [6,502s][info][gc,heap     ] GC(0)      Live:         -
>>>>> 195M (2%)          195M (2%)          195M (2%)             -
>>>>>           -
>>>>> [6,502s][info][gc,heap     ] GC(0) Allocated:         -
>>>>> 242M (3%)          270M (3%)          380M (5%)             -
>>>>>           -
>>>>> [6,502s][info][gc,heap     ] GC(0)   Garbage:         -
>>>>> 638M (8%)          606M (8%)           24M (0%)             -
>>>>>           -
>>>>> [6,502s][info][gc,heap     ] GC(0) Reclaimed:         -
>>>>>     -                32M (0%)          614M (8%)             -
>>>>>          -
>>>>>
>>>>> I hope that is useful?
>>>>>
>>>>> Thanks,
>>>>> Roman
>>>>>
>>>>
>>>
Reply | Threaded
Open this post in threaded view
|

Re: Can GC implementations provide a cheap estimation of live set size?

Jaroslav Bachorík
On Mon, Feb 15, 2021 at 11:24 AM Per Liden <[hidden email]> wrote:

>
> Hi,
>
> On 2/15/21 10:44 AM, Jaroslav Bachorík wrote:
> > Hi again,
> >
> > I continued experimenting with Shenandoah and ZGC which already are
> > tracking liveness. I am emitting a (partially filled) GCHeapSummary
> > JFR event to capture used/live sizes.
> > For Shenandoah the event is emitted at the very end of the
> > `ShenandoahConcurrentGC::op_final_mark()` method and for ZGC it is the
> > `ZMark::end()` method. The exact changes can be checked via branch
> > comparison (https://github.com/openjdk/jdk/compare/master...DataDog:jb/live_set_1)
> > but bear in mind that this is just an experimental code with no
> > intention being checked in in its current form.
> >
> > Unfortunately, when I run an application on such modified JVM and
> > collect a JFR recording the live set size numbers seem a bit 'low' -
> > eg. on both ZGC and Shenandoah (using an already available liveness
> > info) the reported liveness is ~50% of the reported usage. Is there a
> > good explanation for this?
>
> When you create the GCHeapSummary, the "live" value reflects what was
> live after marking, while the "used" value reflects the usage when the
> GC cycle ended. So, after marking ended, some amount of garbage was
> likely reclaimed, but then new objects were also allocated. For ZGC
> (don't know if Shenandoah shows this), you can see details of how much
> was reclaimed and how much was allocated in the GC log.

Definitely - it's just that a diff of >100MB (eg. for ZGC 350MB used
vs. 170MB live) struck me as a bit suspicious. But maybe it is
expected.

-JB-

>
> /Per
>
> >
> > Thanks!
> >
> > -JB-
> >
> > On Thu, Feb 11, 2021 at 7:09 PM Jaroslav Bachorík
> > <[hidden email]> wrote:
> >>
> >> On Thu, Feb 11, 2021 at 6:55 PM Roman Kennke <[hidden email]> wrote:
> >>>
> >>> Notice that liveness information is only somewhat reliable right after
> >>> marking. In Shenandoah, this is in the final-mark pause, and then the
> >>
> >> Yes, I understand this. What I am looking at is to have something like
> >> 'last known liveness' value - captured at a well defined point and
> >> providing an estimate within the bounds of GC implementation.
> >>
> >>> program is at a safepoint already. This is where you'd want to emit a
> >>> JMX event or something similar. You can't simply query a counter and
> >>> assume it represents current liveness in the middle or outside of GC
> >>> cycle. This should be true for all GCs.
> >>>
> >>> For Serial and Parallel I am not sure at all that you can do this.
> >>> AFAIK, they don't count liveness at all.
> >>>
> >>> Roman
> >>>
> >>>> Hi Roman,
> >>>>
> >>>> Thanks for your response. I checked ZGC implementation and, indeed, it
> >>>> is very easy to get the liveness information just by extending
> >>>> `ZStatHeap` class to report the last valid value of
> >>>> `_at_mark_end.live`.
> >>>>
> >>>> I am also able to get this info from Shenandoah, although my first
> >>>> attempt still involves a safepointing VM operation since I need to
> >>>> iterate over regions to get the liveness info for each of them and sum
> >>>> it up. I think it is still an acceptable trade-off, though.
> >>>>
> >>>> The next one in the queue is the Serial GC. My assumptions, based on
> >>>> reading the code, are that for young gen 'live = used' at the end of
> >>>> DefNewGeneration::collect() method and for old gen 'live = used -
> >>>> slack' (slack is the cumulative size of objects considered to be alive
> >>>> for the purpose of compaction although they are really dead - see
> >>>> CompactibleSpace::scan_and_forward()). Does this sound reasonable?
> >>>>
> >>>> I will post my findings for Parallel GC and G1 GC later.
> >>>>
> >>>> Cheers,
> >>>>
> >>>> -JB-
> >>>>
> >>>> On Wed, Feb 10, 2021 at 11:34 AM Roman Kennke <[hidden email]> wrote:
> >>>>>
> >>>>> Hello Jaroslav,
> >>>>>
> >>>>>> In connection with https://bugs.openjdk.java.net/browse/JDK-8258431 I
> >>>>>> am trying to figure out whether providing a cheap estimation of live
> >>>>>> set size is something actually achievable across various GC
> >>>>>> implementations.
> >>>>>>
> >>>>>> What I am looking at is piggy-backing on a concurrent mark task to get
> >>>>>> the summary size of live objects - using the 'straight-forward'
> >>>>>> heap-inspection like approach is prohibitively expensive.
> >>>>>
> >>>>> In Shenandoah, this information is already collected during concurrent
> >>>>> marking. We currently don't print it directly, but we could certainly do
> >>>>> that. I'll look into implementing it. I'll also look into exposing
> >>>>> liveness info via JMX.
> >>>>>
> >>>>> I'm not quite sure about G1: that information would only be collected
> >>>>> during mixed or full collections. I am not sure if G1 prints it, though.
> >>>>>
> >>>>> ZGC prints this under -Xlog:gc+heap:
> >>>>>
> >>>>> [6,502s][info][gc,heap     ] GC(0)                Mark Start
> >>>>> Mark End        Relocate Start      Relocate End           High
> >>>>>          Low
> >>>>> [6,502s][info][gc,heap     ] GC(0)  Capacity:      834M (10%)
> >>>>> 1076M (13%)        1092M (14%)        1092M (14%)        1092M (14%)
> >>>>>         834M (10%)
> >>>>> [6,502s][info][gc,heap     ] GC(0)      Free:     7154M (90%)
> >>>>> 6912M (87%)        6916M (87%)        7388M (92%)        7388M (92%)
> >>>>>        6896M (86%)
> >>>>> [6,502s][info][gc,heap     ] GC(0)      Used:      834M (10%)
> >>>>> 1076M (13%)        1072M (13%)         600M (8%)         1092M (14%)
> >>>>>         600M (8%)
> >>>>> [6,502s][info][gc,heap     ] GC(0)      Live:         -
> >>>>> 195M (2%)          195M (2%)          195M (2%)             -
> >>>>>           -
> >>>>> [6,502s][info][gc,heap     ] GC(0) Allocated:         -
> >>>>> 242M (3%)          270M (3%)          380M (5%)             -
> >>>>>           -
> >>>>> [6,502s][info][gc,heap     ] GC(0)   Garbage:         -
> >>>>> 638M (8%)          606M (8%)           24M (0%)             -
> >>>>>           -
> >>>>> [6,502s][info][gc,heap     ] GC(0) Reclaimed:         -
> >>>>>     -                32M (0%)          614M (8%)             -
> >>>>>          -
> >>>>>
> >>>>> I hope that is useful?
> >>>>>
> >>>>> Thanks,
> >>>>> Roman
> >>>>>
> >>>>
> >>>
Reply | Threaded
Open this post in threaded view
|

Re: Can GC implementations provide a cheap estimation of live set size?

Per Liden
On 2/15/21 11:47 AM, Jaroslav Bachorík wrote:

> On Mon, Feb 15, 2021 at 11:24 AM Per Liden <[hidden email]> wrote:
>>
>> Hi,
>>
>> On 2/15/21 10:44 AM, Jaroslav Bachorík wrote:
>>> Hi again,
>>>
>>> I continued experimenting with Shenandoah and ZGC which already are
>>> tracking liveness. I am emitting a (partially filled) GCHeapSummary
>>> JFR event to capture used/live sizes.
>>> For Shenandoah the event is emitted at the very end of the
>>> `ShenandoahConcurrentGC::op_final_mark()` method and for ZGC it is the
>>> `ZMark::end()` method. The exact changes can be checked via branch
>>> comparison (https://github.com/openjdk/jdk/compare/master...DataDog:jb/live_set_1)
>>> but bear in mind that this is just an experimental code with no
>>> intention being checked in in its current form.
>>>
>>> Unfortunately, when I run an application on such modified JVM and
>>> collect a JFR recording the live set size numbers seem a bit 'low' -
>>> eg. on both ZGC and Shenandoah (using an already available liveness
>>> info) the reported liveness is ~50% of the reported usage. Is there a
>>> good explanation for this?
>>
>> When you create the GCHeapSummary, the "live" value reflects what was
>> live after marking, while the "used" value reflects the usage when the
>> GC cycle ended. So, after marking ended, some amount of garbage was
>> likely reclaimed, but then new objects were also allocated. For ZGC
>> (don't know if Shenandoah shows this), you can see details of how much
>> was reclaimed and how much was allocated in the GC log.
>
> Definitely - it's just that a diff of >100MB (eg. for ZGC 350MB used
> vs. 170MB live) struck me as a bit suspicious. But maybe it is
> expected.

It's impossible to say if it's expected or not, without knowing what the
application is doing, it's allocation rate, etc. The application could
be allocating several gigabytes per second, in which case the diff could
be large. However, if the application is just idling and isn't
allocating anything, then live is expected to be equal (or close to
equal) to used.

/Per

>
> -JB-
>
>>
>> /Per
>>
>>>
>>> Thanks!
>>>
>>> -JB-
>>>
>>> On Thu, Feb 11, 2021 at 7:09 PM Jaroslav Bachorík
>>> <[hidden email]> wrote:
>>>>
>>>> On Thu, Feb 11, 2021 at 6:55 PM Roman Kennke <[hidden email]> wrote:
>>>>>
>>>>> Notice that liveness information is only somewhat reliable right after
>>>>> marking. In Shenandoah, this is in the final-mark pause, and then the
>>>>
>>>> Yes, I understand this. What I am looking at is to have something like
>>>> 'last known liveness' value - captured at a well defined point and
>>>> providing an estimate within the bounds of GC implementation.
>>>>
>>>>> program is at a safepoint already. This is where you'd want to emit a
>>>>> JMX event or something similar. You can't simply query a counter and
>>>>> assume it represents current liveness in the middle or outside of GC
>>>>> cycle. This should be true for all GCs.
>>>>>
>>>>> For Serial and Parallel I am not sure at all that you can do this.
>>>>> AFAIK, they don't count liveness at all.
>>>>>
>>>>> Roman
>>>>>
>>>>>> Hi Roman,
>>>>>>
>>>>>> Thanks for your response. I checked ZGC implementation and, indeed, it
>>>>>> is very easy to get the liveness information just by extending
>>>>>> `ZStatHeap` class to report the last valid value of
>>>>>> `_at_mark_end.live`.
>>>>>>
>>>>>> I am also able to get this info from Shenandoah, although my first
>>>>>> attempt still involves a safepointing VM operation since I need to
>>>>>> iterate over regions to get the liveness info for each of them and sum
>>>>>> it up. I think it is still an acceptable trade-off, though.
>>>>>>
>>>>>> The next one in the queue is the Serial GC. My assumptions, based on
>>>>>> reading the code, are that for young gen 'live = used' at the end of
>>>>>> DefNewGeneration::collect() method and for old gen 'live = used -
>>>>>> slack' (slack is the cumulative size of objects considered to be alive
>>>>>> for the purpose of compaction although they are really dead - see
>>>>>> CompactibleSpace::scan_and_forward()). Does this sound reasonable?
>>>>>>
>>>>>> I will post my findings for Parallel GC and G1 GC later.
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> -JB-
>>>>>>
>>>>>> On Wed, Feb 10, 2021 at 11:34 AM Roman Kennke <[hidden email]> wrote:
>>>>>>>
>>>>>>> Hello Jaroslav,
>>>>>>>
>>>>>>>> In connection with https://bugs.openjdk.java.net/browse/JDK-8258431 I
>>>>>>>> am trying to figure out whether providing a cheap estimation of live
>>>>>>>> set size is something actually achievable across various GC
>>>>>>>> implementations.
>>>>>>>>
>>>>>>>> What I am looking at is piggy-backing on a concurrent mark task to get
>>>>>>>> the summary size of live objects - using the 'straight-forward'
>>>>>>>> heap-inspection like approach is prohibitively expensive.
>>>>>>>
>>>>>>> In Shenandoah, this information is already collected during concurrent
>>>>>>> marking. We currently don't print it directly, but we could certainly do
>>>>>>> that. I'll look into implementing it. I'll also look into exposing
>>>>>>> liveness info via JMX.
>>>>>>>
>>>>>>> I'm not quite sure about G1: that information would only be collected
>>>>>>> during mixed or full collections. I am not sure if G1 prints it, though.
>>>>>>>
>>>>>>> ZGC prints this under -Xlog:gc+heap:
>>>>>>>
>>>>>>> [6,502s][info][gc,heap     ] GC(0)                Mark Start
>>>>>>> Mark End        Relocate Start      Relocate End           High
>>>>>>>           Low
>>>>>>> [6,502s][info][gc,heap     ] GC(0)  Capacity:      834M (10%)
>>>>>>> 1076M (13%)        1092M (14%)        1092M (14%)        1092M (14%)
>>>>>>>          834M (10%)
>>>>>>> [6,502s][info][gc,heap     ] GC(0)      Free:     7154M (90%)
>>>>>>> 6912M (87%)        6916M (87%)        7388M (92%)        7388M (92%)
>>>>>>>         6896M (86%)
>>>>>>> [6,502s][info][gc,heap     ] GC(0)      Used:      834M (10%)
>>>>>>> 1076M (13%)        1072M (13%)         600M (8%)         1092M (14%)
>>>>>>>          600M (8%)
>>>>>>> [6,502s][info][gc,heap     ] GC(0)      Live:         -
>>>>>>> 195M (2%)          195M (2%)          195M (2%)             -
>>>>>>>            -
>>>>>>> [6,502s][info][gc,heap     ] GC(0) Allocated:         -
>>>>>>> 242M (3%)          270M (3%)          380M (5%)             -
>>>>>>>            -
>>>>>>> [6,502s][info][gc,heap     ] GC(0)   Garbage:         -
>>>>>>> 638M (8%)          606M (8%)           24M (0%)             -
>>>>>>>            -
>>>>>>> [6,502s][info][gc,heap     ] GC(0) Reclaimed:         -
>>>>>>>      -                32M (0%)          614M (8%)             -
>>>>>>>           -
>>>>>>>
>>>>>>> I hope that is useful?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Roman
>>>>>>>
>>>>>>
>>>>>