Quantcast

RFR: 10: 8160638: solaris JVM unable to allocate more than 2GB of direct byte buffers when max heap is <= 2GB

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RFR: 10: 8160638: solaris JVM unable to allocate more than 2GB of direct byte buffers when max heap is <= 2GB

vladimir kempik
Hello

Please review this fix for bug JDK-8160638
<https://bugs.openjdk.java.net/browse/JDK-8160638>

The issue is with solaris only.

When java mmaps heap (with compressed Oops enabled), mmaped heap become
upper limit for any native mallocs.

So when heap is starting at 2 gb, the maximum we can malloc is 2gb.

Native malloc is used by Direct Memory Buffers, so even with
-XX:MaxDirectMemorySize=100g we are still limited with less than 2 gb of
memory for Direct Memory Buffers.

The fix moves HeapBaseMinAddress to upper space when it's needed for
MaxDirectMemorySize to operate properly, leaving about 1 gb of native
memory for java's needs.

Testing: jprt, included testcase.

Webrev - http://cr.openjdk.java.net/~vkempik/8160638/webrev.00/

Bug - https://bugs.openjdk.java.net/browse/JDK-8160638

Thanks, Vladimir

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: RFR: 10: 8160638: solaris JVM unable to allocate more than 2GB of direct byte buffers when max heap is <= 2GB

vladimir kempik
Can I have somebody to take a look at this please.

Thanks, Vladimir


10.04.17 16:30, Vladimir Kempik пишет:

>
> Hello
>
> Please review this fix for bug JDK-8160638
> <https://bugs.openjdk.java.net/browse/JDK-8160638>
>
> The issue is with solaris only.
>
> When java mmaps heap (with compressed Oops enabled), mmaped heap
> become upper limit for any native mallocs.
>
> So when heap is starting at 2 gb, the maximum we can malloc is 2gb.
>
> Native malloc is used by Direct Memory Buffers, so even with
> -XX:MaxDirectMemorySize=100g we are still limited with less than 2 gb
> of memory for Direct Memory Buffers.
>
> The fix moves HeapBaseMinAddress to upper space when it's needed for
> MaxDirectMemorySize to operate properly, leaving about 1 gb of native
> memory for java's needs.
>
> Testing: jprt, included testcase.
>
> Webrev - http://cr.openjdk.java.net/~vkempik/8160638/webrev.00/
>
> Bug - https://bugs.openjdk.java.net/browse/JDK-8160638
>
> Thanks, Vladimir
>

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: RFR: 10: 8160638: solaris JVM unable to allocate more than 2GB of direct byte buffers when max heap is <= 2GB

Vladimir Kozlov
In reply to this post by vladimir kempik
Looks good to me. Someone from Runtime have to review this too.

Thanks,
Vladimir

On 4/10/17 6:30 AM, Vladimir Kempik wrote:

> Hello
>
> Please review this fix for bug JDK-8160638
> <https://bugs.openjdk.java.net/browse/JDK-8160638>
>
> The issue is with solaris only.
>
> When java mmaps heap (with compressed Oops enabled), mmaped heap become
> upper limit for any native mallocs.
>
> So when heap is starting at 2 gb, the maximum we can malloc is 2gb.
>
> Native malloc is used by Direct Memory Buffers, so even with
> -XX:MaxDirectMemorySize=100g we are still limited with less than 2 gb of
> memory for Direct Memory Buffers.
>
> The fix moves HeapBaseMinAddress to upper space when it's needed for
> MaxDirectMemorySize to operate properly, leaving about 1 gb of native
> memory for java's needs.
>
> Testing: jprt, included testcase.
>
> Webrev - http://cr.openjdk.java.net/~vkempik/8160638/webrev.00/
>
> Bug - https://bugs.openjdk.java.net/browse/JDK-8160638
>
> Thanks, Vladimir
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: RFR: 10: 8160638: solaris JVM unable to allocate more than 2GB of direct byte buffers when max heap is <= 2GB

David Holmes
In reply to this post by vladimir kempik
Hi Vladimir,

On 10/04/2017 11:30 PM, Vladimir Kempik wrote:

> Hello
>
> Please review this fix for bug JDK-8160638
> <https://bugs.openjdk.java.net/browse/JDK-8160638>
>
> The issue is with solaris only.
>
> When java mmaps heap (with compressed Oops enabled), mmaped heap become
> upper limit for any native mallocs.
>
> So when heap is starting at 2 gb, the maximum we can malloc is 2gb.
>
> Native malloc is used by Direct Memory Buffers, so even with
> -XX:MaxDirectMemorySize=100g we are still limited with less than 2 gb of
> memory for Direct Memory Buffers.
>
> The fix moves HeapBaseMinAddress to upper space when it's needed for
> MaxDirectMemorySize to operate properly, leaving about 1 gb of native
> memory for java's needs.

Why is this not handled directly in Arguments::set_heap_size()?

I admit I'm unclear about the "magic numbers" involved here. From the
bug report we have to have a heap >2G for things to work okay. So not
sure why we adjust the HeapBaseMinAddress the way you do, or why 1GB is
significant in the calculations ??

May be best for a GC person to review this.

Thanks,
David

> Testing: jprt, included testcase.
>
> Webrev - http://cr.openjdk.java.net/~vkempik/8160638/webrev.00/
>
> Bug - https://bugs.openjdk.java.net/browse/JDK-8160638
>
> Thanks, Vladimir
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: RFR: 10: 8160638: solaris JVM unable to allocate more than 2GB of direct byte buffers when max heap is <= 2GB

vladimir kempik
Hello

Sorry if it sounds a bit complicated, it's actually not that much.


18.04.17 7:03, David Holmes пишет:

> Hi Vladimir,
>
> On 10/04/2017 11:30 PM, Vladimir Kempik wrote:
>> Hello
>>
>> Please review this fix for bug JDK-8160638
>> <https://bugs.openjdk.java.net/browse/JDK-8160638>
>>
>> The issue is with solaris only.
>>
>> When java mmaps heap (with compressed Oops enabled), mmaped heap become
>> upper limit for any native mallocs.
>>
>> So when heap is starting at 2 gb, the maximum we can malloc is 2gb.
>>
>> Native malloc is used by Direct Memory Buffers, so even with
>> -XX:MaxDirectMemorySize=100g we are still limited with less than 2 gb of
>> memory for Direct Memory Buffers.
>>
>> The fix moves HeapBaseMinAddress to upper space when it's needed for
>> MaxDirectMemorySize to operate properly, leaving about 1 gb of native
>> memory for java's needs.
>
> Why is this not handled directly in Arguments::set_heap_size()?

I don't change heap size here, only location of heap.
>
> I admit I'm unclear about the "magic numbers" involved here. From the
> bug report we have to have a heap >2G for things to work okay. So not
> sure why we adjust the HeapBaseMinAddress the way you do, or why 1GB
> is significant in the calculations ??
heap size doesn't have much to do with the issue, the issue happens when
compressedOops get enabled, and it usualy happens with heap <=2G.

for example, if we have 2gb of java heap (with compressed Oops),
starting at 2gb location (so java heap is located from 2gb to 4gb in
memory map) then we are limited with malloc allocations on solaris,
malloc will only allocate until we hit the wall of java heap starting at
2gb. malloc won't be able to allocate past 4gb limit in this case.
By default the heap is starting at 2gb on amd64, and 6gb on sparcv9,
however 0-4gb space is already used on sparcv9, so sparcv9 has same 2G
of free space before heap base.
since java internals is using this 2gb of c++ heap as well I thought it
would be good to not occupy more than half of that space with Direct
Memory Buffers.

Thanks, Vladimir

>
> May be best for a GC person to review this.
>
> Thanks,
> David
>
>> Testing: jprt, included testcase.
>>
>> Webrev - http://cr.openjdk.java.net/~vkempik/8160638/webrev.00/
>>
>> Bug - https://bugs.openjdk.java.net/browse/JDK-8160638
>>
>> Thanks, Vladimir
>>

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: RFR: 10: 8160638: solaris JVM unable to allocate more than 2GB of direct byte buffers when max heap is <= 2GB

David Holmes
On 18/04/2017 8:51 PM, Vladimir Kempik wrote:
> Hello
>
> Sorry if it sounds a bit complicated, it's actually not that much.

:)

> 18.04.17 7:03, David Holmes пишет:
>> Hi Vladimir,
>>
>> On 10/04/2017 11:30 PM, Vladimir Kempik wrote:
>>> Hello
>>>
>>> Please review this fix for bug JDK-8160638
>>> <https://bugs.openjdk.java.net/browse/JDK-8160638>
>>>
>>> The issue is with solaris only.
>>>
>>> When java mmaps heap (with compressed Oops enabled), mmaped heap become
>>> upper limit for any native mallocs.
>>>
>>> So when heap is starting at 2 gb, the maximum we can malloc is 2gb.
>>>
>>> Native malloc is used by Direct Memory Buffers, so even with
>>> -XX:MaxDirectMemorySize=100g we are still limited with less than 2 gb of
>>> memory for Direct Memory Buffers.
>>>
>>> The fix moves HeapBaseMinAddress to upper space when it's needed for
>>> MaxDirectMemorySize to operate properly, leaving about 1 gb of native
>>> memory for java's needs.
>>
>> Why is this not handled directly in Arguments::set_heap_size()?
>
> I don't change heap size here, only location of heap.

True, but set_heap_size already does some checking of, and potential
adjustment of HeapBaseMinAddress. And it isn't immediately obvious when
your checks happen in relation to that code.

>>
>> I admit I'm unclear about the "magic numbers" involved here. From the
>> bug report we have to have a heap >2G for things to work okay. So not
>> sure why we adjust the HeapBaseMinAddress the way you do, or why 1GB
>> is significant in the calculations ??
> heap size doesn't have much to do with the issue, the issue happens when
> compressedOops get enabled, and it usualy happens with heap <=2G.
>
> for example, if we have 2gb of java heap (with compressed Oops),
> starting at 2gb location (so java heap is located from 2gb to 4gb in
> memory map) then we are limited with malloc allocations on solaris,
> malloc will only allocate until we hit the wall of java heap starting at
> 2gb. malloc won't be able to allocate past 4gb limit in this case.
> By default the heap is starting at 2gb on amd64, and 6gb on sparcv9,
> however 0-4gb space is already used on sparcv9, so sparcv9 has same 2G
> of free space before heap base.
> since java internals is using this 2gb of c++ heap as well I thought it
> would be good to not occupy more than half of that space with Direct
> Memory Buffers.

So if I understand this right, the direct byte buffers will always be
allocated from this before-heap-memory ie below HeapBaseMinAddress, so
we're shifting that up to make room for the space requested by
MaxDirectMemorySize if it is >2GB.

I don't get the 1GB adjustment though - doesn't that reduce what is
available for direct byte buffers if "java internals" use more than 1GB
itself?

I think a picture would paint a thousand words here :)

Thanks,
David

>
> Thanks, Vladimir
>>
>> May be best for a GC person to review this.
>>
>> Thanks,
>> David
>>
>>> Testing: jprt, included testcase.
>>>
>>> Webrev - http://cr.openjdk.java.net/~vkempik/8160638/webrev.00/
>>>
>>> Bug - https://bugs.openjdk.java.net/browse/JDK-8160638
>>>
>>> Thanks, Vladimir
>>>
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: RFR: 10: 8160638: solaris JVM unable to allocate more than 2GB of direct byte buffers when max heap is <= 2GB

Zhengyu Gu-2


On 04/18/2017 08:22 AM, David Holmes wrote:

> On 18/04/2017 8:51 PM, Vladimir Kempik wrote:
>> Hello
>>
>> Sorry if it sounds a bit complicated, it's actually not that much.
>
> :)
>
>> 18.04.17 7:03, David Holmes пишет:
>>> Hi Vladimir,
>>>
>>> On 10/04/2017 11:30 PM, Vladimir Kempik wrote:
>>>> Hello
>>>>
>>>> Please review this fix for bug JDK-8160638
>>>> <https://bugs.openjdk.java.net/browse/JDK-8160638>
>>>>
>>>> The issue is with solaris only.
>>>>
>>>> When java mmaps heap (with compressed Oops enabled), mmaped heap become
>>>> upper limit for any native mallocs.
>>>>
>>>> So when heap is starting at 2 gb, the maximum we can malloc is 2gb.
>>>>
>>>> Native malloc is used by Direct Memory Buffers, so even with
>>>> -XX:MaxDirectMemorySize=100g we are still limited with less than 2
>>>> gb of
>>>> memory for Direct Memory Buffers.
>>>>
>>>> The fix moves HeapBaseMinAddress to upper space when it's needed for
>>>> MaxDirectMemorySize to operate properly, leaving about 1 gb of native
>>>> memory for java's needs.
>>>
>>> Why is this not handled directly in Arguments::set_heap_size()?
>>
>> I don't change heap size here, only location of heap.
>
> True, but set_heap_size already does some checking of, and potential
> adjustment of HeapBaseMinAddress. And it isn't immediately obvious when
> your checks happen in relation to that code.
>
>>>
>>> I admit I'm unclear about the "magic numbers" involved here. From the
>>> bug report we have to have a heap >2G for things to work okay. So not
>>> sure why we adjust the HeapBaseMinAddress the way you do, or why 1GB
>>> is significant in the calculations ??
>> heap size doesn't have much to do with the issue, the issue happens when
>> compressedOops get enabled, and it usualy happens with heap <=2G.
>>
>> for example, if we have 2gb of java heap (with compressed Oops),
>> starting at 2gb location (so java heap is located from 2gb to 4gb in
>> memory map) then we are limited with malloc allocations on solaris,
>> malloc will only allocate until we hit the wall of java heap starting at
>> 2gb. malloc won't be able to allocate past 4gb limit in this case.
>> By default the heap is starting at 2gb on amd64, and 6gb on sparcv9,
>> however 0-4gb space is already used on sparcv9, so sparcv9 has same 2G
>> of free space before heap base.
>> since java internals is using this 2gb of c++ heap as well I thought it
>> would be good to not occupy more than half of that space with Direct
>> Memory Buffers.
>
> So if I understand this right, the direct byte buffers will always be
> allocated from this before-heap-memory ie below HeapBaseMinAddress, so
> we're shifting that up to make room for the space requested by
> MaxDirectMemorySize if it is >2GB.

This is really the malloc implementation problem in Solaris, which has
low malloc base address by default and uses sbrk to expand malloc heap.

It will still run into trouble if "UseCompressedClassPointers" flag is on.

Thanks,

-Zhengyu

>
> I don't get the 1GB adjustment though - doesn't that reduce what is
> available for direct byte buffers if "java internals" use more than 1GB
> itself?
>
> I think a picture would paint a thousand words here :)
>
> Thanks,
> David
>
>>
>> Thanks, Vladimir
>>>
>>> May be best for a GC person to review this.
>>>
>>> Thanks,
>>> David
>>>
>>>> Testing: jprt, included testcase.
>>>>
>>>> Webrev - http://cr.openjdk.java.net/~vkempik/8160638/webrev.00/
>>>>
>>>> Bug - https://bugs.openjdk.java.net/browse/JDK-8160638
>>>>
>>>> Thanks, Vladimir
>>>>
>>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: RFR: 10: 8160638: solaris JVM unable to allocate more than 2GB of direct byte buffers when max heap is <= 2GB

Jiangli Zhou
In reply to this post by vladimir kempik
Hi Vladimir,

This looks safe to me. With your change, the HeapBaseMinAddress is adjusted higher with an offset of (MaxDirectMemorySize - 1g) on solaris. It’s done via set_ergonmic_flags(), which is before the HeapBaseMinAddressConstraintFunc check. If the moved HeapBaseMinAddress violates the constraint, it will be caught by the check.

Thanks,
Jiangli

> On Apr 14, 2017, at 5:28 AM, Vladimir Kempik <[hidden email]> wrote:
>
> Can I have somebody to take a look at this please.
>
> Thanks, Vladimir
>
>
> 10.04.17 16:30, Vladimir Kempik пишет:
>>
>> Hello
>>
>> Please review this fix for bug JDK-8160638 <https://bugs.openjdk.java.net/browse/JDK-8160638>
>>
>> The issue is with solaris only.
>>
>> When java mmaps heap (with compressed Oops enabled), mmaped heap become upper limit for any native mallocs.
>>
>> So when heap is starting at 2 gb, the maximum we can malloc is 2gb.
>>
>> Native malloc is used by Direct Memory Buffers, so even with -XX:MaxDirectMemorySize=100g we are still limited with less than 2 gb of memory for Direct Memory Buffers.
>>
>> The fix moves HeapBaseMinAddress to upper space when it's needed for MaxDirectMemorySize to operate properly, leaving about 1 gb of native memory for java's needs.
>>
>> Testing: jprt, included testcase.
>>
>> Webrev - http://cr.openjdk.java.net/~vkempik/8160638/webrev.00/
>>
>> Bug - https://bugs.openjdk.java.net/browse/JDK-8160638
>>
>> Thanks, Vladimir
>>
>

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: RFR: 10: 8160638: solaris JVM unable to allocate more than 2GB of direct byte buffers when max heap is <= 2GB

vladimir kempik
In reply to this post by Zhengyu Gu-2
Hello Zhengyu Gu

How will UseCompressedClassPointers run into trouble?

UseCompressedClassPointers requires UseCompressedOops to be enabled, and
in this case the code will adjust heap into higher space.

and java  -Xmx2g -XX:MaxDirectMemorySize=9g -XX:+UseCompressedOops
-XX:+UseCompressedCPointers -XX:HeapBaseMinAddress=10g MaxDirectMemory

causes no troubles for me in test on jdk with no patch.

Thanks, Vladimir


18.04.17 15:50, Zhengyu Gu пишет:

>
>
> On 04/18/2017 08:22 AM, David Holmes wrote:
>> On 18/04/2017 8:51 PM, Vladimir Kempik wrote:
>>> Hello
>>>
>>> Sorry if it sounds a bit complicated, it's actually not that much.
>>
>> :)
>>
>>> 18.04.17 7:03, David Holmes пишет:
>>>> Hi Vladimir,
>>>>
>>>> On 10/04/2017 11:30 PM, Vladimir Kempik wrote:
>>>>> Hello
>>>>>
>>>>> Please review this fix for bug JDK-8160638
>>>>> <https://bugs.openjdk.java.net/browse/JDK-8160638>
>>>>>
>>>>> The issue is with solaris only.
>>>>>
>>>>> When java mmaps heap (with compressed Oops enabled), mmaped heap
>>>>> become
>>>>> upper limit for any native mallocs.
>>>>>
>>>>> So when heap is starting at 2 gb, the maximum we can malloc is 2gb.
>>>>>
>>>>> Native malloc is used by Direct Memory Buffers, so even with
>>>>> -XX:MaxDirectMemorySize=100g we are still limited with less than 2
>>>>> gb of
>>>>> memory for Direct Memory Buffers.
>>>>>
>>>>> The fix moves HeapBaseMinAddress to upper space when it's needed for
>>>>> MaxDirectMemorySize to operate properly, leaving about 1 gb of native
>>>>> memory for java's needs.
>>>>
>>>> Why is this not handled directly in Arguments::set_heap_size()?
>>>
>>> I don't change heap size here, only location of heap.
>>
>> True, but set_heap_size already does some checking of, and potential
>> adjustment of HeapBaseMinAddress. And it isn't immediately obvious when
>> your checks happen in relation to that code.
>>
>>>>
>>>> I admit I'm unclear about the "magic numbers" involved here. From the
>>>> bug report we have to have a heap >2G for things to work okay. So not
>>>> sure why we adjust the HeapBaseMinAddress the way you do, or why 1GB
>>>> is significant in the calculations ??
>>> heap size doesn't have much to do with the issue, the issue happens
>>> when
>>> compressedOops get enabled, and it usualy happens with heap <=2G.
>>>
>>> for example, if we have 2gb of java heap (with compressed Oops),
>>> starting at 2gb location (so java heap is located from 2gb to 4gb in
>>> memory map) then we are limited with malloc allocations on solaris,
>>> malloc will only allocate until we hit the wall of java heap
>>> starting at
>>> 2gb. malloc won't be able to allocate past 4gb limit in this case.
>>> By default the heap is starting at 2gb on amd64, and 6gb on sparcv9,
>>> however 0-4gb space is already used on sparcv9, so sparcv9 has same 2G
>>> of free space before heap base.
>>> since java internals is using this 2gb of c++ heap as well I thought it
>>> would be good to not occupy more than half of that space with Direct
>>> Memory Buffers.
>>
>> So if I understand this right, the direct byte buffers will always be
>> allocated from this before-heap-memory ie below HeapBaseMinAddress, so
>> we're shifting that up to make room for the space requested by
>> MaxDirectMemorySize if it is >2GB.
>
> This is really the malloc implementation problem in Solaris, which has
> low malloc base address by default and uses sbrk to expand malloc heap.
>
> It will still run into trouble if "UseCompressedClassPointers" flag is
> on.

>
> Thanks,
>
> -Zhengyu
>
>>
>> I don't get the 1GB adjustment though - doesn't that reduce what is
>> available for direct byte buffers if "java internals" use more than 1GB
>> itself?
>>
>> I think a picture would paint a thousand words here :)
>>
>> Thanks,
>> David
>>
>>>
>>> Thanks, Vladimir
>>>>
>>>> May be best for a GC person to review this.
>>>>
>>>> Thanks,
>>>> David
>>>>
>>>>> Testing: jprt, included testcase.
>>>>>
>>>>> Webrev - http://cr.openjdk.java.net/~vkempik/8160638/webrev.00/
>>>>>
>>>>> Bug - https://bugs.openjdk.java.net/browse/JDK-8160638
>>>>>
>>>>> Thanks, Vladimir
>>>>>
>>>

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: RFR: 10: 8160638: solaris JVM unable to allocate more than 2GB of direct byte buffers when max heap is <= 2GB

Zhengyu Gu-2
Hi Vladimir,

Sorry, my bad. Somehow, I had impression that compressed class space is
below the heap, and has address < 4G.

-Zhengyu




On 04/18/2017 02:17 PM, Vladimir Kempik wrote:

> Hello Zhengyu Gu
>
> How will UseCompressedClassPointers run into trouble?
>
> UseCompressedClassPointers requires UseCompressedOops to be enabled, and
> in this case the code will adjust heap into higher space.
>
> and java  -Xmx2g -XX:MaxDirectMemorySize=9g -XX:+UseCompressedOops
> -XX:+UseCompressedCPointers -XX:HeapBaseMinAddress=10g MaxDirectMemory
>
> causes no troubles for me in test on jdk with no patch.
>
> Thanks, Vladimir
>
>
> 18.04.17 15:50, Zhengyu Gu пишет:
>>
>>
>> On 04/18/2017 08:22 AM, David Holmes wrote:
>>> On 18/04/2017 8:51 PM, Vladimir Kempik wrote:
>>>> Hello
>>>>
>>>> Sorry if it sounds a bit complicated, it's actually not that much.
>>>
>>> :)
>>>
>>>> 18.04.17 7:03, David Holmes пишет:
>>>>> Hi Vladimir,
>>>>>
>>>>> On 10/04/2017 11:30 PM, Vladimir Kempik wrote:
>>>>>> Hello
>>>>>>
>>>>>> Please review this fix for bug JDK-8160638
>>>>>> <https://bugs.openjdk.java.net/browse/JDK-8160638>
>>>>>>
>>>>>> The issue is with solaris only.
>>>>>>
>>>>>> When java mmaps heap (with compressed Oops enabled), mmaped heap
>>>>>> become
>>>>>> upper limit for any native mallocs.
>>>>>>
>>>>>> So when heap is starting at 2 gb, the maximum we can malloc is 2gb.
>>>>>>
>>>>>> Native malloc is used by Direct Memory Buffers, so even with
>>>>>> -XX:MaxDirectMemorySize=100g we are still limited with less than 2
>>>>>> gb of
>>>>>> memory for Direct Memory Buffers.
>>>>>>
>>>>>> The fix moves HeapBaseMinAddress to upper space when it's needed for
>>>>>> MaxDirectMemorySize to operate properly, leaving about 1 gb of native
>>>>>> memory for java's needs.
>>>>>
>>>>> Why is this not handled directly in Arguments::set_heap_size()?
>>>>
>>>> I don't change heap size here, only location of heap.
>>>
>>> True, but set_heap_size already does some checking of, and potential
>>> adjustment of HeapBaseMinAddress. And it isn't immediately obvious when
>>> your checks happen in relation to that code.
>>>
>>>>>
>>>>> I admit I'm unclear about the "magic numbers" involved here. From the
>>>>> bug report we have to have a heap >2G for things to work okay. So not
>>>>> sure why we adjust the HeapBaseMinAddress the way you do, or why 1GB
>>>>> is significant in the calculations ??
>>>> heap size doesn't have much to do with the issue, the issue happens
>>>> when
>>>> compressedOops get enabled, and it usualy happens with heap <=2G.
>>>>
>>>> for example, if we have 2gb of java heap (with compressed Oops),
>>>> starting at 2gb location (so java heap is located from 2gb to 4gb in
>>>> memory map) then we are limited with malloc allocations on solaris,
>>>> malloc will only allocate until we hit the wall of java heap
>>>> starting at
>>>> 2gb. malloc won't be able to allocate past 4gb limit in this case.
>>>> By default the heap is starting at 2gb on amd64, and 6gb on sparcv9,
>>>> however 0-4gb space is already used on sparcv9, so sparcv9 has same 2G
>>>> of free space before heap base.
>>>> since java internals is using this 2gb of c++ heap as well I thought it
>>>> would be good to not occupy more than half of that space with Direct
>>>> Memory Buffers.
>>>
>>> So if I understand this right, the direct byte buffers will always be
>>> allocated from this before-heap-memory ie below HeapBaseMinAddress, so
>>> we're shifting that up to make room for the space requested by
>>> MaxDirectMemorySize if it is >2GB.
>>
>> This is really the malloc implementation problem in Solaris, which has
>> low malloc base address by default and uses sbrk to expand malloc heap.
>>
>> It will still run into trouble if "UseCompressedClassPointers" flag is
>> on.
>
>>
>> Thanks,
>>
>> -Zhengyu
>>
>>>
>>> I don't get the 1GB adjustment though - doesn't that reduce what is
>>> available for direct byte buffers if "java internals" use more than 1GB
>>> itself?
>>>
>>> I think a picture would paint a thousand words here :)
>>>
>>> Thanks,
>>> David
>>>
>>>>
>>>> Thanks, Vladimir
>>>>>
>>>>> May be best for a GC person to review this.
>>>>>
>>>>> Thanks,
>>>>> David
>>>>>
>>>>>> Testing: jprt, included testcase.
>>>>>>
>>>>>> Webrev - http://cr.openjdk.java.net/~vkempik/8160638/webrev.00/
>>>>>>
>>>>>> Bug - https://bugs.openjdk.java.net/browse/JDK-8160638
>>>>>>
>>>>>> Thanks, Vladimir
>>>>>>
>>>>
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: RFR: 10: 8160638: solaris JVM unable to allocate more than 2GB of direct byte buffers when max heap is <= 2GB

vladimir kempik
In reply to this post by David Holmes
Hello


18.04.17 15:22, David Holmes пишет:

> On 18/04/2017 8:51 PM, Vladimir Kempik wrote:
>> Hello
>>
>> Sorry if it sounds a bit complicated, it's actually not that much.
>
> :)
>
>> 18.04.17 7:03, David Holmes пишет:
>>> Hi Vladimir,
>>>
>>> On 10/04/2017 11:30 PM, Vladimir Kempik wrote:
>>>> Hello
>>>>
>>>> Please review this fix for bug JDK-8160638
>>>> <https://bugs.openjdk.java.net/browse/JDK-8160638>
>>>>
>>>> The issue is with solaris only.
>>>>
>>>> When java mmaps heap (with compressed Oops enabled), mmaped heap
>>>> become
>>>> upper limit for any native mallocs.
>>>>
>>>> So when heap is starting at 2 gb, the maximum we can malloc is 2gb.
>>>>
>>>> Native malloc is used by Direct Memory Buffers, so even with
>>>> -XX:MaxDirectMemorySize=100g we are still limited with less than 2
>>>> gb of
>>>> memory for Direct Memory Buffers.
>>>>
>>>> The fix moves HeapBaseMinAddress to upper space when it's needed for
>>>> MaxDirectMemorySize to operate properly, leaving about 1 gb of native
>>>> memory for java's needs.
>>>
>>> Why is this not handled directly in Arguments::set_heap_size()?
>>
>> I don't change heap size here, only location of heap.
>
> True, but set_heap_size already does some checking of, and potential
> adjustment of HeapBaseMinAddress. And it isn't immediately obvious
> when your checks happen in relation to that code.
>
After Jiangli's comment:
 >It’s done via set_ergonmic_flags(), which is before the
HeapBaseMinAddressConstraintFunc check. If the moved HeapBaseMinAddress
violates the constraint, it will be caught by the check.
do you still think it's an issue ?

>>>
>>> I admit I'm unclear about the "magic numbers" involved here. From the
>>> bug report we have to have a heap >2G for things to work okay. So not
>>> sure why we adjust the HeapBaseMinAddress the way you do, or why 1GB
>>> is significant in the calculations ??
>> heap size doesn't have much to do with the issue, the issue happens when
>> compressedOops get enabled, and it usualy happens with heap <=2G.
>>
>> for example, if we have 2gb of java heap (with compressed Oops),
>> starting at 2gb location (so java heap is located from 2gb to 4gb in
>> memory map) then we are limited with malloc allocations on solaris,
>> malloc will only allocate until we hit the wall of java heap starting at
>> 2gb. malloc won't be able to allocate past 4gb limit in this case.
>> By default the heap is starting at 2gb on amd64, and 6gb on sparcv9,
>> however 0-4gb space is already used on sparcv9, so sparcv9 has same 2G
>> of free space before heap base.
>> since java internals is using this 2gb of c++ heap as well I thought it
>> would be good to not occupy more than half of that space with Direct
>> Memory Buffers.
>
> So if I understand this right, the direct byte buffers will always be
> allocated from this before-heap-memory ie below HeapBaseMinAddress, so
> we're shifting that up to make room for the space requested by
> MaxDirectMemorySize if it is >2GB.
>
> I don't get the 1GB adjustment though - doesn't that reduce what is
> available for direct byte buffers if "java internals" use more than
> 1GB itself?
>
> I think a picture would paint a thousand words here :)
Do you think it's better to leave all of initial 2gb c++ heap to java
internals ?
THanks, Vladimir

>
> Thanks,
> David
>
>>
>> Thanks, Vladimir
>>>
>>> May be best for a GC person to review this.
>>>
>>> Thanks,
>>> David
>>>
>>>> Testing: jprt, included testcase.
>>>>
>>>> Webrev - http://cr.openjdk.java.net/~vkempik/8160638/webrev.00/
>>>>
>>>> Bug - https://bugs.openjdk.java.net/browse/JDK-8160638
>>>>
>>>> Thanks, Vladimir
>>>>
>>

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: RFR: 10: 8160638: solaris JVM unable to allocate more than 2GB of direct byte buffers when max heap is <= 2GB

David Holmes
Hi Vladimir,

On 19/04/2017 11:14 PM, Vladimir Kempik wrote:

> 18.04.17 15:22, David Holmes пишет:
>> On 18/04/2017 8:51 PM, Vladimir Kempik wrote:
>>> Sorry if it sounds a bit complicated, it's actually not that much.
>>
>> :)
>>
>>> 18.04.17 7:03, David Holmes пишет:
>>>> On 10/04/2017 11:30 PM, Vladimir Kempik wrote:
>>>>> Please review this fix for bug JDK-8160638
>>>>> <https://bugs.openjdk.java.net/browse/JDK-8160638>
>>>>>
>>>>> The issue is with solaris only.
>>>>>
>>>>> When java mmaps heap (with compressed Oops enabled), mmaped heap
>>>>> become upper limit for any native mallocs.
>>>>>
>>>>> So when heap is starting at 2 gb, the maximum we can malloc is 2gb.
>>>>>
>>>>> Native malloc is used by Direct Memory Buffers, so even with
>>>>> -XX:MaxDirectMemorySize=100g we are still limited with less than 2
>>>>> gb of memory for Direct Memory Buffers.
>>>>>
>>>>> The fix moves HeapBaseMinAddress to upper space when it's needed for
>>>>> MaxDirectMemorySize to operate properly, leaving about 1 gb of native
>>>>> memory for java's needs.
>>>>
>>>> Why is this not handled directly in Arguments::set_heap_size()?
>>>
>>> I don't change heap size here, only location of heap.
>>
>> True, but set_heap_size already does some checking of, and potential
>> adjustment of HeapBaseMinAddress. And it isn't immediately obvious
>> when your checks happen in relation to that code.
>>
> After Jiangli's comment:
>>It’s done via set_ergonmic_flags(), which is before the
> HeapBaseMinAddressConstraintFunc check. If the moved HeapBaseMinAddress
> violates the constraint, it will be caught by the check.
> do you still think it's an issue ?

Jiangli's comment simply confirms that the location is okay with respect
to other checks. My point was that it is not immediately obvious in what
order the various uses of HeapBaseMinAddress occur - you have trace
through the call sequence. Placing the new check/update in the same
function as another check/update seems simpler and clearer to me. But it
is a minor point.

>>>>
>>>> I admit I'm unclear about the "magic numbers" involved here. From the
>>>> bug report we have to have a heap >2G for things to work okay. So not
>>>> sure why we adjust the HeapBaseMinAddress the way you do, or why 1GB
>>>> is significant in the calculations ??
>>> heap size doesn't have much to do with the issue, the issue happens when
>>> compressedOops get enabled, and it usualy happens with heap <=2G.
>>>
>>> for example, if we have 2gb of java heap (with compressed Oops),
>>> starting at 2gb location (so java heap is located from 2gb to 4gb in
>>> memory map) then we are limited with malloc allocations on solaris,
>>> malloc will only allocate until we hit the wall of java heap starting at
>>> 2gb. malloc won't be able to allocate past 4gb limit in this case.
>>> By default the heap is starting at 2gb on amd64, and 6gb on sparcv9,
>>> however 0-4gb space is already used on sparcv9, so sparcv9 has same 2G
>>> of free space before heap base.
>>> since java internals is using this 2gb of c++ heap as well I thought it
>>> would be good to not occupy more than half of that space with Direct
>>> Memory Buffers.
>>
>> So if I understand this right, the direct byte buffers will always be
>> allocated from this before-heap-memory ie below HeapBaseMinAddress, so
>> we're shifting that up to make room for the space requested by
>> MaxDirectMemorySize if it is >2GB.
>>
>> I don't get the 1GB adjustment though - doesn't that reduce what is
>> available for direct byte buffers if "java internals" use more than
>> 1GB itself?
>>
>> I think a picture would paint a thousand words here :)
> Do you think it's better to leave all of initial 2gb c++ heap to java
> internals ?

I'm just trying to establish that the change actually makes
MaxDirectMemorySize memory available for use.

Thanks,
David

> THanks, Vladimir
>>
>> Thanks,
>> David
>>
>>>
>>> Thanks, Vladimir
>>>>
>>>> May be best for a GC person to review this.
>>>>
>>>> Thanks,
>>>> David
>>>>
>>>>> Testing: jprt, included testcase.
>>>>>
>>>>> Webrev - http://cr.openjdk.java.net/~vkempik/8160638/webrev.00/
>>>>>
>>>>> Bug - https://bugs.openjdk.java.net/browse/JDK-8160638
>>>>>
>>>>> Thanks, Vladimir
>>>>>
>>>
>
Loading...