Size of pool for agent vms

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Size of pool for agent vms

Zeller, Arno
Hi,

I have a question regarding the -agentvm parameter of jtreg. The documentation states the following:
-avm | -agentvm -  Run tests using a pool of reusable JVMs.

When I use this parameter to execute jtreg tests for OpenJDK hotspot and execute the test group ':hotspot_all_no_apps' (even with a concurrency of one) I end up with a pool of more than 20 VMs. This tends to exhaust my native memory and I end up with "out of native memory" errors in the jtreg driver VM or the agent VMs.

Is there a way to limit the size of the pool?

Best regards,
Arno

Reply | Threaded
Open this post in threaded view
|

Re: Size of pool for agent vms

Jonathan Gibbons
Arno,

The pool is currently a function of the concurrency you use and the VM
options specified on the command line.

If you use the same VM options to compile and run the tests, the size of
the pool is the concurrency value ... one VM in the pool for each thread.

If you use different VM options to compile and run tests, the size of
the pool is double the concurrency value ... one VM in the pool for each
thread to compile tests, and anotherVM in the pool for each thread to
run tests.

In addition, there may be additional VMs outside the pool for /othervm
tests, or for tests that (internally) start their own JVM.

In general, if you want fewer VMs running, reduce the value of your
concurrency option.  Also, consider using VM options to limit the
resources given to each VM.

If you are specifying "concurrency 1" you should have at most two JVMs
in the agent pool.

-- Jon



On 11/21/2018 06:38 AM, Zeller, Arno wrote:

> Hi,
>
> I have a question regarding the -agentvm parameter of jtreg. The documentation states the following:
> -avm | -agentvm -  Run tests using a pool of reusable JVMs.
>
> When I use this parameter to execute jtreg tests for OpenJDK hotspot and execute the test group ':hotspot_all_no_apps' (even with a concurrency of one) I end up with a pool of more than 20 VMs. This tends to exhaust my native memory and I end up with "out of native memory" errors in the jtreg driver VM or the agent VMs.
>
> Is there a way to limit the size of the pool?
>
> Best regards,
> Arno
>

Reply | Threaded
Open this post in threaded view
|

RE: Size of pool for agent vms

Zeller, Arno
Hi Jonathan,

I found https://bugs.openjdk.java.net/browse/CODETOOLS-7902454 and this seems to be just the problem I have too. The VM pooling seems to behave unexpected and when running a lot of test with high concurrency I run out of native memory on some machines because there are much more than double the concurrency number of VMs alive and using memory.

Do you think you could integrate the proposed patch from Aleksey? That would really help me in our automated test setup.

Best regards,
Arno


On 11/21/2018 Jonathan Gibbons wrote:

>Arno,
>
>The pool is currently a function of the concurrency you use and the VM
>options specified on the command line.
>
>If you use the same VM options to compile and run the tests, the size of
>the pool is the concurrency value ... one VM in the pool for each thread.
>
>If you use different VM options to compile and run tests, the size of
>the pool is double the concurrency value ... one VM in the pool for each
>thread to compile tests, and anotherVM in the pool for each thread to
>run tests.
>
>In addition, there may be additional VMs outside the pool for /othervm
>tests, or for tests that (internally) start their own JVM.
>
>In general, if you want fewer VMs running, reduce the value of your
>concurrency option.  Also, consider using VM options to limit the
>resources given to each VM.
>
>If you are specifying "concurrency 1" you should have at most two JVMs
>in the agent pool.
>
>-- Jon
>
>
>
>On 11/21/2018 06:38 AM, Zeller, Arno wrote:
>> Hi,
>>
>> I have a question regarding the -agentvm parameter of jtreg. The
>documentation states the following:
>> -avm | -agentvm -  Run tests using a pool of reusable JVMs.
>>
>> When I use this parameter to execute jtreg tests for OpenJDK hotspot and
>execute the test group ':hotspot_all_no_apps' (even with a concurrency of
>one) I end up with a pool of more than 20 VMs. This tends to exhaust my
>native memory and I end up with "out of native memory" errors in the jtreg
>driver VM or the agent VMs.
>>
>> Is there a way to limit the size of the pool?
>>
>> Best regards,
>> Arno
>>

Reply | Threaded
Open this post in threaded view
|

Re: Size of pool for agent vms

Jonathan Gibbons
I believe there is an underlying bug that needs to be investigated and
addressed, that too many VMs are being created in the first place. I
believe the fix from Aleksey is at best a workaround to cover up the
underlying bad behavior.

-- Jon

On 5/8/19 3:26 AM, Zeller, Arno wrote:

> Hi Jonathan,
>
> I found https://bugs.openjdk.java.net/browse/CODETOOLS-7902454 and this seems to be just the problem I have too. The VM pooling seems to behave unexpected and when running a lot of test with high concurrency I run out of native memory on some machines because there are much more than double the concurrency number of VMs alive and using memory.
>
> Do you think you could integrate the proposed patch from Aleksey? That would really help me in our automated test setup.
>
> Best regards,
> Arno
>
>
> On 11/21/2018 Jonathan Gibbons wrote:
>> Arno,
>>
>> The pool is currently a function of the concurrency you use and the VM
>> options specified on the command line.
>>
>> If you use the same VM options to compile and run the tests, the size of
>> the pool is the concurrency value ... one VM in the pool for each thread.
>>
>> If you use different VM options to compile and run tests, the size of
>> the pool is double the concurrency value ... one VM in the pool for each
>> thread to compile tests, and anotherVM in the pool for each thread to
>> run tests.
>>
>> In addition, there may be additional VMs outside the pool for /othervm
>> tests, or for tests that (internally) start their own JVM.
>>
>> In general, if you want fewer VMs running, reduce the value of your
>> concurrency option.  Also, consider using VM options to limit the
>> resources given to each VM.
>>
>> If you are specifying "concurrency 1" you should have at most two JVMs
>> in the agent pool.
>>
>> -- Jon
>>
>>
>>
>> On 11/21/2018 06:38 AM, Zeller, Arno wrote:
>>> Hi,
>>>
>>> I have a question regarding the -agentvm parameter of jtreg. The
>> documentation states the following:
>>> -avm | -agentvm -  Run tests using a pool of reusable JVMs.
>>>
>>> When I use this parameter to execute jtreg tests for OpenJDK hotspot and
>> execute the test group ':hotspot_all_no_apps' (even with a concurrency of
>> one) I end up with a pool of more than 20 VMs. This tends to exhaust my
>> native memory and I end up with "out of native memory" errors in the jtreg
>> driver VM or the agent VMs.
>>> Is there a way to limit the size of the pool?
>>>
>>> Best regards,
>>> Arno
>>>