RFR: 8261505: Test test/hotspot/jtreg/gc/parallel/TestDynShrinkHeap.java killed by Linux OOM Killer

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

RFR: 8261505: Test test/hotspot/jtreg/gc/parallel/TestDynShrinkHeap.java killed by Linux OOM Killer

Christoph Göttschkes-2
On memory constrained devices, the test might get killed by the linux kernel OOM Killer.

Executing the test with the JTreg test harness makes the test fail and get killed by the OOM Killer.
Executing the test manually, by using the JTreg provided "rerun" command line, the test succeeds.
This happened on a Raspberry PI 2, which has only 1G of memory available.

I added an "os.maxMemory" requirement, so the test gets skipped.

-------------

Commit messages:
 - Adds os.maxMemory requirement.

Changes: https://git.openjdk.java.net/jdk/pull/2507/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=2507&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8261505
  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2507.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2507/head:pull/2507

PR: https://git.openjdk.java.net/jdk/pull/2507
Reply | Threaded
Open this post in threaded view
|

Re: RFR: 8261505: Test test/hotspot/jtreg/gc/parallel/TestDynShrinkHeap.java killed by Linux OOM Killer

Stefan Johansson
On Wed, 10 Feb 2021 12:13:54 GMT, Christoph Göttschkes <[hidden email]> wrote:

> On memory constrained devices, the test might get killed by the linux kernel OOM Killer.
>
> Executing the test with the JTreg test harness makes the test fail and get killed by the OOM Killer.
> Executing the test manually, by using the JTreg provided "rerun" command line, the test succeeds.
> This happened on a Raspberry PI 2, which has only 1G of memory available.
>
> I added an "os.maxMemory" requirement, so the test gets skipped.

Marked as reviewed by sjohanss (Reviewer).

Hi Christoph,

This looks good. The test is setting a 1GB max heap so it seems reasonable to require the system to have at least that.

Another thing to look at when tests are getting killed by the OOM killer is the number of concurrent test jobs. For a system with 1GB of memory that should be 1, so in your case you can't go lower. To be certain you only run one test at a time you could run `configure` with `--with-test-jobs=1 `, but according to `doc/testing.md` this should be the default for your system:
The test concurrency (`-concurrency`).

Defaults to TEST_JOBS (if set by `--with-test-jobs=`), otherwise it defaults to
JOBS, except for Hotspot, where the default is *number of CPU cores/2*,
but never more than *memory size in GB/2*.
 

The reason that the rerun succeeds is most likely because then you don't have the JTREG process running along side the test and consuming resources.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2507
Reply | Threaded
Open this post in threaded view
|

Re: RFR: 8261505: Test test/hotspot/jtreg/gc/parallel/TestDynShrinkHeap.java killed by Linux OOM Killer

Christoph Göttschkes-2
On Thu, 11 Feb 2021 19:39:23 GMT, Stefan Johansson <[hidden email]> wrote:

>> On memory constrained devices, the test might get killed by the linux kernel OOM Killer.
>>
>> Executing the test with the JTreg test harness makes the test fail and get killed by the OOM Killer.
>> Executing the test manually, by using the JTreg provided "rerun" command line, the test succeeds.
>> This happened on a Raspberry PI 2, which has only 1G of memory available.
>>
>> I added an "os.maxMemory" requirement, so the test gets skipped.
>
> Marked as reviewed by sjohanss (Reviewer).

Hi Stefan,

thanks for the review. I am aware of the concurrency feature of the JTreg runner and am always using a concurrency of 1 on embedded devices. Even if they are more powerful, since it makes the test execution less reliable.

I found some more tests with the same problem, but will file a single bug and fix all in one go, as soon as I have time for that.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2507
Reply | Threaded
Open this post in threaded view
|

Re: RFR: 8261505: Test test/hotspot/jtreg/gc/parallel/TestDynShrinkHeap.java killed by Linux OOM Killer

Stefan Johansson
On Fri, 12 Feb 2021 07:38:38 GMT, Christoph Göttschkes <[hidden email]> wrote:

>> Marked as reviewed by sjohanss (Reviewer).
>
> Hi Stefan,
>
> thanks for the review. I am aware of the concurrency feature of the JTreg runner and am always using a concurrency of 1 on embedded devices. Even if they are more powerful, since it makes the test execution less reliable.
>
> I found some more tests with the same problem, but will file a single bug and fix all in one go, as soon as I have time for that.

I'll wait for a second reviewer before sponsoring this, just in case anyone has a different view on how to handle this.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2507
Reply | Threaded
Open this post in threaded view
|

Re: RFR: 8261505: Test test/hotspot/jtreg/gc/parallel/TestDynShrinkHeap.java killed by Linux OOM Killer

Thomas Schatzl-4
In reply to this post by Christoph Göttschkes-2
On Wed, 10 Feb 2021 12:13:54 GMT, Christoph Göttschkes <[hidden email]> wrote:

> On memory constrained devices, the test might get killed by the linux kernel OOM Killer.
>
> Executing the test with the JTreg test harness makes the test fail and get killed by the OOM Killer.
> Executing the test manually, by using the JTreg provided "rerun" command line, the test succeeds.
> This happened on a Raspberry PI 2, which has only 1G of memory available.
>
> I added an "os.maxMemory" requirement, so the test gets skipped.

Lgtm.

-------------

Marked as reviewed by tschatzl (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/2507
Reply | Threaded
Open this post in threaded view
|

Integrated: 8261505: Test test/hotspot/jtreg/gc/parallel/TestDynShrinkHeap.java killed by Linux OOM Killer

Christoph Göttschkes-2
In reply to this post by Christoph Göttschkes-2
On Wed, 10 Feb 2021 12:13:54 GMT, Christoph Göttschkes <[hidden email]> wrote:

> On memory constrained devices, the test might get killed by the linux kernel OOM Killer.
>
> Executing the test with the JTreg test harness makes the test fail and get killed by the OOM Killer.
> Executing the test manually, by using the JTreg provided "rerun" command line, the test succeeds.
> This happened on a Raspberry PI 2, which has only 1G of memory available.
>
> I added an "os.maxMemory" requirement, so the test gets skipped.

This pull request has now been integrated.

Changeset: ebaa58d9
Author:    Christoph Göttschkes <[hidden email]>
Committer: Stefan Johansson <[hidden email]>
URL:       https://git.openjdk.java.net/jdk/commit/ebaa58d9
Stats:     1 line in 1 file changed: 0 ins; 0 del; 1 mod

8261505: Test test/hotspot/jtreg/gc/parallel/TestDynShrinkHeap.java killed by Linux OOM Killer

Reviewed-by: sjohanss, tschatzl

-------------

PR: https://git.openjdk.java.net/jdk/pull/2507