OpenJDK › OpenJDK Build Infrastructure

RFR: 8257913: Add more known library locations to simplify Linux cross-compilation

Classic

List

Threaded

15 messages Options

Aleksey Shipilev-5

RFR: 8257913: Add more known library locations to simplify Linux cross-compilation

Current cross-compilation configurations require adding the whole lot of configure options to make it find the dependencies: freetype and X11 libs are the only problematic dependencies. We should be able to specify just the `--with-sysroot`, and get everything else autodetected.

After this patch, doing `sh ./configure --openjdk-target=powerpc64le-linux-gnu --with-sysroot=/chroots/ppc64el`, where chroot is `debootstrap`-ed chroot as per our build docs, is enough to get everything configured. See the GH Actions definition to see what options are not needed anymore.

Some explanations:
- `*-gnuabi64` is needed for `mips64el`
- `*-gnueabihf` is needed for `armhf`
- `$OPENJDK_TARGET_CPU_AUTOCONF-linux-gnu` is needed for `powerpc*`, because OpenJDK build rewrites `VAR_CPU` to `ppc`, leading to mismatch.

Additional testing:
- [x] Configure with target `x86_64-linux-gnu`
- [x] Configure with cross-target `i686-linux-gnu`
- [x] Configure with cross-target `alpha-linux-gnu`
- [x] Configure with cross-target `aarch64-linux-gnu`
- [x] Configure with cross-target `arm-linux-gnueabihf`
- [x] Configure with cross-target `arm-linux-gnueabi`
- [x] Configure with cross-target `mipsel-linux-gnu`
- [x] Configure with cross-target `mips64el-linux-gnuabi64`
- [x] Configure with cross-target `powerpc-linux-gnu`
- [x] Configure with cross-target `powerpc64-linux-gnu`
- [x] Configure with cross-target `powerpc64le-linux-gnu`
- [x] Configure with cross-target `m68k-linux-gnu`
- [x] Configure with cross-target `sh4-linux-gnu`
- [ ] Build with target `x86_64-linux-gnu`
- [x] Build with cross-target `i686-linux-gnu`
- [ ] Build with cross-target `alpha-linux-gnu`
- [ ] Build with cross-target `aarch64-linux-gnu`
- [ ] Build with cross-target `arm-linux-gnueabihf`
- [ ] Build with cross-target `arm-linux-gnueabi`
- [ ] Build with cross-target `mipsel-linux-gnu`
- [ ] Build with cross-target `mips64el-linux-gnuabi64`
- [ ] Build with cross-target `powerpc-linux-gnu`
- [ ] Build with cross-target `powerpc64-linux-gnu`
- [ ] Build with cross-target `powerpc64le-linux-gnu`
- [ ] Build with cross-target `m68k-linux-gnu`
- [ ] Build with cross-target `sh4-linux-gnu`

-------------

Commit messages:
- 8257913: Enable cross-compilation with only sysroot specified

Changes: https://git.openjdk.java.net/jdk/pull/2833/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=2833&range=00
Issue: https://bugs.openjdk.java.net/browse/JDK-8257913
Stats: 32 lines in 4 files changed: 28 ins; 4 del; 0 mod
Patch: https://git.openjdk.java.net/jdk/pull/2833.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/2833/head:pull/2833

PR: https://git.openjdk.java.net/jdk/pull/2833

Erik Joelsson-2

Re: RFR: 8257913: Add more known library locations to simplify Linux cross-compilation

On Thu, 4 Mar 2021 18:58:03 GMT, Aleksey Shipilev <[hidden email]> wrote:

> Current cross-compilation configurations require adding the whole lot of configure options to make it find the dependencies: freetype and X11 libs are the only problematic dependencies. We should be able to specify just the `--with-sysroot`, and get everything else autodetected.
>
> After this patch, doing `sh ./configure --openjdk-target=powerpc64le-linux-gnu --with-sysroot=/chroots/ppc64el`, where chroot is `debootstrap`-ed chroot as per our build docs, is enough to get everything configured. See the GH Actions definition to see what options are not needed anymore.
>
> Some explanations:
> - `*-gnuabi64` is needed for `mips64el`
> - `*-gnueabihf` is needed for `armhf`
> - `$OPENJDK_TARGET_CPU_AUTOCONF-linux-gnu` is needed for `powerpc*`, because OpenJDK build rewrites `VAR_CPU` to `ppc`, leading to mismatch.
>
> Additional testing:
> - [x] Configure with target `x86_64-linux-gnu`
> - [x] Configure with cross-target `i686-linux-gnu`
> - [x] Configure with cross-target `alpha-linux-gnu`
> - [x] Configure with cross-target `aarch64-linux-gnu`
> - [x] Configure with cross-target `arm-linux-gnueabihf`
> - [x] Configure with cross-target `arm-linux-gnueabi`
> - [x] Configure with cross-target `mipsel-linux-gnu`
> - [x] Configure with cross-target `mips64el-linux-gnuabi64`
> - [x] Configure with cross-target `powerpc-linux-gnu`
> - [x] Configure with cross-target `powerpc64-linux-gnu`
> - [x] Configure with cross-target `powerpc64le-linux-gnu`
> - [x] Configure with cross-target `m68k-linux-gnu`
> - [x] Configure with cross-target `sh4-linux-gnu`
> - [ ] Build with target `x86_64-linux-gnu`
> - [x] Build with cross-target `i686-linux-gnu`
> - [ ] Build with cross-target `alpha-linux-gnu`
> - [ ] Build with cross-target `aarch64-linux-gnu`
> - [ ] Build with cross-target `arm-linux-gnueabihf`
> - [ ] Build with cross-target `arm-linux-gnueabi`
> - [ ] Build with cross-target `mipsel-linux-gnu`
> - [ ] Build with cross-target `mips64el-linux-gnuabi64`
> - [ ] Build with cross-target `powerpc-linux-gnu`
> - [ ] Build with cross-target `powerpc64-linux-gnu`
> - [ ] Build with cross-target `powerpc64le-linux-gnu`
> - [ ] Build with cross-target `m68k-linux-gnu`
> - [ ] Build with cross-target `sh4-linux-gnu`

make/autoconf/lib-freetype.m4 line 201:

> 199: if test "x$FOUND_FREETYPE" != "xyes" ; then
> 200: LIB_CHECK_POTENTIAL_FREETYPE([$FREETYPE_BASE_DIR/include],
> 201: [$FREETYPE_BASE_DIR/lib/$OPENJDK_TARGET_CPU-linux-gnueabihf], [well-known location])

Do we really need to explicitly list all the gnu* variants here? Isn't the subdir simply the value of openjdk-target? At least it looks that way in the submit.yml.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2833

Erik Joelsson-2

Re: RFR: 8257913: Add more known library locations to simplify Linux cross-compilation

On Thu, 4 Mar 2021 19:30:27 GMT, Erik Joelsson <[hidden email]> wrote:

>> Current cross-compilation configurations require adding the whole lot of configure options to make it find the dependencies: freetype and X11 libs are the only problematic dependencies. We should be able to specify just the `--with-sysroot`, and get everything else autodetected.
>>
>> After this patch, doing `sh ./configure --openjdk-target=powerpc64le-linux-gnu --with-sysroot=/chroots/ppc64el`, where chroot is `debootstrap`-ed chroot as per our build docs, is enough to get everything configured. See the GH Actions definition to see what options are not needed anymore.
>>
>> Some explanations:
>> - `*-gnuabi64` is needed for `mips64el`
>> - `*-gnueabihf` is needed for `armhf`
>> - `$OPENJDK_TARGET_CPU_AUTOCONF-linux-gnu` is needed for `powerpc*`, because OpenJDK build rewrites `VAR_CPU` to `ppc`, leading to mismatch.
>>
>> Additional testing:
>> - [x] Configure with target `x86_64-linux-gnu`
>> - [x] Configure with cross-target `i686-linux-gnu`
>> - [x] Configure with cross-target `alpha-linux-gnu`
>> - [x] Configure with cross-target `aarch64-linux-gnu`
>> - [x] Configure with cross-target `arm-linux-gnueabihf`
>> - [x] Configure with cross-target `arm-linux-gnueabi`
>> - [x] Configure with cross-target `mipsel-linux-gnu`
>> - [x] Configure with cross-target `mips64el-linux-gnuabi64`
>> - [x] Configure with cross-target `powerpc-linux-gnu`
>> - [x] Configure with cross-target `powerpc64-linux-gnu`
>> - [x] Configure with cross-target `powerpc64le-linux-gnu`
>> - [x] Configure with cross-target `m68k-linux-gnu`
>> - [x] Configure with cross-target `sh4-linux-gnu`
>> - [ ] Build with target `x86_64-linux-gnu`
>> - [x] Build with cross-target `i686-linux-gnu`
>> - [ ] Build with cross-target `alpha-linux-gnu`
>> - [ ] Build with cross-target `aarch64-linux-gnu`
>> - [ ] Build with cross-target `arm-linux-gnueabihf`
>> - [ ] Build with cross-target `arm-linux-gnueabi`
>> - [ ] Build with cross-target `mipsel-linux-gnu`
>> - [ ] Build with cross-target `mips64el-linux-gnuabi64`
>> - [ ] Build with cross-target `powerpc-linux-gnu`
>> - [ ] Build with cross-target `powerpc64-linux-gnu`
>> - [ ] Build with cross-target `powerpc64le-linux-gnu`
>> - [ ] Build with cross-target `m68k-linux-gnu`
>> - [ ] Build with cross-target `sh4-linux-gnu`
>
> make/autoconf/lib-freetype.m4 line 201:
>
>> 199: if test "x$FOUND_FREETYPE" != "xyes" ; then
>> 200: LIB_CHECK_POTENTIAL_FREETYPE([$FREETYPE_BASE_DIR/include],
>> 201: [$FREETYPE_BASE_DIR/lib/$OPENJDK_TARGET_CPU-linux-gnueabihf], [well-known location])
>
> Do we really need to explicitly list all the gnu* variants here? Isn't the subdir simply the value of openjdk-target? At least it looks that way in the submit.yml.

I think that value should be accessible from OPENJDK_TARGET_AUTOCONF_NAME.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2833

Aleksey Shipilev-5

Re: RFR: 8257913: Add more known library locations to simplify Linux cross-compilation

On Thu, 4 Mar 2021 19:34:16 GMT, Erik Joelsson <[hidden email]> wrote:

>> make/autoconf/lib-freetype.m4 line 201:
>>
>>> 199: if test "x$FOUND_FREETYPE" != "xyes" ; then
>>> 200: LIB_CHECK_POTENTIAL_FREETYPE([$FREETYPE_BASE_DIR/include],
>>> 201: [$FREETYPE_BASE_DIR/lib/$OPENJDK_TARGET_CPU-linux-gnueabihf], [well-known location])
>>
>> Do we really need to explicitly list all the gnu* variants here? Isn't the subdir simply the value of openjdk-target? At least it looks that way in the submit.yml.
>
> I think that value should be accessible from OPENJDK_TARGET_AUTOCONF_NAME.

Unfortunately, AUTOCONF_NAME is a quadruplet. For example, for ARM `OPENJDK_TARGET_AUTOCONF_NAME` is `arm-unknown-linux-gnueabihf`, but the subdirectory is `arm-linux-gnueabihf`. I don't see how can I get the original `--openjdk-target` provided by user here. Seems to me it is not passed to autoconf?

-------------

PR: https://git.openjdk.java.net/jdk/pull/2833

Aleksey Shipilev-5

Re: RFR: 8257913: Add more known library locations to simplify Linux cross-compilation [v2]

In reply to this post by Aleksey Shipilev-5

Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:

Rework to handle different ABIs

-------------

Changes:
- all: https://git.openjdk.java.net/jdk/pull/2833/files
- new: https://git.openjdk.java.net/jdk/pull/2833/files/652710e4..1ccf6aa1

Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2833&range=01
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2833&range=00-01

Stats: 55 lines in 3 files changed: 36 ins; 13 del; 6 mod
Patch: https://git.openjdk.java.net/jdk/pull/2833.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/2833/head:pull/2833

PR: https://git.openjdk.java.net/jdk/pull/2833

Aleksey Shipilev-5

Re: RFR: 8257913: Add more known library locations to simplify Linux cross-compilation [v2]

In reply to this post by Aleksey Shipilev-5

On Fri, 5 Mar 2021 07:06:24 GMT, Aleksey Shipilev <[hidden email]> wrote:

>> I think that value should be accessible from OPENJDK_TARGET_AUTOCONF_NAME.
>
> Yes, original `openjdk-target` would be enough. Unfortunately, `AUTOCONF_NAME` seems to be the quadruplet generated from original `--openjdk-target`. For example, for ARM we supply `--openjdk-target=arm-linux-gnueabihf`, but `OPENJDK_TARGET_AUTOCONF_NAME` is `arm-unknown-linux-gnueabihf`, which is not the same as the subdirectory `arm-linux-gnueabihf`. I don't see how can I get the original `--openjdk-target` provided by user here. Seems to me it is not passed to autoconf?
>
> I'd be happy to use original `openjdk-target` if I understood how to pass it through... Without it, we have to resort listing gnu* variants for the platforms we build for.

What I can do is parse the ABI from the original string, and then reconstruct the triplet as `$OPENJDK_TARGET_CPU-linux-$OPENJDK_TARGET_ABI`. See the update.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2833

Aleksey Shipilev-5

Re: RFR: 8257913: Add more known library locations to simplify Linux cross-compilation [v3]

In reply to this post by Aleksey Shipilev-5

Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:

Replace with linux with TARGET_OS

-------------

Changes:
- all: https://git.openjdk.java.net/jdk/pull/2833/files
- new: https://git.openjdk.java.net/jdk/pull/2833/files/1ccf6aa1..5b64487f

Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2833&range=02
- incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2833&range=01-02

Stats: 6 lines in 2 files changed: 0 ins; 0 del; 6 mod
Patch: https://git.openjdk.java.net/jdk/pull/2833.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/2833/head:pull/2833

PR: https://git.openjdk.java.net/jdk/pull/2833

Erik Joelsson-2

Re: RFR: 8257913: Add more known library locations to simplify Linux cross-compilation [v3]

In reply to this post by Aleksey Shipilev-5

On Fri, 5 Mar 2021 07:35:44 GMT, Aleksey Shipilev <[hidden email]> wrote:

>> Yes, original `openjdk-target` would be enough. Unfortunately, `AUTOCONF_NAME` seems to be the quadruplet generated from original `--openjdk-target`. For example, for ARM we supply `--openjdk-target=arm-linux-gnueabihf`, but `OPENJDK_TARGET_AUTOCONF_NAME` is `arm-unknown-linux-gnueabihf`, which is not the same as the subdirectory `arm-linux-gnueabihf`. I don't see how can I get the original `--openjdk-target` provided by user here. Seems to me it is not passed to autoconf?
>>
>> I'd be happy to use original `openjdk-target` if I understood how to pass it through... Without it, we have to resort listing gnu* variants for the platforms we build for.
>
> What I can do is parse the ABI from the original string, and then reconstruct the triplet as `$OPENJDK_TARGET_CPU-linux-$OPENJDK_TARGET_ABI`. See the update.

This is certainly much better. Would it make sense to replace "linux" with $OPENJDK_TARGET_OS as well, just to avoid having any hard coded values there?

-------------

PR: https://git.openjdk.java.net/jdk/pull/2833

Aleksey Shipilev-5

Re: RFR: 8257913: Add more known library locations to simplify Linux cross-compilation [v3]

On Fri, 5 Mar 2021 14:06:05 GMT, Erik Joelsson <[hidden email]> wrote:

>> What I can do is parse the ABI from the original string, and then reconstruct the triplet as `$OPENJDK_TARGET_CPU-linux-$OPENJDK_TARGET_ABI`. See the update.
>
> This is certainly much better. Would it make sense to replace "linux" with $OPENJDK_TARGET_OS as well, just to avoid having any hard coded values there?

Yeah, I can, see update.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2833

Erik Joelsson-2

Re: RFR: 8257913: Add more known library locations to simplify Linux cross-compilation [v3]

In reply to this post by Aleksey Shipilev-5

On Fri, 5 Mar 2021 16:03:18 GMT, Aleksey Shipilev <[hidden email]> wrote:

Looks good now and definitely a nice change to find these libs automatically.

-------------

Marked as reviewed by erikj (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/2833

Aleksey Shipilev-5

Integrated: 8257913: Add more known library locations to simplify Linux cross-compilation

In reply to this post by Aleksey Shipilev-5

On Thu, 4 Mar 2021 18:58:03 GMT, Aleksey Shipilev <[hidden email]> wrote:

This pull request has now been integrated.

Changeset: 61cff4da
Author: Aleksey Shipilev <[hidden email]>
URL: https://git.openjdk.java.net/jdk/commit/61cff4da
Stats: 55 lines in 4 files changed: 51 ins; 4 del; 0 mod

8257913: Add more known library locations to simplify Linux cross-compilation

Reviewed-by: erikj

-------------

PR: https://git.openjdk.java.net/jdk/pull/2833

Magnus Ihse Bursie-3

Re: RFR: 8257913: Add more known library locations to simplify Linux cross-compilation [v3]

In reply to this post by Aleksey Shipilev-5

On Fri, 5 Mar 2021 16:03:18 GMT, Aleksey Shipilev <[hidden email]> wrote:

make/autoconf/platform.m4 line 364:

> 362: AC_SUBST(OPENJDK_TARGET_CPU_AUTOCONF)
> 363: AC_SUBST(OPENJDK_TARGET_LIBC)
> 364: AC_SUBST(OPENJDK_TARGET_ABI)

I realize this has already been integrated, but I don't fully understand this. There is no corresponding change in spec.gmk.in adding OPENJDK_TARGET/BUILD_CPU_AUTOCONF and OPENJDK_TARGET/BUILD_ABI.

So why the AC_SUBST()?

-------------

PR: https://git.openjdk.java.net/jdk/pull/2833

Aleksey Shipilev-5

Re: RFR: 8257913: Add more known library locations to simplify Linux cross-compilation [v3]

On Mon, 8 Mar 2021 16:11:20 GMT, Magnus Ihse Bursie <[hidden email]> wrote:

>> Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Replace with linux with TARGET_OS
>
> make/autoconf/platform.m4 line 364:
>
>> 362: AC_SUBST(OPENJDK_TARGET_CPU_AUTOCONF)
>> 363: AC_SUBST(OPENJDK_TARGET_LIBC)
>> 364: AC_SUBST(OPENJDK_TARGET_ABI)
>
> I realize this has already been integrated, but I don't fully understand this. There is no corresponding change in spec.gmk.in adding OPENJDK_TARGET/BUILD_CPU_AUTOCONF and OPENJDK_TARGET/BUILD_ABI.
>
> So why the AC_SUBST()?

I thought we need to declare these as `AC_SUBST` as we define the variables outside this file? I can revert these additions if you think these are incorrect.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2833

Magnus Ihse Bursie-3

Re: RFR: 8257913: Add more known library locations to simplify Linux cross-compilation [v3]

On Mon, 8 Mar 2021 16:14:21 GMT, Aleksey Shipilev <[hidden email]> wrote:

>> make/autoconf/platform.m4 line 364:
>>
>>> 362: AC_SUBST(OPENJDK_TARGET_CPU_AUTOCONF)
>>> 363: AC_SUBST(OPENJDK_TARGET_LIBC)
>>> 364: AC_SUBST(OPENJDK_TARGET_ABI)
>>
>> I realize this has already been integrated, but I don't fully understand this. There is no corresponding change in spec.gmk.in adding OPENJDK_TARGET/BUILD_CPU_AUTOCONF and OPENJDK_TARGET/BUILD_ABI.
>>
>> So why the AC_SUBST()?
>
> I thought we need to declare these as `AC_SUBST` as we define the variables outside this file? I can revert these additions if you think these are incorrect.

No, they are only needed to be able to populate the @FOO@ tags in spec.gmk.in. All the *.m4 files are concatenated into a single shell script by autoconf, so referring to variables in a different m4 file does not really matter.

I don't think it's worth a separate bug to remove them, but if you touch the area again you can get rid of them while you're at it :)

-------------

PR: https://git.openjdk.java.net/jdk/pull/2833

Aleksey Shipilev-5

Re: RFR: 8257913: Add more known library locations to simplify Linux cross-compilation [v3]

On Mon, 8 Mar 2021 23:29:25 GMT, Magnus Ihse Bursie <[hidden email]> wrote:

>> I thought we need to declare these as `AC_SUBST` as we define the variables outside this file? I can revert these additions if you think these are incorrect.
>
> No, they are only needed to be able to populate the @FOO@ tags in spec.gmk.in. All the *.m4 files are concatenated into a single shell script by autoconf, so referring to variables in a different m4 file does not really matter.
>
> I don't think it's worth a separate bug to remove them, but if you touch the area again you can get rid of them while you're at it :)

Noted. Thanks!

-------------

PR: https://git.openjdk.java.net/jdk/pull/2833