Building OpenJDK9 on MSYS2

classic Classic list List threaded Threaded
38 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Building OpenJDK9 on MSYS2

Peter Budai
Hello,

According to http://hg.openjdk.java.net/jdk9/jdk9/file/a08cbfc0e4ec/common/doc/building.html “msys2 and the new Windows Subsystem for Linux (WSL) would likely be possible to support in a future version but that would require a community effort to implement”

I’d like to help making the OpenJDK 9 build working on msys2. What is the best way to move forward? Is there a similar effort in progress?

Thank you and best regards,

Peter


Reply | Threaded
Open this post in threaded view
|

Re: Building OpenJDK9 on MSYS2

Magnus Ihse Bursie
I gave msys2 a shot some time ago, but it ended up too much trouble.
I'll share some of my notes from that attempt, for what it's worth.

To install package X/Y, run "pacman -S X/Y". Missing tools and packages
where to find them:
cmp: msys/diffutils
tar: msys/tar
make: msys/make
unzip: msys/unzip
zip: msys/zip

config.sub reports msys as "x86_64-pc-mingw32" but msys2 as
"x86_64-pc-msys". This patch adds postprocessing in "our" config.sub to
report msys2 similar to msys. (Opinions, including my own :-) may vary
if this really is the best way..)

diff -r b88023f46daa common/autoconf/build-aux/config.sub
--- a/common/autoconf/build-aux/config.sub      Fri Jan 27 10:15:41 2017
+0100
+++ b/common/autoconf/build-aux/config.sub      Fri Feb 03 05:00:25 2017
-0700
@@ -30,7 +30,7 @@
  DIR=`dirname $0`

  # First, filter out everything that doesn't begin with "aarch64-"
-if ! echo $* | grep '^aarch64-' >/dev/null ; then
+if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ; then
      . $DIR/autoconf-config.sub "$@"
      # autoconf-config.sub exits, so we never reach here, but just in
      # case we do:
@@ -45,6 +45,10 @@
              config=`echo $1 | sed 's/^aarch64-/arm-/'`
              sub_args="$sub_args $config"
              shift; ;;
+        *-msys )
+            config=`echo $1 | sed 's/msys/mingw32/'`
+            sub_args="$sub_args $config"
+            shift; ;;
          - )    # Use stdin as input.
              sub_args="$sub_args $1"
              shift; break ;;

If I remember correctly, this got me past the configure stage at the time.

I don't think it's very hard to get it to work on msys2, I just ran into
one snag too many and didn't think msys2 would be used by anyone.

/Magnus

On 2017-10-03 17:20, Peter Budai wrote:

> Hello,
>
> According to http://hg.openjdk.java.net/jdk9/jdk9/file/a08cbfc0e4ec/common/doc/building.html “msys2 and the new Windows Subsystem for Linux (WSL) would likely be possible to support in a future version but that would require a community effort to implement”
>
> I’d like to help making the OpenJDK 9 build working on msys2. What is the best way to move forward? Is there a similar effort in progress?
>
> Thank you and best regards,
>
> Peter
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Building OpenJDK9 on MSYS2

Magnus Ihse Bursie
Actually, it wasn't so much remaining trouble. :-) I fired up msys2 and
checked out where I left off. It turned out that the remaining snag was
that msys2 tries to convert command lines automatically, from "unix"
style paths to "windows" style paths. Unfortunately, it does not do this
very well and it breaks all sorts of things. We already have a FIXPATH
solution in place which deals with this, so basically all I had to do
was disable this (by setting MSYS2_ARG_CONV_EXCL to "*"). However, this
broke our cygpath replacement hack (!) so I had to disable it there.
Sigh. Anyway, with those fixes it ran and worked well. (I also
discovered and fixed a bug related to how we set up the FIXPATH variable
on msys, but it only triggers in certain circumstances).

With this patch I now jdk9 seems to build fine on msys2. It should apply
cleanly on jdk9/jdk9. Since it turned out to be so trivial, I'll try to
get it in in jdk10.

Here's the patch if you want to apply it yourself:

diff -r a08cbfc0e4ec common/autoconf/basics_windows.m4
--- a/common/autoconf/basics_windows.m4    Thu Aug 03 18:56:56 2017 +0000
+++ b/common/autoconf/basics_windows.m4    Wed Oct 04 00:53:58 2017 +0200
@@ -42,7 +42,7 @@
      windows_path=`$CYGPATH -m "$unix_path"`
      $1="$windows_path"
    elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then
-    windows_path=`cmd //c echo $unix_path`
+    windows_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $unix_path`
      $1="$windows_path"
    fi
  ])
@@ -136,6 +136,16 @@
    fi
  ])

+AC_DEFUN([BASIC_MSYS_UPDATE_FIXPATH],
+[
+  # Take all collected prefixes and turn them into a
-m/c/foo@/c/bar@... command line
+  # @ was chosen as separator to minimize risk of other tools messing
around with it
+  all_unique_prefixes=`echo "${all_fixpath_prefixes@<:@@@:>@}" \
+      | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
+  fixpath_argument_list=`echo $all_unique_prefixes  | tr ' ' '@'`
+  FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
+])
+
  AC_DEFUN([BASIC_FIXUP_PATH_MSYS],
  [
    path="[$]$1"
@@ -143,7 +153,7 @@
    new_path="$path"
    if test "x$has_colon" = x; then
      # Not in mixed or Windows style, start by that.
-    new_path=`cmd //c echo $path`
+    new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $path`
    fi

    BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
@@ -155,6 +165,8 @@

    # Save the first 10 bytes of this path to the storage, so fixpath
can work.
    all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
"${new_path:0:10}")
+  # We might need to re-evaluate FIXPATH.
+  BASIC_MSYS_UPDATE_FIXPATH
  ])

  AC_DEFUN([BASIC_FIXUP_EXECUTABLE_CYGWIN],
@@ -293,7 +305,7 @@
      # Do not save /bin paths to all_fixpath_prefixes!
    else
      # Not in mixed or Windows style, start by that.
-    new_path=`cmd //c echo $new_path`
+    new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $new_path`
      BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
      # Output is in $new_path
      BASIC_WINDOWS_REWRITE_AS_UNIX_PATH(new_path)
@@ -302,6 +314,8 @@

      # Save the first 10 bytes of this path to the storage, so fixpath
can work.
      all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
"${new_path:0:10}")
+    # We might need to re-evaluate FIXPATH.
+    BASIC_MSYS_UPDATE_FIXPATH
    fi
  ])

@@ -347,6 +361,10 @@
      WINDOWS_ENV_VENDOR='msys'
      WINDOWS_ENV_VERSION="$MSYS_VERSION"

+    # Prohibit msys2 path conversion from trying to be "intelligent",
and rely
+    # on fixpath instead.
+    export MSYS2_ARG_CONV_EXCL="*"
+
      AC_MSG_CHECKING([msys root directory as unix-style path])
      # The cmd output ends with Windows line endings (CR/LF), the grep
command will strip that away
      MSYS_ROOT_PATH=`cd / ; cmd /c cd | $GREP ".*"`
@@ -391,10 +409,7 @@
      elif test "x$OPENJDK_BUILD_OS_ENV" = xwindows.msys; then
        # Take all collected prefixes and turn them into a
-m/c/foo@/c/bar@... command line
        # @ was chosen as separator to minimize risk of other tools
messing around with it
-      all_unique_prefixes=`echo "${all_fixpath_prefixes@<:@@@:>@}" \
-          | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
-      fixpath_argument_list=`echo $all_unique_prefixes  | tr ' ' '@'`
-      FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
+      BASIC_MSYS_UPDATE_FIXPATH
      fi
      FIXPATH_SRC_W="$FIXPATH_SRC"
      FIXPATH_BIN_W="$FIXPATH_BIN"
diff -r a08cbfc0e4ec common/autoconf/build-aux/config.sub
--- a/common/autoconf/build-aux/config.sub    Thu Aug 03 18:56:56 2017 +0000
+++ b/common/autoconf/build-aux/config.sub    Wed Oct 04 00:53:58 2017 +0200
@@ -30,7 +30,7 @@
  DIR=`dirname $0`

  # First, filter out everything that doesn't begin with "aarch64-"
-if ! echo $* | grep '^aarch64-' >/dev/null ; then
+if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ; then
      . $DIR/autoconf-config.sub "$@"
      # autoconf-config.sub exits, so we never reach here, but just in
      # case we do:
@@ -38,13 +38,17 @@
  fi

  while test $# -gt 0 ; do
-    case $1 in
+    case $1 in
          -- )   # Stop option processing
              shift; break ;;
          aarch64-* )
              config=`echo $1 | sed 's/^aarch64-/arm-/'`
              sub_args="$sub_args $config"
              shift; ;;
+        *-msys )
+            config=`echo $1 | sed 's/msys/mingw32/'`
+            sub_args="$sub_args $config"
+            shift; ;;
          - )    # Use stdin as input.
              sub_args="$sub_args $1"
              shift; break ;;
diff -r a08cbfc0e4ec common/autoconf/spec.gmk.in
--- a/common/autoconf/spec.gmk.in    Thu Aug 03 18:56:56 2017 +0000
+++ b/common/autoconf/spec.gmk.in    Wed Oct 04 00:53:58 2017 +0200
@@ -120,6 +120,13 @@
    # On Windows, the Visual Studio toolchain needs the PATH to be adjusted
    # to include Visual Studio tools (this needs to be in cygwin/msys
style).
    export PATH:=@VS_PATH@
+
+endif
+
+ifeq ($(OPENJDK_TARGET_OS_ENV), windows.msys)
+  # On msys2, prohibit msys path conversion from trying to be
+  # "intelligent", and rely on fixpath instead.
+  export MSYS2_ARG_CONV_EXCL:=*
  endif

  SYSROOT_CFLAGS := @SYSROOT_CFLAGS@

/Magnus

On 2017-10-03 22:34, Magnus Ihse Bursie wrote:

> I gave msys2 a shot some time ago, but it ended up too much trouble.
> I'll share some of my notes from that attempt, for what it's worth.
>
> To install package X/Y, run "pacman -S X/Y". Missing tools and
> packages where to find them:
> cmp: msys/diffutils
> tar: msys/tar
> make: msys/make
> unzip: msys/unzip
> zip: msys/zip
>
> config.sub reports msys as "x86_64-pc-mingw32" but msys2 as
> "x86_64-pc-msys". This patch adds postprocessing in "our" config.sub
> to report msys2 similar to msys. (Opinions, including my own :-) may
> vary if this really is the best way..)
>
> diff -r b88023f46daa common/autoconf/build-aux/config.sub
> --- a/common/autoconf/build-aux/config.sub      Fri Jan 27 10:15:41
> 2017 +0100
> +++ b/common/autoconf/build-aux/config.sub      Fri Feb 03 05:00:25
> 2017 -0700
> @@ -30,7 +30,7 @@
>  DIR=`dirname $0`
>
>  # First, filter out everything that doesn't begin with "aarch64-"
> -if ! echo $* | grep '^aarch64-' >/dev/null ; then
> +if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ; then
>      . $DIR/autoconf-config.sub "$@"
>      # autoconf-config.sub exits, so we never reach here, but just in
>      # case we do:
> @@ -45,6 +45,10 @@
>              config=`echo $1 | sed 's/^aarch64-/arm-/'`
>              sub_args="$sub_args $config"
>              shift; ;;
> +        *-msys )
> +            config=`echo $1 | sed 's/msys/mingw32/'`
> +            sub_args="$sub_args $config"
> +            shift; ;;
>          - )    # Use stdin as input.
>              sub_args="$sub_args $1"
>              shift; break ;;
>
> If I remember correctly, this got me past the configure stage at the
> time.
>
> I don't think it's very hard to get it to work on msys2, I just ran
> into one snag too many and didn't think msys2 would be used by anyone.
>
> /Magnus
>
> On 2017-10-03 17:20, Peter Budai wrote:
>> Hello,
>>
>> According to
>> http://hg.openjdk.java.net/jdk9/jdk9/file/a08cbfc0e4ec/common/doc/building.html 
>> “msys2 and the new Windows Subsystem for Linux (WSL) would likely be
>> possible to support in a future version but that would require a
>> community effort to implement”
>>
>> I’d like to help making the OpenJDK 9 build working on msys2. What is
>> the best way to move forward? Is there a similar effort in progress?
>>
>> Thank you and best regards,
>>
>> Peter
>>
>>
>

Reply | Threaded
Open this post in threaded view
|

RE: Building OpenJDK9 on MSYS2

Peter Budai
Hi Magnus,

Thanks for the quick reply I’ll check these patches with msys2.

Let me specify with more details what I’d like to achieve: I’d like to build OpenJDK9 with MSYS2 MINGW64 environment using gcc toolchain. (I’m not sure how familiar are you with MSYS2, but there are 3 different environments: MSYS2, MINGW32 and MINGW64). In theory MINGW64 with gcc is the closes you can get on Windows platform as a gcc unix like build environment, which produces still a native 64-bit executable on Windows.

I’m not very familiar with OpenJDK yet, so therefore I’d like to hear your opinion: how realistic is that?

As a side note: with MINGW64 I have managed to run configure phase successfully for OpenJDK. The compile process has also started and went for a while, but interestingly I run into some kind of race conditions as make stopped with an error. Using LOG=debug I have fond the failing line and then copying the failed command and pasting it to the bash prompt it successfully generated the output target, and then the build process run further when a similar situation happened. Also pasting the failed command run in the bash without any problem, and build continued… until the next.

I have tried to run make JOBS=1, but did not help, strangely I have still seen in the log make[3] and make[4] logs which suggested that there are more than one make jobs were running. Also tried .configure --with-output-sync=recurse without success (same symptoms)

Let me know your thoughts.

Best regards,

Peter

Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10

From: Magnus Ihse Bursie<mailto:[hidden email]>
Sent: Wednesday, October 4, 2017 1:04 AM
To: Peter Budai<mailto:[hidden email]>; [hidden email]<mailto:[hidden email]>
Subject: Re: Building OpenJDK9 on MSYS2

Actually, it wasn't so much remaining trouble. :-) I fired up msys2 and
checked out where I left off. It turned out that the remaining snag was
that msys2 tries to convert command lines automatically, from "unix"
style paths to "windows" style paths. Unfortunately, it does not do this
very well and it breaks all sorts of things. We already have a FIXPATH
solution in place which deals with this, so basically all I had to do
was disable this (by setting MSYS2_ARG_CONV_EXCL to "*"). However, this
broke our cygpath replacement hack (!) so I had to disable it there.
Sigh. Anyway, with those fixes it ran and worked well. (I also
discovered and fixed a bug related to how we set up the FIXPATH variable
on msys, but it only triggers in certain circumstances).

With this patch I now jdk9 seems to build fine on msys2. It should apply
cleanly on jdk9/jdk9. Since it turned out to be so trivial, I'll try to
get it in in jdk10.

Here's the patch if you want to apply it yourself:

diff -r a08cbfc0e4ec common/autoconf/basics_windows.m4
--- a/common/autoconf/basics_windows.m4    Thu Aug 03 18:56:56 2017 +0000
+++ b/common/autoconf/basics_windows.m4    Wed Oct 04 00:53:58 2017 +0200
@@ -42,7 +42,7 @@
      windows_path=`$CYGPATH -m "$unix_path"`
      $1="$windows_path"
    elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then
-    windows_path=`cmd //c echo $unix_path`
+    windows_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $unix_path`
      $1="$windows_path"
    fi
  ])
@@ -136,6 +136,16 @@
    fi
  ])

+AC_DEFUN([BASIC_MSYS_UPDATE_FIXPATH],
+[
+  # Take all collected prefixes and turn them into a
-m/c/foo@/c/bar@... command line
+  # @ was chosen as separator to minimize risk of other tools messing
around with it
+  all_unique_prefixes=`echo "${all_fixpath_prefixes@<:@@@:>@}" \
+      | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
+  fixpath_argument_list=`echo $all_unique_prefixes  | tr ' ' '@'`
+  FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
+])
+
  AC_DEFUN([BASIC_FIXUP_PATH_MSYS],
  [
    path="[$]$1"
@@ -143,7 +153,7 @@
    new_path="$path"
    if test "x$has_colon" = x; then
      # Not in mixed or Windows style, start by that.
-    new_path=`cmd //c echo $path`
+    new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $path`
    fi

    BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
@@ -155,6 +165,8 @@

    # Save the first 10 bytes of this path to the storage, so fixpath
can work.
    all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
"${new_path:0:10}")
+  # We might need to re-evaluate FIXPATH.
+  BASIC_MSYS_UPDATE_FIXPATH
  ])

  AC_DEFUN([BASIC_FIXUP_EXECUTABLE_CYGWIN],
@@ -293,7 +305,7 @@
      # Do not save /bin paths to all_fixpath_prefixes!
    else
      # Not in mixed or Windows style, start by that.
-    new_path=`cmd //c echo $new_path`
+    new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $new_path`
      BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
      # Output is in $new_path
      BASIC_WINDOWS_REWRITE_AS_UNIX_PATH(new_path)
@@ -302,6 +314,8 @@

      # Save the first 10 bytes of this path to the storage, so fixpath
can work.
      all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
"${new_path:0:10}")
+    # We might need to re-evaluate FIXPATH.
+    BASIC_MSYS_UPDATE_FIXPATH
    fi
  ])

@@ -347,6 +361,10 @@
      WINDOWS_ENV_VENDOR='msys'
      WINDOWS_ENV_VERSION="$MSYS_VERSION"

+    # Prohibit msys2 path conversion from trying to be "intelligent",
and rely
+    # on fixpath instead.
+    export MSYS2_ARG_CONV_EXCL="*"
+
      AC_MSG_CHECKING([msys root directory as unix-style path])
      # The cmd output ends with Windows line endings (CR/LF), the grep
command will strip that away
      MSYS_ROOT_PATH=`cd / ; cmd /c cd | $GREP ".*"`
@@ -391,10 +409,7 @@
      elif test "x$OPENJDK_BUILD_OS_ENV" = xwindows.msys; then
        # Take all collected prefixes and turn them into a
-m/c/foo@/c/bar@... command line
        # @ was chosen as separator to minimize risk of other tools
messing around with it
-      all_unique_prefixes=`echo "${all_fixpath_prefixes@<:@@@:>@}" \
-          | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
-      fixpath_argument_list=`echo $all_unique_prefixes  | tr ' ' '@'`
-      FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
+      BASIC_MSYS_UPDATE_FIXPATH
      fi
      FIXPATH_SRC_W="$FIXPATH_SRC"
      FIXPATH_BIN_W="$FIXPATH_BIN"
diff -r a08cbfc0e4ec common/autoconf/build-aux/config.sub
--- a/common/autoconf/build-aux/config.sub    Thu Aug 03 18:56:56 2017 +0000
+++ b/common/autoconf/build-aux/config.sub    Wed Oct 04 00:53:58 2017 +0200
@@ -30,7 +30,7 @@
  DIR=`dirname $0`

  # First, filter out everything that doesn't begin with "aarch64-"
-if ! echo $* | grep '^aarch64-' >/dev/null ; then
+if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ; then
      . $DIR/autoconf-config.sub "$@"
      # autoconf-config.sub exits, so we never reach here, but just in
      # case we do:
@@ -38,13 +38,17 @@
  fi

  while test $# -gt 0 ; do
-    case $1 in
+    case $1 in
          -- )   # Stop option processing
              shift; break ;;
          aarch64-* )
              config=`echo $1 | sed 's/^aarch64-/arm-/'`
              sub_args="$sub_args $config"
              shift; ;;
+        *-msys )
+            config=`echo $1 | sed 's/msys/mingw32/'`
+            sub_args="$sub_args $config"
+            shift; ;;
          - )    # Use stdin as input.
              sub_args="$sub_args $1"
              shift; break ;;
diff -r a08cbfc0e4ec common/autoconf/spec.gmk.in
--- a/common/autoconf/spec.gmk.in    Thu Aug 03 18:56:56 2017 +0000
+++ b/common/autoconf/spec.gmk.in    Wed Oct 04 00:53:58 2017 +0200
@@ -120,6 +120,13 @@
    # On Windows, the Visual Studio toolchain needs the PATH to be adjusted
    # to include Visual Studio tools (this needs to be in cygwin/msys
style).
    export PATH:=@VS_PATH@
+
+endif
+
+ifeq ($(OPENJDK_TARGET_OS_ENV), windows.msys)
+  # On msys2, prohibit msys path conversion from trying to be
+  # "intelligent", and rely on fixpath instead.
+  export MSYS2_ARG_CONV_EXCL:=*
  endif

  SYSROOT_CFLAGS := @SYSROOT_CFLAGS@

/Magnus

On 2017-10-03 22:34, Magnus Ihse Bursie wrote:

> I gave msys2 a shot some time ago, but it ended up too much trouble.
> I'll share some of my notes from that attempt, for what it's worth.
>
> To install package X/Y, run "pacman -S X/Y". Missing tools and
> packages where to find them:
> cmp: msys/diffutils
> tar: msys/tar
> make: msys/make
> unzip: msys/unzip
> zip: msys/zip
>
> config.sub reports msys as "x86_64-pc-mingw32" but msys2 as
> "x86_64-pc-msys". This patch adds postprocessing in "our" config.sub
> to report msys2 similar to msys. (Opinions, including my own :-) may
> vary if this really is the best way..)
>
> diff -r b88023f46daa common/autoconf/build-aux/config.sub
> --- a/common/autoconf/build-aux/config.sub      Fri Jan 27 10:15:41
> 2017 +0100
> +++ b/common/autoconf/build-aux/config.sub      Fri Feb 03 05:00:25
> 2017 -0700
> @@ -30,7 +30,7 @@
>  DIR=`dirname $0`
>
>  # First, filter out everything that doesn't begin with "aarch64-"
> -if ! echo $* | grep '^aarch64-' >/dev/null ; then
> +if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ; then
>      . $DIR/autoconf-config.sub "$@"
>      # autoconf-config.sub exits, so we never reach here, but just in
>      # case we do:
> @@ -45,6 +45,10 @@
>              config=`echo $1 | sed 's/^aarch64-/arm-/'`
>              sub_args="$sub_args $config"
>              shift; ;;
> +        *-msys )
> +            config=`echo $1 | sed 's/msys/mingw32/'`
> +            sub_args="$sub_args $config"
> +            shift; ;;
>          - )    # Use stdin as input.
>              sub_args="$sub_args $1"
>              shift; break ;;
>
> If I remember correctly, this got me past the configure stage at the
> time.
>
> I don't think it's very hard to get it to work on msys2, I just ran
> into one snag too many and didn't think msys2 would be used by anyone.
>
> /Magnus
>
> On 2017-10-03 17:20, Peter Budai wrote:
>> Hello,
>>
>> According to
>> http://hg.openjdk.java.net/jdk9/jdk9/file/a08cbfc0e4ec/common/doc/building.html
>> “msys2 and the new Windows Subsystem for Linux (WSL) would likely be
>> possible to support in a future version but that would require a
>> community effort to implement”
>>
>> I’d like to help making the OpenJDK 9 build working on msys2. What is
>> the best way to move forward? Is there a similar effort in progress?
>>
>> Thank you and best regards,
>>
>> Peter
>>
>>
>

Reply | Threaded
Open this post in threaded view
|

Re: Building OpenJDK9 on MSYS2

Erik Joelsson
Hello Peter,


On 2017-10-04 21:15, Peter Budai wrote:
> Hi Magnus,
>
> Thanks for the quick reply I’ll check these patches with msys2.
>
> Let me specify with more details what I’d like to achieve: I’d like to build OpenJDK9 with MSYS2 MINGW64 environment using gcc toolchain. (I’m not sure how familiar are you with MSYS2, but there are 3 different environments: MSYS2, MINGW32 and MINGW64). In theory MINGW64 with gcc is the closes you can get on Windows platform as a gcc unix like build environment, which produces still a native 64-bit executable on Windows.
>
> I’m not very familiar with OpenJDK yet, so therefore I’d like to hear your opinion: how realistic is that?
Sorry to disappoint, but I would say that requires major work. There is
a strong historic assumption that windows builds are done using Visual
Studio. We have abstracted away some of it in configure (see
TOOLCHAIN_TYPE), but it's very far from enough to change compiler
environment for a Windows build. The native sources are also bound to
make a lot of such assumptions. I would expect the changes needed to be
in the thousands of lines of code.

When we say supporting the build in msys2 instead of cygwin, we just
mean using msys2 as the unix emulating layer for our tools like
make/bash/grep/sed etc.

One think I have done successfully is running the build in WSL (Windows
Subsystem for Linux), but that isn't all that helpful as WSL for
practical purposes is more or less like running Linux in a VM, so the
build sees a Linux system and builds a Linux binary.
> As a side note: with MINGW64 I have managed to run configure phase successfully for OpenJDK. The compile process has also started and went for a while, but interestingly I run into some kind of race conditions as make stopped with an error. Using LOG=debug I have fond the failing line and then copying the failed command and pasting it to the bash prompt it successfully generated the output target, and then the build process run further when a similar situation happened. Also pasting the failed command run in the bash without any problem, and build continued… until the next.
Without seeing the errors I can't say much. I very much doubt that you
are running with gcc as the compiler though. Configure isn't easily
fooled into using a different compiler to what it prefers, and I would
expect things to crash and burn pretty early if you actually did.

/Erik

>
> I have tried to run make JOBS=1, but did not help, strangely I have still seen in the log make[3] and make[4] logs which suggested that there are more than one make jobs were running. Also tried .configure --with-output-sync=recurse without success (same symptoms)
>
> Let me know your thoughts.
>
> Best regards,
>
> Peter
>
> Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10
>
> From: Magnus Ihse Bursie<mailto:[hidden email]>
> Sent: Wednesday, October 4, 2017 1:04 AM
> To: Peter Budai<mailto:[hidden email]>; [hidden email]<mailto:[hidden email]>
> Subject: Re: Building OpenJDK9 on MSYS2
>
> Actually, it wasn't so much remaining trouble. :-) I fired up msys2 and
> checked out where I left off. It turned out that the remaining snag was
> that msys2 tries to convert command lines automatically, from "unix"
> style paths to "windows" style paths. Unfortunately, it does not do this
> very well and it breaks all sorts of things. We already have a FIXPATH
> solution in place which deals with this, so basically all I had to do
> was disable this (by setting MSYS2_ARG_CONV_EXCL to "*"). However, this
> broke our cygpath replacement hack (!) so I had to disable it there.
> Sigh. Anyway, with those fixes it ran and worked well. (I also
> discovered and fixed a bug related to how we set up the FIXPATH variable
> on msys, but it only triggers in certain circumstances).
>
> With this patch I now jdk9 seems to build fine on msys2. It should apply
> cleanly on jdk9/jdk9. Since it turned out to be so trivial, I'll try to
> get it in in jdk10.
>
> Here's the patch if you want to apply it yourself:
>
> diff -r a08cbfc0e4ec common/autoconf/basics_windows.m4
> --- a/common/autoconf/basics_windows.m4    Thu Aug 03 18:56:56 2017 +0000
> +++ b/common/autoconf/basics_windows.m4    Wed Oct 04 00:53:58 2017 +0200
> @@ -42,7 +42,7 @@
>        windows_path=`$CYGPATH -m "$unix_path"`
>        $1="$windows_path"
>      elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then
> -    windows_path=`cmd //c echo $unix_path`
> +    windows_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $unix_path`
>        $1="$windows_path"
>      fi
>    ])
> @@ -136,6 +136,16 @@
>      fi
>    ])
>
> +AC_DEFUN([BASIC_MSYS_UPDATE_FIXPATH],
> +[
> +  # Take all collected prefixes and turn them into a
> -m/c/foo@/c/bar@... command line
> +  # @ was chosen as separator to minimize risk of other tools messing
> around with it
> +  all_unique_prefixes=`echo "${all_fixpath_prefixes@<:@@@:>@}" \
> +      | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
> +  fixpath_argument_list=`echo $all_unique_prefixes  | tr ' ' '@'`
> +  FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
> +])
> +
>    AC_DEFUN([BASIC_FIXUP_PATH_MSYS],
>    [
>      path="[$]$1"
> @@ -143,7 +153,7 @@
>      new_path="$path"
>      if test "x$has_colon" = x; then
>        # Not in mixed or Windows style, start by that.
> -    new_path=`cmd //c echo $path`
> +    new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $path`
>      fi
>
>      BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
> @@ -155,6 +165,8 @@
>
>      # Save the first 10 bytes of this path to the storage, so fixpath
> can work.
>      all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
> "${new_path:0:10}")
> +  # We might need to re-evaluate FIXPATH.
> +  BASIC_MSYS_UPDATE_FIXPATH
>    ])
>
>    AC_DEFUN([BASIC_FIXUP_EXECUTABLE_CYGWIN],
> @@ -293,7 +305,7 @@
>        # Do not save /bin paths to all_fixpath_prefixes!
>      else
>        # Not in mixed or Windows style, start by that.
> -    new_path=`cmd //c echo $new_path`
> +    new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $new_path`
>        BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
>        # Output is in $new_path
>        BASIC_WINDOWS_REWRITE_AS_UNIX_PATH(new_path)
> @@ -302,6 +314,8 @@
>
>        # Save the first 10 bytes of this path to the storage, so fixpath
> can work.
>        all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
> "${new_path:0:10}")
> +    # We might need to re-evaluate FIXPATH.
> +    BASIC_MSYS_UPDATE_FIXPATH
>      fi
>    ])
>
> @@ -347,6 +361,10 @@
>        WINDOWS_ENV_VENDOR='msys'
>        WINDOWS_ENV_VERSION="$MSYS_VERSION"
>
> +    # Prohibit msys2 path conversion from trying to be "intelligent",
> and rely
> +    # on fixpath instead.
> +    export MSYS2_ARG_CONV_EXCL="*"
> +
>        AC_MSG_CHECKING([msys root directory as unix-style path])
>        # The cmd output ends with Windows line endings (CR/LF), the grep
> command will strip that away
>        MSYS_ROOT_PATH=`cd / ; cmd /c cd | $GREP ".*"`
> @@ -391,10 +409,7 @@
>        elif test "x$OPENJDK_BUILD_OS_ENV" = xwindows.msys; then
>          # Take all collected prefixes and turn them into a
> -m/c/foo@/c/bar@... command line
>          # @ was chosen as separator to minimize risk of other tools
> messing around with it
> -      all_unique_prefixes=`echo "${all_fixpath_prefixes@<:@@@:>@}" \
> -          | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
> -      fixpath_argument_list=`echo $all_unique_prefixes  | tr ' ' '@'`
> -      FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
> +      BASIC_MSYS_UPDATE_FIXPATH
>        fi
>        FIXPATH_SRC_W="$FIXPATH_SRC"
>        FIXPATH_BIN_W="$FIXPATH_BIN"
> diff -r a08cbfc0e4ec common/autoconf/build-aux/config.sub
> --- a/common/autoconf/build-aux/config.sub    Thu Aug 03 18:56:56 2017 +0000
> +++ b/common/autoconf/build-aux/config.sub    Wed Oct 04 00:53:58 2017 +0200
> @@ -30,7 +30,7 @@
>    DIR=`dirname $0`
>
>    # First, filter out everything that doesn't begin with "aarch64-"
> -if ! echo $* | grep '^aarch64-' >/dev/null ; then
> +if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ; then
>        . $DIR/autoconf-config.sub "$@"
>        # autoconf-config.sub exits, so we never reach here, but just in
>        # case we do:
> @@ -38,13 +38,17 @@
>    fi
>
>    while test $# -gt 0 ; do
> -    case $1 in
> +    case $1 in
>            -- )   # Stop option processing
>                shift; break ;;
>            aarch64-* )
>                config=`echo $1 | sed 's/^aarch64-/arm-/'`
>                sub_args="$sub_args $config"
>                shift; ;;
> +        *-msys )
> +            config=`echo $1 | sed 's/msys/mingw32/'`
> +            sub_args="$sub_args $config"
> +            shift; ;;
>            - )    # Use stdin as input.
>                sub_args="$sub_args $1"
>                shift; break ;;
> diff -r a08cbfc0e4ec common/autoconf/spec.gmk.in
> --- a/common/autoconf/spec.gmk.in    Thu Aug 03 18:56:56 2017 +0000
> +++ b/common/autoconf/spec.gmk.in    Wed Oct 04 00:53:58 2017 +0200
> @@ -120,6 +120,13 @@
>      # On Windows, the Visual Studio toolchain needs the PATH to be adjusted
>      # to include Visual Studio tools (this needs to be in cygwin/msys
> style).
>      export PATH:=@VS_PATH@
> +
> +endif
> +
> +ifeq ($(OPENJDK_TARGET_OS_ENV), windows.msys)
> +  # On msys2, prohibit msys path conversion from trying to be
> +  # "intelligent", and rely on fixpath instead.
> +  export MSYS2_ARG_CONV_EXCL:=*
>    endif
>
>    SYSROOT_CFLAGS := @SYSROOT_CFLAGS@
>
> /Magnus
>
> On 2017-10-03 22:34, Magnus Ihse Bursie wrote:
>> I gave msys2 a shot some time ago, but it ended up too much trouble.
>> I'll share some of my notes from that attempt, for what it's worth.
>>
>> To install package X/Y, run "pacman -S X/Y". Missing tools and
>> packages where to find them:
>> cmp: msys/diffutils
>> tar: msys/tar
>> make: msys/make
>> unzip: msys/unzip
>> zip: msys/zip
>>
>> config.sub reports msys as "x86_64-pc-mingw32" but msys2 as
>> "x86_64-pc-msys". This patch adds postprocessing in "our" config.sub
>> to report msys2 similar to msys. (Opinions, including my own :-) may
>> vary if this really is the best way..)
>>
>> diff -r b88023f46daa common/autoconf/build-aux/config.sub
>> --- a/common/autoconf/build-aux/config.sub      Fri Jan 27 10:15:41
>> 2017 +0100
>> +++ b/common/autoconf/build-aux/config.sub      Fri Feb 03 05:00:25
>> 2017 -0700
>> @@ -30,7 +30,7 @@
>>   DIR=`dirname $0`
>>
>>   # First, filter out everything that doesn't begin with "aarch64-"
>> -if ! echo $* | grep '^aarch64-' >/dev/null ; then
>> +if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ; then
>>       . $DIR/autoconf-config.sub "$@"
>>       # autoconf-config.sub exits, so we never reach here, but just in
>>       # case we do:
>> @@ -45,6 +45,10 @@
>>               config=`echo $1 | sed 's/^aarch64-/arm-/'`
>>               sub_args="$sub_args $config"
>>               shift; ;;
>> +        *-msys )
>> +            config=`echo $1 | sed 's/msys/mingw32/'`
>> +            sub_args="$sub_args $config"
>> +            shift; ;;
>>           - )    # Use stdin as input.
>>               sub_args="$sub_args $1"
>>               shift; break ;;
>>
>> If I remember correctly, this got me past the configure stage at the
>> time.
>>
>> I don't think it's very hard to get it to work on msys2, I just ran
>> into one snag too many and didn't think msys2 would be used by anyone.
>>
>> /Magnus
>>
>> On 2017-10-03 17:20, Peter Budai wrote:
>>> Hello,
>>>
>>> According to
>>> http://hg.openjdk.java.net/jdk9/jdk9/file/a08cbfc0e4ec/common/doc/building.html
>>> “msys2 and the new Windows Subsystem for Linux (WSL) would likely be
>>> possible to support in a future version but that would require a
>>> community effort to implement”
>>>
>>> I’d like to help making the OpenJDK 9 build working on msys2. What is
>>> the best way to move forward? Is there a similar effort in progress?
>>>
>>> Thank you and best regards,
>>>
>>> Peter
>>>
>>>

Reply | Threaded
Open this post in threaded view
|

Re: Building OpenJDK9 on MSYS2

Magnus Ihse Bursie
On 2017-10-05 10:10, Erik Joelsson wrote:

> Hello Peter,
>
>
> On 2017-10-04 21:15, Peter Budai wrote:
>> Hi Magnus,
>>
>> Thanks for the quick reply I’ll check these patches with msys2.
>>
>> Let me specify with more details what I’d like to achieve: I’d like
>> to build OpenJDK9 with MSYS2 MINGW64 environment using gcc toolchain.
>> (I’m not sure how familiar are you with MSYS2, but there are 3
>> different environments: MSYS2, MINGW32 and MINGW64). In theory
>> MINGW64 with gcc is the closes you can get on Windows platform as a
>> gcc unix like build environment, which produces still a native 64-bit
>> executable on Windows.
>>
>> I’m not very familiar with OpenJDK yet, so therefore I’d like to hear
>> your opinion: how realistic is that?
> Sorry to disappoint, but I would say that requires major work. There
> is a strong historic assumption that windows builds are done using
> Visual Studio. We have abstracted away some of it in configure (see
> TOOLCHAIN_TYPE), but it's very far from enough to change compiler
> environment for a Windows build. The native sources are also bound to
> make a lot of such assumptions. I would expect the changes needed to
> be in the thousands of lines of code.

I agree that it requires hard work (even if "thousands" might be an
overestimation I think, but "hundreds" is not enough, so it's the right
magnitude). On the other hand, it would be really good if we did sort
things out, so that we had proper conditions based on OS vs
compiler/toolchain.

If you really want to start, the first thing is to patch toolchain.m4 to
VALID_TOOLCHAINS_windows="microsoft gcc"
and then call configure using "bash configure --with-toolchain-type=gcc".

As Erik, I doubt you will come very far before things starts tumbling down.

>
> When we say supporting the build in msys2 instead of cygwin, we just
> mean using msys2 as the unix emulating layer for our tools like
> make/bash/grep/sed etc.
>
> One think I have done successfully is running the build in WSL
> (Windows Subsystem for Linux), but that isn't all that helpful as WSL
> for practical purposes is more or less like running Linux in a VM, so
> the build sees a Linux system and builds a Linux binary.
>> As a side note: with MINGW64 I have managed to run configure phase
>> successfully for OpenJDK. The compile process has also started and
>> went for a while, but interestingly I run into some kind of race
>> conditions as make stopped with an error. Using LOG=debug I have fond
>> the failing line and then copying the failed command and pasting it
>> to the bash prompt it successfully generated the output target, and
>> then the build process run further when a similar situation happened.
>> Also pasting the failed command run in the bash without any problem,
>> and build continued… until the next.
> Without seeing the errors I can't say much. I very much doubt that you
> are running with gcc as the compiler though. Configure isn't easily
> fooled into using a different compiler to what it prefers, and I would
> expect things to crash and burn pretty early if you actually did.
>
> /Erik
>>
>> I have tried to run make JOBS=1, but did not help, strangely I have
>> still seen in the log make[3] and make[4] logs which suggested that
>> there are more than one make jobs were running. Also tried .configure
>> --with-output-sync=recurse without success (same symptoms)
>>
>> Let me know your thoughts.
>>
>> Best regards,
>>
>> Peter
>>
>> Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for
>> Windows 10
>>
>> From: Magnus Ihse Bursie<mailto:[hidden email]>
>> Sent: Wednesday, October 4, 2017 1:04 AM
>> To: Peter Budai<mailto:[hidden email]>;
>> [hidden email]<mailto:[hidden email]>
>> Subject: Re: Building OpenJDK9 on MSYS2
>>
>> Actually, it wasn't so much remaining trouble. :-) I fired up msys2 and
>> checked out where I left off. It turned out that the remaining snag was
>> that msys2 tries to convert command lines automatically, from "unix"
>> style paths to "windows" style paths. Unfortunately, it does not do this
>> very well and it breaks all sorts of things. We already have a FIXPATH
>> solution in place which deals with this, so basically all I had to do
>> was disable this (by setting MSYS2_ARG_CONV_EXCL to "*"). However, this
>> broke our cygpath replacement hack (!) so I had to disable it there.
>> Sigh. Anyway, with those fixes it ran and worked well. (I also
>> discovered and fixed a bug related to how we set up the FIXPATH variable
>> on msys, but it only triggers in certain circumstances).
>>
>> With this patch I now jdk9 seems to build fine on msys2. It should apply
>> cleanly on jdk9/jdk9. Since it turned out to be so trivial, I'll try to
>> get it in in jdk10.
>>
>> Here's the patch if you want to apply it yourself:
>>
>> diff -r a08cbfc0e4ec common/autoconf/basics_windows.m4
>> --- a/common/autoconf/basics_windows.m4    Thu Aug 03 18:56:56 2017
>> +0000
>> +++ b/common/autoconf/basics_windows.m4    Wed Oct 04 00:53:58 2017
>> +0200
>> @@ -42,7 +42,7 @@
>>        windows_path=`$CYGPATH -m "$unix_path"`
>>        $1="$windows_path"
>>      elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then
>> -    windows_path=`cmd //c echo $unix_path`
>> +    windows_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $unix_path`
>>        $1="$windows_path"
>>      fi
>>    ])
>> @@ -136,6 +136,16 @@
>>      fi
>>    ])
>>
>> +AC_DEFUN([BASIC_MSYS_UPDATE_FIXPATH],
>> +[
>> +  # Take all collected prefixes and turn them into a
>> -m/c/foo@/c/bar@... command line
>> +  # @ was chosen as separator to minimize risk of other tools messing
>> around with it
>> +  all_unique_prefixes=`echo "${all_fixpath_prefixes@<:@@@:>@}" \
>> +      | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
>> +  fixpath_argument_list=`echo $all_unique_prefixes  | tr ' ' '@'`
>> +  FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
>> +])
>> +
>>    AC_DEFUN([BASIC_FIXUP_PATH_MSYS],
>>    [
>>      path="[$]$1"
>> @@ -143,7 +153,7 @@
>>      new_path="$path"
>>      if test "x$has_colon" = x; then
>>        # Not in mixed or Windows style, start by that.
>> -    new_path=`cmd //c echo $path`
>> +    new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $path`
>>      fi
>>
>>      BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
>> @@ -155,6 +165,8 @@
>>
>>      # Save the first 10 bytes of this path to the storage, so fixpath
>> can work.
>> all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
>> "${new_path:0:10}")
>> +  # We might need to re-evaluate FIXPATH.
>> +  BASIC_MSYS_UPDATE_FIXPATH
>>    ])
>>
>>    AC_DEFUN([BASIC_FIXUP_EXECUTABLE_CYGWIN],
>> @@ -293,7 +305,7 @@
>>        # Do not save /bin paths to all_fixpath_prefixes!
>>      else
>>        # Not in mixed or Windows style, start by that.
>> -    new_path=`cmd //c echo $new_path`
>> +    new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $new_path`
>>        BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
>>        # Output is in $new_path
>>        BASIC_WINDOWS_REWRITE_AS_UNIX_PATH(new_path)
>> @@ -302,6 +314,8 @@
>>
>>        # Save the first 10 bytes of this path to the storage, so fixpath
>> can work.
>> all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
>> "${new_path:0:10}")
>> +    # We might need to re-evaluate FIXPATH.
>> +    BASIC_MSYS_UPDATE_FIXPATH
>>      fi
>>    ])
>>
>> @@ -347,6 +361,10 @@
>>        WINDOWS_ENV_VENDOR='msys'
>>        WINDOWS_ENV_VERSION="$MSYS_VERSION"
>>
>> +    # Prohibit msys2 path conversion from trying to be "intelligent",
>> and rely
>> +    # on fixpath instead.
>> +    export MSYS2_ARG_CONV_EXCL="*"
>> +
>>        AC_MSG_CHECKING([msys root directory as unix-style path])
>>        # The cmd output ends with Windows line endings (CR/LF), the grep
>> command will strip that away
>>        MSYS_ROOT_PATH=`cd / ; cmd /c cd | $GREP ".*"`
>> @@ -391,10 +409,7 @@
>>        elif test "x$OPENJDK_BUILD_OS_ENV" = xwindows.msys; then
>>          # Take all collected prefixes and turn them into a
>> -m/c/foo@/c/bar@... command line
>>          # @ was chosen as separator to minimize risk of other tools
>> messing around with it
>> -      all_unique_prefixes=`echo "${all_fixpath_prefixes@<:@@@:>@}" \
>> -          | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
>> -      fixpath_argument_list=`echo $all_unique_prefixes  | tr ' ' '@'`
>> -      FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
>> +      BASIC_MSYS_UPDATE_FIXPATH
>>        fi
>>        FIXPATH_SRC_W="$FIXPATH_SRC"
>>        FIXPATH_BIN_W="$FIXPATH_BIN"
>> diff -r a08cbfc0e4ec common/autoconf/build-aux/config.sub
>> --- a/common/autoconf/build-aux/config.sub    Thu Aug 03 18:56:56
>> 2017 +0000
>> +++ b/common/autoconf/build-aux/config.sub    Wed Oct 04 00:53:58
>> 2017 +0200
>> @@ -30,7 +30,7 @@
>>    DIR=`dirname $0`
>>
>>    # First, filter out everything that doesn't begin with "aarch64-"
>> -if ! echo $* | grep '^aarch64-' >/dev/null ; then
>> +if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ; then
>>        . $DIR/autoconf-config.sub "$@"
>>        # autoconf-config.sub exits, so we never reach here, but just in
>>        # case we do:
>> @@ -38,13 +38,17 @@
>>    fi
>>
>>    while test $# -gt 0 ; do
>> -    case $1 in
>> +    case $1 in
>>            -- )   # Stop option processing
>>                shift; break ;;
>>            aarch64-* )
>>                config=`echo $1 | sed 's/^aarch64-/arm-/'`
>>                sub_args="$sub_args $config"
>>                shift; ;;
>> +        *-msys )
>> +            config=`echo $1 | sed 's/msys/mingw32/'`
>> +            sub_args="$sub_args $config"
>> +            shift; ;;
>>            - )    # Use stdin as input.
>>                sub_args="$sub_args $1"
>>                shift; break ;;
>> diff -r a08cbfc0e4ec common/autoconf/spec.gmk.in
>> --- a/common/autoconf/spec.gmk.in    Thu Aug 03 18:56:56 2017 +0000
>> +++ b/common/autoconf/spec.gmk.in    Wed Oct 04 00:53:58 2017 +0200
>> @@ -120,6 +120,13 @@
>>      # On Windows, the Visual Studio toolchain needs the PATH to be
>> adjusted
>>      # to include Visual Studio tools (this needs to be in cygwin/msys
>> style).
>>      export PATH:=@VS_PATH@
>> +
>> +endif
>> +
>> +ifeq ($(OPENJDK_TARGET_OS_ENV), windows.msys)
>> +  # On msys2, prohibit msys path conversion from trying to be
>> +  # "intelligent", and rely on fixpath instead.
>> +  export MSYS2_ARG_CONV_EXCL:=*
>>    endif
>>
>>    SYSROOT_CFLAGS := @SYSROOT_CFLAGS@
>>
>> /Magnus
>>
>> On 2017-10-03 22:34, Magnus Ihse Bursie wrote:
>>> I gave msys2 a shot some time ago, but it ended up too much trouble.
>>> I'll share some of my notes from that attempt, for what it's worth.
>>>
>>> To install package X/Y, run "pacman -S X/Y". Missing tools and
>>> packages where to find them:
>>> cmp: msys/diffutils
>>> tar: msys/tar
>>> make: msys/make
>>> unzip: msys/unzip
>>> zip: msys/zip
>>>
>>> config.sub reports msys as "x86_64-pc-mingw32" but msys2 as
>>> "x86_64-pc-msys". This patch adds postprocessing in "our" config.sub
>>> to report msys2 similar to msys. (Opinions, including my own :-) may
>>> vary if this really is the best way..)
>>>
>>> diff -r b88023f46daa common/autoconf/build-aux/config.sub
>>> --- a/common/autoconf/build-aux/config.sub      Fri Jan 27 10:15:41
>>> 2017 +0100
>>> +++ b/common/autoconf/build-aux/config.sub      Fri Feb 03 05:00:25
>>> 2017 -0700
>>> @@ -30,7 +30,7 @@
>>>   DIR=`dirname $0`
>>>
>>>   # First, filter out everything that doesn't begin with "aarch64-"
>>> -if ! echo $* | grep '^aarch64-' >/dev/null ; then
>>> +if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ; then
>>>       . $DIR/autoconf-config.sub "$@"
>>>       # autoconf-config.sub exits, so we never reach here, but just in
>>>       # case we do:
>>> @@ -45,6 +45,10 @@
>>>               config=`echo $1 | sed 's/^aarch64-/arm-/'`
>>>               sub_args="$sub_args $config"
>>>               shift; ;;
>>> +        *-msys )
>>> +            config=`echo $1 | sed 's/msys/mingw32/'`
>>> +            sub_args="$sub_args $config"
>>> +            shift; ;;
>>>           - )    # Use stdin as input.
>>>               sub_args="$sub_args $1"
>>>               shift; break ;;
>>>
>>> If I remember correctly, this got me past the configure stage at the
>>> time.
>>>
>>> I don't think it's very hard to get it to work on msys2, I just ran
>>> into one snag too many and didn't think msys2 would be used by anyone.
>>>
>>> /Magnus
>>>
>>> On 2017-10-03 17:20, Peter Budai wrote:
>>>> Hello,
>>>>
>>>> According to
>>>> http://hg.openjdk.java.net/jdk9/jdk9/file/a08cbfc0e4ec/common/doc/building.html 
>>>>
>>>> “msys2 and the new Windows Subsystem for Linux (WSL) would likely be
>>>> possible to support in a future version but that would require a
>>>> community effort to implement”
>>>>
>>>> I’d like to help making the OpenJDK 9 build working on msys2. What is
>>>> the best way to move forward? Is there a similar effort in progress?
>>>>
>>>> Thank you and best regards,
>>>>
>>>> Peter
>>>>
>>>>
>

Reply | Threaded
Open this post in threaded view
|

RE: Building OpenJDK9 on MSYS2

Peter Budai
Hi Magnus and Erik,

I really appreciate your quick feedback. I assumed that it won’t be easy, but I just don’t feel I should give up now  - maybe later when I see the real scale of work. So bear with me for a time being.

Attached is a patch which already includes Magnus’ changes, plus a few which I have added:

  *   basically enabling gcc for windows,
  *   and modifying a logic for compiling fixpath (before that it was using hard-coded MS VSC compile flags)


So here is what I have as the result of configure:
====================================================
The existing configuration has been successfully updated in
/C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release
using configure arguments '--disable-freetype-bundling --disable-javac-server'.

Configuration summary:
* Debug level:    release
* HS debug level: product
* JDK variant:    normal
* JVM variants:   server
* OpenJDK target: OS: windows, CPU architecture: x86, address length: 64
* Version string: 9-internal+0-adhoc.peterbud.jdk9 (9-internal)

Tools summary:
* Environment:    msys version 2.9.0(0.318/5/3) (root at /C/msys64)
* Boot JDK:       java version "1.8.0_144"  Java(TM) SE Runtime Environment (build 1.8.0_144-b01)  Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)   (at /c/progra~1/java/jdk18~1.0_1)
* Toolchain:      gcc (GNU Compiler Collection)
* C Compiler:     Version 7.2.0 (at /C/msys64/mingw64/bin/gcc)
* C++ Compiler:   Version 7.2.0 (at /c/msys64/mingw64/bin/g++)

Build performance summary:
* Cores to use:   4
* Memory limit:   16216 MB

Its clear says that the toolchain is gcc 7.2 (BTW there is no Visual Studio on this machine)

Now for the details of the config log, you can see here: https://pastebin.com/MN2ZYcHH

And about the build process and the error I get:

$ make JOBS=1
Building target 'default (exploded-image)' in configuration 'windows-x86_64-normal-server-release'
Compiling 8 files for BUILD_TOOLS_LANGTOOLS
Compiling 17 properties into resource bundles for jdk.compiler
Parsing 1 properties into enum-like class for jdk.compiler
Compiling 19 properties into resource bundles for jdk.javadoc
Compiling 12 properties into resource bundles for jdk.jdeps
Compiling 7 properties into resource bundles for jdk.jshell
Compiling 117 files for BUILD_INTERIM_java.compiler
Compiling 396 files for BUILD_INTERIM_jdk.compiler
Compiling 61 files for BUILD_INTERIM_jdk.jdeps
Compiling 457 files for BUILD_INTERIM_jdk.javadoc
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Compiling 159 files for BUILD_TOOLS_JDK
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
make[3]: *** [GensrcMisc.gmk:78: /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java] Error 1
make[3]: *** Deleting file '/C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java'
make[2]: *** [make/Main.gmk:115: java.base-gensrc-jdk] Error 2

ERROR: Build failed for target 'default (exploded-image)' in configuration 'windows-x86_64-normal-server-release' (exit code 2)

No indication of failed target found.
Hint: Try searching the build log for '] Error'.
Hint: See common/doc/building.html#troubleshooting for assistance.

make[1]: *** [/home/peterbud/jdk9/make/Init.gmk:296: main] Error 2
make: *** [/home/peterbud/jdk9/make/Init.gmk:185: default] Error 2

If I run here
make JOBS=1 LOG=debug
The failing line seems to be this:

( /usr/bin/gawk '/@@END_COPYRIGHT@@/{exit}1' /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template && /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/configure-support/bin/fixpath.exe -m/C/msys64/@/c/msys64/@/c/progra~ /C/msys64/mingw64/bin/gcc -E -x c  /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template 2> >(/usr/bin/grep -v '^SocketOptionRegistry.java.template$' >&2) | /usr/bin/gawk '/@@START_HERE@@/,0' |  /usr/bin/sed -e 's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED FILE - DO NOT EDIT/' -e 's/PREFIX_//' -e 's/^#.*//' ) > /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java
make[3]: *** [GensrcMisc.gmk:78: /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java] Error 1

Now the interesting is: if I copy this line above to the bash prompt, it runs without problem, and the file support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java
Is produced.

Then I can again issue
make JOBS=1 LOG=debug

And the compile process is being continued, until a similar error pops up with a different generated file. I have an assumption that this happens because make is still running parallel jobs, despite JOBS=1 but I’m not sure.

How could I best tackle this?

Thank you and best regards,

Peter

Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10

From: Magnus Ihse Bursie<mailto:[hidden email]>
Sent: Thursday, October 5, 2017 11:33 AM
To: Erik Joelsson<mailto:[hidden email]>; Peter Budai<mailto:[hidden email]>; [hidden email]<mailto:[hidden email]>
Subject: Re: Building OpenJDK9 on MSYS2

On 2017-10-05 10:10, Erik Joelsson wrote:

> Hello Peter,
>
>
> On 2017-10-04 21:15, Peter Budai wrote:
>> Hi Magnus,
>>
>> Thanks for the quick reply I’ll check these patches with msys2.
>>
>> Let me specify with more details what I’d like to achieve: I’d like
>> to build OpenJDK9 with MSYS2 MINGW64 environment using gcc toolchain.
>> (I’m not sure how familiar are you with MSYS2, but there are 3
>> different environments: MSYS2, MINGW32 and MINGW64). In theory
>> MINGW64 with gcc is the closes you can get on Windows platform as a
>> gcc unix like build environment, which produces still a native 64-bit
>> executable on Windows.
>>
>> I’m not very familiar with OpenJDK yet, so therefore I’d like to hear
>> your opinion: how realistic is that?
> Sorry to disappoint, but I would say that requires major work. There
> is a strong historic assumption that windows builds are done using
> Visual Studio. We have abstracted away some of it in configure (see
> TOOLCHAIN_TYPE), but it's very far from enough to change compiler
> environment for a Windows build. The native sources are also bound to
> make a lot of such assumptions. I would expect the changes needed to
> be in the thousands of lines of code.

I agree that it requires hard work (even if "thousands" might be an
overestimation I think, but "hundreds" is not enough, so it's the right
magnitude). On the other hand, it would be really good if we did sort
things out, so that we had proper conditions based on OS vs
compiler/toolchain.

If you really want to start, the first thing is to patch toolchain.m4 to
VALID_TOOLCHAINS_windows="microsoft gcc"
and then call configure using "bash configure --with-toolchain-type=gcc".

As Erik, I doubt you will come very far before things starts tumbling down.

>
> When we say supporting the build in msys2 instead of cygwin, we just
> mean using msys2 as the unix emulating layer for our tools like
> make/bash/grep/sed etc.
>
> One think I have done successfully is running the build in WSL
> (Windows Subsystem for Linux), but that isn't all that helpful as WSL
> for practical purposes is more or less like running Linux in a VM, so
> the build sees a Linux system and builds a Linux binary.
>> As a side note: with MINGW64 I have managed to run configure phase
>> successfully for OpenJDK. The compile process has also started and
>> went for a while, but interestingly I run into some kind of race
>> conditions as make stopped with an error. Using LOG=debug I have fond
>> the failing line and then copying the failed command and pasting it
>> to the bash prompt it successfully generated the output target, and
>> then the build process run further when a similar situation happened.
>> Also pasting the failed command run in the bash without any problem,
>> and build continued… until the next.
> Without seeing the errors I can't say much. I very much doubt that you
> are running with gcc as the compiler though. Configure isn't easily
> fooled into using a different compiler to what it prefers, and I would
> expect things to crash and burn pretty early if you actually did.
>
> /Erik
>>
>> I have tried to run make JOBS=1, but did not help, strangely I have
>> still seen in the log make[3] and make[4] logs which suggested that
>> there are more than one make jobs were running. Also tried .configure
>> --with-output-sync=recurse without success (same symptoms)
>>
>> Let me know your thoughts.
>>
>> Best regards,
>>
>> Peter
>>
>> Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for
>> Windows 10
>>
>> From: Magnus Ihse Bursie<mailto:[hidden email]>
>> Sent: Wednesday, October 4, 2017 1:04 AM
>> To: Peter Budai<mailto:[hidden email]>;
>> [hidden email]<mailto:[hidden email]>
>> Subject: Re: Building OpenJDK9 on MSYS2
>>
>> Actually, it wasn't so much remaining trouble. :-) I fired up msys2 and
>> checked out where I left off. It turned out that the remaining snag was
>> that msys2 tries to convert command lines automatically, from "unix"
>> style paths to "windows" style paths. Unfortunately, it does not do this
>> very well and it breaks all sorts of things. We already have a FIXPATH
>> solution in place which deals with this, so basically all I had to do
>> was disable this (by setting MSYS2_ARG_CONV_EXCL to "*"). However, this
>> broke our cygpath replacement hack (!) so I had to disable it there.
>> Sigh. Anyway, with those fixes it ran and worked well. (I also
>> discovered and fixed a bug related to how we set up the FIXPATH variable
>> on msys, but it only triggers in certain circumstances).
>>
>> With this patch I now jdk9 seems to build fine on msys2. It should apply
>> cleanly on jdk9/jdk9. Since it turned out to be so trivial, I'll try to
>> get it in in jdk10.
>>
>> Here's the patch if you want to apply it yourself:
>>
>> diff -r a08cbfc0e4ec common/autoconf/basics_windows.m4
>> --- a/common/autoconf/basics_windows.m4    Thu Aug 03 18:56:56 2017
>> +0000
>> +++ b/common/autoconf/basics_windows.m4    Wed Oct 04 00:53:58 2017
>> +0200
>> @@ -42,7 +42,7 @@
>>        windows_path=`$CYGPATH -m "$unix_path"`
>>        $1="$windows_path"
>>      elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then
>> -    windows_path=`cmd //c echo $unix_path`
>> +    windows_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $unix_path`
>>        $1="$windows_path"
>>      fi
>>    ])
>> @@ -136,6 +136,16 @@
>>      fi
>>    ])
>>
>> +AC_DEFUN([BASIC_MSYS_UPDATE_FIXPATH],
>> +[
>> +  # Take all collected prefixes and turn them into a
>> -m/c/foo@/c/bar@... command line
>> +  # @ was chosen as separator to minimize risk of other tools messing
>> around with it
>> +  all_unique_prefixes=`echo "${all_fixpath_prefixes@<:@@@:>@}" \
>> +      | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
>> +  fixpath_argument_list=`echo $all_unique_prefixes  | tr ' ' '@'`
>> +  FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
>> +])
>> +
>>    AC_DEFUN([BASIC_FIXUP_PATH_MSYS],
>>    [
>>      path="[$]$1"
>> @@ -143,7 +153,7 @@
>>      new_path="$path"
>>      if test "x$has_colon" = x; then
>>        # Not in mixed or Windows style, start by that.
>> -    new_path=`cmd //c echo $path`
>> +    new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $path`
>>      fi
>>
>>      BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
>> @@ -155,6 +165,8 @@
>>
>>      # Save the first 10 bytes of this path to the storage, so fixpath
>> can work.
>> all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
>> "${new_path:0:10}")
>> +  # We might need to re-evaluate FIXPATH.
>> +  BASIC_MSYS_UPDATE_FIXPATH
>>    ])
>>
>>    AC_DEFUN([BASIC_FIXUP_EXECUTABLE_CYGWIN],
>> @@ -293,7 +305,7 @@
>>        # Do not save /bin paths to all_fixpath_prefixes!
>>      else
>>        # Not in mixed or Windows style, start by that.
>> -    new_path=`cmd //c echo $new_path`
>> +    new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $new_path`
>>        BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
>>        # Output is in $new_path
>>        BASIC_WINDOWS_REWRITE_AS_UNIX_PATH(new_path)
>> @@ -302,6 +314,8 @@
>>
>>        # Save the first 10 bytes of this path to the storage, so fixpath
>> can work.
>> all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
>> "${new_path:0:10}")
>> +    # We might need to re-evaluate FIXPATH.
>> +    BASIC_MSYS_UPDATE_FIXPATH
>>      fi
>>    ])
>>
>> @@ -347,6 +361,10 @@
>>        WINDOWS_ENV_VENDOR='msys'
>>        WINDOWS_ENV_VERSION="$MSYS_VERSION"
>>
>> +    # Prohibit msys2 path conversion from trying to be "intelligent",
>> and rely
>> +    # on fixpath instead.
>> +    export MSYS2_ARG_CONV_EXCL="*"
>> +
>>        AC_MSG_CHECKING([msys root directory as unix-style path])
>>        # The cmd output ends with Windows line endings (CR/LF), the grep
>> command will strip that away
>>        MSYS_ROOT_PATH=`cd / ; cmd /c cd | $GREP ".*"`
>> @@ -391,10 +409,7 @@
>>        elif test "x$OPENJDK_BUILD_OS_ENV" = xwindows.msys; then
>>          # Take all collected prefixes and turn them into a
>> -m/c/foo@/c/bar@... command line
>>          # @ was chosen as separator to minimize risk of other tools
>> messing around with it
>> -      all_unique_prefixes=`echo "${all_fixpath_prefixes@<:@@@:>@}" \
>> -          | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
>> -      fixpath_argument_list=`echo $all_unique_prefixes  | tr ' ' '@'`
>> -      FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
>> +      BASIC_MSYS_UPDATE_FIXPATH
>>        fi
>>        FIXPATH_SRC_W="$FIXPATH_SRC"
>>        FIXPATH_BIN_W="$FIXPATH_BIN"
>> diff -r a08cbfc0e4ec common/autoconf/build-aux/config.sub
>> --- a/common/autoconf/build-aux/config.sub    Thu Aug 03 18:56:56
>> 2017 +0000
>> +++ b/common/autoconf/build-aux/config.sub    Wed Oct 04 00:53:58
>> 2017 +0200
>> @@ -30,7 +30,7 @@
>>    DIR=`dirname $0`
>>
>>    # First, filter out everything that doesn't begin with "aarch64-"
>> -if ! echo $* | grep '^aarch64-' >/dev/null ; then
>> +if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ; then
>>        . $DIR/autoconf-config.sub "$@"
>>        # autoconf-config.sub exits, so we never reach here, but just in
>>        # case we do:
>> @@ -38,13 +38,17 @@
>>    fi
>>
>>    while test $# -gt 0 ; do
>> -    case $1 in
>> +    case $1 in
>>            -- )   # Stop option processing
>>                shift; break ;;
>>            aarch64-* )
>>                config=`echo $1 | sed 's/^aarch64-/arm-/'`
>>                sub_args="$sub_args $config"
>>                shift; ;;
>> +        *-msys )
>> +            config=`echo $1 | sed 's/msys/mingw32/'`
>> +            sub_args="$sub_args $config"
>> +            shift; ;;
>>            - )    # Use stdin as input.
>>                sub_args="$sub_args $1"
>>                shift; break ;;
>> diff -r a08cbfc0e4ec common/autoconf/spec.gmk.in
>> --- a/common/autoconf/spec.gmk.in    Thu Aug 03 18:56:56 2017 +0000
>> +++ b/common/autoconf/spec.gmk.in    Wed Oct 04 00:53:58 2017 +0200
>> @@ -120,6 +120,13 @@
>>      # On Windows, the Visual Studio toolchain needs the PATH to be
>> adjusted
>>      # to include Visual Studio tools (this needs to be in cygwin/msys
>> style).
>>      export PATH:=@VS_PATH@
>> +
>> +endif
>> +
>> +ifeq ($(OPENJDK_TARGET_OS_ENV), windows.msys)
>> +  # On msys2, prohibit msys path conversion from trying to be
>> +  # "intelligent", and rely on fixpath instead.
>> +  export MSYS2_ARG_CONV_EXCL:=*
>>    endif
>>
>>    SYSROOT_CFLAGS := @SYSROOT_CFLAGS@
>>
>> /Magnus
>>
>> On 2017-10-03 22:34, Magnus Ihse Bursie wrote:
>>> I gave msys2 a shot some time ago, but it ended up too much trouble.
>>> I'll share some of my notes from that attempt, for what it's worth.
>>>
>>> To install package X/Y, run "pacman -S X/Y". Missing tools and
>>> packages where to find them:
>>> cmp: msys/diffutils
>>> tar: msys/tar
>>> make: msys/make
>>> unzip: msys/unzip
>>> zip: msys/zip
>>>
>>> config.sub reports msys as "x86_64-pc-mingw32" but msys2 as
>>> "x86_64-pc-msys". This patch adds postprocessing in "our" config.sub
>>> to report msys2 similar to msys. (Opinions, including my own :-) may
>>> vary if this really is the best way..)
>>>
>>> diff -r b88023f46daa common/autoconf/build-aux/config.sub
>>> --- a/common/autoconf/build-aux/config.sub      Fri Jan 27 10:15:41
>>> 2017 +0100
>>> +++ b/common/autoconf/build-aux/config.sub      Fri Feb 03 05:00:25
>>> 2017 -0700
>>> @@ -30,7 +30,7 @@
>>>   DIR=`dirname $0`
>>>
>>>   # First, filter out everything that doesn't begin with "aarch64-"
>>> -if ! echo $* | grep '^aarch64-' >/dev/null ; then
>>> +if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ; then
>>>       . $DIR/autoconf-config.sub "$@"
>>>       # autoconf-config.sub exits, so we never reach here, but just in
>>>       # case we do:
>>> @@ -45,6 +45,10 @@
>>>               config=`echo $1 | sed 's/^aarch64-/arm-/'`
>>>               sub_args="$sub_args $config"
>>>               shift; ;;
>>> +        *-msys )
>>> +            config=`echo $1 | sed 's/msys/mingw32/'`
>>> +            sub_args="$sub_args $config"
>>> +            shift; ;;
>>>           - )    # Use stdin as input.
>>>               sub_args="$sub_args $1"
>>>               shift; break ;;
>>>
>>> If I remember correctly, this got me past the configure stage at the
>>> time.
>>>
>>> I don't think it's very hard to get it to work on msys2, I just ran
>>> into one snag too many and didn't think msys2 would be used by anyone.
>>>
>>> /Magnus
>>>
>>> On 2017-10-03 17:20, Peter Budai wrote:
>>>> Hello,
>>>>
>>>> According to
>>>> http://hg.openjdk.java.net/jdk9/jdk9/file/a08cbfc0e4ec/common/doc/building.html
>>>>
>>>> “msys2 and the new Windows Subsystem for Linux (WSL) would likely be
>>>> possible to support in a future version but that would require a
>>>> community effort to implement”
>>>>
>>>> I’d like to help making the OpenJDK 9 build working on msys2. What is
>>>> the best way to move forward? Is there a similar effort in progress?
>>>>
>>>> Thank you and best regards,
>>>>
>>>> Peter
>>>>
>>>>
>

Reply | Threaded
Open this post in threaded view
|

RE: Building OpenJDK9 on MSYS2

Peter Budai
Now the attachment as well…

Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10

From: Peter Budai<mailto:[hidden email]>
Sent: Thursday, October 5, 2017 11:59 AM
To: Magnus Ihse Bursie<mailto:[hidden email]>; Erik Joelsson<mailto:[hidden email]>; [hidden email]<mailto:[hidden email]>
Subject: RE: Building OpenJDK9 on MSYS2

Hi Magnus and Erik,

I really appreciate your quick feedback. I assumed that it won’t be easy, but I just don’t feel I should give up now  - maybe later when I see the real scale of work. So bear with me for a time being.

Attached is a patch which already includes Magnus’ changes, plus a few which I have added:

  *   basically enabling gcc for windows,
  *   and modifying a logic for compiling fixpath (before that it was using hard-coded MS VSC compile flags)


So here is what I have as the result of configure:
====================================================
The existing configuration has been successfully updated in
/C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release
using configure arguments '--disable-freetype-bundling --disable-javac-server'.

Configuration summary:
* Debug level:    release
* HS debug level: product
* JDK variant:    normal
* JVM variants:   server
* OpenJDK target: OS: windows, CPU architecture: x86, address length: 64
* Version string: 9-internal+0-adhoc.peterbud.jdk9 (9-internal)

Tools summary:
* Environment:    msys version 2.9.0(0.318/5/3) (root at /C/msys64)
* Boot JDK:       java version "1.8.0_144"  Java(TM) SE Runtime Environment (build 1.8.0_144-b01)  Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)   (at /c/progra~1/java/jdk18~1.0_1)
* Toolchain:      gcc (GNU Compiler Collection)
* C Compiler:     Version 7.2.0 (at /C/msys64/mingw64/bin/gcc)
* C++ Compiler:   Version 7.2.0 (at /c/msys64/mingw64/bin/g++)

Build performance summary:
* Cores to use:   4
* Memory limit:   16216 MB

Its clear says that the toolchain is gcc 7.2 (BTW there is no Visual Studio on this machine)

Now for the details of the config log, you can see here: https://pastebin.com/MN2ZYcHH

And about the build process and the error I get:

$ make JOBS=1
Building target 'default (exploded-image)' in configuration 'windows-x86_64-normal-server-release'
Compiling 8 files for BUILD_TOOLS_LANGTOOLS
Compiling 17 properties into resource bundles for jdk.compiler
Parsing 1 properties into enum-like class for jdk.compiler
Compiling 19 properties into resource bundles for jdk.javadoc
Compiling 12 properties into resource bundles for jdk.jdeps
Compiling 7 properties into resource bundles for jdk.jshell
Compiling 117 files for BUILD_INTERIM_java.compiler
Compiling 396 files for BUILD_INTERIM_jdk.compiler
Compiling 61 files for BUILD_INTERIM_jdk.jdeps
Compiling 457 files for BUILD_INTERIM_jdk.javadoc
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Compiling 159 files for BUILD_TOOLS_JDK
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
make[3]: *** [GensrcMisc.gmk:78: /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java] Error 1
make[3]: *** Deleting file '/C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java'
make[2]: *** [make/Main.gmk:115: java.base-gensrc-jdk] Error 2

ERROR: Build failed for target 'default (exploded-image)' in configuration 'windows-x86_64-normal-server-release' (exit code 2)

No indication of failed target found.
Hint: Try searching the build log for '] Error'.
Hint: See common/doc/building.html#troubleshooting for assistance.

make[1]: *** [/home/peterbud/jdk9/make/Init.gmk:296: main] Error 2
make: *** [/home/peterbud/jdk9/make/Init.gmk:185: default] Error 2

If I run here
make JOBS=1 LOG=debug
The failing line seems to be this:

( /usr/bin/gawk '/@@END_COPYRIGHT@@/{exit}1' /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template && /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/configure-support/bin/fixpath.exe -m/C/msys64/@/c/msys64/@/c/progra~ /C/msys64/mingw64/bin/gcc -E -x c  /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template 2> >(/usr/bin/grep -v '^SocketOptionRegistry.java.template$' >&2) | /usr/bin/gawk '/@@START_HERE@@/,0' |  /usr/bin/sed -e 's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED FILE - DO NOT EDIT/' -e 's/PREFIX_//' -e 's/^#.*//' ) > /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java
make[3]: *** [GensrcMisc.gmk:78: /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java] Error 1

Now the interesting is: if I copy this line above to the bash prompt, it runs without problem, and the file support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java
Is produced.

Then I can again issue
make JOBS=1 LOG=debug

And the compile process is being continued, until a similar error pops up with a different generated file. I have an assumption that this happens because make is still running parallel jobs, despite JOBS=1 but I’m not sure.

How could I best tackle this?

Thank you and best regards,

Peter

Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10

From: Magnus Ihse Bursie<mailto:[hidden email]>
Sent: Thursday, October 5, 2017 11:33 AM
To: Erik Joelsson<mailto:[hidden email]>; Peter Budai<mailto:[hidden email]>; [hidden email]<mailto:[hidden email]>
Subject: Re: Building OpenJDK9 on MSYS2

On 2017-10-05 10:10, Erik Joelsson wrote:

> Hello Peter,
>
>
> On 2017-10-04 21:15, Peter Budai wrote:
>> Hi Magnus,
>>
>> Thanks for the quick reply I’ll check these patches with msys2.
>>
>> Let me specify with more details what I’d like to achieve: I’d like
>> to build OpenJDK9 with MSYS2 MINGW64 environment using gcc toolchain.
>> (I’m not sure how familiar are you with MSYS2, but there are 3
>> different environments: MSYS2, MINGW32 and MINGW64). In theory
>> MINGW64 with gcc is the closes you can get on Windows platform as a
>> gcc unix like build environment, which produces still a native 64-bit
>> executable on Windows.
>>
>> I’m not very familiar with OpenJDK yet, so therefore I’d like to hear
>> your opinion: how realistic is that?
> Sorry to disappoint, but I would say that requires major work. There
> is a strong historic assumption that windows builds are done using
> Visual Studio. We have abstracted away some of it in configure (see
> TOOLCHAIN_TYPE), but it's very far from enough to change compiler
> environment for a Windows build. The native sources are also bound to
> make a lot of such assumptions. I would expect the changes needed to
> be in the thousands of lines of code.

I agree that it requires hard work (even if "thousands" might be an
overestimation I think, but "hundreds" is not enough, so it's the right
magnitude). On the other hand, it would be really good if we did sort
things out, so that we had proper conditions based on OS vs
compiler/toolchain.

If you really want to start, the first thing is to patch toolchain.m4 to
VALID_TOOLCHAINS_windows="microsoft gcc"
and then call configure using "bash configure --with-toolchain-type=gcc".

As Erik, I doubt you will come very far before things starts tumbling down.

>
> When we say supporting the build in msys2 instead of cygwin, we just
> mean using msys2 as the unix emulating layer for our tools like
> make/bash/grep/sed etc.
>
> One think I have done successfully is running the build in WSL
> (Windows Subsystem for Linux), but that isn't all that helpful as WSL
> for practical purposes is more or less like running Linux in a VM, so
> the build sees a Linux system and builds a Linux binary.
>> As a side note: with MINGW64 I have managed to run configure phase
>> successfully for OpenJDK. The compile process has also started and
>> went for a while, but interestingly I run into some kind of race
>> conditions as make stopped with an error. Using LOG=debug I have fond
>> the failing line and then copying the failed command and pasting it
>> to the bash prompt it successfully generated the output target, and
>> then the build process run further when a similar situation happened.
>> Also pasting the failed command run in the bash without any problem,
>> and build continued… until the next.
> Without seeing the errors I can't say much. I very much doubt that you
> are running with gcc as the compiler though. Configure isn't easily
> fooled into using a different compiler to what it prefers, and I would
> expect things to crash and burn pretty early if you actually did.
>
> /Erik
>>
>> I have tried to run make JOBS=1, but did not help, strangely I have
>> still seen in the log make[3] and make[4] logs which suggested that
>> there are more than one make jobs were running. Also tried .configure
>> --with-output-sync=recurse without success (same symptoms)
>>
>> Let me know your thoughts.
>>
>> Best regards,
>>
>> Peter
>>
>> Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for
>> Windows 10
>>
>> From: Magnus Ihse Bursie<mailto:[hidden email]>
>> Sent: Wednesday, October 4, 2017 1:04 AM
>> To: Peter Budai<mailto:[hidden email]>;
>> [hidden email]<mailto:[hidden email]>
>> Subject: Re: Building OpenJDK9 on MSYS2
>>
>> Actually, it wasn't so much remaining trouble. :-) I fired up msys2 and
>> checked out where I left off. It turned out that the remaining snag was
>> that msys2 tries to convert command lines automatically, from "unix"
>> style paths to "windows" style paths. Unfortunately, it does not do this
>> very well and it breaks all sorts of things. We already have a FIXPATH
>> solution in place which deals with this, so basically all I had to do
>> was disable this (by setting MSYS2_ARG_CONV_EXCL to "*"). However, this
>> broke our cygpath replacement hack (!) so I had to disable it there.
>> Sigh. Anyway, with those fixes it ran and worked well. (I also
>> discovered and fixed a bug related to how we set up the FIXPATH variable
>> on msys, but it only triggers in certain circumstances).
>>
>> With this patch I now jdk9 seems to build fine on msys2. It should apply
>> cleanly on jdk9/jdk9. Since it turned out to be so trivial, I'll try to
>> get it in in jdk10.
>>
>> Here's the patch if you want to apply it yourself:
>>
>> diff -r a08cbfc0e4ec common/autoconf/basics_windows.m4
>> --- a/common/autoconf/basics_windows.m4    Thu Aug 03 18:56:56 2017
>> +0000
>> +++ b/common/autoconf/basics_windows.m4    Wed Oct 04 00:53:58 2017
>> +0200
>> @@ -42,7 +42,7 @@
>>        windows_path=`$CYGPATH -m "$unix_path"`
>>        $1="$windows_path"
>>      elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then
>> -    windows_path=`cmd //c echo $unix_path`
>> +    windows_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $unix_path`
>>        $1="$windows_path"
>>      fi
>>    ])
>> @@ -136,6 +136,16 @@
>>      fi
>>    ])
>>
>> +AC_DEFUN([BASIC_MSYS_UPDATE_FIXPATH],
>> +[
>> +  # Take all collected prefixes and turn them into a
>> -m/c/foo@/c/bar@... command line
>> +  # @ was chosen as separator to minimize risk of other tools messing
>> around with it
>> +  all_unique_prefixes=`echo "${all_fixpath_prefixes@<:@@@:>@}" \
>> +      | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
>> +  fixpath_argument_list=`echo $all_unique_prefixes  | tr ' ' '@'`
>> +  FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
>> +])
>> +
>>    AC_DEFUN([BASIC_FIXUP_PATH_MSYS],
>>    [
>>      path="[$]$1"
>> @@ -143,7 +153,7 @@
>>      new_path="$path"
>>      if test "x$has_colon" = x; then
>>        # Not in mixed or Windows style, start by that.
>> -    new_path=`cmd //c echo $path`
>> +    new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $path`
>>      fi
>>
>>      BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
>> @@ -155,6 +165,8 @@
>>
>>      # Save the first 10 bytes of this path to the storage, so fixpath
>> can work.
>> all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
>> "${new_path:0:10}")
>> +  # We might need to re-evaluate FIXPATH.
>> +  BASIC_MSYS_UPDATE_FIXPATH
>>    ])
>>
>>    AC_DEFUN([BASIC_FIXUP_EXECUTABLE_CYGWIN],
>> @@ -293,7 +305,7 @@
>>        # Do not save /bin paths to all_fixpath_prefixes!
>>      else
>>        # Not in mixed or Windows style, start by that.
>> -    new_path=`cmd //c echo $new_path`
>> +    new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $new_path`
>>        BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
>>        # Output is in $new_path
>>        BASIC_WINDOWS_REWRITE_AS_UNIX_PATH(new_path)
>> @@ -302,6 +314,8 @@
>>
>>        # Save the first 10 bytes of this path to the storage, so fixpath
>> can work.
>> all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
>> "${new_path:0:10}")
>> +    # We might need to re-evaluate FIXPATH.
>> +    BASIC_MSYS_UPDATE_FIXPATH
>>      fi
>>    ])
>>
>> @@ -347,6 +361,10 @@
>>        WINDOWS_ENV_VENDOR='msys'
>>        WINDOWS_ENV_VERSION="$MSYS_VERSION"
>>
>> +    # Prohibit msys2 path conversion from trying to be "intelligent",
>> and rely
>> +    # on fixpath instead.
>> +    export MSYS2_ARG_CONV_EXCL="*"
>> +
>>        AC_MSG_CHECKING([msys root directory as unix-style path])
>>        # The cmd output ends with Windows line endings (CR/LF), the grep
>> command will strip that away
>>        MSYS_ROOT_PATH=`cd / ; cmd /c cd | $GREP ".*"`
>> @@ -391,10 +409,7 @@
>>        elif test "x$OPENJDK_BUILD_OS_ENV" = xwindows.msys; then
>>          # Take all collected prefixes and turn them into a
>> -m/c/foo@/c/bar@... command line
>>          # @ was chosen as separator to minimize risk of other tools
>> messing around with it
>> -      all_unique_prefixes=`echo "${all_fixpath_prefixes@<:@@@:>@}" \
>> -          | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
>> -      fixpath_argument_list=`echo $all_unique_prefixes  | tr ' ' '@'`
>> -      FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
>> +      BASIC_MSYS_UPDATE_FIXPATH
>>        fi
>>        FIXPATH_SRC_W="$FIXPATH_SRC"
>>        FIXPATH_BIN_W="$FIXPATH_BIN"
>> diff -r a08cbfc0e4ec common/autoconf/build-aux/config.sub
>> --- a/common/autoconf/build-aux/config.sub    Thu Aug 03 18:56:56
>> 2017 +0000
>> +++ b/common/autoconf/build-aux/config.sub    Wed Oct 04 00:53:58
>> 2017 +0200
>> @@ -30,7 +30,7 @@
>>    DIR=`dirname $0`
>>
>>    # First, filter out everything that doesn't begin with "aarch64-"
>> -if ! echo $* | grep '^aarch64-' >/dev/null ; then
>> +if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ; then
>>        . $DIR/autoconf-config.sub "$@"
>>        # autoconf-config.sub exits, so we never reach here, but just in
>>        # case we do:
>> @@ -38,13 +38,17 @@
>>    fi
>>
>>    while test $# -gt 0 ; do
>> -    case $1 in
>> +    case $1 in
>>            -- )   # Stop option processing
>>                shift; break ;;
>>            aarch64-* )
>>                config=`echo $1 | sed 's/^aarch64-/arm-/'`
>>                sub_args="$sub_args $config"
>>                shift; ;;
>> +        *-msys )
>> +            config=`echo $1 | sed 's/msys/mingw32/'`
>> +            sub_args="$sub_args $config"
>> +            shift; ;;
>>            - )    # Use stdin as input.
>>                sub_args="$sub_args $1"
>>                shift; break ;;
>> diff -r a08cbfc0e4ec common/autoconf/spec.gmk.in
>> --- a/common/autoconf/spec.gmk.in    Thu Aug 03 18:56:56 2017 +0000
>> +++ b/common/autoconf/spec.gmk.in    Wed Oct 04 00:53:58 2017 +0200
>> @@ -120,6 +120,13 @@
>>      # On Windows, the Visual Studio toolchain needs the PATH to be
>> adjusted
>>      # to include Visual Studio tools (this needs to be in cygwin/msys
>> style).
>>      export PATH:=@VS_PATH@
>> +
>> +endif
>> +
>> +ifeq ($(OPENJDK_TARGET_OS_ENV), windows.msys)
>> +  # On msys2, prohibit msys path conversion from trying to be
>> +  # "intelligent", and rely on fixpath instead.
>> +  export MSYS2_ARG_CONV_EXCL:=*
>>    endif
>>
>>    SYSROOT_CFLAGS := @SYSROOT_CFLAGS@
>>
>> /Magnus
>>
>> On 2017-10-03 22:34, Magnus Ihse Bursie wrote:
>>> I gave msys2 a shot some time ago, but it ended up too much trouble.
>>> I'll share some of my notes from that attempt, for what it's worth.
>>>
>>> To install package X/Y, run "pacman -S X/Y". Missing tools and
>>> packages where to find them:
>>> cmp: msys/diffutils
>>> tar: msys/tar
>>> make: msys/make
>>> unzip: msys/unzip
>>> zip: msys/zip
>>>
>>> config.sub reports msys as "x86_64-pc-mingw32" but msys2 as
>>> "x86_64-pc-msys". This patch adds postprocessing in "our" config.sub
>>> to report msys2 similar to msys. (Opinions, including my own :-) may
>>> vary if this really is the best way..)
>>>
>>> diff -r b88023f46daa common/autoconf/build-aux/config.sub
>>> --- a/common/autoconf/build-aux/config.sub      Fri Jan 27 10:15:41
>>> 2017 +0100
>>> +++ b/common/autoconf/build-aux/config.sub      Fri Feb 03 05:00:25
>>> 2017 -0700
>>> @@ -30,7 +30,7 @@
>>>   DIR=`dirname $0`
>>>
>>>   # First, filter out everything that doesn't begin with "aarch64-"
>>> -if ! echo $* | grep '^aarch64-' >/dev/null ; then
>>> +if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ; then
>>>       . $DIR/autoconf-config.sub "$@"
>>>       # autoconf-config.sub exits, so we never reach here, but just in
>>>       # case we do:
>>> @@ -45,6 +45,10 @@
>>>               config=`echo $1 | sed 's/^aarch64-/arm-/'`
>>>               sub_args="$sub_args $config"
>>>               shift; ;;
>>> +        *-msys )
>>> +            config=`echo $1 | sed 's/msys/mingw32/'`
>>> +            sub_args="$sub_args $config"
>>> +            shift; ;;
>>>           - )    # Use stdin as input.
>>>               sub_args="$sub_args $1"
>>>               shift; break ;;
>>>
>>> If I remember correctly, this got me past the configure stage at the
>>> time.
>>>
>>> I don't think it's very hard to get it to work on msys2, I just ran
>>> into one snag too many and didn't think msys2 would be used by anyone.
>>>
>>> /Magnus
>>>
>>> On 2017-10-03 17:20, Peter Budai wrote:
>>>> Hello,
>>>>
>>>> According to
>>>> http://hg.openjdk.java.net/jdk9/jdk9/file/a08cbfc0e4ec/common/doc/building.html
>>>>
>>>> “msys2 and the new Windows Subsystem for Linux (WSL) would likely be
>>>> possible to support in a future version but that would require a
>>>> community effort to implement”
>>>>
>>>> I’d like to help making the OpenJDK 9 build working on msys2. What is
>>>> the best way to move forward? Is there a similar effort in progress?
>>>>
>>>> Thank you and best regards,
>>>>
>>>> Peter
>>>>
>>>>
>


Reply | Threaded
Open this post in threaded view
|

Re: Building OpenJDK9 on MSYS2

David Holmes
On 5/10/2017 8:00 PM, Peter Budai wrote:
> Now the attachment as well…

Attachments get stripped by the mail list servers.

David

> Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10
>
> From: Peter Budai<mailto:[hidden email]>
> Sent: Thursday, October 5, 2017 11:59 AM
> To: Magnus Ihse Bursie<mailto:[hidden email]>; Erik Joelsson<mailto:[hidden email]>; [hidden email]<mailto:[hidden email]>
> Subject: RE: Building OpenJDK9 on MSYS2
>
> Hi Magnus and Erik,
>
> I really appreciate your quick feedback. I assumed that it won’t be easy, but I just don’t feel I should give up now  - maybe later when I see the real scale of work. So bear with me for a time being.
>
> Attached is a patch which already includes Magnus’ changes, plus a few which I have added:
>
>    *   basically enabling gcc for windows,
>    *   and modifying a logic for compiling fixpath (before that it was using hard-coded MS VSC compile flags)
>
>
> So here is what I have as the result of configure:
> ====================================================
> The existing configuration has been successfully updated in
> /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release
> using configure arguments '--disable-freetype-bundling --disable-javac-server'.
>
> Configuration summary:
> * Debug level:    release
> * HS debug level: product
> * JDK variant:    normal
> * JVM variants:   server
> * OpenJDK target: OS: windows, CPU architecture: x86, address length: 64
> * Version string: 9-internal+0-adhoc.peterbud.jdk9 (9-internal)
>
> Tools summary:
> * Environment:    msys version 2.9.0(0.318/5/3) (root at /C/msys64)
> * Boot JDK:       java version "1.8.0_144"  Java(TM) SE Runtime Environment (build 1.8.0_144-b01)  Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)   (at /c/progra~1/java/jdk18~1.0_1)
> * Toolchain:      gcc (GNU Compiler Collection)
> * C Compiler:     Version 7.2.0 (at /C/msys64/mingw64/bin/gcc)
> * C++ Compiler:   Version 7.2.0 (at /c/msys64/mingw64/bin/g++)
>
> Build performance summary:
> * Cores to use:   4
> * Memory limit:   16216 MB
>
> Its clear says that the toolchain is gcc 7.2 (BTW there is no Visual Studio on this machine)
>
> Now for the details of the config log, you can see here: https://pastebin.com/MN2ZYcHH
>
> And about the build process and the error I get:
>
> $ make JOBS=1
> Building target 'default (exploded-image)' in configuration 'windows-x86_64-normal-server-release'
> Compiling 8 files for BUILD_TOOLS_LANGTOOLS
> Compiling 17 properties into resource bundles for jdk.compiler
> Parsing 1 properties into enum-like class for jdk.compiler
> Compiling 19 properties into resource bundles for jdk.javadoc
> Compiling 12 properties into resource bundles for jdk.jdeps
> Compiling 7 properties into resource bundles for jdk.jshell
> Compiling 117 files for BUILD_INTERIM_java.compiler
> Compiling 396 files for BUILD_INTERIM_jdk.compiler
> Compiling 61 files for BUILD_INTERIM_jdk.jdeps
> Compiling 457 files for BUILD_INTERIM_jdk.javadoc
> Note: Some input files use or override a deprecated API.
> Note: Recompile with -Xlint:deprecation for details.
> Compiling 159 files for BUILD_TOOLS_JDK
> Note: Some input files use unchecked or unsafe operations.
> Note: Recompile with -Xlint:unchecked for details.
> make[3]: *** [GensrcMisc.gmk:78: /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java] Error 1
> make[3]: *** Deleting file '/C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java'
> make[2]: *** [make/Main.gmk:115: java.base-gensrc-jdk] Error 2
>
> ERROR: Build failed for target 'default (exploded-image)' in configuration 'windows-x86_64-normal-server-release' (exit code 2)
>
> No indication of failed target found.
> Hint: Try searching the build log for '] Error'.
> Hint: See common/doc/building.html#troubleshooting for assistance.
>
> make[1]: *** [/home/peterbud/jdk9/make/Init.gmk:296: main] Error 2
> make: *** [/home/peterbud/jdk9/make/Init.gmk:185: default] Error 2
>
> If I run here
> make JOBS=1 LOG=debug
> The failing line seems to be this:
>
> ( /usr/bin/gawk '/@@END_COPYRIGHT@@/{exit}1' /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template && /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/configure-support/bin/fixpath.exe -m/C/msys64/@/c/msys64/@/c/progra~ /C/msys64/mingw64/bin/gcc -E -x c  /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template 2> >(/usr/bin/grep -v '^SocketOptionRegistry.java.template$' >&2) | /usr/bin/gawk '/@@START_HERE@@/,0' |  /usr/bin/sed -e 's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED FILE - DO NOT EDIT/' -e 's/PREFIX_//' -e 's/^#.*//' ) > /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java
> make[3]: *** [GensrcMisc.gmk:78: /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java] Error 1
>
> Now the interesting is: if I copy this line above to the bash prompt, it runs without problem, and the file support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java
> Is produced.
>
> Then I can again issue
> make JOBS=1 LOG=debug
>
> And the compile process is being continued, until a similar error pops up with a different generated file. I have an assumption that this happens because make is still running parallel jobs, despite JOBS=1 but I’m not sure.
>
> How could I best tackle this?
>
> Thank you and best regards,
>
> Peter
>
> Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10
>
> From: Magnus Ihse Bursie<mailto:[hidden email]>
> Sent: Thursday, October 5, 2017 11:33 AM
> To: Erik Joelsson<mailto:[hidden email]>; Peter Budai<mailto:[hidden email]>; [hidden email]<mailto:[hidden email]>
> Subject: Re: Building OpenJDK9 on MSYS2
>
> On 2017-10-05 10:10, Erik Joelsson wrote:
>> Hello Peter,
>>
>>
>> On 2017-10-04 21:15, Peter Budai wrote:
>>> Hi Magnus,
>>>
>>> Thanks for the quick reply I’ll check these patches with msys2.
>>>
>>> Let me specify with more details what I’d like to achieve: I’d like
>>> to build OpenJDK9 with MSYS2 MINGW64 environment using gcc toolchain.
>>> (I’m not sure how familiar are you with MSYS2, but there are 3
>>> different environments: MSYS2, MINGW32 and MINGW64). In theory
>>> MINGW64 with gcc is the closes you can get on Windows platform as a
>>> gcc unix like build environment, which produces still a native 64-bit
>>> executable on Windows.
>>>
>>> I’m not very familiar with OpenJDK yet, so therefore I’d like to hear
>>> your opinion: how realistic is that?
>> Sorry to disappoint, but I would say that requires major work. There
>> is a strong historic assumption that windows builds are done using
>> Visual Studio. We have abstracted away some of it in configure (see
>> TOOLCHAIN_TYPE), but it's very far from enough to change compiler
>> environment for a Windows build. The native sources are also bound to
>> make a lot of such assumptions. I would expect the changes needed to
>> be in the thousands of lines of code.
>
> I agree that it requires hard work (even if "thousands" might be an
> overestimation I think, but "hundreds" is not enough, so it's the right
> magnitude). On the other hand, it would be really good if we did sort
> things out, so that we had proper conditions based on OS vs
> compiler/toolchain.
>
> If you really want to start, the first thing is to patch toolchain.m4 to
> VALID_TOOLCHAINS_windows="microsoft gcc"
> and then call configure using "bash configure --with-toolchain-type=gcc".
>
> As Erik, I doubt you will come very far before things starts tumbling down.
>
>>
>> When we say supporting the build in msys2 instead of cygwin, we just
>> mean using msys2 as the unix emulating layer for our tools like
>> make/bash/grep/sed etc.
>>
>> One think I have done successfully is running the build in WSL
>> (Windows Subsystem for Linux), but that isn't all that helpful as WSL
>> for practical purposes is more or less like running Linux in a VM, so
>> the build sees a Linux system and builds a Linux binary.
>>> As a side note: with MINGW64 I have managed to run configure phase
>>> successfully for OpenJDK. The compile process has also started and
>>> went for a while, but interestingly I run into some kind of race
>>> conditions as make stopped with an error. Using LOG=debug I have fond
>>> the failing line and then copying the failed command and pasting it
>>> to the bash prompt it successfully generated the output target, and
>>> then the build process run further when a similar situation happened.
>>> Also pasting the failed command run in the bash without any problem,
>>> and build continued… until the next.
>> Without seeing the errors I can't say much. I very much doubt that you
>> are running with gcc as the compiler though. Configure isn't easily
>> fooled into using a different compiler to what it prefers, and I would
>> expect things to crash and burn pretty early if you actually did.
>>
>> /Erik
>>>
>>> I have tried to run make JOBS=1, but did not help, strangely I have
>>> still seen in the log make[3] and make[4] logs which suggested that
>>> there are more than one make jobs were running. Also tried .configure
>>> --with-output-sync=recurse without success (same symptoms)
>>>
>>> Let me know your thoughts.
>>>
>>> Best regards,
>>>
>>> Peter
>>>
>>> Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for
>>> Windows 10
>>>
>>> From: Magnus Ihse Bursie<mailto:[hidden email]>
>>> Sent: Wednesday, October 4, 2017 1:04 AM
>>> To: Peter Budai<mailto:[hidden email]>;
>>> [hidden email]<mailto:[hidden email]>
>>> Subject: Re: Building OpenJDK9 on MSYS2
>>>
>>> Actually, it wasn't so much remaining trouble. :-) I fired up msys2 and
>>> checked out where I left off. It turned out that the remaining snag was
>>> that msys2 tries to convert command lines automatically, from "unix"
>>> style paths to "windows" style paths. Unfortunately, it does not do this
>>> very well and it breaks all sorts of things. We already have a FIXPATH
>>> solution in place which deals with this, so basically all I had to do
>>> was disable this (by setting MSYS2_ARG_CONV_EXCL to "*"). However, this
>>> broke our cygpath replacement hack (!) so I had to disable it there.
>>> Sigh. Anyway, with those fixes it ran and worked well. (I also
>>> discovered and fixed a bug related to how we set up the FIXPATH variable
>>> on msys, but it only triggers in certain circumstances).
>>>
>>> With this patch I now jdk9 seems to build fine on msys2. It should apply
>>> cleanly on jdk9/jdk9. Since it turned out to be so trivial, I'll try to
>>> get it in in jdk10.
>>>
>>> Here's the patch if you want to apply it yourself:
>>>
>>> diff -r a08cbfc0e4ec common/autoconf/basics_windows.m4
>>> --- a/common/autoconf/basics_windows.m4    Thu Aug 03 18:56:56 2017
>>> +0000
>>> +++ b/common/autoconf/basics_windows.m4    Wed Oct 04 00:53:58 2017
>>> +0200
>>> @@ -42,7 +42,7 @@
>>>         windows_path=`$CYGPATH -m "$unix_path"`
>>>         $1="$windows_path"
>>>       elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then
>>> -    windows_path=`cmd //c echo $unix_path`
>>> +    windows_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $unix_path`
>>>         $1="$windows_path"
>>>       fi
>>>     ])
>>> @@ -136,6 +136,16 @@
>>>       fi
>>>     ])
>>>
>>> +AC_DEFUN([BASIC_MSYS_UPDATE_FIXPATH],
>>> +[
>>> +  # Take all collected prefixes and turn them into a
>>> -m/c/foo@/c/bar@... command line
>>> +  # @ was chosen as separator to minimize risk of other tools messing
>>> around with it
>>> +  all_unique_prefixes=`echo "${all_fixpath_prefixes@<:@@@:>@}" \
>>> +      | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
>>> +  fixpath_argument_list=`echo $all_unique_prefixes  | tr ' ' '@'`
>>> +  FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
>>> +])
>>> +
>>>     AC_DEFUN([BASIC_FIXUP_PATH_MSYS],
>>>     [
>>>       path="[$]$1"
>>> @@ -143,7 +153,7 @@
>>>       new_path="$path"
>>>       if test "x$has_colon" = x; then
>>>         # Not in mixed or Windows style, start by that.
>>> -    new_path=`cmd //c echo $path`
>>> +    new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $path`
>>>       fi
>>>
>>>       BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
>>> @@ -155,6 +165,8 @@
>>>
>>>       # Save the first 10 bytes of this path to the storage, so fixpath
>>> can work.
>>> all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
>>> "${new_path:0:10}")
>>> +  # We might need to re-evaluate FIXPATH.
>>> +  BASIC_MSYS_UPDATE_FIXPATH
>>>     ])
>>>
>>>     AC_DEFUN([BASIC_FIXUP_EXECUTABLE_CYGWIN],
>>> @@ -293,7 +305,7 @@
>>>         # Do not save /bin paths to all_fixpath_prefixes!
>>>       else
>>>         # Not in mixed or Windows style, start by that.
>>> -    new_path=`cmd //c echo $new_path`
>>> +    new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $new_path`
>>>         BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
>>>         # Output is in $new_path
>>>         BASIC_WINDOWS_REWRITE_AS_UNIX_PATH(new_path)
>>> @@ -302,6 +314,8 @@
>>>
>>>         # Save the first 10 bytes of this path to the storage, so fixpath
>>> can work.
>>> all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
>>> "${new_path:0:10}")
>>> +    # We might need to re-evaluate FIXPATH.
>>> +    BASIC_MSYS_UPDATE_FIXPATH
>>>       fi
>>>     ])
>>>
>>> @@ -347,6 +361,10 @@
>>>         WINDOWS_ENV_VENDOR='msys'
>>>         WINDOWS_ENV_VERSION="$MSYS_VERSION"
>>>
>>> +    # Prohibit msys2 path conversion from trying to be "intelligent",
>>> and rely
>>> +    # on fixpath instead.
>>> +    export MSYS2_ARG_CONV_EXCL="*"
>>> +
>>>         AC_MSG_CHECKING([msys root directory as unix-style path])
>>>         # The cmd output ends with Windows line endings (CR/LF), the grep
>>> command will strip that away
>>>         MSYS_ROOT_PATH=`cd / ; cmd /c cd | $GREP ".*"`
>>> @@ -391,10 +409,7 @@
>>>         elif test "x$OPENJDK_BUILD_OS_ENV" = xwindows.msys; then
>>>           # Take all collected prefixes and turn them into a
>>> -m/c/foo@/c/bar@... command line
>>>           # @ was chosen as separator to minimize risk of other tools
>>> messing around with it
>>> -      all_unique_prefixes=`echo "${all_fixpath_prefixes@<:@@@:>@}" \
>>> -          | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
>>> -      fixpath_argument_list=`echo $all_unique_prefixes  | tr ' ' '@'`
>>> -      FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
>>> +      BASIC_MSYS_UPDATE_FIXPATH
>>>         fi
>>>         FIXPATH_SRC_W="$FIXPATH_SRC"
>>>         FIXPATH_BIN_W="$FIXPATH_BIN"
>>> diff -r a08cbfc0e4ec common/autoconf/build-aux/config.sub
>>> --- a/common/autoconf/build-aux/config.sub    Thu Aug 03 18:56:56
>>> 2017 +0000
>>> +++ b/common/autoconf/build-aux/config.sub    Wed Oct 04 00:53:58
>>> 2017 +0200
>>> @@ -30,7 +30,7 @@
>>>     DIR=`dirname $0`
>>>
>>>     # First, filter out everything that doesn't begin with "aarch64-"
>>> -if ! echo $* | grep '^aarch64-' >/dev/null ; then
>>> +if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ; then
>>>         . $DIR/autoconf-config.sub "$@"
>>>         # autoconf-config.sub exits, so we never reach here, but just in
>>>         # case we do:
>>> @@ -38,13 +38,17 @@
>>>     fi
>>>
>>>     while test $# -gt 0 ; do
>>> -    case $1 in
>>> +    case $1 in
>>>             -- )   # Stop option processing
>>>                 shift; break ;;
>>>             aarch64-* )
>>>                 config=`echo $1 | sed 's/^aarch64-/arm-/'`
>>>                 sub_args="$sub_args $config"
>>>                 shift; ;;
>>> +        *-msys )
>>> +            config=`echo $1 | sed 's/msys/mingw32/'`
>>> +            sub_args="$sub_args $config"
>>> +            shift; ;;
>>>             - )    # Use stdin as input.
>>>                 sub_args="$sub_args $1"
>>>                 shift; break ;;
>>> diff -r a08cbfc0e4ec common/autoconf/spec.gmk.in
>>> --- a/common/autoconf/spec.gmk.in    Thu Aug 03 18:56:56 2017 +0000
>>> +++ b/common/autoconf/spec.gmk.in    Wed Oct 04 00:53:58 2017 +0200
>>> @@ -120,6 +120,13 @@
>>>       # On Windows, the Visual Studio toolchain needs the PATH to be
>>> adjusted
>>>       # to include Visual Studio tools (this needs to be in cygwin/msys
>>> style).
>>>       export PATH:=@VS_PATH@
>>> +
>>> +endif
>>> +
>>> +ifeq ($(OPENJDK_TARGET_OS_ENV), windows.msys)
>>> +  # On msys2, prohibit msys path conversion from trying to be
>>> +  # "intelligent", and rely on fixpath instead.
>>> +  export MSYS2_ARG_CONV_EXCL:=*
>>>     endif
>>>
>>>     SYSROOT_CFLAGS := @SYSROOT_CFLAGS@
>>>
>>> /Magnus
>>>
>>> On 2017-10-03 22:34, Magnus Ihse Bursie wrote:
>>>> I gave msys2 a shot some time ago, but it ended up too much trouble.
>>>> I'll share some of my notes from that attempt, for what it's worth.
>>>>
>>>> To install package X/Y, run "pacman -S X/Y". Missing tools and
>>>> packages where to find them:
>>>> cmp: msys/diffutils
>>>> tar: msys/tar
>>>> make: msys/make
>>>> unzip: msys/unzip
>>>> zip: msys/zip
>>>>
>>>> config.sub reports msys as "x86_64-pc-mingw32" but msys2 as
>>>> "x86_64-pc-msys". This patch adds postprocessing in "our" config.sub
>>>> to report msys2 similar to msys. (Opinions, including my own :-) may
>>>> vary if this really is the best way..)
>>>>
>>>> diff -r b88023f46daa common/autoconf/build-aux/config.sub
>>>> --- a/common/autoconf/build-aux/config.sub      Fri Jan 27 10:15:41
>>>> 2017 +0100
>>>> +++ b/common/autoconf/build-aux/config.sub      Fri Feb 03 05:00:25
>>>> 2017 -0700
>>>> @@ -30,7 +30,7 @@
>>>>    DIR=`dirname $0`
>>>>
>>>>    # First, filter out everything that doesn't begin with "aarch64-"
>>>> -if ! echo $* | grep '^aarch64-' >/dev/null ; then
>>>> +if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ; then
>>>>        . $DIR/autoconf-config.sub "$@"
>>>>        # autoconf-config.sub exits, so we never reach here, but just in
>>>>        # case we do:
>>>> @@ -45,6 +45,10 @@
>>>>                config=`echo $1 | sed 's/^aarch64-/arm-/'`
>>>>                sub_args="$sub_args $config"
>>>>                shift; ;;
>>>> +        *-msys )
>>>> +            config=`echo $1 | sed 's/msys/mingw32/'`
>>>> +            sub_args="$sub_args $config"
>>>> +            shift; ;;
>>>>            - )    # Use stdin as input.
>>>>                sub_args="$sub_args $1"
>>>>                shift; break ;;
>>>>
>>>> If I remember correctly, this got me past the configure stage at the
>>>> time.
>>>>
>>>> I don't think it's very hard to get it to work on msys2, I just ran
>>>> into one snag too many and didn't think msys2 would be used by anyone.
>>>>
>>>> /Magnus
>>>>
>>>> On 2017-10-03 17:20, Peter Budai wrote:
>>>>> Hello,
>>>>>
>>>>> According to
>>>>> http://hg.openjdk.java.net/jdk9/jdk9/file/a08cbfc0e4ec/common/doc/building.html
>>>>>
>>>>> “msys2 and the new Windows Subsystem for Linux (WSL) would likely be
>>>>> possible to support in a future version but that would require a
>>>>> community effort to implement”
>>>>>
>>>>> I’d like to help making the OpenJDK 9 build working on msys2. What is
>>>>> the best way to move forward? Is there a similar effort in progress?
>>>>>
>>>>> Thank you and best regards,
>>>>>
>>>>> Peter
>>>>>
>>>>>
>>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Building OpenJDK9 on MSYS2

Magnus Ihse Bursie
In reply to this post by Peter Budai

On 2017-10-05 11:59, Peter Budai wrote:

>
> Hi Magnus and Erik,
>
> I really appreciate your quick feedback. I assumed that it won’t be
> easy, but I just don’t feel I should give up now  - maybe later when I
> see the real scale of work. So bear with me for a time being.
>
> Attached is a patch which already includes Magnus’ changes, plus a few
> which I have added:
>
>   * basically enabling gcc for windows,
>   * and modifying a logic for compiling fixpath (before that it was
>     using hard-coded MS VSC compile flags)
>

Actually, you must make sure fixpath is *not* used for the toolchain,
since gcc uses unix style paths.
(However, other tools such as java will still need it.)


> So here is what I have as the result of configure:
>
> ====================================================
>
> The existing configuration has been successfully updated in
>
> /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release
>
> using configure arguments '--disable-freetype-bundling
> --disable-javac-server'.
>
> Configuration summary:
>
> * Debug level:    release
>
> * HS debug level: product
>
> * JDK variant:    normal
>
> * JVM variants:   server
>
> * OpenJDK target: OS: windows, CPU architecture: x86, address length: 64
>
> * Version string: 9-internal+0-adhoc.peterbud.jdk9 (9-internal)
>
> Tools summary:
>
> * Environment:    msys version 2.9.0(0.318/5/3) (root at /C/msys64)
>
> * Boot JDK:       java version "1.8.0_144" Java(TM) SE Runtime
> Environment (build 1.8.0_144-b01)  Java HotSpot(TM) 64-Bit Server VM
> (build 25.144-b01, mixed mode)   (at /c/progra~1/java/jdk18~1.0_1)
>
> * Toolchain:      gcc (GNU Compiler Collection)
>
> * C Compiler:     Version 7.2.0 (at /C/msys64/mingw64/bin/gcc)
>
> * C++ Compiler:   Version 7.2.0 (at /c/msys64/mingw64/bin/g++)
>
> Build performance summary:
>
> * Cores to use:   4
>
> * Memory limit:   16216 MB
>
> Its clear says that the toolchain is gcc 7.2 (BTW there is no Visual
> Studio on this machine)
>
> Now for the details of the config log, you can see here:
> https://pastebin.com/MN2ZYcHH <https://pastebin.com/MN2ZYcHH>
>
> And about the build process and the error I get:
>
> $ make JOBS=1
>
> Building target 'default (exploded-image)' in configuration
> 'windows-x86_64-normal-server-release'
>
> Compiling 8 files for BUILD_TOOLS_LANGTOOLS
>
> Compiling 17 properties into resource bundles for jdk.compiler
>
> Parsing 1 properties into enum-like class for jdk.compiler
>
> Compiling 19 properties into resource bundles for jdk.javadoc
>
> Compiling 12 properties into resource bundles for jdk.jdeps
>
> Compiling 7 properties into resource bundles for jdk.jshell
>
> Compiling 117 files for BUILD_INTERIM_java.compiler
>
> Compiling 396 files for BUILD_INTERIM_jdk.compiler
>
> Compiling 61 files for BUILD_INTERIM_jdk.jdeps
>
> Compiling 457 files for BUILD_INTERIM_jdk.javadoc
>
> Note: Some input files use or override a deprecated API.
>
> Note: Recompile with -Xlint:deprecation for details.
>
> Compiling 159 files for BUILD_TOOLS_JDK
>
> Note: Some input files use unchecked or unsafe operations.
>
> Note: Recompile with -Xlint:unchecked for details.
>
> make[3]: *** [GensrcMisc.gmk:78:
> /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java]
> Error 1
>
> make[3]: *** Deleting file
> '/C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java'
>
> make[2]: *** [make/Main.gmk:115: java.base-gensrc-jdk] Error 2
>
> ERROR: Build failed for target 'default (exploded-image)' in
> configuration 'windows-x86_64-normal-server-release' (exit code 2)
>
> No indication of failed target found.
>
> Hint: Try searching the build log for '] Error'.
>
> Hint: See common/doc/building.html#troubleshooting for assistance.
>
> make[1]: *** [/home/peterbud/jdk9/make/Init.gmk:296: main] Error 2
>
> make: *** [/home/peterbud/jdk9/make/Init.gmk:185: default] Error 2
>
> If I run here
>
> make JOBS=1 LOG=debug
>
> The failing line seems to be this:
>
> ( /usr/bin/gawk '/@@END_COPYRIGHT@@/{exit}1'
> /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template
> &&
> /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/configure-support/bin/fixpath.exe
> -m/C/msys64/@/c/msys64/@/c/progra~ /C/msys64/mingw64/bin/gcc -E -x c
> /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template
> 2> >(/usr/bin/grep -v '^SocketOptionRegistry.java.template$' >&2) |
> /usr/bin/gawk '/@@START_HERE@@/,0' |  /usr/bin/sed -e
> 's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED FILE - DO NOT EDIT/' -e
> 's/PREFIX_//' -e 's/^#.*//' ) >
> /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java
>
> make[3]: *** [GensrcMisc.gmk:78:
> /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java]
> Error 1
>
> Now the interesting is: if I copy this line above to the bash prompt,
> it runs without problem, and the file
> support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java
>
That's surprising, since gcc is prefixed with fixpath, which it should not.

I have a hard time believing this is a race condition. On the other
hand, this stuff is weird, we're misusing the C preprocessor to process
defines in java code, so I'm not suprised it breaks down. I don't know
why it succeeded when run on the command line, though. My suggestion is
to just do some quick and dirty hack around this: take the file you
manage to generate and just copy it in during the build instead. If you
can get round this, you might start seeing some *real* problems. :-)

Also, my suggestion is that you try running "make hotspot" to cut to the
chase. Compiling hotspot will likely be the hardest thing. Or even "make
-k hotspot" to get an assessment of the amount of work ahead of you.

/Magnus

> Is produced.
>
> Then I can again issue
>
> make JOBS=1 LOG=debug
>
> And the compile process is being continued, until a similar error pops
> up with a different generated file. I have an assumption that this
> happens because make is still running parallel jobs, despite JOBS=1
> but I’m not sure.
>
> How could I best tackle this?
>
> Thank you and best regards,
>
> Peter
>
> Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for
> Windows 10
>
> *From: *Magnus Ihse Bursie <mailto:[hidden email]>
> *Sent: *Thursday, October 5, 2017 11:33 AM
> *To: *Erik Joelsson <mailto:[hidden email]>; Peter Budai
> <mailto:[hidden email]>; [hidden email]
> <mailto:[hidden email]>
> *Subject: *Re: Building OpenJDK9 on MSYS2
>
> On 2017-10-05 10:10, Erik Joelsson wrote:
> > Hello Peter,
> >
> >
> > On 2017-10-04 21:15, Peter Budai wrote:
> >> Hi Magnus,
> >>
> >> Thanks for the quick reply I’ll check these patches with msys2.
> >>
> >> Let me specify with more details what I’d like to achieve: I’d like
> >> to build OpenJDK9 with MSYS2 MINGW64 environment using gcc toolchain.
> >> (I’m not sure how familiar are you with MSYS2, but there are 3
> >> different environments: MSYS2, MINGW32 and MINGW64). In theory
> >> MINGW64 with gcc is the closes you can get on Windows platform as a
> >> gcc unix like build environment, which produces still a native 64-bit
> >> executable on Windows.
> >>
> >> I’m not very familiar with OpenJDK yet, so therefore I’d like to hear
> >> your opinion: how realistic is that?
> > Sorry to disappoint, but I would say that requires major work. There
> > is a strong historic assumption that windows builds are done using
> > Visual Studio. We have abstracted away some of it in configure (see
> > TOOLCHAIN_TYPE), but it's very far from enough to change compiler
> > environment for a Windows build. The native sources are also bound to
> > make a lot of such assumptions. I would expect the changes needed to
> > be in the thousands of lines of code.
>
> I agree that it requires hard work (even if "thousands" might be an
> overestimation I think, but "hundreds" is not enough, so it's the right
> magnitude). On the other hand, it would be really good if we did sort
> things out, so that we had proper conditions based on OS vs
> compiler/toolchain.
>
> If you really want to start, the first thing is to patch toolchain.m4 to
> VALID_TOOLCHAINS_windows="microsoft gcc"
> and then call configure using "bash configure --with-toolchain-type=gcc".
>
> As Erik, I doubt you will come very far before things starts tumbling
> down.
>
> >
> > When we say supporting the build in msys2 instead of cygwin, we just
> > mean using msys2 as the unix emulating layer for our tools like
> > make/bash/grep/sed etc.
> >
> > One think I have done successfully is running the build in WSL
> > (Windows Subsystem for Linux), but that isn't all that helpful as WSL
> > for practical purposes is more or less like running Linux in a VM, so
> > the build sees a Linux system and builds a Linux binary.
> >> As a side note: with MINGW64 I have managed to run configure phase
> >> successfully for OpenJDK. The compile process has also started and
> >> went for a while, but interestingly I run into some kind of race
> >> conditions as make stopped with an error. Using LOG=debug I have fond
> >> the failing line and then copying the failed command and pasting it
> >> to the bash prompt it successfully generated the output target, and
> >> then the build process run further when a similar situation happened.
> >> Also pasting the failed command run in the bash without any problem,
> >> and build continued… until the next.
> > Without seeing the errors I can't say much. I very much doubt that you
> > are running with gcc as the compiler though. Configure isn't easily
> > fooled into using a different compiler to what it prefers, and I would
> > expect things to crash and burn pretty early if you actually did.
> >
> > /Erik
> >>
> >> I have tried to run make JOBS=1, but did not help, strangely I have
> >> still seen in the log make[3] and make[4] logs which suggested that
> >> there are more than one make jobs were running. Also tried .configure
> >> --with-output-sync=recurse without success (same symptoms)
> >>
> >> Let me know your thoughts.
> >>
> >> Best regards,
> >>
> >> Peter
> >>
> >> Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for
> >> Windows 10
> >>
> >> From: Magnus Ihse Bursie<mailto:[hidden email]>
> >> Sent: Wednesday, October 4, 2017 1:04 AM
> >> To: Peter Budai<mailto:[hidden email]>;
> >> [hidden email]<mailto:[hidden email]>
> >> Subject: Re: Building OpenJDK9 on MSYS2
> >>
> >> Actually, it wasn't so much remaining trouble. :-) I fired up msys2 and
> >> checked out where I left off. It turned out that the remaining snag was
> >> that msys2 tries to convert command lines automatically, from "unix"
> >> style paths to "windows" style paths. Unfortunately, it does not do
> this
> >> very well and it breaks all sorts of things. We already have a FIXPATH
> >> solution in place which deals with this, so basically all I had to do
> >> was disable this (by setting MSYS2_ARG_CONV_EXCL to "*"). However, this
> >> broke our cygpath replacement hack (!) so I had to disable it there.
> >> Sigh. Anyway, with those fixes it ran and worked well. (I also
> >> discovered and fixed a bug related to how we set up the FIXPATH
> variable
> >> on msys, but it only triggers in certain circumstances).
> >>
> >> With this patch I now jdk9 seems to build fine on msys2. It should
> apply
> >> cleanly on jdk9/jdk9. Since it turned out to be so trivial, I'll try to
> >> get it in in jdk10.
> >>
> >> Here's the patch if you want to apply it yourself:
> >>
> >> diff -r a08cbfc0e4ec common/autoconf/basics_windows.m4
> >> --- a/common/autoconf/basics_windows.m4    Thu Aug 03 18:56:56 2017
> >> +0000
> >> +++ b/common/autoconf/basics_windows.m4    Wed Oct 04 00:53:58 2017
> >> +0200
> >> @@ -42,7 +42,7 @@
> >>        windows_path=`$CYGPATH -m "$unix_path"`
> >>        $1="$windows_path"
> >>      elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then
> >> -    windows_path=`cmd //c echo $unix_path`
> >> +    windows_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $unix_path`
> >>        $1="$windows_path"
> >>      fi
> >>    ])
> >> @@ -136,6 +136,16 @@
> >>      fi
> >>    ])
> >>
> >> +AC_DEFUN([BASIC_MSYS_UPDATE_FIXPATH],
> >> +[
> >> +  # Take all collected prefixes and turn them into a
> >> -m/c/foo@/c/bar@... command line
> >> +  # @ was chosen as separator to minimize risk of other tools messing
> >> around with it
> >> +  all_unique_prefixes=`echo "${all_fixpath_prefixes@<:@@@:>@}" \
> >> +      | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
> >> +  fixpath_argument_list=`echo $all_unique_prefixes  | tr ' ' '@'`
> >> +  FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
> >> +])
> >> +
> >>    AC_DEFUN([BASIC_FIXUP_PATH_MSYS],
> >>    [
> >>      path="[$]$1"
> >> @@ -143,7 +153,7 @@
> >>      new_path="$path"
> >>      if test "x$has_colon" = x; then
> >>        # Not in mixed or Windows style, start by that.
> >> -    new_path=`cmd //c echo $path`
> >> +    new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $path`
> >>      fi
> >>
> >> BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
> >> @@ -155,6 +165,8 @@
> >>
> >>      # Save the first 10 bytes of this path to the storage, so fixpath
> >> can work.
> >> all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
> >> "${new_path:0:10}")
> >> +  # We might need to re-evaluate FIXPATH.
> >> +  BASIC_MSYS_UPDATE_FIXPATH
> >>    ])
> >>
> >>    AC_DEFUN([BASIC_FIXUP_EXECUTABLE_CYGWIN],
> >> @@ -293,7 +305,7 @@
> >>        # Do not save /bin paths to all_fixpath_prefixes!
> >>      else
> >>        # Not in mixed or Windows style, start by that.
> >> -    new_path=`cmd //c echo $new_path`
> >> +    new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $new_path`
> >> BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
> >>        # Output is in $new_path
> >>        BASIC_WINDOWS_REWRITE_AS_UNIX_PATH(new_path)
> >> @@ -302,6 +314,8 @@
> >>
> >>        # Save the first 10 bytes of this path to the storage, so
> fixpath
> >> can work.
> >> all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
> >> "${new_path:0:10}")
> >> +    # We might need to re-evaluate FIXPATH.
> >> +    BASIC_MSYS_UPDATE_FIXPATH
> >>      fi
> >>    ])
> >>
> >> @@ -347,6 +361,10 @@
> >>        WINDOWS_ENV_VENDOR='msys'
> >>        WINDOWS_ENV_VERSION="$MSYS_VERSION"
> >>
> >> +    # Prohibit msys2 path conversion from trying to be "intelligent",
> >> and rely
> >> +    # on fixpath instead.
> >> +    export MSYS2_ARG_CONV_EXCL="*"
> >> +
> >>        AC_MSG_CHECKING([msys root directory as unix-style path])
> >>        # The cmd output ends with Windows line endings (CR/LF), the
> grep
> >> command will strip that away
> >>        MSYS_ROOT_PATH=`cd / ; cmd /c cd | $GREP ".*"`
> >> @@ -391,10 +409,7 @@
> >>        elif test "x$OPENJDK_BUILD_OS_ENV" = xwindows.msys; then
> >>          # Take all collected prefixes and turn them into a
> >> -m/c/foo@/c/bar@... command line
> >>          # @ was chosen as separator to minimize risk of other tools
> >> messing around with it
> >> -      all_unique_prefixes=`echo "${all_fixpath_prefixes@<:@@@:>@}" \
> >> -          | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
> >> -      fixpath_argument_list=`echo $all_unique_prefixes  | tr ' ' '@'`
> >> -      FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
> >> +      BASIC_MSYS_UPDATE_FIXPATH
> >>        fi
> >>        FIXPATH_SRC_W="$FIXPATH_SRC"
> >>        FIXPATH_BIN_W="$FIXPATH_BIN"
> >> diff -r a08cbfc0e4ec common/autoconf/build-aux/config.sub
> >> --- a/common/autoconf/build-aux/config.sub    Thu Aug 03 18:56:56
> >> 2017 +0000
> >> +++ b/common/autoconf/build-aux/config.sub    Wed Oct 04 00:53:58
> >> 2017 +0200
> >> @@ -30,7 +30,7 @@
> >>    DIR=`dirname $0`
> >>
> >>    # First, filter out everything that doesn't begin with "aarch64-"
> >> -if ! echo $* | grep '^aarch64-' >/dev/null ; then
> >> +if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ; then
> >>        . $DIR/autoconf-config.sub "$@"
> >>        # autoconf-config.sub exits, so we never reach here, but just in
> >>        # case we do:
> >> @@ -38,13 +38,17 @@
> >>    fi
> >>
> >>    while test $# -gt 0 ; do
> >> -    case $1 in
> >> +    case $1 in
> >>            -- )   # Stop option processing
> >>                shift; break ;;
> >>            aarch64-* )
> >>                config=`echo $1 | sed 's/^aarch64-/arm-/'`
> >>                sub_args="$sub_args $config"
> >>                shift; ;;
> >> +        *-msys )
> >> +            config=`echo $1 | sed 's/msys/mingw32/'`
> >> +            sub_args="$sub_args $config"
> >> +            shift; ;;
> >>            - )    # Use stdin as input.
> >>                sub_args="$sub_args $1"
> >>                shift; break ;;
> >> diff -r a08cbfc0e4ec common/autoconf/spec.gmk.in
> >> --- a/common/autoconf/spec.gmk.in    Thu Aug 03 18:56:56 2017 +0000
> >> +++ b/common/autoconf/spec.gmk.in    Wed Oct 04 00:53:58 2017 +0200
> >> @@ -120,6 +120,13 @@
> >>      # On Windows, the Visual Studio toolchain needs the PATH to be
> >> adjusted
> >>      # to include Visual Studio tools (this needs to be in cygwin/msys
> >> style).
> >>      export PATH:=@VS_PATH@
> >> +
> >> +endif
> >> +
> >> +ifeq ($(OPENJDK_TARGET_OS_ENV), windows.msys)
> >> +  # On msys2, prohibit msys path conversion from trying to be
> >> +  # "intelligent", and rely on fixpath instead.
> >> +  export MSYS2_ARG_CONV_EXCL:=*
> >>    endif
> >>
> >>    SYSROOT_CFLAGS := @SYSROOT_CFLAGS@
> >>
> >> /Magnus
> >>
> >> On 2017-10-03 22:34, Magnus Ihse Bursie wrote:
> >>> I gave msys2 a shot some time ago, but it ended up too much trouble.
> >>> I'll share some of my notes from that attempt, for what it's worth.
> >>>
> >>> To install package X/Y, run "pacman -S X/Y". Missing tools and
> >>> packages where to find them:
> >>> cmp: msys/diffutils
> >>> tar: msys/tar
> >>> make: msys/make
> >>> unzip: msys/unzip
> >>> zip: msys/zip
> >>>
> >>> config.sub reports msys as "x86_64-pc-mingw32" but msys2 as
> >>> "x86_64-pc-msys". This patch adds postprocessing in "our" config.sub
> >>> to report msys2 similar to msys. (Opinions, including my own :-) may
> >>> vary if this really is the best way..)
> >>>
> >>> diff -r b88023f46daa common/autoconf/build-aux/config.sub
> >>> --- a/common/autoconf/build-aux/config.sub Fri Jan 27 10:15:41
> >>> 2017 +0100
> >>> +++ b/common/autoconf/build-aux/config.sub Fri Feb 03 05:00:25
> >>> 2017 -0700
> >>> @@ -30,7 +30,7 @@
> >>>   DIR=`dirname $0`
> >>>
> >>>   # First, filter out everything that doesn't begin with "aarch64-"
> >>> -if ! echo $* | grep '^aarch64-' >/dev/null ; then
> >>> +if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ; then
> >>>       . $DIR/autoconf-config.sub "$@"
> >>>       # autoconf-config.sub exits, so we never reach here, but just in
> >>>       # case we do:
> >>> @@ -45,6 +45,10 @@
> >>>               config=`echo $1 | sed 's/^aarch64-/arm-/'`
> >>>               sub_args="$sub_args $config"
> >>>               shift; ;;
> >>> +        *-msys )
> >>> +            config=`echo $1 | sed 's/msys/mingw32/'`
> >>> +            sub_args="$sub_args $config"
> >>> +            shift; ;;
> >>>           - )    # Use stdin as input.
> >>>               sub_args="$sub_args $1"
> >>>               shift; break ;;
> >>>
> >>> If I remember correctly, this got me past the configure stage at the
> >>> time.
> >>>
> >>> I don't think it's very hard to get it to work on msys2, I just ran
> >>> into one snag too many and didn't think msys2 would be used by anyone.
> >>>
> >>> /Magnus
> >>>
> >>> On 2017-10-03 17:20, Peter Budai wrote:
> >>>> Hello,
> >>>>
> >>>> According to
> >>>>
> http://hg.openjdk.java.net/jdk9/jdk9/file/a08cbfc0e4ec/common/doc/building.html 
>
> >>>>
> >>>> “msys2 and the new Windows Subsystem for Linux (WSL) would likely be
> >>>> possible to support in a future version but that would require a
> >>>> community effort to implement”
> >>>>
> >>>> I’d like to help making the OpenJDK 9 build working on msys2. What is
> >>>> the best way to move forward? Is there a similar effort in progress?
> >>>>
> >>>> Thank you and best regards,
> >>>>
> >>>> Peter
> >>>>
> >>>>
> >
>

Reply | Threaded
Open this post in threaded view
|

RE: Building OpenJDK9 on MSYS2

Peter Budai
Hi Magnus,

So first of all, here is the current patch, which I was not able to attach: https://pastebin.com/pwT4Ynxc

>> That's surprising, since gcc is prefixed with fixpath, which it should not.
Actually you DO need fixpath IMHO.
This is a mingw64 version of the gcc (/C/msys64/mingw64/bin/gcc), which is a fully functional Windows executable, which expects Windows formatted path arguments.
As the updated build process uses EXPORT MSYS2_ARG_CONV_EXCL=* (see that patch), none of the command line arguments are converted  from the unix path to Windows, but fixpath does that conversion. There is a wiki describing more details on this:
https://github.com/msys2/msys2/wiki/Porting#user-content-filesystem-namespaces



>>I have a hard time believing this is a race condition. On the other hand, this stuff is weird, we're misusing the C preprocessor to process defines in java code, so I'm not surprised it breaks down.
>>I don't know why it succeeded when run on the command line, though.
When I execute that command from the bash command line there is no EXPORT MSYS2_ARG_CONV_EXCL, but the bash itself does the automatic conversion of the arguments. Maybe it has something to do how fixpath does CreateProcess?

Does that help?

Best regards,

Peter

Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10

From: Magnus Ihse Bursie<mailto:[hidden email]>
Sent: Thursday, October 5, 2017 12:13 PM
To: Peter Budai<mailto:[hidden email]>; Erik Joelsson<mailto:[hidden email]>; [hidden email]<mailto:[hidden email]>
Subject: Re: Building OpenJDK9 on MSYS2


On 2017-10-05 11:59, Peter Budai wrote:
Hi Magnus and Erik,

I really appreciate your quick feedback. I assumed that it won’t be easy, but I just don’t feel I should give up now  - maybe later when I see the real scale of work. So bear with me for a time being.

Attached is a patch which already includes Magnus’ changes, plus a few which I have added:
·       basically enabling gcc for windows,
·       and modifying a logic for compiling fixpath (before that it was using hard-coded MS VSC compile flags)

Actually, you must make sure fixpath is *not* used for the toolchain, since gcc uses unix style paths.
(However, other tools such as java will still need it.)





So here is what I have as the result of configure:
====================================================
The existing configuration has been successfully updated in
/C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release
using configure arguments '--disable-freetype-bundling --disable-javac-server'.

Configuration summary:
* Debug level:    release
* HS debug level: product
* JDK variant:    normal
* JVM variants:   server
* OpenJDK target: OS: windows, CPU architecture: x86, address length: 64
* Version string: 9-internal+0-adhoc.peterbud.jdk9 (9-internal)

Tools summary:
* Environment:    msys version 2.9.0(0.318/5/3) (root at /C/msys64)
* Boot JDK:       java version "1.8.0_144"  Java(TM) SE Runtime Environment (build 1.8.0_144-b01)  Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)   (at /c/progra~1/java/jdk18~1.0_1)
* Toolchain:      gcc (GNU Compiler Collection)
* C Compiler:     Version 7.2.0 (at /C/msys64/mingw64/bin/gcc)
* C++ Compiler:   Version 7.2.0 (at /c/msys64/mingw64/bin/g++)

Build performance summary:
* Cores to use:   4
* Memory limit:   16216 MB

Its clear says that the toolchain is gcc 7.2 (BTW there is no Visual Studio on this machine)

Now for the details of the config log, you can see here: https://pastebin.com/MN2ZYcHH

And about the build process and the error I get:

$ make JOBS=1
Building target 'default (exploded-image)' in configuration 'windows-x86_64-normal-server-release'
Compiling 8 files for BUILD_TOOLS_LANGTOOLS
Compiling 17 properties into resource bundles for jdk.compiler
Parsing 1 properties into enum-like class for jdk.compiler
Compiling 19 properties into resource bundles for jdk.javadoc
Compiling 12 properties into resource bundles for jdk.jdeps
Compiling 7 properties into resource bundles for jdk.jshell
Compiling 117 files for BUILD_INTERIM_java.compiler
Compiling 396 files for BUILD_INTERIM_jdk.compiler
Compiling 61 files for BUILD_INTERIM_jdk.jdeps
Compiling 457 files for BUILD_INTERIM_jdk.javadoc
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Compiling 159 files for BUILD_TOOLS_JDK
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
make[3]: *** [GensrcMisc.gmk:78: /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java] Error 1
make[3]: *** Deleting file '/C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java'
make[2]: *** [make/Main.gmk:115: java.base-gensrc-jdk] Error 2

ERROR: Build failed for target 'default (exploded-image)' in configuration 'windows-x86_64-normal-server-release' (exit code 2)

No indication of failed target found.
Hint: Try searching the build log for '] Error'.
Hint: See common/doc/building.html#troubleshooting for assistance.

make[1]: *** [/home/peterbud/jdk9/make/Init.gmk:296: main] Error 2
make: *** [/home/peterbud/jdk9/make/Init.gmk:185: default] Error 2

If I run here
make JOBS=1 LOG=debug
The failing line seems to be this:

( /usr/bin/gawk '/@@END_COPYRIGHT@@/{exit}1' /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template && /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/configure-support/bin/fixpath.exe -m/C/msys64/@/c/msys64/@/c/progra~ /C/msys64/mingw64/bin/gcc -E -x c  /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template 2> >(/usr/bin/grep -v '^SocketOptionRegistry.java.template$' >&2) | /usr/bin/gawk '/@@START_HERE@@/,0' |  /usr/bin/sed -e 's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED FILE - DO NOT EDIT/' -e 's/PREFIX_//' -e 's/^#.*//' ) > /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java
make[3]: *** [GensrcMisc.gmk:78: /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java] Error 1

Now the interesting is: if I copy this line above to the bash prompt, it runs without problem, and the file support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java
That's surprising, since gcc is prefixed with fixpath, which it should not.

I have a hard time believing this is a race condition. On the other hand, this stuff is weird, we're misusing the C preprocessor to process defines in java code, so I'm not suprised it breaks down. I don't know why it succeeded when run on the command line, though. My suggestion is to just do some quick and dirty hack around this: take the file you manage to generate and just copy it in during the build instead. If you can get round this, you might start seeing some *real* problems. :-)

Also, my suggestion is that you try running "make hotspot" to cut to the chase. Compiling hotspot will likely be the hardest thing. Or even "make -k hotspot" to get an assessment of the amount of work ahead of you.

/Magnus


Is produced.

Then I can again issue
make JOBS=1 LOG=debug

And the compile process is being continued, until a similar error pops up with a different generated file. I have an assumption that this happens because make is still running parallel jobs, despite JOBS=1 but I’m not sure.

How could I best tackle this?

Thank you and best regards,

Peter

Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10

From: Magnus Ihse Bursie<mailto:[hidden email]>
Sent: Thursday, October 5, 2017 11:33 AM
To: Erik Joelsson<mailto:[hidden email]>; Peter Budai<mailto:[hidden email]>; [hidden email]<mailto:[hidden email]>
Subject: Re: Building OpenJDK9 on MSYS2

On 2017-10-05 10:10, Erik Joelsson wrote:

> Hello Peter,
>
>
> On 2017-10-04 21:15, Peter Budai wrote:
>> Hi Magnus,
>>
>> Thanks for the quick reply I’ll check these patches with msys2.
>>
>> Let me specify with more details what I’d like to achieve: I’d like
>> to build OpenJDK9 with MSYS2 MINGW64 environment using gcc toolchain.
>> (I’m not sure how familiar are you with MSYS2, but there are 3
>> different environments: MSYS2, MINGW32 and MINGW64). In theory
>> MINGW64 with gcc is the closes you can get on Windows platform as a
>> gcc unix like build environment, which produces still a native 64-bit
>> executable on Windows.
>>
>> I’m not very familiar with OpenJDK yet, so therefore I’d like to hear
>> your opinion: how realistic is that?
> Sorry to disappoint, but I would say that requires major work. There
> is a strong historic assumption that windows builds are done using
> Visual Studio. We have abstracted away some of it in configure (see
> TOOLCHAIN_TYPE), but it's very far from enough to change compiler
> environment for a Windows build. The native sources are also bound to
> make a lot of such assumptions. I would expect the changes needed to
> be in the thousands of lines of code.

I agree that it requires hard work (even if "thousands" might be an
overestimation I think, but "hundreds" is not enough, so it's the right
magnitude). On the other hand, it would be really good if we did sort
things out, so that we had proper conditions based on OS vs
compiler/toolchain.

If you really want to start, the first thing is to patch toolchain.m4 to
VALID_TOOLCHAINS_windows="microsoft gcc"
and then call configure using "bash configure --with-toolchain-type=gcc".

As Erik, I doubt you will come very far before things starts tumbling down.

>
> When we say supporting the build in msys2 instead of cygwin, we just
> mean using msys2 as the unix emulating layer for our tools like
> make/bash/grep/sed etc.
>
> One think I have done successfully is running the build in WSL
> (Windows Subsystem for Linux), but that isn't all that helpful as WSL
> for practical purposes is more or less like running Linux in a VM, so
> the build sees a Linux system and builds a Linux binary.
>> As a side note: with MINGW64 I have managed to run configure phase
>> successfully for OpenJDK. The compile process has also started and
>> went for a while, but interestingly I run into some kind of race
>> conditions as make stopped with an error. Using LOG=debug I have fond
>> the failing line and then copying the failed command and pasting it
>> to the bash prompt it successfully generated the output target, and
>> then the build process run further when a similar situation happened.
>> Also pasting the failed command run in the bash without any problem,
>> and build continued… until the next.
> Without seeing the errors I can't say much. I very much doubt that you
> are running with gcc as the compiler though. Configure isn't easily
> fooled into using a different compiler to what it prefers, and I would
> expect things to crash and burn pretty early if you actually did.
>
> /Erik
>>
>> I have tried to run make JOBS=1, but did not help, strangely I have
>> still seen in the log make[3] and make[4] logs which suggested that
>> there are more than one make jobs were running. Also tried .configure
>> --with-output-sync=recurse without success (same symptoms)
>>
>> Let me know your thoughts.
>>
>> Best regards,
>>
>> Peter
>>
>> Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for
>> Windows 10
>>
>> From: Magnus Ihse Bursie<mailto:[hidden email]>
>> Sent: Wednesday, October 4, 2017 1:04 AM
>> To: Peter Budai<mailto:[hidden email]>;
>> [hidden email]<mailto:[hidden email]><mailto:[hidden email]><mailto:[hidden email]>
>> Subject: Re: Building OpenJDK9 on MSYS2
>>
>> Actually, it wasn't so much remaining trouble. :-) I fired up msys2 and
>> checked out where I left off. It turned out that the remaining snag was
>> that msys2 tries to convert command lines automatically, from "unix"
>> style paths to "windows" style paths. Unfortunately, it does not do this
>> very well and it breaks all sorts of things. We already have a FIXPATH
>> solution in place which deals with this, so basically all I had to do
>> was disable this (by setting MSYS2_ARG_CONV_EXCL to "*"). However, this
>> broke our cygpath replacement hack (!) so I had to disable it there.
>> Sigh. Anyway, with those fixes it ran and worked well. (I also
>> discovered and fixed a bug related to how we set up the FIXPATH variable
>> on msys, but it only triggers in certain circumstances).
>>
>> With this patch I now jdk9 seems to build fine on msys2. It should apply
>> cleanly on jdk9/jdk9. Since it turned out to be so trivial, I'll try to
>> get it in in jdk10.
>>
>> Here's the patch if you want to apply it yourself:
>>
>> diff -r a08cbfc0e4ec common/autoconf/basics_windows.m4
>> --- a/common/autoconf/basics_windows.m4    Thu Aug 03 18:56:56 2017
>> +0000
>> +++ b/common/autoconf/basics_windows.m4    Wed Oct 04 00:53:58 2017
>> +0200
>> @@ -42,7 +42,7 @@
>>        windows_path=`$CYGPATH -m "$unix_path"`
>>        $1="$windows_path"
>>      elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then
>> -    windows_path=`cmd //c echo $unix_path`
>> +    windows_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $unix_path`
>>        $1="$windows_path"
>>      fi
>>    ])
>> @@ -136,6 +136,16 @@
>>      fi
>>    ])
>>
>> +AC_DEFUN([BASIC_MSYS_UPDATE_FIXPATH],
>> +[
>> +  # Take all collected prefixes and turn them into a
>> -m/c/foo@/c/bar@... command line
>> +  # @ was chosen as separator to minimize risk of other tools messing
>> around with it
>> +  all_unique_prefixes=`echo "${all_fixpath_prefixes@<:@@@:>@}" \
>> +      | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
>> +  fixpath_argument_list=`echo $all_unique_prefixes  | tr ' ' '@'`
>> +  FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
>> +])
>> +
>>    AC_DEFUN([BASIC_FIXUP_PATH_MSYS],
>>    [
>>      path="[$]$1"
>> @@ -143,7 +153,7 @@
>>      new_path="$path"
>>      if test "x$has_colon" = x; then
>>        # Not in mixed or Windows style, start by that.
>> -    new_path=`cmd //c echo $path`
>> +    new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $path`
>>      fi
>>
>>      BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
>> @@ -155,6 +165,8 @@
>>
>>      # Save the first 10 bytes of this path to the storage, so fixpath
>> can work.
>> all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
>> "${new_path:0:10}")
>> +  # We might need to re-evaluate FIXPATH.
>> +  BASIC_MSYS_UPDATE_FIXPATH
>>    ])
>>
>>    AC_DEFUN([BASIC_FIXUP_EXECUTABLE_CYGWIN],
>> @@ -293,7 +305,7 @@
>>        # Do not save /bin paths to all_fixpath_prefixes!
>>      else
>>        # Not in mixed or Windows style, start by that.
>> -    new_path=`cmd //c echo $new_path`
>> +    new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $new_path`
>>        BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
>>        # Output is in $new_path
>>        BASIC_WINDOWS_REWRITE_AS_UNIX_PATH(new_path)
>> @@ -302,6 +314,8 @@
>>
>>        # Save the first 10 bytes of this path to the storage, so fixpath
>> can work.
>> all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
>> "${new_path:0:10}")
>> +    # We might need to re-evaluate FIXPATH.
>> +    BASIC_MSYS_UPDATE_FIXPATH
>>      fi
>>    ])
>>
>> @@ -347,6 +361,10 @@
>>        WINDOWS_ENV_VENDOR='msys'
>>        WINDOWS_ENV_VERSION="$MSYS_VERSION"
>>
>> +    # Prohibit msys2 path conversion from trying to be "intelligent",
>> and rely
>> +    # on fixpath instead.
>> +    export MSYS2_ARG_CONV_EXCL="*"
>> +
>>        AC_MSG_CHECKING([msys root directory as unix-style path])
>>        # The cmd output ends with Windows line endings (CR/LF), the grep
>> command will strip that away
>>        MSYS_ROOT_PATH=`cd / ; cmd /c cd | $GREP ".*"`
>> @@ -391,10 +409,7 @@
>>        elif test "x$OPENJDK_BUILD_OS_ENV" = xwindows.msys; then
>>          # Take all collected prefixes and turn them into a
>> -m/c/foo@/c/bar@... command line
>>          # @ was chosen as separator to minimize risk of other tools
>> messing around with it
>> -      all_unique_prefixes=`echo "${all_fixpath_prefixes@<:@@@:>@}" \
>> -          | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
>> -      fixpath_argument_list=`echo $all_unique_prefixes  | tr ' ' '@'`
>> -      FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
>> +      BASIC_MSYS_UPDATE_FIXPATH
>>        fi
>>        FIXPATH_SRC_W="$FIXPATH_SRC"
>>        FIXPATH_BIN_W="$FIXPATH_BIN"
>> diff -r a08cbfc0e4ec common/autoconf/build-aux/config.sub
>> --- a/common/autoconf/build-aux/config.sub    Thu Aug 03 18:56:56
>> 2017 +0000
>> +++ b/common/autoconf/build-aux/config.sub    Wed Oct 04 00:53:58
>> 2017 +0200
>> @@ -30,7 +30,7 @@
>>    DIR=`dirname $0`
>>
>>    # First, filter out everything that doesn't begin with "aarch64-"
>> -if ! echo $* | grep '^aarch64-' >/dev/null ; then
>> +if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ; then
>>        . $DIR/autoconf-config.sub "$@"
>>        # autoconf-config.sub exits, so we never reach here, but just in
>>        # case we do:
>> @@ -38,13 +38,17 @@
>>    fi
>>
>>    while test $# -gt 0 ; do
>> -    case $1 in
>> +    case $1 in
>>            -- )   # Stop option processing
>>                shift; break ;;
>>            aarch64-* )
>>                config=`echo $1 | sed 's/^aarch64-/arm-/'`
>>                sub_args="$sub_args $config"
>>                shift; ;;
>> +        *-msys )
>> +            config=`echo $1 | sed 's/msys/mingw32/'`
>> +            sub_args="$sub_args $config"
>> +            shift; ;;
>>            - )    # Use stdin as input.
>>                sub_args="$sub_args $1"
>>                shift; break ;;
>> diff -r a08cbfc0e4ec common/autoconf/spec.gmk.in
>> --- a/common/autoconf/spec.gmk.in    Thu Aug 03 18:56:56 2017 +0000
>> +++ b/common/autoconf/spec.gmk.in    Wed Oct 04 00:53:58 2017 +0200
>> @@ -120,6 +120,13 @@
>>      # On Windows, the Visual Studio toolchain needs the PATH to be
>> adjusted
>>      # to include Visual Studio tools (this needs to be in cygwin/msys
>> style).
>>      export PATH:=@VS_PATH@
>> +
>> +endif
>> +
>> +ifeq ($(OPENJDK_TARGET_OS_ENV), windows.msys)
>> +  # On msys2, prohibit msys path conversion from trying to be
>> +  # "intelligent", and rely on fixpath instead.
>> +  export MSYS2_ARG_CONV_EXCL:=*
>>    endif
>>
>>    SYSROOT_CFLAGS := @SYSROOT_CFLAGS@
>>
>> /Magnus
>>
>> On 2017-10-03 22:34, Magnus Ihse Bursie wrote:
>>> I gave msys2 a shot some time ago, but it ended up too much trouble.
>>> I'll share some of my notes from that attempt, for what it's worth.
>>>
>>> To install package X/Y, run "pacman -S X/Y". Missing tools and
>>> packages where to find them:
>>> cmp: msys/diffutils
>>> tar: msys/tar
>>> make: msys/make
>>> unzip: msys/unzip
>>> zip: msys/zip
>>>
>>> config.sub reports msys as "x86_64-pc-mingw32" but msys2 as
>>> "x86_64-pc-msys". This patch adds postprocessing in "our" config.sub
>>> to report msys2 similar to msys. (Opinions, including my own :-) may
>>> vary if this really is the best way..)
>>>
>>> diff -r b88023f46daa common/autoconf/build-aux/config.sub
>>> --- a/common/autoconf/build-aux/config.sub      Fri Jan 27 10:15:41
>>> 2017 +0100
>>> +++ b/common/autoconf/build-aux/config.sub      Fri Feb 03 05:00:25
>>> 2017 -0700
>>> @@ -30,7 +30,7 @@
>>>   DIR=`dirname $0`
>>>
>>>   # First, filter out everything that doesn't begin with "aarch64-"
>>> -if ! echo $* | grep '^aarch64-' >/dev/null ; then
>>> +if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ; then
>>>       . $DIR/autoconf-config.sub "$@"
>>>       # autoconf-config.sub exits, so we never reach here, but just in
>>>       # case we do:
>>> @@ -45,6 +45,10 @@
>>>               config=`echo $1 | sed 's/^aarch64-/arm-/'`
>>>               sub_args="$sub_args $config"
>>>               shift; ;;
>>> +        *-msys )
>>> +            config=`echo $1 | sed 's/msys/mingw32/'`
>>> +            sub_args="$sub_args $config"
>>> +            shift; ;;
>>>           - )    # Use stdin as input.
>>>               sub_args="$sub_args $1"
>>>               shift; break ;;
>>>
>>> If I remember correctly, this got me past the configure stage at the
>>> time.
>>>
>>> I don't think it's very hard to get it to work on msys2, I just ran
>>> into one snag too many and didn't think msys2 would be used by anyone.
>>>
>>> /Magnus
>>>
>>> On 2017-10-03 17:20, Peter Budai wrote:
>>>> Hello,
>>>>
>>>> According to
>>>> http://hg.openjdk.java.net/jdk9/jdk9/file/a08cbfc0e4ec/common/doc/building.html
>>>>
>>>> “msys2 and the new Windows Subsystem for Linux (WSL) would likely be
>>>> possible to support in a future version but that would require a
>>>> community effort to implement”
>>>>
>>>> I’d like to help making the OpenJDK 9 build working on msys2. What is
>>>> the best way to move forward? Is there a similar effort in progress?
>>>>
>>>> Thank you and best regards,
>>>>
>>>> Peter
>>>>
>>>>
>



Reply | Threaded
Open this post in threaded view
|

RE: Building OpenJDK9 on MSYS2

Peter Budai
Magnus,

I have followed your suggestion and removed the fixpath prefixes from gcc related compile tools, and left only the fixpath prefix _only_ for the Boot JDK related tools in place.


  1.  As  I follow the process, all java and javac related compile steps are running properly
  2.  When the process reaches gcc related steps I got the error message at the same place as before (no fixpath). If I execute that command from the bash prompt, it creates the output:

$ ( /usr/bin/gawk '/@@END_COPYRIGHT@@/{exit}1' /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template && /C/msys64/mingw64/bin/gcc -E -x c  /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template 2> >(/usr/bin/grep -v '^SocketOptionRegistry.java.template$' >&2) | /usr/bin/gawk '/@@START_HERE@@/,0' |  /usr/bin/sed -e 's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED FILE - DO NOT EDIT/' -e 's/PREFIX_//' -e 's/^#.*//' ) > /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java

As I have mentioned the parameters are replaced by the bash automatically

  1.  Then build continues, then little later stops at a super simple command:

mv /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBuffer.java.tmp /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBuffer.java
           Needless to say, the ByteBuffer.java.tmp file DOES exist. And running the above command from the bash works, and build continues.

  1.  A few similar cases (stops) with DirectByteBuffer and DirectByteBufferR


Currently I try to explore how that might relate to the MSYS2 bash and make, somehow it behaves differently

If you have any other suggestion, let me know.

Best regards,

Peter

From: Peter Budai<mailto:[hidden email]>
Sent: Thursday, October 5, 2017 3:52 PM
To: Magnus Ihse Bursie<mailto:[hidden email]>; Erik Joelsson<mailto:[hidden email]>; [hidden email]<mailto:[hidden email]>
Subject: RE: Building OpenJDK9 on MSYS2

Hi Magnus,

So first of all, here is the current patch, which I was not able to attach: https://pastebin.com/pwT4Ynxc

>> That's surprising, since gcc is prefixed with fixpath, which it should not.
Actually you DO need fixpath IMHO.
This is a mingw64 version of the gcc (/C/msys64/mingw64/bin/gcc), which is a fully functional Windows executable, which expects Windows formatted path arguments.
As the updated build process uses EXPORT MSYS2_ARG_CONV_EXCL=* (see that patch), none of the command line arguments are converted  from the unix path to Windows, but fixpath does that conversion. There is a wiki describing more details on this:
https://github.com/msys2/msys2/wiki/Porting#user-content-filesystem-namespaces



>>I have a hard time believing this is a race condition. On the other hand, this stuff is weird, we're misusing the C preprocessor to process defines in java code, so I'm not surprised it breaks down.
>>I don't know why it succeeded when run on the command line, though.
When I execute that command from the bash command line there is no EXPORT MSYS2_ARG_CONV_EXCL, but the bash itself does the automatic conversion of the arguments. Maybe it has something to do how fixpath does CreateProcess?

Does that help?

Best regards,

Peter

Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10

From: Magnus Ihse Bursie<mailto:[hidden email]>
Sent: Thursday, October 5, 2017 12:13 PM
To: Peter Budai<mailto:[hidden email]>; Erik Joelsson<mailto:[hidden email]>; [hidden email]<mailto:[hidden email]>
Subject: Re: Building OpenJDK9 on MSYS2


On 2017-10-05 11:59, Peter Budai wrote:
Hi Magnus and Erik,

I really appreciate your quick feedback. I assumed that it won’t be easy, but I just don’t feel I should give up now  - maybe later when I see the real scale of work. So bear with me for a time being.

Attached is a patch which already includes Magnus’ changes, plus a few which I have added:
·       basically enabling gcc for windows,
·       and modifying a logic for compiling fixpath (before that it was using hard-coded MS VSC compile flags)

Actually, you must make sure fixpath is *not* used for the toolchain, since gcc uses unix style paths.
(However, other tools such as java will still need it.)




So here is what I have as the result of configure:
====================================================
The existing configuration has been successfully updated in
/C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release
using configure arguments '--disable-freetype-bundling --disable-javac-server'.

Configuration summary:
* Debug level:    release
* HS debug level: product
* JDK variant:    normal
* JVM variants:   server
* OpenJDK target: OS: windows, CPU architecture: x86, address length: 64
* Version string: 9-internal+0-adhoc.peterbud.jdk9 (9-internal)

Tools summary:
* Environment:    msys version 2.9.0(0.318/5/3) (root at /C/msys64)
* Boot JDK:       java version "1.8.0_144"  Java(TM) SE Runtime Environment (build 1.8.0_144-b01)  Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)   (at /c/progra~1/java/jdk18~1.0_1)
* Toolchain:      gcc (GNU Compiler Collection)
* C Compiler:     Version 7.2.0 (at /C/msys64/mingw64/bin/gcc)
* C++ Compiler:   Version 7.2.0 (at /c/msys64/mingw64/bin/g++)

Build performance summary:
* Cores to use:   4
* Memory limit:   16216 MB

Its clear says that the toolchain is gcc 7.2 (BTW there is no Visual Studio on this machine)

Now for the details of the config log, you can see here: https://pastebin.com/MN2ZYcHH

And about the build process and the error I get:

$ make JOBS=1
Building target 'default (exploded-image)' in configuration 'windows-x86_64-normal-server-release'
Compiling 8 files for BUILD_TOOLS_LANGTOOLS
Compiling 17 properties into resource bundles for jdk.compiler
Parsing 1 properties into enum-like class for jdk.compiler
Compiling 19 properties into resource bundles for jdk.javadoc
Compiling 12 properties into resource bundles for jdk.jdeps
Compiling 7 properties into resource bundles for jdk.jshell
Compiling 117 files for BUILD_INTERIM_java.compiler
Compiling 396 files for BUILD_INTERIM_jdk.compiler
Compiling 61 files for BUILD_INTERIM_jdk.jdeps
Compiling 457 files for BUILD_INTERIM_jdk.javadoc
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Compiling 159 files for BUILD_TOOLS_JDK
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
make[3]: *** [GensrcMisc.gmk:78: /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java] Error 1
make[3]: *** Deleting file '/C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java'
make[2]: *** [make/Main.gmk:115: java.base-gensrc-jdk] Error 2

ERROR: Build failed for target 'default (exploded-image)' in configuration 'windows-x86_64-normal-server-release' (exit code 2)

No indication of failed target found.
Hint: Try searching the build log for '] Error'.
Hint: See common/doc/building.html#troubleshooting for assistance.

make[1]: *** [/home/peterbud/jdk9/make/Init.gmk:296: main] Error 2
make: *** [/home/peterbud/jdk9/make/Init.gmk:185: default] Error 2

If I run here
make JOBS=1 LOG=debug
The failing line seems to be this:

( /usr/bin/gawk '/@@END_COPYRIGHT@@/{exit}1' /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template && /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/configure-support/bin/fixpath.exe -m/C/msys64/@/c/msys64/@/c/progra~ /C/msys64/mingw64/bin/gcc -E -x c  /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template 2> >(/usr/bin/grep -v '^SocketOptionRegistry.java.template$' >&2) | /usr/bin/gawk '/@@START_HERE@@/,0' |  /usr/bin/sed -e 's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED FILE - DO NOT EDIT/' -e 's/PREFIX_//' -e 's/^#.*//' ) > /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java
make[3]: *** [GensrcMisc.gmk:78: /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java] Error 1

Now the interesting is: if I copy this line above to the bash prompt, it runs without problem, and the file support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java
That's surprising, since gcc is prefixed with fixpath, which it should not.

I have a hard time believing this is a race condition. On the other hand, this stuff is weird, we're misusing the C preprocessor to process defines in java code, so I'm not suprised it breaks down. I don't know why it succeeded when run on the command line, though. My suggestion is to just do some quick and dirty hack around this: take the file you manage to generate and just copy it in during the build instead. If you can get round this, you might start seeing some *real* problems. :-)

Also, my suggestion is that you try running "make hotspot" to cut to the chase. Compiling hotspot will likely be the hardest thing. Or even "make -k hotspot" to get an assessment of the amount of work ahead of you.

/Magnus

Is produced.

Then I can again issue
make JOBS=1 LOG=debug

And the compile process is being continued, until a similar error pops up with a different generated file. I have an assumption that this happens because make is still running parallel jobs, despite JOBS=1 but I’m not sure.

How could I best tackle this?

Thank you and best regards,

Peter

Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10

From: Magnus Ihse Bursie<mailto:[hidden email]>
Sent: Thursday, October 5, 2017 11:33 AM
To: Erik Joelsson<mailto:[hidden email]>; Peter Budai<mailto:[hidden email]>; [hidden email]<mailto:[hidden email]>
Subject: Re: Building OpenJDK9 on MSYS2

On 2017-10-05 10:10, Erik Joelsson wrote:

> Hello Peter,
>
>
> On 2017-10-04 21:15, Peter Budai wrote:
>> Hi Magnus,
>>
>> Thanks for the quick reply I’ll check these patches with msys2.
>>
>> Let me specify with more details what I’d like to achieve: I’d like
>> to build OpenJDK9 with MSYS2 MINGW64 environment using gcc toolchain.
>> (I’m not sure how familiar are you with MSYS2, but there are 3
>> different environments: MSYS2, MINGW32 and MINGW64). In theory
>> MINGW64 with gcc is the closes you can get on Windows platform as a
>> gcc unix like build environment, which produces still a native 64-bit
>> executable on Windows.
>>
>> I’m not very familiar with OpenJDK yet, so therefore I’d like to hear
>> your opinion: how realistic is that?
> Sorry to disappoint, but I would say that requires major work. There
> is a strong historic assumption that windows builds are done using
> Visual Studio. We have abstracted away some of it in configure (see
> TOOLCHAIN_TYPE), but it's very far from enough to change compiler
> environment for a Windows build. The native sources are also bound to
> make a lot of such assumptions. I would expect the changes needed to
> be in the thousands of lines of code.

I agree that it requires hard work (even if "thousands" might be an
overestimation I think, but "hundreds" is not enough, so it's the right
magnitude). On the other hand, it would be really good if we did sort
things out, so that we had proper conditions based on OS vs
compiler/toolchain.

If you really want to start, the first thing is to patch toolchain.m4 to
VALID_TOOLCHAINS_windows="microsoft gcc"
and then call configure using "bash configure --with-toolchain-type=gcc".

As Erik, I doubt you will come very far before things starts tumbling down.

>
> When we say supporting the build in msys2 instead of cygwin, we just
> mean using msys2 as the unix emulating layer for our tools like
> make/bash/grep/sed etc.
>
> One think I have done successfully is running the build in WSL
> (Windows Subsystem for Linux), but that isn't all that helpful as WSL
> for practical purposes is more or less like running Linux in a VM, so
> the build sees a Linux system and builds a Linux binary.
>> As a side note: with MINGW64 I have managed to run configure phase
>> successfully for OpenJDK. The compile process has also started and
>> went for a while, but interestingly I run into some kind of race
>> conditions as make stopped with an error. Using LOG=debug I have fond
>> the failing line and then copying the failed command and pasting it
>> to the bash prompt it successfully generated the output target, and
>> then the build process run further when a similar situation happened.
>> Also pasting the failed command run in the bash without any problem,
>> and build continued… until the next.
> Without seeing the errors I can't say much. I very much doubt that you
> are running with gcc as the compiler though. Configure isn't easily
> fooled into using a different compiler to what it prefers, and I would
> expect things to crash and burn pretty early if you actually did.
>
> /Erik
>>
>> I have tried to run make JOBS=1, but did not help, strangely I have
>> still seen in the log make[3] and make[4] logs which suggested that
>> there are more than one make jobs were running. Also tried .configure
>> --with-output-sync=recurse without success (same symptoms)
>>
>> Let me know your thoughts.
>>
>> Best regards,
>>
>> Peter
>>
>> Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for
>> Windows 10
>>
>> From: Magnus Ihse Bursie<mailto:[hidden email]>
>> Sent: Wednesday, October 4, 2017 1:04 AM
>> To: Peter Budai<mailto:[hidden email]>;
>> [hidden email]<mailto:[hidden email]><mailto:[hidden email]><mailto:[hidden email]>
>> Subject: Re: Building OpenJDK9 on MSYS2
>>
>> Actually, it wasn't so much remaining trouble. :-) I fired up msys2 and
>> checked out where I left off. It turned out that the remaining snag was
>> that msys2 tries to convert command lines automatically, from "unix"
>> style paths to "windows" style paths. Unfortunately, it does not do this
>> very well and it breaks all sorts of things. We already have a FIXPATH
>> solution in place which deals with this, so basically all I had to do
>> was disable this (by setting MSYS2_ARG_CONV_EXCL to "*"). However, this
>> broke our cygpath replacement hack (!) so I had to disable it there.
>> Sigh. Anyway, with those fixes it ran and worked well. (I also
>> discovered and fixed a bug related to how we set up the FIXPATH variable
>> on msys, but it only triggers in certain circumstances).
>>
>> With this patch I now jdk9 seems to build fine on msys2. It should apply
>> cleanly on jdk9/jdk9. Since it turned out to be so trivial, I'll try to
>> get it in in jdk10.
>>
>> Here's the patch if you want to apply it yourself:
>>
>> diff -r a08cbfc0e4ec common/autoconf/basics_windows.m4
>> --- a/common/autoconf/basics_windows.m4    Thu Aug 03 18:56:56 2017
>> +0000
>> +++ b/common/autoconf/basics_windows.m4    Wed Oct 04 00:53:58 2017
>> +0200
>> @@ -42,7 +42,7 @@
>>        windows_path=`$CYGPATH -m "$unix_path"`
>>        $1="$windows_path"
>>      elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then
>> -    windows_path=`cmd //c echo $unix_path`
>> +    windows_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $unix_path`
>>        $1="$windows_path"
>>      fi
>>    ])
>> @@ -136,6 +136,16 @@
>>      fi
>>    ])
>>
>> +AC_DEFUN([BASIC_MSYS_UPDATE_FIXPATH],
>> +[
>> +  # Take all collected prefixes and turn them into a
>> -m/c/foo@/c/bar@... command line
>> +  # @ was chosen as separator to minimize risk of other tools messing
>> around with it
>> +  all_unique_prefixes=`echo "${all_fixpath_prefixes@<:@@@:>@}" \
>> +      | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
>> +  fixpath_argument_list=`echo $all_unique_prefixes  | tr ' ' '@'`
>> +  FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
>> +])
>> +
>>    AC_DEFUN([BASIC_FIXUP_PATH_MSYS],
>>    [
>>      path="[$]$1"
>> @@ -143,7 +153,7 @@
>>      new_path="$path"
>>      if test "x$has_colon" = x; then
>>        # Not in mixed or Windows style, start by that.
>> -    new_path=`cmd //c echo $path`
>> +    new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $path`
>>      fi
>>
>>      BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
>> @@ -155,6 +165,8 @@
>>
>>      # Save the first 10 bytes of this path to the storage, so fixpath
>> can work.
>> all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
>> "${new_path:0:10}")
>> +  # We might need to re-evaluate FIXPATH.
>> +  BASIC_MSYS_UPDATE_FIXPATH
>>    ])
>>
>>    AC_DEFUN([BASIC_FIXUP_EXECUTABLE_CYGWIN],
>> @@ -293,7 +305,7 @@
>>        # Do not save /bin paths to all_fixpath_prefixes!
>>      else
>>        # Not in mixed or Windows style, start by that.
>> -    new_path=`cmd //c echo $new_path`
>> +    new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $new_path`
>>        BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
>>        # Output is in $new_path
>>        BASIC_WINDOWS_REWRITE_AS_UNIX_PATH(new_path)
>> @@ -302,6 +314,8 @@
>>
>>        # Save the first 10 bytes of this path to the storage, so fixpath
>> can work.
>> all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
>> "${new_path:0:10}")
>> +    # We might need to re-evaluate FIXPATH.
>> +    BASIC_MSYS_UPDATE_FIXPATH
>>      fi
>>    ])
>>
>> @@ -347,6 +361,10 @@
>>        WINDOWS_ENV_VENDOR='msys'
>>        WINDOWS_ENV_VERSION="$MSYS_VERSION"
>>
>> +    # Prohibit msys2 path conversion from trying to be "intelligent",
>> and rely
>> +    # on fixpath instead.
>> +    export MSYS2_ARG_CONV_EXCL="*"
>> +
>>        AC_MSG_CHECKING([msys root directory as unix-style path])
>>        # The cmd output ends with Windows line endings (CR/LF), the grep
>> command will strip that away
>>        MSYS_ROOT_PATH=`cd / ; cmd /c cd | $GREP ".*"`
>> @@ -391,10 +409,7 @@
>>        elif test "x$OPENJDK_BUILD_OS_ENV" = xwindows.msys; then
>>          # Take all collected prefixes and turn them into a
>> -m/c/foo@/c/bar@... command line
>>          # @ was chosen as separator to minimize risk of other tools
>> messing around with it
>> -      all_unique_prefixes=`echo "${all_fixpath_prefixes@<:@@@:>@}" \
>> -          | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
>> -      fixpath_argument_list=`echo $all_unique_prefixes  | tr ' ' '@'`
>> -      FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
>> +      BASIC_MSYS_UPDATE_FIXPATH
>>        fi
>>        FIXPATH_SRC_W="$FIXPATH_SRC"
>>        FIXPATH_BIN_W="$FIXPATH_BIN"
>> diff -r a08cbfc0e4ec common/autoconf/build-aux/config.sub
>> --- a/common/autoconf/build-aux/config.sub    Thu Aug 03 18:56:56
>> 2017 +0000
>> +++ b/common/autoconf/build-aux/config.sub    Wed Oct 04 00:53:58
>> 2017 +0200
>> @@ -30,7 +30,7 @@
>>    DIR=`dirname $0`
>>
>>    # First, filter out everything that doesn't begin with "aarch64-"
>> -if ! echo $* | grep '^aarch64-' >/dev/null ; then
>> +if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ; then
>>        . $DIR/autoconf-config.sub "$@"
>>        # autoconf-config.sub exits, so we never reach here, but just in
>>        # case we do:
>> @@ -38,13 +38,17 @@
>>    fi
>>
>>    while test $# -gt 0 ; do
>> -    case $1 in
>> +    case $1 in
>>            -- )   # Stop option processing
>>                shift; break ;;
>>            aarch64-* )
>>                config=`echo $1 | sed 's/^aarch64-/arm-/'`
>>                sub_args="$sub_args $config"
>>                shift; ;;
>> +        *-msys )
>> +            config=`echo $1 | sed 's/msys/mingw32/'`
>> +            sub_args="$sub_args $config"
>> +            shift; ;;
>>            - )    # Use stdin as input.
>>                sub_args="$sub_args $1"
>>                shift; break ;;
>> diff -r a08cbfc0e4ec common/autoconf/spec.gmk.in
>> --- a/common/autoconf/spec.gmk.in    Thu Aug 03 18:56:56 2017 +0000
>> +++ b/common/autoconf/spec.gmk.in    Wed Oct 04 00:53:58 2017 +0200
>> @@ -120,6 +120,13 @@
>>      # On Windows, the Visual Studio toolchain needs the PATH to be
>> adjusted
>>      # to include Visual Studio tools (this needs to be in cygwin/msys
>> style).
>>      export PATH:=@VS_PATH@
>> +
>> +endif
>> +
>> +ifeq ($(OPENJDK_TARGET_OS_ENV), windows.msys)
>> +  # On msys2, prohibit msys path conversion from trying to be
>> +  # "intelligent", and rely on fixpath instead.
>> +  export MSYS2_ARG_CONV_EXCL:=*
>>    endif
>>
>>    SYSROOT_CFLAGS := @SYSROOT_CFLAGS@
>>
>> /Magnus
>>
>> On 2017-10-03 22:34, Magnus Ihse Bursie wrote:
>>> I gave msys2 a shot some time ago, but it ended up too much trouble.
>>> I'll share some of my notes from that attempt, for what it's worth.
>>>
>>> To install package X/Y, run "pacman -S X/Y". Missing tools and
>>> packages where to find them:
>>> cmp: msys/diffutils
>>> tar: msys/tar
>>> make: msys/make
>>> unzip: msys/unzip
>>> zip: msys/zip
>>>
>>> config.sub reports msys as "x86_64-pc-mingw32" but msys2 as
>>> "x86_64-pc-msys". This patch adds postprocessing in "our" config.sub
>>> to report msys2 similar to msys. (Opinions, including my own :-) may
>>> vary if this really is the best way..)
>>>
>>> diff -r b88023f46daa common/autoconf/build-aux/config.sub
>>> --- a/common/autoconf/build-aux/config.sub      Fri Jan 27 10:15:41
>>> 2017 +0100
>>> +++ b/common/autoconf/build-aux/config.sub      Fri Feb 03 05:00:25
>>> 2017 -0700
>>> @@ -30,7 +30,7 @@
>>>   DIR=`dirname $0`
>>>
>>>   # First, filter out everything that doesn't begin with "aarch64-"
>>> -if ! echo $* | grep '^aarch64-' >/dev/null ; then
>>> +if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ; then
>>>       . $DIR/autoconf-config.sub "$@"
>>>       # autoconf-config.sub exits, so we never reach here, but just in
>>>       # case we do:
>>> @@ -45,6 +45,10 @@
>>>               config=`echo $1 | sed 's/^aarch64-/arm-/'`
>>>               sub_args="$sub_args $config"
>>>               shift; ;;
>>> +        *-msys )
>>> +            config=`echo $1 | sed 's/msys/mingw32/'`
>>> +            sub_args="$sub_args $config"
>>> +            shift; ;;
>>>           - )    # Use stdin as input.
>>>               sub_args="$sub_args $1"
>>>               shift; break ;;
>>>
>>> If I remember correctly, this got me past the configure stage at the
>>> time.
>>>
>>> I don't think it's very hard to get it to work on msys2, I just ran
>>> into one snag too many and didn't think msys2 would be used by anyone.
>>>
>>> /Magnus
>>>
>>> On 2017-10-03 17:20, Peter Budai wrote:
>>>> Hello,
>>>>
>>>> According to
>>>> http://hg.openjdk.java.net/jdk9/jdk9/file/a08cbfc0e4ec/common/doc/building.html
>>>>
>>>> “msys2 and the new Windows Subsystem for Linux (WSL) would likely be
>>>> possible to support in a future version but that would require a
>>>> community effort to implement”
>>>>
>>>> I’d like to help making the OpenJDK 9 build working on msys2. What is
>>>> the best way to move forward? Is there a similar effort in progress?
>>>>
>>>> Thank you and best regards,
>>>>
>>>> Peter
>>>>
>>>>
>




Reply | Threaded
Open this post in threaded view
|

RE: Building OpenJDK9 on MSYS2

Peter Budai
The configure of OpenJDK overwrites the SHELL. Actually it is using bash, but for the arguments it was using “-e -o pipefail”. I have figured that for MSYS2 bash what is needed as bash arguments is “-e -l -c -o pipefail”

That looks like solving this problem, and now the real issues are surfacing.

Peter

From: Peter Budai<mailto:[hidden email]>
Sent: Friday, October 6, 2017 6:43 PM
To: Magnus Ihse Bursie<mailto:[hidden email]>; Erik Joelsson<mailto:[hidden email]>; [hidden email]<mailto:[hidden email]>
Subject: RE: Building OpenJDK9 on MSYS2

Magnus,

I have followed your suggestion and removed the fixpath prefixes from gcc related compile tools, and left only the fixpath prefix _only_ for the Boot JDK related tools in place.


  1.  As  I follow the process, all java and javac related compile steps are running properly
  2.  When the process reaches gcc related steps I got the error message at the same place as before (no fixpath). If I execute that command from the bash prompt, it creates the output:

$ ( /usr/bin/gawk '/@@END_COPYRIGHT@@/{exit}1' /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template && /C/msys64/mingw64/bin/gcc -E -x c  /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template 2> >(/usr/bin/grep -v '^SocketOptionRegistry.java.template$' >&2) | /usr/bin/gawk '/@@START_HERE@@/,0' |  /usr/bin/sed -e 's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED FILE - DO NOT EDIT/' -e 's/PREFIX_//' -e 's/^#.*//' ) > /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java

As I have mentioned the parameters are replaced by the bash automatically

  1.  Then build continues, then little later stops at a super simple command:

mv /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBuffer.java.tmp /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBuffer.java
           Needless to say, the ByteBuffer.java.tmp file DOES exist. And running the above command from the bash works, and build continues.

  1.  A few similar cases (stops) with DirectByteBuffer and DirectByteBufferR


Currently I try to explore how that might relate to the MSYS2 bash and make, somehow it behaves differently

If you have any other suggestion, let me know.

Best regards,

Peter

From: Peter Budai<mailto:[hidden email]>
Sent: Thursday, October 5, 2017 3:52 PM
To: Magnus Ihse Bursie<mailto:[hidden email]>; Erik Joelsson<mailto:[hidden email]>; [hidden email]<mailto:[hidden email]>
Subject: RE: Building OpenJDK9 on MSYS2

Hi Magnus,

So first of all, here is the current patch, which I was not able to attach: https://pastebin.com/pwT4Ynxc

>> That's surprising, since gcc is prefixed with fixpath, which it should not.
Actually you DO need fixpath IMHO.
This is a mingw64 version of the gcc (/C/msys64/mingw64/bin/gcc), which is a fully functional Windows executable, which expects Windows formatted path arguments.
As the updated build process uses EXPORT MSYS2_ARG_CONV_EXCL=* (see that patch), none of the command line arguments are converted  from the unix path to Windows, but fixpath does that conversion. There is a wiki describing more details on this:
https://github.com/msys2/msys2/wiki/Porting#user-content-filesystem-namespaces



>>I have a hard time believing this is a race condition. On the other hand, this stuff is weird, we're misusing the C preprocessor to process defines in java code, so I'm not surprised it breaks down.
>>I don't know why it succeeded when run on the command line, though.
When I execute that command from the bash command line there is no EXPORT MSYS2_ARG_CONV_EXCL, but the bash itself does the automatic conversion of the arguments. Maybe it has something to do how fixpath does CreateProcess?

Does that help?

Best regards,

Peter

Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10

From: Magnus Ihse Bursie<mailto:[hidden email]>
Sent: Thursday, October 5, 2017 12:13 PM
To: Peter Budai<mailto:[hidden email]>; Erik Joelsson<mailto:[hidden email]>; [hidden email]<mailto:[hidden email]>
Subject: Re: Building OpenJDK9 on MSYS2


On 2017-10-05 11:59, Peter Budai wrote:
Hi Magnus and Erik,

I really appreciate your quick feedback. I assumed that it won’t be easy, but I just don’t feel I should give up now  - maybe later when I see the real scale of work. So bear with me for a time being.

Attached is a patch which already includes Magnus’ changes, plus a few which I have added:
·       basically enabling gcc for windows,
·       and modifying a logic for compiling fixpath (before that it was using hard-coded MS VSC compile flags)

Actually, you must make sure fixpath is *not* used for the toolchain, since gcc uses unix style paths.
(However, other tools such as java will still need it.)



So here is what I have as the result of configure:
====================================================
The existing configuration has been successfully updated in
/C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release
using configure arguments '--disable-freetype-bundling --disable-javac-server'.

Configuration summary:
* Debug level:    release
* HS debug level: product
* JDK variant:    normal
* JVM variants:   server
* OpenJDK target: OS: windows, CPU architecture: x86, address length: 64
* Version string: 9-internal+0-adhoc.peterbud.jdk9 (9-internal)

Tools summary:
* Environment:    msys version 2.9.0(0.318/5/3) (root at /C/msys64)
* Boot JDK:       java version "1.8.0_144"  Java(TM) SE Runtime Environment (build 1.8.0_144-b01)  Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)   (at /c/progra~1/java/jdk18~1.0_1)
* Toolchain:      gcc (GNU Compiler Collection)
* C Compiler:     Version 7.2.0 (at /C/msys64/mingw64/bin/gcc)
* C++ Compiler:   Version 7.2.0 (at /c/msys64/mingw64/bin/g++)

Build performance summary:
* Cores to use:   4
* Memory limit:   16216 MB

Its clear says that the toolchain is gcc 7.2 (BTW there is no Visual Studio on this machine)

Now for the details of the config log, you can see here: https://pastebin.com/MN2ZYcHH

And about the build process and the error I get:

$ make JOBS=1
Building target 'default (exploded-image)' in configuration 'windows-x86_64-normal-server-release'
Compiling 8 files for BUILD_TOOLS_LANGTOOLS
Compiling 17 properties into resource bundles for jdk.compiler
Parsing 1 properties into enum-like class for jdk.compiler
Compiling 19 properties into resource bundles for jdk.javadoc
Compiling 12 properties into resource bundles for jdk.jdeps
Compiling 7 properties into resource bundles for jdk.jshell
Compiling 117 files for BUILD_INTERIM_java.compiler
Compiling 396 files for BUILD_INTERIM_jdk.compiler
Compiling 61 files for BUILD_INTERIM_jdk.jdeps
Compiling 457 files for BUILD_INTERIM_jdk.javadoc
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Compiling 159 files for BUILD_TOOLS_JDK
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
make[3]: *** [GensrcMisc.gmk:78: /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java] Error 1
make[3]: *** Deleting file '/C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java'
make[2]: *** [make/Main.gmk:115: java.base-gensrc-jdk] Error 2

ERROR: Build failed for target 'default (exploded-image)' in configuration 'windows-x86_64-normal-server-release' (exit code 2)

No indication of failed target found.
Hint: Try searching the build log for '] Error'.
Hint: See common/doc/building.html#troubleshooting for assistance.

make[1]: *** [/home/peterbud/jdk9/make/Init.gmk:296: main] Error 2
make: *** [/home/peterbud/jdk9/make/Init.gmk:185: default] Error 2

If I run here
make JOBS=1 LOG=debug
The failing line seems to be this:

( /usr/bin/gawk '/@@END_COPYRIGHT@@/{exit}1' /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template && /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/configure-support/bin/fixpath.exe -m/C/msys64/@/c/msys64/@/c/progra~ /C/msys64/mingw64/bin/gcc -E -x c  /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template 2> >(/usr/bin/grep -v '^SocketOptionRegistry.java.template$' >&2) | /usr/bin/gawk '/@@START_HERE@@/,0' |  /usr/bin/sed -e 's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED FILE - DO NOT EDIT/' -e 's/PREFIX_//' -e 's/^#.*//' ) > /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java
make[3]: *** [GensrcMisc.gmk:78: /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java] Error 1

Now the interesting is: if I copy this line above to the bash prompt, it runs without problem, and the file support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java
That's surprising, since gcc is prefixed with fixpath, which it should not.

I have a hard time believing this is a race condition. On the other hand, this stuff is weird, we're misusing the C preprocessor to process defines in java code, so I'm not suprised it breaks down. I don't know why it succeeded when run on the command line, though. My suggestion is to just do some quick and dirty hack around this: take the file you manage to generate and just copy it in during the build instead. If you can get round this, you might start seeing some *real* problems. :-)

Also, my suggestion is that you try running "make hotspot" to cut to the chase. Compiling hotspot will likely be the hardest thing. Or even "make -k hotspot" to get an assessment of the amount of work ahead of you.

/Magnus
Is produced.

Then I can again issue
make JOBS=1 LOG=debug

And the compile process is being continued, until a similar error pops up with a different generated file. I have an assumption that this happens because make is still running parallel jobs, despite JOBS=1 but I’m not sure.

How could I best tackle this?

Thank you and best regards,

Peter

Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10

From: Magnus Ihse Bursie<mailto:[hidden email]>
Sent: Thursday, October 5, 2017 11:33 AM
To: Erik Joelsson<mailto:[hidden email]>; Peter Budai<mailto:[hidden email]>; [hidden email]<mailto:[hidden email]>
Subject: Re: Building OpenJDK9 on MSYS2

On 2017-10-05 10:10, Erik Joelsson wrote:

> Hello Peter,
>
>
> On 2017-10-04 21:15, Peter Budai wrote:
>> Hi Magnus,
>>
>> Thanks for the quick reply I’ll check these patches with msys2.
>>
>> Let me specify with more details what I’d like to achieve: I’d like
>> to build OpenJDK9 with MSYS2 MINGW64 environment using gcc toolchain.
>> (I’m not sure how familiar are you with MSYS2, but there are 3
>> different environments: MSYS2, MINGW32 and MINGW64). In theory
>> MINGW64 with gcc is the closes you can get on Windows platform as a
>> gcc unix like build environment, which produces still a native 64-bit
>> executable on Windows.
>>
>> I’m not very familiar with OpenJDK yet, so therefore I’d like to hear
>> your opinion: how realistic is that?
> Sorry to disappoint, but I would say that requires major work. There
> is a strong historic assumption that windows builds are done using
> Visual Studio. We have abstracted away some of it in configure (see
> TOOLCHAIN_TYPE), but it's very far from enough to change compiler
> environment for a Windows build. The native sources are also bound to
> make a lot of such assumptions. I would expect the changes needed to
> be in the thousands of lines of code.

I agree that it requires hard work (even if "thousands" might be an
overestimation I think, but "hundreds" is not enough, so it's the right
magnitude). On the other hand, it would be really good if we did sort
things out, so that we had proper conditions based on OS vs
compiler/toolchain.

If you really want to start, the first thing is to patch toolchain.m4 to
VALID_TOOLCHAINS_windows="microsoft gcc"
and then call configure using "bash configure --with-toolchain-type=gcc".

As Erik, I doubt you will come very far before things starts tumbling down.

>
> When we say supporting the build in msys2 instead of cygwin, we just
> mean using msys2 as the unix emulating layer for our tools like
> make/bash/grep/sed etc.
>
> One think I have done successfully is running the build in WSL
> (Windows Subsystem for Linux), but that isn't all that helpful as WSL
> for practical purposes is more or less like running Linux in a VM, so
> the build sees a Linux system and builds a Linux binary.
>> As a side note: with MINGW64 I have managed to run configure phase
>> successfully for OpenJDK. The compile process has also started and
>> went for a while, but interestingly I run into some kind of race
>> conditions as make stopped with an error. Using LOG=debug I have fond
>> the failing line and then copying the failed command and pasting it
>> to the bash prompt it successfully generated the output target, and
>> then the build process run further when a similar situation happened.
>> Also pasting the failed command run in the bash without any problem,
>> and build continued… until the next.
> Without seeing the errors I can't say much. I very much doubt that you
> are running with gcc as the compiler though. Configure isn't easily
> fooled into using a different compiler to what it prefers, and I would
> expect things to crash and burn pretty early if you actually did.
>
> /Erik
>>
>> I have tried to run make JOBS=1, but did not help, strangely I have
>> still seen in the log make[3] and make[4] logs which suggested that
>> there are more than one make jobs were running. Also tried .configure
>> --with-output-sync=recurse without success (same symptoms)
>>
>> Let me know your thoughts.
>>
>> Best regards,
>>
>> Peter
>>
>> Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for
>> Windows 10
>>
>> From: Magnus Ihse Bursie<mailto:[hidden email]>
>> Sent: Wednesday, October 4, 2017 1:04 AM
>> To: Peter Budai<mailto:[hidden email]>;
>> [hidden email]<mailto:[hidden email]><mailto:[hidden email]><mailto:[hidden email]>
>> Subject: Re: Building OpenJDK9 on MSYS2
>>
>> Actually, it wasn't so much remaining trouble. :-) I fired up msys2 and
>> checked out where I left off. It turned out that the remaining snag was
>> that msys2 tries to convert command lines automatically, from "unix"
>> style paths to "windows" style paths. Unfortunately, it does not do this
>> very well and it breaks all sorts of things. We already have a FIXPATH
>> solution in place which deals with this, so basically all I had to do
>> was disable this (by setting MSYS2_ARG_CONV_EXCL to "*"). However, this
>> broke our cygpath replacement hack (!) so I had to disable it there.
>> Sigh. Anyway, with those fixes it ran and worked well. (I also
>> discovered and fixed a bug related to how we set up the FIXPATH variable
>> on msys, but it only triggers in certain circumstances).
>>
>> With this patch I now jdk9 seems to build fine on msys2. It should apply
>> cleanly on jdk9/jdk9. Since it turned out to be so trivial, I'll try to
>> get it in in jdk10.
>>
>> Here's the patch if you want to apply it yourself:
>>
>> diff -r a08cbfc0e4ec common/autoconf/basics_windows.m4
>> --- a/common/autoconf/basics_windows.m4    Thu Aug 03 18:56:56 2017
>> +0000
>> +++ b/common/autoconf/basics_windows.m4    Wed Oct 04 00:53:58 2017
>> +0200
>> @@ -42,7 +42,7 @@
>>        windows_path=`$CYGPATH -m "$unix_path"`
>>        $1="$windows_path"
>>      elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then
>> -    windows_path=`cmd //c echo $unix_path`
>> +    windows_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $unix_path`
>>        $1="$windows_path"
>>      fi
>>    ])
>> @@ -136,6 +136,16 @@
>>      fi
>>    ])
>>
>> +AC_DEFUN([BASIC_MSYS_UPDATE_FIXPATH],
>> +[
>> +  # Take all collected prefixes and turn them into a
>> -m/c/foo@/c/bar@... command line
>> +  # @ was chosen as separator to minimize risk of other tools messing
>> around with it
>> +  all_unique_prefixes=`echo "${all_fixpath_prefixes@<:@@@:>@}" \
>> +      | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
>> +  fixpath_argument_list=`echo $all_unique_prefixes  | tr ' ' '@'`
>> +  FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
>> +])
>> +
>>    AC_DEFUN([BASIC_FIXUP_PATH_MSYS],
>>    [
>>      path="[$]$1"
>> @@ -143,7 +153,7 @@
>>      new_path="$path"
>>      if test "x$has_colon" = x; then
>>        # Not in mixed or Windows style, start by that.
>> -    new_path=`cmd //c echo $path`
>> +    new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $path`
>>      fi
>>
>>      BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
>> @@ -155,6 +165,8 @@
>>
>>      # Save the first 10 bytes of this path to the storage, so fixpath
>> can work.
>> all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
>> "${new_path:0:10}")
>> +  # We might need to re-evaluate FIXPATH.
>> +  BASIC_MSYS_UPDATE_FIXPATH
>>    ])
>>
>>    AC_DEFUN([BASIC_FIXUP_EXECUTABLE_CYGWIN],
>> @@ -293,7 +305,7 @@
>>        # Do not save /bin paths to all_fixpath_prefixes!
>>      else
>>        # Not in mixed or Windows style, start by that.
>> -    new_path=`cmd //c echo $new_path`
>> +    new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $new_path`
>>        BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
>>        # Output is in $new_path
>>        BASIC_WINDOWS_REWRITE_AS_UNIX_PATH(new_path)
>> @@ -302,6 +314,8 @@
>>
>>        # Save the first 10 bytes of this path to the storage, so fixpath
>> can work.
>> all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
>> "${new_path:0:10}")
>> +    # We might need to re-evaluate FIXPATH.
>> +    BASIC_MSYS_UPDATE_FIXPATH
>>      fi
>>    ])
>>
>> @@ -347,6 +361,10 @@
>>        WINDOWS_ENV_VENDOR='msys'
>>        WINDOWS_ENV_VERSION="$MSYS_VERSION"
>>
>> +    # Prohibit msys2 path conversion from trying to be "intelligent",
>> and rely
>> +    # on fixpath instead.
>> +    export MSYS2_ARG_CONV_EXCL="*"
>> +
>>        AC_MSG_CHECKING([msys root directory as unix-style path])
>>        # The cmd output ends with Windows line endings (CR/LF), the grep
>> command will strip that away
>>        MSYS_ROOT_PATH=`cd / ; cmd /c cd | $GREP ".*"`
>> @@ -391,10 +409,7 @@
>>        elif test "x$OPENJDK_BUILD_OS_ENV" = xwindows.msys; then
>>          # Take all collected prefixes and turn them into a
>> -m/c/foo@/c/bar@... command line
>>          # @ was chosen as separator to minimize risk of other tools
>> messing around with it
>> -      all_unique_prefixes=`echo "${all_fixpath_prefixes@<:@@@:>@}" \
>> -          | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
>> -      fixpath_argument_list=`echo $all_unique_prefixes  | tr ' ' '@'`
>> -      FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
>> +      BASIC_MSYS_UPDATE_FIXPATH
>>        fi
>>        FIXPATH_SRC_W="$FIXPATH_SRC"
>>        FIXPATH_BIN_W="$FIXPATH_BIN"
>> diff -r a08cbfc0e4ec common/autoconf/build-aux/config.sub
>> --- a/common/autoconf/build-aux/config.sub    Thu Aug 03 18:56:56
>> 2017 +0000
>> +++ b/common/autoconf/build-aux/config.sub    Wed Oct 04 00:53:58
>> 2017 +0200
>> @@ -30,7 +30,7 @@
>>    DIR=`dirname $0`
>>
>>    # First, filter out everything that doesn't begin with "aarch64-"
>> -if ! echo $* | grep '^aarch64-' >/dev/null ; then
>> +if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ; then
>>        . $DIR/autoconf-config.sub "$@"
>>        # autoconf-config.sub exits, so we never reach here, but just in
>>        # case we do:
>> @@ -38,13 +38,17 @@
>>    fi
>>
>>    while test $# -gt 0 ; do
>> -    case $1 in
>> +    case $1 in
>>            -- )   # Stop option processing
>>                shift; break ;;
>>            aarch64-* )
>>                config=`echo $1 | sed 's/^aarch64-/arm-/'`
>>                sub_args="$sub_args $config"
>>                shift; ;;
>> +        *-msys )
>> +            config=`echo $1 | sed 's/msys/mingw32/'`
>> +            sub_args="$sub_args $config"
>> +            shift; ;;
>>            - )    # Use stdin as input.
>>                sub_args="$sub_args $1"
>>                shift; break ;;
>> diff -r a08cbfc0e4ec common/autoconf/spec.gmk.in
>> --- a/common/autoconf/spec.gmk.in    Thu Aug 03 18:56:56 2017 +0000
>> +++ b/common/autoconf/spec.gmk.in    Wed Oct 04 00:53:58 2017 +0200
>> @@ -120,6 +120,13 @@
>>      # On Windows, the Visual Studio toolchain needs the PATH to be
>> adjusted
>>      # to include Visual Studio tools (this needs to be in cygwin/msys
>> style).
>>      export PATH:=@VS_PATH@
>> +
>> +endif
>> +
>> +ifeq ($(OPENJDK_TARGET_OS_ENV), windows.msys)
>> +  # On msys2, prohibit msys path conversion from trying to be
>> +  # "intelligent", and rely on fixpath instead.
>> +  export MSYS2_ARG_CONV_EXCL:=*
>>    endif
>>
>>    SYSROOT_CFLAGS := @SYSROOT_CFLAGS@
>>
>> /Magnus
>>
>> On 2017-10-03 22:34, Magnus Ihse Bursie wrote:
>>> I gave msys2 a shot some time ago, but it ended up too much trouble.
>>> I'll share some of my notes from that attempt, for what it's worth.
>>>
>>> To install package X/Y, run "pacman -S X/Y". Missing tools and
>>> packages where to find them:
>>> cmp: msys/diffutils
>>> tar: msys/tar
>>> make: msys/make
>>> unzip: msys/unzip
>>> zip: msys/zip
>>>
>>> config.sub reports msys as "x86_64-pc-mingw32" but msys2 as
>>> "x86_64-pc-msys". This patch adds postprocessing in "our" config.sub
>>> to report msys2 similar to msys. (Opinions, including my own :-) may
>>> vary if this really is the best way..)
>>>
>>> diff -r b88023f46daa common/autoconf/build-aux/config.sub
>>> --- a/common/autoconf/build-aux/config.sub      Fri Jan 27 10:15:41
>>> 2017 +0100
>>> +++ b/common/autoconf/build-aux/config.sub      Fri Feb 03 05:00:25
>>> 2017 -0700
>>> @@ -30,7 +30,7 @@
>>>   DIR=`dirname $0`
>>>
>>>   # First, filter out everything that doesn't begin with "aarch64-"
>>> -if ! echo $* | grep '^aarch64-' >/dev/null ; then
>>> +if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ; then
>>>       . $DIR/autoconf-config.sub "$@"
>>>       # autoconf-config.sub exits, so we never reach here, but just in
>>>       # case we do:
>>> @@ -45,6 +45,10 @@
>>>               config=`echo $1 | sed 's/^aarch64-/arm-/'`
>>>               sub_args="$sub_args $config"
>>>               shift; ;;
>>> +        *-msys )
>>> +            config=`echo $1 | sed 's/msys/mingw32/'`
>>> +            sub_args="$sub_args $config"
>>> +            shift; ;;
>>>           - )    # Use stdin as input.
>>>               sub_args="$sub_args $1"
>>>               shift; break ;;
>>>
>>> If I remember correctly, this got me past the configure stage at the
>>> time.
>>>
>>> I don't think it's very hard to get it to work on msys2, I just ran
>>> into one snag too many and didn't think msys2 would be used by anyone.
>>>
>>> /Magnus
>>>
>>> On 2017-10-03 17:20, Peter Budai wrote:
>>>> Hello,
>>>>
>>>> According to
>>>> http://hg.openjdk.java.net/jdk9/jdk9/file/a08cbfc0e4ec/common/doc/building.html
>>>>
>>>> “msys2 and the new Windows Subsystem for Linux (WSL) would likely be
>>>> possible to support in a future version but that would require a
>>>> community effort to implement”
>>>>
>>>> I’d like to help making the OpenJDK 9 build working on msys2. What is
>>>> the best way to move forward? Is there a similar effort in progress?
>>>>
>>>> Thank you and best regards,
>>>>
>>>> Peter
>>>>
>>>>
>





Reply | Threaded
Open this post in threaded view
|

Re: Building OpenJDK9 on MSYS2

Erik Joelsson
Nice work so far and good luck with trying to fix the real issues!

/Erik


On 2017-10-07 10:14, Peter Budai wrote:

>
> The configure of OpenJDK overwrites the SHELL. Actually it is using
> bash, but for the arguments it was using “-e -o pipefail”. I have
> figured that for MSYS2 bash what is needed as bash arguments is “-e -l
> -c -o pipefail”
>
> That looks like solving this problem, and now the real issues are
> surfacing.
>
> Peter
>
> *From: *Peter Budai <mailto:[hidden email]>
> *Sent: *Friday, October 6, 2017 6:43 PM
> *To: *Magnus Ihse Bursie <mailto:[hidden email]>; Erik
> Joelsson <mailto:[hidden email]>; [hidden email]
> <mailto:[hidden email]>
> *Subject: *RE: Building OpenJDK9 on MSYS2
>
> Magnus,
>
> I have followed your suggestion and removed the fixpath prefixes from
> gcc related compile tools, and left only the fixpath prefix _only_ for
> the Boot JDK related tools in place.
>
>  1. As  I follow the process, all java and javac related compile steps
>     are running properly
>  2. When the process reaches gcc related steps I got the error message
>     at the same place as before (no fixpath). If I execute that
>     command from the bash prompt, it creates the output:
>
> $ ( /usr/bin/gawk '/@@END_COPYRIGHT@@/{exit}1'
> /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template
> && /C/msys64/mingw64/bin/gcc -E -x c
> /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template
> 2> >(/usr/bin/grep -v '^SocketOptionRegistry.java.template$' >&2) |
> /usr/bin/gawk '/@@START_HERE@@/,0' |  /usr/bin/sed -e
> 's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED FILE - DO NOT EDIT/' -e
> 's/PREFIX_//' -e 's/^#.*//' ) >
> /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java
>
> As I have mentioned the parameters are replaced by the bash automatically
>
>  3. Then build continues, then little later stops at a super simple
>     command:
>
> mv
> /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBuffer.java.tmp
> /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBuffer.java
>
>            Needless to say, the ByteBuffer.java.tmp file DOES exist.
> And running the above command from the bash works, and build continues.
>
>  4. A few similar cases (stops) with DirectByteBuffer and
>     DirectByteBufferR
>
> Currently I try to explore how that might relate to the MSYS2 bash and
> make, somehow it behaves differently
>
> If you have any other suggestion, let me know.
>
> Best regards,
>
> Peter
>
> *From: *Peter Budai <mailto:[hidden email]>
> *Sent: *Thursday, October 5, 2017 3:52 PM
> *To: *Magnus Ihse Bursie <mailto:[hidden email]>; Erik
> Joelsson <mailto:[hidden email]>; [hidden email]
> <mailto:[hidden email]>
> *Subject: *RE: Building OpenJDK9 on MSYS2
>
> Hi Magnus,
>
> So first of all, here is the current patch, which I was not able to
> attach: https://pastebin.com/pwT4Ynxc
>
> >> That's surprising, since gcc is prefixed with fixpath, which it should not.
>
> Actually you DO need fixpath IMHO.
>
> This is a mingw64 version of the gcc (/C/msys64/mingw64/bin/gcc),
> which is a fully functional Windows executable, /which expects Windows
> formatted path arguments/.
>
> As the updated build process uses EXPORT MSYS2_ARG_CONV_EXCL=* (see
> that patch), none of the command line arguments are converted from the
> unix path to Windows, but fixpath does that conversion. There is a
> wiki describing more details on this:
>
> https://github.com/msys2/msys2/wiki/Porting#user-content-filesystem-namespaces
>
>
> >>I have a hard time believing this is a race condition. On the other
> hand, this stuff is weird, we're misusing the C preprocessor to
> process defines in java code, so I'm not surprised it breaks down.
>
> >>I don't know why it succeeded when run on the command line, though.
>
> When I execute that command from the /bash/ command line there is no
> EXPORT MSYS2_ARG_CONV_EXCL, but the bash itself does the automatic
> conversion of the arguments. Maybe it has something to do how fixpath
> does CreateProcess?
>
> Does that help?
>
> Best regards,
>
> Peter
>
> Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for
> Windows 10
>
> *From: *Magnus Ihse Bursie <mailto:[hidden email]>
> *Sent: *Thursday, October 5, 2017 12:13 PM
> *To: *Peter Budai <mailto:[hidden email]>; Erik Joelsson
> <mailto:[hidden email]>; [hidden email]
> <mailto:[hidden email]>
> *Subject: *Re: Building OpenJDK9 on MSYS2
>
> On 2017-10-05 11:59, Peter Budai wrote:
>
>     Hi Magnus and Erik,
>
>     I really appreciate your quick feedback. I assumed that it won’t
>     be easy, but I just don’t feel I should give up now  - maybe later
>     when I see the real scale of work. So bear with me for a time being.
>
>     Attached is a patch which already includes Magnus’ changes, plus a
>     few which I have added:
>
>     ·basically enabling gcc for windows,
>
>     ·and modifying a logic for compiling fixpath (before that it was
>     using hard-coded MS VSC compile flags)
>
>
> Actually, you must make sure fixpath is *not* used for the toolchain,
> since gcc uses unix style paths.
> (However, other tools such as java will still need it.)
>
>     So here is what I have as the result of configure:
>
>     ====================================================
>
>     The existing configuration has been successfully updated in
>
>     /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release
>
>     using configure arguments '--disable-freetype-bundling
>     --disable-javac-server'.
>
>     Configuration summary:
>
>     * Debug level:    release
>
>     * HS debug level: product
>
>     * JDK variant:    normal
>
>     * JVM variants:   server
>
>     * OpenJDK target: OS: windows, CPU architecture: x86, address
>     length: 64
>
>     * Version string: 9-internal+0-adhoc.peterbud.jdk9 (9-internal)
>
>     Tools summary:
>
>     * Environment:    msys version 2.9.0(0.318/5/3) (root at /C/msys64)
>
>     * Boot JDK:       java version "1.8.0_144"  Java(TM) SE Runtime
>     Environment (build 1.8.0_144-b01)  Java HotSpot(TM) 64-Bit Server
>     VM (build 25.144-b01, mixed mode)   (at /c/progra~1/java/jdk18~1.0_1)
>
>     * Toolchain:      gcc (GNU Compiler Collection)
>
>     * C Compiler:     Version 7.2.0 (at /C/msys64/mingw64/bin/gcc)
>
>     * C++ Compiler:   Version 7.2.0 (at /c/msys64/mingw64/bin/g++)
>
>     Build performance summary:
>
>     * Cores to use:   4
>
>     * Memory limit:   16216 MB
>
>     Its clear says that the toolchain is gcc 7.2 (BTW there is no
>     Visual Studio on this machine)
>
>     Now for the details of the config log, you can see here:
>     https://pastebin.com/MN2ZYcHH
>
>     And about the build process and the error I get:
>
>     $ make JOBS=1
>
>     Building target 'default (exploded-image)' in configuration
>     'windows-x86_64-normal-server-release'
>
>     Compiling 8 files for BUILD_TOOLS_LANGTOOLS
>
>     Compiling 17 properties into resource bundles for jdk.compiler
>
>     Parsing 1 properties into enum-like class for jdk.compiler
>
>     Compiling 19 properties into resource bundles for jdk.javadoc
>
>     Compiling 12 properties into resource bundles for jdk.jdeps
>
>     Compiling 7 properties into resource bundles for jdk.jshell
>
>     Compiling 117 files for BUILD_INTERIM_java.compiler
>
>     Compiling 396 files for BUILD_INTERIM_jdk.compiler
>
>     Compiling 61 files for BUILD_INTERIM_jdk.jdeps
>
>     Compiling 457 files for BUILD_INTERIM_jdk.javadoc
>
>     Note: Some input files use or override a deprecated API.
>
>     Note: Recompile with -Xlint:deprecation for details.
>
>     Compiling 159 files for BUILD_TOOLS_JDK
>
>     Note: Some input files use unchecked or unsafe operations.
>
>     Note: Recompile with -Xlint:unchecked for details.
>
>     make[3]: *** [GensrcMisc.gmk:78:
>     /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java]
>     Error 1
>
>     make[3]: *** Deleting file
>     '/C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java'
>
>     make[2]: *** [make/Main.gmk:115: java.base-gensrc-jdk] Error 2
>
>     ERROR: Build failed for target 'default (exploded-image)' in
>     configuration 'windows-x86_64-normal-server-release' (exit code 2)
>
>     No indication of failed target found.
>
>     Hint: Try searching the build log for '] Error'.
>
>     Hint: See common/doc/building.html#troubleshooting for assistance.
>
>     make[1]: *** [/home/peterbud/jdk9/make/Init.gmk:296: main] Error 2
>
>     make: *** [/home/peterbud/jdk9/make/Init.gmk:185: default] Error 2
>
>     If I run here
>
>     make JOBS=1 LOG=debug
>
>     The failing line seems to be this:
>
>     ( /usr/bin/gawk '/@@END_COPYRIGHT@@/{exit}1'
>     /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template
>     &&
>     /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/configure-support/bin/fixpath.exe
>     -m/C/msys64/@/c/msys64/@/c/progra~ /C/msys64/mingw64/bin/gcc -E -x
>     c
>     /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template
>     2> >(/usr/bin/grep -v '^SocketOptionRegistry.java.template$' >&2)
>     | /usr/bin/gawk '/@@START_HERE@@/,0' |  /usr/bin/sed -e
>     's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED FILE - DO NOT
>     EDIT/' -e 's/PREFIX_//' -e 's/^#.*//' ) >
>     /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java
>
>     make[3]: *** [GensrcMisc.gmk:78:
>     /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java]
>     Error 1
>
>     Now the interesting is: if I copy this line above to the bash
>     prompt, it runs without problem, and the file
>     support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java
>
> That's surprising, since gcc is prefixed with fixpath, which it should
> not.
>
> I have a hard time believing this is a race condition. On the other
> hand, this stuff is weird, we're misusing the C preprocessor to
> process defines in java code, so I'm not suprised it breaks down. I
> don't know why it succeeded when run on the command line, though. My
> suggestion is to just do some quick and dirty hack around this: take
> the file you manage to generate and just copy it in during the build
> instead. If you can get round this, you might start seeing some *real*
> problems. :-)
>
> Also, my suggestion is that you try running "make hotspot" to cut to
> the chase. Compiling hotspot will likely be the hardest thing. Or even
> "make -k hotspot" to get an assessment of the amount of work ahead of you.
>
> /Magnus
>
>     Is produced.
>
>     Then I can again issue
>
>     make JOBS=1 LOG=debug
>
>     And the compile process is being continued, until a similar error
>     pops up with a different generated file. I have an assumption that
>     this happens because make is still running parallel jobs, despite
>     JOBS=1 but I’m not sure.
>
>     How could I best tackle this?
>
>     Thank you and best regards,
>
>     Peter
>
>     Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986>
>     for Windows 10
>
>     *From: *Magnus Ihse Bursie <mailto:[hidden email]>
>     *Sent: *Thursday, October 5, 2017 11:33 AM
>     *To: *Erik Joelsson <mailto:[hidden email]>; Peter Budai
>     <mailto:[hidden email]>; [hidden email]
>     <mailto:[hidden email]>
>     *Subject: *Re: Building OpenJDK9 on MSYS2
>
>     On 2017-10-05 10:10, Erik Joelsson wrote:
>     > Hello Peter,
>     >
>     >
>     > On 2017-10-04 21:15, Peter Budai wrote:
>     >> Hi Magnus,
>     >>
>     >> Thanks for the quick reply I’ll check these patches with msys2.
>     >>
>     >> Let me specify with more details what I’d like to achieve: I’d
>     like
>     >> to build OpenJDK9 with MSYS2 MINGW64 environment using gcc
>     toolchain.
>     >> (I’m not sure how familiar are you with MSYS2, but there are 3
>     >> different environments: MSYS2, MINGW32 and MINGW64). In theory
>     >> MINGW64 with gcc is the closes you can get on Windows platform
>     as a
>     >> gcc unix like build environment, which produces still a native
>     64-bit
>     >> executable on Windows.
>     >>
>     >> I’m not very familiar with OpenJDK yet, so therefore I’d like
>     to hear
>     >> your opinion: how realistic is that?
>     > Sorry to disappoint, but I would say that requires major work.
>     There
>     > is a strong historic assumption that windows builds are done using
>     > Visual Studio. We have abstracted away some of it in configure (see
>     > TOOLCHAIN_TYPE), but it's very far from enough to change compiler
>     > environment for a Windows build. The native sources are also
>     bound to
>     > make a lot of such assumptions. I would expect the changes
>     needed to
>     > be in the thousands of lines of code.
>
>     I agree that it requires hard work (even if "thousands" might be an
>     overestimation I think, but "hundreds" is not enough, so it's the
>     right
>     magnitude). On the other hand, it would be really good if we did sort
>     things out, so that we had proper conditions based on OS vs
>     compiler/toolchain.
>
>     If you really want to start, the first thing is to patch
>     toolchain.m4 to
>     VALID_TOOLCHAINS_windows="microsoft gcc"
>     and then call configure using "bash configure
>     --with-toolchain-type=gcc".
>
>     As Erik, I doubt you will come very far before things starts
>     tumbling down.
>
>     >
>     > When we say supporting the build in msys2 instead of cygwin, we
>     just
>     > mean using msys2 as the unix emulating layer for our tools like
>     > make/bash/grep/sed etc.
>     >
>     > One think I have done successfully is running the build in WSL
>     > (Windows Subsystem for Linux), but that isn't all that helpful
>     as WSL
>     > for practical purposes is more or less like running Linux in a
>     VM, so
>     > the build sees a Linux system and builds a Linux binary.
>     >> As a side note: with MINGW64 I have managed to run configure phase
>     >> successfully for OpenJDK. The compile process has also started and
>     >> went for a while, but interestingly I run into some kind of race
>     >> conditions as make stopped with an error. Using LOG=debug I
>     have fond
>     >> the failing line and then copying the failed command and
>     pasting it
>     >> to the bash prompt it successfully generated the output target,
>     and
>     >> then the build process run further when a similar situation
>     happened.
>     >> Also pasting the failed command run in the bash without any
>     problem,
>     >> and build continued… until the next.
>     > Without seeing the errors I can't say much. I very much doubt
>     that you
>     > are running with gcc as the compiler though. Configure isn't easily
>     > fooled into using a different compiler to what it prefers, and I
>     would
>     > expect things to crash and burn pretty early if you actually did.
>     >
>     > /Erik
>     >>
>     >> I have tried to run make JOBS=1, but did not help, strangely I
>     have
>     >> still seen in the log make[3] and make[4] logs which suggested
>     that
>     >> there are more than one make jobs were running. Also tried
>     .configure
>     >> --with-output-sync=recurse without success (same symptoms)
>     >>
>     >> Let me know your thoughts.
>     >>
>     >> Best regards,
>     >>
>     >> Peter
>     >>
>     >> Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for
>     >> Windows 10
>     >>
>     >> From: Magnus Ihse Bursie<mailto:[hidden email]>
>     >> Sent: Wednesday, October 4, 2017 1:04 AM
>     >> To: Peter Budai<mailto:[hidden email]>;
>     >> [hidden email]
>     <mailto:[hidden email]><mailto:[hidden email]>
>     <mailto:[hidden email]>
>     >> Subject: Re: Building OpenJDK9 on MSYS2
>     >>
>     >> Actually, it wasn't so much remaining trouble. :-) I fired up
>     msys2 and
>     >> checked out where I left off. It turned out that the remaining
>     snag was
>     >> that msys2 tries to convert command lines automatically, from
>     "unix"
>     >> style paths to "windows" style paths. Unfortunately, it does
>     not do this
>     >> very well and it breaks all sorts of things. We already have a
>     FIXPATH
>     >> solution in place which deals with this, so basically all I had
>     to do
>     >> was disable this (by setting MSYS2_ARG_CONV_EXCL to "*").
>     However, this
>     >> broke our cygpath replacement hack (!) so I had to disable it
>     there.
>     >> Sigh. Anyway, with those fixes it ran and worked well. (I also
>     >> discovered and fixed a bug related to how we set up the FIXPATH
>     variable
>     >> on msys, but it only triggers in certain circumstances).
>     >>
>     >> With this patch I now jdk9 seems to build fine on msys2. It
>     should apply
>     >> cleanly on jdk9/jdk9. Since it turned out to be so trivial,
>     I'll try to
>     >> get it in in jdk10.
>     >>
>     >> Here's the patch if you want to apply it yourself:
>     >>
>     >> diff -r a08cbfc0e4ec common/autoconf/basics_windows.m4
>     >> --- a/common/autoconf/basics_windows.m4    Thu Aug 03 18:56:56
>     2017
>     >> +0000
>     >> +++ b/common/autoconf/basics_windows.m4    Wed Oct 04 00:53:58
>     2017
>     >> +0200
>     >> @@ -42,7 +42,7 @@
>     >>        windows_path=`$CYGPATH -m "$unix_path"`
>     >>        $1="$windows_path"
>     >>      elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then
>     >> -    windows_path=`cmd //c echo $unix_path`
>     >> +    windows_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $unix_path`
>     >>        $1="$windows_path"
>     >>      fi
>     >>    ])
>     >> @@ -136,6 +136,16 @@
>     >>      fi
>     >>    ])
>     >>
>     >> +AC_DEFUN([BASIC_MSYS_UPDATE_FIXPATH],
>     >> +[
>     >> +  # Take all collected prefixes and turn them into a
>     >> -m/c/foo@/c/bar@... command line
>     >> +  # @ was chosen as separator to minimize risk of other tools
>     messing
>     >> around with it
>     >> +  all_unique_prefixes=`echo "${all_fixpath_prefixes@<:@@@:>@}" \
>     >> +      | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
>     >> +  fixpath_argument_list=`echo $all_unique_prefixes  | tr ' ' '@'`
>     >> +  FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
>     >> +])
>     >> +
>     >>    AC_DEFUN([BASIC_FIXUP_PATH_MSYS],
>     >>    [
>     >>      path="[$]$1"
>     >> @@ -143,7 +153,7 @@
>     >>      new_path="$path"
>     >>      if test "x$has_colon" = x; then
>     >>        # Not in mixed or Windows style, start by that.
>     >> -    new_path=`cmd //c echo $path`
>     >> +    new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $path`
>     >>      fi
>     >>
>     >> BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
>     >> @@ -155,6 +165,8 @@
>     >>
>     >>      # Save the first 10 bytes of this path to the storage, so
>     fixpath
>     >> can work.
>     >> all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
>     >> "${new_path:0:10}")
>     >> +  # We might need to re-evaluate FIXPATH.
>     >> +  BASIC_MSYS_UPDATE_FIXPATH
>     >>    ])
>     >>
>     >>    AC_DEFUN([BASIC_FIXUP_EXECUTABLE_CYGWIN],
>     >> @@ -293,7 +305,7 @@
>     >>        # Do not save /bin paths to all_fixpath_prefixes!
>     >>      else
>     >>        # Not in mixed or Windows style, start by that.
>     >> -    new_path=`cmd //c echo $new_path`
>     >> +    new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $new_path`
>     >> BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
>     >>        # Output is in $new_path
>     >> BASIC_WINDOWS_REWRITE_AS_UNIX_PATH(new_path)
>     >> @@ -302,6 +314,8 @@
>     >>
>     >>        # Save the first 10 bytes of this path to the storage,
>     so fixpath
>     >> can work.
>     >> all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
>     >> "${new_path:0:10}")
>     >> +    # We might need to re-evaluate FIXPATH.
>     >> +    BASIC_MSYS_UPDATE_FIXPATH
>     >>      fi
>     >>    ])
>     >>
>     >> @@ -347,6 +361,10 @@
>     >>        WINDOWS_ENV_VENDOR='msys'
>     >>        WINDOWS_ENV_VERSION="$MSYS_VERSION"
>     >>
>     >> +    # Prohibit msys2 path conversion from trying to be
>     "intelligent",
>     >> and rely
>     >> +    # on fixpath instead.
>     >> +    export MSYS2_ARG_CONV_EXCL="*"
>     >> +
>     >>        AC_MSG_CHECKING([msys root directory as unix-style path])
>     >>        # The cmd output ends with Windows line endings (CR/LF),
>     the grep
>     >> command will strip that away
>     >>        MSYS_ROOT_PATH=`cd / ; cmd /c cd | $GREP ".*"`
>     >> @@ -391,10 +409,7 @@
>     >>        elif test "x$OPENJDK_BUILD_OS_ENV" = xwindows.msys; then
>     >>          # Take all collected prefixes and turn them into a
>     >> -m/c/foo@/c/bar@... command line
>     >>          # @ was chosen as separator to minimize risk of other
>     tools
>     >> messing around with it
>     >> -      all_unique_prefixes=`echo
>     "${all_fixpath_prefixes@<:@@@:>@}" \
>     >> -          | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
>     >> -      fixpath_argument_list=`echo $all_unique_prefixes  | tr '
>     ' '@'`
>     >> -      FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
>     >> +      BASIC_MSYS_UPDATE_FIXPATH
>     >>        fi
>     >>        FIXPATH_SRC_W="$FIXPATH_SRC"
>     >>        FIXPATH_BIN_W="$FIXPATH_BIN"
>     >> diff -r a08cbfc0e4ec common/autoconf/build-aux/config.sub
>     >> --- a/common/autoconf/build-aux/config.sub    Thu Aug 03 18:56:56
>     >> 2017 +0000
>     >> +++ b/common/autoconf/build-aux/config.sub    Wed Oct 04 00:53:58
>     >> 2017 +0200
>     >> @@ -30,7 +30,7 @@
>     >>    DIR=`dirname $0`
>     >>
>     >>    # First, filter out everything that doesn't begin with
>     "aarch64-"
>     >> -if ! echo $* | grep '^aarch64-' >/dev/null ; then
>     >> +if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ; then
>     >>        . $DIR/autoconf-config.sub "$@"
>     >>        # autoconf-config.sub exits, so we never reach here, but
>     just in
>     >>        # case we do:
>     >> @@ -38,13 +38,17 @@
>     >>    fi
>     >>
>     >>    while test $# -gt 0 ; do
>     >> -    case $1 in
>     >> +    case $1 in
>     >>            -- )   # Stop option processing
>     >>                shift; break ;;
>     >>            aarch64-* )
>     >>                config=`echo $1 | sed 's/^aarch64-/arm-/'`
>     >>                sub_args="$sub_args $config"
>     >>                shift; ;;
>     >> +        *-msys )
>     >> +            config=`echo $1 | sed 's/msys/mingw32/'`
>     >> +            sub_args="$sub_args $config"
>     >> +            shift; ;;
>     >>            - )    # Use stdin as input.
>     >>                sub_args="$sub_args $1"
>     >>                shift; break ;;
>     >> diff -r a08cbfc0e4ec common/autoconf/spec.gmk.in
>     >> --- a/common/autoconf/spec.gmk.in    Thu Aug 03 18:56:56 2017 +0000
>     >> +++ b/common/autoconf/spec.gmk.in    Wed Oct 04 00:53:58 2017 +0200
>     >> @@ -120,6 +120,13 @@
>     >>      # On Windows, the Visual Studio toolchain needs the PATH
>     to be
>     >> adjusted
>     >>      # to include Visual Studio tools (this needs to be in
>     cygwin/msys
>     >> style).
>     >>      export PATH:=@VS_PATH@
>     >> +
>     >> +endif
>     >> +
>     >> +ifeq ($(OPENJDK_TARGET_OS_ENV), windows.msys)
>     >> +  # On msys2, prohibit msys path conversion from trying to be
>     >> +  # "intelligent", and rely on fixpath instead.
>     >> +  export MSYS2_ARG_CONV_EXCL:=*
>     >>    endif
>     >>
>     >>    SYSROOT_CFLAGS := @SYSROOT_CFLAGS@
>     >>
>     >> /Magnus
>     >>
>     >> On 2017-10-03 22:34, Magnus Ihse Bursie wrote:
>     >>> I gave msys2 a shot some time ago, but it ended up too much
>     trouble.
>     >>> I'll share some of my notes from that attempt, for what it's
>     worth.
>     >>>
>     >>> To install package X/Y, run "pacman -S X/Y". Missing tools and
>     >>> packages where to find them:
>     >>> cmp: msys/diffutils
>     >>> tar: msys/tar
>     >>> make: msys/make
>     >>> unzip: msys/unzip
>     >>> zip: msys/zip
>     >>>
>     >>> config.sub reports msys as "x86_64-pc-mingw32" but msys2 as
>     >>> "x86_64-pc-msys". This patch adds postprocessing in "our"
>     config.sub
>     >>> to report msys2 similar to msys. (Opinions, including my own
>     :-) may
>     >>> vary if this really is the best way..)
>     >>>
>     >>> diff -r b88023f46daa common/autoconf/build-aux/config.sub
>     >>> --- a/common/autoconf/build-aux/config.sub      Fri Jan 27
>     10:15:41
>     >>> 2017 +0100
>     >>> +++ b/common/autoconf/build-aux/config.sub      Fri Feb 03
>     05:00:25
>     >>> 2017 -0700
>     >>> @@ -30,7 +30,7 @@
>     >>>   DIR=`dirname $0`
>     >>>
>     >>>   # First, filter out everything that doesn't begin with
>     "aarch64-"
>     >>> -if ! echo $* | grep '^aarch64-' >/dev/null ; then
>     >>> +if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ; then
>     >>>       . $DIR/autoconf-config.sub "$@"
>     >>>       # autoconf-config.sub exits, so we never reach here, but
>     just in
>     >>>       # case we do:
>     >>> @@ -45,6 +45,10 @@
>     >>>               config=`echo $1 | sed 's/^aarch64-/arm-/'`
>     >>>               sub_args="$sub_args $config"
>     >>>               shift; ;;
>     >>> +        *-msys )
>     >>> +            config=`echo $1 | sed 's/msys/mingw32/'`
>     >>> +            sub_args="$sub_args $config"
>     >>> +            shift; ;;
>     >>>           - )    # Use stdin as input.
>     >>>               sub_args="$sub_args $1"
>     >>>               shift; break ;;
>     >>>
>     >>> If I remember correctly, this got me past the configure stage
>     at the
>     >>> time.
>     >>>
>     >>> I don't think it's very hard to get it to work on msys2, I
>     just ran
>     >>> into one snag too many and didn't think msys2 would be used by
>     anyone.
>     >>>
>     >>> /Magnus
>     >>>
>     >>> On 2017-10-03 17:20, Peter Budai wrote:
>     >>>> Hello,
>     >>>>
>     >>>> According to
>     >>>>
>     http://hg.openjdk.java.net/jdk9/jdk9/file/a08cbfc0e4ec/common/doc/building.html
>
>     >>>>
>     >>>> “msys2 and the new Windows Subsystem for Linux (WSL) would
>     likely be
>     >>>> possible to support in a future version but that would require a
>     >>>> community effort to implement”
>     >>>>
>     >>>> I’d like to help making the OpenJDK 9 build working on msys2.
>     What is
>     >>>> the best way to move forward? Is there a similar effort in
>     progress?
>     >>>>
>     >>>> Thank you and best regards,
>     >>>>
>     >>>> Peter
>     >>>>
>     >>>>
>     >
>

Reply | Threaded
Open this post in threaded view
|

Re: Building OpenJDK9 on MSYS2

Magnus Ihse Bursie
In reply to this post by Peter Budai
On 2017-10-07 10:14, Peter Budai wrote:
>
> The configure of OpenJDK overwrites the SHELL. Actually it is using
> bash, but for the arguments it was using “-e -o pipefail”. I have
> figured that for MSYS2 bash what is needed as bash arguments is “-e -l
> -c -o pipefail”
>
> That looks like solving this problem, and now the real issues are
> surfacing.
>

FWIW, "-l" makes bash behave like a login shell. Most likely you are
changing bash's behavior in one of your login scripts, and that change
is what's really needed.

/Magnus

> Peter
>
> *From: *Peter Budai <mailto:[hidden email]>
> *Sent: *Friday, October 6, 2017 6:43 PM
> *To: *Magnus Ihse Bursie <mailto:[hidden email]>; Erik
> Joelsson <mailto:[hidden email]>; [hidden email]
> <mailto:[hidden email]>
> *Subject: *RE: Building OpenJDK9 on MSYS2
>
> Magnus,
>
> I have followed your suggestion and removed the fixpath prefixes from
> gcc related compile tools, and left only the fixpath prefix _only_ for
> the Boot JDK related tools in place.
>
>  1. As  I follow the process, all java and javac related compile steps
>     are running properly
>  2. When the process reaches gcc related steps I got the error message
>     at the same place as before (no fixpath). If I execute that
>     command from the bash prompt, it creates the output:
>
> $ ( /usr/bin/gawk '/@@END_COPYRIGHT@@/{exit}1'
> /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template
> && /C/msys64/mingw64/bin/gcc -E -x c
> /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template
> 2> >(/usr/bin/grep -v '^SocketOptionRegistry.java.template$' >&2) |
> /usr/bin/gawk '/@@START_HERE@@/,0' |  /usr/bin/sed -e
> 's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED FILE - DO NOT EDIT/' -e
> 's/PREFIX_//' -e 's/^#.*//' ) >
> /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java
>
> As I have mentioned the parameters are replaced by the bash automatically
>
>  3. Then build continues, then little later stops at a super simple
>     command:
>
> mv
> /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBuffer.java.tmp
> /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBuffer.java
>
>            Needless to say, the ByteBuffer.java.tmp file DOES exist.
> And running the above command from the bash works, and build continues.
>
>  4. A few similar cases (stops) with DirectByteBuffer and
>     DirectByteBufferR
>
> Currently I try to explore how that might relate to the MSYS2 bash and
> make, somehow it behaves differently
>
> If you have any other suggestion, let me know.
>
> Best regards,
>
> Peter
>
> *From: *Peter Budai <mailto:[hidden email]>
> *Sent: *Thursday, October 5, 2017 3:52 PM
> *To: *Magnus Ihse Bursie <mailto:[hidden email]>; Erik
> Joelsson <mailto:[hidden email]>; [hidden email]
> <mailto:[hidden email]>
> *Subject: *RE: Building OpenJDK9 on MSYS2
>
> Hi Magnus,
>
> So first of all, here is the current patch, which I was not able to
> attach: https://pastebin.com/pwT4Ynxc
>
> >> That's surprising, since gcc is prefixed with fixpath, which it should not.
>
> Actually you DO need fixpath IMHO.
>
> This is a mingw64 version of the gcc (/C/msys64/mingw64/bin/gcc),
> which is a fully functional Windows executable, /which expects Windows
> formatted path arguments/.
>
> As the updated build process uses EXPORT MSYS2_ARG_CONV_EXCL=* (see
> that patch), none of the command line arguments are converted from the
> unix path to Windows, but fixpath does that conversion. There is a
> wiki describing more details on this:
>
> https://github.com/msys2/msys2/wiki/Porting#user-content-filesystem-namespaces
>
>
> >>I have a hard time believing this is a race condition. On the other
> hand, this stuff is weird, we're misusing the C preprocessor to
> process defines in java code, so I'm not surprised it breaks down.
>
> >>I don't know why it succeeded when run on the command line, though.
>
> When I execute that command from the /bash/ command line there is no
> EXPORT MSYS2_ARG_CONV_EXCL, but the bash itself does the automatic
> conversion of the arguments. Maybe it has something to do how fixpath
> does CreateProcess?
>
> Does that help?
>
> Best regards,
>
> Peter
>
> Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for
> Windows 10
>
> *From: *Magnus Ihse Bursie <mailto:[hidden email]>
> *Sent: *Thursday, October 5, 2017 12:13 PM
> *To: *Peter Budai <mailto:[hidden email]>; Erik Joelsson
> <mailto:[hidden email]>; [hidden email]
> <mailto:[hidden email]>
> *Subject: *Re: Building OpenJDK9 on MSYS2
>
> On 2017-10-05 11:59, Peter Budai wrote:
>
>     Hi Magnus and Erik,
>
>     I really appreciate your quick feedback. I assumed that it won’t
>     be easy, but I just don’t feel I should give up now  - maybe later
>     when I see the real scale of work. So bear with me for a time being.
>
>     Attached is a patch which already includes Magnus’ changes, plus a
>     few which I have added:
>
>     •basically enabling gcc for windows,
>
>     •and modifying a logic for compiling fixpath (before that it was
>     using hard-coded MS VSC compile flags)
>
>
> Actually, you must make sure fixpath is *not* used for the toolchain,
> since gcc uses unix style paths.
> (However, other tools such as java will still need it.)
>
>     So here is what I have as the result of configure:
>
>     ====================================================
>
>     The existing configuration has been successfully updated in
>
>     /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release
>
>     using configure arguments '--disable-freetype-bundling
>     --disable-javac-server'.
>
>     Configuration summary:
>
>     * Debug level:    release
>
>     * HS debug level: product
>
>     * JDK variant:    normal
>
>     * JVM variants:   server
>
>     * OpenJDK target: OS: windows, CPU architecture: x86, address
>     length: 64
>
>     * Version string: 9-internal+0-adhoc.peterbud.jdk9 (9-internal)
>
>     Tools summary:
>
>     * Environment:    msys version 2.9.0(0.318/5/3) (root at /C/msys64)
>
>     * Boot JDK:       java version "1.8.0_144"  Java(TM) SE Runtime
>     Environment (build 1.8.0_144-b01)  Java HotSpot(TM) 64-Bit Server
>     VM (build 25.144-b01, mixed mode)   (at /c/progra~1/java/jdk18~1.0_1)
>
>     * Toolchain:      gcc (GNU Compiler Collection)
>
>     * C Compiler:     Version 7.2.0 (at /C/msys64/mingw64/bin/gcc)
>
>     * C++ Compiler:   Version 7.2.0 (at /c/msys64/mingw64/bin/g++)
>
>     Build performance summary:
>
>     * Cores to use:   4
>
>     * Memory limit:   16216 MB
>
>     Its clear says that the toolchain is gcc 7.2 (BTW there is no
>     Visual Studio on this machine)
>
>     Now for the details of the config log, you can see here:
>     https://pastebin.com/MN2ZYcHH
>
>     And about the build process and the error I get:
>
>     $ make JOBS=1
>
>     Building target 'default (exploded-image)' in configuration
>     'windows-x86_64-normal-server-release'
>
>     Compiling 8 files for BUILD_TOOLS_LANGTOOLS
>
>     Compiling 17 properties into resource bundles for jdk.compiler
>
>     Parsing 1 properties into enum-like class for jdk.compiler
>
>     Compiling 19 properties into resource bundles for jdk.javadoc
>
>     Compiling 12 properties into resource bundles for jdk.jdeps
>
>     Compiling 7 properties into resource bundles for jdk.jshell
>
>     Compiling 117 files for BUILD_INTERIM_java.compiler
>
>     Compiling 396 files for BUILD_INTERIM_jdk.compiler
>
>     Compiling 61 files for BUILD_INTERIM_jdk.jdeps
>
>     Compiling 457 files for BUILD_INTERIM_jdk.javadoc
>
>     Note: Some input files use or override a deprecated API.
>
>     Note: Recompile with -Xlint:deprecation for details.
>
>     Compiling 159 files for BUILD_TOOLS_JDK
>
>     Note: Some input files use unchecked or unsafe operations.
>
>     Note: Recompile with -Xlint:unchecked for details.
>
>     make[3]: *** [GensrcMisc.gmk:78:
>     /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java]
>     Error 1
>
>     make[3]: *** Deleting file
>     '/C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java'
>
>     make[2]: *** [make/Main.gmk:115: java.base-gensrc-jdk] Error 2
>
>     ERROR: Build failed for target 'default (exploded-image)' in
>     configuration 'windows-x86_64-normal-server-release' (exit code 2)
>
>     No indication of failed target found.
>
>     Hint: Try searching the build log for '] Error'.
>
>     Hint: See common/doc/building.html#troubleshooting for assistance.
>
>     make[1]: *** [/home/peterbud/jdk9/make/Init.gmk:296: main] Error 2
>
>     make: *** [/home/peterbud/jdk9/make/Init.gmk:185: default] Error 2
>
>     If I run here
>
>     make JOBS=1 LOG=debug
>
>     The failing line seems to be this:
>
>     ( /usr/bin/gawk '/@@END_COPYRIGHT@@/{exit}1'
>     /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template
>     &&
>     /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/configure-support/bin/fixpath.exe
>     -m/C/msys64/@/c/msys64/@/c/progra~ /C/msys64/mingw64/bin/gcc -E -x
>     c
>     /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template
>     2> >(/usr/bin/grep -v '^SocketOptionRegistry.java.template$' >&2)
>     | /usr/bin/gawk '/@@START_HERE@@/,0' |  /usr/bin/sed -e
>     's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED FILE - DO NOT
>     EDIT/' -e 's/PREFIX_//' -e 's/^#.*//' ) >
>     /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java
>
>     make[3]: *** [GensrcMisc.gmk:78:
>     /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java]
>     Error 1
>
>     Now the interesting is: if I copy this line above to the bash
>     prompt, it runs without problem, and the file
>     support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java
>
> That's surprising, since gcc is prefixed with fixpath, which it should
> not.
>
> I have a hard time believing this is a race condition. On the other
> hand, this stuff is weird, we're misusing the C preprocessor to
> process defines in java code, so I'm not suprised it breaks down. I
> don't know why it succeeded when run on the command line, though. My
> suggestion is to just do some quick and dirty hack around this: take
> the file you manage to generate and just copy it in during the build
> instead. If you can get round this, you might start seeing some *real*
> problems. :-)
>
> Also, my suggestion is that you try running "make hotspot" to cut to
> the chase. Compiling hotspot will likely be the hardest thing. Or even
> "make -k hotspot" to get an assessment of the amount of work ahead of you.
>
> /Magnus
>
>     Is produced.
>
>     Then I can again issue
>
>     make JOBS=1 LOG=debug
>
>     And the compile process is being continued, until a similar error
>     pops up with a different generated file. I have an assumption that
>     this happens because make is still running parallel jobs, despite
>     JOBS=1 but I’m not sure.
>
>     How could I best tackle this?
>
>     Thank you and best regards,
>
>     Peter
>
>     Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986>
>     for Windows 10
>
>     *From: *Magnus Ihse Bursie <mailto:[hidden email]>
>     *Sent: *Thursday, October 5, 2017 11:33 AM
>     *To: *Erik Joelsson <mailto:[hidden email]>; Peter Budai
>     <mailto:[hidden email]>; [hidden email]
>     <mailto:[hidden email]>
>     *Subject: *Re: Building OpenJDK9 on MSYS2
>
>     On 2017-10-05 10:10, Erik Joelsson wrote:
>     > Hello Peter,
>     >
>     >
>     > On 2017-10-04 21:15, Peter Budai wrote:
>     >> Hi Magnus,
>     >>
>     >> Thanks for the quick reply I’ll check these patches with msys2.
>     >>
>     >> Let me specify with more details what I’d like to achieve: I’d
>     like
>     >> to build OpenJDK9 with MSYS2 MINGW64 environment using gcc
>     toolchain.
>     >> (I’m not sure how familiar are you with MSYS2, but there are 3
>     >> different environments: MSYS2, MINGW32 and MINGW64). In theory
>     >> MINGW64 with gcc is the closes you can get on Windows platform
>     as a
>     >> gcc unix like build environment, which produces still a native
>     64-bit
>     >> executable on Windows.
>     >>
>     >> I’m not very familiar with OpenJDK yet, so therefore I’d like
>     to hear
>     >> your opinion: how realistic is that?
>     > Sorry to disappoint, but I would say that requires major work.
>     There
>     > is a strong historic assumption that windows builds are done using
>     > Visual Studio. We have abstracted away some of it in configure (see
>     > TOOLCHAIN_TYPE), but it's very far from enough to change compiler
>     > environment for a Windows build. The native sources are also
>     bound to
>     > make a lot of such assumptions. I would expect the changes
>     needed to
>     > be in the thousands of lines of code.
>
>     I agree that it requires hard work (even if "thousands" might be an
>     overestimation I think, but "hundreds" is not enough, so it's the
>     right
>     magnitude). On the other hand, it would be really good if we did sort
>     things out, so that we had proper conditions based on OS vs
>     compiler/toolchain.
>
>     If you really want to start, the first thing is to patch
>     toolchain.m4 to
>     VALID_TOOLCHAINS_windows="microsoft gcc"
>     and then call configure using "bash configure
>     --with-toolchain-type=gcc".
>
>     As Erik, I doubt you will come very far before things starts
>     tumbling down.
>
>     >
>     > When we say supporting the build in msys2 instead of cygwin, we
>     just
>     > mean using msys2 as the unix emulating layer for our tools like
>     > make/bash/grep/sed etc.
>     >
>     > One think I have done successfully is running the build in WSL
>     > (Windows Subsystem for Linux), but that isn't all that helpful
>     as WSL
>     > for practical purposes is more or less like running Linux in a
>     VM, so
>     > the build sees a Linux system and builds a Linux binary.
>     >> As a side note: with MINGW64 I have managed to run configure phase
>     >> successfully for OpenJDK. The compile process has also started and
>     >> went for a while, but interestingly I run into some kind of race
>     >> conditions as make stopped with an error. Using LOG=debug I
>     have fond
>     >> the failing line and then copying the failed command and
>     pasting it
>     >> to the bash prompt it successfully generated the output target,
>     and
>     >> then the build process run further when a similar situation
>     happened.
>     >> Also pasting the failed command run in the bash without any
>     problem,
>     >> and build continued… until the next.
>     > Without seeing the errors I can't say much. I very much doubt
>     that you
>     > are running with gcc as the compiler though. Configure isn't easily
>     > fooled into using a different compiler to what it prefers, and I
>     would
>     > expect things to crash and burn pretty early if you actually did.
>     >
>     > /Erik
>     >>
>     >> I have tried to run make JOBS=1, but did not help, strangely I
>     have
>     >> still seen in the log make[3] and make[4] logs which suggested
>     that
>     >> there are more than one make jobs were running. Also tried
>     .configure
>     >> --with-output-sync=recurse without success (same symptoms)
>     >>
>     >> Let me know your thoughts.
>     >>
>     >> Best regards,
>     >>
>     >> Peter
>     >>
>     >> Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for
>     >> Windows 10
>     >>
>     >> From: Magnus Ihse Bursie<mailto:[hidden email]>
>     >> Sent: Wednesday, October 4, 2017 1:04 AM
>     >> To: Peter Budai<mailto:[hidden email]>;
>     >> [hidden email]
>     <mailto:[hidden email]><mailto:[hidden email]>
>     <mailto:[hidden email]>
>     >> Subject: Re: Building OpenJDK9 on MSYS2
>     >>
>     >> Actually, it wasn't so much remaining trouble. :-) I fired up
>     msys2 and
>     >> checked out where I left off. It turned out that the remaining
>     snag was
>     >> that msys2 tries to convert command lines automatically, from
>     "unix"
>     >> style paths to "windows" style paths. Unfortunately, it does
>     not do this
>     >> very well and it breaks all sorts of things. We already have a
>     FIXPATH
>     >> solution in place which deals with this, so basically all I had
>     to do
>     >> was disable this (by setting MSYS2_ARG_CONV_EXCL to "*").
>     However, this
>     >> broke our cygpath replacement hack (!) so I had to disable it
>     there.
>     >> Sigh. Anyway, with those fixes it ran and worked well. (I also
>     >> discovered and fixed a bug related to how we set up the FIXPATH
>     variable
>     >> on msys, but it only triggers in certain circumstances).
>     >>
>     >> With this patch I now jdk9 seems to build fine on msys2. It
>     should apply
>     >> cleanly on jdk9/jdk9. Since it turned out to be so trivial,
>     I'll try to
>     >> get it in in jdk10.
>     >>
>     >> Here's the patch if you want to apply it yourself:
>     >>
>     >> diff -r a08cbfc0e4ec common/autoconf/basics_windows.m4
>     >> --- a/common/autoconf/basics_windows.m4    Thu Aug 03 18:56:56
>     2017
>     >> +0000
>     >> +++ b/common/autoconf/basics_windows.m4    Wed Oct 04 00:53:58
>     2017
>     >> +0200
>     >> @@ -42,7 +42,7 @@
>     >>        windows_path=`$CYGPATH -m "$unix_path"`
>     >>        $1="$windows_path"
>     >>      elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then
>     >> -    windows_path=`cmd //c echo $unix_path`
>     >> +    windows_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $unix_path`
>     >>        $1="$windows_path"
>     >>      fi
>     >>    ])
>     >> @@ -136,6 +136,16 @@
>     >>      fi
>     >>    ])
>     >>
>     >> +AC_DEFUN([BASIC_MSYS_UPDATE_FIXPATH],
>     >> +[
>     >> +  # Take all collected prefixes and turn them into a
>     >> -m/c/foo@/c/bar@... command line
>     >> +  # @ was chosen as separator to minimize risk of other tools
>     messing
>     >> around with it
>     >> +  all_unique_prefixes=`echo "${all_fixpath_prefixes@<:@@@:>@}" \
>     >> +      | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
>     >> +  fixpath_argument_list=`echo $all_unique_prefixes  | tr ' ' '@'`
>     >> +  FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
>     >> +])
>     >> +
>     >>    AC_DEFUN([BASIC_FIXUP_PATH_MSYS],
>     >>    [
>     >>      path="[$]$1"
>     >> @@ -143,7 +153,7 @@
>     >>      new_path="$path"
>     >>      if test "x$has_colon" = x; then
>     >>        # Not in mixed or Windows style, start by that.
>     >> -    new_path=`cmd //c echo $path`
>     >> +    new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $path`
>     >>      fi
>     >>
>     >> BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
>     >> @@ -155,6 +165,8 @@
>     >>
>     >>      # Save the first 10 bytes of this path to the storage, so
>     fixpath
>     >> can work.
>     >> all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
>     >> "${new_path:0:10}")
>     >> +  # We might need to re-evaluate FIXPATH.
>     >> +  BASIC_MSYS_UPDATE_FIXPATH
>     >>    ])
>     >>
>     >>    AC_DEFUN([BASIC_FIXUP_EXECUTABLE_CYGWIN],
>     >> @@ -293,7 +305,7 @@
>     >>        # Do not save /bin paths to all_fixpath_prefixes!
>     >>      else
>     >>        # Not in mixed or Windows style, start by that.
>     >> -    new_path=`cmd //c echo $new_path`
>     >> +    new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $new_path`
>     >> BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
>     >>        # Output is in $new_path
>     >> BASIC_WINDOWS_REWRITE_AS_UNIX_PATH(new_path)
>     >> @@ -302,6 +314,8 @@
>     >>
>     >>        # Save the first 10 bytes of this path to the storage,
>     so fixpath
>     >> can work.
>     >> all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
>     >> "${new_path:0:10}")
>     >> +    # We might need to re-evaluate FIXPATH.
>     >> +    BASIC_MSYS_UPDATE_FIXPATH
>     >>      fi
>     >>    ])
>     >>
>     >> @@ -347,6 +361,10 @@
>     >>        WINDOWS_ENV_VENDOR='msys'
>     >>        WINDOWS_ENV_VERSION="$MSYS_VERSION"
>     >>
>     >> +    # Prohibit msys2 path conversion from trying to be
>     "intelligent",
>     >> and rely
>     >> +    # on fixpath instead.
>     >> +    export MSYS2_ARG_CONV_EXCL="*"
>     >> +
>     >>        AC_MSG_CHECKING([msys root directory as unix-style path])
>     >>        # The cmd output ends with Windows line endings (CR/LF),
>     the grep
>     >> command will strip that away
>     >>        MSYS_ROOT_PATH=`cd / ; cmd /c cd | $GREP ".*"`
>     >> @@ -391,10 +409,7 @@
>     >>        elif test "x$OPENJDK_BUILD_OS_ENV" = xwindows.msys; then
>     >>          # Take all collected prefixes and turn them into a
>     >> -m/c/foo@/c/bar@... command line
>     >>          # @ was chosen as separator to minimize risk of other
>     tools
>     >> messing around with it
>     >> -      all_unique_prefixes=`echo
>     "${all_fixpath_prefixes@<:@@@:>@}" \
>     >> -          | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
>     >> -      fixpath_argument_list=`echo $all_unique_prefixes  | tr '
>     ' '@'`
>     >> -      FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
>     >> +      BASIC_MSYS_UPDATE_FIXPATH
>     >>        fi
>     >>        FIXPATH_SRC_W="$FIXPATH_SRC"
>     >>        FIXPATH_BIN_W="$FIXPATH_BIN"
>     >> diff -r a08cbfc0e4ec common/autoconf/build-aux/config.sub
>     >> --- a/common/autoconf/build-aux/config.sub    Thu Aug 03 18:56:56
>     >> 2017 +0000
>     >> +++ b/common/autoconf/build-aux/config.sub    Wed Oct 04 00:53:58
>     >> 2017 +0200
>     >> @@ -30,7 +30,7 @@
>     >>    DIR=`dirname $0`
>     >>
>     >>    # First, filter out everything that doesn't begin with
>     "aarch64-"
>     >> -if ! echo $* | grep '^aarch64-' >/dev/null ; then
>     >> +if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ; then
>     >>        . $DIR/autoconf-config.sub "$@"
>     >>        # autoconf-config.sub exits, so we never reach here, but
>     just in
>     >>        # case we do:
>     >> @@ -38,13 +38,17 @@
>     >>    fi
>     >>
>     >>    while test $# -gt 0 ; do
>     >> -    case $1 in
>     >> +    case $1 in
>     >>            -- )   # Stop option processing
>     >>                shift; break ;;
>     >>            aarch64-* )
>     >>                config=`echo $1 | sed 's/^aarch64-/arm-/'`
>     >>                sub_args="$sub_args $config"
>     >>                shift; ;;
>     >> +        *-msys )
>     >> +            config=`echo $1 | sed 's/msys/mingw32/'`
>     >> +            sub_args="$sub_args $config"
>     >> +            shift; ;;
>     >>            - )    # Use stdin as input.
>     >>                sub_args="$sub_args $1"
>     >>                shift; break ;;
>     >> diff -r a08cbfc0e4ec common/autoconf/spec.gmk.in
>     >> --- a/common/autoconf/spec.gmk.in    Thu Aug 03 18:56:56 2017 +0000
>     >> +++ b/common/autoconf/spec.gmk.in    Wed Oct 04 00:53:58 2017 +0200
>     >> @@ -120,6 +120,13 @@
>     >>      # On Windows, the Visual Studio toolchain needs the PATH
>     to be
>     >> adjusted
>     >>      # to include Visual Studio tools (this needs to be in
>     cygwin/msys
>     >> style).
>     >>      export PATH:=@VS_PATH@
>     >> +
>     >> +endif
>     >> +
>     >> +ifeq ($(OPENJDK_TARGET_OS_ENV), windows.msys)
>     >> +  # On msys2, prohibit msys path conversion from trying to be
>     >> +  # "intelligent", and rely on fixpath instead.
>     >> +  export MSYS2_ARG_CONV_EXCL:=*
>     >>    endif
>     >>
>     >>    SYSROOT_CFLAGS := @SYSROOT_CFLAGS@
>     >>
>     >> /Magnus
>     >>
>     >> On 2017-10-03 22:34, Magnus Ihse Bursie wrote:
>     >>> I gave msys2 a shot some time ago, but it ended up too much
>     trouble.
>     >>> I'll share some of my notes from that attempt, for what it's
>     worth.
>     >>>
>     >>> To install package X/Y, run "pacman -S X/Y". Missing tools and
>     >>> packages where to find them:
>     >>> cmp: msys/diffutils
>     >>> tar: msys/tar
>     >>> make: msys/make
>     >>> unzip: msys/unzip
>     >>> zip: msys/zip
>     >>>
>     >>> config.sub reports msys as "x86_64-pc-mingw32" but msys2 as
>     >>> "x86_64-pc-msys". This patch adds postprocessing in "our"
>     config.sub
>     >>> to report msys2 similar to msys. (Opinions, including my own
>     :-) may
>     >>> vary if this really is the best way..)
>     >>>
>     >>> diff -r b88023f46daa common/autoconf/build-aux/config.sub
>     >>> --- a/common/autoconf/build-aux/config.sub      Fri Jan 27
>     10:15:41
>     >>> 2017 +0100
>     >>> +++ b/common/autoconf/build-aux/config.sub      Fri Feb 03
>     05:00:25
>     >>> 2017 -0700
>     >>> @@ -30,7 +30,7 @@
>     >>>   DIR=`dirname $0`
>     >>>
>     >>>   # First, filter out everything that doesn't begin with
>     "aarch64-"
>     >>> -if ! echo $* | grep '^aarch64-' >/dev/null ; then
>     >>> +if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ; then
>     >>>       . $DIR/autoconf-config.sub "$@"
>     >>>       # autoconf-config.sub exits, so we never reach here, but
>     just in
>     >>>       # case we do:
>     >>> @@ -45,6 +45,10 @@
>     >>>               config=`echo $1 | sed 's/^aarch64-/arm-/'`
>     >>>               sub_args="$sub_args $config"
>     >>>               shift; ;;
>     >>> +        *-msys )
>     >>> +            config=`echo $1 | sed 's/msys/mingw32/'`
>     >>> +            sub_args="$sub_args $config"
>     >>> +            shift; ;;
>     >>>           - )    # Use stdin as input.
>     >>>               sub_args="$sub_args $1"
>     >>>               shift; break ;;
>     >>>
>     >>> If I remember correctly, this got me past the configure stage
>     at the
>     >>> time.
>     >>>
>     >>> I don't think it's very hard to get it to work on msys2, I
>     just ran
>     >>> into one snag too many and didn't think msys2 would be used by
>     anyone.
>     >>>
>     >>> /Magnus
>     >>>
>     >>> On 2017-10-03 17:20, Peter Budai wrote:
>     >>>> Hello,
>     >>>>
>     >>>> According to
>     >>>>
>     http://hg.openjdk.java.net/jdk9/jdk9/file/a08cbfc0e4ec/common/doc/building.html
>
>     >>>>
>     >>>> “msys2 and the new Windows Subsystem for Linux (WSL) would
>     likely be
>     >>>> possible to support in a future version but that would require a
>     >>>> community effort to implement”
>     >>>>
>     >>>> I’d like to help making the OpenJDK 9 build working on msys2.
>     What is
>     >>>> the best way to move forward? Is there a similar effort in
>     progress?
>     >>>>
>     >>>> Thank you and best regards,
>     >>>>
>     >>>> Peter
>     >>>>
>     >>>>
>     >
>

Reply | Threaded
Open this post in threaded view
|

RE: Building OpenJDK9 on MSYS2

Peter Budai
Thanks Magnus & Erik

First thanks for your support and kind words!

Magnus, I have checked .bash_profile, .bashrc but they seem to be empty (everything is commented out). You can check with a default MSYS2 install, I have not changed these files at all. If you find thee something specific I can give a try here as well.

Let me give also a quick status update, where am I with building hotspot:

  *   I guess its still the beginning, but I have managed to compile jvm.dll with almost 700 object file: with debug info the dll is around 700 MB 😊
  *   I made only surgical, minimal changes to the source, and so far it looks reasonable. I have encountered 3 scenarios where changes were necessary:
     *   When in makefiles conditionals were using assuming that if target_os is windows then it is visual studio compiler/linker. Obviously these conditionals had to be reviewed in a few places and if necessary were changes to check the toolchain=Microsoft
     *   I got a few warnings as gcc 7.2 uncovered some code problems in windows specific codes, where before that MSVC I guess did not say a word…
     *   And I had like 6-7 places where the code was using MSVC specific __try … __except structures which gcc does not know. Do you have a suggestion how to approach them? I can do ugly #ifdefs (I would avoid that) but I have also seen some solutions to replace them with a code which gcc can compile (http://www.programmingunlimited.net/siteexec/content.cgi?page=mingw-seh ) – but before doing that  though I would ask first you on the purpose of those
  *   What bothers me is that I was not able to do incremental builds: when an error occurs, and build stops, then after making change in the CPP source the build cannot continue, I always got an error message:

/C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/hotspot/variant-server/tools/adlc/objs/adlparse.d.targets:1: *** missing target pattern.  Stop.
make[2]: *** [make/Main.gmk:256: hotspot-server-gensrc] Error 2

If I do a ‘make clean’ and restart the build then it nicely compiles.

Question 1: Is there a way to  resume such builds without ‘make clean’?

Question 2: What would be the best way to submit/share the patches for your thorough review?

P.

Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10

From: Magnus Ihse Bursie<mailto:[hidden email]>
Sent: Tuesday, October 10, 2017 10:04 AM
To: Peter Budai<mailto:[hidden email]>; Erik Joelsson<mailto:[hidden email]>; [hidden email]<mailto:[hidden email]>
Subject: Re: Building OpenJDK9 on MSYS2

On 2017-10-07 10:14, Peter Budai wrote:
The configure of OpenJDK overwrites the SHELL. Actually it is using bash, but for the arguments it was using “-e -o pipefail”. I have figured that for MSYS2 bash what is needed as bash arguments is “-e -l -c -o pipefail”

That looks like solving this problem, and now the real issues are surfacing.

FWIW, "-l" makes bash behave like a login shell. Most likely you are changing bash's behavior in one of your login scripts, and that change is what's really needed.

/Magnus



Peter

From: Peter Budai<mailto:[hidden email]>
Sent: Friday, October 6, 2017 6:43 PM
To: Magnus Ihse Bursie<mailto:[hidden email]>; Erik Joelsson<mailto:[hidden email]>; [hidden email]<mailto:[hidden email]>
Subject: RE: Building OpenJDK9 on MSYS2

Magnus,

I have followed your suggestion and removed the fixpath prefixes from gcc related compile tools, and left only the fixpath prefix _only_ for the Boot JDK related tools in place.

1)      As  I follow the process, all java and javac related compile steps are running properly
2)      When the process reaches gcc related steps I got the error message at the same place as before (no fixpath). If I execute that command from the bash prompt, it creates the output:

$ ( /usr/bin/gawk '/@@END_COPYRIGHT@@/{exit}1' /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template && /C/msys64/mingw64/bin/gcc -E -x c  /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template 2> >(/usr/bin/grep -v '^SocketOptionRegistry.java.template$' >&2) | /usr/bin/gawk '/@@START_HERE@@/,0' |  /usr/bin/sed -e 's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED FILE - DO NOT EDIT/' -e 's/PREFIX_//' -e 's/^#.*//' ) > /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java

As I have mentioned the parameters are replaced by the bash automatically
3)      Then build continues, then little later stops at a super simple command:

mv /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBuffer.java.tmp /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBuffer.java
           Needless to say, the ByteBuffer.java.tmp file DOES exist. And running the above command from the bash works, and build continues.
4)      A few similar cases (stops) with DirectByteBuffer and DirectByteBufferR


Currently I try to explore how that might relate to the MSYS2 bash and make, somehow it behaves differently

If you have any other suggestion, let me know.

Best regards,

Peter

From: Peter Budai<mailto:[hidden email]>
Sent: Thursday, October 5, 2017 3:52 PM
To: Magnus Ihse Bursie<mailto:[hidden email]>; Erik Joelsson<mailto:[hidden email]>; [hidden email]<mailto:[hidden email]>
Subject: RE: Building OpenJDK9 on MSYS2

Hi Magnus,

So first of all, here is the current patch, which I was not able to attach: https://pastebin.com/pwT4Ynxc

>> That's surprising, since gcc is prefixed with fixpath, which it should not.
Actually you DO need fixpath IMHO.
This is a mingw64 version of the gcc (/C/msys64/mingw64/bin/gcc), which is a fully functional Windows executable, which expects Windows formatted path arguments.
As the updated build process uses EXPORT MSYS2_ARG_CONV_EXCL=* (see that patch), none of the command line arguments are converted  from the unix path to Windows, but fixpath does that conversion. There is a wiki describing more details on this:
https://github.com/msys2/msys2/wiki/Porting#user-content-filesystem-namespaces



>>I have a hard time believing this is a race condition. On the other hand, this stuff is weird, we're misusing the C preprocessor to process defines in java code, so I'm not surprised it breaks down.
>>I don't know why it succeeded when run on the command line, though.
When I execute that command from the bash command line there is no EXPORT MSYS2_ARG_CONV_EXCL, but the bash itself does the automatic conversion of the arguments. Maybe it has something to do how fixpath does CreateProcess?

Does that help?

Best regards,

Peter

Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10

From: Magnus Ihse Bursie<mailto:[hidden email]>
Sent: Thursday, October 5, 2017 12:13 PM
To: Peter Budai<mailto:[hidden email]>; Erik Joelsson<mailto:[hidden email]>; [hidden email]<mailto:[hidden email]>
Subject: Re: Building OpenJDK9 on MSYS2


On 2017-10-05 11:59, Peter Budai wrote:
Hi Magnus and Erik,

I really appreciate your quick feedback. I assumed that it won’t be easy, but I just don’t feel I should give up now  - maybe later when I see the real scale of work. So bear with me for a time being.

Attached is a patch which already includes Magnus’ changes, plus a few which I have added:
·       basically enabling gcc for windows,
·       and modifying a logic for compiling fixpath (before that it was using hard-coded MS VSC compile flags)

Actually, you must make sure fixpath is *not* used for the toolchain, since gcc uses unix style paths.
(However, other tools such as java will still need it.)




So here is what I have as the result of configure:
====================================================
The existing configuration has been successfully updated in
/C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release
using configure arguments '--disable-freetype-bundling --disable-javac-server'.

Configuration summary:
* Debug level:    release
* HS debug level: product
* JDK variant:    normal
* JVM variants:   server
* OpenJDK target: OS: windows, CPU architecture: x86, address length: 64
* Version string: 9-internal+0-adhoc.peterbud.jdk9 (9-internal)

Tools summary:
* Environment:    msys version 2.9.0(0.318/5/3) (root at /C/msys64)
* Boot JDK:       java version "1.8.0_144"  Java(TM) SE Runtime Environment (build 1.8.0_144-b01)  Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)   (at /c/progra~1/java/jdk18~1.0_1)
* Toolchain:      gcc (GNU Compiler Collection)
* C Compiler:     Version 7.2.0 (at /C/msys64/mingw64/bin/gcc)
* C++ Compiler:   Version 7.2.0 (at /c/msys64/mingw64/bin/g++)

Build performance summary:
* Cores to use:   4
* Memory limit:   16216 MB

Its clear says that the toolchain is gcc 7.2 (BTW there is no Visual Studio on this machine)

Now for the details of the config log, you can see here: https://pastebin.com/MN2ZYcHH

And about the build process and the error I get:

$ make JOBS=1
Building target 'default (exploded-image)' in configuration 'windows-x86_64-normal-server-release'
Compiling 8 files for BUILD_TOOLS_LANGTOOLS
Compiling 17 properties into resource bundles for jdk.compiler
Parsing 1 properties into enum-like class for jdk.compiler
Compiling 19 properties into resource bundles for jdk.javadoc
Compiling 12 properties into resource bundles for jdk.jdeps
Compiling 7 properties into resource bundles for jdk.jshell
Compiling 117 files for BUILD_INTERIM_java.compiler
Compiling 396 files for BUILD_INTERIM_jdk.compiler
Compiling 61 files for BUILD_INTERIM_jdk.jdeps
Compiling 457 files for BUILD_INTERIM_jdk.javadoc
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Compiling 159 files for BUILD_TOOLS_JDK
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
make[3]: *** [GensrcMisc.gmk:78: /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java] Error 1
make[3]: *** Deleting file '/C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java'
make[2]: *** [make/Main.gmk:115: java.base-gensrc-jdk] Error 2

ERROR: Build failed for target 'default (exploded-image)' in configuration 'windows-x86_64-normal-server-release' (exit code 2)

No indication of failed target found.
Hint: Try searching the build log for '] Error'.
Hint: See common/doc/building.html#troubleshooting for assistance.

make[1]: *** [/home/peterbud/jdk9/make/Init.gmk:296: main] Error 2
make: *** [/home/peterbud/jdk9/make/Init.gmk:185: default] Error 2

If I run here
make JOBS=1 LOG=debug
The failing line seems to be this:

( /usr/bin/gawk '/@@END_COPYRIGHT@@/{exit}1' /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template && /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/configure-support/bin/fixpath.exe -m/C/msys64/@/c/msys64/@/c/progra~ /C/msys64/mingw64/bin/gcc -E -x c  /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template 2> >(/usr/bin/grep -v '^SocketOptionRegistry.java.template$' >&2) | /usr/bin/gawk '/@@START_HERE@@/,0' |  /usr/bin/sed -e 's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED FILE - DO NOT EDIT/' -e 's/PREFIX_//' -e 's/^#.*//' ) > /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java
make[3]: *** [GensrcMisc.gmk:78: /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java] Error 1

Now the interesting is: if I copy this line above to the bash prompt, it runs without problem, and the file support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java
That's surprising, since gcc is prefixed with fixpath, which it should not.

I have a hard time believing this is a race condition. On the other hand, this stuff is weird, we're misusing the C preprocessor to process defines in java code, so I'm not suprised it breaks down. I don't know why it succeeded when run on the command line, though. My suggestion is to just do some quick and dirty hack around this: take the file you manage to generate and just copy it in during the build instead. If you can get round this, you might start seeing some *real* problems. :-)

Also, my suggestion is that you try running "make hotspot" to cut to the chase. Compiling hotspot will likely be the hardest thing. Or even "make -k hotspot" to get an assessment of the amount of work ahead of you.

/Magnus
Is produced.

Then I can again issue
make JOBS=1 LOG=debug

And the compile process is being continued, until a similar error pops up with a different generated file. I have an assumption that this happens because make is still running parallel jobs, despite JOBS=1 but I’m not sure.

How could I best tackle this?

Thank you and best regards,

Peter

Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10

From: Magnus Ihse Bursie<mailto:[hidden email]>
Sent: Thursday, October 5, 2017 11:33 AM
To: Erik Joelsson<mailto:[hidden email]>; Peter Budai<mailto:[hidden email]>; [hidden email]<mailto:[hidden email]>
Subject: Re: Building OpenJDK9 on MSYS2

On 2017-10-05 10:10, Erik Joelsson wrote:

> Hello Peter,
>
>
> On 2017-10-04 21:15, Peter Budai wrote:
>> Hi Magnus,
>>
>> Thanks for the quick reply I’ll check these patches with msys2.
>>
>> Let me specify with more details what I’d like to achieve: I’d like
>> to build OpenJDK9 with MSYS2 MINGW64 environment using gcc toolchain.
>> (I’m not sure how familiar are you with MSYS2, but there are 3
>> different environments: MSYS2, MINGW32 and MINGW64). In theory
>> MINGW64 with gcc is the closes you can get on Windows platform as a
>> gcc unix like build environment, which produces still a native 64-bit
>> executable on Windows.
>>
>> I’m not very familiar with OpenJDK yet, so therefore I’d like to hear
>> your opinion: how realistic is that?
> Sorry to disappoint, but I would say that requires major work. There
> is a strong historic assumption that windows builds are done using
> Visual Studio. We have abstracted away some of it in configure (see
> TOOLCHAIN_TYPE), but it's very far from enough to change compiler
> environment for a Windows build. The native sources are also bound to
> make a lot of such assumptions. I would expect the changes needed to
> be in the thousands of lines of code.

I agree that it requires hard work (even if "thousands" might be an
overestimation I think, but "hundreds" is not enough, so it's the right
magnitude). On the other hand, it would be really good if we did sort
things out, so that we had proper conditions based on OS vs
compiler/toolchain.

If you really want to start, the first thing is to patch toolchain.m4 to
VALID_TOOLCHAINS_windows="microsoft gcc"
and then call configure using "bash configure --with-toolchain-type=gcc".

As Erik, I doubt you will come very far before things starts tumbling down.

>
> When we say supporting the build in msys2 instead of cygwin, we just
> mean using msys2 as the unix emulating layer for our tools like
> make/bash/grep/sed etc.
>
> One think I have done successfully is running the build in WSL
> (Windows Subsystem for Linux), but that isn't all that helpful as WSL
> for practical purposes is more or less like running Linux in a VM, so
> the build sees a Linux system and builds a Linux binary.
>> As a side note: with MINGW64 I have managed to run configure phase
>> successfully for OpenJDK. The compile process has also started and
>> went for a while, but interestingly I run into some kind of race
>> conditions as make stopped with an error. Using LOG=debug I have fond
>> the failing line and then copying the failed command and pasting it
>> to the bash prompt it successfully generated the output target, and
>> then the build process run further when a similar situation happened.
>> Also pasting the failed command run in the bash without any problem,
>> and build continued… until the next.
> Without seeing the errors I can't say much. I very much doubt that you
> are running with gcc as the compiler though. Configure isn't easily
> fooled into using a different compiler to what it prefers, and I would
> expect things to crash and burn pretty early if you actually did.
>
> /Erik
>>
>> I have tried to run make JOBS=1, but did not help, strangely I have
>> still seen in the log make[3] and make[4] logs which suggested that
>> there are more than one make jobs were running. Also tried .configure
>> --with-output-sync=recurse without success (same symptoms)
>>
>> Let me know your thoughts.
>>
>> Best regards,
>>
>> Peter
>>
>> Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for
>> Windows 10
>>
>> From: Magnus Ihse Bursie<mailto:[hidden email]>
>> Sent: Wednesday, October 4, 2017 1:04 AM
>> To: Peter Budai<mailto:[hidden email]>;
>> [hidden email]<mailto:[hidden email]><mailto:[hidden email]><mailto:[hidden email]>
>> Subject: Re: Building OpenJDK9 on MSYS2
>>
>> Actually, it wasn't so much remaining trouble. :-) I fired up msys2 and
>> checked out where I left off. It turned out that the remaining snag was
>> that msys2 tries to convert command lines automatically, from "unix"
>> style paths to "windows" style paths. Unfortunately, it does not do this
>> very well and it breaks all sorts of things. We already have a FIXPATH
>> solution in place which deals with this, so basically all I had to do
>> was disable this (by setting MSYS2_ARG_CONV_EXCL to "*"). However, this
>> broke our cygpath replacement hack (!) so I had to disable it there.
>> Sigh. Anyway, with those fixes it ran and worked well. (I also
>> discovered and fixed a bug related to how we set up the FIXPATH variable
>> on msys, but it only triggers in certain circumstances).
>>
>> With this patch I now jdk9 seems to build fine on msys2. It should apply
>> cleanly on jdk9/jdk9. Since it turned out to be so trivial, I'll try to
>> get it in in jdk10.
>>
>> Here's the patch if you want to apply it yourself:
>>
>> diff -r a08cbfc0e4ec common/autoconf/basics_windows.m4
>> --- a/common/autoconf/basics_windows.m4    Thu Aug 03 18:56:56 2017
>> +0000
>> +++ b/common/autoconf/basics_windows.m4    Wed Oct 04 00:53:58 2017
>> +0200
>> @@ -42,7 +42,7 @@
>>        windows_path=`$CYGPATH -m "$unix_path"`
>>        $1="$windows_path"
>>      elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then
>> -    windows_path=`cmd //c echo $unix_path`
>> +    windows_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $unix_path`
>>        $1="$windows_path"
>>      fi
>>    ])
>> @@ -136,6 +136,16 @@
>>      fi
>>    ])
>>
>> +AC_DEFUN([BASIC_MSYS_UPDATE_FIXPATH],
>> +[
>> +  # Take all collected prefixes and turn them into a
>> -m/c/foo@/c/bar@... command line
>> +  # @ was chosen as separator to minimize risk of other tools messing
>> around with it
>> +  all_unique_prefixes=`echo "${all_fixpath_prefixes@<:@@@:>@}" \
>> +      | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
>> +  fixpath_argument_list=`echo $all_unique_prefixes  | tr ' ' '@'`
>> +  FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
>> +])
>> +
>>    AC_DEFUN([BASIC_FIXUP_PATH_MSYS],
>>    [
>>      path="[$]$1"
>> @@ -143,7 +153,7 @@
>>      new_path="$path"
>>      if test "x$has_colon" = x; then
>>        # Not in mixed or Windows style, start by that.
>> -    new_path=`cmd //c echo $path`
>> +    new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $path`
>>      fi
>>
>>      BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
>> @@ -155,6 +165,8 @@
>>
>>      # Save the first 10 bytes of this path to the storage, so fixpath
>> can work.
>> all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
>> "${new_path:0:10}")
>> +  # We might need to re-evaluate FIXPATH.
>> +  BASIC_MSYS_UPDATE_FIXPATH
>>    ])
>>
>>    AC_DEFUN([BASIC_FIXUP_EXECUTABLE_CYGWIN],
>> @@ -293,7 +305,7 @@
>>        # Do not save /bin paths to all_fixpath_prefixes!
>>      else
>>        # Not in mixed or Windows style, start by that.
>> -    new_path=`cmd //c echo $new_path`
>> +    new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $new_path`
>>        BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
>>        # Output is in $new_path
>>        BASIC_WINDOWS_REWRITE_AS_UNIX_PATH(new_path)
>> @@ -302,6 +314,8 @@
>>
>>        # Save the first 10 bytes of this path to the storage, so fixpath
>> can work.
>> all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
>> "${new_path:0:10}")
>> +    # We might need to re-evaluate FIXPATH.
>> +    BASIC_MSYS_UPDATE_FIXPATH
>>      fi
>>    ])
>>
>> @@ -347,6 +361,10 @@
>>        WINDOWS_ENV_VENDOR='msys'
>>        WINDOWS_ENV_VERSION="$MSYS_VERSION"
>>
>> +    # Prohibit msys2 path conversion from trying to be "intelligent",
>> and rely
>> +    # on fixpath instead.
>> +    export MSYS2_ARG_CONV_EXCL="*"
>> +
>>        AC_MSG_CHECKING([msys root directory as unix-style path])
>>        # The cmd output ends with Windows line endings (CR/LF), the grep
>> command will strip that away
>>        MSYS_ROOT_PATH=`cd / ; cmd /c cd | $GREP ".*"`
>> @@ -391,10 +409,7 @@
>>        elif test "x$OPENJDK_BUILD_OS_ENV" = xwindows.msys; then
>>          # Take all collected prefixes and turn them into a
>> -m/c/foo@/c/bar@... command line
>>          # @ was chosen as separator to minimize risk of other tools
>> messing around with it
>> -      all_unique_prefixes=`echo "${all_fixpath_prefixes@<:@@@:>@}" \
>> -          | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
>> -      fixpath_argument_list=`echo $all_unique_prefixes  | tr ' ' '@'`
>> -      FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
>> +      BASIC_MSYS_UPDATE_FIXPATH
>>        fi
>>        FIXPATH_SRC_W="$FIXPATH_SRC"
>>        FIXPATH_BIN_W="$FIXPATH_BIN"
>> diff -r a08cbfc0e4ec common/autoconf/build-aux/config.sub
>> --- a/common/autoconf/build-aux/config.sub    Thu Aug 03 18:56:56
>> 2017 +0000
>> +++ b/common/autoconf/build-aux/config.sub    Wed Oct 04 00:53:58
>> 2017 +0200
>> @@ -30,7 +30,7 @@
>>    DIR=`dirname $0`
>>
>>    # First, filter out everything that doesn't begin with "aarch64-"
>> -if ! echo $* | grep '^aarch64-' >/dev/null ; then
>> +if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ; then
>>        . $DIR/autoconf-config.sub "$@"
>>        # autoconf-config.sub exits, so we never reach here, but just in
>>        # case we do:
>> @@ -38,13 +38,17 @@
>>    fi
>>
>>    while test $# -gt 0 ; do
>> -    case $1 in
>> +    case $1 in
>>            -- )   # Stop option processing
>>                shift; break ;;
>>            aarch64-* )
>>                config=`echo $1 | sed 's/^aarch64-/arm-/'`
>>                sub_args="$sub_args $config"
>>                shift; ;;
>> +        *-msys )
>> +            config=`echo $1 | sed 's/msys/mingw32/'`
>> +            sub_args="$sub_args $config"
>> +            shift; ;;
>>            - )    # Use stdin as input.
>>                sub_args="$sub_args $1"
>>                shift; break ;;
>> diff -r a08cbfc0e4ec common/autoconf/spec.gmk.in
>> --- a/common/autoconf/spec.gmk.in    Thu Aug 03 18:56:56 2017 +0000
>> +++ b/common/autoconf/spec.gmk.in    Wed Oct 04 00:53:58 2017 +0200
>> @@ -120,6 +120,13 @@
>>      # On Windows, the Visual Studio toolchain needs the PATH to be
>> adjusted
>>      # to include Visual Studio tools (this needs to be in cygwin/msys
>> style).
>>      export PATH:=@VS_PATH@
>> +
>> +endif
>> +
>> +ifeq ($(OPENJDK_TARGET_OS_ENV), windows.msys)
>> +  # On msys2, prohibit msys path conversion from trying to be
>> +  # "intelligent", and rely on fixpath instead.
>> +  export MSYS2_ARG_CONV_EXCL:=*
>>    endif
>>
>>    SYSROOT_CFLAGS := @SYSROOT_CFLAGS@
>>
>> /Magnus
>>
>> On 2017-10-03 22:34, Magnus Ihse Bursie wrote:
>>> I gave msys2 a shot some time ago, but it ended up too much trouble.
>>> I'll share some of my notes from that attempt, for what it's worth.
>>>
>>> To install package X/Y, run "pacman -S X/Y". Missing tools and
>>> packages where to find them:
>>> cmp: msys/diffutils
>>> tar: msys/tar
>>> make: msys/make
>>> unzip: msys/unzip
>>> zip: msys/zip
>>>
>>> config.sub reports msys as "x86_64-pc-mingw32" but msys2 as
>>> "x86_64-pc-msys". This patch adds postprocessing in "our" config.sub
>>> to report msys2 similar to msys. (Opinions, including my own :-) may
>>> vary if this really is the best way..)
>>>
>>> diff -r b88023f46daa common/autoconf/build-aux/config.sub
>>> --- a/common/autoconf/build-aux/config.sub      Fri Jan 27 10:15:41
>>> 2017 +0100
>>> +++ b/common/autoconf/build-aux/config.sub      Fri Feb 03 05:00:25
>>> 2017 -0700
>>> @@ -30,7 +30,7 @@
>>>   DIR=`dirname $0`
>>>
>>>   # First, filter out everything that doesn't begin with "aarch64-"
>>> -if ! echo $* | grep '^aarch64-' >/dev/null ; then
>>> +if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ; then
>>>       . $DIR/autoconf-config.sub "$@"
>>>       # autoconf-config.sub exits, so we never reach here, but just in
>>>       # case we do:
>>> @@ -45,6 +45,10 @@
>>>               config=`echo $1 | sed 's/^aarch64-/arm-/'`
>>>               sub_args="$sub_args $config"
>>>               shift; ;;
>>> +        *-msys )
>>> +            config=`echo $1 | sed 's/msys/mingw32/'`
>>> +            sub_args="$sub_args $config"
>>> +            shift; ;;
>>>           - )    # Use stdin as input.
>>>               sub_args="$sub_args $1"
>>>               shift; break ;;
>>>
>>> If I remember correctly, this got me past the configure stage at the
>>> time.
>>>
>>> I don't think it's very hard to get it to work on msys2, I just ran
>>> into one snag too many and didn't think msys2 would be used by anyone.
>>>
>>> /Magnus
>>>
>>> On 2017-10-03 17:20, Peter Budai wrote:
>>>> Hello,
>>>>
>>>> According to
>>>> http://hg.openjdk.java.net/jdk9/jdk9/file/a08cbfc0e4ec/common/doc/building.html
>>>>
>>>> “msys2 and the new Windows Subsystem for Linux (WSL) would likely be
>>>> possible to support in a future version but that would require a
>>>> community effort to implement”
>>>>
>>>> I’d like to help making the OpenJDK 9 build working on msys2. What is
>>>> the best way to move forward? Is there a similar effort in progress?
>>>>
>>>> Thank you and best regards,
>>>>
>>>> Peter
>>>>
>>>>
>







Reply | Threaded
Open this post in threaded view
|

Re: Building OpenJDK9 on MSYS2

Erik Joelsson
Hello Peter,


On 2017-10-11 00:18, Peter Budai wrote:

>
> Thanks Magnus & Erik
>
> First thanks for your support and kind words!
>
> Magnus, I have checked .bash_profile, .bashrc but they seem to be
> empty (everything is commented out). You can check with a default
> MSYS2 install, I have not changed these files at all. If you find thee
> something specific I can give a try here as well.
>
> Let me give also a quick status update, where am I with building hotspot:
>
>   * I guess its still the beginning, but I have managed to compile
>     jvm.dll with almost 700 object file: with debug info the dll is
>     around 700 MB 😊
>   * I made only surgical, minimal changes to the source, and so far it
>     looks reasonable. I have encountered 3 scenarios where changes
>     were necessary:
>       o When in makefiles conditionals were using assuming that if
>         target_os is windows then it is visual studio compiler/linker.
>         Obviously these conditionals had to be reviewed in a few
>         places and if necessary were changes to check the
>         toolchain=Microsoft
>
These are not surprising and should be pretty straight forward to fix
and it seems you know what to do.
>
>  *
>       o I got a few warnings as gcc 7.2 uncovered some code problems
>         in windows specific codes, where before that MSVC I guess did
>         not say a word…
>
To get around this you can configure with --disable-warnings-as-errors
until you get things working properly. This is commonly needed when
using compiler versions that we normally don't use.

>
>  *
>       o And I had like 6-7 places where the code was using MSVC
>         specific __try … __except structures which gcc does not know.
>         Do you have a suggestion how to approach them? I can do ugly
>         #ifdefs (I would avoid that) but I have also seen some
>         solutions to replace them with a code which gcc can compile
>         (http://www.programmingunlimited.net/siteexec/content.cgi?page=mingw-seh
>         ) – but before doing that  though I would ask first you on the
>         purpose of those
>
This kind of question is probably best to bring to the hotspot mailing
list.

>
>   * What bothers me is that I was not able to do incremental builds:
>     when an error occurs, and build stops, then after making change in
>     the CPP source the build cannot continue, I always got an error
>     message:
>
> |/C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/hotspot/variant-server/tools/adlc/objs/adlparse.d.targets:1:
> *** missing target pattern.  Stop.|
> |make[2]: *** [make/Main.gmk:256: hotspot-server-gensrc] Error 2|
>
> If I do a ‘make clean’ and restart the build then it nicely compiles.
>
> Question 1: Is there a way to  resume such builds without ‘make clean’?
>
Well, incremental builds is supposed to work well. We have several extra
tricks in there to handle cases where normal make builds would fail. The
*.d.targets files is one such trick and it seems to backfire for you.
The contents of that file should be something like:

/localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/adlparse.cpp :
/localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/adlc.hpp :
/localhome/hg/jdk9-dev/hotspot/src/share/vm/opto/opcodes.hpp :
/localhome/hg/jdk9-dev/hotspot/src/share/vm/opto/classes.hpp :
/localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/arena.hpp :
/localhome/hg/jdk9-dev/hotspot/src/share/vm/opto/adlcVMDeps.hpp :
/localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/filebuff.hpp :
/localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/dict2.hpp :
/localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/forms.hpp :
/localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/formsopt.hpp :
/localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/formssel.hpp :
/localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/archDesc.hpp :
/localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/adlparse.hpp :

Basically an empty rule for each dependency for the corresponding object
file. Declaring these rules makes it possible to delete source files
without having to build clean. It seems your file is not generated
correctly so please have a look inside it. The file is in
make/common/NativeCompilation.gmk, look for DEPENDENCY_TARGET_SED_PATTERN.
>
> Question 2: What would be the best way to submit/share the patches for
> your thorough review?
>
Well, first of all, have you signed the OCA?

As for publishing patches and reviews, there is a bit of chicken and egg
problem. Once you become an "author" in any of the OpenJDK projects, you
get a user name and should be able to publish reviews on
cr.openjdk.java.net. Before that, if the patch is small, it can be
posted inline in an email to the list. If it's large, you will need a
current OpenJDK user to host it for you. At least that's how I
understand it. Hopefully someone who knows the process better can chime
in here.

I should also let you know that getting this into JDK 9 is most likely
not going to happen. AFAIK we are only doing security updates for 9. It
would have to go into the currently active release. I should also warn
you that new ports generally need a certain amount of backing to be
accepted. It may be that this would have to live in a porting side
project. Hopefully someone who knows this better can chime in here as well.

/Erik

>
> P.
>
> Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for
> Windows 10
>
> *From: *Magnus Ihse Bursie <mailto:[hidden email]>
> *Sent: *Tuesday, October 10, 2017 10:04 AM
> *To: *Peter Budai <mailto:[hidden email]>; Erik Joelsson
> <mailto:[hidden email]>; [hidden email]
> <mailto:[hidden email]>
> *Subject: *Re: Building OpenJDK9 on MSYS2
>
> On 2017-10-07 10:14, Peter Budai wrote:
>
>     The configure of OpenJDK overwrites the SHELL. Actually it is
>     using bash, but for the arguments it was using “-e -o pipefail”. I
>     have figured that for MSYS2 bash what is needed as bash arguments
>     is “-e -l -c -o pipefail”
>
>     That looks like solving this problem, and now the real issues are
>     surfacing.
>
>
> FWIW, "-l" makes bash behave like a login shell. Most likely you are
> changing bash's behavior in one of your login scripts, and that change
> is what's really needed.
>
> /Magnus
>
>
>     Peter
>
>     *From: *Peter Budai <mailto:[hidden email]>
>     *Sent: *Friday, October 6, 2017 6:43 PM
>     *To: *Magnus Ihse Bursie <mailto:[hidden email]>;
>     Erik Joelsson <mailto:[hidden email]>;
>     [hidden email] <mailto:[hidden email]>
>     *Subject: *RE: Building OpenJDK9 on MSYS2
>
>     Magnus,
>
>     I have followed your suggestion and removed the fixpath prefixes
>     from gcc related compile tools, and left only the fixpath prefix
>     _only_ for the Boot JDK related tools in place.
>
>     1)As  I follow the process, all java and javac related compile
>     steps are running properly
>
>     2)When the process reaches gcc related steps I got the error
>     message at the same place as before (no fixpath). If I execute
>     that command from the bash prompt, it creates the output:
>
>     $ ( /usr/bin/gawk '/@@END_COPYRIGHT@@/{exit}1'
>     /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template
>     && /C/msys64/mingw64/bin/gcc -E -x c
>     /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template
>     2> >(/usr/bin/grep -v '^SocketOptionRegistry.java.template$' >&2)
>     | /usr/bin/gawk '/@@START_HERE@@/,0' |  /usr/bin/sed -e
>     's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED FILE - DO NOT
>     EDIT/' -e 's/PREFIX_//' -e 's/^#.*//' ) >
>     /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java
>
>     As I have mentioned the parameters are replaced by the bash
>     automatically
>
>     3)Then build continues, then little later stops at a super simple
>     command:
>
>     mv
>     /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBuffer.java.tmp
>     /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBuffer.java
>
>     Needless to say, the ByteBuffer.java.tmp file DOES exist. And
>     running the above command from the bash works, and build continues.
>
>     4)A few similar cases (stops) with DirectByteBuffer and
>     DirectByteBufferR
>
>     Currently I try to explore how that might relate to the MSYS2 bash
>     and make, somehow it behaves differently
>
>     If you have any other suggestion, let me know.
>
>     Best regards,
>
>     Peter
>
>     *From: *Peter Budai <mailto:[hidden email]>
>     *Sent: *Thursday, October 5, 2017 3:52 PM
>     *To: *Magnus Ihse Bursie <mailto:[hidden email]>;
>     Erik Joelsson <mailto:[hidden email]>;
>     [hidden email] <mailto:[hidden email]>
>     *Subject: *RE: Building OpenJDK9 on MSYS2
>
>     Hi Magnus,
>
>     So first of all, here is the current patch, which I was not able
>     to attach: https://pastebin.com/pwT4Ynxc
>
>     >> That's surprising, since gcc is prefixed with fixpath, which it should not.
>
>     Actually you DO need fixpath IMHO.
>
>     This is a mingw64 version of the gcc (/C/msys64/mingw64/bin/gcc),
>     which is a fully functional Windows executable, /which expects
>     Windows formatted path arguments/.
>
>     As the updated build process uses EXPORT MSYS2_ARG_CONV_EXCL=*
>     (see that patch), none of the command line arguments are converted
>     from the unix path to Windows, but fixpath does that conversion.
>     There is a wiki describing more details on this:
>
>     https://github.com/msys2/msys2/wiki/Porting#user-content-filesystem-namespaces
>
>
>     >>I have a hard time believing this is a race condition. On the
>     other hand, this stuff is weird, we're misusing the C preprocessor
>     to process defines in java code, so I'm not surprised it breaks down.
>
>     >>I don't know why it succeeded when run on the command line, though.
>
>     When I execute that command from the /bash/ command line there is
>     no EXPORT MSYS2_ARG_CONV_EXCL, but the bash itself does the
>     automatic conversion of the arguments. Maybe it has something to
>     do how fixpath does CreateProcess?
>
>     Does that help?
>
>     Best regards,
>
>     Peter
>
>     Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986>
>     for Windows 10
>
>     *From: *Magnus Ihse Bursie <mailto:[hidden email]>
>     *Sent: *Thursday, October 5, 2017 12:13 PM
>     *To: *Peter Budai <mailto:[hidden email]>; Erik Joelsson
>     <mailto:[hidden email]>; [hidden email]
>     <mailto:[hidden email]>
>     *Subject: *Re: Building OpenJDK9 on MSYS2
>
>     On 2017-10-05 11:59, Peter Budai wrote:
>
>         Hi Magnus and Erik,
>
>         I really appreciate your quick feedback. I assumed that it
>         won’t be easy, but I just don’t feel I should give up now  -
>         maybe later when I see the real scale of work. So bear with me
>         for a time being.
>
>         Attached is a patch which already includes Magnus’ changes,
>         plus a few which I have added:
>
>         ·basically enabling gcc for windows,
>
>         ·and modifying a logic for compiling fixpath (before that it
>         was using hard-coded MS VSC compile flags)
>
>
>     Actually, you must make sure fixpath is *not* used for the
>     toolchain, since gcc uses unix style paths.
>     (However, other tools such as java will still need it.)
>
>
>         So here is what I have as the result of configure:
>
>         ====================================================
>
>         The existing configuration has been successfully updated in
>
>         /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release
>
>         using configure arguments '--disable-freetype-bundling
>         --disable-javac-server'.
>
>         Configuration summary:
>
>         * Debug level:    release
>
>         * HS debug level: product
>
>         * JDK variant:    normal
>
>         * JVM variants:   server
>
>         * OpenJDK target: OS: windows, CPU architecture: x86, address
>         length: 64
>
>         * Version string: 9-internal+0-adhoc.peterbud.jdk9 (9-internal)
>
>         Tools summary:
>
>         * Environment:    msys version 2.9.0(0.318/5/3) (root at
>         /C/msys64)
>
>         * Boot JDK:       java version "1.8.0_144"  Java(TM) SE
>         Runtime Environment (build 1.8.0_144-b01)  Java HotSpot(TM)
>         64-Bit Server VM (build 25.144-b01, mixed mode)   (at
>         /c/progra~1/java/jdk18~1.0_1)
>
>         * Toolchain:    gcc (GNU Compiler Collection)
>
>         * C Compiler:     Version 7.2.0 (at /C/msys64/mingw64/bin/gcc)
>
>         * C++ Compiler:   Version 7.2.0 (at /c/msys64/mingw64/bin/g++)
>
>         Build performance summary:
>
>         * Cores to use:   4
>
>         * Memory limit:   16216 MB
>
>         Its clear says that the toolchain is gcc 7.2 (BTW there is no
>         Visual Studio on this machine)
>
>         Now for the details of the config log, you can see here:
>         https://pastebin.com/MN2ZYcHH
>
>         And about the build process and the error I get:
>
>         $ make JOBS=1
>
>         Building target 'default (exploded-image)' in configuration
>         'windows-x86_64-normal-server-release'
>
>         Compiling 8 files for BUILD_TOOLS_LANGTOOLS
>
>         Compiling 17 properties into resource bundles for jdk.compiler
>
>         Parsing 1 properties into enum-like class for jdk.compiler
>
>         Compiling 19 properties into resource bundles for jdk.javadoc
>
>         Compiling 12 properties into resource bundles for jdk.jdeps
>
>         Compiling 7 properties into resource bundles for jdk.jshell
>
>         Compiling 117 files for BUILD_INTERIM_java.compiler
>
>         Compiling 396 files for BUILD_INTERIM_jdk.compiler
>
>         Compiling 61 files for BUILD_INTERIM_jdk.jdeps
>
>         Compiling 457 files for BUILD_INTERIM_jdk.javadoc
>
>         Note: Some input files use or override a deprecated API.
>
>         Note: Recompile with -Xlint:deprecation for details.
>
>         Compiling 159 files for BUILD_TOOLS_JDK
>
>         Note: Some input files use unchecked or unsafe operations.
>
>         Note: Recompile with -Xlint:unchecked for details.
>
>         make[3]: *** [GensrcMisc.gmk:78:
>         /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java]
>         Error 1
>
>         make[3]: *** Deleting file
>         '/C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java'
>
>         make[2]: *** [make/Main.gmk:115: java.base-gensrc-jdk] Error 2
>
>         ERROR: Build failed for target 'default (exploded-image)' in
>         configuration 'windows-x86_64-normal-server-release' (exit code 2)
>
>         No indication of failed target found.
>
>         Hint: Try searching the build log for '] Error'.
>
>         Hint: See common/doc/building.html#troubleshooting for assistance.
>
>         make[1]: *** [/home/peterbud/jdk9/make/Init.gmk:296: main] Error 2
>
>         make: *** [/home/peterbud/jdk9/make/Init.gmk:185: default] Error 2
>
>         If I run here
>
>         make JOBS=1 LOG=debug
>
>         The failing line seems to be this:
>
>         ( /usr/bin/gawk '/@@END_COPYRIGHT@@/{exit}1'
>         /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template
>         &&
>         /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/configure-support/bin/fixpath.exe
>         -m/C/msys64/@/c/msys64/@/c/progra~ /C/msys64/mingw64/bin/gcc
>         -E -x c
>         /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template
>         2> >(/usr/bin/grep -v '^SocketOptionRegistry.java.template$'
>         >&2) | /usr/bin/gawk '/@@START_HERE@@/,0' |  /usr/bin/sed -e
>         's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED FILE - DO NOT
>         EDIT/' -e 's/PREFIX_//' -e 's/^#.*//' ) >
>         /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java
>
>         make[3]: *** [GensrcMisc.gmk:78:
>         /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java]
>         Error 1
>
>         Now the interesting is: if I copy this line above to the bash
>         prompt, it runs without problem, and the file
>         support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java
>
>     That's surprising, since gcc is prefixed with fixpath, which it
>     should not.
>
>     I have a hard time believing this is a race condition. On the
>     other hand, this stuff is weird, we're misusing the C preprocessor
>     to process defines in java code, so I'm not suprised it breaks
>     down. I don't know why it succeeded when run on the command line,
>     though. My suggestion is to just do some quick and dirty hack
>     around this: take the file you manage to generate and just copy it
>     in during the build instead. If you can get round this, you might
>     start seeing some *real* problems. :-)
>
>     Also, my suggestion is that you try running "make hotspot" to cut
>     to the chase. Compiling hotspot will likely be the hardest thing.
>     Or even "make -k hotspot" to get an assessment of the amount of
>     work ahead of you.
>
>     /Magnus
>
>         Is produced.
>
>         Then I can again issue
>
>         make JOBS=1 LOG=debug
>
>         And the compile process is being continued, until a similar
>         error pops up with a different generated file. I have an
>         assumption that this happens because make is still running
>         parallel jobs, despite JOBS=1 but I’m not sure.
>
>         How could I best tackle this?
>
>         Thank you and best regards,
>
>         Peter
>
>         Sent from Mail
>         <https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10
>
>         *From: *Magnus Ihse Bursie <mailto:[hidden email]>
>         *Sent: *Thursday, October 5, 2017 11:33 AM
>         *To: *Erik Joelsson <mailto:[hidden email]>; Peter
>         Budai <mailto:[hidden email]>;
>         [hidden email] <mailto:[hidden email]>
>         *Subject: *Re: Building OpenJDK9 on MSYS2
>
>         On 2017-10-05 10:10, Erik Joelsson wrote:
>         > Hello Peter,
>         >
>         >
>         > On 2017-10-04 21:15, Peter Budai wrote:
>         >> Hi Magnus,
>         >>
>         >> Thanks for the quick reply I’ll check these patches with msys2.
>         >>
>         >> Let me specify with more details what I’d like to achieve:
>         I’d like
>         >> to build OpenJDK9 with MSYS2 MINGW64 environment using gcc
>         toolchain.
>         >> (I’m not sure how familiar are you with MSYS2, but there are 3
>         >> different environments: MSYS2, MINGW32 and MINGW64). In theory
>         >> MINGW64 with gcc is the closes you can get on Windows
>         platform as a
>         >> gcc unix like build environment, which produces still a
>         native 64-bit
>         >> executable on Windows.
>         >>
>         >> I’m not very familiar with OpenJDK yet, so therefore I’d
>         like to hear
>         >> your opinion: how realistic is that?
>         > Sorry to disappoint, but I would say that requires major
>         work. There
>         > is a strong historic assumption that windows builds are done
>         using
>         > Visual Studio. We have abstracted away some of it in
>         configure (see
>         > TOOLCHAIN_TYPE), but it's very far from enough to change
>         compiler
>         > environment for a Windows build. The native sources are also
>         bound to
>         > make a lot of such assumptions. I would expect the changes
>         needed to
>         > be in the thousands of lines of code.
>
>         I agree that it requires hard work (even if "thousands" might
>         be an
>         overestimation I think, but "hundreds" is not enough, so it's
>         the right
>         magnitude). On the other hand, it would be really good if we
>         did sort
>         things out, so that we had proper conditions based on OS vs
>         compiler/toolchain.
>
>         If you really want to start, the first thing is to patch
>         toolchain.m4 to
>         VALID_TOOLCHAINS_windows="microsoft gcc"
>         and then call configure using "bash configure
>         --with-toolchain-type=gcc".
>
>         As Erik, I doubt you will come very far before things starts
>         tumbling down.
>
>         >
>         > When we say supporting the build in msys2 instead of cygwin,
>         we just
>         > mean using msys2 as the unix emulating layer for our tools like
>         > make/bash/grep/sed etc.
>         >
>         > One think I have done successfully is running the build in WSL
>         > (Windows Subsystem for Linux), but that isn't all that
>         helpful as WSL
>         > for practical purposes is more or less like running Linux in
>         a VM, so
>         > the build sees a Linux system and builds a Linux binary.
>         >> As a side note: with MINGW64 I have managed to run
>         configure phase
>         >> successfully for OpenJDK. The compile process has also
>         started and
>         >> went for a while, but interestingly I run into some kind of
>         race
>         >> conditions as make stopped with an error. Using LOG=debug I
>         have fond
>         >> the failing line and then copying the failed command and
>         pasting it
>         >> to the bash prompt it successfully generated the output
>         target, and
>         >> then the build process run further when a similar situation
>         happened.
>         >> Also pasting the failed command run in the bash without any
>         problem,
>         >> and build continued… until the next.
>         > Without seeing the errors I can't say much. I very much
>         doubt that you
>         > are running with gcc as the compiler though. Configure isn't
>         easily
>         > fooled into using a different compiler to what it prefers,
>         and I would
>         > expect things to crash and burn pretty early if you actually
>         did.
>         >
>         > /Erik
>         >>
>         >> I have tried to run make JOBS=1, but did not help,
>         strangely I have
>         >> still seen in the log make[3] and make[4] logs which
>         suggested that
>         >> there are more than one make jobs were running. Also tried
>         .configure
>         >> --with-output-sync=recurse without success (same symptoms)
>         >>
>         >> Let me know your thoughts.
>         >>
>         >> Best regards,
>         >>
>         >> Peter
>         >>
>         >> Sent from
>         Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for
>         >> Windows 10
>         >>
>         >> From: Magnus Ihse Bursie<mailto:[hidden email]>
>         >> Sent: Wednesday, October 4, 2017 1:04 AM
>         >> To: Peter Budai<mailto:[hidden email]>;
>         >> [hidden email]
>         <mailto:[hidden email]><mailto:[hidden email]>
>         <mailto:[hidden email]>
>         >> Subject: Re: Building OpenJDK9 on MSYS2
>         >>
>         >> Actually, it wasn't so much remaining trouble. :-) I fired
>         up msys2 and
>         >> checked out where I left off. It turned out that the
>         remaining snag was
>         >> that msys2 tries to convert command lines automatically,
>         from "unix"
>         >> style paths to "windows" style paths. Unfortunately, it
>         does not do this
>         >> very well and it breaks all sorts of things. We already
>         have a FIXPATH
>         >> solution in place which deals with this, so basically all I
>         had to do
>         >> was disable this (by setting MSYS2_ARG_CONV_EXCL to "*").
>         However, this
>         >> broke our cygpath replacement hack (!) so I had to disable
>         it there.
>         >> Sigh. Anyway, with those fixes it ran and worked well. (I also
>         >> discovered and fixed a bug related to how we set up the
>         FIXPATH variable
>         >> on msys, but it only triggers in certain circumstances).
>         >>
>         >> With this patch I now jdk9 seems to build fine on msys2. It
>         should apply
>         >> cleanly on jdk9/jdk9. Since it turned out to be so trivial,
>         I'll try to
>         >> get it in in jdk10.
>         >>
>         >> Here's the patch if you want to apply it yourself:
>         >>
>         >> diff -r a08cbfc0e4ec common/autoconf/basics_windows.m4
>         >> --- a/common/autoconf/basics_windows.m4    Thu Aug 03
>         18:56:56 2017
>         >> +0000
>         >> +++ b/common/autoconf/basics_windows.m4    Wed Oct 04
>         00:53:58 2017
>         >> +0200
>         >> @@ -42,7 +42,7 @@
>         >>        windows_path=`$CYGPATH -m "$unix_path"`
>         >>        $1="$windows_path"
>         >>      elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then
>         >> -    windows_path=`cmd //c echo $unix_path`
>         >> +    windows_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo
>         $unix_path`
>         >>        $1="$windows_path"
>         >>      fi
>         >>    ])
>         >> @@ -136,6 +136,16 @@
>         >>      fi
>         >>    ])
>         >>
>         >> +AC_DEFUN([BASIC_MSYS_UPDATE_FIXPATH],
>         >> +[
>         >> +  # Take all collected prefixes and turn them into a
>         >> -m/c/foo@/c/bar@... command line
>         >> +  # @ was chosen as separator to minimize risk of other
>         tools messing
>         >> around with it
>         >> +  all_unique_prefixes=`echo
>         "${all_fixpath_prefixes@<:@@@:>@}" \
>         >> +      | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
>         >> +  fixpath_argument_list=`echo $all_unique_prefixes  | tr '
>         ' '@'`
>         >> +  FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
>         >> +])
>         >> +
>         >>    AC_DEFUN([BASIC_FIXUP_PATH_MSYS],
>         >>    [
>         >>      path="[$]$1"
>         >> @@ -143,7 +153,7 @@
>         >>      new_path="$path"
>         >>      if test "x$has_colon" = x; then
>         >>        # Not in mixed or Windows style, start by that.
>         >> -    new_path=`cmd //c echo $path`
>         >> +    new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $path`
>         >>      fi
>         >>
>         >> BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
>         >> @@ -155,6 +165,8 @@
>         >>
>         >>      # Save the first 10 bytes of this path to the storage,
>         so fixpath
>         >> can work.
>         >> all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
>         >> "${new_path:0:10}")
>         >> +  # We might need to re-evaluate FIXPATH.
>         >> +  BASIC_MSYS_UPDATE_FIXPATH
>         >>    ])
>         >>
>         >>    AC_DEFUN([BASIC_FIXUP_EXECUTABLE_CYGWIN],
>         >> @@ -293,7 +305,7 @@
>         >>        # Do not save /bin paths to all_fixpath_prefixes!
>         >>      else
>         >>        # Not in mixed or Windows style, start by that.
>         >> -    new_path=`cmd //c echo $new_path`
>         >> +    new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $new_path`
>         >> BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
>         >>        # Output is in $new_path
>         >> BASIC_WINDOWS_REWRITE_AS_UNIX_PATH(new_path)
>         >> @@ -302,6 +314,8 @@
>         >>
>         >>        # Save the first 10 bytes of this path to the
>         storage, so fixpath
>         >> can work.
>         >> all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
>         >> "${new_path:0:10}")
>         >> +    # We might need to re-evaluate FIXPATH.
>         >> +    BASIC_MSYS_UPDATE_FIXPATH
>         >>      fi
>         >>    ])
>         >>
>         >> @@ -347,6 +361,10 @@
>         >>        WINDOWS_ENV_VENDOR='msys'
>         >>        WINDOWS_ENV_VERSION="$MSYS_VERSION"
>         >>
>         >> +    # Prohibit msys2 path conversion from trying to be
>         "intelligent",
>         >> and rely
>         >> +    # on fixpath instead.
>         >> +    export MSYS2_ARG_CONV_EXCL="*"
>         >> +
>         >>        AC_MSG_CHECKING([msys root directory as unix-style
>         path])
>         >>        # The cmd output ends with Windows line endings
>         (CR/LF), the grep
>         >> command will strip that away
>         >>        MSYS_ROOT_PATH=`cd / ; cmd /c cd | $GREP ".*"`
>         >> @@ -391,10 +409,7 @@
>         >>        elif test "x$OPENJDK_BUILD_OS_ENV" = xwindows.msys; then
>         >>          # Take all collected prefixes and turn them into a
>         >> -m/c/foo@/c/bar@... command line
>         >>          # @ was chosen as separator to minimize risk of
>         other tools
>         >> messing around with it
>         >> -      all_unique_prefixes=`echo
>         "${all_fixpath_prefixes@<:@@@:>@}" \
>         >> -          | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
>         >> -      fixpath_argument_list=`echo $all_unique_prefixes  |
>         tr ' ' '@'`
>         >> -      FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
>         >> +      BASIC_MSYS_UPDATE_FIXPATH
>         >>        fi
>         >>        FIXPATH_SRC_W="$FIXPATH_SRC"
>         >>        FIXPATH_BIN_W="$FIXPATH_BIN"
>         >> diff -r a08cbfc0e4ec common/autoconf/build-aux/config.sub
>         >> --- a/common/autoconf/build-aux/config.sub Thu Aug 03 18:56:56
>         >> 2017 +0000
>         >> +++ b/common/autoconf/build-aux/config.sub Wed Oct 04 00:53:58
>         >> 2017 +0200
>         >> @@ -30,7 +30,7 @@
>         >>    DIR=`dirname $0`
>         >>
>         >>    # First, filter out everything that doesn't begin with
>         "aarch64-"
>         >> -if ! echo $* | grep '^aarch64-' >/dev/null ; then
>         >> +if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ; then
>         >>        . $DIR/autoconf-config.sub "$@"
>         >>        # autoconf-config.sub exits, so we never reach here,
>         but just in
>         >>        # case we do:
>         >> @@ -38,13 +38,17 @@
>         >>    fi
>         >>
>         >>    while test $# -gt 0 ; do
>         >> -    case $1 in
>         >> +    case $1 in
>         >>            -- )   # Stop option processing
>         >>                shift; break ;;
>         >>            aarch64-* )
>         >>                config=`echo $1 | sed 's/^aarch64-/arm-/'`
>         >>                sub_args="$sub_args $config"
>         >>                shift; ;;
>         >> +        *-msys )
>         >> +            config=`echo $1 | sed 's/msys/mingw32/'`
>         >> +            sub_args="$sub_args $config"
>         >> +            shift; ;;
>         >>            - )    # Use stdin as input.
>         >>                sub_args="$sub_args $1"
>         >>                shift; break ;;
>         >> diff -r a08cbfc0e4ec common/autoconf/spec.gmk.in
>         >> --- a/common/autoconf/spec.gmk.in    Thu Aug 03 18:56:56
>         2017 +0000
>         >> +++ b/common/autoconf/spec.gmk.in    Wed Oct 04 00:53:58
>         2017 +0200
>         >> @@ -120,6 +120,13 @@
>         >>      # On Windows, the Visual Studio toolchain needs the
>         PATH to be
>         >> adjusted
>         >>      # to include Visual Studio tools (this needs to be in
>         cygwin/msys
>         >> style).
>         >>      export PATH:=@VS_PATH@
>         >> +
>         >> +endif
>         >> +
>         >> +ifeq ($(OPENJDK_TARGET_OS_ENV), windows.msys)
>         >> +  # On msys2, prohibit msys path conversion from trying to be
>         >> +  # "intelligent", and rely on fixpath instead.
>         >> +  export MSYS2_ARG_CONV_EXCL:=*
>         >>    endif
>         >>
>         >>    SYSROOT_CFLAGS := @SYSROOT_CFLAGS@
>         >>
>         >> /Magnus
>         >>
>         >> On 2017-10-03 22:34, Magnus Ihse Bursie wrote:
>         >>> I gave msys2 a shot some time ago, but it ended up too
>         much trouble.
>         >>> I'll share some of my notes from that attempt, for what
>         it's worth.
>         >>>
>         >>> To install package X/Y, run "pacman -S X/Y". Missing tools and
>         >>> packages where to find them:
>         >>> cmp: msys/diffutils
>         >>> tar: msys/tar
>         >>> make: msys/make
>         >>> unzip: msys/unzip
>         >>> zip: msys/zip
>         >>>
>         >>> config.sub reports msys as "x86_64-pc-mingw32" but msys2 as
>         >>> "x86_64-pc-msys". This patch adds postprocessing in "our"
>         config.sub
>         >>> to report msys2 similar to msys. (Opinions, including my
>         own :-) may
>         >>> vary if this really is the best way..)
>         >>>
>         >>> diff -r b88023f46daa common/autoconf/build-aux/config.sub
>         >>> --- a/common/autoconf/build-aux/config.sub      Fri Jan 27
>         10:15:41
>         >>> 2017 +0100
>         >>> +++ b/common/autoconf/build-aux/config.sub      Fri Feb 03
>         05:00:25
>         >>> 2017 -0700
>         >>> @@ -30,7 +30,7 @@
>         >>>   DIR=`dirname $0`
>         >>>
>         >>>   # First, filter out everything that doesn't begin with
>         "aarch64-"
>         >>> -if ! echo $* | grep '^aarch64-' >/dev/null ; then
>         >>> +if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ;
>         then
>         >>>       . $DIR/autoconf-config.sub "$@"
>         >>>       # autoconf-config.sub exits, so we never reach here,
>         but just in
>         >>>       # case we do:
>         >>> @@ -45,6 +45,10 @@
>         >>>               config=`echo $1 | sed 's/^aarch64-/arm-/'`
>         >>>               sub_args="$sub_args $config"
>         >>>               shift; ;;
>         >>> +        *-msys )
>         >>> +            config=`echo $1 | sed 's/msys/mingw32/'`
>         >>> +            sub_args="$sub_args $config"
>         >>> +            shift; ;;
>         >>>           - )    # Use stdin as input.
>         >>>               sub_args="$sub_args $1"
>         >>>               shift; break ;;
>         >>>
>         >>> If I remember correctly, this got me past the configure
>         stage at the
>         >>> time.
>         >>>
>         >>> I don't think it's very hard to get it to work on msys2, I
>         just ran
>         >>> into one snag too many and didn't think msys2 would be
>         used by anyone.
>         >>>
>         >>> /Magnus
>         >>>
>         >>> On 2017-10-03 17:20, Peter Budai wrote:
>         >>>> Hello,
>         >>>>
>         >>>> According to
>         >>>>
>         http://hg.openjdk.java.net/jdk9/jdk9/file/a08cbfc0e4ec/common/doc/building.html
>
>         >>>>
>         >>>> “msys2 and the new Windows Subsystem for Linux (WSL)
>         would likely be
>         >>>> possible to support in a future version but that would
>         require a
>         >>>> community effort to implement”
>         >>>>
>         >>>> I’d like to help making the OpenJDK 9 build working on
>         msys2. What is
>         >>>> the best way to move forward? Is there a similar effort
>         in progress?
>         >>>>
>         >>>> Thank you and best regards,
>         >>>>
>         >>>> Peter
>         >>>>
>         >>>>
>         >
>

Reply | Threaded
Open this post in threaded view
|

RE: Building OpenJDK9 on MSYS2

Peter Budai
Hi Erik,

The .d file looks like this:
C:/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/hotspot/variant-server/tools/adlc/objs/adlparse.obj: \
C:/msys64/home/peterbud/jdk9/hotspot/src/share/vm/adlc/adlparse.cpp \

I have checked .d.targets file, and looks like it has the first line has not been deleted, and the file names below are also wrong:
/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/hotspot/variant-server/tools/adlc/objs/adlparse.obj : :
/msys64/home/peterbud/jdk9/hotspot/src/share/vm/adlc/adlparse.cpp :
/msys64/home/peterbud/jdk9/hotspot/src/share/vm/adlc/adlc.hpp :

I guess this part in the DEPENDENCY_TARGET_SED_PATTERN is fooled by the “C:/”
-e 's/^[^:]*: *//'


Peter

Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10

From: Erik Joelsson<mailto:[hidden email]>
Sent: Wednesday, October 11, 2017 12:16 PM
To: Peter Budai<mailto:[hidden email]>; Magnus Ihse Bursie<mailto:[hidden email]>; [hidden email]<mailto:[hidden email]>
Subject: Re: Building OpenJDK9 on MSYS2


Hello Peter,

On 2017-10-11 00:18, Peter Budai wrote:
Thanks Magnus & Erik

First thanks for your support and kind words!

Magnus, I have checked .bash_profile, .bashrc but they seem to be empty (everything is commented out). You can check with a default MSYS2 install, I have not changed these files at all. If you find thee something specific I can give a try here as well.

Let me give also a quick status update, where am I with building hotspot:
·       I guess its still the beginning, but I have managed to compile jvm.dll with almost 700 object file: with debug info the dll is around 700 MB 😊
·       I made only surgical, minimal changes to the source, and so far it looks reasonable. I have encountered 3 scenarios where changes were necessary:
o   When in makefiles conditionals were using assuming that if target_os is windows then it is visual studio compiler/linker. Obviously these conditionals had to be reviewed in a few places and if necessary were changes to check the toolchain=Microsoft
These are not surprising and should be pretty straight forward to fix and it seems you know what to do.

·
o   I got a few warnings as gcc 7.2 uncovered some code problems in windows specific codes, where before that MSVC I guess did not say a word…
To get around this you can configure with --disable-warnings-as-errors until you get things working properly. This is commonly needed when using compiler versions that we normally don't use.

·
o   And I had like 6-7 places where the code was using MSVC specific __try … __except structures which gcc does not know. Do you have a suggestion how to approach them? I can do ugly #ifdefs (I would avoid that) but I have also seen some solutions to replace them with a code which gcc can compile (http://www.programmingunlimited.net/siteexec/content.cgi?page=mingw-seh ) – but before doing that  though I would ask first you on the purpose of those
This kind of question is probably best to bring to the hotspot mailing list.
·       What bothers me is that I was not able to do incremental builds: when an error occurs, and build stops, then after making change in the CPP source the build cannot continue, I always got an error message:

/C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/hotspot/variant-server/tools/adlc/objs/adlparse.d.targets:1: *** missing target pattern.  Stop.
make[2]: *** [make/Main.gmk:256: hotspot-server-gensrc] Error 2

If I do a ‘make clean’ and restart the build then it nicely compiles.

Question 1: Is there a way to  resume such builds without ‘make clean’?
Well, incremental builds is supposed to work well. We have several extra tricks in there to handle cases where normal make builds would fail. The *.d.targets files is one such trick and it seems to backfire for you. The contents of that file should be something like:

/localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/adlparse.cpp :
/localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/adlc.hpp :
/localhome/hg/jdk9-dev/hotspot/src/share/vm/opto/opcodes.hpp :
/localhome/hg/jdk9-dev/hotspot/src/share/vm/opto/classes.hpp :
/localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/arena.hpp :
/localhome/hg/jdk9-dev/hotspot/src/share/vm/opto/adlcVMDeps.hpp :
/localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/filebuff.hpp :
/localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/dict2.hpp :
/localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/forms.hpp :
/localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/formsopt.hpp :
/localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/formssel.hpp :
/localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/archDesc.hpp :
/localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/adlparse.hpp :

Basically an empty rule for each dependency for the corresponding object file. Declaring these rules makes it possible to delete source files without having to build clean. It seems your file is not generated correctly so please have a look inside it. The file is in make/common/NativeCompilation.gmk, look for DEPENDENCY_TARGET_SED_PATTERN.


Question 2: What would be the best way to submit/share the patches for your thorough review?

Well, first of all, have you signed the OCA?

As for publishing patches and reviews, there is a bit of chicken and egg problem. Once you become an "author" in any of the OpenJDK projects, you get a user name and should be able to publish reviews on cr.openjdk.java.net. Before that, if the patch is small, it can be posted inline in an email to the list. If it's large, you will need a current OpenJDK user to host it for you. At least that's how I understand it. Hopefully someone who knows the process better can chime in here.

I should also let you know that getting this into JDK 9 is most likely not going to happen. AFAIK we are only doing security updates for 9. It would have to go into the currently active release. I should also warn you that new ports generally need a certain amount of backing to be accepted. It may be that this would have to live in a porting side project. Hopefully someone who knows this better can chime in here as well.

/Erik

P.

Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10

From: Magnus Ihse Bursie<mailto:[hidden email]>
Sent: Tuesday, October 10, 2017 10:04 AM
To: Peter Budai<mailto:[hidden email]>; Erik Joelsson<mailto:[hidden email]>; [hidden email]<mailto:[hidden email]>
Subject: Re: Building OpenJDK9 on MSYS2

On 2017-10-07 10:14, Peter Budai wrote:
The configure of OpenJDK overwrites the SHELL. Actually it is using bash, but for the arguments it was using “-e -o pipefail”. I have figured that for MSYS2 bash what is needed as bash arguments is “-e -l -c -o pipefail”

That looks like solving this problem, and now the real issues are surfacing.

FWIW, "-l" makes bash behave like a login shell. Most likely you are changing bash's behavior in one of your login scripts, and that change is what's really needed.

/Magnus




Peter

From: Peter Budai<mailto:[hidden email]>
Sent: Friday, October 6, 2017 6:43 PM
To: Magnus Ihse Bursie<mailto:[hidden email]>; Erik Joelsson<mailto:[hidden email]>; [hidden email]<mailto:[hidden email]>
Subject: RE: Building OpenJDK9 on MSYS2

Magnus,

I have followed your suggestion and removed the fixpath prefixes from gcc related compile tools, and left only the fixpath prefix _only_ for the Boot JDK related tools in place.

1)      As  I follow the process, all java and javac related compile steps are running properly
2)      When the process reaches gcc related steps I got the error message at the same place as before (no fixpath). If I execute that command from the bash prompt, it creates the output:

$ ( /usr/bin/gawk '/@@END_COPYRIGHT@@/{exit}1' /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template && /C/msys64/mingw64/bin/gcc -E -x c  /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template 2> >(/usr/bin/grep -v '^SocketOptionRegistry.java.template$' >&2) | /usr/bin/gawk '/@@START_HERE@@/,0' |  /usr/bin/sed -e 's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED FILE - DO NOT EDIT/' -e 's/PREFIX_//' -e 's/^#.*//' ) > /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java

As I have mentioned the parameters are replaced by the bash automatically
3)      Then build continues, then little later stops at a super simple command:

mv /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBuffer.java.tmp /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBuffer.java
           Needless to say, the ByteBuffer.java.tmp file DOES exist. And running the above command from the bash works, and build continues.
4)      A few similar cases (stops) with DirectByteBuffer and DirectByteBufferR


Currently I try to explore how that might relate to the MSYS2 bash and make, somehow it behaves differently

If you have any other suggestion, let me know.

Best regards,

Peter

From: Peter Budai<mailto:[hidden email]>
Sent: Thursday, October 5, 2017 3:52 PM
To: Magnus Ihse Bursie<mailto:[hidden email]>; Erik Joelsson<mailto:[hidden email]>; [hidden email]<mailto:[hidden email]>
Subject: RE: Building OpenJDK9 on MSYS2

Hi Magnus,

So first of all, here is the current patch, which I was not able to attach: https://pastebin.com/pwT4Ynxc

>> That's surprising, since gcc is prefixed with fixpath, which it should not.
Actually you DO need fixpath IMHO.
This is a mingw64 version of the gcc (/C/msys64/mingw64/bin/gcc), which is a fully functional Windows executable, which expects Windows formatted path arguments.
As the updated build process uses EXPORT MSYS2_ARG_CONV_EXCL=* (see that patch), none of the command line arguments are converted  from the unix path to Windows, but fixpath does that conversion. There is a wiki describing more details on this:
https://github.com/msys2/msys2/wiki/Porting#user-content-filesystem-namespaces



>>I have a hard time believing this is a race condition. On the other hand, this stuff is weird, we're misusing the C preprocessor to process defines in java code, so I'm not surprised it breaks down.
>>I don't know why it succeeded when run on the command line, though.
When I execute that command from the bash command line there is no EXPORT MSYS2_ARG_CONV_EXCL, but the bash itself does the automatic conversion of the arguments. Maybe it has something to do how fixpath does CreateProcess?

Does that help?

Best regards,

Peter

Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10

From: Magnus Ihse Bursie<mailto:[hidden email]>
Sent: Thursday, October 5, 2017 12:13 PM
To: Peter Budai<mailto:[hidden email]>; Erik Joelsson<mailto:[hidden email]>; [hidden email]<mailto:[hidden email]>
Subject: Re: Building OpenJDK9 on MSYS2


On 2017-10-05 11:59, Peter Budai wrote:
Hi Magnus and Erik,

I really appreciate your quick feedback. I assumed that it won’t be easy, but I just don’t feel I should give up now  - maybe later when I see the real scale of work. So bear with me for a time being.

Attached is a patch which already includes Magnus’ changes, plus a few which I have added:
·       basically enabling gcc for windows,
·       and modifying a logic for compiling fixpath (before that it was using hard-coded MS VSC compile flags)

Actually, you must make sure fixpath is *not* used for the toolchain, since gcc uses unix style paths.
(However, other tools such as java will still need it.)





So here is what I have as the result of configure:
====================================================
The existing configuration has been successfully updated in
/C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release
using configure arguments '--disable-freetype-bundling --disable-javac-server'.

Configuration summary:
* Debug level:    release
* HS debug level: product
* JDK variant:    normal
* JVM variants:   server
* OpenJDK target: OS: windows, CPU architecture: x86, address length: 64
* Version string: 9-internal+0-adhoc.peterbud.jdk9 (9-internal)

Tools summary:
* Environment:    msys version 2.9.0(0.318/5/3) (root at /C/msys64)
* Boot JDK:       java version "1.8.0_144"  Java(TM) SE Runtime Environment (build 1.8.0_144-b01)  Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)   (at /c/progra~1/java/jdk18~1.0_1)
* Toolchain:      gcc (GNU Compiler Collection)
* C Compiler:     Version 7.2.0 (at /C/msys64/mingw64/bin/gcc)
* C++ Compiler:   Version 7.2.0 (at /c/msys64/mingw64/bin/g++)

Build performance summary:
* Cores to use:   4
* Memory limit:   16216 MB

Its clear says that the toolchain is gcc 7.2 (BTW there is no Visual Studio on this machine)

Now for the details of the config log, you can see here: https://pastebin.com/MN2ZYcHH

And about the build process and the error I get:

$ make JOBS=1
Building target 'default (exploded-image)' in configuration 'windows-x86_64-normal-server-release'
Compiling 8 files for BUILD_TOOLS_LANGTOOLS
Compiling 17 properties into resource bundles for jdk.compiler
Parsing 1 properties into enum-like class for jdk.compiler
Compiling 19 properties into resource bundles for jdk.javadoc
Compiling 12 properties into resource bundles for jdk.jdeps
Compiling 7 properties into resource bundles for jdk.jshell
Compiling 117 files for BUILD_INTERIM_java.compiler
Compiling 396 files for BUILD_INTERIM_jdk.compiler
Compiling 61 files for BUILD_INTERIM_jdk.jdeps
Compiling 457 files for BUILD_INTERIM_jdk.javadoc
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Compiling 159 files for BUILD_TOOLS_JDK
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
make[3]: *** [GensrcMisc.gmk:78: /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java] Error 1
make[3]: *** Deleting file '/C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java'
make[2]: *** [make/Main.gmk:115: java.base-gensrc-jdk] Error 2

ERROR: Build failed for target 'default (exploded-image)' in configuration 'windows-x86_64-normal-server-release' (exit code 2)

No indication of failed target found.
Hint: Try searching the build log for '] Error'.
Hint: See common/doc/building.html#troubleshooting for assistance.

make[1]: *** [/home/peterbud/jdk9/make/Init.gmk:296: main] Error 2
make: *** [/home/peterbud/jdk9/make/Init.gmk:185: default] Error 2

If I run here
make JOBS=1 LOG=debug
The failing line seems to be this:

( /usr/bin/gawk '/@@END_COPYRIGHT@@/{exit}1' /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template && /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/configure-support/bin/fixpath.exe -m/C/msys64/@/c/msys64/@/c/progra~ /C/msys64/mingw64/bin/gcc -E -x c  /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template 2> >(/usr/bin/grep -v '^SocketOptionRegistry.java.template$' >&2) | /usr/bin/gawk '/@@START_HERE@@/,0' |  /usr/bin/sed -e 's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED FILE - DO NOT EDIT/' -e 's/PREFIX_//' -e 's/^#.*//' ) > /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java
make[3]: *** [GensrcMisc.gmk:78: /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java] Error 1

Now the interesting is: if I copy this line above to the bash prompt, it runs without problem, and the file support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java
That's surprising, since gcc is prefixed with fixpath, which it should not.

I have a hard time believing this is a race condition. On the other hand, this stuff is weird, we're misusing the C preprocessor to process defines in java code, so I'm not suprised it breaks down. I don't know why it succeeded when run on the command line, though. My suggestion is to just do some quick and dirty hack around this: take the file you manage to generate and just copy it in during the build instead. If you can get round this, you might start seeing some *real* problems. :-)

Also, my suggestion is that you try running "make hotspot" to cut to the chase. Compiling hotspot will likely be the hardest thing. Or even "make -k hotspot" to get an assessment of the amount of work ahead of you.

/Magnus
Is produced.

Then I can again issue
make JOBS=1 LOG=debug

And the compile process is being continued, until a similar error pops up with a different generated file. I have an assumption that this happens because make is still running parallel jobs, despite JOBS=1 but I’m not sure.

How could I best tackle this?

Thank you and best regards,

Peter

Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10

From: Magnus Ihse Bursie<mailto:[hidden email]>
Sent: Thursday, October 5, 2017 11:33 AM
To: Erik Joelsson<mailto:[hidden email]>; Peter Budai<mailto:[hidden email]>; [hidden email]<mailto:[hidden email]>
Subject: Re: Building OpenJDK9 on MSYS2

On 2017-10-05 10:10, Erik Joelsson wrote:

> Hello Peter,
>
>
> On 2017-10-04 21:15, Peter Budai wrote:
>> Hi Magnus,
>>
>> Thanks for the quick reply I’ll check these patches with msys2.
>>
>> Let me specify with more details what I’d like to achieve: I’d like
>> to build OpenJDK9 with MSYS2 MINGW64 environment using gcc toolchain.
>> (I’m not sure how familiar are you with MSYS2, but there are 3
>> different environments: MSYS2, MINGW32 and MINGW64). In theory
>> MINGW64 with gcc is the closes you can get on Windows platform as a
>> gcc unix like build environment, which produces still a native 64-bit
>> executable on Windows.
>>
>> I’m not very familiar with OpenJDK yet, so therefore I’d like to hear
>> your opinion: how realistic is that?
> Sorry to disappoint, but I would say that requires major work. There
> is a strong historic assumption that windows builds are done using
> Visual Studio. We have abstracted away some of it in configure (see
> TOOLCHAIN_TYPE), but it's very far from enough to change compiler
> environment for a Windows build. The native sources are also bound to
> make a lot of such assumptions. I would expect the changes needed to
> be in the thousands of lines of code.

I agree that it requires hard work (even if "thousands" might be an
overestimation I think, but "hundreds" is not enough, so it's the right
magnitude). On the other hand, it would be really good if we did sort
things out, so that we had proper conditions based on OS vs
compiler/toolchain.

If you really want to start, the first thing is to patch toolchain.m4 to
VALID_TOOLCHAINS_windows="microsoft gcc"
and then call configure using "bash configure --with-toolchain-type=gcc".

As Erik, I doubt you will come very far before things starts tumbling down.

>
> When we say supporting the build in msys2 instead of cygwin, we just
> mean using msys2 as the unix emulating layer for our tools like
> make/bash/grep/sed etc.
>
> One think I have done successfully is running the build in WSL
> (Windows Subsystem for Linux), but that isn't all that helpful as WSL
> for practical purposes is more or less like running Linux in a VM, so
> the build sees a Linux system and builds a Linux binary.
>> As a side note: with MINGW64 I have managed to run configure phase
>> successfully for OpenJDK. The compile process has also started and
>> went for a while, but interestingly I run into some kind of race
>> conditions as make stopped with an error. Using LOG=debug I have fond
>> the failing line and then copying the failed command and pasting it
>> to the bash prompt it successfully generated the output target, and
>> then the build process run further when a similar situation happened.
>> Also pasting the failed command run in the bash without any problem,
>> and build continued… until the next.
> Without seeing the errors I can't say much. I very much doubt that you
> are running with gcc as the compiler though. Configure isn't easily
> fooled into using a different compiler to what it prefers, and I would
> expect things to crash and burn pretty early if you actually did.
>
> /Erik
>>
>> I have tried to run make JOBS=1, but did not help, strangely I have
>> still seen in the log make[3] and make[4] logs which suggested that
>> there are more than one make jobs were running. Also tried .configure
>> --with-output-sync=recurse without success (same symptoms)
>>
>> Let me know your thoughts.
>>
>> Best regards,
>>
>> Peter
>>
>> Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for
>> Windows 10
>>
>> From: Magnus Ihse Bursie<mailto:[hidden email]>
>> Sent: Wednesday, October 4, 2017 1:04 AM
>> To: Peter Budai<mailto:[hidden email]>;
>> [hidden email]<mailto:[hidden email]><mailto:[hidden email]><mailto:[hidden email]>
>> Subject: Re: Building OpenJDK9 on MSYS2
>>
>> Actually, it wasn't so much remaining trouble. :-) I fired up msys2 and
>> checked out where I left off. It turned out that the remaining snag was
>> that msys2 tries to convert command lines automatically, from "unix"
>> style paths to "windows" style paths. Unfortunately, it does not do this
>> very well and it breaks all sorts of things. We already have a FIXPATH
>> solution in place which deals with this, so basically all I had to do
>> was disable this (by setting MSYS2_ARG_CONV_EXCL to "*"). However, this
>> broke our cygpath replacement hack (!) so I had to disable it there.
>> Sigh. Anyway, with those fixes it ran and worked well. (I also
>> discovered and fixed a bug related to how we set up the FIXPATH variable
>> on msys, but it only triggers in certain circumstances).
>>
>> With this patch I now jdk9 seems to build fine on msys2. It should apply
>> cleanly on jdk9/jdk9. Since it turned out to be so trivial, I'll try to
>> get it in in jdk10.
>>
>> Here's the patch if you want to apply it yourself:
>>
>> diff -r a08cbfc0e4ec common/autoconf/basics_windows.m4
>> --- a/common/autoconf/basics_windows.m4    Thu Aug 03 18:56:56 2017
>> +0000
>> +++ b/common/autoconf/basics_windows.m4    Wed Oct 04 00:53:58 2017
>> +0200
>> @@ -42,7 +42,7 @@
>>        windows_path=`$CYGPATH -m "$unix_path"`
>>        $1="$windows_path"
>>      elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then
>> -    windows_path=`cmd //c echo $unix_path`
>> +    windows_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $unix_path`
>>        $1="$windows_path"
>>      fi
>>    ])
>> @@ -136,6 +136,16 @@
>>      fi
>>    ])
>>
>> +AC_DEFUN([BASIC_MSYS_UPDATE_FIXPATH],
>> +[
>> +  # Take all collected prefixes and turn them into a
>> -m/c/foo@/c/bar@... command line
>> +  # @ was chosen as separator to minimize risk of other tools messing
>> around with it
>> +  all_unique_prefixes=`echo "${all_fixpath_prefixes@<:@@@:>@}" \
>> +      | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
>> +  fixpath_argument_list=`echo $all_unique_prefixes  | tr ' ' '@'`
>> +  FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
>> +])
>> +
>>    AC_DEFUN([BASIC_FIXUP_PATH_MSYS],
>>    [
>>      path="[$]$1"
>> @@ -143,7 +153,7 @@
>>      new_path="$path"
>>      if test "x$has_colon" = x; then
>>        # Not in mixed or Windows style, start by that.
>> -    new_path=`cmd //c echo $path`
>> +    new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $path`
>>      fi
>>
>>      BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
>> @@ -155,6 +165,8 @@
>>
>>      # Save the first 10 bytes of this path to the storage, so fixpath
>> can work.
>> all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
>> "${new_path:0:10}")
>> +  # We might need to re-evaluate FIXPATH.
>> +  BASIC_MSYS_UPDATE_FIXPATH
>>    ])
>>
>>    AC_DEFUN([BASIC_FIXUP_EXECUTABLE_CYGWIN],
>> @@ -293,7 +305,7 @@
>>        # Do not save /bin paths to all_fixpath_prefixes!
>>      else
>>        # Not in mixed or Windows style, start by that.
>> -    new_path=`cmd //c echo $new_path`
>> +    new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $new_path`
>>        BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
>>        # Output is in $new_path
>>        BASIC_WINDOWS_REWRITE_AS_UNIX_PATH(new_path)
>> @@ -302,6 +314,8 @@
>>
>>        # Save the first 10 bytes of this path to the storage, so fixpath
>> can work.
>> all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
>> "${new_path:0:10}")
>> +    # We might need to re-evaluate FIXPATH.
>> +    BASIC_MSYS_UPDATE_FIXPATH
>>      fi
>>    ])
>>
>> @@ -347,6 +361,10 @@
>>        WINDOWS_ENV_VENDOR='msys'
>>        WINDOWS_ENV_VERSION="$MSYS_VERSION"
>>
>> +    # Prohibit msys2 path conversion from trying to be "intelligent",
>> and rely
>> +    # on fixpath instead.
>> +    export MSYS2_ARG_CONV_EXCL="*"
>> +
>>        AC_MSG_CHECKING([msys root directory as unix-style path])
>>        # The cmd output ends with Windows line endings (CR/LF), the grep
>> command will strip that away
>>        MSYS_ROOT_PATH=`cd / ; cmd /c cd | $GREP ".*"`
>> @@ -391,10 +409,7 @@
>>        elif test "x$OPENJDK_BUILD_OS_ENV" = xwindows.msys; then
>>          # Take all collected prefixes and turn them into a
>> -m/c/foo@/c/bar@... command line
>>          # @ was chosen as separator to minimize risk of other tools
>> messing around with it
>> -      all_unique_prefixes=`echo "${all_fixpath_prefixes@<:@@@:>@}" \
>> -          | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
>> -      fixpath_argument_list=`echo $all_unique_prefixes  | tr ' ' '@'`
>> -      FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
>> +      BASIC_MSYS_UPDATE_FIXPATH
>>        fi
>>        FIXPATH_SRC_W="$FIXPATH_SRC"
>>        FIXPATH_BIN_W="$FIXPATH_BIN"
>> diff -r a08cbfc0e4ec common/autoconf/build-aux/config.sub
>> --- a/common/autoconf/build-aux/config.sub    Thu Aug 03 18:56:56
>> 2017 +0000
>> +++ b/common/autoconf/build-aux/config.sub    Wed Oct 04 00:53:58
>> 2017 +0200
>> @@ -30,7 +30,7 @@
>>    DIR=`dirname $0`
>>
>>    # First, filter out everything that doesn't begin with "aarch64-"
>> -if ! echo $* | grep '^aarch64-' >/dev/null ; then
>> +if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ; then
>>        . $DIR/autoconf-config.sub "$@"
>>        # autoconf-config.sub exits, so we never reach here, but just in
>>        # case we do:
>> @@ -38,13 +38,17 @@
>>    fi
>>
>>    while test $# -gt 0 ; do
>> -    case $1 in
>> +    case $1 in
>>            -- )   # Stop option processing
>>                shift; break ;;
>>            aarch64-* )
>>                config=`echo $1 | sed 's/^aarch64-/arm-/'`
>>                sub_args="$sub_args $config"
>>                shift; ;;
>> +        *-msys )
>> +            config=`echo $1 | sed 's/msys/mingw32/'`
>> +            sub_args="$sub_args $config"
>> +            shift; ;;
>>            - )    # Use stdin as input.
>>                sub_args="$sub_args $1"
>>                shift; break ;;
>> diff -r a08cbfc0e4ec common/autoconf/spec.gmk.in
>> --- a/common/autoconf/spec.gmk.in    Thu Aug 03 18:56:56 2017 +0000
>> +++ b/common/autoconf/spec.gmk.in    Wed Oct 04 00:53:58 2017 +0200
>> @@ -120,6 +120,13 @@
>>      # On Windows, the Visual Studio toolchain needs the PATH to be
>> adjusted
>>      # to include Visual Studio tools (this needs to be in cygwin/msys
>> style).
>>      export PATH:=@VS_PATH@
>> +
>> +endif
>> +
>> +ifeq ($(OPENJDK_TARGET_OS_ENV), windows.msys)
>> +  # On msys2, prohibit msys path conversion from trying to be
>> +  # "intelligent", and rely on fixpath instead.
>> +  export MSYS2_ARG_CONV_EXCL:=*
>>    endif
>>
>>    SYSROOT_CFLAGS := @SYSROOT_CFLAGS@
>>
>> /Magnus
>>
>> On 2017-10-03 22:34, Magnus Ihse Bursie wrote:
>>> I gave msys2 a shot some time ago, but it ended up too much trouble.
>>> I'll share some of my notes from that attempt, for what it's worth.
>>>
>>> To install package X/Y, run "pacman -S X/Y". Missing tools and
>>> packages where to find them:
>>> cmp: msys/diffutils
>>> tar: msys/tar
>>> make: msys/make
>>> unzip: msys/unzip
>>> zip: msys/zip
>>>
>>> config.sub reports msys as "x86_64-pc-mingw32" but msys2 as
>>> "x86_64-pc-msys". This patch adds postprocessing in "our" config.sub
>>> to report msys2 similar to msys. (Opinions, including my own :-) may
>>> vary if this really is the best way..)
>>>
>>> diff -r b88023f46daa common/autoconf/build-aux/config.sub
>>> --- a/common/autoconf/build-aux/config.sub      Fri Jan 27 10:15:41
>>> 2017 +0100
>>> +++ b/common/autoconf/build-aux/config.sub      Fri Feb 03 05:00:25
>>> 2017 -0700
>>> @@ -30,7 +30,7 @@
>>>   DIR=`dirname $0`
>>>
>>>   # First, filter out everything that doesn't begin with "aarch64-"
>>> -if ! echo $* | grep '^aarch64-' >/dev/null ; then
>>> +if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ; then
>>>       . $DIR/autoconf-config.sub "$@"
>>>       # autoconf-config.sub exits, so we never reach here, but just in
>>>       # case we do:
>>> @@ -45,6 +45,10 @@
>>>               config=`echo $1 | sed 's/^aarch64-/arm-/'`
>>>               sub_args="$sub_args $config"
>>>               shift; ;;
>>> +        *-msys )
>>> +            config=`echo $1 | sed 's/msys/mingw32/'`
>>> +            sub_args="$sub_args $config"
>>> +            shift; ;;
>>>           - )    # Use stdin as input.
>>>               sub_args="$sub_args $1"
>>>               shift; break ;;
>>>
>>> If I remember correctly, this got me past the configure stage at the
>>> time.
>>>
>>> I don't think it's very hard to get it to work on msys2, I just ran
>>> into one snag too many and didn't think msys2 would be used by anyone.
>>>
>>> /Magnus
>>>
>>> On 2017-10-03 17:20, Peter Budai wrote:
>>>> Hello,
>>>>
>>>> According to
>>>> http://hg.openjdk.java.net/jdk9/jdk9/file/a08cbfc0e4ec/common/doc/building.html
>>>>
>>>> “msys2 and the new Windows Subsystem for Linux (WSL) would likely be
>>>> possible to support in a future version but that would require a
>>>> community effort to implement”
>>>>
>>>> I’d like to help making the OpenJDK 9 build working on msys2. What is
>>>> the best way to move forward? Is there a similar effort in progress?
>>>>
>>>> Thank you and best regards,
>>>>
>>>> Peter
>>>>
>>>>
>









Reply | Threaded
Open this post in threaded view
|

Re: Building OpenJDK9 on MSYS2

Magnus Ihse Bursie
For gcc, we let the compiler generate the .d file. For the Microsoft tool chain, we use a clever sed script to extract and create it ourself.

I think that logic is checking for "Windows", not "Microsoft". That might be your cause of trouble.

Look in NativeCompilation.gmk.

/Magnus

> 11 okt. 2017 kl. 14:43 skrev Peter Budai <[hidden email]>:
>
> Hi Erik,
>  
> The .d file looks like this:
> C:/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/hotspot/variant-server/tools/adlc/objs/adlparse.obj: \
> C:/msys64/home/peterbud/jdk9/hotspot/src/share/vm/adlc/adlparse.cpp \
>  
> I have checked .d.targets file, and looks like it has the first line has not been deleted, and the file names below are also wrong:
> /msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/hotspot/variant-server/tools/adlc/objs/adlparse.obj : :
> /msys64/home/peterbud/jdk9/hotspot/src/share/vm/adlc/adlparse.cpp :
> /msys64/home/peterbud/jdk9/hotspot/src/share/vm/adlc/adlc.hpp :
>  
> I guess this part in the DEPENDENCY_TARGET_SED_PATTERN is fooled by the “C:/”
> -e 's/^[^:]*: *//'
>  
>  
> Peter
>  
> Sent from Mail for Windows 10
>  
> From: Erik Joelsson
> Sent: Wednesday, October 11, 2017 12:16 PM
> To: Peter Budai; Magnus Ihse Bursie; [hidden email]
> Subject: Re: Building OpenJDK9 on MSYS2
>  
> Hello Peter,
>
>  
> On 2017-10-11 00:18, Peter Budai wrote:
> Thanks Magnus & Erik
>  
> First thanks for your support and kind words!
>  
> Magnus, I have checked .bash_profile, .bashrc but they seem to be empty (everything is commented out). You can check with a default MSYS2 install, I have not changed these files at all. If you find thee something specific I can give a try here as well.
>  
> Let me give also a quick status update, where am I with building hotspot:
> ·       I guess its still the beginning, but I have managed to compile jvm.dll with almost 700 object file: with debug info the dll is around 700 MB 😊
> ·       I made only surgical, minimal changes to the source, and so far it looks reasonable. I have encountered 3 scenarios where changes were necessary:
> o   When in makefiles conditionals were using assuming that if target_os is windows then it is visual studio compiler/linker. Obviously these conditionals had to be reviewed in a few places and if necessary were changes to check the toolchain=Microsoft
> These are not surprising and should be pretty straight forward to fix and it seems you know what to do.
>
> ·        
> o   I got a few warnings as gcc 7.2 uncovered some code problems in windows specific codes, where before that MSVC I guess did not say a word…
> To get around this you can configure with --disable-warnings-as-errors until you get things working properly. This is commonly needed when using compiler versions that we normally don't use.
>
> ·        
> o   And I had like 6-7 places where the code was using MSVC specific __try … __except structures which gcc does not know. Do you have a suggestion how to approach them? I can do ugly #ifdefs (I would avoid that) but I have also seen some solutions to replace them with a code which gcc can compile (http://www.programmingunlimited.net/siteexec/content.cgi?page=mingw-seh ) – but before doing that  though I would ask first you on the purpose of those
> This kind of question is probably best to bring to the hotspot mailing list.
> ·       What bothers me is that I was not able to do incremental builds: when an error occurs, and build stops, then after making change in the CPP source the build cannot continue, I always got an error message:
> /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/hotspot/variant-server/tools/adlc/objs/adlparse.d.targets:1: *** missing target pattern.  Stop.
> make[2]: *** [make/Main.gmk:256: hotspot-server-gensrc] Error 2
>
> If I do a ‘make clean’ and restart the build then it nicely compiles.
> Question 1: Is there a way to  resume such builds without ‘make clean’?
> Well, incremental builds is supposed to work well. We have several extra tricks in there to handle cases where normal make builds would fail. The *.d.targets files is one such trick and it seems to backfire for you. The contents of that file should be something like:
>
> /localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/adlparse.cpp :
> /localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/adlc.hpp :
> /localhome/hg/jdk9-dev/hotspot/src/share/vm/opto/opcodes.hpp :
> /localhome/hg/jdk9-dev/hotspot/src/share/vm/opto/classes.hpp :
> /localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/arena.hpp :
> /localhome/hg/jdk9-dev/hotspot/src/share/vm/opto/adlcVMDeps.hpp :
> /localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/filebuff.hpp :
> /localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/dict2.hpp :
> /localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/forms.hpp :
> /localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/formsopt.hpp :
> /localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/formssel.hpp :
> /localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/archDesc.hpp :
> /localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/adlparse.hpp :
>
> Basically an empty rule for each dependency for the corresponding object file. Declaring these rules makes it possible to delete source files without having to build clean. It seems your file is not generated correctly so please have a look inside it. The file is in make/common/NativeCompilation.gmk, look for DEPENDENCY_TARGET_SED_PATTERN.
>
> Question 2: What would be the best way to submit/share the patches for your thorough review?
>  
> Well, first of all, have you signed the OCA?
>
> As for publishing patches and reviews, there is a bit of chicken and egg problem. Once you become an "author" in any of the OpenJDK projects, you get a user name and should be able to publish reviews on cr.openjdk.java.net. Before that, if the patch is small, it can be posted inline in an email to the list. If it's large, you will need a current OpenJDK user to host it for you. At least that's how I understand it. Hopefully someone who knows the process better can chime in here.
>
> I should also let you know that getting this into JDK 9 is most likely not going to happen. AFAIK we are only doing security updates for 9. It would have to go into the currently active release. I should also warn you that new ports generally need a certain amount of backing to be accepted. It may be that this would have to live in a porting side project. Hopefully someone who knows this better can chime in here as well.
>
> /Erik
>
> P.
>  
> Sent from Mail for Windows 10
>  
> From: Magnus Ihse Bursie
> Sent: Tuesday, October 10, 2017 10:04 AM
> To: Peter Budai; Erik Joelsson; [hidden email]
> Subject: Re: Building OpenJDK9 on MSYS2
>  
> On 2017-10-07 10:14, Peter Budai wrote:
> The configure of OpenJDK overwrites the SHELL. Actually it is using bash, but for the arguments it was using “-e -o pipefail”. I have figured that for MSYS2 bash what is needed as bash arguments is “-e -l -c -o pipefail”
>  
> That looks like solving this problem, and now the real issues are surfacing.
>
> FWIW, "-l" makes bash behave like a login shell. Most likely you are changing bash's behavior in one of your login scripts, and that change is what's really needed.
>
> /Magnus
>
>
>
>  
> Peter
>  
> From: Peter Budai
> Sent: Friday, October 6, 2017 6:43 PM
> To: Magnus Ihse Bursie; Erik Joelsson; [hidden email]
> Subject: RE: Building OpenJDK9 on MSYS2
>  
> Magnus,
>  
> I have followed your suggestion and removed the fixpath prefixes from gcc related compile tools, and left only the fixpath prefix _only_ for the Boot JDK related tools in place.
>  
> 1)      As  I follow the process, all java and javac related compile steps are running properly
> 2)      When the process reaches gcc related steps I got the error message at the same place as before (no fixpath). If I execute that command from the bash prompt, it creates the output:
> $ ( /usr/bin/gawk '/@@END_COPYRIGHT@@/{exit}1' /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template && /C/msys64/mingw64/bin/gcc -E -x c  /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template 2> >(/usr/bin/grep -v '^SocketOptionRegistry.java.template$' >&2) | /usr/bin/gawk '/@@START_HERE@@/,0' |  /usr/bin/sed -e 's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED FILE - DO NOT EDIT/' -e 's/PREFIX_//' -e 's/^#.*//' ) > /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java
> As I have mentioned the parameters are replaced by the bash automatically
> 3)      Then build continues, then little later stops at a super simple command:
> mv /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBuffer.java.tmp /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBuffer.java
>            Needless to say, the ByteBuffer.java.tmp file DOES exist. And running the above command from the bash works, and build continues.
> 4)      A few similar cases (stops) with DirectByteBuffer and DirectByteBufferR
>  
> Currently I try to explore how that might relate to the MSYS2 bash and make, somehow it behaves differently
>  
> If you have any other suggestion, let me know.
>  
> Best regards,
>  
> Peter
>  
> From: Peter Budai
> Sent: Thursday, October 5, 2017 3:52 PM
> To: Magnus Ihse Bursie; Erik Joelsson; [hidden email]
> Subject: RE: Building OpenJDK9 on MSYS2
>  
> Hi Magnus,
>  
> So first of all, here is the current patch, which I was not able to attach: https://pastebin.com/pwT4Ynxc
>  
> >> That's surprising, since gcc is prefixed with fixpath, which it should not.
> Actually you DO need fixpath IMHO.
> This is a mingw64 version of the gcc (/C/msys64/mingw64/bin/gcc), which is a fully functional Windows executable, which expects Windows formatted path arguments.
> As the updated build process uses EXPORT MSYS2_ARG_CONV_EXCL=* (see that patch), none of the command line arguments are converted  from the unix path to Windows, but fixpath does that conversion. There is a wiki describing more details on this:
> https://github.com/msys2/msys2/wiki/Porting#user-content-filesystem-namespaces
>  
>  
>
> >>I have a hard time believing this is a race condition. On the other hand, this stuff is weird, we're misusing the C preprocessor to process defines in java code, so I'm not surprised it breaks down.
> >>I don't know why it succeeded when run on the command line, though.
> When I execute that command from the bash command line there is no EXPORT MSYS2_ARG_CONV_EXCL, but the bash itself does the automatic conversion of the arguments. Maybe it has something to do how fixpath does CreateProcess?
>  
> Does that help?
>  
> Best regards,
>  
> Peter
>  
> Sent from Mail for Windows 10
>  
> From: Magnus Ihse Bursie
> Sent: Thursday, October 5, 2017 12:13 PM
> To: Peter Budai; Erik Joelsson; [hidden email]
> Subject: Re: Building OpenJDK9 on MSYS2
>  
>  
> On 2017-10-05 11:59, Peter Budai wrote:
> Hi Magnus and Erik,
>  
> I really appreciate your quick feedback. I assumed that it won’t be easy, but I just don’t feel I should give up now  - maybe later when I see the real scale of work. So bear with me for a time being.
>  
> Attached is a patch which already includes Magnus’ changes, plus a few which I have added:
> ·       basically enabling gcc for windows,
> ·       and modifying a logic for compiling fixpath (before that it was using hard-coded MS VSC compile flags)
>
> Actually, you must make sure fixpath is *not* used for the toolchain, since gcc uses unix style paths.
> (However, other tools such as java will still need it.)
>
>
>
>
>  
>  
> So here is what I have as the result of configure:
> ====================================================
> The existing configuration has been successfully updated in
> /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release
> using configure arguments '--disable-freetype-bundling --disable-javac-server'.
>  
> Configuration summary:
> * Debug level:    release
> * HS debug level: product
> * JDK variant:    normal
> * JVM variants:   server
> * OpenJDK target: OS: windows, CPU architecture: x86, address length: 64
> * Version string: 9-internal+0-adhoc.peterbud.jdk9 (9-internal)
>  
> Tools summary:
> * Environment:    msys version 2.9.0(0.318/5/3) (root at /C/msys64)
> * Boot JDK:       java version "1.8.0_144"  Java(TM) SE Runtime Environment (build 1.8.0_144-b01)  Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)   (at /c/progra~1/java/jdk18~1.0_1)
> * Toolchain:      gcc (GNU Compiler Collection)
> * C Compiler:     Version 7.2.0 (at /C/msys64/mingw64/bin/gcc)
> * C++ Compiler:   Version 7.2.0 (at /c/msys64/mingw64/bin/g++)
>  
> Build performance summary:
> * Cores to use:   4
> * Memory limit:   16216 MB
>  
> Its clear says that the toolchain is gcc 7.2 (BTW there is no Visual Studio on this machine)
>  
> Now for the details of the config log, you can see here: https://pastebin.com/MN2ZYcHH
>  
> And about the build process and the error I get:
>  
> $ make JOBS=1
> Building target 'default (exploded-image)' in configuration 'windows-x86_64-normal-server-release'
> Compiling 8 files for BUILD_TOOLS_LANGTOOLS
> Compiling 17 properties into resource bundles for jdk.compiler
> Parsing 1 properties into enum-like class for jdk.compiler
> Compiling 19 properties into resource bundles for jdk.javadoc
> Compiling 12 properties into resource bundles for jdk.jdeps
> Compiling 7 properties into resource bundles for jdk.jshell
> Compiling 117 files for BUILD_INTERIM_java.compiler
> Compiling 396 files for BUILD_INTERIM_jdk.compiler
> Compiling 61 files for BUILD_INTERIM_jdk.jdeps
> Compiling 457 files for BUILD_INTERIM_jdk.javadoc
> Note: Some input files use or override a deprecated API.
> Note: Recompile with -Xlint:deprecation for details.
> Compiling 159 files for BUILD_TOOLS_JDK
> Note: Some input files use unchecked or unsafe operations.
> Note: Recompile with -Xlint:unchecked for details.
> make[3]: *** [GensrcMisc.gmk:78: /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java] Error 1
> make[3]: *** Deleting file '/C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java'
> make[2]: *** [make/Main.gmk:115: java.base-gensrc-jdk] Error 2
>  
> ERROR: Build failed for target 'default (exploded-image)' in configuration 'windows-x86_64-normal-server-release' (exit code 2)
>  
> No indication of failed target found.
> Hint: Try searching the build log for '] Error'.
> Hint: See common/doc/building.html#troubleshooting for assistance.
>  
> make[1]: *** [/home/peterbud/jdk9/make/Init.gmk:296: main] Error 2
> make: *** [/home/peterbud/jdk9/make/Init.gmk:185: default] Error 2
>  
> If I run here
> make JOBS=1 LOG=debug
> The failing line seems to be this:
>  
> ( /usr/bin/gawk '/@@END_COPYRIGHT@@/{exit}1' /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template && /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/configure-support/bin/fixpath.exe -m/C/msys64/@/c/msys64/@/c/progra~ /C/msys64/mingw64/bin/gcc -E -x c  /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template 2> >(/usr/bin/grep -v '^SocketOptionRegistry.java.template$' >&2) | /usr/bin/gawk '/@@START_HERE@@/,0' |  /usr/bin/sed -e 's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED FILE - DO NOT EDIT/' -e 's/PREFIX_//' -e 's/^#.*//' ) > /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java
> make[3]: *** [GensrcMisc.gmk:78: /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java] Error 1
>  
> Now the interesting is: if I copy this line above to the bash prompt, it runs without problem, and the file support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java
> That's surprising, since gcc is prefixed with fixpath, which it should not.
>
> I have a hard time believing this is a race condition. On the other hand, this stuff is weird, we're misusing the C preprocessor to process defines in java code, so I'm not suprised it breaks down. I don't know why it succeeded when run on the command line, though. My suggestion is to just do some quick and dirty hack around this: take the file you manage to generate and just copy it in during the build instead. If you can get round this, you might start seeing some *real* problems. :-)
>
> Also, my suggestion is that you try running "make hotspot" to cut to the chase. Compiling hotspot will likely be the hardest thing. Or even "make -k hotspot" to get an assessment of the amount of work ahead of you.
>
> /Magnus
>
> Is produced.
>  
> Then I can again issue
> make JOBS=1 LOG=debug
>  
> And the compile process is being continued, until a similar error pops up with a different generated file. I have an assumption that this happens because make is still running parallel jobs, despite JOBS=1 but I’m not sure.
>  
> How could I best tackle this?
>  
> Thank you and best regards,
>  
> Peter
>  
> Sent from Mail for Windows 10
>  
> From: Magnus Ihse Bursie
> Sent: Thursday, October 5, 2017 11:33 AM
> To: Erik Joelsson; Peter Budai; [hidden email]
> Subject: Re: Building OpenJDK9 on MSYS2
>  
> On 2017-10-05 10:10, Erik Joelsson wrote:
> > Hello Peter,
> >
> >
> > On 2017-10-04 21:15, Peter Budai wrote:
> >> Hi Magnus,
> >>
> >> Thanks for the quick reply I’ll check these patches with msys2.
> >>
> >> Let me specify with more details what I’d like to achieve: I’d like
> >> to build OpenJDK9 with MSYS2 MINGW64 environment using gcc toolchain.
> >> (I’m not sure how familiar are you with MSYS2, but there are 3
> >> different environments: MSYS2, MINGW32 and MINGW64). In theory
> >> MINGW64 with gcc is the closes you can get on Windows platform as a
> >> gcc unix like build environment, which produces still a native 64-bit
> >> executable on Windows.
> >>
> >> I’m not very familiar with OpenJDK yet, so therefore I’d like to hear
> >> your opinion: how realistic is that?
> > Sorry to disappoint, but I would say that requires major work. There
> > is a strong historic assumption that windows builds are done using
> > Visual Studio. We have abstracted away some of it in configure (see
> > TOOLCHAIN_TYPE), but it's very far from enough to change compiler
> > environment for a Windows build. The native sources are also bound to
> > make a lot of such assumptions. I would expect the changes needed to
> > be in the thousands of lines of code.
>
> I agree that it requires hard work (even if "thousands" might be an
> overestimation I think, but "hundreds" is not enough, so it's the right
> magnitude). On the other hand, it would be really good if we did sort
> things out, so that we had proper conditions based on OS vs
> compiler/toolchain.
>
> If you really want to start, the first thing is to patch toolchain.m4 to
> VALID_TOOLCHAINS_windows="microsoft gcc"
> and then call configure using "bash configure --with-toolchain-type=gcc".
>
> As Erik, I doubt you will come very far before things starts tumbling down.
>
> >
> > When we say supporting the build in msys2 instead of cygwin, we just
> > mean using msys2 as the unix emulating layer for our tools like
> > make/bash/grep/sed etc.
> >
> > One think I have done successfully is running the build in WSL
> > (Windows Subsystem for Linux), but that isn't all that helpful as WSL
> > for practical purposes is more or less like running Linux in a VM, so
> > the build sees a Linux system and builds a Linux binary.
> >> As a side note: with MINGW64 I have managed to run configure phase
> >> successfully for OpenJDK. The compile process has also started and
> >> went for a while, but interestingly I run into some kind of race
> >> conditions as make stopped with an error. Using LOG=debug I have fond
> >> the failing line and then copying the failed command and pasting it
> >> to the bash prompt it successfully generated the output target, and
> >> then the build process run further when a similar situation happened.
> >> Also pasting the failed command run in the bash without any problem,
> >> and build continued… until the next.
> > Without seeing the errors I can't say much. I very much doubt that you
> > are running with gcc as the compiler though. Configure isn't easily
> > fooled into using a different compiler to what it prefers, and I would
> > expect things to crash and burn pretty early if you actually did.
> >
> > /Erik
> >>
> >> I have tried to run make JOBS=1, but did not help, strangely I have
> >> still seen in the log make[3] and make[4] logs which suggested that
> >> there are more than one make jobs were running. Also tried .configure
> >> --with-output-sync=recurse without success (same symptoms)
> >>
> >> Let me know your thoughts.
> >>
> >> Best regards,
> >>
> >> Peter
> >>
> >> Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for
> >> Windows 10
> >>
> >> From: Magnus Ihse Bursie<mailto:[hidden email]>
> >> Sent: Wednesday, October 4, 2017 1:04 AM
> >> To: Peter Budai<mailto:[hidden email]>;
> >> [hidden email]<mailto:[hidden email]>
> >> Subject: Re: Building OpenJDK9 on MSYS2
> >>
> >> Actually, it wasn't so much remaining trouble. :-) I fired up msys2 and
> >> checked out where I left off. It turned out that the remaining snag was
> >> that msys2 tries to convert command lines automatically, from "unix"
> >> style paths to "windows" style paths. Unfortunately, it does not do this
> >> very well and it breaks all sorts of things. We already have a FIXPATH
> >> solution in place which deals with this, so basically all I had to do
> >> was disable this (by setting MSYS2_ARG_CONV_EXCL to "*"). However, this
> >> broke our cygpath replacement hack (!) so I had to disable it there.
> >> Sigh. Anyway, with those fixes it ran and worked well. (I also
> >> discovered and fixed a bug related to how we set up the FIXPATH variable
> >> on msys, but it only triggers in certain circumstances).
> >>
> >> With this patch I now jdk9 seems to build fine on msys2. It should apply
> >> cleanly on jdk9/jdk9. Since it turned out to be so trivial, I'll try to
> >> get it in in jdk10.
> >>
> >> Here's the patch if you want to apply it yourself:
> >>
> >> diff -r a08cbfc0e4ec common/autoconf/basics_windows.m4
> >> --- a/common/autoconf/basics_windows.m4    Thu Aug 03 18:56:56 2017
> >> +0000
> >> +++ b/common/autoconf/basics_windows.m4    Wed Oct 04 00:53:58 2017
> >> +0200
> >> @@ -42,7 +42,7 @@
> >>        windows_path=`$CYGPATH -m "$unix_path"`
> >>        $1="$windows_path"
> >>      elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then
> >> -    windows_path=`cmd //c echo $unix_path`
> >> +    windows_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $unix_path`
> >>        $1="$windows_path"
> >>      fi
> >>    ])
> >> @@ -136,6 +136,16 @@
> >>      fi
> >>    ])
> >>
> >> +AC_DEFUN([BASIC_MSYS_UPDATE_FIXPATH],
> >> +[
> >> +  # Take all collected prefixes and turn them into a
> >> -m/c/foo@/c/bar@... command line
> >> +  # @ was chosen as separator to minimize risk of other tools messing
> >> around with it
> >> +  all_unique_prefixes=`echo "${all_fixpath_prefixes@<:@@@:>@}" \
> >> +      | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
> >> +  fixpath_argument_list=`echo $all_unique_prefixes  | tr ' ' '@'`
> >> +  FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
> >> +])
> >> +
> >>    AC_DEFUN([BASIC_FIXUP_PATH_MSYS],
> >>    [
> >>      path="[$]$1"
> >> @@ -143,7 +153,7 @@
> >>      new_path="$path"
> >>      if test "x$has_colon" = x; then
> >>        # Not in mixed or Windows style, start by that.
> >> -    new_path=`cmd //c echo $path`
> >> +    new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $path`
> >>      fi
> >>
> >>      BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
> >> @@ -155,6 +165,8 @@
> >>
> >>      # Save the first 10 bytes of this path to the storage, so fixpath
> >> can work.
> >> all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
> >> "${new_path:0:10}")
> >> +  # We might need to re-evaluate FIXPATH.
> >> +  BASIC_MSYS_UPDATE_FIXPATH
> >>    ])
> >>
> >>    AC_DEFUN([BASIC_FIXUP_EXECUTABLE_CYGWIN],
> >> @@ -293,7 +305,7 @@
> >>        # Do not save /bin paths to all_fixpath_prefixes!
> >>      else
> >>        # Not in mixed or Windows style, start by that.
> >> -    new_path=`cmd //c echo $new_path`
> >> +    new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $new_path`
> >>        BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
> >>        # Output is in $new_path
> >>        BASIC_WINDOWS_REWRITE_AS_UNIX_PATH(new_path)
> >> @@ -302,6 +314,8 @@
> >>
> >>        # Save the first 10 bytes of this path to the storage, so fixpath
> >> can work.
> >> all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
> >> "${new_path:0:10}")
> >> +    # We might need to re-evaluate FIXPATH.
> >> +    BASIC_MSYS_UPDATE_FIXPATH
> >>      fi
> >>    ])
> >>
> >> @@ -347,6 +361,10 @@
> >>        WINDOWS_ENV_VENDOR='msys'
> >>        WINDOWS_ENV_VERSION="$MSYS_VERSION"
> >>
> >> +    # Prohibit msys2 path conversion from trying to be "intelligent",
> >> and rely
> >> +    # on fixpath instead.
> >> +    export MSYS2_ARG_CONV_EXCL="*"
> >> +
> >>        AC_MSG_CHECKING([msys root directory as unix-style path])
> >>        # The cmd output ends with Windows line endings (CR/LF), the grep
> >> command will strip that away
> >>        MSYS_ROOT_PATH=`cd / ; cmd /c cd | $GREP ".*"`
> >> @@ -391,10 +409,7 @@
> >>        elif test "x$OPENJDK_BUILD_OS_ENV" = xwindows.msys; then
> >>          # Take all collected prefixes and turn them into a
> >> -m/c/foo@/c/bar@... command line
> >>          # @ was chosen as separator to minimize risk of other tools
> >> messing around with it
> >> -      all_unique_prefixes=`echo "${all_fixpath_prefixes@<:@@@:>@}" \
> >> -          | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
> >> -      fixpath_argument_list=`echo $all_unique_prefixes  | tr ' ' '@'`
> >> -      FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
> >> +      BASIC_MSYS_UPDATE_FIXPATH
> >>        fi
> >>        FIXPATH_SRC_W="$FIXPATH_SRC"
> >>        FIXPATH_BIN_W="$FIXPATH_BIN"
> >> diff -r a08cbfc0e4ec common/autoconf/build-aux/config.sub
> >> --- a/common/autoconf/build-aux/config.sub    Thu Aug 03 18:56:56
> >> 2017 +0000
> >> +++ b/common/autoconf/build-aux/config.sub    Wed Oct 04 00:53:58
> >> 2017 +0200
> >> @@ -30,7 +30,7 @@
> >>    DIR=`dirname $0`
> >>
> >>    # First, filter out everything that doesn't begin with "aarch64-"
> >> -if ! echo $* | grep '^aarch64-' >/dev/null ; then
> >> +if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ; then
> >>        . $DIR/autoconf-config.sub "$@"
> >>        # autoconf-config.sub exits, so we never reach here, but just in
> >>        # case we do:
> >> @@ -38,13 +38,17 @@
> >>    fi
> >>
> >>    while test $# -gt 0 ; do
> >> -    case $1 in
> >> +    case $1 in
> >>            -- )   # Stop option processing
> >>                shift; break ;;
> >>            aarch64-* )
> >>                config=`echo $1 | sed 's/^aarch64-/arm-/'`
> >>                sub_args="$sub_args $config"
> >>                shift; ;;
> >> +        *-msys )
> >> +            config=`echo $1 | sed 's/msys/mingw32/'`
> >> +            sub_args="$sub_args $config"
> >> +            shift; ;;
> >>            - )    # Use stdin as input.
> >>                sub_args="$sub_args $1"
> >>                shift; break ;;
> >> diff -r a08cbfc0e4ec common/autoconf/spec.gmk.in
> >> --- a/common/autoconf/spec.gmk.in    Thu Aug 03 18:56:56 2017 +0000
> >> +++ b/common/autoconf/spec.gmk.in    Wed Oct 04 00:53:58 2017 +0200
> >> @@ -120,6 +120,13 @@
> >>      # On Windows, the Visual Studio toolchain needs the PATH to be
> >> adjusted
> >>      # to include Visual Studio tools (this needs to be in cygwin/msys
> >> style).
> >>      export PATH:=@VS_PATH@
> >> +
> >> +endif
> >> +
> >> +ifeq ($(OPENJDK_TARGET_OS_ENV), windows.msys)
> >> +  # On msys2, prohibit msys path conversion from trying to be
> >> +  # "intelligent", and rely on fixpath instead.
> >> +  export MSYS2_ARG_CONV_EXCL:=*
> >>    endif
> >>
> >>    SYSROOT_CFLAGS := @SYSROOT_CFLAGS@
> >>
> >> /Magnus
> >>
> >> On 2017-10-03 22:34, Magnus Ihse Bursie wrote:
> >>> I gave msys2 a shot some time ago, but it ended up too much trouble.
> >>> I'll share some of my notes from that attempt, for what it's worth.
> >>>
> >>> To install package X/Y, run "pacman -S X/Y". Missing tools and
> >>> packages where to find them:
> >>> cmp: msys/diffutils
> >>> tar: msys/tar
> >>> make: msys/make
> >>> unzip: msys/unzip
> >>> zip: msys/zip
> >>>
> >>> config.sub reports msys as "x86_64-pc-mingw32" but msys2 as
> >>> "x86_64-pc-msys". This patch adds postprocessing in "our" config.sub
> >>> to report msys2 similar to msys. (Opinions, including my own :-) may
> >>> vary if this really is the best way..)
> >>>
> >>> diff -r b88023f46daa common/autoconf/build-aux/config.sub
> >>> --- a/common/autoconf/build-aux/config.sub      Fri Jan 27 10:15:41
> >>> 2017 +0100
> >>> +++ b/common/autoconf/build-aux/config.sub      Fri Feb 03 05:00:25
> >>> 2017 -0700
> >>> @@ -30,7 +30,7 @@
> >>>   DIR=`dirname $0`
> >>>
> >>>   # First, filter out everything that doesn't begin with "aarch64-"
> >>> -if ! echo $* | grep '^aarch64-' >/dev/null ; then
> >>> +if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ; then
> >>>       . $DIR/autoconf-config.sub "$@"
> >>>       # autoconf-config.sub exits, so we never reach here, but just in
> >>>       # case we do:
> >>> @@ -45,6 +45,10 @@
> >>>               config=`echo $1 | sed 's/^aarch64-/arm-/'`
> >>>               sub_args="$sub_args $config"
> >>>               shift; ;;
> >>> +        *-msys )
> >>> +            config=`echo $1 | sed 's/msys/mingw32/'`
> >>> +            sub_args="$sub_args $config"
> >>> +            shift; ;;
> >>>           - )    # Use stdin as input.
> >>>               sub_args="$sub_args $1"
> >>>               shift; break ;;
> >>>
> >>> If I remember correctly, this got me past the configure stage at the
> >>> time.
> >>>
> >>> I don't think it's very hard to get it to work on msys2, I just ran
> >>> into one snag too many and didn't think msys2 would be used by anyone.
> >>>
> >>> /Magnus
> >>>
> >>> On 2017-10-03 17:20, Peter Budai wrote:
> >>>> Hello,
> >>>>
> >>>> According to
> >>>> http://hg.openjdk.java.net/jdk9/jdk9/file/a08cbfc0e4ec/common/doc/building.html 
> >>>>
> >>>> “msys2 and the new Windows Subsystem for Linux (WSL) would likely be
> >>>> possible to support in a future version but that would require a
> >>>> community effort to implement”
> >>>>
> >>>> I’d like to help making the OpenJDK 9 build working on msys2. What is
> >>>> the best way to move forward? Is there a similar effort in progress?
> >>>>
> >>>> Thank you and best regards,
> >>>>
> >>>> Peter
> >>>>
> >>>>
> >
>
>  
>  
>  
>  
>  
>  
>  
>  
>  
Reply | Threaded
Open this post in threaded view
|

Re: Building OpenJDK9 on MSYS2

Erik Joelsson
Hello,

On 2017-10-11 15:48, Magnus Ihse Bursie wrote:
> For gcc, we let the compiler generate the .d file. For the Microsoft
> tool chain, we use a clever sed script to extract and create it ourself.
>
> I think that logic is checking for "Windows", not "Microsoft". That
> might be your cause of trouble.
>
> Look in NativeCompilation.gmk.
>
That was my initial thought as well, but we do correctly check  for
microsoft. Also it's not the .d files that are the problem. As Peter
just wrote, they look fine. It's the .d.target files which we create
using the same technique on all platforms. What we don't account for is
the compiler putting Windows mixed paths in the .d files.

> /Magnus
>
> 11 okt. 2017 kl. 14:43 skrev Peter Budai <[hidden email]
> <mailto:[hidden email]>>:
>
>> Hi Erik,
>>
>> The .d file looks like this:
>>
>> C:/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/hotspot/variant-server/tools/adlc/objs/adlparse.obj:
>> \
>>
>> C:/msys64/home/peterbud/jdk9/hotspot/src/share/vm/adlc/adlparse.cpp \
>>
>> I have checked .d.targets file, and looks like it has the first line
>> has not been deleted, and the file names below are also wrong:
>>
>> /msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/hotspot/variant-server/tools/adlc/objs/adlparse.obj
>> : :
>>
>> /msys64/home/peterbud/jdk9/hotspot/src/share/vm/adlc/adlparse.cpp :
>>
>> /msys64/home/peterbud/jdk9/hotspot/src/share/vm/adlc/adlc.hpp :
>>
>> I guess this part in the DEPENDENCY_TARGET_SED_PATTERN is fooled by
>> the “C:/”
>>
>> -e 's/^[^:]*: *//'
>>
Yes, that does indeed look like the problem. I suppose the regexp is
unnecessarily strict. It should be ok to rewrite it as something like this:
-e 's/^.*: *//'

Basically just make sure it ends with : and any number of spaces.

/Erik

>> Peter
>>
>> Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for
>> Windows 10
>>
>> *From: *Erik Joelsson <mailto:[hidden email]>
>> *Sent: *Wednesday, October 11, 2017 12:16 PM
>> *To: *Peter Budai <mailto:[hidden email]>; Magnus Ihse Bursie
>> <mailto:[hidden email]>; [hidden email]
>> <mailto:[hidden email]>
>> *Subject: *Re: Building OpenJDK9 on MSYS2
>>
>> Hello Peter,
>>
>> On 2017-10-11 00:18, Peter Budai wrote:
>>
>>     Thanks Magnus & Erik
>>
>>     First thanks for your support and kind words!
>>
>>     Magnus, I have checked .bash_profile, .bashrc but they seem to be
>>     empty (everything is commented out). You can check with a default
>>     MSYS2 install, I have not changed these files at all. If you find
>>     thee something specific I can give a try here as well.
>>
>>     Let me give also a quick status update, where am I with building
>>     hotspot:
>>
>>     ·I guess its still the beginning, but I have managed to compile
>>     jvm.dll with almost 700 object file: with debug info the dll is
>>     around 700 MB 😊
>>
>>     ·I made only surgical, minimal changes to the source, and so far
>>     it looks reasonable. I have encountered 3 scenarios where changes
>>     were necessary:
>>
>>     oWhen in makefiles conditionals were using assuming that if
>>     target_os is windows then it is visual studio compiler/linker.
>>     Obviously these conditionals had to be reviewed in a few places
>>     and if necessary were changes to check the toolchain=Microsoft
>>
>> These are not surprising and should be pretty straight forward to fix
>> and it seems you know what to do.
>>
>>     ·
>>
>>     oI got a few warnings as gcc 7.2 uncovered some code problems in
>>     windows specific codes, where before that MSVC I guess did not
>>     say a word…
>>
>> To get around this you can configure with
>> --disable-warnings-as-errors until you get things working properly.
>> This is commonly needed when using compiler versions that we normally
>> don't use.
>>
>>     ·
>>
>>     oAnd I had like 6-7 places where the code was using MSVC specific
>>     __try … __except structures which gcc does not know. Do you have
>>     a suggestion how to approach them? I can do ugly #ifdefs (I would
>>     avoid that) but I have also seen some solutions to replace them
>>     with a code which gcc can compile
>>     (http://www.programmingunlimited.net/siteexec/content.cgi?page=mingw-seh
>>     ) – but before doing that  though I would ask first you on the
>>     purpose of those
>>
>> This kind of question is probably best to bring to the hotspot
>> mailing list.
>>
>>     ·What bothers me is that I was not able to do incremental builds:
>>     when an error occurs, and build stops, then after making change
>>     in the CPP source the build cannot continue, I always got an
>>     error message:
>>
>>     |/C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/hotspot/variant-server/tools/adlc/objs/adlparse.d.targets:1:
>>     *** missing target pattern.  Stop.|
>>     |make[2]: *** [make/Main.gmk:256: hotspot-server-gensrc] Error 2|
>>
>>     If I do a ‘make clean’ and restart the build then it nicely compiles.
>>
>>     Question 1: Is there a way to  resume such builds without ‘make
>>     clean’?
>>
>> Well, incremental builds is supposed to work well. We have several
>> extra tricks in there to handle cases where normal make builds would
>> fail. The *.d.targets files is one such trick and it seems to
>> backfire for you. The contents of that file should be something like:
>>
>> /localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/adlparse.cpp :
>> /localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/adlc.hpp :
>> /localhome/hg/jdk9-dev/hotspot/src/share/vm/opto/opcodes.hpp :
>> /localhome/hg/jdk9-dev/hotspot/src/share/vm/opto/classes.hpp :
>> /localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/arena.hpp :
>> /localhome/hg/jdk9-dev/hotspot/src/share/vm/opto/adlcVMDeps.hpp :
>> /localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/filebuff.hpp :
>> /localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/dict2.hpp :
>> /localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/forms.hpp :
>> /localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/formsopt.hpp :
>> /localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/formssel.hpp :
>> /localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/archDesc.hpp :
>> /localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/adlparse.hpp :
>>
>> Basically an empty rule for each dependency for the corresponding
>> object file. Declaring these rules makes it possible to delete source
>> files without having to build clean. It seems your file is not
>> generated correctly so please have a look inside it. The file is in
>> make/common/NativeCompilation.gmk, look for
>> DEPENDENCY_TARGET_SED_PATTERN.
>>
>>     Question 2: What would be the best way to submit/share the
>>     patches for your thorough review?
>>
>> Well, first of all, have you signed the OCA?
>>
>> As for publishing patches and reviews, there is a bit of chicken and
>> egg problem. Once you become an "author" in any of the OpenJDK
>> projects, you get a user name and should be able to publish reviews
>> on cr.openjdk.java.net <http://cr.openjdk.java.net>. Before that, if
>> the patch is small, it can be posted inline in an email to the list.
>> If it's large, you will need a current OpenJDK user to host it for
>> you. At least that's how I understand it. Hopefully someone who knows
>> the process better can chime in here.
>>
>> I should also let you know that getting this into JDK 9 is most
>> likely not going to happen. AFAIK we are only doing security updates
>> for 9. It would have to go into the currently active release. I
>> should also warn you that new ports generally need a certain amount
>> of backing to be accepted. It may be that this would have to live in
>> a porting side project. Hopefully someone who knows this better can
>> chime in here as well.
>>
>> /Erik
>>
>>     P.
>>
>>     Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986>
>>     for Windows 10
>>
>>     *From: *Magnus Ihse Bursie <mailto:[hidden email]>
>>     *Sent: *Tuesday, October 10, 2017 10:04 AM
>>     *To: *Peter Budai <mailto:[hidden email]>; Erik Joelsson
>>     <mailto:[hidden email]>; [hidden email]
>>     <mailto:[hidden email]>
>>     *Subject: *Re: Building OpenJDK9 on MSYS2
>>
>>     On 2017-10-07 10:14, Peter Budai wrote:
>>
>>         The configure of OpenJDK overwrites the SHELL. Actually it is
>>         using bash, but for the arguments it was using “-e -o
>>         pipefail”. I have figured that for MSYS2 bash what is needed
>>         as bash arguments is “-e -l -c -o pipefail”
>>
>>         That looks like solving this problem, and now the real issues
>>         are surfacing.
>>
>>
>>     FWIW, "-l" makes bash behave like a login shell. Most likely you
>>     are changing bash's behavior in one of your login scripts, and
>>     that change is what's really needed.
>>
>>     /Magnus
>>
>>
>>
>>         Peter
>>
>>         *From: *Peter Budai <mailto:[hidden email]>
>>         *Sent: *Friday, October 6, 2017 6:43 PM
>>         *To: *Magnus Ihse Bursie
>>         <mailto:[hidden email]>; Erik Joelsson
>>         <mailto:[hidden email]>; [hidden email]
>>         <mailto:[hidden email]>
>>         *Subject: *RE: Building OpenJDK9 on MSYS2
>>
>>         Magnus,
>>
>>         I have followed your suggestion and removed the fixpath
>>         prefixes from gcc related compile tools, and left only the
>>         fixpath prefix _only_ for the Boot JDK related tools in place.
>>
>>         1)As  I follow the process, all java and javac related
>>         compile steps are running properly
>>
>>         2)When the process reaches gcc related steps I got the error
>>         message at the same place as before (no fixpath). If I
>>         execute that command from the bash prompt, it creates the
>>         output:
>>
>>         $ ( /usr/bin/gawk '/@@END_COPYRIGHT@@/{exit}1'
>>         /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template
>>         && /C/msys64/mingw64/bin/gcc -E -x c
>>         /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template
>>         2> >(/usr/bin/grep -v '^SocketOptionRegistry.java.template$'
>>         >&2) | /usr/bin/gawk '/@@START_HERE@@/,0' |  /usr/bin/sed -e
>>         's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED FILE - DO NOT
>>         EDIT/' -e 's/PREFIX_//' -e 's/^#.*//' ) >
>>         /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java
>>
>>         As I have mentioned the parameters are replaced by the bash
>>         automatically
>>
>>         3)Then build continues, then little later stops at a super
>>         simple command:
>>
>>         mv
>>         /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBuffer.java.tmp
>>         /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBuffer.java
>>
>>         Needless to say, the ByteBuffer.java.tmp file DOES exist. And
>>         running the above command from the bash works, and build
>>         continues.
>>
>>         4)A few similar cases (stops) with DirectByteBuffer and
>>         DirectByteBufferR
>>
>>         Currently I try to explore how that might relate to the MSYS2
>>         bash and make, somehow it behaves differently
>>
>>         If you have any other suggestion, let me know.
>>
>>         Best regards,
>>
>>         Peter
>>
>>         *From: *Peter Budai <mailto:[hidden email]>
>>         *Sent: *Thursday, October 5, 2017 3:52 PM
>>         *To: *Magnus Ihse Bursie
>>         <mailto:[hidden email]>; Erik Joelsson
>>         <mailto:[hidden email]>; [hidden email]
>>         <mailto:[hidden email]>
>>         *Subject: *RE: Building OpenJDK9 on MSYS2
>>
>>         Hi Magnus,
>>
>>         So first of all, here is the current patch, which I was not
>>         able to attach: https://pastebin.com/pwT4Ynxc
>>
>>         >> That's surprising, since gcc is prefixed with fixpath, which
>>         it should not.
>>
>>         Actually you DO need fixpath IMHO.
>>
>>         This is a mingw64 version of the gcc
>>         (/C/msys64/mingw64/bin/gcc), which is a fully functional
>>         Windows executable, /which expects Windows formatted path
>>         arguments/.
>>
>>         As the updated build process uses EXPORT
>>         MSYS2_ARG_CONV_EXCL=* (see that patch), none of the command
>>         line arguments are converted  from the unix path to Windows,
>>         but fixpath does that conversion. There is a wiki describing
>>         more details on this:
>>
>>         https://github.com/msys2/msys2/wiki/Porting#user-content-filesystem-namespaces
>>
>>
>>         >>I have a hard time believing this is a race condition. On
>>         the other hand, this stuff is weird, we're misusing the C
>>         preprocessor to process defines in java code, so I'm not
>>         surprised it breaks down.
>>
>>         >>I don't know why it succeeded when run on the command line,
>>         though.
>>
>>         When I execute that command from the /bash/ command line
>>         there is no EXPORT MSYS2_ARG_CONV_EXCL, but the bash itself
>>         does the automatic conversion of the arguments. Maybe it has
>>         something to do how fixpath does CreateProcess?
>>
>>         Does that help?
>>
>>         Best regards,
>>
>>         Peter
>>
>>         Sent from Mail
>>         <https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10
>>
>>         *From: *Magnus Ihse Bursie <mailto:[hidden email]>
>>         *Sent: *Thursday, October 5, 2017 12:13 PM
>>         *To: *Peter Budai <mailto:[hidden email]>; Erik
>>         Joelsson <mailto:[hidden email]>;
>>         [hidden email] <mailto:[hidden email]>
>>         *Subject: *Re: Building OpenJDK9 on MSYS2
>>
>>         On 2017-10-05 11:59, Peter Budai wrote:
>>
>>             Hi Magnus and Erik,
>>
>>             I really appreciate your quick feedback. I assumed that
>>             it won’t be easy, but I just don’t feel I should give up
>>             now  - maybe later when I see the real scale of work. So
>>             bear with me for a time being.
>>
>>             Attached is a patch which already includes Magnus’
>>             changes, plus a few which I have added:
>>
>>             ·basically enabling gcc for windows,
>>
>>             ·and modifying a logic for compiling fixpath (before that
>>             it was using hard-coded MS VSC compile flags)
>>
>>
>>         Actually, you must make sure fixpath is *not* used for the
>>         toolchain, since gcc uses unix style paths.
>>         (However, other tools such as java will still need it.)
>>
>>
>>
>>             So here is what I have as the result of configure:
>>
>>             ====================================================
>>
>>             The existing configuration has been successfully updated in
>>
>>             /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release
>>
>>             using configure arguments '--disable-freetype-bundling
>>             --disable-javac-server'.
>>
>>             Configuration summary:
>>
>>             * Debug level:    release
>>
>>             * HS debug level: product
>>
>>             * JDK variant:    normal
>>
>>             * JVM variants:   server
>>
>>             * OpenJDK target: OS: windows, CPU architecture: x86,
>>             address length: 64
>>
>>             * Version string: 9-internal+0-adhoc.peterbud.jdk9
>>             (9-internal)
>>
>>             Tools summary:
>>
>>             * Environment:    msys version 2.9.0(0.318/5/3) (root at
>>             /C/msys64)
>>
>>             * Boot JDK:       java version "1.8.0_144"  Java(TM) SE
>>             Runtime Environment (build 1.8.0_144-b01)  Java
>>             HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed
>>             mode)   (at /c/progra~1/java/jdk18~1.0_1)
>>
>>             * Toolchain:      gcc (GNU Compiler Collection)
>>
>>             * C Compiler:     Version 7.2.0 (at
>>             /C/msys64/mingw64/bin/gcc)
>>
>>             * C++ Compiler:   Version 7.2.0 (at
>>             /c/msys64/mingw64/bin/g++)
>>
>>             Build performance summary:
>>
>>             * Cores to use:   4
>>
>>             * Memory limit:   16216 MB
>>
>>             Its clear says that the toolchain is gcc 7.2 (BTW there
>>             is no Visual Studio on this machine)
>>
>>             Now for the details of the config log, you can see here:
>>             https://pastebin.com/MN2ZYcHH
>>
>>             And about the build process and the error I get:
>>
>>             $ make JOBS=1
>>
>>             Building target 'default (exploded-image)' in
>>             configuration 'windows-x86_64-normal-server-release'
>>
>>             Compiling 8 files for BUILD_TOOLS_LANGTOOLS
>>
>>             Compiling 17 properties into resource bundles for
>>             jdk.compiler
>>
>>             Parsing 1 properties into enum-like class for jdk.compiler
>>
>>             Compiling 19 properties into resource bundles for jdk.javadoc
>>
>>             Compiling 12 properties into resource bundles for jdk.jdeps
>>
>>             Compiling 7 properties into resource bundles for jdk.jshell
>>
>>             Compiling 117 files for BUILD_INTERIM_java.compiler
>>
>>             Compiling 396 files for BUILD_INTERIM_jdk.compiler
>>
>>             Compiling 61 files for BUILD_INTERIM_jdk.jdeps
>>
>>             Compiling 457 files for BUILD_INTERIM_jdk.javadoc
>>
>>             Note: Some input files use or override a deprecated API.
>>
>>             Note: Recompile with -Xlint:deprecation for details.
>>
>>             Compiling 159 files for BUILD_TOOLS_JDK
>>
>>             Note: Some input files use unchecked or unsafe operations.
>>
>>             Note: Recompile with -Xlint:unchecked for details.
>>
>>             make[3]: *** [GensrcMisc.gmk:78:
>>             /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java]
>>             Error 1
>>
>>             make[3]: *** Deleting file
>>             '/C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java'
>>
>>             make[2]: *** [make/Main.gmk:115: java.base-gensrc-jdk]
>>             Error 2
>>
>>             ERROR: Build failed for target 'default (exploded-image)'
>>             in configuration 'windows-x86_64-normal-server-release'
>>             (exit code 2)
>>
>>             No indication of failed target found.
>>
>>             Hint: Try searching the build log for '] Error'.
>>
>>             Hint: See common/doc/building.html#troubleshooting for
>>             assistance.
>>
>>             make[1]: *** [/home/peterbud/jdk9/make/Init.gmk:296:
>>             main] Error 2
>>
>>             make: *** [/home/peterbud/jdk9/make/Init.gmk:185:
>>             default] Error 2
>>
>>             If I run here
>>
>>             make JOBS=1 LOG=debug
>>
>>             The failing line seems to be this:
>>
>>             ( /usr/bin/gawk '/@@END_COPYRIGHT@@/{exit}1'
>>             /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template
>>             &&
>>             /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/configure-support/bin/fixpath.exe
>>             -m/C/msys64/@/c/msys64/@/c/progra~
>>             /C/msys64/mingw64/bin/gcc -E -x c
>>             /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template
>>             2> >(/usr/bin/grep -v
>>             '^SocketOptionRegistry.java.template$' >&2) |
>>             /usr/bin/gawk '/@@START_HERE@@/,0' | /usr/bin/sed -e
>>             's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED FILE - DO
>>             NOT EDIT/' -e 's/PREFIX_//' -e 's/^#.*//' ) >
>>             /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java
>>
>>             make[3]: *** [GensrcMisc.gmk:78:
>>             /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java]
>>             Error 1
>>
>>             Now the interesting is: if I copy this line above to the
>>             bash prompt, it runs without problem, and the file
>>             support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java
>>
>>         That's surprising, since gcc is prefixed with fixpath, which
>>         it should not.
>>
>>         I have a hard time believing this is a race condition. On the
>>         other hand, this stuff is weird, we're misusing the C
>>         preprocessor to process defines in java code, so I'm not
>>         suprised it breaks down. I don't know why it succeeded when
>>         run on the command line, though. My suggestion is to just do
>>         some quick and dirty hack around this: take the file you
>>         manage to generate and just copy it in during the build
>>         instead. If you can get round this, you might start seeing
>>         some *real* problems. :-)
>>
>>         Also, my suggestion is that you try running "make hotspot" to
>>         cut to the chase. Compiling hotspot will likely be the
>>         hardest thing. Or even "make -k hotspot" to get an assessment
>>         of the amount of work ahead of you.
>>
>>         /Magnus
>>
>>             Is produced.
>>
>>             Then I can again issue
>>
>>             make JOBS=1 LOG=debug
>>
>>             And the compile process is being continued, until a
>>             similar error pops up with a different generated file. I
>>             have an assumption that this happens because make is
>>             still running parallel jobs, despite JOBS=1 but I’m not sure.
>>
>>             How could I best tackle this?
>>
>>             Thank you and best regards,
>>
>>             Peter
>>
>>             Sent from Mail
>>             <https://go.microsoft.com/fwlink/?LinkId=550986> for
>>             Windows 10
>>
>>             *From: *Magnus Ihse Bursie
>>             <mailto:[hidden email]>
>>             *Sent: *Thursday, October 5, 2017 11:33 AM
>>             *To: *Erik Joelsson <mailto:[hidden email]>;
>>             Peter Budai <mailto:[hidden email]>;
>>             [hidden email]
>>             <mailto:[hidden email]>
>>             *Subject: *Re: Building OpenJDK9 on MSYS2
>>
>>             On 2017-10-05 10:10, Erik Joelsson wrote:
>>             > Hello Peter,
>>             >
>>             >
>>             > On 2017-10-04 21:15, Peter Budai wrote:
>>             >> Hi Magnus,
>>             >>
>>             >> Thanks for the quick reply I’ll check these patches
>>             with msys2.
>>             >>
>>             >> Let me specify with more details what I’d like to
>>             achieve: I’d like
>>             >> to build OpenJDK9 with MSYS2 MINGW64 environment using
>>             gcc toolchain.
>>             >> (I’m not sure how familiar are you with MSYS2, but
>>             there are 3
>>             >> different environments: MSYS2, MINGW32 and MINGW64).
>>             In theory
>>             >> MINGW64 with gcc is the closes you can get on Windows
>>             platform as a
>>             >> gcc unix like build environment, which produces still
>>             a native 64-bit
>>             >> executable on Windows.
>>             >>
>>             >> I’m not very familiar with OpenJDK yet, so therefore
>>             I’d like to hear
>>             >> your opinion: how realistic is that?
>>             > Sorry to disappoint, but I would say that requires
>>             major work. There
>>             > is a strong historic assumption that windows builds are
>>             done using
>>             > Visual Studio. We have abstracted away some of it in
>>             configure (see
>>             > TOOLCHAIN_TYPE), but it's very far from enough to
>>             change compiler
>>             > environment for a Windows build. The native sources are
>>             also bound to
>>             > make a lot of such assumptions. I would expect the
>>             changes needed to
>>             > be in the thousands of lines of code.
>>
>>             I agree that it requires hard work (even if "thousands"
>>             might be an
>>             overestimation I think, but "hundreds" is not enough, so
>>             it's the right
>>             magnitude). On the other hand, it would be really good if
>>             we did sort
>>             things out, so that we had proper conditions based on OS vs
>>             compiler/toolchain.
>>
>>             If you really want to start, the first thing is to patch
>>             toolchain.m4 to
>>             VALID_TOOLCHAINS_windows="microsoft gcc"
>>             and then call configure using "bash configure
>>             --with-toolchain-type=gcc".
>>
>>             As Erik, I doubt you will come very far before things
>>             starts tumbling down.
>>
>>             >
>>             > When we say supporting the build in msys2 instead of
>>             cygwin, we just
>>             > mean using msys2 as the unix emulating layer for our
>>             tools like
>>             > make/bash/grep/sed etc.
>>             >
>>             > One think I have done successfully is running the build
>>             in WSL
>>             > (Windows Subsystem for Linux), but that isn't all that
>>             helpful as WSL
>>             > for practical purposes is more or less like running
>>             Linux in a VM, so
>>             > the build sees a Linux system and builds a Linux binary.
>>             >> As a side note: with MINGW64 I have managed to run
>>             configure phase
>>             >> successfully for OpenJDK. The compile process has also
>>             started and
>>             >> went for a while, but interestingly I run into some
>>             kind of race
>>             >> conditions as make stopped with an error. Using
>>             LOG=debug I have fond
>>             >> the failing line and then copying the failed command
>>             and pasting it
>>             >> to the bash prompt it successfully generated the
>>             output target, and
>>             >> then the build process run further when a similar
>>             situation happened.
>>             >> Also pasting the failed command run in the bash
>>             without any problem,
>>             >> and build continued… until the next.
>>             > Without seeing the errors I can't say much. I very much
>>             doubt that you
>>             > are running with gcc as the compiler though. Configure
>>             isn't easily
>>             > fooled into using a different compiler to what it
>>             prefers, and I would
>>             > expect things to crash and burn pretty early if you
>>             actually did.
>>             >
>>             > /Erik
>>             >>
>>             >> I have tried to run make JOBS=1, but did not help,
>>             strangely I have
>>             >> still seen in the log make[3] and make[4] logs which
>>             suggested that
>>             >> there are more than one make jobs were running. Also
>>             tried .configure
>>             >> --with-output-sync=recurse without success (same symptoms)
>>             >>
>>             >> Let me know your thoughts.
>>             >>
>>             >> Best regards,
>>             >>
>>             >> Peter
>>             >>
>>             >> Sent from
>>             Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for
>>             >> Windows 10
>>             >>
>>             >> From: Magnus Ihse
>>             Bursie<mailto:[hidden email]>
>>             >> Sent: Wednesday, October 4, 2017 1:04 AM
>>             >> To: Peter Budai<mailto:[hidden email]>;
>>             >> [hidden email]
>>             <mailto:[hidden email]><mailto:[hidden email]>
>>             <mailto:[hidden email]>
>>             >> Subject: Re: Building OpenJDK9 on MSYS2
>>             >>
>>             >> Actually, it wasn't so much remaining trouble. :-) I
>>             fired up msys2 and
>>             >> checked out where I left off. It turned out that the
>>             remaining snag was
>>             >> that msys2 tries to convert command lines
>>             automatically, from "unix"
>>             >> style paths to "windows" style paths. Unfortunately,
>>             it does not do this
>>             >> very well and it breaks all sorts of things. We
>>             already have a FIXPATH
>>             >> solution in place which deals with this, so basically
>>             all I had to do
>>             >> was disable this (by setting MSYS2_ARG_CONV_EXCL to
>>             "*"). However, this
>>             >> broke our cygpath replacement hack (!) so I had to
>>             disable it there.
>>             >> Sigh. Anyway, with those fixes it ran and worked well.
>>             (I also
>>             >> discovered and fixed a bug related to how we set up
>>             the FIXPATH variable
>>             >> on msys, but it only triggers in certain circumstances).
>>             >>
>>             >> With this patch I now jdk9 seems to build fine on
>>             msys2. It should apply
>>             >> cleanly on jdk9/jdk9. Since it turned out to be so
>>             trivial, I'll try to
>>             >> get it in in jdk10.
>>             >>
>>             >> Here's the patch if you want to apply it yourself:
>>             >>
>>             >> diff -r a08cbfc0e4ec common/autoconf/basics_windows.m4
>>             >> --- a/common/autoconf/basics_windows.m4    Thu Aug 03
>>             18:56:56 2017
>>             >> +0000
>>             >> +++ b/common/autoconf/basics_windows.m4    Wed Oct 04
>>             00:53:58 2017
>>             >> +0200
>>             >> @@ -42,7 +42,7 @@
>>             >>        windows_path=`$CYGPATH -m "$unix_path"`
>>             >>        $1="$windows_path"
>>             >>      elif test "x$OPENJDK_BUILD_OS_ENV" =
>>             "xwindows.msys"; then
>>             >> -    windows_path=`cmd //c echo $unix_path`
>>             >> +    windows_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo
>>             $unix_path`
>>             >>        $1="$windows_path"
>>             >>      fi
>>             >>    ])
>>             >> @@ -136,6 +136,16 @@
>>             >>      fi
>>             >>    ])
>>             >>
>>             >> +AC_DEFUN([BASIC_MSYS_UPDATE_FIXPATH],
>>             >> +[
>>             >> +  # Take all collected prefixes and turn them into a
>>             >> -m/c/foo@/c/bar@... command line
>>             >> +  # @ was chosen as separator to minimize risk of
>>             other tools messing
>>             >> around with it
>>             >> +  all_unique_prefixes=`echo
>>             "${all_fixpath_prefixes@<:@@@:>@}" \
>>             >> +      | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
>>             >> +  fixpath_argument_list=`echo $all_unique_prefixes  |
>>             tr ' ' '@'`
>>             >> +  FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
>>             >> +])
>>             >> +
>>             >>    AC_DEFUN([BASIC_FIXUP_PATH_MSYS],
>>             >>    [
>>             >>      path="[$]$1"
>>             >> @@ -143,7 +153,7 @@
>>             >>      new_path="$path"
>>             >>      if test "x$has_colon" = x; then
>>             >>        # Not in mixed or Windows style, start by that.
>>             >> -    new_path=`cmd //c echo $path`
>>             >> +    new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $path`
>>             >>      fi
>>             >>
>>             >> BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
>>             >> @@ -155,6 +165,8 @@
>>             >>
>>             >>      # Save the first 10 bytes of this path to the
>>             storage, so fixpath
>>             >> can work.
>>             >> all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
>>             >> "${new_path:0:10}")
>>             >> +  # We might need to re-evaluate FIXPATH.
>>             >> +  BASIC_MSYS_UPDATE_FIXPATH
>>             >>    ])
>>             >>
>>             >> AC_DEFUN([BASIC_FIXUP_EXECUTABLE_CYGWIN],
>>             >> @@ -293,7 +305,7 @@
>>             >>        # Do not save /bin paths to all_fixpath_prefixes!
>>             >>      else
>>             >>        # Not in mixed or Windows style, start by that.
>>             >> -    new_path=`cmd //c echo $new_path`
>>             >> +    new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo
>>             $new_path`
>>             >> BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
>>             >>        # Output is in $new_path
>>             >> BASIC_WINDOWS_REWRITE_AS_UNIX_PATH(new_path)
>>             >> @@ -302,6 +314,8 @@
>>             >>
>>             >>        # Save the first 10 bytes of this path to the
>>             storage, so fixpath
>>             >> can work.
>>             >> all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
>>             >> "${new_path:0:10}")
>>             >> +    # We might need to re-evaluate FIXPATH.
>>             >> +    BASIC_MSYS_UPDATE_FIXPATH
>>             >>      fi
>>             >>    ])
>>             >>
>>             >> @@ -347,6 +361,10 @@
>>             >>        WINDOWS_ENV_VENDOR='msys'
>>             >> WINDOWS_ENV_VERSION="$MSYS_VERSION"
>>             >>
>>             >> +    # Prohibit msys2 path conversion from trying to
>>             be "intelligent",
>>             >> and rely
>>             >> +    # on fixpath instead.
>>             >> +    export MSYS2_ARG_CONV_EXCL="*"
>>             >> +
>>             >>        AC_MSG_CHECKING([msys root directory as
>>             unix-style path])
>>             >>        # The cmd output ends with Windows line endings
>>             (CR/LF), the grep
>>             >> command will strip that away
>>             >>        MSYS_ROOT_PATH=`cd / ; cmd /c cd | $GREP ".*"`
>>             >> @@ -391,10 +409,7 @@
>>             >>        elif test "x$OPENJDK_BUILD_OS_ENV" =
>>             xwindows.msys; then
>>             >>          # Take all collected prefixes and turn them
>>             into a
>>             >> -m/c/foo@/c/bar@... command line
>>             >>          # @ was chosen as separator to minimize risk
>>             of other tools
>>             >> messing around with it
>>             >> -      all_unique_prefixes=`echo
>>             "${all_fixpath_prefixes@<:@@@:>@}" \
>>             >> -          | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
>>             >> -      fixpath_argument_list=`echo
>>             $all_unique_prefixes  | tr ' ' '@'`
>>             >> -      FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
>>             >> +      BASIC_MSYS_UPDATE_FIXPATH
>>             >>        fi
>>             >>        FIXPATH_SRC_W="$FIXPATH_SRC"
>>             >>        FIXPATH_BIN_W="$FIXPATH_BIN"
>>             >> diff -r a08cbfc0e4ec common/autoconf/build-aux/config.sub
>>             >> --- a/common/autoconf/build-aux/config.sub    Thu Aug
>>             03 18:56:56
>>             >> 2017 +0000
>>             >> +++ b/common/autoconf/build-aux/config.sub    Wed Oct
>>             04 00:53:58
>>             >> 2017 +0200
>>             >> @@ -30,7 +30,7 @@
>>             >>    DIR=`dirname $0`
>>             >>
>>             >>    # First, filter out everything that doesn't begin
>>             with "aarch64-"
>>             >> -if ! echo $* | grep '^aarch64-' >/dev/null ; then
>>             >> +if ! echo $* | grep -e '^aarch64-' -e 'msys'
>>             >/dev/null ; then
>>             >>        . $DIR/autoconf-config.sub "$@"
>>             >>        # autoconf-config.sub exits, so we never reach
>>             here, but just in
>>             >>        # case we do:
>>             >> @@ -38,13 +38,17 @@
>>             >>    fi
>>             >>
>>             >>    while test $# -gt 0 ; do
>>             >> -    case $1 in
>>             >> +    case $1 in
>>             >>            -- )   # Stop option processing
>>             >>                shift; break ;;
>>             >>            aarch64-* )
>>             >>                config=`echo $1 | sed 's/^aarch64-/arm-/'`
>>             >>                sub_args="$sub_args $config"
>>             >>                shift; ;;
>>             >> +        *-msys )
>>             >> +            config=`echo $1 | sed 's/msys/mingw32/'`
>>             >> +            sub_args="$sub_args $config"
>>             >> +            shift; ;;
>>             >>            - )    # Use stdin as input.
>>             >>                sub_args="$sub_args $1"
>>             >>                shift; break ;;
>>             >> diff -r a08cbfc0e4ec common/autoconf/spec.gmk.in
>>             >> --- a/common/autoconf/spec.gmk.in    Thu Aug 03
>>             18:56:56 2017 +0000
>>             >> +++ b/common/autoconf/spec.gmk.in    Wed Oct 04
>>             00:53:58 2017 +0200
>>             >> @@ -120,6 +120,13 @@
>>             >>      # On Windows, the Visual Studio toolchain needs
>>             the PATH to be
>>             >> adjusted
>>             >>      # to include Visual Studio tools (this needs to
>>             be in cygwin/msys
>>             >> style).
>>             >>      export PATH:=@VS_PATH@
>>             >> +
>>             >> +endif
>>             >> +
>>             >> +ifeq ($(OPENJDK_TARGET_OS_ENV), windows.msys)
>>             >> +  # On msys2, prohibit msys path conversion from
>>             trying to be
>>             >> +  # "intelligent", and rely on fixpath instead.
>>             >> +  export MSYS2_ARG_CONV_EXCL:=*
>>             >>    endif
>>             >>
>>             >>    SYSROOT_CFLAGS := @SYSROOT_CFLAGS@
>>             >>
>>             >> /Magnus
>>             >>
>>             >> On 2017-10-03 22:34, Magnus Ihse Bursie wrote:
>>             >>> I gave msys2 a shot some time ago, but it ended up
>>             too much trouble.
>>             >>> I'll share some of my notes from that attempt, for
>>             what it's worth.
>>             >>>
>>             >>> To install package X/Y, run "pacman -S X/Y". Missing
>>             tools and
>>             >>> packages where to find them:
>>             >>> cmp: msys/diffutils
>>             >>> tar: msys/tar
>>             >>> make: msys/make
>>             >>> unzip: msys/unzip
>>             >>> zip: msys/zip
>>             >>>
>>             >>> config.sub reports msys as "x86_64-pc-mingw32" but
>>             msys2 as
>>             >>> "x86_64-pc-msys". This patch adds postprocessing in
>>             "our" config.sub
>>             >>> to report msys2 similar to msys. (Opinions, including
>>             my own :-) may
>>             >>> vary if this really is the best way..)
>>             >>>
>>             >>> diff -r b88023f46daa common/autoconf/build-aux/config.sub
>>             >>> --- a/common/autoconf/build-aux/config.sub      Fri
>>             Jan 27 10:15:41
>>             >>> 2017 +0100
>>             >>> +++ b/common/autoconf/build-aux/config.sub      Fri
>>             Feb 03 05:00:25
>>             >>> 2017 -0700
>>             >>> @@ -30,7 +30,7 @@
>>             >>>   DIR=`dirname $0`
>>             >>>
>>             >>>   # First, filter out everything that doesn't begin
>>             with "aarch64-"
>>             >>> -if ! echo $* | grep '^aarch64-' >/dev/null ; then
>>             >>> +if ! echo $* | grep -e '^aarch64-' -e 'msys'
>>             >/dev/null ; then
>>             >>>       . $DIR/autoconf-config.sub "$@"
>>             >>>       # autoconf-config.sub exits, so we never reach
>>             here, but just in
>>             >>>       # case we do:
>>             >>> @@ -45,6 +45,10 @@
>>             >>>               config=`echo $1 | sed 's/^aarch64-/arm-/'`
>>             >>>               sub_args="$sub_args $config"
>>             >>>               shift; ;;
>>             >>> +        *-msys )
>>             >>> +            config=`echo $1 | sed 's/msys/mingw32/'`
>>             >>> +            sub_args="$sub_args $config"
>>             >>> +            shift; ;;
>>             >>>           - )    # Use stdin as input.
>>             >>>               sub_args="$sub_args $1"
>>             >>>               shift; break ;;
>>             >>>
>>             >>> If I remember correctly, this got me past the
>>             configure stage at the
>>             >>> time.
>>             >>>
>>             >>> I don't think it's very hard to get it to work on
>>             msys2, I just ran
>>             >>> into one snag too many and didn't think msys2 would
>>             be used by anyone.
>>             >>>
>>             >>> /Magnus
>>             >>>
>>             >>> On 2017-10-03 17:20, Peter Budai wrote:
>>             >>>> Hello,
>>             >>>>
>>             >>>> According to
>>             >>>>
>>             http://hg.openjdk.java.net/jdk9/jdk9/file/a08cbfc0e4ec/common/doc/building.html
>>
>>             >>>>
>>             >>>> “msys2 and the new Windows Subsystem for Linux (WSL)
>>             would likely be
>>             >>>> possible to support in a future version but that
>>             would require a
>>             >>>> community effort to implement”
>>             >>>>
>>             >>>> I’d like to help making the OpenJDK 9 build working
>>             on msys2. What is
>>             >>>> the best way to move forward? Is there a similar
>>             effort in progress?
>>             >>>>
>>             >>>> Thank you and best regards,
>>             >>>>
>>             >>>> Peter
>>             >>>>
>>             >>>>
>>             >
>>

12