RFR: JDK-8190702: JMC packaging causing errors in JDK 10 consolidated repo (macOS)

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

RFR: JDK-8190702: JMC packaging causing errors in JDK 10 consolidated repo (macOS)

Erik Joelsson
When I added support for copying files with spaces in their names, I
missed a case. On Mac (the only platform where we currently, in the
Oracle build, have a directory with space in it), when rebuilding images
after actually touching the source, the build fails with:

cp:
/Users/danielfuchs/workspaces/jdk/jdk10/build/macosx-x64/images/jdk/lib/missioncontrol/Java
Mission Control.app/Contents/Eclipse/jmc.ini: No such file or directory

This is caused by the $(call MakeDir, $(@D)) line in the install-file
macro not actually creating the parent directory before trying to copy
the file. This in turn was caused by two things:

First, if a target file that contains spaces, that is defined using the
? wildcard trick, exists when the makefile is parsed, make will resolve
those wildcards to spaces, so the target file variable ($@) will have
spaces in it instead of ?. This has the consequence that $(@D), which is
just a shortcut for $(dir $@) will actually apply the dir call on each
space separated entity in $@. So the first fix was to make sure the
input to $(dir ) is actually SpaceEncoded.

Second, the wildcard function seem to be doing some kind of clever
caching of results and this caching breaks down when there are spaces in
filenames. Because of this, if the argument of $(wildcard ) existed when
the makefile was parsed, but was then deleted as part of a prerequisite
rule, $(wildcard ) will still return the file as existing when called
from a subsequent recipe line. This only happens if the file argument to
wildcard contains spaces, so my best guess is that some string matching
is happening in some caching table and fails (but I haven't actually
checked the gnu make source). My fix for this is to change the MakeDir
macro so that it always runs mkdir if the target dir contains ?.

I also have modified the copy-files tests to reproduce this situation
and verified that the fix solves it there as well.

Bug: https://bugs.openjdk.java.net/browse/JDK-8190702

Webrev: http://cr.openjdk.java.net/~erikj/8190702/webrev.01/

/Erik

Reply | Threaded
Open this post in threaded view
|

Re: RFR: JDK-8190702: JMC packaging causing errors in JDK 10 consolidated repo (macOS)

Daniel Fuchs

Thanks for that Erik!
It's been annoying me for a couple of days :-)

I have imported your patch locally in my repo and it seems to
be working fine.

Hopefully someone from the build team will give you a thumbs up!

best regards,

-- daniel


On 14/11/2017 22:35, Erik Joelsson wrote:

> When I added support for copying files with spaces in their names, I
> missed a case. On Mac (the only platform where we currently, in the
> Oracle build, have a directory with space in it), when rebuilding images
> after actually touching the source, the build fails with:
>
> cp:
> /Users/danielfuchs/workspaces/jdk/jdk10/build/macosx-x64/images/jdk/lib/missioncontrol/Java
> Mission Control.app/Contents/Eclipse/jmc.ini: No such file or directory
>
> This is caused by the $(call MakeDir, $(@D)) line in the install-file
> macro not actually creating the parent directory before trying to copy
> the file. This in turn was caused by two things:
>
> First, if a target file that contains spaces, that is defined using the
> ? wildcard trick, exists when the makefile is parsed, make will resolve
> those wildcards to spaces, so the target file variable ($@) will have
> spaces in it instead of ?. This has the consequence that $(@D), which is
> just a shortcut for $(dir $@) will actually apply the dir call on each
> space separated entity in $@. So the first fix was to make sure the
> input to $(dir ) is actually SpaceEncoded.
>
> Second, the wildcard function seem to be doing some kind of clever
> caching of results and this caching breaks down when there are spaces in
> filenames. Because of this, if the argument of $(wildcard ) existed when
> the makefile was parsed, but was then deleted as part of a prerequisite
> rule, $(wildcard ) will still return the file as existing when called
> from a subsequent recipe line. This only happens if the file argument to
> wildcard contains spaces, so my best guess is that some string matching
> is happening in some caching table and fails (but I haven't actually
> checked the gnu make source). My fix for this is to change the MakeDir
> macro so that it always runs mkdir if the target dir contains ?.
>
> I also have modified the copy-files tests to reproduce this situation
> and verified that the fix solves it there as well.
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8190702
>
> Webrev: http://cr.openjdk.java.net/~erikj/8190702/webrev.01/
>
> /Erik
>

Reply | Threaded
Open this post in threaded view
|

Re: RFR: JDK-8190702: JMC packaging causing errors in JDK 10 consolidated repo (macOS)

Tim Bell
Erik:

Looks good to me as well.

Tim

On 11/15/17 07:52, Daniel Fuchs wrote:

>
> Thanks for that Erik!
> It's been annoying me for a couple of days :-)
>
> I have imported your patch locally in my repo and it seems to
> be working fine.
>
> Hopefully someone from the build team will give you a thumbs up!
>
> best regards,
>
> -- daniel
>
>
> On 14/11/2017 22:35, Erik Joelsson wrote:
>> When I added support for copying files with spaces in their names, I
>> missed a case. On Mac (the only platform where we currently, in the
>> Oracle build, have a directory with space in it), when rebuilding
>> images after actually touching the source, the build fails with:
>>
>> cp:
>> /Users/danielfuchs/workspaces/jdk/jdk10/build/macosx-x64/images/jdk/lib/missioncontrol/Java
>> Mission Control.app/Contents/Eclipse/jmc.ini: No such file or directory
>>
>> This is caused by the $(call MakeDir, $(@D)) line in the install-file
>> macro not actually creating the parent directory before trying to copy
>> the file. This in turn was caused by two things:
>>
>> First, if a target file that contains spaces, that is defined using
>> the ? wildcard trick, exists when the makefile is parsed, make will
>> resolve those wildcards to spaces, so the target file variable ($@)
>> will have spaces in it instead of ?. This has the consequence that
>> $(@D), which is just a shortcut for $(dir $@) will actually apply the
>> dir call on each space separated entity in $@. So the first fix was to
>> make sure the input to $(dir ) is actually SpaceEncoded.
>>
>> Second, the wildcard function seem to be doing some kind of clever
>> caching of results and this caching breaks down when there are spaces
>> in filenames. Because of this, if the argument of $(wildcard ) existed
>> when the makefile was parsed, but was then deleted as part of a
>> prerequisite rule, $(wildcard ) will still return the file as existing
>> when called from a subsequent recipe line. This only happens if the
>> file argument to wildcard contains spaces, so my best guess is that
>> some string matching is happening in some caching table and fails (but
>> I haven't actually checked the gnu make source). My fix for this is to
>> change the MakeDir macro so that it always runs mkdir if the target
>> dir contains ?.
>>
>> I also have modified the copy-files tests to reproduce this situation
>> and verified that the fix solves it there as well.
>>
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8190702
>>
>> Webrev: http://cr.openjdk.java.net/~erikj/8190702/webrev.01/
>>
>> /Erik
>>
>