Howto build an old version of hotspot (pre forest consolidation)

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Howto build an old version of hotspot (pre forest consolidation)

Volker Simonis
Hi,

can somebody provide any hints on how to build an older version of
hotspot in the correct context?

I have a reproducible hotspot crash which occurs with jdk-9+105 but
not with jdk-9+104. I can sync to both tags and successfully build the
corresponding jdk. But it seems impossible to update the hotspot repo
to a change between  jdk-9+104 and jdk-9+105 and build the complete
jdk because I always get some wired build errors (no difference if the
other repos are on  jdk-9+104 or jdk-9+105). It seems that the hotspot
revisions between jdk-9+104 and jdk-9+105 are based on some older
versions of jdk. Is there s simple way how I can find out on which
version?

Thank you and best regards,
Volker
Reply | Threaded
Open this post in threaded view
|

Re: Howto build an old version of hotspot (pre forest consolidation)

Erik Joelsson
Hello Volker,

The recommended way of actually building source from pre-consolidation
is to clone a tree from that day and age (jdk9 in this case). That said,
it should be possible to build the consolidated repo as well (as long as
you are working in an OpenJDK only situation).

The merge of the repos is only actually correct at the tags. Between
tags, each old repo moves on its own, with the rest of the repos sitting
still on the last tag, then at the next tag, they are all merged
together again. So in your case (provided I got it all right), if you
first sync to jdk-9+104, that is a verified equivalent to the combined
tags of jdk-9+104 pre-consolidation. If you then sync to a hotspot
change after that, the rest of the repos should sit still at 104. This
does not guarantee that the consolidated repo builds on all or any of
those changes since we can't know which combination of changes were ever
actually a working set of tips in the old forest, except for the
promoted tags.

Also note that just because a change in the hotspot subtree has a higher
number than the tagged jdk-9+104, it does not necessarily mean it is a
descendant of jdk-9+104. Especially for hotspot, since it was merged
through one or two side forests, many of the new changes in 105 likely
have parents much older than that. To find the relevant list of hotspot
changes to try to build, you need to do a more advanced revset query,
looking for descendants of 104 and ancestors of 105. Many of these
changes will be merges that brought the actual changes in, typically
when the integrator brought 104 down from dev.

/Erik


On 2017-11-13 08:09, Volker Simonis wrote:

> Hi,
>
> can somebody provide any hints on how to build an older version of
> hotspot in the correct context?
>
> I have a reproducible hotspot crash which occurs with jdk-9+105 but
> not with jdk-9+104. I can sync to both tags and successfully build the
> corresponding jdk. But it seems impossible to update the hotspot repo
> to a change between  jdk-9+104 and jdk-9+105 and build the complete
> jdk because I always get some wired build errors (no difference if the
> other repos are on  jdk-9+104 or jdk-9+105). It seems that the hotspot
> revisions between jdk-9+104 and jdk-9+105 are based on some older
> versions of jdk. Is there s simple way how I can find out on which
> version?
>
> Thank you and best regards,
> Volker

Reply | Threaded
Open this post in threaded view
|

Re: Howto build an old version of hotspot (pre forest consolidation)

Volker Simonis
Hi Erik,

thanks for the explanation and suggestions. I've managed to find the
offending change  (8145322: Code generated from unsafe loops can be
slightly improved) just to find out that Roland already sent out a RFR
for the fix (RFR(S): 8190375: Java Crash in
JavaBug.formatPos(I)Ljava/lang/String :)

In the end I could actually only find the offending change by building
"hotspot-only" and copying libjvm.so into a pre-built image. The
pre-build image was from jdk-9+100, the hotspot changes I was testing
were from hotspot between jdk-9+104 and jdk-9+105.

From my understanding (and from what you wrote) I thought there must
be an older, tagged version of jdk and the other sub-repos (i.e. older
than jdk-9+104) against which the changes in hotspot, which finally
ended up between jdk-9+104 and jdk-9+105 in the main repo, must have
been developed. Unfortunately, I couldn't do a complete, clean build
of the offending hotspot changes with the other repos being synced at
any of the tags between jdk-9+95 and jdk-9+105.

Another problem is that the date attribute on the changesets is
actually unusable because it is fooled by tools like Mercurial Queues
which many developers (including myself) are using. The problem is
that the date attribute on a changeset is the date, when the change
was first committed in the developers repo, which can be potentially
quite some time before it was integrated or completely bogus if the
developers local time was not accurate or he deliberately used a bad
"--date" argument when committing.

Taking all this into account, I don't see how I could clone a tree
"from that day and age" (although that would be usefull)? JBS contains
the correct check-in times, but unfortunately that's an external tool
and cumbersome to look-up.

After all, I finally found the offending change and hopefully these
kind of problems will disappear in the future with the consolidated
repo.

Thank you and best regards,
Volker



On Mon, Nov 13, 2017 at 5:43 PM, Erik Joelsson <[hidden email]> wrote:

> Hello Volker,
>
> The recommended way of actually building source from pre-consolidation is to
> clone a tree from that day and age (jdk9 in this case). That said, it should
> be possible to build the consolidated repo as well (as long as you are
> working in an OpenJDK only situation).
>
> The merge of the repos is only actually correct at the tags. Between tags,
> each old repo moves on its own, with the rest of the repos sitting still on
> the last tag, then at the next tag, they are all merged together again. So
> in your case (provided I got it all right), if you first sync to jdk-9+104,
> that is a verified equivalent to the combined tags of jdk-9+104
> pre-consolidation. If you then sync to a hotspot change after that, the rest
> of the repos should sit still at 104. This does not guarantee that the
> consolidated repo builds on all or any of those changes since we can't know
> which combination of changes were ever actually a working set of tips in the
> old forest, except for the promoted tags.
>
> Also note that just because a change in the hotspot subtree has a higher
> number than the tagged jdk-9+104, it does not necessarily mean it is a
> descendant of jdk-9+104. Especially for hotspot, since it was merged through
> one or two side forests, many of the new changes in 105 likely have parents
> much older than that. To find the relevant list of hotspot changes to try to
> build, you need to do a more advanced revset query, looking for descendants
> of 104 and ancestors of 105. Many of these changes will be merges that
> brought the actual changes in, typically when the integrator brought 104
> down from dev.
>
> /Erik
>
>
>
> On 2017-11-13 08:09, Volker Simonis wrote:
>>
>> Hi,
>>
>> can somebody provide any hints on how to build an older version of
>> hotspot in the correct context?
>>
>> I have a reproducible hotspot crash which occurs with jdk-9+105 but
>> not with jdk-9+104. I can sync to both tags and successfully build the
>> corresponding jdk. But it seems impossible to update the hotspot repo
>> to a change between  jdk-9+104 and jdk-9+105 and build the complete
>> jdk because I always get some wired build errors (no difference if the
>> other repos are on  jdk-9+104 or jdk-9+105). It seems that the hotspot
>> revisions between jdk-9+104 and jdk-9+105 are based on some older
>> versions of jdk. Is there s simple way how I can find out on which
>> version?
>>
>> Thank you and best regards,
>> Volker
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Howto build an old version of hotspot (pre forest consolidation)

Magnus Ihse Bursie
On 2017-11-13 19:06, Volker Simonis wrote:

> Hi Erik,
>
> thanks for the explanation and suggestions. I've managed to find the
> offending change  (8145322: Code generated from unsafe loops can be
> slightly improved) just to find out that Roland already sent out a RFR
> for the fix (RFR(S): 8190375: Java Crash in
> JavaBug.formatPos(I)Ljava/lang/String :)
>
> In the end I could actually only find the offending change by building
> "hotspot-only" and copying libjvm.so into a pre-built image. The
> pre-build image was from jdk-9+100, the hotspot changes I was testing
> were from hotspot between jdk-9+104 and jdk-9+105.
>
>  From my understanding (and from what you wrote) I thought there must
> be an older, tagged version of jdk and the other sub-repos (i.e. older
> than jdk-9+104) against which the changes in hotspot, which finally
> ended up between jdk-9+104 and jdk-9+105 in the main repo, must have
> been developed. Unfortunately, I couldn't do a complete, clean build
> of the offending hotspot changes with the other repos being synced at
> any of the tags between jdk-9+95 and jdk-9+105.
>
> Another problem is that the date attribute on the changesets is
> actually unusable because it is fooled by tools like Mercurial Queues
> which many developers (including myself) are using. The problem is
> that the date attribute on a changeset is the date, when the change
> was first committed in the developers repo, which can be potentially
> quite some time before it was integrated or completely bogus if the
> developers local time was not accurate or he deliberately used a bad
> "--date" argument when committing.
>
> Taking all this into account, I don't see how I could clone a tree
> "from that day and age" (although that would be usefull)? JBS contains
> the correct check-in times, but unfortunately that's an external tool
> and cumbersome to look-up.
Unfortunately, there is no such way. :-( The "best" approximation of
what consituted the complete source code tree at any point in time is
provided by the jdk*-changes mailing lists. You would have to dig up the
mails there, check their timestamps and then reconstruate the timeline
of the entire forst.

Yes, this sucks big time. :-(

It means that interval halving your way to find a problem when the code
spans multiple repos is virtually impossible in practice. I've done it a
few times for the build system when I found no other way out. In normal
circumstances, bisecting commits would be an almost automated process
that should be on top of your list of tools. On the flip side, this
should now work better, as long as you stick to OpenJDK only with no
closed extensions. :)

/Magnus

>
> After all, I finally found the offending change and hopefully these
> kind of problems will disappear in the future with the consolidated
> repo.
>
> Thank you and best regards,
> Volker
>
>
>
> On Mon, Nov 13, 2017 at 5:43 PM, Erik Joelsson <[hidden email]> wrote:
>> Hello Volker,
>>
>> The recommended way of actually building source from pre-consolidation is to
>> clone a tree from that day and age (jdk9 in this case). That said, it should
>> be possible to build the consolidated repo as well (as long as you are
>> working in an OpenJDK only situation).
>>
>> The merge of the repos is only actually correct at the tags. Between tags,
>> each old repo moves on its own, with the rest of the repos sitting still on
>> the last tag, then at the next tag, they are all merged together again. So
>> in your case (provided I got it all right), if you first sync to jdk-9+104,
>> that is a verified equivalent to the combined tags of jdk-9+104
>> pre-consolidation. If you then sync to a hotspot change after that, the rest
>> of the repos should sit still at 104. This does not guarantee that the
>> consolidated repo builds on all or any of those changes since we can't know
>> which combination of changes were ever actually a working set of tips in the
>> old forest, except for the promoted tags.
>>
>> Also note that just because a change in the hotspot subtree has a higher
>> number than the tagged jdk-9+104, it does not necessarily mean it is a
>> descendant of jdk-9+104. Especially for hotspot, since it was merged through
>> one or two side forests, many of the new changes in 105 likely have parents
>> much older than that. To find the relevant list of hotspot changes to try to
>> build, you need to do a more advanced revset query, looking for descendants
>> of 104 and ancestors of 105. Many of these changes will be merges that
>> brought the actual changes in, typically when the integrator brought 104
>> down from dev.
>>
>> /Erik
>>
>>
>>
>> On 2017-11-13 08:09, Volker Simonis wrote:
>>> Hi,
>>>
>>> can somebody provide any hints on how to build an older version of
>>> hotspot in the correct context?
>>>
>>> I have a reproducible hotspot crash which occurs with jdk-9+105 but
>>> not with jdk-9+104. I can sync to both tags and successfully build the
>>> corresponding jdk. But it seems impossible to update the hotspot repo
>>> to a change between  jdk-9+104 and jdk-9+105 and build the complete
>>> jdk because I always get some wired build errors (no difference if the
>>> other repos are on  jdk-9+104 or jdk-9+105). It seems that the hotspot
>>> revisions between jdk-9+104 and jdk-9+105 are based on some older
>>> versions of jdk. Is there s simple way how I can find out on which
>>> version?
>>>
>>> Thank you and best regards,
>>> Volker
>>