Open source TCK was: JDK 9: General Availability

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Open source TCK was: JDK 9: General Availability

David Herron-2
On Sat, Sep 23, 2017 at 4:25 PM, Volker Simonis <[hidden email]>
wrote:

> dalibor topic <[hidden email]> schrieb am Fr. 22. Sep. 2017 um
> 22:30:
>
> > On 22.09.2017 18:21, Volker Simonis wrote:
> >
> > > improving the conformance tests and only then it will be possible to
> > > transparently certify one's own implementation as well as verifying
> > > the implementation of others being standards conforming.
> >
> > 'Verifying' wouldn't work, if you think a bit further about how that
> > would play out in practice. Failure to reproduce results of others could
> > have any number of benign reasons without cause for alarm.
>
>
> > For a related, timely discussion in the field of science, please see
> >
> > https://news.northeastern.edu/2015/09/failure-to-reproduce-
> results-is-a-normal-part-of-how-science-works/
> > .
> >
>
> I hope you don't want to propose that nobody should publish any scientific
> findings in the future just because their reproduction by others may fail.
> That sounds a little "Trumpish" to make obscurity great again :)
>
> For our concrete problem (i.e. a JCK certification run) publishing the
> complete certification data (especially all the .jtr files) would give a
> pretty good overview of what people really did. And if this data still
> leaves open questions which are debatable - that would actually be great!
> That would be fruitful for everybody: the Java community, the Java
> implementations, the Java specification and last but not least, for the TCK
> itself!
>
> I know that big companies love "security by obscurity" (I'm working for one
> myself ;) But that's not my personal opinion. Instead, I think an open TCK
> would help the Java ecosystem just as much as the OpenJDK did.
>
> Open sourcing the TCK for Java EE is a good step into the right direction.
> However if Oracle wants to be taken seriously with this step, open sourcing
> the Jave SE TCK must be the direct consequence. Otherwise there's a danger
> that the whole Java EE open sourcing story may look a little like riding a
> dead horse :)
>
> Regards,
> Volker
>
>
 You make a compelling case and I largely agree with the goal of "open
sourcing" the TCK's.  Doing so would go a long way towards truly opening
the OpenJDK project.

There is a practical question regarding the primary use for the TCK's -
which is to certify compliance with the specifications.

An open source software project is free to be downloaded, modified, and the
modified version distributed at will.

Hence, if someone tested with a modified TCK how can there be certainty
about the result?  Can the results of testing with a modified TCK be
trusted for certification of compliance?

For the purpose of sharing of test results, open sourcing the TCK's would
be great.  The TCK's have a different purpose than just verifying
functionality.

David Herron
Reply | Threaded
Open this post in threaded view
|

Re: Open source TCK was: JDK 9: General Availability

Andrew Haley
On 24/09/17 00:51, David Herron wrote:
>  You make a compelling case and I largely agree with the goal of "open
> sourcing" the TCK's.  Doing so would go a long way towards truly opening
> the OpenJDK project.
>
> There is a practical question regarding the primary use for the
> TCK's - which is to certify compliance with the specifications.
>
> An open source software project is free to be downloaded, modified,
> and the modified version distributed at will.

Not necessarily: there are variants of free licences which don't
permit some sections to be changed.  But it doesn't really matter what
licence is used for the TCK itself: the Java Compatible badge would be
reserved for implementations which had passed TCK Version n.n, as
posted at java.net.  That would be the canonical one true TCK.  Sure,
people would be able to hack their own copies of the TCK, but what
would be the point?  The purpose of passing the TCK is to ensure
compatibility with other implementations.  There isn't any motive to
claim compliance with a private version of the TCK.

> For the purpose of sharing of test results, open sourcing the TCK's would
> be great.  The TCK's have a different purpose than just verifying
> functionality.
>
> Hence, if someone tested with a modified TCK how can there be certainty
> about the result?  Can the results of testing with a modified TCK be
> trusted for certification of compliance?

No, and no.

--
Andrew Haley
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671
Reply | Threaded
Open this post in threaded view
|

Re: Open source TCK was: JDK 9: General Availability

Geir Magnusson Jr.-3

> On Sep 24, 2017, at 2:46 AM, Andrew Haley <[hidden email]> wrote:
>
> On 24/09/17 00:51, David Herron wrote:
>> You make a compelling case and I largely agree with the goal of "open
>> sourcing" the TCK's.  Doing so would go a long way towards truly opening
>> the OpenJDK project.
>>
>> There is a practical question regarding the primary use for the
>> TCK's - which is to certify compliance with the specifications.
>>
>> An open source software project is free to be downloaded, modified,
>> and the modified version distributed at will.
>
> Not necessarily: there are variants of free licences which don't
> permit some sections to be changed.  But it doesn't really matter what
> licence is used for the TCK itself: the Java Compatible badge would be
> reserved for implementations which had passed TCK Version n.n, as
> posted at java.net.  That would be the canonical one true TCK.  Sure,
> people would be able to hack their own copies of the TCK, but what
> would be the point?  The purpose of passing the TCK is to ensure
> compatibility with other implementations.  There isn't any motive to
> claim compliance with a private version of the TCK.

Generally agree.  Only quibble is that there may be plenty of motive (for example, AIUI, the Java SE TCK still has limiting constraints when licensed to third parties…), but still, no benefit.  A private version of the TCK wouldn’t be the TCK as a derivative work of the TCK isn’t the TCK.  

Only the software suite defined by and distributed by the spec lead is the TCK.

(Similar parallel with OpenJDK.  Derivatives aren’t the OpenJDK.)

>
>> For the purpose of sharing of test results, open sourcing the TCK's would
>> be great.  The TCK's have a different purpose than just verifying
>> functionality.
>>
>> Hence, if someone tested with a modified TCK how can there be certainty
>> about the result?  Can the results of testing with a modified TCK be
>> trusted for certification of compliance?
>
> No, and no.
>
> --
> Andrew Haley
> Java Platform Lead Engineer
> Red Hat UK Ltd. <https://www.redhat.com>
> EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671

Reply | Threaded
Open this post in threaded view
|

Re: Open source TCK was: JDK 9: General Availability

Andrew Hughes-8
In reply to this post by Andrew Haley
On 24 September 2017 at 07:46, Andrew Haley <[hidden email]> wrote:

> On 24/09/17 00:51, David Herron wrote:
>>  You make a compelling case and I largely agree with the goal of "open
>> sourcing" the TCK's.  Doing so would go a long way towards truly opening
>> the OpenJDK project.
>>
>> There is a practical question regarding the primary use for the
>> TCK's - which is to certify compliance with the specifications.
>>
>> An open source software project is free to be downloaded, modified,
>> and the modified version distributed at will.
>
> Not necessarily: there are variants of free licences which don't
> permit some sections to be changed.  But it doesn't really matter what
> licence is used for the TCK itself: the Java Compatible badge would be
> reserved for implementations which had passed TCK Version n.n, as
> posted at java.net.  That would be the canonical one true TCK.  Sure,
> people would be able to hack their own copies of the TCK, but what
> would be the point?  The purpose of passing the TCK is to ensure
> compatibility with other implementations.  There isn't any motive to
> claim compliance with a private version of the TCK.
>

I tend to agree with Andrew on this. Passing the TCK is already a
process based more on trust than source code licensing. It is
taken as part of making such an assertion that the TCK is being
run as shipped with only approved exclusions.

On the other hand, I think trust in the TCK itself could be improved
by making the source code public. Only then is it possible for anyone
to verify that the TCK is testing against the specification, rather than
a particular implementation.

Thanks,
--
Andrew :)

Senior Free Java Software Engineer
Red Hat, Inc. (http://www.redhat.com)

Web Site: http://fuseyism.com
Twitter: https://twitter.com/gnu_andrew_java
PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net)
Fingerprint = 5132 579D D154 0ED2 3E04  C5A0 CFDA 0F9B 3596 4222