Possible error in PackTestZip64.java!?

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Possible error in PackTestZip64.java!?

Kharbas, Kishor
Hi!
I came across an exception when running pack200 test PackTestZip64.java when I use a non-bundled zlib library with JDK8. This causes the test to fail.
Upon examining the code, I believe the test fails incorrectly. The error is an side-effect of how new JarEntry's are created.

Below is the detailed report :

---------------------------------------------------------------------------------------------------------------------------------------------------
ERROR TITLE:
PackTestZip64 fails incorrectly with ZipException with JDK using system zlib

PRODUCT VERSION:
openjdk version "1.8.0-internal"
OpenJDK Runtime Environment (build 1.8.0-internal-root_2017_02_21_13_06-b00)
OpenJDK 64-Bit Server VM (build 25.60-b23, mixed mode)

OS VERSION:
Linux kkharbas-desk2.amr.corp.intel.com 4.9.12-200.fc25.x86_64 #1 SMP Thu Feb 23 19:31:49 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

A DESCRIPTION OF THE PROBLEM:
When using an alternative zlib implementation (in this case the implementation at https://github.com/jtkukunas/zlib) PackTestZip64 test fails incorrectly.
The test fails with following exception:
java.util.zip.ZipException: invalid entry compressed size (expected 88 but got 89 bytes)
    at java.util.zip.ZipOutputStream.closeEntry(ZipOutputStream.java:267)
    at java.util.zip.ZipOutputStream.putNextEntry(ZipOutputStream.java:192)
    at java.util.jar.JarOutputStream.putNextEntry(JarOutputStream.java:109)
    at PackTestZip64.generateLargeJar(PackTestZip64.java:137)
    at PackTestZip64.testPacking(PackTestZip64.java:55)
    at PackTestZip64.main(PackTestZip64.java:43)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:110)
    at java.lang.Thread.run(Thread.java:745)

Here is what happens :
The test generates a new jar file ( in generateLargeJar()) from existing files in golden.jar and also adds a large number of small dummy files.
Because the JarEntry is created from existing jar file, the compressed size for it is set.
Later ZipOutputStream.closeEntry() validates the compressed size of a file by comparing it to number of bytes written,
                if (e.csize != def.getBytesWritten()) {
                    throw new ZipException(
                        "invalid entry compressed size (expected " +
                        e.csize + " but got " + def.getBytesWritten() + " bytes)");
                }

The above check fails, throwing an exception.
However, the Zlib specification gives implementations a flexibility of choosing a string search algorithm, thus generating different bit streams.
A workaround is to reset the compressed size of 'JarEntry' before its added to the new jar.

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
1. Build JDK specifying configure option : --with-zlib=system
2. Download and build the zlib from https://github.com/jtkukunas/zlib
3. Override the Linux zlib at /usr/lib64/
4. Run PackTestZip64 test with jtreg
---------------------------------------------------------------------------------------------------------------------------------------------------

If the community thinks this a valid diagnosis, an error should be filed.

Thanks
Kishor
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: Possible error in PackTestZip64.java!?

Kharbas, Kishor

Some more clarification on the error-

The origin of the error is this statement in generateLargeJar() in PackTestZip64.java:
    copyTo.putNextEntry(je);

A solution would be to reset the compressed size of 'je'
    je.setCompressedSize(-1);
    copyTo.putNextEntry(je);

Thanks
Kishor

From: Kharbas, Kishor
Sent: Monday, March 20, 2017 7:02 PM
To: [hidden email]
Cc: Kharbas, Kishor <[hidden email]>
Subject: Possible error in PackTestZip64.java!?

Hi!
I came across an exception when running pack200 test PackTestZip64.java when I use a non-bundled zlib library with JDK8. This causes the test to fail.
Upon examining the code, I believe the test fails incorrectly. The error is an side-effect of how new JarEntry's are created.

Below is the detailed report :

---------------------------------------------------------------------------------------------------------------------------------------------------
ERROR TITLE:
PackTestZip64 fails incorrectly with ZipException with JDK using system zlib

PRODUCT VERSION:
openjdk version "1.8.0-internal"
OpenJDK Runtime Environment (build 1.8.0-internal-root_2017_02_21_13_06-b00)
OpenJDK 64-Bit Server VM (build 25.60-b23, mixed mode)

OS VERSION:
Linux kkharbas-desk2.amr.corp.intel.com 4.9.12-200.fc25.x86_64 #1 SMP Thu Feb 23 19:31:49 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

A DESCRIPTION OF THE PROBLEM:
When using an alternative zlib implementation (in this case the implementation at https://github.com/jtkukunas/zlib) PackTestZip64 test fails incorrectly.
The test fails with following exception:
java.util.zip.ZipException: invalid entry compressed size (expected 88 but got 89 bytes)
    at java.util.zip.ZipOutputStream.closeEntry(ZipOutputStream.java:267)
    at java.util.zip.ZipOutputStream.putNextEntry(ZipOutputStream.java:192)
    at java.util.jar.JarOutputStream.putNextEntry(JarOutputStream.java:109)
    at PackTestZip64.generateLargeJar(PackTestZip64.java:137)
    at PackTestZip64.testPacking(PackTestZip64.java:55)
    at PackTestZip64.main(PackTestZip64.java:43)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:110)
    at java.lang.Thread.run(Thread.java:745)

Here is what happens :
The test generates a new jar file ( in generateLargeJar()) from existing files in golden.jar and also adds a large number of small dummy files.
Because the JarEntry is created from existing jar file, the compressed size for it is set.
Later ZipOutputStream.closeEntry() validates the compressed size of a file by comparing it to number of bytes written,
                if (e.csize != def.getBytesWritten()) {
                    throw new ZipException(
                        "invalid entry compressed size (expected " +
                        e.csize + " but got " + def.getBytesWritten() + " bytes)");
                }

The above check fails, throwing an exception.
However, the Zlib specification gives implementations a flexibility of choosing a string search algorithm, thus generating different bit streams.
A workaround is to reset the compressed size of 'JarEntry' before its added to the new jar.

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
1. Build JDK specifying configure option : --with-zlib=system
2. Download and build the zlib from https://github.com/jtkukunas/zlib
3. Override the Linux zlib at /usr/lib64/
4. Run PackTestZip64 test with jtreg
---------------------------------------------------------------------------------------------------------------------------------------------------

If the community thinks this a valid diagnosis, an error should be filed.

Thanks
Kishor
Loading...