git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Job jumbo jar can't be extracted on Windows


Looking at javadoc of separatorChar :

     * The system-dependent default name-separator character.  This field is

     * initialized to contain the first character of the value of the system

     * property <code>file.separator</code>.  On UNIX systems the value of
this

     * field is <code>'/'</code>; on Microsoft Windows systems it is <code>
'\\'</code>.

Looks like the code you quoted should get the correct separator.

On Tue, Jul 3, 2018 at 10:43 AM, NEKRASSOV, ALEXEI <an4828@xxxxxxx> wrote:

> When I try to "flink run" a job jar that includes dependent jar's - it
> fails on Windows with the following Exception:
>
> org.apache.flink.client.program.ProgramInvocationException: Unknown I/O
> error while extracting contained jar files.
>         at org.apache.flink.client.program.PackagedProgram.
> extractContainedLibraries(PackagedProgram.java:752)
>         at org.apache.flink.client.program.PackagedProgram.<init>
> (PackagedProgram.java:194)
>         at org.apache.flink.client.cli.CliFrontend.buildProgram(
> CliFrontend.java:833)
>         at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.
> java:201)
>         at org.apache.flink.client.cli.CliFrontend.parseParameters(
> CliFrontend.java:1020)
>         at org.apache.flink.client.cli.CliFrontend.lambda$main$9(
> CliFrontend.java:1096)
>         at org.apache.flink.runtime.security.NoOpSecurityContext.
> runSecured(NoOpSecurityContext.java:30)
>         at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.
> java:1096)
> Caused by: org.apache.flink.client.program.ProgramInvocationException: An
> I/O error occurred while creating temporary file to extract nested library
> 'lib/flink-connector-kafka-0.10_2.11-1.4.1.jar'.
>         at org.apache.flink.client.program.PackagedProgram.
> extractContainedLibraries(PackagedProgram.java:708)
>         ... 7 more
> Caused by: java.io.IOException: Unable to create temporary file,
> C:\Users\XXX\AppData\Local\Temp\1863208689_4625824260650653022lib\flink-
> connector-kafka-0.10_2.11-1.4.1.jar
>         at java.io.File$TempDirectory.generateFile(Unknown Source)
>         at java.io.File.createTempFile(Unknown Source)
>         at java.io.File.createTempFile(Unknown Source)
>         at org.apache.flink.client.program.PackagedProgram.
> extractContainedLibraries(PackagedProgram.java:702)
>         ... 7 more
>
> I think we have a problem in PackagedProgram.java - it fails to extract
> contained jar's on Windows, because on Windows File.separatorChar is '\',
> but JarEntry.getName() returns '/' as file separator.
> Line 699 of https://github.com/apache/flink/blob/master/flink-
> clients/src/main/java/org/apache/flink/client/program/PackagedProgram.java
> is no-op on Windows.
>
> I think we need to replace line 699 with
>             name = name.replace('/', '_');
> as I don't see a way to have JarEntry use platform-specific file
> separator, and ZIP spec dictates the use of forward-slashes anyway.
> Line 680 already uses hard-coded '/'.
>
>
> Alex Nekrassov
>