[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[jira] [Created] (ARROW-3936) Add _O_NOINHERIT to the file open flags on Windows

Philip Felton created ARROW-3936:

             Summary: Add _O_NOINHERIT to the file open flags on Windows
                 Key: ARROW-3936
             Project: Apache Arrow
          Issue Type: Bug
            Reporter: Philip Felton

Unlike Linux, Windows doesn't let you delete files that are currently opened by another process. So if you create a child process while a Parquet file is open, with the current code the file handle is inherited to the child process, and the parent process can't then delete the file after closing it without the child process terminating first.

By default, Win32 file handles are not inheritable (likely because of the aforementioned problems). Except for _wsopen_s, which tries to maintain POSIX compatibility.

This is a serious problem for us.

We would argue that specifying _O_NOINHERIT by default in the _MSC_VER path is a sensible approach and would likely be the correct behaviour as it matches the main Win32 API.

However, it could be that some developers rely on the current inheritable behaviour. In which case, the Arrow public API should take a boolean argument on whether the created file descriptor should be inheritable. But this would break API backward compatibility (unless a new overloaded method is introduced).

Is forking and inheriting Arrow internal file descriptor something that Arrow actually means to support?

What do we think of the proposed fix?

This message was sent by Atlassian JIRA