You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
subprocess gained the ability to use close_range() in #84603, however, it's only used as a fallback if procfs is not available, which means that at least on Linux it's almost never used. Its performance benefits are significant (see numbers at the end). I propose to use it by default, with procfs being the second option.
This requires some extra changes because _Py_closerange() was designed to be standalone and tries all available methods, including brute force, and also because it's not required to be async-signal-safe, but we must preserve async-signal-safety of fd closing code on Linux. I'll open a PR with a proposed implementation.
Performance comparison that I did on a test version with the following script (may require you to raise the fd limit in your shell):
import os, subprocess, sys, timeit
from resource import *
soft, hard = getrlimit(RLIMIT_NOFILE)
setrlimit(RLIMIT_NOFILE, (hard, hard))
num_fds, num_iter = map(int, sys.argv[1:3])
for i in range(num_fds):
os.open('/dev/null', os.O_RDONLY)
print(timeit.timeit(lambda: subprocess.run('/bin/true'), number=num_iter))
…#92303)
#92301: subprocess: Prefer `close_range()` to procfs-based fd closing.
`close_range()` is much faster for large number of file descriptors, e.g.
4 times faster for 1000 descriptors in a Linux 5.16-based environment.
We prefer close_range() only if it's known to be async-signal-safe.
subprocess gained the ability to use close_range() in #84603, however, it's only used as a fallback if procfs is not available, which means that at least on Linux it's almost never used. Its performance benefits are significant (see numbers at the end). I propose to use it by default, with procfs being the second option.
This requires some extra changes because _Py_closerange() was designed to be standalone and tries all available methods, including brute force, and also because it's not required to be async-signal-safe, but we must preserve async-signal-safety of fd closing code on Linux. I'll open a PR with a proposed implementation.
Performance comparison that I did on a test version with the following script (may require you to raise the fd limit in your shell):
Before:
After:
The text was updated successfully, but these errors were encountered: