Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bpo-26175: Fix SpooledTemporaryFile IOBase abstract #3249

Closed
wants to merge 13 commits into from
Closed

bpo-26175: Fix SpooledTemporaryFile IOBase abstract #3249

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
83d6152
Fix SpooledTemporaryFile IOBase abstract
GFernie Aug 30, 2017
3b69baa
Add news entry for bpo-26175 patch
GFernie Aug 30, 2017
753b4fb
Update 2017-08-30-23-14-17.bpo-26175.GOaj9y.rst
merwok Sep 20, 2017
a70113e
Make SpooledTemporaryFile subclass IOBase
GFernie Sep 30, 2017
fb28362
Refactor tests to more reliably test abstract
GFernie Sep 30, 2017
bc9d0a9
Return new absolute position from underlying file on seek
GFernie Sep 9, 2018
b588dec
Do nothing on __del__
GFernie Sep 9, 2018
ab49dd1
Return new file size from underlying file on truncate
GFernie Sep 9, 2018
4251b21
Merge remote-tracking branch 'origin/master' into bpo-26175_fix-Spool…
GFernie Oct 30, 2019
19f4c08
Add versionchanged tag
GFernie Oct 30, 2019
4519b9c
Use :class: instead of :py:data:
GFernie Oct 30, 2019
554aad6
Revert "Do nothing on __del__"
GFernie Nov 23, 2019
be094b3
Add test for rolled file finalisation
GFernie Nov 23, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Fix SpooledTemporaryFile IOBase abstract 
One would assume that this class implements the IOBase abstract. As the
underlying file-like object is either io.BytesIO, io.StringIO, or a true
file object, this is a reasonable abstract expect and to implement.

Regardless, the behaviour of this class does not change much in the case
of the attribute being missing from the underlying file-like; an
AttributeError is still raised, albeit from one additional frame on the
stack trace.
  • Loading branch information
@GFernie
GFernie committed Aug 30, 2017
commit 83d6152028ad56d7eb0e4038c9c6d5c643908629
15 changes: 15 additions & 0 deletions Lib/tempfile.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -685,6 +685,9 @@ def __exit__(self, exc, value, tb):
def __iter__(self):
return self._file.__iter__()

def __del__(self):
Copy link

@pppery pppery Jun 6, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This shouldn't be added: deleting the SpooledTemporaryFile will null out the reference to the underline file, and therefor call its __del__

Copy link
Member

@vadmium vadmium Jun 9, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think if you inherit the default IOBase.__del__ implementation, it will call close and defeat any ResourceWarning that might otherwise be emitted. Perhaps it is better to make __del__ do nothing, or set it to object.__del__. [Seems that last option doesn’t exist.]

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree: the underlying file should not be explicitly deleted as this is not expected behaviour. I can imagine a situation where someone is deliberately holding a reference to the underlying file and they would not expect/want it to be closed until their own reference has fallen out of scope. I've changed the method to do nothing and added a docstring to reflect this.

Thanks for your feedback

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree: the underlying file should not be explicitly deleted as this is not expected behaviour

The doc says: """This function operates exactly as TemporaryFile() does, except [irrelevant differences]."""

And then, about TemporaryFile: """On completion of the context or destruction of the file object the temporary file will be removed from the filesystem."""

So it seems the underlying file should be deleted when the file object disappears.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The point is that IOBase implements a __del__ which has some side-effects. Those side effects are not desirable here. Any underlying buffers being wrapped, e.g. the TemporaryFile or the BytesIO, will be gc'd and handled as they should.

I think this implementation is correct.

return self._file.__del__()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, calling the __del__ method directly like that is not correct. __del__ is only to be called when all external references are gone. Instead I would suggest something like the following:

    def __del__(self):
        if not self.closed:
            import warnings
            warnings.warn('unclosed file %r' % (self,), ResourceWarning,
                          stacklevel=2, source=self)
            self.close()

(this is adapted from FileIO.__del__ in Lib/pyio.py)


def close(self):
self._file.close()

Expand Down Expand Up @@ -737,6 +740,12 @@ def newlines(self):
def read(self, *args):
return self._file.read(*args)

def readable(self):
return self._file.readable()

def readinto(self, b):
return self._file.readinto(b)

def readline(self, *args):
return self._file.readline(*args)

Expand All @@ -746,6 +755,9 @@ def readlines(self, *args):
def seek(self, *args):
self._file.seek(*args)

def seekable(self):
return self._file.seekable()

@property
def softspace(self):
return self._file.softspace
Expand All @@ -767,6 +779,9 @@ def write(self, s):
self._check(file)
return rv

def writable(self):
return self._file.writable()

def writelines(self, iterable):
file = self._file
rv = file.writelines(iterable)
Expand Down
29 changes: 29 additions & 0 deletions Lib/test/test_tempfile.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -982,6 +982,35 @@ def test_basic(self):
f = self.do_create(max_size=100, pre="a", suf=".txt")
self.assertFalse(f._rolled)

def test_iobase_abstract(self):
# SpooledTemporaryFile should implement the IOBase abstract
iobase_abstract = (
'close',
'closed',
'fileno',
'flush',
'isatty',
'read',
'readable',
'readinto',
'readline',
'readlines',
'seek',
'seekable',
'tell',
'truncate',
'write',
'writable',
'writelines',
'__del__',
)
f = self.do_create()
for attribute in iobase_abstract:
self.assertTrue(
hasattr(f, attribute),
'{} attribute missing'.format(attribute)
)
merwok marked this conversation as resolved.
Show resolved Hide resolved

def test_del_on_close(self):
# A SpooledTemporaryFile is deleted when closed
dir = tempfile.mkdtemp()
Expand Down