Control: clone -1 -2 Control: retitle -2 python3-apt: add support for non-gzip compressed file objects Control: severity -2 wishlist On Thu, Nov 02, 2023 at 01:18:23PM +0100, Cyril Brulebois wrote: > Cyril Brulebois <kibi@debian.org> (2023-11-02): > > Today I had a few more minutes to spend on this, so here's a little > > debugging session. My main system is still bullseye, but the same tests > > in a bookworm chroots fail the same way. > > “But maybe it's a bug in the lzma library?” one might ask. > > Adding a bzip2 test between gzip and lzma leads to the following, again > on both bullseye and bookworm (after creating a Test.bz2/Packages.bz2 > from one of the other files): > > With bug-932491-aa.py (bug-932491-a.py + bzip2): > > $ ./bug-932491-aa.py Test > gz == bz: True > gz == xz: True > gz: section 1 size: 29 > gz: section 1 keys: ['Package', 'Desc'] > gz: section 2 size: 47 > gz: section 2 keys: ['Package', 'Desc'] > Traceback (most recent call last): > File "/home/kibi/tmp/./bug-932491-c.py", line 37, in <module> > tf_bz.step() > apt_pkg.Error: E:Unable to parse package file (1) > > $ ./bug-932491-aa.py Packages > gz == bz: True > gz == xz: True > gz: section 1 size: 1281 > gz: section 1 keys: ['Package', 'Version', 'Installed-Size', 'Maintainer', 'Architecture', 'Depends', 'Pre-Depends', 'Description', 'Homepage', 'Description-md5', 'Tag', 'Section', 'Priority', 'Filename', 'Size', 'MD5sum', 'SHA256'] > gz: section 2 size: 585 > gz: section 2 keys: ['Package', 'Version', 'Installed-Size', 'Maintainer', 'Architecture', 'Pre-Depends', 'Suggests', 'Description', 'Homepage', 'Description-md5', 'Tag', 'Section', 'Priority', 'Filename', 'Size', 'MD5sum', 'SHA256'] > bz: section 1 size: 1410 > Segmentation fault > > With bug-932491-bb.py (bug-932491-b.py + bzip2): > > $ ./bug-932491-bb.py Test > gz packages: 2 > Traceback (most recent call last): > File "/home/kibi/tmp/./bug-932491-bb.py", line 26, in <module> > for stanza in tf_bz: > apt_pkg.Error: E:Unable to parse package file (1) > > $ ./bug-932491-bb.py Packages > gz packages: 50771 > Traceback (most recent call last): > File "/home/kibi/tmp/./bug-932491-bb.py", line 27, in <module> > bz_packages.append(stanza['Package']) > ~~~~~~^^^^^^^^^^^ > KeyError: 'Package' > > > It looks like we might be getting chunks of different sizes depending on > the underlying file objects, and some buffering/seeking code is buggy on > the apt_pkg side? You are literally just fuzzing the tagfile parser with compressed streams, there is no decompression going on. We don't talk to the the file-like object you pass to at all, we just call it's fileno() method to get the underlying file descriptor, and then apt's gzip support reads from that, and that works automagically because zlib just passes through uncompressed content. If you want it to automatically guess the compressor, you can do that by passing a filename with the right file extension. For existing open files, the right way to approach this arguably is o provide apt_pkg.FileFd bindings to the FileFd class such that you can specify a decompressor, and then parse the FileFd to TagFile. But I think this is a different issue than the segfault because we probably still should not be segfaulting on fuzzing with random data like you do, we probably ought to error out at some point. -- debian developer - deb.li/jak | jak-linux.org - free software dev ubuntu core developer i speak de, en
Attachment:
signature.asc
Description: PGP signature