--- Begin Message ---
- To: submit@bugs.debian.org
- Subject: apt python library leak memory and file descriptors
- From: Petter Reinholdtsen <pere@hungry.com>
- Date: Tue, 22 Apr 2014 11:23:24 +0200
- Message-id: <2fllhuxsooj.fsf@diskless.uio.no>
Package: python-apt
Version: 0.8.8.2
Severity: important
Since I wrote the isenkram package, it have been plagued with a nasty
resourse leak. It leak memory and file descriptors every time some
hardware is inserted. The amount of leaked memory varies, but seem to
be 30-40 MiB every time. The amount of file descriptors depend on the
number of APT sources listed in /etc/apt/.
<URL: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=730704 >
is an example on what happen when the process run out of file
descriptors, and
<URL: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=719837 > is the
initial report about leaking memory.
I've been able to track down the problem to the python-apt library, and
the following test program demonstrate the leak:
#!/usr/bin/python
import apt
while True:
cache = apt.Cache()
cache.open(None)
When left alone running for a while, it crashes like this:
% ./apt-leak.py
Traceback (most recent call last):
File "./apt-leak.py", line 6, in <module>
cache.open(None)
File "/usr/lib/python2.7/dist-packages/apt/cache.py", line 147, in open
self._records = apt_pkg.PackageRecords(self._cache)
SystemError: E:Could not open file /var/lib/apt/lists/security.debian.org_dists_wheezy_updates_contrib_binary-amd64_Packages - open (24: Too many open files), E:Could not open file /var/lib/apt/lists/security.debian.org_dists_wheezy_updates_main_binary-amd64_Packages - open (24: Too many open files), E:Could not open file /var/lib/apt/lists/ftp.skolelinux.org_skolelinux_dists_wheezy-test_local_binary-amd64_Packages - open (24: Too many open files), E:Could not open file /var/lib/apt/lists/http.debian.net_debian_dists_wheezy-backports_main_i18n_Translation-en - open (24: Too many open files), E:Could not open file /var/lib/apt/lists/http.debian.net_debian_dists_wheezy-backports_main_binary-amd64_Packages - open (24: Too many open files), E:Could not open file /var/lib/apt/lists/http.debian.net_debian_dists_wheezy_non-free_i18n_Translation-en - open (24: Too many open files), E:Could not open file /var/lib/apt/lists/http.debian.net_debian_dists_wheezy_main_i18n_Translation-en - open (24: Too many open files), E:Could not open file /var/lib/apt/lists/http.debian.net_debian_dists_wheezy_contrib_i18n_Translation-en - open (24: Too many open files), E:Could not open file /var/lib/apt/lists/http.debian.net_debian_dists_wheezy_contrib_binary-amd64_Packages - open (24: Too many open files), E:Could not open file /var/lib/apt/lists/http.debian.net_debian_dists_wheezy_non-free_binary-amd64_Packages - open (24: Too many open files), E:Could not open file /var/lib/apt/lists/http.debian.net_debian_dists_wheezy_main_binary-amd64_Packages - open (24: Too many open files)
%
Am I using the library wrong (ie should I do something to release the
resources when I am done with the cache), or is it a bug in the library
leaking memory and file descriptors?
Setting severity to important. I guess one could argue that it should
be critical because it causes isenkram to fail, but that severity seemed
a bit high when I do not know if I am using the library wrong or not.
--
Happy hacking
Petter Reinholdtsen
--- End Message ---
--- Begin Message ---
- To: 745487-close@bugs.debian.org
- Subject: Bug#745487: fixed in python-apt 0.9.3.12
- From: Julian Andres Klode <jak@debian.org>
- Date: Tue, 23 Jun 2015 19:32:08 +0000
- Message-id: <E1Z7TvU-0006aw-Si@franck.debian.org>
Source: python-apt
Source-Version: 0.9.3.12
We believe that the bug you reported is fixed in the latest version of
python-apt, which is due to be installed in the Debian FTP archive.
A summary of the changes between this version and the previous one is
attached.
Thank you for reporting the bug, which will now be closed. If you
have further comments please address them to 745487@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.
Debian distribution maintenance software
pp.
Julian Andres Klode <jak@debian.org> (supplier of updated python-apt package)
(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
Format: 1.8
Date: Tue, 16 Jun 2015 11:35:11 +0200
Source: python-apt
Binary: python-apt python-apt-doc python-apt-dbg python-apt-dev python-apt-common python3-apt python3-apt-dbg
Architecture: source amd64 all
Version: 0.9.3.12
Distribution: jessie
Urgency: medium
Maintainer: APT Development Team <deity@lists.debian.org>
Changed-By: Julian Andres Klode <jak@debian.org>
Description:
python-apt - Python interface to libapt-pkg
python-apt-common - Python interface to libapt-pkg (locales)
python-apt-dbg - Python interface to libapt-pkg (debug extension)
python-apt-dev - Python interface to libapt-pkg (development files)
python-apt-doc - Python interface to libapt-pkg (API documentation)
python3-apt - Python 3 interface to libapt-pkg
python3-apt-dbg - Python 3 interface to libapt-pkg (debug extension)
Closes: 745487 750189 751770
Changes:
python-apt (0.9.3.12) jessie; urgency=medium
.
[ Julian Andres Klode ]
* apt/cache.py: Work around a cyclic reference from Cache to its methods
(Closes: #745487)
* python/arfile.cc: LFS: Use long long instead of long for file sizes
* python/arfile.cc: Do not allow files larger than SIZE_MAX to be mapped
* python/tarfile.cc: LFS: Handle too large file
* apt.debfile: Fix splitting of multi-lines Binary fields in dsc files
(Closes: #751770)
* apt/debfile.py: Arch-qualify in compare_to_version_in_cache()
(Closes: #750189)
.
[ Michael Vogt ]
* Fix apt.Package.installed_files for multi-arch packages (LP: #1313699)
Checksums-Sha1:
23b012f8286b24ad10ef43561f950ba1a5f8eff9 2234 python-apt_0.9.3.12.dsc
4bc6693dc97b89c610bb1268a3fefc30e9eda2fb 309896 python-apt_0.9.3.12.tar.xz
b4df7d0073ad42e97f22d8c47d3be79f64cef5fa 170568 python-apt_0.9.3.12_amd64.deb
aca170f3425e1242dd82787db2409a834e02b1cf 152336 python-apt-doc_0.9.3.12_all.deb
5981ee9dd8852a48008ec735e5dc369f847f6597 1570586 python-apt-dbg_0.9.3.12_amd64.deb
ca5a86c1b89984b29507be9edfc39ce0cd59c0d4 7444 python-apt-dev_0.9.3.12_all.deb
54181af434918d49c3825e00352fccead5f52ec9 89888 python-apt-common_0.9.3.12_all.deb
afc630ac10d99906663b1f7666f82a70c8c4ffef 168648 python3-apt_0.9.3.12_amd64.deb
7d930a5829b0f462d8dbc82fa782005241c17666 1584930 python3-apt-dbg_0.9.3.12_amd64.deb
Checksums-Sha256:
6bdbdd663556269cd5b8a90f0e2155456b0d2b50cf2baf0aac3606dc5a053e82 2234 python-apt_0.9.3.12.dsc
0a889130fd643be9cc3b4d972819bcb2ba9cc70b923077d4210c297aba07843b 309896 python-apt_0.9.3.12.tar.xz
7fb490a0514d3251dd80b03e4e118ddc512180a5831c4074c75052327bef5467 170568 python-apt_0.9.3.12_amd64.deb
040c1f97549f107d5543fa2b9053570483f2c1af8d054dc101199fd3a615f7c8 152336 python-apt-doc_0.9.3.12_all.deb
98ada68ee23ef604e38c095640fa20652d07dae05e35d154b9d80b158b4d4487 1570586 python-apt-dbg_0.9.3.12_amd64.deb
27e083d0ae16436d5cb411f4441e42601251f02cc508e2bd51859f2d1d688d4c 7444 python-apt-dev_0.9.3.12_all.deb
ae1c9e04a00f73f087c17370d563e53179d91824faaf2367374591ff7196e6be 89888 python-apt-common_0.9.3.12_all.deb
18a49d0bf33133d2ccbf644a6f6b0bb92ac301053675d1bf4a76d8cef2ab8b80 168648 python3-apt_0.9.3.12_amd64.deb
73bc3ead4e8020c34bb73f4a1899e66c147e383dda46f353385a708cb8d4c81f 1584930 python3-apt-dbg_0.9.3.12_amd64.deb
Files:
99af948d30a167c79b1c4c6fcdb569bc 2234 python standard python-apt_0.9.3.12.dsc
0b5d87d5fc854f427dec4ee0977bf677 309896 python standard python-apt_0.9.3.12.tar.xz
73d902f1e7b46423a5ebfeeafe5c6dea 170568 python standard python-apt_0.9.3.12_amd64.deb
303f9e9b4a9d330a8f93ccde0e4bcbc8 152336 doc optional python-apt-doc_0.9.3.12_all.deb
54ad2d4121d5245a2ed8f8cc997c838d 1570586 debug extra python-apt-dbg_0.9.3.12_amd64.deb
70f8b7f74a264833acabd948d3642c8f 7444 python optional python-apt-dev_0.9.3.12_all.deb
e9270a43f98a0fd01605b19fc0ea81ac 89888 python optional python-apt-common_0.9.3.12_all.deb
ccebfc569cf27cd164cbd1325566163a 168648 python optional python3-apt_0.9.3.12_amd64.deb
bd6c1be58ad1da616bada1339ec4a523 1584930 debug extra python3-apt-dbg_0.9.3.12_amd64.deb
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iQIcBAEBCgAGBQJVhcoMAAoJENc8OeVlgLOGc7IP/1Cw33oZGv2VXGwCWjGMcbjf
Oc4Xs9CvMxOYDvPyH2h74LxDpoiv0KryUBnC4ppmDWiLU45nR4KG8PySg0q+KeFD
S/jrU3ICdW/FcdK+8a9ZbS3rPUwC0/N62iH3Hr9jUeHQHGbmKDphjkuTiF1jAoQg
YVRzg3L5WwyATGqvTlrgpgCpThbYvm8Ww/2nyTeuu6AHXsqrjF+fuP+pH9N19f5u
tyIwQQkSKJkJ37VgZcpJe4g01vAtPfDb8QenkdMHRkV82atS3PgHxeuWMiT5gzKj
Vdvs7szkVU7D7o8fLAwFqssTRoYTcfKmKqBdHITuJICwGFAKRcE1nDQORQDGNCfu
vcRb2w7QkIVX73bdB+/OLon5+lxde7G/mfY9L8OAAW1IPlm7rcH2qShjxXwatStw
rWf8PlmFt5p6D7WoPdi556fuKsisM4GZhVqrf08qpNNnnzm+4CIa8uFIZSIjocDi
nDcg6q8yuHB9VD/xejvQ5cSnBFZOE/B16OJLcb+XYIRLIwC95N6KrlgNUEoHC6yb
35VLXY/OAi9khpr908tbTXBd0j1bKa4vThTbVUkzGFDH6ocB7Xh8ApBwNcGR6B1r
aybQ2xHF6VF9CVc5OISleThCtSN7sqG1GfnF2H9RbLZj8JMBRctFm2KXkuGUFFhx
FDBFtRBGfzIeeYCwdEaa
=k+Ot
-----END PGP SIGNATURE-----
--- End Message ---