Re: pyyaml 6
On November 2, 2022 8:35:35 AM UTC, Gordon Ball <gordon@chronitis.net> wrote:
>On 19/10/2022 22:30, Gordon Ball wrote:
>> On 09/10/2022 21:39, Gordon Ball wrote:
>>> On 07/10/2022 01:13, Timo Röhling wrote:
>>>> Hi Gordon,
>>>>
>>>> * Gordon Ball <gordon@chronitis.net> [2022-10-07 00:10]:
>>>>> * Upload to unstable and see what breaks?
>>>>> * Request an archive rebuild with this version and see what breaks?
>>>>> * File bugs against all likely affected packages with a fixed date for
>>>>> an upload?
>>>>> * Wait until after the freeze?
>>>> Considering that PyYAML has been issuing a YAMLLoadWarning since
>>>> version 5.1 (i.e. September 2019), I would expect that many (most?)
>>>> reverse dependencies have been fixed, and anything that still breaks
>>>> is probably in dire need of maintenance anyway.
>>>
>>> Yes, that's a good point.
>>>
>>> I've done some more investigation of reverse-depends and looking for
>>> likely bad patterns with codesearch.d.n, and the set of packages which
>>> A) appear to contain plain `yaml.load(f)` or equivalent, where `yaml` is
>>> pyyaml and B) actually depend or build-depend on `python3-yaml` is
>>> smaller than I feared. This is what I came up with:
>>>
>>> ```
>>> ansible # confirm - in ansible_collections/community/general/plugins
>>> buildbot # confirm - in porttostable
>>> ceph # confirm, in civetweb/third_party/duktape, src/test/crimson
>>> docker-compose # confirm, in tests
>>> dose3 # confirm, in scripts
>>> elasticsearch-curator # confirm, in test/unit
>>> etm # confirm, in etmTk/data
>>> ganeti # confirm, in qa, test/py
>>> gnocchi # confirm, in gnocchi/gendoc
>>> gr-dab # confirm, in python/app
>>> jeepyb # confirm, in cmd/notify_impact
>>> knitpy # confirm, in knitpy
>>> labgrid # confirm, in remote/coordinator
>>> lecm # confirm, in lecm/configuration
>>> lirc # confirm, in lirc/database
>>> lqa # confirm, in lqa_api/job
>>> open-adventure # confirm, in tests
>>> owslib # confirm, in ogcapi
>>> policyd-rate-limit # confirm, in config, tests
>>> python-aptly # confirm, in publisher
>>> python-canmatrix # confirm, in canmatrix/formats/yaml
>>> python-multipart # confirm, in multipart/tests
>>> python-pybedtools # confirm, in test, contrib
>>> python-seqcluster # confirm, in install
>>> python-tempestconf # confirm, in config_tempest/profile
>>> qcat # confirm, in adapters
>>> qcengine # confirm, in config
>>> refstack-client # confirm, in refstack_client
>>> relatorio # confirm, in templates/chart
>>> ripe-atlas-tools # confirm, in tools/settings
>>> ros-genpy # confirm, in test
>>> ros-rosinstall # confirm, in test, src/rosinstall
>>> ros-rosinstall-generator # confirm, in test, src/rosinstall_generator
>>> spades # confirm, in assembler/src/test/teamcity,
>>> assembler/src/projects/mts, assembler/src/spades_pipeline
>>> xrstools # confirm, in WIZARD
>>> zlmdb # confirm, in _database
>>> ```
>>>
>>> I don't know if all these codepaths are actually ever reached, or tests
>>> ever run, so the packages won't necessarily break (or the breakage is
>>> very minor, such as a single community plugin in ansible). I don't
>>> guarantee there are no omissions from the list though.
>>>
>>> There is a significantly longer list of packages which appear to contain
>>> a use of yaml.load _somewhere_, but it is usually in some
>>> maintainer/release script, or an optional path somewhere, and the
>>> package itself doesn't [build-]depend on python3-yaml.
>>>
>>
>> I've filed bugs for (most) of the packages listed above (some were dropped on review because they weren't in main, or the affected file appeared to be unused at build time and not installed). I've had a couple of acknowledgements/fixes already.
>>
>> A number of affected package look very sparsely used and/or maintained, and I doubt are likely to be fixed. However, of the packages above, only two have autopkgtests which fail on experimental migration, so I wouldn't be surprised if some are not already broken by other past changes.
>>
>> I propose to wait ~2 weeks until the beginning of November and upload pyyaml 6 to unstable then.
>>
>
>pyyaml 6 has now landed in unstable. About half the bugs I filed have been resolved, and there is only one package (ganeti) with autopkgtests blocking migration.
>
There were two more, but I retried the migration reference jobs and they failed (which makes them no longer block since they are no longer a regression).
Ganeti can't currently be fixed since it's unbuildable. It's only due to an archive hiccup that it's even in Testing. I was planning on asking the Release Team to ignore it, so pyyaml can transition (worst case it's stuck until ganeti gets removed from Testing in a week and a half).
Scott K
Reply to: