Re: can I help with pyvista packaging?

[Andreas Tille <andreas@an3as.eu>]

Hi Francesco,

please excuse my violation of the netiquette when forwarding private
mail to a public mailing list.  But this technical discussion should
simply not be hidden in private mailboxes.  I think the Debian Science
mailing list is appropriate for this.

Am Mon, Jan 29, 2024 at 07:14:46AM +0100 schrieb Francesco Ballarin:
> Hi,
> a few updates on trame:
> 
> 1) pyvista with trame depends on ipywidgets. It seems that
> python3-ipywidgets on unstable is very old, and much newer versions
> haven't been migrated from experimental to unstable in the last six
> months. I've sent an email to the uploader inquiring on the status of
> the package.

Maybe its a good thing to discuss this issue at
debian-python@lists.debian.org (keeping the actual uploaders Gorden and
Ximin in CC) to check out the blockers for the migration.  Finally we
need to make sure that we will not lag too much behind upstream.
 
> To be sure that the issue is sorted I'll wait working on trame until I
> have an answer from him.
> 
> 2) even before starting to work, it's clear that the chain of
> dependencies it's quite complex (at least for a beginner like me) due
> to the six interconnected packages that should be updated in sync (5
> for trame + 1 for pyvista).
> Furthermore, it's quite difficult to keep track of the minimum
> required version of the 5 trame dependencies, see for instance
> https://github.com/pyvista/pyvista/blob/main/pyproject.toml#L55
> https://github.com/Kitware/trame/blob/master/setup.cfg#L29
> and note how the minimum version numbers are different.

Sometimes its some solution to patch out these minimum versions.
The test suite will usually show whether lower versions might work
or not.  I experienced that upstream set some restrictions that
work for them but it turns out that they are to strict.
 
> Actions I envision to take:
> a] add a README.Debian that documents the right order in which to
> carry out the updates (for instance, trame-client must be the first,
> and pyvista the last)

About what packages README.Debian are you talking?  I think
README.Debian is not the right place since it is user oriented while you
want talk to other developers.  I sometimes drop some

   TODO: this and that

right into debian/changelog since I take it for granted that other
developers are reading this file.  Alternatively there is
debian/README.source or simply some debian/TODO.

> b] for a selected list of dependencies (e.g., trame ones), add a check
> with grep (or something similar) that the minimum version reported in
> debian/control is the same as the one in setup.cfg or pyproject.toml

I admit I do not fully understand what you mean.  Why not adding
versioned Build-Depends and let dpkg check the proper versions?
 
> Questions for you:
> * would b] be acceptable, or would it go against some rule/policy?
> * what's the best way to include b] into the build system? I guess I
> should be overriding one of the dh steps, but I am not sure which one.

I can't answer this question since I do not understand the suggestion.
 
> 3) a user that installs pyvista with pip must use "pip install
> pyvista[jupyter]" if they want to install trame and jupyter plotting.
> I think we should do something similar, otherwise a user installing
> python3-pyvista would have a lot of dependencies installed on their
> system, notably bits and pieces of jupyter, and they may not actually
> be interested in having them at all.

You need to decide between

   Recommends: python3-jupyter-WHATEVER
or
   Suggests: python3-jupyter-WHATEVER

depending what is really needed (maybe WHATEVER=core, or core as
Recommends + others as Suggests ... I simply have no knowledge about all
this).

> I would probably want to change debian/control to have a second
> package called python3-pyvista-jupyter, that depends on
> python3-pyvista and trame packages, so that the subset of users
> (Francesco, for sure ;) ) who want to use pyvista in jupyter can
> install python3-pyvista-jupyter, while the rest of the users will
> install python3-pyvista instead.

This sounds very sensible.
 
> Questions for you:
> * is that reasonable?

Yes.

> * how should one ask for an exemption to "Package Content" at
> https://ftp-master.debian.org/REJECT-FAQ.html? The
> python3-pyvista-jupyter would be empty, since its only goal would be
> to pull in more dependencies.

No.  Just create another binary package containing just the
dependencies.  That's fine.
 
Hope this helps
    Andreas.
 
> On Sun, Jan 28, 2024 at 6:58 AM Francesco Ballarin
> <francesco.ballarin@unicatt.it> wrote:
> >
> > (I dropped the bug from cc)
> >
> > Great, thanks!
> >
> > Can one of you guys start opening the following repos
> > =====
> > trame
> > trame-client
> > trame-server
> > trame-vtk
> > trame-vuetify
> > =====
> >
> > under the science team?
> >
> > If so, can you please manually add me to each one of those with the
> > highest possible gitlab permissions, so that I can tweak salsa CI?
> > (I am in the science group and therefore I can access every science
> > team repo, but only with the lowest gitlab permissions)
> >
> > It's probably going to take me a bit longer with trame, because I
> > don't have as much experience with it as with pyvista, but I'd like to
> > get started in the next few days.
> >
> > Cheers,
> > Francesco
> >
> > On Sat, Jan 27, 2024 at 11:34 PM Andreas Tille <andreas@an3as.eu> wrote:
> > >
> > > Hi Francesco,
> > >
> > > thanks a lot for the fix - the package is uploaded to new.
> > >
> > > Kind regards
> > >    Andreas.
> > >
> > > Am Sat, Jan 27, 2024 at 10:23:20PM +0100 schrieb Francesco Ballarin:
> > > > A first version is available at
> > > > https://salsa.debian.org/science-team/python-pyvista
> > > > in case you want to have a look before I/we change it from UNRELEASED
> > > > to unstable.
> > > >
> > > > autopkgtest passes.
> > > > test-crossbuild-arm64 is failing, but is allowed to do so, not sure if
> > > > it is relevant.
> > > >
> > > > As agreed, interactive plotting within jupyter is not considered for now.
> > > >
> > > > Cheers,
> > > > Francesco
> > > >
> > > >
> > > > On Sat, Jan 27, 2024 at 7:08 PM Andreas Tille <andreas@an3as.eu> wrote:
> > > > >
> > > > > Am Sat, Jan 27, 2024 at 06:28:14PM +0100 schrieb Francesco Ballarin:
> > > > > > OK Andreas, I'll push to master. Let me take the lead on that, and I'll come back to you and Drew with progress and questions.
> > > > >
> > > > > Perfectly fine for me.
> > > > >
> > > > > > I think I have some ideas on how to get started on the basic package.
> > > > > >
> > > > > > The full package (i.e., all optional components that one can install with "pip install pyvista[all]") will be much more complex, because it depends on trame, which comes split in five interdependent packages
> > > > > > ====
> > > > > > trame                    3.2.4
> > > > > > trame-client             2.11.2
> > > > > > trame-server             2.11.7
> > > > > > trame-vtk                2.5.8
> > > > > > trame-vuetify            2.3.1
> > > > > > ====
> > > > > > and who knows how many dependencies each one of those have.
> > > > >
> > > > > Sounds complex and I'd love to stay in background given that I need to
> > > > > care for >1000 packages.
> > > > >
> > > > > > Down the line I'd want to have that too, because that's what I actually need for plotting within jupyter notebooks in my libraries, see for instance
> > > > > > https://viskex.github.io/tutorials/01_introduction/tutorial_plot_subdomains_3d_dolfinx.html
> > > > > > but let's start with a less ambitious goal ;)
> > > > >
> > > > > +1
> > > > >
> > > > > > I think that the error you see is because python3-vtk9 is only built for python 3.11, but unit tests are getting run with python 3.12.
> > > > > > At the moment, I'd like to disable that part adding an empty override_dh_auto_test-indep and run tests with autopkgtest only, so that I can have a clearer idea on the difference between dependencies required for building the package vs dependencies required for testing it, because I know from experience that there are a few packages that are only needed for testing pyvista.
> > > > >
> > > > > Perfectly fine for me.  Feel free to turn the RFP in ITP and I can
> > > > > sponsor the upload if needed.
> > > > >
> > > > > Thanks a lot for your help
> > > > >     Andreas.
> > > > >
> > > > > --
> > > > > http://fam-tille.de
> > > > [http://static.unicatt.it/ext-portale/5xmille_firma_mail_2023.jpg] <https://www.unicatt.it/uc/5xmille>
> > > >
> > > >
> > >
> > > --
> > > http://fam-tille.de
> [http://static.unicatt.it/ext-portale/5xmille_firma_mail_2023.jpg] <https://www.unicatt.it/uc/5xmille>
> 
> 

-- 
http://fam-tille.de