Bug#1030743: help is welcome about the 'source-is-missing' error with the cherrytree packaging

[Andrius Merkys <merkys@debian.org>]

Hello,

On Mon, 15 May 2023 15:54:23 +0300 Andrius Merkys <merkys@debian.org> wrote:
> On 2023-05-12 21:34, Patrice Duroux wrote:
> > What is astonishing me is that the Perl code of lintian uses calls to encode_utf8/decode_uf8 in many places of different modules instead of fixing utf8 IO once in each.
> > I finally got lost in translation. 😄
>
> There might be reasons for such usage we are not aware of. Nevertheless 
> lintian should output correct Unicode and use it for override matching. 
> I suspect there is double encoding problem somewhere, as when I replace 
> line 74 of /usr/share/lintian/lib/Lintian/Pointer/Item.pm with
>
>      my $text = decode_utf8($self->item->name);
>
> I no longer have either source-is-missing nor mismatched-override.

I attempted rebuilding lintian with this change and got a bunch of test 
failures:

> Test Summary Report
> -------------------
> 
debian/test-out/eval/checks/documentation/manual/manpage-errors-from-man/generic.t 
                                                 (Wstat: 256 (exited 1) 
Tests: 1 Failed: 1)
>   Failed test:  1
>   Non-zero exit status: 1
> 
debian/test-out/eval/checks/documentation/manual/manpages-general/generic.t 
                                                        (Wstat: 256 
(exited 1) Tests: 1 Failed: 1)
>   Failed test:  1
>   Non-zero exit status: 1
> 
debian/test-out/eval/checks/documentation/manual/surplus-manpage/generic.t 
                                                         (Wstat: 256 
(exited 1) Tests: 1 Failed: 1)
>   Failed test:  1
>   Non-zero exit status: 1

I cannot see how the above are related to my change.

> debian/test-out/eval/checks/files/names/files-general/generic.t 
                                                              (Wstat: 
65280 (exited 255) Tests: 0 Failed: 0)
>   Non-zero exit status: 255
>   Parse errors: Bad plan.  You planned 1 tests but ran 0.
> debian/test-out/eval/checks/files/names/legacy-filenames/generic.t 
                                                              (Wstat: 
65280 (exited 255) Tests: 0 Failed: 0)
>   Non-zero exit status: 255
>   Parse errors: Bad plan.  You planned 1 tests but ran 0.
> 
debian/test-out/eval/checks/files/names/national-encoding-in-orig/generic.t 
                                                        (Wstat: 65280 
(exited 255) Tests: 0 Failed: 0)
>   Non-zero exit status: 255
>   Parse errors: Bad plan.  You planned 1 tests but ran 0.
> 
debian/test-out/eval/checks/files/unicode/trojan/exe-vs-gif-in-patched-filename/generic.t 
                                          (Wstat: 256 (exited 1) Tests: 
1 Failed: 1)
>   Failed test:  1
>   Non-zero exit status: 1
> Files=1486, Tests=63476, 1441 wallclock secs (10.63 usr  6.15 sys + 
4448.39 cusr 876.07 csys = 5341.24 CPU)
> Result: FAIL

These seem to show that lintian intentionally omits decode_utf8() to 
catch and process various ill-formatted utf8 octet sequences. 
Nevertheless I would suggest calling decode_utf8() in try/catch block 
reverting to the old behavior should the decoding fail. I will try that 
and report how it goes.

Best,
Andrius