[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#748962: texlive-bastexlive-base: fmtutil-sys --all may not return error exit code under low free space condition.



Hi,

This is going to be my last post here.

I posted a very lengthy post to tex-live mailing list to
follow up on my finding here.

http://tug.org/pipermail/tex-live/2014-May/thread.html

I posted a proper patch there. It is near the end since it was posted
a couple of days ago.

Good news is that I tested the modified fmtutil.sh under two Debian
linux systems by making "/var" partition full (totally full, and
partially full by creating a large file under /var/tmp), and I am
confident that the modified fmtutil.sh works much better:

 - it prints the warning and error messages much more reliably than the
   current version.
  (It succeeded under the near full/full conditions of my testing.)

 - it returns proper error code in each failure case.

I believe I have covered all the corner cases.

Well, Famous Last Words.
So, I don't claim it is perfect, but it is much better.

TWO MAJOR BUGS UNCOVERED

The two major bugs discovered after my last post to Debian bug
tracking are follows. I was not aware of the second problem below and
so was wondering in detail if the tricky syntax of "verbose" could be
the cause of the strange behavior in previous postings.  (It still
could be, but even after my modification to "verbose", the following
two bugs showed up..)

It turned out that a couple of commands may have been responsible for

 - the failure of flushing the warning and error messages at the end.
  (A command was not properly checked and so it triggered trap
   processing bypassing "byebye" processing that would have flushed
   the messages.)
   It would still return exit code 1. Hmm...

 - the failure to return the error code properly. An "mv" command was
   executed in the following manner.

   if mv .... < /dev/null
   then
     ...
     ...[lengthy processing]
     ...
   fi

   Note that there was NO "else" block to handle the error of "mv"
   (say, due to full file system) by calling "log_failure", etc.
   Possible error caused by full file system, etc. was used up by "if"
   checking itself, and no error exit code was ever produced.  In this
   case, error code is forgotten!  Bad, bad, bad.

(The second problem did not show up on a linux installation, but
it became apparent on a different linux installation. That is how I
noticed the bug.)

In the proposed patch to tex-live@tug.org, I reverted the test
condition of "if mv ... < /dev/null" to "if ! mv ... < /dev/null" so
that the error processing block comes first before the lengthy normal
processing. It is much easier this way to understand the structure of
the code.
Last time I checked, the very long normal processing block made me
lose the fact that there was no matching "else" block. I mistook
another "else" block for the missing one.

I will monitor tex-live mailing list although, as noted by Norbert,
people seem to be very busy preparing 2014 TeXLive image.

We have to wait for the stable upstream version.  But it will be great
if we can incorporate the gist of changes into Debian packages
eventually. ("set -e" was not in TeXlive 2013, it is a Debian
addition, for example.)

TIA

PS: In the end, rewriting fmtutil.sh in Perl, C, java, or whatever
reasonably powerful programming language might be
better because the loose standards regarding the
trap handling of the failure of command when "set -e" is issued.

I see "byebye called" in the posted log to tex-live
mailing list: the log was produced under almost full /var partition condition.
E.g.:
 [...]
'mv tex.fmt /var/lib/texmf/web2c/tex/tex.fmt' failed
'mv dviluatex.fmt /var/lib/texmf/web2c/luatex/dviluatex.fmt' failed
byebye  called
We had error(s).

If somehow subtle change of trap handling occurs due the differences of
shell versions, some installation failure may see

byebye 1 called

instead. (This is how "byebye" is called in my modified version from
trap processing.)  Such a line will give us the clue of the different
way of trap handling under the particular shell used.


Thank you again.


2014-05-23 15:01 GMT+09:00 Norbert Preining <preining@logic.at>:
Hi,

> second
> execution would print error message to console (unlike the re-directed
> execution on LHS of "||").

Yes, that was quite surely his intention ;-) But you can also try to
ask him ;-)

Anyway, rewriting fmtutil in perl is on my personal todo list.
I have already rewritten (based on previous work) updmap in perl,
and at the end I want to get rid of the separation c-code for Windpws
and shell code for the rest of fmtutil, by replacing it with all
perl code.

At that point, my aim is also to have a decent error handling ;-)

> You are right. My point here was that the script seems to "forget" that the
> error was there during the re-production of the whole set of .fmt files, and
> did not return error code (my casual reading. Maybe I am wrong.) at the end.
> I think it should.

Actually, that is a known problem. fmtutil does not do *any* proper
error handling. We are well aware of that, one other reason why I want
to rewrite it in perl ;-)

> Thank you again. I noticed by reading tex live mailing list that
> you and a few others are at the core of maintaining tex live.

Indeed, indeed. Although I also maintain the Debian packages, my
main part is in upstream (now, not originally).

> (Yeah, I even tried to install OLD ASCII ptex on my own after encountering

Oh, bad bad idea. There is Okumura-sensei's TeX Q-A and there was
ptetex and ptexlive, but I incorporated all the pTeX stuff and
support packages over the last years, with lots of support from
many of the Japanese TeX community, into TeX Live. Before it was
a drag to get pTeX (upTeX etc) running.

> I wonder what people would do when old TeX files that were once typeset
> using the vertical writing feature supported by ASCII's ptex needed to be
> modified afresh and typeset again.

That is not a problem. You are speaking about the *format*.
The format can be regenerated. There has been a problem 6 years
ago when the released version we had in Debian just fell out
of these 5 years, and we had to patch the latex.ltx, but that
has no influence. You can still run old code and get the same
output (unless you use many packages that have changed
considerably - if you *reall* want 100% reproducibility in the
future, you have to stay with *plain* ptex and not use latex
and add-on packages!).

> We may need a time machine (of a sort) where the old typeset software is
> kept intact so that in the future the old documents that need modification

We have mirrors of old TeX Live's, so you can go back at least with
TeX Live.


A, one more remark: TeX Live is in the hot phase of pre-release,
so don't expect toooo much activity and response *now* on fmtutil
requests, as everyone of (the few of us) is busy for the final
image burning etc etc.

Thanks for all your work on fmtutil, much appreciated.

Norbert

------------------------------------------------------------------------
PREINING, Norbert                               http://www.preining.info
JAIST, Japan                                 TeX Live & Debian Developer
GPG: 0x860CDC13   fp: F7D8 A928 26E3 16A1 9FA0  ACF0 6CAC A448 860C DC13
------------------------------------------------------------------------


Reply to: