enhance mark extraction to account for forced breaks within the material

base/changes.txt

@@ -6,6 +6,20 @@ to completeness or accuracy and it contains some references to files that are
 not part of the distribution.
 ================================================================================
 
+2023-11-14  Frank Mittelbach  <[email protected]>
+
+	* ltmarks.dtx:
+	Some internal commands renamed and extended.
+
+	Generalized the support for extracting marks to cover cases like multicols.
+
+	Renamed \__mark_update_structure:nn to \__mark_update_structure_from_material:nn
+
+	Added \__mark_get_marks_for_reinsertion:nNN
+
+	Added \ShowMarksAt for debugging (might not stay this way)
+
+
 2023-11-09 Yukai Chou  <[email protected]>
 	* clsguide-historic.tex, usrguide.tex:
 	Replace quotation with quote envs for zero para indent

base/doc/ltnews39.tex

@@ -145,6 +145,60 @@
 \section{Introduction}
 
 
+\section{Enhancements to the new mark mechanism}
+
+In June 2022 we introduced a new mark mechanism in
+\LaTeX{}~\cite[p.~76]{39:ltnews} that allows keeping track of multiple
+independent marks. It also  properly supports top marks, something that wasn't
+reliably possible with \LaTeX{} before.
+
+There was, however, one limitation: to retrieve the marks from the
+page data it was necessary to \cs{vsplit} that data artificially so
+that \TeX{} would produce split marks that the mechanism could then
+use. Unfortunately, \TeX{} gets very upset if it finds infinite
+negative glue (e.g., from \cs{vss}) within this data. This is not
+totally surprising because such glue would allow splitting off any
+amount of material as such glue would hide the size of it. \TeX{}
+therefore responds with an error message if it find such glue while
+doing a \cs{vsplit} operation (and it does so even if a later glue
+item cancels the infinite glue).
+
+To account for this, the code in 2022 attempted to detect this
+situation beforehand and if so did not do any splitting but, of
+course, it would then also not extract any mark information.
+
+In this release the approach has been changed and we always do a
+\cs{vsplit} operation and thus always get the right mark data
+extracted. While it is not possible to avoid upsetting \TeX{} in case
+we have infinite negative glue present, it is possible to hide this
+(more or less) from the user.\footnote{A note to \pkg{l3build} users
+that make use of its testing capabilities: the new mechanism
+temporarily changes \cs{interactionmode} and, for implementation
+reasons in \TeX{}, that results in extra newlines in the \texttt{.log}
+file, so instead of seeing \texttt{[1] [2]} you will see each on
+separate lines. This means that test files might show difference of
+that nature, once the code is active, and must therefore be
+regenerated as necessary.}  With the new code \TeX{} will neither stop
+nor show anything on the terminal. What we can't do, though, is to
+avoid that an error is written to the log file, but to make it clear
+that this error is harmless and should be ignored we have arranged the
+code so that the error message, if it is issued, takes the following
+format:
+\begin{verbatim}
+! Infinite glue shrinkage found in box being split.
+<argument> Infinite shrink error above ignored ! 
+l. ...  }
+\end{verbatim}
+Not perfect (especially the somewhat unmotivated \texttt{<argument>}),
+but you can only do so much if error messages and their texts are
+hard-wired in the engine.
+
+So why all this? There are two reasons: we do not lose marks in edge
+cases any longer and perhaps more importantly we are now also reliably
+able to extract marks from arbitrarily boxed data, something that
+wasn't possible at all before. This is, for example, necessary to
+support extended marks in \env{multicols} environments or extract them
+from floats, marginpars, etc.
 
 \section{New or improved commands}
 
@@ -209,19 +263,19 @@ \subsection{\pkg{verbatim}: \cs{verb} showed visible spaces}
 \githubissue{1160}
 
 
-\subsection{\pkg{multicol}: \cs{columnbreak} interfers with mark mechanism}
+\subsection{\pkg{multicol}: \cs{columnbreak} interferes with mark mechanism}
 
 The \pkg{multicol} package has to keep track of marks (from
 \cs{markright} or \cs{markboth}) as part of its output routine code
 and can't rely on \LaTeX{} handling that automatically. It does so by
-artifically splitting page data with \cs{vsplit} to extract the mark
+artificially splitting page data with \cs{vsplit} to extract the mark
 data. With the introduction of \cs{columnbreak} that code failed
 sometimes, because it was not seeing any mark that followed such a
 forced column break.
 
 This has now been corrected, but there is further work to do, because
-as of now \pkg{multicol} does not handle marks using the new mark
-mechanism at all.
+as of now \pkg{multicol} does not yet handle marks using the new mark
+mechanism\Dash see the discussion at the beginning of the newsletter.
 %
 \githubissue{1130}
 
@@ -253,10 +307,10 @@ \subsection{Correct definition of \cs{k}}
 %\newblock ISBN 0-201-52983-1.
 %\newblock Reprinted with corrections in 1996.
 %
-%
-%\bibitem{39:ltnews} \LaTeX{} Project Team.
-%  \emph{\LaTeXe{} news 1--39}.\\
-%  \url{https://latex-project.org/news/latex2e-news/ltnews.pdf}
+
+\bibitem{39:ltnews} \LaTeX{} Project Team.
+  \emph{\LaTeXe{} news 1--39}.\\
+  \url{https://latex-project.org/news/latex2e-news/ltnews.pdf}
 %
 %\bibitem{39:blueprint} Frank Mittelbach and Chris Rowley.
 %  \emph{\LaTeX{} Tagged PDF \Dash A blueprint for a large project}.\\