enhance mark extraction to account for forced breaks within the material #1177

FrankMittelbach · 2023-11-10T15:12:20Z

Internal housekeeping

Status of pull request

Ready to merge

Checklist of required changes before merge will be approved

Test file(s) added
Version and date string updated in changed source files
Relevant \changes entries in source included
Relevant changes.txt updated
Rollback provided (if necessary)?
ltnewsX.tex (and/or latexchanges.tex) updated

…ial (WIP)

josephwright

Changes.txt entry?

FrankMittelbach · 2023-11-12T22:02:08Z

Changes.txt entry?

sure, but too early for that in my opinion. That's why it says under development and nothing is checkmarked above (I'm one of those folks who are really using Will's checkmark list :-) ). So right now all I'm really interested in is whether the logical seems to be right.

josephwright · 2023-11-13T08:07:46Z

Changes.txt entry?

sure, but too early for that in my opinion.

Obviously we view this a bit differently: I thought the changes.txt had to be updated when you edit the source, even if that means we have a series of edits to changes.txtduring a PR. I am though happy if you want to leave until the end.

FrankMittelbach · 2023-11-13T09:59:53Z

@josephwright I agree that doing this in parallel is useful most of the time. However, in this case I expect many more changes (and possibly alternative implementation), so I'm going lightweight on programmer-level docs and do them when things are a bit more settled.

Use it to also provide \__mark_get_marks_for_reinsertion:nNN for use in multicols (and elsewhere) eventually

FrankMittelbach · 2023-11-13T18:25:45Z

I altered and extended the implementation in anticipation what is necessary for multicols and/or in general for extracting marks out of box material to reinsert them afterwards. This is now closer to what I think is needed so change entries will come up soon @josephwright :-)

muzimuzhi · 2023-11-13T18:44:47Z

To reduce the ambiguity, perhaps a new item can be added to section "Status of pull request", in pull request template, to indicate that the PR author is aware of the missing changes.txt entries and friends.

For example,

- In early stage (`changes.txt` entries and friends might be missing)

FrankMittelbach · 2023-11-13T18:59:18Z

To reduce the ambiguity, perhaps a new item can be added to section "Status of pull request", in pull request template, to indicate that the PR author is aware of the missing changes.txt entries and friends.

sorry don't get it. Of course, I'm aware (and so should everybody else) since I didn't checkmark those item nor did I put n/a (not applicable) into the checkmark box. So it is all there at the very top. Why should a different string from

Feedback wanted
Under development

plus the checkmark list, provide any additional info? It is more that often people do not use the checkmark list, but that is not helped I think by asking for providing a different status string.

davidcarlisle · 2023-11-14T08:58:56Z

base/ltmarks.dtx

+%    executes \verb/#1{#3}/ to do something with the marks, e.g., to
+%    update the mark structure for the region.
+%  
+%    If it finds infinite negative glue it generates an error message


could you get any other tex error at this point, since the material is already a vlist?If not I suppose you could consider setting interactionmode and restoring it to suppress this error?

Interesting thought, but I don't think there is a way to detect then that the error has happened and it is an error, ie the user is losing the marks and so I think they should learn that this is the case. But perhaps if it is enough if that is something you can find in the .log? It would certainly has the advantage that then the processing doesn't stop if they can't avoid the infinite negative glue. Maybe.

hmm, back to the drawing board perhaps. If we run with \interactionmode 0, the error only appears in the log and we actually get to the marks even if there is real infinite negative glue (because TeX makes it finite).

In other words, if we accept that we may see

! Infinite glue shrinkage found in box being split. \__box_dim_eval:n ...box_dim_eval:w #1\scan_stop: l. ... } The box you are \vsplitting contains some infinitely shrinkable glue, e.g., `\vss' or `\vskip 0pt minus 1fil'. Such glue doesn't belong there; but you can safely proceed, since the offensive shrinkability has been made finite. ! Infinite glue shrinkage found in box being split.

in the log then we can get at the marks in all cases, which seems better then issuing an error or a warning that it doesn't work.

If we want to we could additionally do

\wlog{Any immediately following 'Infinite glue shrinkage' error is harmless and can be ignored}

but I'm not sure this is better, because most of the time there will be none and we only end up with these extra lines. Thoughts?

yes wlog would be confusing I think, for the reason you say.

You could use \ignore_infinite_shrinkage_error defined as \let\ignore_infinite_shrinkage_error\__box_dim_eval:n perhaps....

??? not sure I understand the suggestion, could you perhaps simply implement it? the branch is up to date and I don't intend to alter it right now

\expandafter\def\csname Ignore any infinite glue shrinkage error on this line\endcsname{ \setbox0\vbox{aaa \vss bbb}% \setbox2=\vsplit0 to \maxdimen } \expandafter\def\expandafter\foo\expandafter{% \csname Ignore any infinite glue shrinkage error on this line\endcsname} { \interactionmode=0 \foo }

makes a log

! Infinite glue shrinkage found in box being split. \Ignore any infinite glue shrinkage error on this line ... l.13 \foo

I could probably try adding something like this later this evening

base/ltmarks.dtx

car222222

Lots of commas (and a few "uncommas"), etc.

base/ltmarks.dtx

typos and corrections

code can be simplified if we go this way.

davidcarlisle · 2023-11-14T19:27:08Z

I pushed a change change the glue shrink message


  ! Infinite glue shrinkage found in box being split.
! \__box_dim_eval:n ...box_dim_eval:w #1\scan_stop: 

--- 113,126 ----

  ! Infinite glue shrinkage found in box being split.
! <argument> \Ignore infinite glue shrinkage error

The test file xmarks-009 fails showing this diff, I didn't update it, to make it easier to back out this commit if you don't like it, and to highlight the diff.
It requires a bit of balancing to get the string at the right expansion level but it seems to work here without breaking the other ltmarks tests.

documentation not yet updated and old code still mostly inside commented out

…an be triggered from ltmarks.dtx?

FrankMittelbach · 2023-11-14T22:41:01Z

The test file xmarks-009 fails showing this diff, I didn't update it, to make it easier to back out this commit if you don't like it, and to highlight the diff. It requires a bit of balancing to get the string at the right expansion level but it seems to work here without breaking the other ltmarks tests.

I like the idea and it makes the code much simpler --- and we get correct results even if there is shrinkage in the material which is great. I altered the txt a bit (not sure if it is for the better). Right now the test suite fails in a very strange way though and I don't understand why.

davidcarlisle · 2023-11-14T23:05:18Z

The test file xmarks-009 fails showing this diff, I didn't update it, to make it easier to back out this commit if you don't like it, and to highlight the diff. It requires a bit of balancing to get the string at the right expansion level but it seems to work here without breaking the other ltmarks tests.

I like the idea and it makes the code much simpler --- and we get correct results even if there is shrinkage in the material which is great. I altered the txt a bit (not sure if it is for the better). Right now the test suite fails in a very strange way though and I don't understand why.

yes lots of newlines in the raw log before the [4 page marker which looks like some extra empty \write but I don't see how that can be. eg the raw log for tlb-dox002 has

[3


]) (./tlb-dox002.gls



[4



])

On the terminal it looks like

[3]) (./tlb-dox002.gls



[4]) (./tlb-dox002.aux)

so the extra newlines are only going to the log

FrankMittelbach · 2023-11-14T23:27:15Z

but I don't see how that can be. eg the raw log for tlb-dox002 has

neither can I but I really would like to understand before I update hundreds of test files

davidcarlisle · 2023-11-14T23:33:47Z

@FrankMittelbach turns out \nonstopmode causes a newline in the log, if you keep all the code apart from this , the breaks go.

possibly from

@<Change the interaction...@>=
begin error_count:=0; interaction:=batch_mode+c-"Q";
print("OK, entering ");
case c of
"Q":begin print_esc("batchmode"); decr(selector);
  end;
"R":print_esc("nonstopmode");
"S":print_esc("scrollmode");
end; {there are no other cases}
print("..."); print_ln; update_terminal; return;
end

but my web is a bit rusty:-)

davidcarlisle · 2023-11-14T23:47:04Z

@FrankMittelbach


\wlog{===}
\wlog{===}
\nonstopmode
\interactionmode=1
\interactionmode=2
\wlog{===}

\bye

produces

===
===



===

but since the test suite uses \scrollmode anyway an alternative plan to avoid changing the tests and adding newlines to end user log files would be to skip saving and reseting \interactionmode unless its current value is 3 (errorstopmode) I think any of the values 0,1,2 would work for this so we don't need to switch?

FrankMittelbach · 2023-11-14T23:54:10Z

I guess I'm more for fixing the tlgs instead of doing something tailored to our test suite. But I have another issue. When I locally run l3build check it seems too loop after processing (or while?) github-0094 and I don't find what happens there. When I run 0094 by itself (or the one that I think comes after it all seems fine

davidcarlisle · 2023-11-15T00:01:39Z

@FrankMittelbach

If I run

l3build check github-0094

It stops in error stop mode at the terminal with

! Undefined control sequence.
\pdfstartlink ->\pdfextension 
                              startlink 
l.25 \leavevmode\pdfstartlink
                             
?

FrankMittelbach · 2023-11-15T00:03:23Z

and the same happens for l3build check tltc001 with xetex (as if the interaction mode it not properly restored), now why is that?

davidcarlisle · 2023-11-15T00:05:49Z

and the same happens for l3build check tltc001 with xetex (as if the interaction mode it not properly restored), now why is that?

well why is it using luatex back end syntax with xetex

davidcarlisle · 2023-11-15T00:12:50Z

me> well why is it using luatex back end syntax with xetex

because it's defined that way in the test file, this ought to be in a pdftex-luatex config, probably

FrankMittelbach · 2023-11-15T00:30:01Z

All true but why is it not running scrollmode?

FrankMittelbach · 2023-11-15T16:23:30Z

After the test suite issues have been identified and corrected, I think this is now ready to be merged. Open questions

Should there be a rollback (would be tricky and I tend to think no)
Is the error message ok, the way it is now? (there is not much space availabel, ie one has to stay with more or less that many chars)
Is the news entry ok?

davidcarlisle

looks good to me, I don't think it needs rollback, I made a couple of minor tweaks to the news text. Error text is OK, I played around in my initial draft but getting the text just to appear in the right place doesn't leave a lot of room, as you noted Your version is fine.

# Conflicts: # base/changes.txt # base/doc/ltnews39.tex # base/ltmarks.dtx

…ial (latex3#1177) * enhance mark extraction to account for forced breaks within the material (WIP) * Generalize mark extraction code; Use it to also provide \__mark_get_marks_for_reinsertion:nNN for use in multicols (and elsewhere) eventually * added change entries typos and corrections * first attempt using \interactionmode (WIP) code can be simplified if we go this way. * improve infinite glue shrinkage log message * some documentation of David's idea * hmm * much simpler this way now (and the marks always correct) documentation not yet updated and old code still mostly inside commented out * the ways of TeX are mysterious --- anybody any idea how this change can be triggered from ltmarks.dtx? * more of those ... strange * and another one * some cleanup and many more test files showing extra lines in the log * too much removed from the doc * and a few more tests ... * even more test files * added ltnews entry * minor wording adjustments * a bit more docu cleanup * tlg updates * correct date --------- Co-authored-by: David Carlisle <[email protected]>

enhance mark extraction to account for forced breaks within the mater…

7adf2ca

…ial (WIP)

FrankMittelbach requested review from PhelypeOleinik, josephwright, davidcarlisle, u-fischer and car222222 November 10, 2023 15:13

josephwright requested changes Nov 12, 2023

View reviewed changes

Generalize mark extraction code;

010bb15

Use it to also provide \__mark_get_marks_for_reinsertion:nNN for use in multicols (and elsewhere) eventually

josephwright approved these changes Nov 13, 2023

View reviewed changes

davidcarlisle approved these changes Nov 14, 2023

View reviewed changes

car222222 approved these changes Nov 14, 2023

View reviewed changes

FrankMittelbach and others added 4 commits November 14, 2023 15:06

Merge branch 'develop' into ltmarks-enhance

1307dfc

added change entries

b6728fe

typos and corrections

first attempt using \interactionmode (WIP)

3e1a3a0

code can be simplified if we go this way.

improve infinite glue shrinkage log message

2a5626b

FrankMittelbach added 6 commits November 14, 2023 22:50

some documentation of David's idea

071e846

hmm

759de63

much simpler this way now (and the marks always correct)

9963c28

documentation not yet updated and old code still mostly inside commented out

the ways of TeX are mysterious --- anybody any idea how this change c…

8bab6ba

…an be triggered from ltmarks.dtx?

more of those ... strange

c05b4ff

and another one

99c537b

FrankMittelbach added 6 commits November 15, 2023 12:26

some cleanup and many more test files showing extra lines in the log

d599d5e

too much removed from the doc

4fb1442

and a few more tests ...

bff9dd7

even more test files

ef86135

Merge branch 'develop' into ltmarks-enhance

d02ebfd

added ltnews entry

cf3cbff

FrankMittelbach requested review from davidcarlisle, josephwright and car222222 November 15, 2023 16:22

minor wording adjustments

7d8f90f

josephwright approved these changes Nov 15, 2023

View reviewed changes

davidcarlisle approved these changes Nov 15, 2023

View reviewed changes

FrankMittelbach added 4 commits November 16, 2023 00:02

a bit more docu cleanup

0d35a5f

Merge branch 'develop' into ltmarks-enhance

0549df4

# Conflicts: # base/changes.txt # base/doc/ltnews39.tex # base/ltmarks.dtx

tlg updates

6829c52

correct date

1e59231

FrankMittelbach merged commit d87b8fc into develop Jan 29, 2024
78 checks passed

FrankMittelbach deleted the ltmarks-enhance branch January 29, 2024 09:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

enhance mark extraction to account for forced breaks within the material #1177

enhance mark extraction to account for forced breaks within the material #1177

FrankMittelbach commented Nov 10, 2023 •

edited

Loading

josephwright left a comment

FrankMittelbach commented Nov 12, 2023

josephwright commented Nov 13, 2023

FrankMittelbach commented Nov 13, 2023 •

edited

Loading

FrankMittelbach commented Nov 13, 2023

muzimuzhi commented Nov 13, 2023

FrankMittelbach commented Nov 13, 2023

davidcarlisle Nov 14, 2023

FrankMittelbach Nov 14, 2023

FrankMittelbach Nov 14, 2023

davidcarlisle Nov 14, 2023

davidcarlisle Nov 14, 2023 •

edited

Loading

FrankMittelbach Nov 14, 2023 •

edited

Loading

davidcarlisle Nov 14, 2023

davidcarlisle Nov 14, 2023

car222222 left a comment

davidcarlisle commented Nov 14, 2023 •

edited

Loading

FrankMittelbach commented Nov 14, 2023

davidcarlisle commented Nov 14, 2023

FrankMittelbach commented Nov 14, 2023

davidcarlisle commented Nov 14, 2023

davidcarlisle commented Nov 14, 2023 •

edited

Loading

FrankMittelbach commented Nov 14, 2023

davidcarlisle commented Nov 15, 2023

FrankMittelbach commented Nov 15, 2023

davidcarlisle commented Nov 15, 2023

davidcarlisle commented Nov 15, 2023

FrankMittelbach commented Nov 15, 2023

FrankMittelbach commented Nov 15, 2023

davidcarlisle left a comment

enhance mark extraction to account for forced breaks within the material #1177

enhance mark extraction to account for forced breaks within the material #1177

Conversation

FrankMittelbach commented Nov 10, 2023 • edited Loading

Internal housekeeping

Status of pull request

Checklist of required changes before merge will be approved

josephwright left a comment

Choose a reason for hiding this comment

FrankMittelbach commented Nov 12, 2023

josephwright commented Nov 13, 2023

FrankMittelbach commented Nov 13, 2023 • edited Loading

FrankMittelbach commented Nov 13, 2023

muzimuzhi commented Nov 13, 2023

FrankMittelbach commented Nov 13, 2023

davidcarlisle Nov 14, 2023

Choose a reason for hiding this comment

FrankMittelbach Nov 14, 2023

Choose a reason for hiding this comment

FrankMittelbach Nov 14, 2023

Choose a reason for hiding this comment

davidcarlisle Nov 14, 2023

Choose a reason for hiding this comment

davidcarlisle Nov 14, 2023 • edited Loading

Choose a reason for hiding this comment

FrankMittelbach Nov 14, 2023 • edited Loading

Choose a reason for hiding this comment

davidcarlisle Nov 14, 2023

Choose a reason for hiding this comment

davidcarlisle Nov 14, 2023

Choose a reason for hiding this comment

car222222 left a comment

Choose a reason for hiding this comment

davidcarlisle commented Nov 14, 2023 • edited Loading

FrankMittelbach commented Nov 14, 2023

davidcarlisle commented Nov 14, 2023

FrankMittelbach commented Nov 14, 2023

davidcarlisle commented Nov 14, 2023

davidcarlisle commented Nov 14, 2023 • edited Loading

FrankMittelbach commented Nov 14, 2023

davidcarlisle commented Nov 15, 2023

FrankMittelbach commented Nov 15, 2023

davidcarlisle commented Nov 15, 2023

davidcarlisle commented Nov 15, 2023

FrankMittelbach commented Nov 15, 2023

FrankMittelbach commented Nov 15, 2023

davidcarlisle left a comment

Choose a reason for hiding this comment

FrankMittelbach commented Nov 10, 2023 •

edited

Loading

FrankMittelbach commented Nov 13, 2023 •

edited

Loading

davidcarlisle Nov 14, 2023 •

edited

Loading

FrankMittelbach Nov 14, 2023 •

edited

Loading

davidcarlisle commented Nov 14, 2023 •

edited

Loading

davidcarlisle commented Nov 14, 2023 •

edited

Loading