Issues/improvements to rasterize-before-printing setup

[jxlin123]

Edit 8/22: promoted one further consideration to TODO status

I wanted to make a list of known issues, or potential improvements to the recent rasterize-before-printing setup from PR #684. While the new setup works, there are a few things that could potentially be addressed to make it better.

I don't necessary expect that every point I mention below should be fixed, or is even fixable. If the current setup, along with its peculiarities, works fine, then we can leave it at that.

Note that there's also an earlier issue, #44, that addresses some other printing improvements.

Thanks once again for all the reviewers of PR #684.

Significant TODOS:

• Ensure PR Removing obsolete ensure absent statements from PR 684 #714 gets merged around the beginning of the semester.
• Update the documentation in the ocfweb repo. See issue docs: Update printing infrastructure docs ocfweb#529.
• Modify the local CUPS settings. So far, I haven't really changed some of the CUPS configuration files (like cupsd.conf, cups-files.conf), and have been relying on the default settings. However, the defaults don't provide a lot of flexibility when interfacing with CUPS (e.g., one needs special permissions to use parts of the CUPS web interface, etc.).
  • It is important to note that print jobs can appear in two places: the local machine, and printhost. Since print jobs pass through the local machine relatively quickly, it may not be worthwhile to pay attention to the local CUPS print queues (e.g., if one wants to cancel a print job, he/she can just do it at https://printhost.ocf.berkeley.edu). Still, staffers may have other reasons to modify the local CUPS settings, so having more relaxed permissions may be useful.

Further considerations:

• General
  • The whole process can take some time, depending on the PDF complexity. I don't anticipate this to be a huge issue, but I can imagine impatient users trying to reprint their documents just because they don't see immediate results. For now, I don't foresee a straightforward fix for this, so probably we can either have opstaff to remind users to be patient as their files print, or somehow include a loading bar of some sorts?
  • If users somehow want to bypass the whole filtering process, should there be a backdoor that lets them directly send their PDFs to printhost, without running through my filter?
• Conversion process
  • While the conversion filter (raster-filter) works for many PDFs, I've noticed it fails for others, especially those with many pages (50+). While this shouldn't affect regular lab users as much (due to the 10/20 page count limit), it may be nice to see if those PDFs can be pre-rasterized as well.
    • This probably can be done by upping the ImageMagick limits...
  • Find a way to provide more information when raster-filter fails. Currently, all that is emailed out is the error message from convert along with the machine the error occurred on. It would be nice to include more information, such as the current user, job details, etc. (similar to what enforcer emails when it encounters an error).
• CUPS interface
  • I've also noted that within the local CUPS web interface, many job details are withheld. Perhaps this a good thing... or should those details be visible? And if so, how? (This might require changing other parts of my setup, not just CUPS settings...)
  • Printing test pages from the CUPS web interface fail, both locally and from whiteout. I don't think I can easily solve this, but one of my earlier versions of my printing setup (with a different architecture, however) allowed test pages to be printed, so...
    • Probably not a major problem, since there isn't much reason for printing test pages from the CUPS web interface...
    • I think whiteout already had this "issue" to begin with. Because my PR made very few changes to printhost, I don't think this feature broke after my setup was added.

If you notice any further issues or bugs, or have any other comments, let me know!

[jxlin123]

Update: also see PR #734, since printing apparently is broken right now?

[cg505]

Printing test pages from the CUPS web interface fail

I believe this is because the "user" for these pages is "anonymous", which confuses enforcer.

[jxlin123]

@cg505 I think you're right, judging from the emails I've seen from whiteout. I've also noticed some CLASS errors as well, but I think that's what happens when you try to directly access a printer queue (like logjam-double) instead of a printer class (like double).

Also, looks like I'm going to have to bump up the local CUPS settings issue -- I've updated my initial post to reflect this. It's very inconvenient when CUPS doesn't let users access some of its functionality just because they don't have elevated permissions... so I'll be working on that...

[jxlin123]

There's been some discussion on Slack regarding the recent printing issues, as well as its implications.

(@cg505 and @fydai correct me if I'm wrong here; you two noticed this before I did)

To summarize, printing jobs weren't going through due to a lack of authentication between the local computers and whiteout. PR #734 fixes this. However, this is not enough. We also need to ensure that the changes are actually deployed.

As far as I know, some of the desktops are still experiencing problems. I think this is because they are not up to date with production, or because somebody needs to restart CUPS on the machines for the changes to apply. (@cg505 suggested introducing a dependency in Puppet to restart CUPS upon any changes to the printer configuration files.)

That being said, @pxhanus noticed that jobs that didn't go through before were later being printed. CUPS retains any queued printing jobs, so once printing is fixed, jobs will start going through again. In her case, this was an issue since sensitive documents were being printed out after the fact. @rakechill suggested automatically clearing the queues after some time... which may work...

All good ideas to keep in mind as we continue working on this...

[cg505]

To add to this:

due to a lack of authentication between local computers and whiteout

"Authentication" isn't exactly the right term here. My hypothesis is that we were attempting to start a TLS connection for ipps with printhost, but we were getting a Let's Encrypt cert for printhost.ocf.berkeley.edu. Since we weren't connecting to the FQDN, cups/ipps thought this was invalid and failed. The confusing part about this is that it surely worked at some point, but this hasn't changed. So I'm not sure why this changed now. This is at best a hypothesis of what was happening, and my PR seems to fix it.

Also, @fydai ran ssh-list to run # systemctl restart cups on all the desktops, so theoretically CUPS should be restarted everywhere. This may not have totally worked. Also, if a job fails, the virtual printer in the local cups will be "paused" and must be re-enabled with # cupsenable <printername>, so it's possible there are paused printers still on some desktops.

[jxlin123]

An update, now that's it's the start of classes:

I've tried working on some of the suggestions noted in this thread; so far, no luck. Looks like the setup is still unchanged...

As far as I know, printing currently works on the desktops. I have tested most of desktops, and I believe fydai and cooperc did so as well.

That being said, if printing ends up breaking loose, I've created a rollback branch on my own OCF Puppet fork here that undoes some (but not all) of the major changes I've made. In that case, one may need to restart CUPS on the machines after deploying the rollbacks.

And for those still in Berkeley: happy first day of classes!

[jxlin123]

Quick update: it seems like printing was decent today, although not without a few (minor) issues. From the looks of it, there wasn't any major complaints or large numbers of jobs not going through, so I think we should be fine for now.

The only opstaff comments I've heard so far was from Elle, who mentioned that Google Docs were giving some people trouble. The Google Docs check has been added back to enforcer.