Some consumed files get a "_01" attached

[klemmi19]

Hi everyone.

First of all: paperless-ng is very nice. I really like it. It is exactly what I was looking for.

I am still doing a setup for my needs.
paperless-ng is running in a docker container on OMV on a rpi4. So far so good, performance is ok.

My problem is that some (not all) of the consumed are moved to the right location according to my FILENAME env variable but get a copy with a _01 attached. So I have file.pdf and file_01.pdf.
This happens to like 50% of my files. I think all of them are scanned.

I assume that this has something to do with the OCR and that the _01 file is a optimized version.

Is there a way to prevent paperless-ng from doing that or just overwriting the original one with a optimized version?

These are the env I added into portainer:

PAPERLESS_FILENAME_FORMAT | ../../{created_year}/{document_type}/{title}
PAPERLESS_TIME_ZONE | Europe/Berlin
PAPERLESS_OCR_MODE | skip_noarchive
PAPERLESS_OCR_CLEAN | none
PAPERLESS_OCR_DESKEW | false
PAPERLESS_OCR_LANGUAGE | deu
PAPERLESS_OCR_ROTATE_PAGES | false
PAPERLESS_OCR_OUTPUT_TYPE | pdf
PAPERLESS_OCR_PAGES | 1

Thanks for your suggestions!

Stephan

[jonaswinkler]

PAPERLESS_FILENAME_FORMAT | ../../{created_year}/{document_type}/{title}

Oh wow. Don't use ../../ here. I didn't even think of that, but I'll probably add a check that prevents that.

Paperless organizes your documents in two folders:

• /media/documents/originals/ stores original documents.
• /media/documents/archive/ stores optimized PDF/A documents, with embedded OCR text. These are what you usually see in the frontend.

When you set PAPERLESS_FILENAME_FORMAT="{title}", this will cause paperless to create the following files for a document with the title "invoice":

• /media/documents/originals/invoice.pdf
• /media/documents/archive/invoice.pdf

If you add "../../" prior to that, paperless would like to create these two files instead:

• /media/invoice.pdf
• /media/invoice.pdf

Since that's impossible, paperless adds _01 to one of the files.

Paperless will always append _01, _02 etc to a file if it would otherwise overwrite an already existing file.

[jonaswinkler]

Also, paperless remembers every file and it's location on disk. As long as you don't move files manually, you can fix this situation by changing the filename format, and running the document_renamer.

[klemmi19]

ah understood. Thank you for the quick answer.

I just tried to get the folders YEAR -> DOCTYPE -> any.pdf to be created outside of documents/archive that the folder tree doesn't get too deep and to have it on root level of my external drive (less clicks when I go through it via samba share).

Is there any way to achieve this? Otherwise I'll live with that...

Thanks!

[klemmi19]

and I am pretty sure that I read it somewhere else to use xpath expression in that variable :-P

[jonaswinkler]

Is there any way to achieve this? Otherwise I'll live with that...

You can always setup a share that points to some specific folder on your hard drive. Other than that, I'd suggest sticking with how paperless organizes the media folder.

I agree that that the folder structure in the media folder could be reduced by at least one level (the "documents" folder is not really needed), but this folder structure has been like that since OG paperless, and I don't want to change it. Makes migration from old to -ng easier. Also, data migration is always a huge PITA to get right.

the document_renamer is a management cli command, yes. It's in the documentation.

Not sure what this xpath stuff is about :)

[klemmi19]

thanks. So I'll stick with it.
is there any way to get rid of one version of the files? with "pdf" and "skip_noarchive" the scanned documents get an archive version, too.
Personally I'd like to have just one version, regardless of "original" or "archive" one.

[jonaswinkler]

No, there's no such option yet. This has been requested quite a few times now, so I'll have a look at the OCR and file handling logic again. Probably an option PAPERLESS_OCR_NOARCHIVE=bool that simply won't store the archived version instead of the current "skip_noarchive", which is somewhat of a hack anyway.

It's pretty tough to make everyone happy. The current solution works out pretty good for the majority, with only a few inconveniencies here and there.

[klemmi19]

Would be really nice but I know the struggle. ;) When I started my job as an it consultant, the first 4 years I did software development all the time. Now I am doing more or less high level tasks and am not really into development anymore, which I really enjoyed. But in my free time I am really saturated working with computers and do not have the drive to get into programming anymore.

So obviously I am not that deep into programming atm, but from the ocrmypdf docs, it doesn't sound too complicated to overwrite a file in place instead:

Modify a file in place
The file will only be overwritten if OCRmyPDF is successful.

ocrmypdf myfile.pdf myfile.pdf