Use changedetection to automatically download files/invoices

[N1c093]

I know, that this feature is pretty out of scope of changedetection, but I think the feature is mostly finished.

Problem:
I have the problem that my internet provider only provide the newest invoice on his customer portal. Therefore I have to login every month into the customer portal and download the invoice manually.

Solution:
Use a tool which automatically login into the customer portal and download the newest invoice. The download should be saved on the docker volume.

As changedetection already have the ability to navigate through webpages with the feature "browser steps", I think it would be pretty easy to expand this feature, to download files.

Thank you in advance.

[dgtlmoon]

Actually one idea this gives me, is that there could be an option like

[x] Attach page/download to notification

Then you could aim it at some .exe or .pdf or whatever, and make it retry once a day, if the target is different (change detection) it would send you a notification with that added as an attachment

[jjmoffitt]

Any hope on getting something like this added? I've been trying to use browser steps, but playwright won't switch to a new tab when I try to open the invoice. It would be even better if I could tell playwright to go to a page and then download the pdf that comes from a link, that I could persist to my paperless consume folder.

Love the software, thanks for the work you do!

[nrasd]

I try to solve something similar:

1. log in to personal account of a website using browser steps.
2. "click" on a link leading to a "_csvexport.php" which generates and returns a csv-file. This file should be stored in order and be available for further use.

I am struggeling with step 2. I tried to run javascript before change detection and download the file to the docker volume (/datastore/c65964be-1b0f-464e-b002-9512ebed7da6/).

I use this code which seems to be running without any errors in the frontend of the changedetection tasks and no errors in the console of the docker container:

(async () => {
  // Warte 5 Sekunden, um sicherzustellen, dass die Seite vollständig geladen ist
  await new Promise(resolve => setTimeout(resolve, 5000));

  // Versuche, das Element mit der ID "#csv" zu finden und darauf zu klicken
  const csvElement = document.getElementById('csv');
  if (csvElement) {
    csvElement.click();

    // Warte kurz auf die Navigation oder füge eine geeignete Wartezeit ein
    await new Promise(resolve => setTimeout(resolve, 3000));

    // Lade die PHP-Datei herunter und speichere sie lokal
    const downloadLink = 'https://www.xxx/_csvexport.php?filename=xxx_20240118.csv';
    const downloadPath = '/datastore/c65964be-1b0f-464e-b002-9512ebed7da6/xxx_20240118.csv';

    await fetch(downloadLink)
      .then(response => response.blob())
      .then(blob => {
        const url = window.URL.createObjectURL(blob);
        const a = document.createElement('a');
        a.href = url;
        a.download = downloadPath;
        document.body.appendChild(a);
        a.click();
        window.URL.revokeObjectURL(url);
      });
  } else {
    console.error('Das Element mit der ID "#csv" wurde nicht gefunden.');
  }
})();

However, no file is stored to this path.
Anyone has an idea how to solve this? Thanks!

[VINTX2]

I am looking for a solution, too.

As an alternative, I stumbled upon https://github.com/invoiceradar
Unfortunately, not open-source, but can be selfhosted (only on Mac and Windows).

[dgtlmoon]

It looks like it may "technically" be possible to automatically download invoices (files)

    # Set the download behavior
    client = await page.target.createCDPSession()
    await client.send(
        "Page.setDownloadBehavior",
        {"behavior": "allow", "downloadPath": download_path},
    )

    # Trigger the download
    await page.goto("https://example.com/some-download-url")

    # Wait for the file to download
    download_complete = await wait_for_download(download_path, expected_filename)

by hacking something nicer into the pypuppeteer library we use...

but question is here, is this really changedetection? how does the changedetection part of it work?

Maybe some special extension that actually uses the "downloaded" content as the change detection - then if a change is detected it can send it to discord/email/etc ?

https://www.getmyinvoices.com/en/ pretty interesting alternative

[VINTX2]

but question is here, is this really changedetection? how does the changedetection part of it work?

changedetection.io already implemented the task to detect and notify about changes. In this case, it could check for newly added links or buttons, which trigger the download.
It also has an UI (browsersteps) for easily implementing the login process and clicks.

The downloaded files could be saved locally for selfhosted users.
For the hosted solution, maybe send the files to the user's dropbox, google drive, onedrive account or send them to the specified notification service (which may not always work). For example, Discord allows file uploads up to 20mb, which is enough for invoices.

https://www.getmyinvoices.com/en/ pretty interesting alternative

I know 3 services which offer such kind of service. invoicefetcher.com, getmyinvoices and invoiceradar (beta), but they are all limited in one way or another, sometimes slow, buggy or expensive.

If it were possible to download files with changedetection, one could build his own solution for every provider he needs.

[jjmoffitt]

The change detection for me is looking for new bills, invoices, receipts etc and downloading it when it’s detected. Because things like this aren’t emailed anymore and automating retrieval is a pain.

Do you think that would make more sense as a separate utility?