Import CSV / XLS / XLSX Operation

[gumpon]

Feature Request

What's the feature you think Backpack should have?

Import csv file to database example user table same column(name, email ) and
check email exsiting or not.
??

Have you already implemented a prototype solution, for your own project?

??

Do you see this as a core feature or an add-on?

Yes, I want to see core feature.
??

[tabacitu]

Hi @gumpon - I also think an IMPORT feature would be pretty cool. I've needed it a few times myself in the past but found ways around it. I don't agree with it being a core feature though - I think it's best that it's a separate package, an add-on, since MOST people won't ever use it. I want to keep the Core as lean as possible, with only what most people use, the 90% use case, so that the main package doesn't become too big. But it could be a first-party package too, indeed, if nobody else builds a good solution. I'll move this to the Addons repo so we can continue the conversation.

Here's how I would see this.

Architecture

A package that provides an operation. Just like the revise operation. We could have an ImportOperation trait, that developers can add on top of the CrudController if they want it.

Interface

That operation would:

• provide an "Import" button inside the top button stack;
• open a new page once you click that button, that allows you to drag&drop (or click) to upload a .CSV , .XLS or .XLSX file with a table; it could/should have a checkbox to declare that "the first row is the header" - since this is a common use case;
• once the file is uploaded, it could/should allow you to map .CSV columns to database columns; the most friendly interface I've seen so far is from MailChimp, where they show the .CSV table in HTML form (only the first 3-5-7 rows though) and for headers they show dropdowns (<select>) that allow you to pick what column from the db this should be mapped to; I found that very intuitive; it also allows you to ignore .CSV columns entirely by picking nothing in the dropdown;
• once the columns are mapped, the user can click an "Add entries" button, which would import all the lines, most likely using PHP Generators so that even if the file is 1.000.000 rows, it'll work just fine;

Challenges

1. This column mapping will be the tricky part, I think. For direct mapping (Name -> product.name) this will be easy, we could just show the uploaded table as an HTML table, with the header being a Select, so that you pick for each .CSV column where the information should be uploaded. But. What happens for relationships?! How can you make it easy to map .CSV columns to other database tables? Is it through the ID? Is it through the names (identifiable attribute) like Excel users do a lot of the times? That will be a tricky bit, in my opinion...

2. Another possible issue with adding A LOT of entries, from a file that might not be properly formatted, at least not for ALL rows, is - what do you do when a few rows cannot be added?! Do you revert adding all other rows, that were valid? Do you just show the invalid rows that failed? What will the user do with those? Is the IMPORT operation considered a success if some rows have failed? What I've done in the past, to avoid this bad UX, was to show the remaining invalid rows as a Handsontable, so that the user can make the valid right then-and-there, instead of having to revert changes and all that. However, since then Handsontable changed their pricing - they no longer offer a Free and Paid version, it's free for non-commercial use (which is awesome) but a whopping $790 per developer for commercial use, which I'm pretty sure a lot of Backpack customers won't be able to afford). So yes, there's a good technical solution to this problem, I think, but not a good economical solution to this problem 😞

Thoughts anyone? Claims anyone?

[vernard]

Leaving this link here as it's somehow providing a solution to this problem.

https://medium.com/@harish.patel/how-ive-built-an-bulk-import-button-for-my-backpack-for-laravel-admin-panel-437542ac1408

[abewartech]

https://github.com/ckhandla94/backpack-import