Replies: 8 comments 3 replies
-
|
I vote for "Ability to start and stop the crawler and resume from the previous position" because gui not that much important for me. Other features are nice to have but can be easily implemented / manipulated by scripting languages or excel/google sheets etc except long and lat feature. |
Beta Was this translation helpful? Give feedback.
-
|
I agree, that's a good feature. I am already working on it but needs more work. |
Beta Was this translation helpful? Give feedback.
-
|
I will try to implement lon and lat customization first. |
Beta Was this translation helpful? Give feedback.
-
|
When scraping a lot of cities next to each other i get a lot of duplicate results. Scraping all cities for a category that is not available in most cities will cause google to zoom out and collect results from other cities. But these results also appear when scraping the city where the result is actually in. For a small country i scraped 20x more then needed. Which is huge. Maybe adding a identifier to one of the columns in gmaps table and doing upsert based on it helps. For example using the google identifier (not sure if you have it at that time). But if first all queries in the txt file are done, followed by the 'scrape jobs' it will reduce the queries significantly. Imagine doing scraping at scale (200k cities/villages in usa) and then do 20x more then needed. This will drastically decrease the total duration. |
Beta Was this translation helpful? Give feedback.
-
|
Would be nice to some how gather emails via regex, emails are most likely what matter we might integrate an alternative page to drag and drop csv and in bulk so that we can throw emails away like Mautic - https://www.mautic.org/ pivoting potential leads generate more revenue for our business and money :) |
Beta Was this translation helpful? Give feedback.
-
|
@micaelparadox Right now the program can extract emails from the first page of the external website (where this is available). can you please explain your idea? |
Beta Was this translation helpful? Give feedback.
-
|
Re-using emails to once we upload the cvs we send emails to people of
business offering they your service for a potencial solution. Increasing
incoming money eventually image grabbing a lot of clients could you pls
show me an example of usage with -email I couldn’t make it work regarding
the email part
…On Tue, 22 Oct 2024 at 01:59 Georgios Komninos ***@***.***> wrote:
@micaelparadox <https://github.com/micaelparadox> Right now the program
can extract emails from the first page of the external website (where this
is available).
can you please explain your idea?
—
Reply to this email directly, view it on GitHub
<#61 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AQRZHX6LD742YD3NJWH2KXTZ4XLR5AVCNFSM6AAAAABJQLD7PKVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCMBRGIZDINY>
.
You are receiving this because you were mentioned.Message ID:
***@***.***
com>
|
Beta Was this translation helpful? Give feedback.
-
|
(1) in the Web UI there is a checkbox in the Advanced section named email. Use that and the program will try to visit the extracted websites and will try to extract emails (only from the first page). (2) I appreciate your idea but I don't believe that sending mass emails is a feature that belongs to this software. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the tip!
…On Tue, 22 Oct 2024 at 02:22 Georgios Komninos ***@***.***> wrote:
(1) in the Web UI there is a checkbox in the Advanced section named email.
Use that and the program will try to visit the extracted websites and will
try to extract emails (only from the first page).
(2) I appreciate your idea but I don't believe that sending mass emails is
a feature that belongs to this software.
—
Reply to this email directly, view it on GitHub
<#61 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AQRZHX3RUBDMZYZD4CFZALLZ4XOJNAVCNFSM6AAAAABJQLD7PKVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCMBRGIZTQNY>
.
You are receiving this because you were mentioned.Message ID:
***@***.***
com>
|
Beta Was this translation helpful? Give feedback.
-
|
[image: image.png]
On Tue, Oct 22, 2024 at 6:33 AM Santana, Micael ***@***.***>
wrote:
… Thanks for the tip!
On Tue, 22 Oct 2024 at 02:22 Georgios Komninos ***@***.***>
wrote:
> (1) in the Web UI there is a checkbox in the Advanced section named
> email. Use that and the program will try to visit the extracted websites
> and will try to extract emails (only from the first page).
>
> (2) I appreciate your idea but I don't believe that sending mass emails
> is a feature that belongs to this software.
>
> —
> Reply to this email directly, view it on GitHub
> <#61 (reply in thread)>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AQRZHX3RUBDMZYZD4CFZALLZ4XOJNAVCNFSM6AAAAABJQLD7PKVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCMBRGIZTQNY>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***
> com>
>
--
Micael Santana
Software Engineer
in/micasan <https://www.linkedin.com/in/micasan/> WhatsApp
<https://wa.me/+5547996428339>
|
Beta Was this translation helpful? Give feedback.
-
|
Does it have progressbar ? |
Beta Was this translation helpful? Give feedback.
-
Please select the feature which you would like to see next.
Voting will be open until 25 Jun 2024
23 votes ·
Beta Was this translation helpful? Give feedback.
All reactions