forked from hermit-crab/ScrapeMate
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathTODO
39 lines (36 loc) · 2.09 KB
/
TODO
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
Features/issues priority:
High:
easy - allow traversing upwards (if rejects are present - kill them)
easy - smart upwards - traverse up for a common selector separate parents
hard - nested selectors
trivial - allow specifying field meta after a colon within field name
(e.g. 'next-page-link:relative') as a way to provide additional info to the actual scraping tool
hard - instead of making any page/css mutations do all highlighting and picking over a click-through overlay div
Implement testing capabilities:
medium - sweep over selected templates downloading and testing them on their respective pages
medium - allow to specify and edit template matching pages
medium - implement some sorta contracts to verify templates
easy - json editor fields sorted
hard - visual intuitive expression editor:
full element(s) path shown, clicking on parents, their classes, id's and other attributes to include inside expression
medium - make css-to-xpath button
very hard - automatic structure discovery by matching html trees of multiple similarly structured pages (e.g. product pages on the same website)
Medium:
medium - exclude from picker by clicking on row in a data tab
easy - n key -> new field, p key -> pick field, esc -> blur current input, esc on main window body -> focus sidebar
easy - dont allow selecting ourselves / selectorgadget controls
easy - syntax highlight html and json inside data control tab and in an editor
medium - highlight target element when hovered over in a .data.many control tab
Low:
easy - sort templates upper to lower domain levels
medium - thorough logging
hard - rework styles into something sensible
medium - beautify and rework html
easy - separator of templates list popup header is below scrollbar
medium - link patterns mode
easy - template shift select many
hard - tweak styles for firefox
hard - fix styles for firefox gtk build
easy - "show hidden elements" button (use jq ":hidden" selector maybe)
hard - "highlight all" mode - highlights all elements on the page using different color for each respective field
hard - also draw field titles written on top of highlight area