forked from Nandaka/PixivUtil2
-
Notifications
You must be signed in to change notification settings - Fork 0
/
changelog.txt
537 lines (432 loc) · 18.9 KB
/
changelog.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
20150725-beta1:
- Fix Issue #88: undefined variable when handling cookie.
20150715:
- Fix Issue #87: Update login logic, now always use https.
20150711:
- Implement #85: Enable infinite loop for download by tags.
- Fix Issue #79: Don't save if the download is not complete.
- Add option to dump tags search page.
- Reorganize config.ini.
20150605:
- Merge pull request #76 from woky/master: Expand ~ in path options.
- Add option to filter images by bookmark count for download by taglist (console input only).
- Fix Issue #81: Divided by Zero when download time is instant.
20150427:
- Add feature to download both private and public image bookmark.
- Fix Issue #70: Update Error code updated and use sys.exit() instead of os._exit()
- Fix Issue #74: add check based on saved filename, old manga data might need to be redownloaded.
- Add ability to export list of user bookmarks for user with given member_id.
- Pull Request #73: Merge export user bookmarks from prototype27/topic/exportUserBookmarksForUser.
20150328:
- Implement #69: Create .ugoira format, open with http://www.bandisoft.com/forum/viewtopic.php?f=8&t=3359
- Implement #69: Add local .ugoira file checking.
- Implement #70: call os._exit() with error code != 0 if got exception.
- Fix Issue #71: Always overwrite files even if local file is larger, use backupoldfile=True to backup the old file!
- Fix Issue #72: Update download by tags parser.
- Fix Issue #69: Add option to delete zip file, set deleteZipFile = True.
20150218:
- Fix Issue #64: Update login form selector.
20150121:
- Implement #66: Add option to download from oldest image first for Download by Tags/Tags list.
20150105:
- Update the SSL login page url to https://www.secure.pixiv.net/login.php.
- Update BeautifulSoup to version 3.2.1.
20141216:
- Fix Issue #63: Update image parser again :(.
20141212b:
- Fix Issue #60: Update image parser to handle old and new layout.
- Fix html dump logic.
20141201:
- Fix Issue #58: Update handler for private image.
- Update handler for deleted image.
20141121:
- Fix issue #57: Type error when downloading from tags.
20141006:
- Ignore userobots setting, now is set to always False.
- Update manga parser again.
20141004c:
- Update parser.
- Update download logic for manga page (20141004b)
- Update parser for the new 2 page manga mode (20141004c)
20140926
- Fix Issue #47: Update manga parser.
20140916
- Fix Issue #44: process image to handle known image exception.
- Implement #43: write ugoira data to separate file.
20140712
- Fix parser for Option 8: Download new illust from bookmark.
20140705
- Update download by member id to accept comma-separated list from console.
- Update download by image id to accept comma-separated list from console.
20140628
- Update parser to support animation (ugoira_view).
20140621
- Update artist token parser.
20140620
- Fix issue #36: Member illustration parser update.
- Fix issue #37: Bookmark parser update.
20140606
- Fix issue #33: fallback to member token if nickname not found
- Add CLI option to load config file from path.
- Implement feature #30: use %bookmark_count% and %image_response_count%
20140325
- Update unknown error handler.
- Add %bookmark_count% for filename format, only available for download by tags.
20140222
- Update model.
- Update test cases.
20140126
- Fix download by Tags.
20131218
- Add 403 handler when processing member.
- Fix Japanese worksdate parser.
20131214
- Add 404 handler when processing member.
- Reset selection when Ctrl-C pressed.
- Reindent and encode source code to UTF-8.
20131122
- Fix Unicode error for Download by Tag.
- Add feature to replace the root path of image in database.
20131005
- Fix image parsing.
- Update test case.
- Add new option dumpMediumPage for debugging.
20130927
- Change some command line syntax:
- Download by Tags (3): wilcard[y/n], start page, end page, tags
- Download by Title Caption (9): start page, end page, title/captions
- Download by Tags and MemberId (10): MemberId, start page, end page, tags
- Download by Group ID (12): GroupId, limit, process external[y/n]
- Fix trailing space before extensions in filename
- Add SOCKS proxy support, use socks5:// or socks4:// format.
- Add temporary fix for datetime parsing (pixiv server bug).
20130912
- Add option to disable html dump and skipDumpFilter.
- Update console title for download by member and download by tags.
- Add download by group id.
20130804
- Move updatedLimitCount to outer loop of member.
20130714
- Add logLevel option.
- Fix UnboundLocalError for time as reported by Janice.
- Add handler for server error: "An error has occured".
20130705
- Update manga parser.
- Update caption parser.
- Add new option to back up old file if the filesize is different (backupOldFile).
- Auto Backup old config.ini if error detected (e.g.: missing entries) to config.ini.error-xxx.
- Update test cases.
20130511
- Refactor out PixivException and PixivBrowserFactory.
- Update error codes.
- Update artist token parser to handle all img-inf in image list.
- Update filename sanitizer to remove all Unicode Control Character (Cc).
20130430
- Update parser.
- Add start/end page for Download from Bookmarked Members (option 5).
- Remove member id from option 6, use option 11, Download bookmarked images by member_id.
- Add url logging for failed login using cookie.
20130330
- Add Ctrl+C handler for download by member id and download by tags to
allow user to skip to next image or abort the whole process.
20130327
- Add option to keep signed in, set keepsignedin = 1 in config.ini on
[Authentication] section.
- Add login error message parser.
20130322
- Add download time and speed.
- Backup config.ini when saving to config.ini.bak.
20130321
- Fix command line option for download by member id and tags to allow single
tag search.
20130316
- Add option to skip image if worksDate < dateDiff, this will skip to next
member, or stop processing for download from new bookmark.
20130308
- Add menu to reload config.ini
- Add %date% for filename format (YYYYMMMDD).
20130222
- Update parser to the latest pixiv layout.
- Update last page detection (20130222b).
- Update sanitizeFilename: remove new line marker '\r\n' (20130222c).
20130201
- Update imageDownload to skip if got 502 error.
- Fix end page detection for download by tags.
20130128
- Fix end page limit.
- Update parser for pixivArtist and download by tags.
20121215
- Fix title, tags, and caption parsing.
- Add new filename format for download by tags:
-%searchTags% ==> for download by tags, add searched tag keyword.
20121208
- Update no image detection for member
- Fix issue #14: ignore showcase for download by tag
- Fix issue #15: Update query for search by tag
- Add R-18 mode in search by tag
20121124
- Update error detection for image page.
- Add R18 mode only for member/member's bookmarks download.
- Set r18mode = True to apply.
- Fix page limit detection.
20121108
- Add feature for download member's bookmarked images.
- Add new filename format for member's bookmark mode.
- %bookmark% ==> for bookmark mode, add 'Bookmarks' string.
- %original_member_id% ==> for bookmark mode, put original member id.
- %original_member_token% ==> for bookmark mode, put original member token.
- %original_artist% ==> for bookmark mode, put original artist name.
20121102
- Add feature to export image information to text file.
- Set writeimageinfo = True to use.
20121028
- Update error message detection for MyPick and Ero/Guro page.
20121013
- Fix irfanView slide show.
20121006
- Fix unbound variable error.
- Update parser to check if user logged in.
20120929
- Update medium image parser to handle no tags.
20120928
- Update medium image page parser for tags and unit test.
20120923
- Fix PixivConfig doesn't allow %page_number% in settings.
20120921
- Fix login method to use correct form.
20120903
- Add option to compact database.
- Format %works_date*% to use dash '-', instead of slash '/'.
20120831
- Update error detection for new error messages.
20120828
- Update readConfig to handle Unicode, please note writing Unicode value still have error.
- Fix createDownloadList to handle relative path / remove library.zip from path.
- Handle IOError: [Errno 28] No space left on device exception.
- Fix mechanize._httperror_seek_wrapper (20120828b)
20120806
- Update PixivConfig to handle Unicode path.
- Fix %page_index% and %page_number% bug: truncate the length to 1 character only.
- Fix PixivConfig: recreate config.ini if there is any error (20120806b)
20120803
- Fix config.ini path resolution to the actual path where the script reside, not the working dir.
- Update page parser and unittest
- Append 0 for %page_number%
20120724
- Update makeFilename
- clean up double space
- Update filenameFormat
- %urlFilename%
- Add filenameMangaFormat
- %page_index%
- %page_number%
- %page_big%
20120704
- Reload config before downloading tags to reset root directory value.
- Fix export bookmark ordering to follow pixiv page.
20120616
- Fix "0 -" in the avatar filename when no subfolder defined in the filename format.
- Add start action 10 - Download by member_id and tags to the start action list.
20120609
- Update Download by tags parsing because of layout update.
- Change filename limit from 254 to 250 + 4 (for extensions)
- Handle '?' in filename
20120601
- Fix search by member_id and tags.
- Increase log size to 10MB and log backup to 10.
- Fix parseBigImage UnboundLocalError exception
20120526
- Add %R-18% filename format for R-18/R-18G tagged images, can be used for creating directory.
- Fix proxy handling.
- Fix UnboundLocalError when using the black list tag.
- Fix duplicate member_id when exporting user bookmark.
20120519
- Add downloaded filesize check.
- Parse image Views/Ratings/Totals information.
- Parse bookmark count information for tags search.
20120505
- Add SSL Login support (set useSSl in Authentication to True).
20120421
- Fix download by member and tags page detection.
20120413
- Update unicode handling.
- Add Ctrl-C handler.
20120406
- Add download by member_id and tags as requested by Drill.
- Update tags and list file unicode handing.
20120330
- update logging for ProcessImage and ProcessMember.
- update regex to handle private/MyPick-only image as reported by soji.
20120317
- Fix start/end page parameter parsing as reported by repu.
- Add option to use the query tags as the save folder (appended to root directory) as requested by Ruca.
- Update tags list parser to handle text files with BOM.
- Add blacklist and suppress tags list.
- useblacklisttags = True/False
Skip image if containing blacklisted tags.
The list is taken from blacklist_tags.txt, each tags is separated by new line.
- usesuppresstags = True/False
Remove the suppressed tags from %tags% meta for filename.
The list is taken from suppress_tags.txt, each tags is separated by new line.
- Add tagsLimit to be used for %tags% meta in the filename as requested by Anonymous User.
- Use -1 to use all tags.
- Fix useTagsAsDir (20120317b)
- Fix start/end page for Download by Tag list (20120317b)
20120309
- Fix list.txt parsing if have drive letter in the path as reported by Dwayne.
- Fix Download by Tags console command, add wildcard option [y/n] as reported by Dwayne.
- Fix Download by Title/Caption as reported by tribe.
20120224
- Fix filename sanitazion, replace tab character with space as reported by asm_demon.
- Source code formatting.
- Fix sanitizeFilename method for non-windows os as reported by Joe.
- Fix Mechanize 'req._tunnel_host = None' if no proxy used as reported by Joe.
- Add new filenameFormat metatag as requested by Yanoflies.
-> %works_date% ==> works date, complete with time.
-> %works_date_only% ==> only the works date.
-> %works_res% ==> image resolution, will be containing the page count if manga.
-> %works_tools% ==> tools used for the image.
- Update unittest.
20120210
- Fix no limit for end page as reported by Troid.
- Fix parseTags method as reported by gsilver.
- Add start/end date for download by tag as requested by UnemployedIncubator.
- Fix member bookmarks parsing as reported by Anonymous.
- Fix new illustration download if run from console as reported by ToadadnoChikara.
20120114
- Add BOM handler for list.txt.
- Add End Page for download new illust from bookmark as requested by Golden Wolf.
- Add Last Page Detection for download by tags as reported by tribe.
- Add use wilcard option for download by tags and download by Title/Caption as requested by tribe.
- Refactor Start/End page number UI.
20111212
- replace os.system() to subprocess.call()
20111202
- Fix PixivHelper, add import sys statement as reported by MikoMiko.
- Update Download/Export from online user bookmark: only process the private/hidden user bookmark.
- Update error logging message.
20111126
- Add error message if R-18/R-18G is disabled.
- Update console swith operation.
- Add max limit for Download New Illust from Bookmark to 100 (Pixiv limitation).
- Update readme.txt, add license agreement section (BSD 2-Clause License)
20111119
- Fix list.txt parsing the user target dir; strip() the line before split().
- Fix bookmark and DB exporting for member_id; remove space if no path defined.
- Change default value for rootdirectory to "." (same folder with apps).
- Add new error message for member_id page parsing.
- Update Console Title when downloading.
- Delete inactive member_id from DB.
- Update download by image bookmark to enter the member_id.
20111111
- Add filename format in the console title.
- Update bookmarks processing to include hidden bookmarks.
- Add download by image bookmarks.
- Add Download new illust from bookmarks (http://www.pixiv.net/bookmark_new_illust.php).
20111028
- Fix artist name parsing.
- Add download by tag list.
- Separate each tags by space or paste the encoded query string.
- Separate with new line for each query.
- Add Export user bookmarks.
- Add Database cleanup. Go to Manage Database -> Clean Up Database.
- Refactor Download by tags.
- Fix bookmark processing if more than 1 page (20111028b).
20111017
- Add download by user bookmark.
- Add createMangaDirectory option. The generated directory is based on the filenameFormat and the filename is
from the page marker '_p##'+ file extension.
20110922
- update DBManager to save custom folder from list.txt.
- fix download proxy error.
- change default value 'alwayscheckfilesize' to 'False'.
- refactor some method to new file.
20110916
- Fix filename bug if you are using member_id=directory format in the list.txt.
- Add option 'downloadavatar' to download artist profile avatar.
20110911
- Update Login process.
- Add filesize checking when downloading avatar image.
- Change the parser from mechanize.DefaultFactory() to mechanize.RobustFactory().
For downloading still using the default factory because there is a problem (ValueError: invalid literal for int()
with base 10: 'some_chars') if using robust factory (20110911b).
- Fix artist token parser when there is no profile image (20110911b).
20110821
- Fix filename for manga mode.
- Add strip() for list.txt parsing.
- Fix downloadlistdirectory parsing for list.txt.
- Update exception handling so the console window doesn't immidiately closed.
- Update the readme.txt.
- Fix DB saving (20110821b).
20110729
- add auto save cookie from login.
- refactor parsing logic to the PixivModel.
- update Python version to 2.7.2
- update mechanize version to 0.2.5
- update BeautifulSoup version to 3.2.0
- add unittest for the parsing logic
- fix 404 in big_manga mode (20110729b).
20110708
- Add cookie option for log in as requested by Demonoid.
- Add __br__.clear_history() in the process_list
20110513:
- Update ProcessByTags: Update with pixiv UI change, now based on regular expression to evaluate the links.
20110502:
- Update ProcessByTags: handle already encoded tags (prefixed with %).
20110327:
- Fixing the error handler, now it will be trying for number of times (set in the config.ini).
- Enable dumping the html page again, if the app cannot dumping the page, it will be listed in the log file.
20110227b:
- Update download by tags to accept kana.
20110227:
- handle httplib.BadStatusLine
- fix download by tag as informed by Sandoi&bububu.
- add start page in procress by tag.
20110205:
- add permission check for downloading image.
- fix overwrite mode.
- fix big manga again.
20110202 v2:
- Fix big manga mode again, they using jQuery template.
20110127:
- fix big manga mode
- add option checkUpdatedLimit to skip member by the count of already downloaded images.
20110101:
- by crust:
- Update manga mode to download the big version.
- merge the code and fix memory leak.
20101202b::
- add additional checking for suspended user.
20101024::
- Fix error when dumping html.
- Change exit behaviour, no need to press enter twice if you select exit.
20101005:
- by Yavos:
- added commandline-option: <-n number> to temporarily overwrite config-settings for <numberofpage>.
- added new options in config.ini (for details see readme.txt).
(<createDownloadLists>, <downloadListDirectory>, <startIrfanView>, <startIrfanSlide>, <IrfanViewPath>)
- added commandline-option <-i> to automatically start IrfanView with list of downloaded Images.
(this option only takes effect when <startIrfanView> and <startIrfanSlide> from config.ini are not set True)
- images will get the extension <.pixiv> until they are complete (prevent incomplete image-downloads).
- fix: all downloaded images will be added in DB (until now it was limited to images downloaded by member_id/list of member_ids).
(note: you can reset it again by setting <Yavos = False> in sourcecode line 27)
- added feature: when (directly or indirectly) set in config.ini pixivUtil will create lists of your downloaded images in <Downloaded_on_YYYY_MM_DD.txt> (using date when pixivUtil is launched).
- added feature: start IrfanView when exiting pixivUtil (Slideshow will be started before normal IrfanView-Window).
- added feature: typing <-all> in menu will temporarily activate/deactivate pagelimit set in config.ini (it will set commandline-option <-n 0>).
20100908::
- Updated find by tags.
- by Yavos:
- added commandline-OptionParser.
(new options: -x, -s <number>, arguments may be passed in combination with -s).
- removed <'> from forbidden symbols list.
- ending dots <.> in foldernames are removed automatically.
- added html-encoding for <'> -> <'>.
- special directorys for every member_id in list may be specified now.
- directorys like rootDirectory in config.ini don't need an ending <\> anymore.
- images by member_id can be downloaded automatically via commandline: <-s 1 (member_id(s))>.
- images by image_id can be downloaded automatically via commandline: <-s 2 (image_id(s))>.
- images by tags can be downloaded automatically via commandline: <-s 3 (tags)>.
- list can be downloaded automatically via commandline: <-s 4 (listfile)>.
- DB-Manger can be loaded via commandline: <-s 5>.
- programm can exit automatically when operation finished via commandline: <-x>.
- fix: output of imagelist in DB-Manger did not work when there are less than 10.000 pics.