Commit Graph

287 Commits (7efc9872df972511ba365d4a310e6cb450020a6e)

Author SHA1 Message Date
Joe Zhou b1cf7ae455 Set page status code for SeleniumDownloader. Fixes #1185 1 month ago
Joe Zhou 8dc417452a Update for next development version 2 months ago
Joe Zhou bf1088bd67 Update versions for release 2 months ago
Joe Zhou 6ed83769e0 Update for next development version 3 months ago
Joe Zhou 6eab7a4155 Update versions for release 3 months ago
Joe Zhou 541ced9eea Change the default status code from 200 to 0 & downloadSuccess from true to false, for Page. 4 months ago
xiezcGitHub 15ec80fcf1
FileCacheQueueScheduler使用BloomFilter进行去重 (#1176)
Co-authored-by: xiezc <blanexie@qq.com>
6 months ago
Joe Zhou 25c2d95e96 Update for next development version 7 months ago
Joe Zhou 3e9cd9b5c3 Update versions for release 7 months ago
Sutra Zhou 16a4fe3e28 Use oxerr-parent instead. 9 months ago
Sutra Zhou 38f240c42e Update to hotfix version 9 months ago
Sutra Zhou a81c4e7627 Update to hotfix version 9 months ago
Sutra Zhou 884f51ba3b Update to hotfix version 9 months ago
Joe Zhou 5196a56ccf Format code. 9 months ago
Joe Zhou b7e0d360ec Upgrade junit from 4.13.2 to 5.10.2. 9 months ago
François Gibier 2df7dca871
Changed refactor of processSingle again, this one is a better version (#1157)
* Refactor of processSingle in PageModelExtractor

* Changed my refactor of processSingle, this one is a lot better

* Changed my refactor of processSingle, this one is a lot better
10 months ago
François Gibier 05e5eefc7d
Refactor of processSingle in PageModelExtractor (#1155) 10 months ago
Sutra Zhou ed7429c293 Rename webmagic-parent to webmagic. 10 months ago
Sutra Zhou 4ebf48f6e3 Replace log4j 1.x with log4j 2.x, refs #534. 10 months ago
Joe Zhou 0ceaf14882 Bump version number from 0.10.1-SNAPSHOT to 1.0.0-SNAPSHOT for Java version updating from 1.8 to 11, refs #1134. 10 months ago
Sutra Zhou 31548deb93
Revert "Refactored code for increased optimization. (#1139)" (#1153)
This reverts commit f051d978e2.
10 months ago
Parthgajera056 f051d978e2
Refactored code for increased optimization. (#1139)
* refactoring by decompose conditional technique

* refactoring by introduction explaining variable technique

* refactoring by rename method/variable technique

* refactoring by introducing explaining variable technique

* Added Extract class refactoring to increase maintainablilty

* Refactoring using replace conditional with polymorphism
10 months ago
ayushi250317 9b9f173c1c
Refactored Code to increase maintainability (#1152)
* Initial Commit

* Assignment 1 Submission

* Resolving Implementation Smells

* Refactoring Code to increase maintainability
10 months ago
ayushi250317 28ac8bf9c4
Refactored Code to Resolve Implementation Code Smells (#1151)
* Initial Commit

* Assignment 1 Submission

* Resolving Implementation Smells
10 months ago
Sutra Zhou e4ab6e27e4 Optimize Request#extras, refs #1148. 11 months ago
Sutra Zhou 1e5c6488ff Update for next development version 1 year ago
Sutra Zhou 73dd2ebbac Update versions for release 1 year ago
Sutra Zhou 67644de3d9 Expose Page to onSuccess & onError. 1 year ago
Joe Zhou 8c008563ff Update for next development version 1 year ago
Joe Zhou 07e83a421b Update versions for release 1 year ago
Joe Zhou 6e8d130113 Update for next development version 2 years ago
Joe Zhou 3688226e32 Update versions for release 2 years ago
Joe Zhou a5fb4e0414 Upgrade dependencies. 2 years ago
Joe Zhou ef616c999e Fix warnings. 2 years ago
Joe Zhou cda8467f95 Update for next development version 2 years ago
Joe Zhou 888682863c Update versions for release 2 years ago
Sutra Zhou 5d5f3bf20e Update for next development version 2 years ago
Sutra Zhou 838c47f1f6 Update versions for release 2 years ago
Sutra Zhou d2b2eed9df Pass the task to onSuccess & onError. 2 years ago
vio.ao 7a62a6cb45 Revert "Revert "Common the downloader status process and pass error information when …""
This reverts commit acfbd7b883.
2 years ago
Sutra Zhou acfbd7b883
Revert "Common the downloader status process and pass error information when …" 2 years ago
vio.ao d01f26333b Common the downloader status process and pass error information when onError 2 years ago
David Hsing 54da7af17e change dependency versions into properties
change dependency versions into properties
update commons-collections from 3.x to 4.4
3 years ago
Sutra Zhou c5a037a807 Update for next development version 4 years ago
Sutra Zhou 4e51a4f68b Update versions for release 4 years ago
wecandoitjustthink 528a8908af 增加了List<SpiderStatusMXBean>属性的get方法,供SpiderMonitor的子类获取. 4 years ago
Sutra Zhou 71b7dfbf9a
Merge pull request #993 from yqia182/master
SpiderStatus中getPagePerSecond()方法,增加验证逻辑,避免空指针,避免除数为零。
4 years ago
JustThink 54127318a4 SpiderStatus中getPagePerSecond()方法,增加验证逻辑,避免空指针,避免除数为零。 4 years ago
Sutra Zhou 0e01550a79 Upgrade dependencies, including the jedis from 2.9.3 to 3.4.1. 4 years ago
Sutra Zhou c489647c4b Revert " Downloader 提供刷新组件的api,方便在spider中操作"
This reverts commit 2e2a0fdf3e.
4 years ago