Commit Graph

287 Commits (b1cf7ae4554e47b64b49b7e1827d731ea71a4cc8)

Author SHA1 Message Date
Joe Zhou b1cf7ae455 Set page status code for SeleniumDownloader. Fixes #1185 2 months ago
Joe Zhou 8dc417452a Update for next development version 2 months ago
Joe Zhou bf1088bd67 Update versions for release 2 months ago
Joe Zhou 6ed83769e0 Update for next development version 3 months ago
Joe Zhou 6eab7a4155 Update versions for release 3 months ago
Joe Zhou 541ced9eea Change the default status code from 200 to 0 & downloadSuccess from true to false, for Page. 4 months ago
xiezcGitHub 15ec80fcf1
FileCacheQueueScheduler使用BloomFilter进行去重 (#1176)
Co-authored-by: xiezc <blanexie@qq.com>
6 months ago
Joe Zhou 25c2d95e96 Update for next development version 7 months ago
Joe Zhou 3e9cd9b5c3 Update versions for release 7 months ago
Sutra Zhou 16a4fe3e28 Use oxerr-parent instead. 9 months ago
Sutra Zhou 38f240c42e Update to hotfix version 10 months ago
Sutra Zhou a81c4e7627 Update to hotfix version 10 months ago
Sutra Zhou 884f51ba3b Update to hotfix version 10 months ago
Joe Zhou 5196a56ccf Format code. 10 months ago
Joe Zhou b7e0d360ec Upgrade junit from 4.13.2 to 5.10.2. 10 months ago
François Gibier 2df7dca871
Changed refactor of processSingle again, this one is a better version (#1157)
* Refactor of processSingle in PageModelExtractor

* Changed my refactor of processSingle, this one is a lot better

* Changed my refactor of processSingle, this one is a lot better
10 months ago
François Gibier 05e5eefc7d
Refactor of processSingle in PageModelExtractor (#1155) 10 months ago
Sutra Zhou ed7429c293 Rename webmagic-parent to webmagic. 10 months ago
Sutra Zhou 4ebf48f6e3 Replace log4j 1.x with log4j 2.x, refs #534. 10 months ago
Joe Zhou 0ceaf14882 Bump version number from 0.10.1-SNAPSHOT to 1.0.0-SNAPSHOT for Java version updating from 1.8 to 11, refs #1134. 10 months ago
Sutra Zhou 31548deb93
Revert "Refactored code for increased optimization. (#1139)" (#1153)
This reverts commit f051d978e2.
10 months ago
Parthgajera056 f051d978e2
Refactored code for increased optimization. (#1139)
* refactoring by decompose conditional technique

* refactoring by introduction explaining variable technique

* refactoring by rename method/variable technique

* refactoring by introducing explaining variable technique

* Added Extract class refactoring to increase maintainablilty

* Refactoring using replace conditional with polymorphism
10 months ago
ayushi250317 9b9f173c1c
Refactored Code to increase maintainability (#1152)
* Initial Commit

* Assignment 1 Submission

* Resolving Implementation Smells

* Refactoring Code to increase maintainability
10 months ago
ayushi250317 28ac8bf9c4
Refactored Code to Resolve Implementation Code Smells (#1151)
* Initial Commit

* Assignment 1 Submission

* Resolving Implementation Smells
11 months ago
Sutra Zhou e4ab6e27e4 Optimize Request#extras, refs #1148. 11 months ago
Sutra Zhou 1e5c6488ff Update for next development version 1 year ago
Sutra Zhou 73dd2ebbac Update versions for release 1 year ago
Sutra Zhou 67644de3d9 Expose Page to onSuccess & onError. 1 year ago
Joe Zhou 8c008563ff Update for next development version 1 year ago
Joe Zhou 07e83a421b Update versions for release 1 year ago
Joe Zhou 6e8d130113 Update for next development version 2 years ago
Joe Zhou 3688226e32 Update versions for release 2 years ago
Joe Zhou a5fb4e0414 Upgrade dependencies. 2 years ago
Joe Zhou ef616c999e Fix warnings. 2 years ago
Joe Zhou cda8467f95 Update for next development version 2 years ago
Joe Zhou 888682863c Update versions for release 2 years ago
Sutra Zhou 5d5f3bf20e Update for next development version 2 years ago
Sutra Zhou 838c47f1f6 Update versions for release 2 years ago
Sutra Zhou d2b2eed9df Pass the task to onSuccess & onError. 2 years ago
vio.ao 7a62a6cb45 Revert "Revert "Common the downloader status process and pass error information when …""
This reverts commit acfbd7b883.
2 years ago
Sutra Zhou acfbd7b883
Revert "Common the downloader status process and pass error information when …" 2 years ago
vio.ao d01f26333b Common the downloader status process and pass error information when onError 2 years ago
David Hsing 54da7af17e change dependency versions into properties
change dependency versions into properties
update commons-collections from 3.x to 4.4
3 years ago
Sutra Zhou c5a037a807 Update for next development version 4 years ago
Sutra Zhou 4e51a4f68b Update versions for release 4 years ago
wecandoitjustthink 528a8908af 增加了List<SpiderStatusMXBean>属性的get方法,供SpiderMonitor的子类获取. 4 years ago
Sutra Zhou 71b7dfbf9a
Merge pull request #993 from yqia182/master
SpiderStatus中getPagePerSecond()方法,增加验证逻辑,避免空指针,避免除数为零。
4 years ago
JustThink 54127318a4 SpiderStatus中getPagePerSecond()方法,增加验证逻辑,避免空指针,避免除数为零。 4 years ago
Sutra Zhou 0e01550a79 Upgrade dependencies, including the jedis from 2.9.3 to 3.4.1. 4 years ago
Sutra Zhou c489647c4b Revert " Downloader 提供刷新组件的api,方便在spider中操作"
This reverts commit 2e2a0fdf3e.
4 years ago