github-forks
/
webmagic
mirror of https://github.com/code4craft/webmagic.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
1,304 Commits
50 Branches
37 Tags
19 MiB
Commit Graph

284 Commits (0a9fe8d3e03e58c96e497efdc727ce6d09684229)

Author SHA1 Message Date
Joe Zhou 6ed83769e0 Update for next development version
Joe Zhou 6eab7a4155 Update versions for release
Joe Zhou 541ced9eea Change the default status code from 200 to 0 & downloadSuccess from true to false, for Page.
xiezcGitHub 15ec80fcf1
FileCacheQueueScheduler使用BloomFilter进行去重 () 
Co-authored-by: xiezc <blanexie@qq.com>
Joe Zhou 25c2d95e96 Update for next development version
Joe Zhou 3e9cd9b5c3 Update versions for release
Sutra Zhou 16a4fe3e28 Use oxerr-parent instead.
Sutra Zhou 38f240c42e Update to hotfix version
Sutra Zhou a81c4e7627 Update to hotfix version
Sutra Zhou 884f51ba3b Update to hotfix version
Joe Zhou 5196a56ccf Format code.
Joe Zhou b7e0d360ec Upgrade junit from 4.13.2 to 5.10.2.
François Gibier 2df7dca871
Changed refactor of processSingle again, this one is a better version () 
* Refactor of processSingle in PageModelExtractor

* Changed my refactor of processSingle, this one is a lot better

* Changed my refactor of processSingle, this one is a lot better
François Gibier 05e5eefc7d
Refactor of processSingle in PageModelExtractor ()
Sutra Zhou ed7429c293 Rename webmagic-parent to webmagic.
Sutra Zhou 4ebf48f6e3 Replace log4j 1.x with log4j 2.x, refs .
Joe Zhou 0ceaf14882 Bump version number from 0.10.1-SNAPSHOT to 1.0.0-SNAPSHOT for Java version updating from 1.8 to 11, refs .
Sutra Zhou 31548deb93
Revert "Refactored code for increased optimization. ()" () 
This reverts commit f051d978e2.
Parthgajera056 f051d978e2
Refactored code for increased optimization. () 
* refactoring by decompose conditional technique

* refactoring by introduction explaining variable technique

* refactoring by rename method/variable technique

* refactoring by introducing explaining variable technique

* Added Extract class refactoring to increase maintainablilty

* Refactoring using replace conditional with polymorphism
ayushi250317 9b9f173c1c
Refactored Code to increase maintainability () 
* Initial Commit

* Assignment 1 Submission

* Resolving Implementation Smells

* Refactoring Code to increase maintainability
ayushi250317 28ac8bf9c4
Refactored Code to Resolve Implementation Code Smells () 
* Initial Commit

* Assignment 1 Submission

* Resolving Implementation Smells
Sutra Zhou e4ab6e27e4 Optimize Request#extras, refs .
Sutra Zhou 1e5c6488ff Update for next development version
Sutra Zhou 73dd2ebbac Update versions for release
Sutra Zhou 67644de3d9 Expose Page to onSuccess & onError.
Joe Zhou 8c008563ff Update for next development version
Joe Zhou 07e83a421b Update versions for release
Joe Zhou 6e8d130113 Update for next development version
Joe Zhou 3688226e32 Update versions for release
Joe Zhou a5fb4e0414 Upgrade dependencies.
Joe Zhou ef616c999e Fix warnings.
Joe Zhou cda8467f95 Update for next development version
Joe Zhou 888682863c Update versions for release
Sutra Zhou 5d5f3bf20e Update for next development version
Sutra Zhou 838c47f1f6 Update versions for release
Sutra Zhou d2b2eed9df Pass the task to onSuccess & onError.
vio.ao 7a62a6cb45 Revert "Revert "Common the downloader status process and pass error information when …"" 
This reverts commit acfbd7b883.
Sutra Zhou acfbd7b883
Revert "Common the downloader status process and pass error information when …"
vio.ao d01f26333b Common the downloader status process and pass error information when onError
David Hsing 54da7af17e change dependency versions into properties 
change dependency versions into properties
update commons-collections from 3.x to 4.4
Sutra Zhou c5a037a807 Update for next development version
Sutra Zhou 4e51a4f68b Update versions for release
wecandoitjustthink 528a8908af 增加了List<SpiderStatusMXBean>属性的get方法,供SpiderMonitor的子类获取.
Sutra Zhou 71b7dfbf9a
Merge pull request from yqia182/master 
SpiderStatus中getPagePerSecond()方法，增加验证逻辑，避免空指针，避免除数为零。
JustThink 54127318a4 SpiderStatus中getPagePerSecond()方法，增加验证逻辑，避免空指针，避免除数为零。
Sutra Zhou 0e01550a79 Upgrade dependencies, including the jedis from 2.9.3 to 3.4.1.
Sutra Zhou c489647c4b Revert " Downloader 提供刷新组件的api,方便在spider中操作" 
This reverts commit 2e2a0fdf3e.
yao 2e2a0fdf3e Downloader 提供刷新组件的api,方便在spider中操作
Sutra Zhou 4b902270b4 Bump version number from 0.7.3 to 0.7.4.
Sutra Zhou 5d14efc50f Serialize request URL only in FileCacheQueueScheduler.