Sutra Zhou
|
acfbd7b883
|
Revert "Common the downloader status process and pass error information when …"
|
2 years ago |
Sutra Zhou
|
ee5a0585d7
|
Merge pull request #1082 from vioao/common-downloader-status-process
Common the downloader status process and pass error information when …
|
2 years ago |
vio.ao
|
d01f26333b
|
Common the downloader status process and pass error information when onError
|
2 years ago |
Sutra Zhou
|
16221e391d
|
Fix xhtml namespace.
|
3 years ago |
Sutra Zhou
|
6747046b6c
|
Merge pull request #1067 from yokelsoft/develop
change dependency versions into properties
|
3 years ago |
David Hsing
|
54da7af17e
|
change dependency versions into properties
change dependency versions into properties
update commons-collections from 3.x to 4.4
|
3 years ago |
Sutra Zhou
|
692605bd75
|
Merge branch 'develop' of github.com:code4craft/webmagic into develop
|
3 years ago |
Sutra Zhou
|
34da2fb3a0
|
Make PageProcessor#getSite be default method. Closes #1040.
|
3 years ago |
Sutra Zhou
|
f110147b62
|
Merge pull request #1033 from carl-don-it/perfect_spider_run
perfect Spider.run to avoid some rare concurrent issue, change the Sp…
|
4 years ago |
carl.don:tjr
|
fcdb9074d6
|
格式化 Spider.run 代码
|
4 years ago |
carl.don:tjr
|
ab5d81a6b6
|
perfect Spider.run to avoid some rare concurrent issue, change the Spider.emptySleepTime to long type
|
4 years ago |
Sutra Zhou
|
c5a037a807
|
Update for next development version
|
4 years ago |
Sutra Zhou
|
4024230e17
|
Merge branch 'release/0.7.5' into develop
|
4 years ago |
Sutra Zhou
|
113eaa4bae
|
Bump version number to 0.7.5.
|
4 years ago |
Sutra Zhou
|
4e51a4f68b
|
Update versions for release
|
4 years ago |
Sutra Zhou
|
e3e66fb270
|
Upgrade webmagic-samples dependencies.
|
4 years ago |
Sutra Zhou
|
14b09a3385
|
Update maven plugin versions.
|
4 years ago |
Sutra Zhou
|
31a00f5f8e
|
Set gitflow-maven-plugin versionTagPrefix.
|
4 years ago |
Sutra Zhou
|
db70b6e095
|
Add maven reports.
|
4 years ago |
Sutra Zhou
|
c7828f44ac
|
Merge pull request #1025 from gkorland/patch-1
Update to Jedis 3.6.0
|
4 years ago |
Guy Korland
|
189c5962e6
|
Update to Jedis 3.6.0
|
4 years ago |
Sutra Zhou
|
dba0ddb92c
|
Remove unknown tag from javadoc.
|
4 years ago |
Sutra Zhou
|
133106a15c
|
Merge pull request #1013 from FreaxLin/Restore_the_already_crawled_example
提交可恢复爬取内容例子
|
4 years ago |
linweisen
|
76f625c02e
|
提交可恢复爬取内容例子
|
4 years ago |
Sutra Zhou
|
be6f5ff771
|
Add missing @Deprecated annotations.
|
4 years ago |
Sutra Zhou
|
4e8a086dae
|
Pass exception to onError. Fixes #1005.
|
4 years ago |
Sutra Zhou
|
dcfd238413
|
Polish java version setting.
|
4 years ago |
Sutra Zhou
|
59fc16101b
|
Merge pull request #1000 from thebirdandfish/develop
增加了List<SpiderStatusMXBean>属性的get方法,供SpiderMonitor的子类获取.
|
4 years ago |
wecandoitjustthink
|
528a8908af
|
增加了List<SpiderStatusMXBean>属性的get方法,供SpiderMonitor的子类获取.
|
4 years ago |
Sutra Zhou
|
71b7dfbf9a
|
Merge pull request #993 from yqia182/master
SpiderStatus中getPagePerSecond()方法,增加验证逻辑,避免空指针,避免除数为零。
|
4 years ago |
JustThink
|
54127318a4
|
SpiderStatus中getPagePerSecond()方法,增加验证逻辑,避免空指针,避免除数为零。
|
4 years ago |
Sutra Zhou
|
d92dc8397f
|
Upgrade htmlcleaner from 2.5 to 2.9, this is the highest version to let Xpath2Selector pass the test cases.
|
4 years ago |
Sutra Zhou
|
124c52b988
|
Downgrade htmlcleaner from 2.24 back to 2.5, to make Xpath2Selector pass the test cases.
|
4 years ago |
Sutra Zhou
|
683db09133
|
Complete testXPath2 assertion.
|
4 years ago |
Sutra Zhou
|
2f71f7912c
|
Fix scm tag.
|
4 years ago |
Sutra Zhou
|
d0e2776991
|
Upgrade xsoup from 0.3.1 to 0.3.2.
|
4 years ago |
Sutra Zhou
|
0e01550a79
|
Upgrade dependencies, including the jedis from 2.9.3 to 3.4.1.
|
4 years ago |
Sutra Zhou
|
0d73f08ef6
|
Upgrade maven plugins.
|
4 years ago |
Sutra Zhou
|
e14a762632
|
Add gitflow-maven-plugin.
|
4 years ago |
Sutra Zhou
|
ab6ff7f809
|
Revert "pageCount修改"
This reverts commit 9a71f0ac92 .
|
4 years ago |
Sutra Zhou
|
30daec4803
|
Revert "提供出现某种异常刷新代理,异常可配置"
This reverts commit 4a6441e7c5 .
|
4 years ago |
Sutra Zhou
|
d0843bee0d
|
Revert "简化代码"
This reverts commit 9cc5287743 .
|
4 years ago |
Sutra Zhou
|
5ceccc62e0
|
Revert "提供异常刷新httpClient,异常可配置,重写getHttpClient代码"
This reverts commit 19465089c3 .
|
4 years ago |
Sutra Zhou
|
33e3fcdf22
|
Revert "代理接口的修改,提供刷星代理API。downloader 下载错误时,提供request,exception,proxyProvider三个参数,"
This reverts commit ba69eba669 .
|
4 years ago |
Sutra Zhou
|
c489647c4b
|
Revert " Downloader 提供刷新组件的api,方便在spider中操作"
This reverts commit 2e2a0fdf3e .
|
4 years ago |
Sutra Zhou
|
4bedd97267
|
Revert " 刷新代理api重构,需要提供旧代理,如果依然是旧代理,才进行刷新,防止应延迟响应造成的过度刷新"
This reverts commit 0aa2c3949d .
|
4 years ago |
Sutra Zhou
|
3f756c9325
|
Revert " 代理功能扩展,对原代理提供商进行拆分,加入lombok"
This reverts commit 33906e36f4 .
|
4 years ago |
Sutra Zhou
|
aabc5584b8
|
Revert " bug修改,对结果提供缓存能力"
This reverts commit f68795d7dd .
|
4 years ago |
Sutra Zhou
|
57dfc7cfb3
|
Merge pull request #977 from sutra/build
Remove useless imports to fix build.
|
4 years ago |
Sutra Zhou
|
328c3e0d7d
|
Remove useless imports to fix build.
|
4 years ago |