Commit Graph

1157 Commits (snyk-fix-305ed17aec2839f91059daec3059c3ce)
 

Author SHA1 Message Date
Sutra Zhou 683db09133 Complete testXPath2 assertion. 4 years ago
Sutra Zhou 2f71f7912c Fix scm tag. 4 years ago
Sutra Zhou d0e2776991 Upgrade xsoup from 0.3.1 to 0.3.2. 4 years ago
Sutra Zhou 0e01550a79 Upgrade dependencies, including the jedis from 2.9.3 to 3.4.1. 4 years ago
Sutra Zhou 0d73f08ef6 Upgrade maven plugins. 4 years ago
Sutra Zhou e14a762632 Add gitflow-maven-plugin. 4 years ago
Sutra Zhou ab6ff7f809 Revert "pageCount修改"
This reverts commit 9a71f0ac92.
4 years ago
Sutra Zhou 30daec4803 Revert "提供出现某种异常刷新代理,异常可配置"
This reverts commit 4a6441e7c5.
4 years ago
Sutra Zhou d0843bee0d Revert "简化代码"
This reverts commit 9cc5287743.
4 years ago
Sutra Zhou 5ceccc62e0 Revert "提供异常刷新httpClient,异常可配置,重写getHttpClient代码"
This reverts commit 19465089c3.
4 years ago
Sutra Zhou 33e3fcdf22 Revert "代理接口的修改,提供刷星代理API。downloader 下载错误时,提供request,exception,proxyProvider三个参数,"
This reverts commit ba69eba669.
4 years ago
Sutra Zhou c489647c4b Revert " Downloader 提供刷新组件的api,方便在spider中操作"
This reverts commit 2e2a0fdf3e.
4 years ago
Sutra Zhou 4bedd97267 Revert " 刷新代理api重构,需要提供旧代理,如果依然是旧代理,才进行刷新,防止应延迟响应造成的过度刷新"
This reverts commit 0aa2c3949d.
4 years ago
Sutra Zhou 3f756c9325 Revert " 代理功能扩展,对原代理提供商进行拆分,加入lombok"
This reverts commit 33906e36f4.
4 years ago
Sutra Zhou aabc5584b8 Revert " bug修改,对结果提供缓存能力"
This reverts commit f68795d7dd.
4 years ago
Sutra Zhou 57dfc7cfb3
Merge pull request #977 from sutra/build
Remove useless imports to fix build.
4 years ago
Sutra Zhou 328c3e0d7d Remove useless imports to fix build. 4 years ago
Sutra Zhou 1d536cf705
Merge pull request #976 from yaoqiangpersonal/master
主要是对代理的功能进行了增加和修改
4 years ago
yao f68795d7dd bug修改,对结果提供缓存能力 4 years ago
yao 33906e36f4 代理功能扩展,对原代理提供商进行拆分,加入lombok 4 years ago
yao 0aa2c3949d 刷新代理api重构,需要提供旧代理,如果依然是旧代理,才进行刷新,防止应延迟响应造成的过度刷新 4 years ago
yao 2e2a0fdf3e Downloader 提供刷新组件的api,方便在spider中操作 4 years ago
yao 19465089c3 提供异常刷新httpClient,异常可配置,重写getHttpClient代码 4 years ago
yao 9cc5287743 简化代码 4 years ago
yao 4a6441e7c5 提供出现某种异常刷新代理,异常可配置 4 years ago
yao ba69eba669 代理接口的修改,提供刷星代理API。downloader 下载错误时,提供request,exception,proxyProvider三个参数, 4 years ago
Sutra Zhou b2aa5e2677
Merge pull request #974 from itranlin/master
子任务可以使用不同的下载器
4 years ago
itranlin fc7ae9ce28 子任务可以使用不同的下载器。。。 4 years ago
yao 9a71f0ac92 pageCount修改 4 years ago
Sutra Zhou 4b902270b4 Bump version number from 0.7.3 to 0.7.4. 4 years ago
Sutra Zhou 1895dba696
Merge pull request #959 from code4craft/snyk-fix-ad3d8147b4d9640ed91bfc34fbfb9e38
[Snyk] Security upgrade com.google.guava:guava from 29.0-jre to 30.0-android
4 years ago
snyk-bot c87af365d6
fix: pom.xml to reduce vulnerabilities
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-JAVA-COMGOOGLEGUAVA-1015415
4 years ago
Sutra Zhou 541f421011
Merge pull request #957 from code4craft/snyk-fix-9aefd7a47e1f776c68e7fad3e85033a4
[Snyk] Security upgrade junit:junit from 4.13 to 4.13.1
4 years ago
snyk-bot 2223552aeb
fix: pom.xml to reduce vulnerabilities
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-JAVA-JUNIT-1017047
4 years ago
Sutra Zhou 50c4e8ccfe
Merge pull request #955 from code4craft/snyk-fix-5cfb952010fd88492e4e2d5516c6237d
[Snyk] Security upgrade org.apache.httpcomponents:httpclient from 4.5.12 to 4.5.13
4 years ago
snyk-bot 7f737626b1
fix: pom.xml to reduce vulnerabilities
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-JAVA-ORGAPACHEHTTPCOMPONENTS-1016906
4 years ago
Sutra Zhou b4b1df85a0 Fix TLSv1.3. Maybe we should expose a API to allow user to use org.apache.http.ssl.SSLContextBuilder. Fixes #948. 4 years ago
Sutra Zhou 94ac7ca3b6
Merge pull request #946 from code4craft/snyk-fix-84ae966502c97386fb9c384e5874967b
[Snyk] Security upgrade com.alibaba:fastjson from 1.2.68 to 1.2.69
4 years ago
snyk-bot e3b3b9afdd
fix: pom.xml to reduce vulnerabilities
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-JAVA-COMALIBABA-570967
4 years ago
Sutra Zhou 96ebe608cc
Merge pull request #939 from bytesgo/manage-version
build: manage plugin version & remove build WARNING
5 years ago
leeyazhou 9aab25f339 build: manage plugin version & remove build WARNING
## use the new dependency of commons-io

[WARNING] The artifact org.apache.commons:commons-io:jar:1.3.2 has been
relocated to commons-io:commons-io:jar:1.3.2

## manage plugin version of maven-jar-plugin and maven-deploy-plugin

[WARNING]
[WARNING] Some problems were encountered while building the effective
model for us.codecraft:webmagic-core:jar:0.7.3
[WARNING] 'build.plugins.plugin.version' for
org.apache.maven.plugins:maven-jar-plugin is missing. @
us.codecraft:webmagic-parent:0.7.3, /opt/code/git/webmagic/pom.xml, line
263, column 21
[WARNING]
[WARNING] Some problems were encountered while building the effective
model for us.codecraft:webmagic-extension:jar:0.7.3
[WARNING] 'build.plugins.plugin.version' for
org.apache.maven.plugins:maven-jar-plugin is missing. @
us.codecraft:webmagic-parent:0.7.3, /opt/code/git/webmagic/pom.xml, line
263, column 21
[WARNING]
[WARNING] Some problems were encountered while building the effective
model for us.codecraft:webmagic-scripts:jar:0.7.3
[WARNING] 'build.plugins.plugin.version' for
org.apache.maven.plugins:maven-jar-plugin is missing. @ line 61, column
21
[WARNING]
[WARNING] Some problems were encountered while building the effective
model for us.codecraft:webmagic-selenium:jar:0.7.3
[WARNING] 'build.plugins.plugin.version' for
org.apache.maven.plugins:maven-jar-plugin is missing. @
us.codecraft:webmagic-parent:0.7.3, /opt/code/git/webmagic/pom.xml, line
263, column 21
[WARNING] 'build.plugins.plugin.version' for
org.apache.maven.plugins:maven-deploy-plugin is missing. @ line 34,
column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective
model for us.codecraft:webmagic-saxon:jar:0.7.3
[WARNING] 'build.plugins.plugin.version' for
org.apache.maven.plugins:maven-jar-plugin is missing. @
us.codecraft:webmagic-parent:0.7.3, /opt/code/git/webmagic/pom.xml, line
263, column 21
[WARNING] 'build.plugins.plugin.version' for
org.apache.maven.plugins:maven-deploy-plugin is missing. @ line 34,
column 21
[WARNING]
[WARNING] Some problems were encountered while building the effective
model for us.codecraft:webmagic-samples:jar:0.7.3
[WARNING] 'build.plugins.plugin.version' for
org.apache.maven.plugins:maven-jar-plugin is missing. @
us.codecraft:webmagic-parent:0.7.3, /opt/code/git/webmagic/pom.xml, line
263, column 21
[WARNING]
[WARNING] Some problems were encountered while building the effective
model for us.codecraft:webmagic-parent:pom:0.7.3
[WARNING] 'build.plugins.plugin.version' for
org.apache.maven.plugins:maven-jar-plugin is missing. @ line 263, column
21
[WARNING]
[WARNING] It is highly recommended to fix these problems because they
threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support
building such malformed projects.
5 years ago
Sutra Zhou 48bc73fbff New method Proxy#create. 5 years ago
Sutra Zhou 6d3f2d9b64 Wrap URISyntaxException as IllegalArgumentException for Proxy#toURI. 5 years ago
Sutra Zhou 236e5ade44 Update Proxy#toString(). 5 years ago
Sutra Zhou 791323a5b0 Add Proxy#scheme. 5 years ago
Sutra Zhou 2413366adb Format code, no actual code changed. 5 years ago
Sutra Zhou 5d14efc50f Serialize request URL only in FileCacheQueueScheduler. 5 years ago
Sutra Zhou 7945c0612d Merge branch 'master' of github.com:code4craft/webmagic 5 years ago
Sutra Zhou 71aa04c89f Upgrade dependencies. 5 years ago
Sutra Zhou 6c05dd8b72 Upgrade maven plugins. 5 years ago