552 Commits (d01f26333bb75561e80596932397512f83b177d4)

Author	SHA1	Message	Date
vio.ao	d01f26333b	Common the downloader status process and pass error information when onError	2 years ago
David Hsing	54da7af17e	change dependency versions into properties ... change dependency versions into properties update commons-collections from 3.x to 4.4	3 years ago
Sutra Zhou	692605bd75	Merge branch 'develop' of github.com:code4craft/webmagic into develop	3 years ago
Sutra Zhou	34da2fb3a0	Make PageProcessor#getSite be default method. Closes #1040.	3 years ago
carl.don:tjr	fcdb9074d6	格式化 Spider.run 代码	4 years ago
carl.don:tjr	ab5d81a6b6	perfect Spider.run to avoid some rare concurrent issue, change the Spider.emptySleepTime to long type	4 years ago
Sutra Zhou	c5a037a807	Update for next development version	4 years ago
Sutra Zhou	4e51a4f68b	Update versions for release	4 years ago
Sutra Zhou	be6f5ff771	Add missing @Deprecated annotations.	4 years ago
Sutra Zhou	4e8a086dae	Pass exception to onError. Fixes #1005.	4 years ago
Sutra Zhou	d0e2776991	Upgrade xsoup from 0.3.1 to 0.3.2.	4 years ago
Sutra Zhou	0e01550a79	Upgrade dependencies, including the jedis from 2.9.3 to 3.4.1.	4 years ago
Sutra Zhou	ab6ff7f809	Revert "pageCount修改" ... This reverts commit 9a71f0ac92.	4 years ago
Sutra Zhou	30daec4803	Revert "提供出现某种异常刷新代理，异常可配置" ... This reverts commit 4a6441e7c5.	4 years ago
Sutra Zhou	d0843bee0d	Revert "简化代码" ... This reverts commit 9cc5287743.	4 years ago
Sutra Zhou	5ceccc62e0	Revert "提供异常刷新httpClient，异常可配置，重写getHttpClient代码" ... This reverts commit 19465089c3.	4 years ago
Sutra Zhou	33e3fcdf22	Revert "代理接口的修改，提供刷星代理API。downloader 下载错误时，提供request,exception,proxyProvider三个参数，" ... This reverts commit ba69eba669.	4 years ago
Sutra Zhou	c489647c4b	Revert " Downloader 提供刷新组件的api,方便在spider中操作" ... This reverts commit 2e2a0fdf3e.	4 years ago
Sutra Zhou	4bedd97267	Revert " 刷新代理api重构，需要提供旧代理，如果依然是旧代理，才进行刷新，防止应延迟响应造成的过度刷新" ... This reverts commit 0aa2c3949d.	4 years ago
Sutra Zhou	3f756c9325	Revert " 代理功能扩展，对原代理提供商进行拆分，加入lombok" ... This reverts commit 33906e36f4.	4 years ago
Sutra Zhou	aabc5584b8	Revert " bug修改，对结果提供缓存能力" ... This reverts commit f68795d7dd.	4 years ago
Sutra Zhou	328c3e0d7d	Remove useless imports to fix build.	4 years ago
Sutra Zhou	1d536cf705	Merge pull request #976 from yaoqiangpersonal/master ... 主要是对代理的功能进行了增加和修改	4 years ago
yao	f68795d7dd	bug修改，对结果提供缓存能力	4 years ago
yao	33906e36f4	代理功能扩展，对原代理提供商进行拆分，加入lombok	4 years ago
yao	0aa2c3949d	刷新代理api重构，需要提供旧代理，如果依然是旧代理，才进行刷新，防止应延迟响应造成的过度刷新	4 years ago
yao	2e2a0fdf3e	Downloader 提供刷新组件的api,方便在spider中操作	4 years ago
yao	19465089c3	提供异常刷新httpClient，异常可配置，重写getHttpClient代码	4 years ago
yao	9cc5287743	简化代码	4 years ago
yao	4a6441e7c5	提供出现某种异常刷新代理，异常可配置	4 years ago
yao	ba69eba669	代理接口的修改，提供刷星代理API。downloader 下载错误时，提供request,exception,proxyProvider三个参数，	4 years ago
itranlin	fc7ae9ce28	子任务可以使用不同的下载器。。。	4 years ago
yao	9a71f0ac92	pageCount修改	4 years ago
Sutra Zhou	4b902270b4	Bump version number from 0.7.3 to 0.7.4.	4 years ago
Sutra Zhou	b4b1df85a0	Fix TLSv1.3. Maybe we should expose a API to allow user to use org.apache.http.ssl.SSLContextBuilder. Fixes #948.	4 years ago
leeyazhou	9aab25f339	build: manage plugin version & remove build WARNING ... ## use the new dependency of commons-io [WARNING] The artifact org.apache.commons:commons-io:jar:1.3.2 has been relocated to commons-io:commons-io:jar:1.3.2 ## manage plugin version of maven-jar-plugin and maven-deploy-plugin [WARNING] [WARNING] Some problems were encountered while building the effective model for us.codecraft:webmagic-core:jar:0.7.3 [WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-jar-plugin is missing. @ us.codecraft:webmagic-parent:0.7.3, /opt/code/git/webmagic/pom.xml, line 263, column 21 [WARNING] [WARNING] Some problems were encountered while building the effective model for us.codecraft:webmagic-extension:jar:0.7.3 [WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-jar-plugin is missing. @ us.codecraft:webmagic-parent:0.7.3, /opt/code/git/webmagic/pom.xml, line 263, column 21 [WARNING] [WARNING] Some problems were encountered while building the effective model for us.codecraft:webmagic-scripts:jar:0.7.3 [WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-jar-plugin is missing. @ line 61, column 21 [WARNING] [WARNING] Some problems were encountered while building the effective model for us.codecraft:webmagic-selenium:jar:0.7.3 [WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-jar-plugin is missing. @ us.codecraft:webmagic-parent:0.7.3, /opt/code/git/webmagic/pom.xml, line 263, column 21 [WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-deploy-plugin is missing. @ line 34, column 12 [WARNING] [WARNING] Some problems were encountered while building the effective model for us.codecraft:webmagic-saxon:jar:0.7.3 [WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-jar-plugin is missing. @ us.codecraft:webmagic-parent:0.7.3, /opt/code/git/webmagic/pom.xml, line 263, column 21 [WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-deploy-plugin is missing. @ line 34, column 21 [WARNING] [WARNING] Some problems were encountered while building the effective model for us.codecraft:webmagic-samples:jar:0.7.3 [WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-jar-plugin is missing. @ us.codecraft:webmagic-parent:0.7.3, /opt/code/git/webmagic/pom.xml, line 263, column 21 [WARNING] [WARNING] Some problems were encountered while building the effective model for us.codecraft:webmagic-parent:pom:0.7.3 [WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-jar-plugin is missing. @ line 263, column 21 [WARNING] [WARNING] It is highly recommended to fix these problems because they threaten the stability of your build. [WARNING] [WARNING] For this reason, future Maven versions might no longer support building such malformed projects.	5 years ago
Sutra Zhou	48bc73fbff	New method Proxy#create.	5 years ago
Sutra Zhou	6d3f2d9b64	Wrap URISyntaxException as IllegalArgumentException for Proxy#toURI.	5 years ago
Sutra Zhou	236e5ade44	Update Proxy#toString().	5 years ago
Sutra Zhou	791323a5b0	Add Proxy#scheme.	5 years ago
Sutra Zhou	2413366adb	Format code, no actual code changed.	5 years ago
Sutra Zhou	7945c0612d	Merge branch 'master' of github.com:code4craft/webmagic	5 years ago
Sutra Zhou	71aa04c89f	Upgrade dependencies.	5 years ago
Sutra Zhou	f24410d7bf	Merge pull request #885 from Diffblue-benchmarks/add-NumberUtils-tests ... Add unit tests for us.codecraft.webmagic.utils.NumberUtils	5 years ago
Sutra Zhou	ab5ac9d796	Fix test failure and javadoc failure.	5 years ago
Sutra Zhou	4078766d0e	Change log level of supportedProtocols.	5 years ago
Sutra Zhou	3e42523141	TLSv1.3 requires Java 11 at least. Fixes #927.	5 years ago
Sutra Zhou	436af97346	Use spaces as indent.	5 years ago
Sutra Zhou	ba1b4017a7	Mark slf4j-log4j12 as optional.	5 years ago
Sutra Zhou	e7476cb8dc	Make Request#getExtra be generic.	5 years ago