607 Commits (6be02f3a0fdbb672033af5092f3a343728b7e3a8)

Author	SHA1	Message	Date
Sutra Zhou	5f80e02abd	Interrupt current thread.	2 years ago
Sutra Zhou	126c32ecd0	Fix compatible issue.	2 years ago
vio.ao	e7a7fbeeeb	Enhance Jsoup could parse tr td tag directly	2 years ago
vio.ao	7a62a6cb45	Revert "Revert "Common the downloader status process and pass error information when …"" ... This reverts commit acfbd7b883.	2 years ago
Sutra Zhou	acfbd7b883	Revert "Common the downloader status process and pass error information when …"	2 years ago
vio.ao	d01f26333b	Common the downloader status process and pass error information when onError	2 years ago
David Hsing	54da7af17e	change dependency versions into properties ... change dependency versions into properties update commons-collections from 3.x to 4.4	3 years ago
Sutra Zhou	692605bd75	Merge branch 'develop' of github.com:code4craft/webmagic into develop	3 years ago
Sutra Zhou	34da2fb3a0	Make PageProcessor#getSite be default method. Closes #1040.	3 years ago
carl.don:tjr	fcdb9074d6	格式化 Spider.run 代码	4 years ago
carl.don:tjr	ab5d81a6b6	perfect Spider.run to avoid some rare concurrent issue, change the Spider.emptySleepTime to long type	4 years ago
Sutra Zhou	c5a037a807	Update for next development version	4 years ago
Sutra Zhou	4e51a4f68b	Update versions for release	4 years ago
Sutra Zhou	be6f5ff771	Add missing @Deprecated annotations.	4 years ago
Sutra Zhou	4e8a086dae	Pass exception to onError. Fixes #1005.	4 years ago
Sutra Zhou	d0e2776991	Upgrade xsoup from 0.3.1 to 0.3.2.	4 years ago
Sutra Zhou	0e01550a79	Upgrade dependencies, including the jedis from 2.9.3 to 3.4.1.	4 years ago
Sutra Zhou	ab6ff7f809	Revert "pageCount修改" ... This reverts commit 9a71f0ac92.	4 years ago
Sutra Zhou	30daec4803	Revert "提供出现某种异常刷新代理，异常可配置" ... This reverts commit 4a6441e7c5.	4 years ago
Sutra Zhou	d0843bee0d	Revert "简化代码" ... This reverts commit 9cc5287743.	4 years ago
Sutra Zhou	5ceccc62e0	Revert "提供异常刷新httpClient，异常可配置，重写getHttpClient代码" ... This reverts commit 19465089c3.	4 years ago
Sutra Zhou	33e3fcdf22	Revert "代理接口的修改，提供刷星代理API。downloader 下载错误时，提供request,exception,proxyProvider三个参数，" ... This reverts commit ba69eba669.	4 years ago
Sutra Zhou	c489647c4b	Revert " Downloader 提供刷新组件的api,方便在spider中操作" ... This reverts commit 2e2a0fdf3e.	4 years ago
Sutra Zhou	4bedd97267	Revert " 刷新代理api重构，需要提供旧代理，如果依然是旧代理，才进行刷新，防止应延迟响应造成的过度刷新" ... This reverts commit 0aa2c3949d.	4 years ago
Sutra Zhou	3f756c9325	Revert " 代理功能扩展，对原代理提供商进行拆分，加入lombok" ... This reverts commit 33906e36f4.	4 years ago
Sutra Zhou	aabc5584b8	Revert " bug修改，对结果提供缓存能力" ... This reverts commit f68795d7dd.	4 years ago
Sutra Zhou	328c3e0d7d	Remove useless imports to fix build.	4 years ago
Sutra Zhou	1d536cf705	Merge pull request #976 from yaoqiangpersonal/master ... 主要是对代理的功能进行了增加和修改	4 years ago
yao	f68795d7dd	bug修改，对结果提供缓存能力	4 years ago
yao	33906e36f4	代理功能扩展，对原代理提供商进行拆分，加入lombok	4 years ago
yao	0aa2c3949d	刷新代理api重构，需要提供旧代理，如果依然是旧代理，才进行刷新，防止应延迟响应造成的过度刷新	4 years ago
yao	2e2a0fdf3e	Downloader 提供刷新组件的api,方便在spider中操作	4 years ago
yao	19465089c3	提供异常刷新httpClient，异常可配置，重写getHttpClient代码	4 years ago
yao	9cc5287743	简化代码	4 years ago
yao	4a6441e7c5	提供出现某种异常刷新代理，异常可配置	4 years ago
yao	ba69eba669	代理接口的修改，提供刷星代理API。downloader 下载错误时，提供request,exception,proxyProvider三个参数，	4 years ago
itranlin	fc7ae9ce28	子任务可以使用不同的下载器。。。	4 years ago
yao	9a71f0ac92	pageCount修改	4 years ago
Sutra Zhou	4b902270b4	Bump version number from 0.7.3 to 0.7.4.	4 years ago
Sutra Zhou	b4b1df85a0	Fix TLSv1.3. Maybe we should expose a API to allow user to use org.apache.http.ssl.SSLContextBuilder. Fixes #948.	4 years ago
leeyazhou	9aab25f339	build: manage plugin version & remove build WARNING ... ## use the new dependency of commons-io [WARNING] The artifact org.apache.commons:commons-io:jar:1.3.2 has been relocated to commons-io:commons-io:jar:1.3.2 ## manage plugin version of maven-jar-plugin and maven-deploy-plugin [WARNING] [WARNING] Some problems were encountered while building the effective model for us.codecraft:webmagic-core:jar:0.7.3 [WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-jar-plugin is missing. @ us.codecraft:webmagic-parent:0.7.3, /opt/code/git/webmagic/pom.xml, line 263, column 21 [WARNING] [WARNING] Some problems were encountered while building the effective model for us.codecraft:webmagic-extension:jar:0.7.3 [WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-jar-plugin is missing. @ us.codecraft:webmagic-parent:0.7.3, /opt/code/git/webmagic/pom.xml, line 263, column 21 [WARNING] [WARNING] Some problems were encountered while building the effective model for us.codecraft:webmagic-scripts:jar:0.7.3 [WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-jar-plugin is missing. @ line 61, column 21 [WARNING] [WARNING] Some problems were encountered while building the effective model for us.codecraft:webmagic-selenium:jar:0.7.3 [WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-jar-plugin is missing. @ us.codecraft:webmagic-parent:0.7.3, /opt/code/git/webmagic/pom.xml, line 263, column 21 [WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-deploy-plugin is missing. @ line 34, column 12 [WARNING] [WARNING] Some problems were encountered while building the effective model for us.codecraft:webmagic-saxon:jar:0.7.3 [WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-jar-plugin is missing. @ us.codecraft:webmagic-parent:0.7.3, /opt/code/git/webmagic/pom.xml, line 263, column 21 [WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-deploy-plugin is missing. @ line 34, column 21 [WARNING] [WARNING] Some problems were encountered while building the effective model for us.codecraft:webmagic-samples:jar:0.7.3 [WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-jar-plugin is missing. @ us.codecraft:webmagic-parent:0.7.3, /opt/code/git/webmagic/pom.xml, line 263, column 21 [WARNING] [WARNING] Some problems were encountered while building the effective model for us.codecraft:webmagic-parent:pom:0.7.3 [WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-jar-plugin is missing. @ line 263, column 21 [WARNING] [WARNING] It is highly recommended to fix these problems because they threaten the stability of your build. [WARNING] [WARNING] For this reason, future Maven versions might no longer support building such malformed projects.	5 years ago
Sutra Zhou	48bc73fbff	New method Proxy#create.	5 years ago
Sutra Zhou	6d3f2d9b64	Wrap URISyntaxException as IllegalArgumentException for Proxy#toURI.	5 years ago
Sutra Zhou	236e5ade44	Update Proxy#toString().	5 years ago
Sutra Zhou	791323a5b0	Add Proxy#scheme.	5 years ago
Sutra Zhou	2413366adb	Format code, no actual code changed.	5 years ago
Sutra Zhou	7945c0612d	Merge branch 'master' of github.com:code4craft/webmagic	5 years ago
Sutra Zhou	71aa04c89f	Upgrade dependencies.	5 years ago
Sutra Zhou	f24410d7bf	Merge pull request #885 from Diffblue-benchmarks/add-NumberUtils-tests ... Add unit tests for us.codecraft.webmagic.utils.NumberUtils	5 years ago
Sutra Zhou	ab5ac9d796	Fix test failure and javadoc failure.	5 years ago
Sutra Zhou	4078766d0e	Change log level of supportedProtocols.	5 years ago
Sutra Zhou	3e42523141	TLSv1.3 requires Java 11 at least. Fixes #927.	5 years ago
Sutra Zhou	436af97346	Use spaces as indent.	5 years ago
Sutra Zhou	ba1b4017a7	Mark slf4j-log4j12 as optional.	5 years ago
Sutra Zhou	e7476cb8dc	Make Request#getExtra be generic.	5 years ago
Sutra Zhou	c46400d126	Fix javadoc of sleep time. Fixes #918.	5 years ago
Sutra Zhou	fe3d52e2a4	Add TLSv1.3 support.	5 years ago
Thomas Perkins	2fd0e192fd	Add unit tests for us.codecraft.webmagic.utils.NumberUtils ... These tests were written using Diffblue Cover.	6 years ago
yihua.huang	e5db538c19	#647 remove ThreadSafe annotation	7 years ago
yihua.huang	b539522ca8	#701 support to tls1.2	7 years ago
yihua.huang	e276b11ea9	vesion 0.7.3	8 years ago
王侃	a7e3090711	setBinaryContent, setCharset 方法返回this, 方便链式调用	8 years ago
yihua.huang	6f5b9e448e	#627 set charset to request	8 years ago
yihua.huang	32f1f2cf44	#613 add charset to page	8 years ago
yihua.huang	65049baca4	#571 fix spell mistake	8 years ago
yihua.huang	f375b9fdba	#629 fix ut	8 years ago
yihua.huang	3266ea15ca	#629 correct illegal url in HttpUriRequestConverter	8 years ago
yihua.huang	5daf92e8b2	#610 CASE_INSENSITIVE for charset detect in Content-Type	8 years ago
yihua.huang	2183ba9b61	#571 add getBytes to Page	8 years ago
yihua.huang	c3bdb20458	#631 remove IllegalArgumentException of HttpRequestBody.json and so on	8 years ago
yihua.huang	f405e642c0	add default constructor for HttpRequestBody #609	8 years ago
yihua.huang	eb376fca74	update jsoup to 1.10.3 #608	8 years ago
yihua.huang	faca38d4ec	update jsonpath to 2.2.0 #606	8 years ago
yihua.huang	1b6394bef9	version 0.7.2	8 years ago
yihua.huang	3ee00015c2	change header from Authorization to Proxy-Authorization for Proxy Authorization #596	8 years ago
yihua.huang	4111b07263	more error log on page code error #601	8 years ago
yihua.huang	6bd1eed25e	fix duplicate call of onSuccess and onError #605	8 years ago
yihua.huang	3a589d4ca6	HttpRequestBody implements Serializable #594	8 years ago
yihua.huang	13cdf82695	update version to 0.7.2-SNAPSHOT	8 years ago
yihua.huang	1e9187f24e	version 0.7.1	8 years ago
yihua.huang	592fa2c0f1	add site header test	8 years ago
yihua.huang	19d34dbb65	not add bracket to regex in RegexSelector #559	8 years ago
Yihua Huang	b6b991a09b	Merge pull request #556 from zhuyuesut/master ... 增加对零宽断言的支持	8 years ago
yihua.huang	bb0eb69acf	update ZhihuPageProcessor example	8 years ago
yihua.huang	2e35e149be	for 0.7.1	8 years ago
yihua.huang	17d8bfa907	docs and pgp version	8 years ago
yihua.huang	17478fcfc4	0.7.0 release	8 years ago
yihua.huang	636359300f	add Site.disableCookieManagement #577	8 years ago
yihua.huang	49de9374cd	new SimpleHttpClient #576	8 years ago
yihua.huang	8999ea9320	add public constructor for SimpleProxyProvider	8 years ago
yihua.huang	a8c2e6c729	alpha release	8 years ago
yihua.huang	3c1338193b	for 0.7.0.alpha	8 years ago
yihua.huang	e8abc28072	#552 add some log when crawler stop	8 years ago
zhuyue	9e1b7ed3f7	Update RegexSelector.java	8 years ago
zhuyue	c80f25edbd	Update RegexSelectorTest.java ... 简单的增加了一点测试	8 years ago
zhuyue	c3183252ac	Update RegexSelector.java	8 years ago
yihua.huang	cbf80af5dd	test for SimpleProxyProvider #535	8 years ago
yihua.huang	eb632a93d3	SimpleProxyProvider #535	8 years ago
yihua.huang	d38d51dfcb	fix javadoc	8 years ago
GZhY	5f34adf938	完善 LinksSelector.selectList 的测试用例	8 years ago