Commit Graph

566 Commits (a7a06936f07152469daeaa85fd67a0b737231aa3)

Author SHA1 Message Date
Sutra Zhou 48bc73fbff New method Proxy#create. 5 years ago
Sutra Zhou 6d3f2d9b64 Wrap URISyntaxException as IllegalArgumentException for Proxy#toURI. 5 years ago
Sutra Zhou 236e5ade44 Update Proxy#toString(). 5 years ago
Sutra Zhou 791323a5b0 Add Proxy#scheme. 5 years ago
Sutra Zhou 2413366adb Format code, no actual code changed. 5 years ago
Sutra Zhou 7945c0612d Merge branch 'master' of github.com:code4craft/webmagic 5 years ago
Sutra Zhou 71aa04c89f Upgrade dependencies. 5 years ago
Sutra Zhou f24410d7bf
Merge pull request #885 from Diffblue-benchmarks/add-NumberUtils-tests
Add unit tests for us.codecraft.webmagic.utils.NumberUtils
5 years ago
Sutra Zhou ab5ac9d796 Fix test failure and javadoc failure. 5 years ago
Sutra Zhou 4078766d0e Change log level of supportedProtocols. 5 years ago
Sutra Zhou 3e42523141 TLSv1.3 requires Java 11 at least. Fixes #927. 5 years ago
Sutra Zhou 436af97346 Use spaces as indent. 5 years ago
Sutra Zhou ba1b4017a7 Mark slf4j-log4j12 as optional. 5 years ago
Sutra Zhou e7476cb8dc Make Request#getExtra be generic. 5 years ago
Sutra Zhou c46400d126 Fix javadoc of sleep time. Fixes #918. 5 years ago
Sutra Zhou fe3d52e2a4 Add TLSv1.3 support. 5 years ago
Thomas Perkins 2fd0e192fd Add unit tests for us.codecraft.webmagic.utils.NumberUtils
These tests were written using Diffblue Cover.
6 years ago
yihua.huang e5db538c19 #647 remove ThreadSafe annotation 7 years ago
yihua.huang b539522ca8 #701 support to tls1.2 7 years ago
yihua.huang e276b11ea9 vesion 0.7.3 8 years ago
王侃 a7e3090711 setBinaryContent, setCharset 方法返回this, 方便链式调用 8 years ago
yihua.huang 6f5b9e448e #627 set charset to request 8 years ago
yihua.huang 32f1f2cf44 #613 add charset to page 8 years ago
yihua.huang 65049baca4 #571 fix spell mistake 8 years ago
yihua.huang f375b9fdba #629 fix ut 8 years ago
yihua.huang 3266ea15ca #629 correct illegal url in HttpUriRequestConverter 8 years ago
yihua.huang 5daf92e8b2 #610 CASE_INSENSITIVE for charset detect in Content-Type 8 years ago
yihua.huang 2183ba9b61 #571 add getBytes to Page 8 years ago
yihua.huang c3bdb20458 #631 remove IllegalArgumentException of HttpRequestBody.json and so on 8 years ago
yihua.huang f405e642c0 add default constructor for HttpRequestBody #609 8 years ago
yihua.huang eb376fca74 update jsoup to 1.10.3 #608 8 years ago
yihua.huang faca38d4ec update jsonpath to 2.2.0 #606 8 years ago
yihua.huang 1b6394bef9 version 0.7.2 8 years ago
yihua.huang 3ee00015c2 change header from Authorization to Proxy-Authorization for Proxy Authorization #596 8 years ago
yihua.huang 4111b07263 more error log on page code error #601 8 years ago
yihua.huang 6bd1eed25e fix duplicate call of onSuccess and onError #605 8 years ago
yihua.huang 3a589d4ca6 HttpRequestBody implements Serializable #594 8 years ago
yihua.huang 13cdf82695 update version to 0.7.2-SNAPSHOT 8 years ago
yihua.huang 1e9187f24e version 0.7.1 8 years ago
yihua.huang 592fa2c0f1 add site header test 8 years ago
yihua.huang 19d34dbb65 not add bracket to regex in RegexSelector #559 8 years ago
Yihua Huang b6b991a09b Merge pull request #556 from zhuyuesut/master
增加对零宽断言的支持
8 years ago
yihua.huang bb0eb69acf update ZhihuPageProcessor example 8 years ago
yihua.huang 2e35e149be for 0.7.1 8 years ago
yihua.huang 17d8bfa907 docs and pgp version 8 years ago
yihua.huang 17478fcfc4 0.7.0 release 8 years ago
yihua.huang 636359300f add Site.disableCookieManagement #577 8 years ago
yihua.huang 49de9374cd new SimpleHttpClient #576 8 years ago
yihua.huang 8999ea9320 add public constructor for SimpleProxyProvider 8 years ago
yihua.huang a8c2e6c729 alpha release 8 years ago
yihua.huang 3c1338193b for 0.7.0.alpha 8 years ago
yihua.huang e8abc28072 #552 add some log when crawler stop 8 years ago
zhuyue 9e1b7ed3f7 Update RegexSelector.java 8 years ago
zhuyue c80f25edbd Update RegexSelectorTest.java
简单的增加了一点测试
8 years ago
zhuyue c3183252ac Update RegexSelector.java 8 years ago
yihua.huang cbf80af5dd test for SimpleProxyProvider #535 8 years ago
yihua.huang eb632a93d3 SimpleProxyProvider #535 8 years ago
yihua.huang d38d51dfcb fix javadoc 8 years ago
GZhY 5f34adf938 完善 LinksSelector.selectList 的测试用例 8 years ago
GZhY ce3f0ac239 删除 fixAllRelativeHrefs 并修复 SeleniumDownloader 对 fixAllRelativeHrefs 的依赖 8 years ago
GZhY bc6e81e00f 修复checkElementAndConvert方法注释中注释错误 8 years ago
yihua.huang 4a2c0f4f97 add returnProxy for proxyProvider 8 years ago
yihua.huang 1b04a7f2b3 #527 move logic check from downloaderto spider 8 years ago
yihua.huang 0f4d6e8b12 #525 remove port in UrlUtils.getDomain() 8 years ago
yihua.huang a1ae632b62 test for request cookies and headers 8 years ago
yihua.huang db67db8103 #523 remove fixAllRelativeHrefs by default, get absolute urls for links() 8 years ago
yihua.huang abd020b45b some comments 8 years ago
yihua.huang 2622b448b8 fix test 8 years ago
yihua.huang b06a248c00 fix test 8 years ago
yihua.huang 1cfbd13aae refacor in httpclientdownloader 8 years ago
yihua.huang 83ada9749e fix test 8 years ago
yihua.huang fe95a6842f Request再次重构:去掉params,仅保留HttpRequestBody 8 years ago
yihua.huang 395396c68e 增加HttpRequestBody 8 years ago
xbynet c93a8a2722 修复字符编码检测BUG 8 years ago
yihua.huang 74110e6ec5 remove useless file 8 years ago
yihua.huang b100dfe273 update version 8 years ago
xbynet@outlook.com 1c24baa8d1 Request支持设置header与cookie
新增POST请求时,XML、JSON参数支持
Page支持获取响应header
8 years ago
yihua.huang 6bd197859b fix test 8 years ago
yihua.huang f23e138c72 add response headers to Page #508 8 years ago
yihua.huang c13110c4cb fix samples 8 years ago
yihua.huang c51ac6017c remove Site.addStartRequest() etc. #494 8 years ago
yihua.huang 68050fc88e test pass 8 years ago
yihua.huang 474b7c9d57 refactor 8 years ago
yihua.huang 25c81013ca new proxy pool api 8 years ago
yihua.huang 46297deaa1 HttpUriRequestConverter 8 years ago
yihua.huang 1d86f7c048 compile passed in httpclientDownloader 8 years ago
yihua.huang b71f379512 fix 8 years ago
yihua.huang a7f9e7cad5 重构一部分httpclient 8 years ago
yihua.huang 221c155060 move release connection before return proxy #396 8 years ago
yihua.huang 68beff42c5 add test #493 8 years ago
wuyifan 79522f941e Bug, add null check to site in HttpClientDownloader & HttpClientGenerator 8 years ago
yihua.huang e9341d0291 complete test #447 8 years ago
yihua.huang e7d35c4846 add params to all method of request #447 8 years ago
yihua.huang 75bad591d7 rewrite hashCode and equals for params #447 8 years ago
Yihua Huang 11c32669b2 Merge pull request #447 from xbynet/master
简化POST参数设置.
8 years ago
yihua.huang aa01e27779 change constructor for Proxy to public #490 8 years ago
mei 791520e6a0 fix a bug of RegexSelector when regex has zero-width assertions. 8 years ago
yihua.huang c175ea88c0 #more test #484 8 years ago
yihua.huang 9b964c0a99 test for #484 8 years ago
yihua.huang fc702fd3b6 introduce mockito for test 8 years ago