Commit Graph

552 Commits (d01f26333bb75561e80596932397512f83b177d4)

Author SHA1 Message Date
Sutra Zhou c46400d126 Fix javadoc of sleep time. Fixes #918. 5 years ago
Sutra Zhou fe3d52e2a4 Add TLSv1.3 support. 5 years ago
Thomas Perkins 2fd0e192fd Add unit tests for us.codecraft.webmagic.utils.NumberUtils
These tests were written using Diffblue Cover.
6 years ago
yihua.huang e5db538c19 #647 remove ThreadSafe annotation 7 years ago
yihua.huang b539522ca8 #701 support to tls1.2 7 years ago
yihua.huang e276b11ea9 vesion 0.7.3 8 years ago
王侃 a7e3090711 setBinaryContent, setCharset 方法返回this, 方便链式调用 8 years ago
yihua.huang 6f5b9e448e #627 set charset to request 8 years ago
yihua.huang 32f1f2cf44 #613 add charset to page 8 years ago
yihua.huang 65049baca4 #571 fix spell mistake 8 years ago
yihua.huang f375b9fdba #629 fix ut 8 years ago
yihua.huang 3266ea15ca #629 correct illegal url in HttpUriRequestConverter 8 years ago
yihua.huang 5daf92e8b2 #610 CASE_INSENSITIVE for charset detect in Content-Type 8 years ago
yihua.huang 2183ba9b61 #571 add getBytes to Page 8 years ago
yihua.huang c3bdb20458 #631 remove IllegalArgumentException of HttpRequestBody.json and so on 8 years ago
yihua.huang f405e642c0 add default constructor for HttpRequestBody #609 8 years ago
yihua.huang eb376fca74 update jsoup to 1.10.3 #608 8 years ago
yihua.huang faca38d4ec update jsonpath to 2.2.0 #606 8 years ago
yihua.huang 1b6394bef9 version 0.7.2 8 years ago
yihua.huang 3ee00015c2 change header from Authorization to Proxy-Authorization for Proxy Authorization #596 8 years ago
yihua.huang 4111b07263 more error log on page code error #601 8 years ago
yihua.huang 6bd1eed25e fix duplicate call of onSuccess and onError #605 8 years ago
yihua.huang 3a589d4ca6 HttpRequestBody implements Serializable #594 8 years ago
yihua.huang 13cdf82695 update version to 0.7.2-SNAPSHOT 8 years ago
yihua.huang 1e9187f24e version 0.7.1 8 years ago
yihua.huang 592fa2c0f1 add site header test 8 years ago
yihua.huang 19d34dbb65 not add bracket to regex in RegexSelector #559 8 years ago
Yihua Huang b6b991a09b Merge pull request #556 from zhuyuesut/master
增加对零宽断言的支持
8 years ago
yihua.huang bb0eb69acf update ZhihuPageProcessor example 8 years ago
yihua.huang 2e35e149be for 0.7.1 8 years ago
yihua.huang 17d8bfa907 docs and pgp version 8 years ago
yihua.huang 17478fcfc4 0.7.0 release 8 years ago
yihua.huang 636359300f add Site.disableCookieManagement #577 8 years ago
yihua.huang 49de9374cd new SimpleHttpClient #576 8 years ago
yihua.huang 8999ea9320 add public constructor for SimpleProxyProvider 8 years ago
yihua.huang a8c2e6c729 alpha release 8 years ago
yihua.huang 3c1338193b for 0.7.0.alpha 8 years ago
yihua.huang e8abc28072 #552 add some log when crawler stop 8 years ago
zhuyue 9e1b7ed3f7 Update RegexSelector.java 8 years ago
zhuyue c80f25edbd Update RegexSelectorTest.java
简单的增加了一点测试
8 years ago
zhuyue c3183252ac Update RegexSelector.java 8 years ago
yihua.huang cbf80af5dd test for SimpleProxyProvider #535 8 years ago
yihua.huang eb632a93d3 SimpleProxyProvider #535 8 years ago
yihua.huang d38d51dfcb fix javadoc 8 years ago
GZhY 5f34adf938 完善 LinksSelector.selectList 的测试用例 8 years ago
GZhY ce3f0ac239 删除 fixAllRelativeHrefs 并修复 SeleniumDownloader 对 fixAllRelativeHrefs 的依赖 8 years ago
GZhY bc6e81e00f 修复checkElementAndConvert方法注释中注释错误 8 years ago
yihua.huang 4a2c0f4f97 add returnProxy for proxyProvider 8 years ago
yihua.huang 1b04a7f2b3 #527 move logic check from downloaderto spider 8 years ago
yihua.huang 0f4d6e8b12 #525 remove port in UrlUtils.getDomain() 8 years ago
yihua.huang a1ae632b62 test for request cookies and headers 8 years ago
yihua.huang db67db8103 #523 remove fixAllRelativeHrefs by default, get absolute urls for links() 8 years ago
yihua.huang abd020b45b some comments 8 years ago
yihua.huang 2622b448b8 fix test 8 years ago
yihua.huang b06a248c00 fix test 8 years ago
yihua.huang 1cfbd13aae refacor in httpclientdownloader 8 years ago
yihua.huang 83ada9749e fix test 8 years ago
yihua.huang fe95a6842f Request再次重构:去掉params,仅保留HttpRequestBody 8 years ago
yihua.huang 395396c68e 增加HttpRequestBody 8 years ago
xbynet c93a8a2722 修复字符编码检测BUG 8 years ago
yihua.huang 74110e6ec5 remove useless file 8 years ago
yihua.huang b100dfe273 update version 8 years ago
xbynet@outlook.com 1c24baa8d1 Request支持设置header与cookie
新增POST请求时,XML、JSON参数支持
Page支持获取响应header
8 years ago
yihua.huang 6bd197859b fix test 8 years ago
yihua.huang f23e138c72 add response headers to Page #508 8 years ago
yihua.huang c13110c4cb fix samples 8 years ago
yihua.huang c51ac6017c remove Site.addStartRequest() etc. #494 8 years ago
yihua.huang 68050fc88e test pass 8 years ago
yihua.huang 474b7c9d57 refactor 8 years ago
yihua.huang 25c81013ca new proxy pool api 8 years ago
yihua.huang 46297deaa1 HttpUriRequestConverter 8 years ago
yihua.huang 1d86f7c048 compile passed in httpclientDownloader 8 years ago
yihua.huang b71f379512 fix 8 years ago
yihua.huang a7f9e7cad5 重构一部分httpclient 8 years ago
yihua.huang 221c155060 move release connection before return proxy #396 8 years ago
yihua.huang 68beff42c5 add test #493 8 years ago
wuyifan 79522f941e Bug, add null check to site in HttpClientDownloader & HttpClientGenerator 8 years ago
yihua.huang e9341d0291 complete test #447 8 years ago
yihua.huang e7d35c4846 add params to all method of request #447 8 years ago
yihua.huang 75bad591d7 rewrite hashCode and equals for params #447 8 years ago
Yihua Huang 11c32669b2 Merge pull request #447 from xbynet/master
简化POST参数设置.
8 years ago
yihua.huang aa01e27779 change constructor for Proxy to public #490 8 years ago
mei 791520e6a0 fix a bug of RegexSelector when regex has zero-width assertions. 8 years ago
yihua.huang c175ea88c0 #more test #484 8 years ago
yihua.huang 9b964c0a99 test for #484 8 years ago
yihua.huang fc702fd3b6 introduce mockito for test 8 years ago
yihua.huang 5215a492cc remove duplicate check for POST request #484 8 years ago
yihua.huang 0a1fb19052 add tests #483 8 years ago
yihua.huang a2e7f0004b Merge branch 'master' of github.com:code4craft/webmagic 8 years ago
yihua.huang ef32571821 rewrite Request.equals and hashCode, add Method to check #483 8 years ago
yihua.huang 8b8f535c30 refactor:extract charset detect to utils 8 years ago
Ckex.zha e645524ad2 fix bug,set ExecutorService 8 years ago
yihua.huang a872a6480e fix code sample for github #348 8 years ago
yihua.huang 1d2171805f add test for #228 8 years ago
yihua.huang bbe0b52ddd remove synchronized in QueueScheduler #410 8 years ago
yihua.huang ad69963005 remove synchronize in Page #411 8 years ago
yihua.huang 3a796b9413 remove duplicate code #421 8 years ago
yihua.huang 42f1018010 remove messy code 8 years ago
xbynet 650468c0e4 解决POST中文参数乱码问题 8 years ago
yihua.huang aaccc93215 new version 8 years ago