1125 Commits (e3e66fb270782813cc35e1189949dd4ac2465299)

 

All Branches   

Search

Author	SHA1	Message	Date
yihua.huang	a960a39c44	fix compile error for example change	8 years ago
yihua.huang	a3ee9e3d08	fix example	8 years ago
yihua.huang	7476ceccee	more stable test	8 years ago
yihua.huang	5ce3fdfe5a	some refactor in log	8 years ago
yihua.huang	98163a3e40	update examples	8 years ago
yihua.huang	243ebc22fa	#374 update httpclient version to 4.5.2	8 years ago
yihua.huang	b090dcd20d	sepcific error page for HttpClientDownloaderTest to avoid test error when local port is available	8 years ago
yihua.huang	8f942d6fe2	#419 修复抓取https链接线程无法结束导致进程一直运行的问题	8 years ago
Jsbd	1b886d48a2	新增PhantomJSDownloader构造函数，支持crawl.js路径自定义，因为当其他项目依赖此jar包时，runtime.exec()执行phantomjs命令时无使用法jar包中的crawl.js	8 years ago
Jsbd	d1f2e65e5d	新增PhantomJSDownloader构造函数，支持crawl.js路径自定义，因为当其他项目依赖此jar包时，runtime.exec()执行phantomjs命令时无使用法jar包中的crawl.js	8 years ago
Jsbd	f8a2328ead	Merge pull request #1 from code4craft/master ... merge	8 years ago
Yihua Huang	1987cd3ae1	Merge pull request #408 from code4craft/0.6.0 ... groovy demo	8 years ago
Yihua Huang	65fe2c4487	Merge pull request #407 from jsbd/master ... 为PhantomJSDownloader添加新的构造函数，支持phantomjs自定义命令	8 years ago
Jsbd	ebc61363c8	为PhantomJSDownloader添加新的构造函数，支持phantomjs自定义命令 ... 为PhantomJSDownloader添加新的构造函数，支持phantomjs自定义命令 example: * phantomjs.exe 支持windows环境 * phantomjs --ignore-ssl-errors=yes 忽略抓取地址是https时的一些错误 * /usr/local/bin/phantomjs 命令的绝对路径，避免因系统环境变量引起的IOException	8 years ago
yihua.huang	fdf39eb99d	open in new page	8 years ago
yihua.huang	1e74494708	add related link	8 years ago
yihua.huang	b92e6b04f0	#400 修复FileCacheQueueScheduler自己设置DuplicateRemover会导致NPE的问题	8 years ago
yihua.huang	dafd2b77ff	fix GithubRepoPageProcessor in example	8 years ago
yihua.huang	cfed860fb9	Merge branch 'master' of github.com:code4craft/webmagic	8 years ago
yihua.huang	2189aab652	fix test	8 years ago
Yihua Huang	1491033534	Merge pull request #377 from jerry-sc/monitor-bug ... fix the monitor bug which the spider will terminate when a seed url with port	8 years ago
Yihua Huang	228911b58c	Merge pull request #370 from gyk001/master ... fixed #301 修复使用注解抽取JSON数据的问题	8 years ago
yihua.huang	507556d0aa	fix test: ProxyTest.testProxy() do not load exist proxy config	8 years ago
yihua.huang	55f131e5ef	#380 update fastjson to 1.2.21	8 years ago
Jerry	e56b8c3efc	fix the monitor bug which the spider will terminate when a seed url with port	9 years ago
郭玉昆	700898fe8a	fixed #301 修复使用注解抽取JSON数据的问题	9 years ago
Yihua Huang	e22d6426fc	Merge pull request #343 from Salon-sai/master ... add: redis scheduler with priority	9 years ago
Salon.sai	f89a6a6826	add: redis scheduler with priority	9 years ago
yihua.huang	448e528140	update StringUtils to apache lang3 #314	9 years ago
yihua.huang	3e33959b7a	#319 fix javadoc	9 years ago
yihua.huang	3a6e246350	Merge branch 'kapsterio-fix'	9 years ago
yihua.huang	8730e3e97a	Merge branch 'fix' of git://github.com/kapsterio/webmagic into kapsterio-fix	9 years ago
Yihua Huang	37cb43b667	Merge pull request #176 from lavenderx/master ... add PhantomJSDownloader	9 years ago
yihua.huang	2400ff7e1a	resovle conflict	9 years ago
yihua.huang	9de64ea0f2	Merge branch 'hepan-master'	9 years ago
yihua.huang	b7f3c4bba0	Merge branch 'master' of git://github.com/hepan/webmagic into hepan-master	9 years ago
yihua.huang	d8f978fd20	fix test in JsonPathSelectorTest #289	9 years ago
yihua.huang	25821c3c06	Merge branch 'EdwardsBean-proxy-strategy'	9 years ago
yihua.huang	61c28a0130	refactor on proxypool	9 years ago
yihua.huang	b871b210c5	Merge branch 'proxy-strategy' of github.com:EdwardsBean/webmagic into EdwardsBean-proxy-strategy	9 years ago
yihua.huang	b5413368de	update ut	9 years ago
yihua.huang	a8c9005381	Merge branch 'oxf1-master'	9 years ago
Jon	83c27ebbc4	增加IP代理认证功能	9 years ago
yihua.huang	ca072c5575	fix URL regex in GithubRepoPageProcessor #305	9 years ago
hepan	89c6e52863	代理增加用户名密码认证	9 years ago
Linker Lin	047cb8ff8f	updated versions to 0.5.4-SNAPSHOT	9 years ago
Linker Lin	b9544424f2	Update pom.xml ... 回滚。。。 我想，主要的问题还是主版本号没了 SNAPSHOT	9 years ago
Linker Lin	b05d605c4b	Update pom.xml ... 在纯净的环境中，需要做上面的改动。	9 years ago
zhangheng09	6b179c3d55	这个改动的原因基于两点：1）代理归还给代理池的时机应该是执行完http请求后就要尽早归还 2)http代理应该是HttpClientDownloader该考虑的事，不应该有Spider来处理，Spider并不知道它的downloader是个HttpClientDownloader	9 years ago
zhangheng09	5f106c9c69	当page为null时，意味着非正常的响应状态，应该抛出异常，否则SpiderListener的onSuccess方法和onError方法都会执行	9 years ago