Commit Graph

214 Commits (0e6eb46ebaf049d2be3203ae9423bc1b18e3d1b7)

Author SHA1 Message Date
yihua.huang 0e6eb46eba add test for jsonpath in pagemodel #462 8 years ago
yihua.huang f5018d569e remove test_issue409 #409 8 years ago
yihua.huang 17ae500c77 test_issue409 #409 8 years ago
yihua.huang 4cd5b4f93e test_issue409 8 years ago
yihua.huang fb0acd710c complete SimpleHttpClientTest 8 years ago
yihua.huang d07941d900 SimpleHttpClientTest 8 years ago
yihua.huang f02f469c69 add test #570 8 years ago
yihua.huang 2d693580fc add test 8 years ago
yihua.huang b879b0eed0 fix redisscheduler #583 8 years ago
Yihua Huang 9903f0367d Merge pull request #570 from SoulZhong/master
修复formatter初始化未传参bug
8 years ago
yihua.huang 2e35e149be for 0.7.1 8 years ago
yihua.huang 17478fcfc4 0.7.0 release 8 years ago
yihua.huang 49de9374cd new SimpleHttpClient #576 8 years ago
yihua.huang 7ffc6998ef add isExtractLinks to OOSpider #575 8 years ago
soul bc828e1384 修复formatter初始化未传参bug 8 years ago
yihua.huang a8c2e6c729 alpha release 8 years ago
yihua.huang 3c1338193b for 0.7.0.alpha 8 years ago
yihua.huang d38d51dfcb fix javadoc 8 years ago
yihua.huang 1b04a7f2b3 #527 move logic check from downloaderto spider 8 years ago
yihua.huang 74110e6ec5 remove useless file 8 years ago
yihua.huang b100dfe273 update version 8 years ago
yihua.huang c13110c4cb fix samples 8 years ago
yihua.huang d87c73b472 change check-and-set to atomic sadd for redis DuplicateRemover #368 8 years ago
yihua.huang aaccc93215 new version 8 years ago
yihua.huang 3e633c6871 version 8 years ago
yihua.huang f45e2f118b for release 8 years ago
Jsbd 6d78d51fc0 Merge branch 'master' into master 8 years ago
yihua.huang d69204b919 0.6.0 8 years ago
yihua.huang 97592d6720 Version 0.6.0 8 years ago
yihua.huang 00dfebbceb #424 remove guava dep and add fix docs 8 years ago
yihua.huang a960a39c44 fix compile error for example change 8 years ago
yihua.huang a3ee9e3d08 fix example 8 years ago
Jsbd 1b886d48a2 新增PhantomJSDownloader构造函数,支持crawl.js路径自定义,因为当其他项目依赖此jar包时,runtime.exec()执行phantomjs命令时无使用法jar包中的crawl.js 8 years ago
Jsbd d1f2e65e5d 新增PhantomJSDownloader构造函数,支持crawl.js路径自定义,因为当其他项目依赖此jar包时,runtime.exec()执行phantomjs命令时无使用法jar包中的crawl.js 8 years ago
Jsbd ebc61363c8 为PhantomJSDownloader添加新的构造函数,支持phantomjs自定义命令
为PhantomJSDownloader添加新的构造函数,支持phantomjs自定义命令
 example: 
   *    phantomjs.exe 支持windows环境
   *    phantomjs --ignore-ssl-errors=yes 忽略抓取地址是https时的一些错误
   *    /usr/local/bin/phantomjs 命令的绝对路径,避免因系统环境变量引起的IOException
8 years ago
yihua.huang b92e6b04f0 #400 修复FileCacheQueueScheduler自己设置DuplicateRemover会导致NPE的问题 8 years ago
Yihua Huang 1491033534 Merge pull request #377 from jerry-sc/monitor-bug
fix the monitor bug which the spider will terminate when a seed url with port
8 years ago
Jerry e56b8c3efc fix the monitor bug which the spider will terminate when a seed url with port 9 years ago
郭玉昆 700898fe8a fixed #301 修复使用注解抽取JSON数据的问题 9 years ago
Salon.sai f89a6a6826 add: redis scheduler with priority 9 years ago
Yihua Huang 37cb43b667 Merge pull request #176 from lavenderx/master
add PhantomJSDownloader
9 years ago
Linker Lin 047cb8ff8f updated versions to 0.5.4-SNAPSHOT 9 years ago
yihua.huang c0b8e8f8ae remove .classpath .project 9 years ago
yihua.huang a8e6de4b90 Merge branch 'master' of git.oschina.net:flashsword20/webmagic 9 years ago
yihua.huang ce5495ecd5 remove useless files 9 years ago
yihua.huang 8265c7dade remove submodules for relase 9 years ago
yihua.huang 7edfa26f90 complete javadoc 9 years ago
yihua.huang 8b90b91e33 complete some javadoc 9 years ago
yihua.huang 2b556cf053 update verison to 0.5.3-SNAPSHOT 9 years ago
yihua.huang 9c5716a543 complete javadoc 9 years ago