Commit Graph

625 Commits (95494d3c4d9e0165299726b27019d138844ae474)
 

Author SHA1 Message Date
fengwuze 95494d3c4d 增加处理meta的逻辑。
遗留:
3、网页没有指定编码的情况下,需要采用cpdetector,但目前cpdetector这个在Maven的中央库里面没有,不清楚如何解决。
11 years ago
yihua.huang dde2d89bbe Ignore content in json when bracket when remove padding #124 11 years ago
Yihua Huang 2913da4763 Merge pull request #123 from gsh199449/master
Update JsonFilePipeline.java #122
11 years ago
yihua.huang 928f98dd93 auto create folder in JsonFilePipeline #122 11 years ago
GaoShen 5883ed93d7 Update JsonFilePipeline.java
JsonFilePipeline可以自动新建尚不存在的文件夹
11 years ago
Yihua Huang 4e65dac249 Merge pull request #121 from ywooer/master
创建指定编码的Writer
11 years ago
ywooer 259f0a16c5 Update FilePipeline.java 11 years ago
ywooer 26d38851b5 add charset to Writer 11 years ago
yihua.huang 7fbe18b8c0 implementation of PageMapper #120 11 years ago
yihua.huang 5dc9fe95a9 interface of PageMapper #120 11 years ago
yihua.huang 7668731f08 update version to snapshot 11 years ago
yihua.huang 5f6f489314 deperate in user manual 11 years ago
yihua.huang 81e6e772ac versions back to 0.5.1 11 years ago
yihua.huang dbebcbe44f docs 11 years ago
yihua.huang 358e906379 [maven-release-plugin] prepare for next development iteration 11 years ago
yihua.huang 470750fc0d [maven-release-plugin] prepare release WebMagic-0.5.1 11 years ago
yihua.huang fc3d2906b0 remove avalon from pom temporary 11 years ago
yihua.huang 01aec7e1ab extension point of geturl #118 11 years ago
yihua.huang ec1c2e8cbc test and so on 11 years ago
yihua.huang 4f22f1210e some bug fix #118 11 years ago
yihua.huang 186b90512e refactor redisscheduler #118 11 years ago
yihua.huang 56f033ce8d set setDuplicateRemover for chain api #118 11 years ago
yihua.huang d1140b9e29 add bloom filter for scheduler #118 11 years ago
yihua.huang 64293cba20 samples 11 years ago
yihua.huang bc1d14fed4 sample 11 years ago
yihua.huang 8e4814bdc5 fix path seperator 11 years ago
yihua.huang e8d4a9be2b fix remove duplicate error #117 11 years ago
yihua.huang 22652c4521 fix dep 11 years ago
yihua.huang f84a858bce update verion of forge 11 years ago
yihua.huang 5c00e59939 Merge branch 'stable' 11 years ago
yihua.huang 66692b2f74 update forger version 11 years ago
yihua.huang c07b32cd85 en docs 11 years ago
yihua.huang 3355624035 docs 11 years ago
yihua.huang a08d8cb167 update verion 11 years ago
yihua.huang 42a2676e8c update version 11 years ago
yihua.huang c892eadb56 contributor 11 years ago
yihua.huang 028f5e8755 readme 11 years ago
yihua.huang c25b32f1ca [maven-release-plugin] prepare for next development iteration 11 years ago
yihua.huang 7ff83bb11a [maven-release-plugin] prepare release WebMagic-0.5.0 11 years ago
yihua.huang dc3c175772 docs 11 years ago
yihua.huang 1104122979 more abstraction in scheduler 11 years ago
yihua.huang b0fb1c3e10 remove copy-dependcies plugin for m2e error 11 years ago
yihua.huang 94a67165e1 remove jmx server for simplify #98 11 years ago
yihua.huang 86a45a6643 change SpiderMonitor to singleton #98 11 years ago
yihua.huang ab4d36806e clean code 11 years ago
yihua.huang 04fde8203b add control for monitor 11 years ago
yihua.huang 2770811a10 update monitor example 11 years ago
yihua.huang 5ecd909ef2 add timeout for wait/notify #111 11 years ago
yihua.huang 964e637264 fix ut #110 11 years ago
yihua.huang c7afdb516e remove thread utils #110 11 years ago