Commit Graph

512 Commits (37666a7151f64a8446b1b4a4a563d59a65bdbcd8)
 

Author SHA1 Message Date
friddle 37666a7151 update the script 11 years ago
yihua.huang c1e7207869 add FileCacheQueueScheduler support for cycleRetryTimes 11 years ago
yihua.huang 969ad1766b change logger style to slf4j for cleaner code 11 years ago
yihua.huang 9b2cb43f47 ConfigurablePageProcessor #91 11 years ago
Yihua Huang 1090d070d9 Merge pull request #90 from ccliangbo/removeUnusedLines
Remove unused variable to make the project cleaner.
11 years ago
Bo LIANG 159eeea2f5 Remove unused variable to make the project cleaner. 11 years ago
yihua.huang c143fc662c add SubPageProcessor #86 11 years ago
Yihua Huang 2b2ce9ce13 Merge pull request #89 from ccliangbo/slf4jFormat
change the formatter of log.
11 years ago
Bo LIANG b043ac76d6 change the formatter of log.
To use slf4j, we should insert {} into the formatter string.
11 years ago
Yihua Huang 474f785dab Merge pull request #86 from sebastian1118/master
new feature: PatternProcessor
11 years ago
yihua.huang 8fe967ba8d [BugFix]exclude log4j.xml from maven jar plugin #82 11 years ago
Tian 38a12f8641 new feature: PatternProcessor 11 years ago
yihua.huang dafd0b5875 [BugFix]multi model in one pageprocessor will be skipped #85 11 years ago
yihua.huang 7aaf837e15 change logger to slf4j style for performance #84 11 years ago
yihua.huang f9b157951d Merge branch 'master' of github.com:code4craft/webmagic 11 years ago
yihua.huang 22c394e629 [doc] 11 years ago
Yihua Huang 3efa774191 Merge pull request #84 from ccliangbo/logInScheduler
Modify the log levels of LocalDuplicatedRemovedScheduler.java
11 years ago
Bo LIANG 762a3973fd Modify the log levels of LocalDuplicatedRemovedScheduler.java
The old version will print a debug level log each time the push method is
called. So sometimes, when a html page have multiple links for the same
page, the debug log will appears more than once. Also, when we meet a
duplicate URL, it will also print a log, which will be confusing.
I change the level of it to trace. And each time a URL is really push into
queue, print a debug level log.
11 years ago
yihua.huang 44293cd894 [doc]add qq group in readme 11 years ago
yihua.huang 9a0a4051ed [doc] ch3 part1 11 years ago
yihua.huang 7ca644cdd9 format readme 11 years ago
yihua.huang a1c7e826f7 fix dep of slf4j-log4j12 11 years ago
yihua.huang a34e92d11a fix huabanprocessor 11 years ago
yihua.huang 50cee4c7bb [doc] complete docs2.0 ch1 11 years ago
yihua.huang 9ec0ca02c6 doc2.0 ch1 11 years ago
yihua.huang 7e0e5b0969 clean ui 11 years ago
yihua.huang 94f97da4dc [Avalon] fix spring config for static and ignore google fonts for better loading speed 11 years ago
yihua.huang 22e8697671 add forger to folder 11 years ago
yihua.huang 05abd566a4 remove submodule 11 years ago
yihua.huang 01848301d4 encode illegal charactors in url #80 11 years ago
yihua.huang 2780423e60 enable blank space in quotes in UrlUtils.fixAllRelativeHrefs #80 11 years ago
yihua.huang 97b6f46280 Bugfix: break loop in addTargetRequests #81 11 years ago
yihua.huang d1563da33b add contributor 11 years ago
yihua.huang b13f1da039 reformat 11 years ago
yihua.huang 7038c00a9a reformat 11 years ago
yihua.huang 6252042ed2 add warning of slf4j #78 11 years ago
yihua.huang f3c2503a29 add warning of slf4j #78 11 years ago
yihua.huang 4e15a68ff4 update xsoup version to 0.2.2
Bugfix: Parsing error when separate chars in quotes cause in xsoup #77
11 years ago
yihua.huang 8d8194bee4 Change HashMap to LinkedHashMap in ResultItems for same order of input and output #76 11 years ago
yihua.huang c72483a220 update en_docs 11 years ago
yihua.huang e008b7b851 update zh_docs 11 years ago
yihua.huang 8958d774f2 add default values for @Formatter 11 years ago
yihua.huang 90447bff02 update xsoup to 0.2.1 #75 11 years ago
yihua.huang 8b35d79569 Do not cache document in Selectable for selected Html element #73 11 years ago
yihua.huang 6201fd6966 add worker as container 11 years ago
yihua.huang 6577a75892 fix spring config path in web.xml of webmagic-worker 11 years ago
yihua.huang c7169328c7 refactor config resource 11 years ago
yihua.huang 23d1151bc0 seperate spring config file 11 years ago
yihua.huang 316980973c invite h2 as test and standalone db, add spring profile for switch 11 years ago
yihua.huang 9a4eab44be add DynamicClassException to DynamicClassService.compileAndSave 11 years ago