paoding分词器 词典大于30M 出现错误

lc82111 2010-09-29
出现的错误如下:



在paoding maker里啦!
处理词典完毕!
compiler.readCompliedDictionaries(p);
get knivf 完毕
knives length:0 当前i=0
net.paoding.analysis.exception.PaodingAnalysisException:
        at net.paoding.analysis.knife.PaodingMaker.implMake(PaodingMaker.java:43
2)
        at net.paoding.analysis.knife.PaodingMaker.make(PaodingMaker.java:115)
        at net.paoding.analysis.analyzer.PaodingAnalyzer.init(PaodingAnalyzer.ja
va:73)
        at net.paoding.analysis.analyzer.PaodingAnalyzer.<init>(PaodingAnalyzer.
java:59)
        at net.paoding.analysis.analyzer.estimate.TryPaodingAnalyzer.analysing(T
ryPaodingAnalyzer.java:99)
        at net.paoding.analysis.analyzer.estimate.TryPaodingAnalyzer.main(TryPao
dingAnalyzer.java:73)
Caused by: java.lang.StringIndexOutOfBoundsException: String index out of range:
-1
        at java.lang.String.substring(Unknown Source)
        at net.paoding.analysis.dictionary.support.filewords.SimpleReadListener2
.onWord(SimpleReadListener2.java:78)
        at net.paoding.analysis.dictionary.support.filewords.FileWordsReader.rea
dWords(FileWordsReader.java:120)
        at net.paoding.analysis.dictionary.support.filewords.FileWordsReader.rea
dWords(FileWordsReader.java:52)
        at net.paoding.analysis.analyzer.impl.CompiledFileDictionaries.getDictio
naryWords(CompiledFileDictionaries.java:287)
        at net.paoding.analysis.analyzer.impl.CompiledFileDictionaries.getVocabu
laryWords(CompiledFileDictionaries.java:298)
        at net.paoding.analysis.analyzer.impl.CompiledFileDictionaries.getVocabu
laryDictionary(CompiledFileDictionaries.java:183)
        at net.paoding.analysis.knife.CJKKnife.setDictionaries(CJKKnife.java:48)

        at net.paoding.analysis.knife.PaodingMaker.setDictionaries(PaodingMaker.
java:532)
        at net.paoding.analysis.knife.PaodingMaker.access$300(PaodingMaker.java:
54)
        at net.paoding.analysis.knife.PaodingMaker$1.run(PaodingMaker.java:406)
        at net.paoding.analysis.knife.PaodingMaker.implMake(PaodingMaker.java:39
1)
        ... 5 more
Global site tag (gtag.js) - Google Analytics