利用文本挖掘技术来找出网络中的“小鲜词”

NewWordDiscoverTest.java

package grid.test;

import grid.common.TextDatReader;
import grid.text.evolution.NewWordDiscover;

import java.io.IOException;
import java.util.Set;


public class NewWordDiscoverTest {

    private final static String path = "text.dat";

    public static void main(String args[]) throws IOException {
        // Replace your document here
        String document = TextDatReader.read(path);

        NewWordDiscover discover = new NewWordDiscover();
        long start = System.currentTimeMillis();
        Set<String> words = discover.discover(document);
        System.out.println("Speed: " + (double) document.length()
                / (System.currentTimeMillis() - start) * 1000);
        System.out.println("New words size: " + words.size());
        System.out.println("New word is: "+"n");
        for (String str : words) {  
            System.out.println(str+"n");
        }  

    }
}

抽词测试，结果如下

ParticipleTest.java

package grid.test;

import grid.text.participle.MechanicalParticiple;

import java.util.Vector;


public class ParticipleTest {

    private static String document = "我是中国人";

    public static void main(String args[]) {
        MechanicalParticiple participle = new MechanicalParticiple();
        Vector<String> vec = participle.partition(document);
        System.out.println(vec);
    }
}

分词测试，结果如下

怎么样，很酷吧，你还可以试着用《天龙八部》数据集玩下，看看主角是不是乔帮主。如果发现了什么新鲜词，请告诉博主，咱也不落后哈！

（编辑：核心网）

【声明】本站内容均来自网络，其相关言论仅代表作者个人观点，不代表本站立场。若无意侵犯到您的权利，请及时与联系站长删除相关内容!