extract n-grams from patterns

Question

I tried to extract ngrams from patterns which is extracted from text documents, patterns have different number of terms.

for example : if the pattern p={t1,t2,t3}

and we need to extract ngram 3

it should be like this

t1
t2
t3

t1, t2
t2,t3

t1,t2,t3

I wrote some code but does not work well.

     public Hashtable<String, Pattern> findGrams(XMLDocument d) {
    ArrayList<Pattern> patterns = d.getPatterns();

    System.out.println("patterns " + d.getPatterns());

    ArrayList terms = new ArrayList();
    Hashtable Grams = new Hashtable();

    String s = "";

    // to extract all terms from the pattern
    for (int i = 0; i < patterns.size(); i++) {
        Pattern pat = (Pattern) patterns.get(i);
        terms.clear();
        for (int z = 0; z < pat.getNumitems(); z++) {
            terms.add(pat.getItems().get(z).toString());
        }

        // to generate grams from the pattern
        int j = 0;
        int x=0;
        for (int y = 1; y <= ngram ; y++) {

             for ( x = 0; x < terms.size() & j != y; x++) {
                    s = terms.get(x).toString();

                    if (y > 1) {
                        for (j = x + 1; j < terms.size() & terms.indexOf(j) < ngram; j++) {
                            s = s + "," + terms.get(j).toString();
                        }
                    }

                    if (!Grams.contains(s)) {
                        System.out.println(s);
                        Grams.put(s, i);
                    }
                }

        }
    }
    return (Grams);
}

any help please,

I found it hard to figure out what you're asking for here. Could you provide a concrete example with a given input and expected output? — Aleksander Blomskøld, Jul 12 '12 at 12:56
for example : if the pattern p={t1,t2,t3} and we need to extract ngram 3 it should be like this for ngram 1 : t1 then t2 then t3 for ngram 2: t1, t2 then t2,t3 for ngram 3:t1,t2,t3 — Mubarak, Jul 12 '12 at 13:02
Is this homework? It may be a duplicate of http://stackoverflow.com/questions/3656762/n-gram-generation-from-a-sentence? — radimpe, Jul 12 '12 at 13:03
There are a lot of fundamental problems with this code. I would suggest you take a look at @radimpe 's link to a previous SO question. You can modify your code based on the solution in that question. It shouldn't be hard to figure out. — Rob Wagner, Jul 12 '12 at 13:10
I suggest you have a look at the 'Related' links on the right hand side of the post. There are numerous answers related to generating N-grams available on SO already. — radimpe, Jul 12 '12 at 13:11

score 0 · Answer 1 · edited Sep 23 '12 at 10:12

i hope this will give you want you needed.

import java.util.*;

public class Test {

    public static List<String> ngrams(int n, String str) {
        List<String> ngrams = new ArrayList<String>();
        String[] words = str.split(" ");
        for (int i = 0; i < words.length - n + 1; i++)
            ngrams.add(concat(words, i, i+n));
        return ngrams;
    }

    public static String concat(String[] words, int start, int end) {
        StringBuilder sb = new StringBuilder();
        for (int i = start; i < end; i++)
            sb.append((i > start ? " " : "") + words[i]);
        return sb.toString();
    }

    public static void main(String[] args) {
        for (int n = 1; n <= 3; n++) {
           for (String ngram : ngrams(n, "t1 t2 t3"))
               System.out.println(ngram);
            System.out.println();
        }
    }
}

extract n-grams from patterns

1 Answers1