Package totalcross.util.regex

Class RETokenizer

java.lang.Object

totalcross.util.regex.RETokenizer

public class RETokenizer
extends java.lang.Object

The Tokenizer class suggests a methods to break a text into tokens using occurences of a pattern as delimiters. There are two ways to obtain a text tokenizer for some pattern:

 Pattern p=new Pattern("\\s+"); //any number of space characters
 String text="blah blah blah";
 //by factory method
 RETokenizer tok1=p.tokenizer(text);
 //or by constructor
 RETokenizer tok2=new RETokenizer(p,text);
 

Now the one way is to use the tokenizer as a token enumeration/iterator:

 while(tok1.hasMore()) System.out.println(tok1.nextToken());
 

and another way is to split it into a String array:

 
 String[] arr=tok2.split();
 for(int i=0;i

See Also:

Pattern.tokenizer(java.lang.String)

Constructor Summary

Constructors

Constructor	Description
RETokenizer(Matcher m, boolean emptyEnabled)
RETokenizer(Pattern pattern, char[] chars, int off, int len)
RETokenizer(Pattern pattern, java.lang.String text)
RETokenizer(Pattern pattern, CharStream r, int len)

Method Summary

Modifier and Type	Method	Description
boolean	hasMore()
boolean	hasMoreElements()
boolean	isEmptyEnabled()
java.lang.Object	nextElement()
java.lang.String	nextToken()
void	reset()
void	setEmptyEnabled(boolean b)
java.lang.String[]	split()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

RETokenizer

public RETokenizer(Pattern pattern,
                   java.lang.String text)

RETokenizer

public RETokenizer(Pattern pattern,
                   char[] chars,
                   int off,
                   int len)

RETokenizer

public RETokenizer(Pattern pattern,
                   CharStream r,
                   int len)
            throws IOException

Throws:

IOException

RETokenizer

public RETokenizer(Matcher m,
                   boolean emptyEnabled)

Method Detail

setEmptyEnabled

public void setEmptyEnabled(boolean b)

isEmptyEnabled

public boolean isEmptyEnabled()

hasMore

public boolean hasMore()

nextToken

public java.lang.String nextToken()
                           throws ElementNotFoundException

Throws:

ElementNotFoundException

split

public java.lang.String[] split()
                         throws ElementNotFoundException

Throws:

ElementNotFoundException

reset

public void reset()

hasMoreElements

public boolean hasMoreElements()

nextElement

public java.lang.Object nextElement()
                             throws ElementNotFoundException

Returns:

a next token as a String

Throws:

ElementNotFoundException