tribble.io
Interface LexerI

All Known Implementing Classes:
LexerAdapter

public interface LexerI

Generic lexical analyzer (lexer) interface.

This interface is used to implement any kind of lexical analyzer (lexer), which reads characters from an input source stream (implementing the java.io.Reader class) and converts it into a stream of tokens (which implement the TokenI interface). Such a lexer stream can be used to read the underlying source text for a parser, such as those implementing the ParserI interface.

Since:
2001-04-16
Version:
$Revision: 1.4 $ $Date: 2003/02/26 04:40:02 $
Author:
David R. Tribble, david@tribble.com.
Copyright 2001-2003 by David R. Tribble all rights reserved.
Permission is granted to freely use and distribute this source code provided that the original copyright and authorship notices remain intact.
See Also:
CharInputStreamI, ParserI, TokenI

Field Summary
static java.lang.String REV
          Revision information.
 
Method Summary
 void close()
          Close the input stream.
 TokenI getToken()
          Read the next token from the input stream.
 boolean getToken(TokenI tok)
          Read the next token from the lexer input stream.
 void setInput(java.io.Reader in)
          Establish the input stream from which to read tokens.
 void setSourceName(java.lang.String fname)
          Establish the input source filename.
 

Field Detail

REV

static final java.lang.String REV
Revision information.

See Also:
Constant Field Values
Method Detail

setInput

void setInput(java.io.Reader in)
Establish the input stream from which to read tokens.

Parameters:
in - A character input stream from which to read tokens.
Since:
1.4, 2003-02-25

setSourceName

void setSourceName(java.lang.String fname)
Establish the input source filename.

Parameters:
fname - The source filename.
Since:
1.1, 2001-05-31

close

void close()
           throws java.io.IOException
Close the input stream.

Throws:
java.io.IOException
Since:
1.1, 2001-04-16

getToken

TokenI getToken()
                throws java.io.IOException
Read the next token from the input stream.

Either this method or getToken(TokenI) may be used interchangeably.

Returns:
The next token read from the input stream, or null if there are no more tokens (i.e., the end of file was reached).
Throws:
java.io.IOException - Thrown if an I/O error or parsing error occurs.
Since:
1.1, 2001-04-16

getToken

boolean getToken(TokenI tok)
                 throws java.io.IOException
Read the next token from the lexer input stream.

Either this method or getToken() may be used interchangeably. This method might be more efficient, since a new token object does not need to be allocated on each call.

Parameters:
tok - The token object which is filled in with the next token read from the input stream.
Returns:
True if a token was successfully read from the input stream, otherwise false.
Throws:
java.io.IOException - Thrown if an I/O error or parsing error occurs.
Since:
1.1, 2001-04-16