tribble.io
Interface LexerI

All Known Implementing Classes:

LexerAdapter

public interface LexerI

Generic lexical analyzer (lexer) interface.

This interface is used to implement any kind of lexical analyzer (lexer), which reads characters from an input source stream (implementing the java.io.Reader class) and converts it into a stream of tokens (which implement the TokenI interface). Such a lexer stream can be used to read the underlying source text for a parser, such as those implementing the ParserI interface.

Since:

2001-04-16

Version:

$Revision: 1.4 $ $Date: 2003/02/26 04:40:02 $

Author:

David R. Tribble, david@tribble.com.
Copyright ©2001-2003 by David R. Tribble all rights reserved.
Permission is granted to freely use and distribute this source code provided that the original copyright and authorship notices remain intact.

See Also:

CharInputStreamI, ParserI, TokenI

Field Summary

static java.lang.String

REV Revision information.

Method Summary

void	close() Close the input stream.
TokenI	getToken() Read the next token from the input stream.
boolean	getToken(TokenI tok) Read the next token from the lexer input stream.
void	setInput(java.io.Reader in) Establish the input stream from which to read tokens.
void	setSourceName(java.lang.String fname) Establish the input source filename.

Field Detail

REV

static final java.lang.String REV

Revision information.

See Also:

Constant Field Values

Method Detail

setInput

void setInput(java.io.Reader in)

Establish the input stream from which to read tokens.

Parameters:

in - A character input stream from which to read tokens.

Since:

1.4, 2003-02-25

setSourceName

void setSourceName(java.lang.String fname)

Establish the input source filename.

Parameters:

fname - The source filename.

Since:

1.1, 2001-05-31

close

void close()
           throws java.io.IOException

Close the input stream.

Throws:

java.io.IOException

Since:

1.1, 2001-04-16

getToken

TokenI getToken()
                throws java.io.IOException

Read the next token from the input stream.

Either this method or getToken(TokenI) may be used interchangeably.

Returns:

The next token read from the input stream, or null if there are no more tokens (i.e., the end of file was reached).

Throws:

java.io.IOException - Thrown if an I/O error or parsing error occurs.

Since:

1.1, 2001-04-16

getToken

boolean getToken(TokenI tok)
                 throws java.io.IOException

Read the next token from the lexer input stream.

Either this method or getToken() may be used interchangeably. This method might be more efficient, since a new token object does not need to be allocated on each call.

Parameters:

tok - The token object which is filled in with the next token read from the input stream.

Returns:

True if a token was successfully read from the input stream, otherwise false.

Throws:

java.io.IOException - Thrown if an I/O error or parsing error occurs.

Since:

1.1, 2001-04-16