tribble.io
Class TokenAdapter

java.lang.Object
  tribble.io.TokenAdapter

All Implemented Interfaces:

TokenI, TraceableI

public class TokenAdapter
extends java.lang.Object
implements TokenI, TraceableI

Default generic lexical token implementation.

This is a simple implementation for any kind of lexical token object, which is constructed by some form of lexical analyzer (lexer) such as any that implements the LexerI interface. A token object represents a single lexical entity from an input source text stream. A lexer provides a stream of tokens read from such an input stream, which can then be used by a syntactical parser (such as any that implements the ParserI interface) for parsing sequences of source tokens into a symbol table or parse tree.

A token object has the following attributes:

text

The textual contents of the token, which is typically the characters it is composed of as it occurs in the source text stream.

type

A code used to indicate what kind of token it is. This is typically a particular value from a predefined set of token code constants.

line number

The source line number, corresponding to the physical text line that the token was read from.

column number

The source column number, corresponding to the physical text column that the token was read from. This can be either the column number at which the token starts or ends.

filename

The source filename, corresponding to the name of the physical file or stream that the token was read from.

Since:

2003-02-22

Version:

$Revision: 1.3 $ $Date: 2003/02/23 03:06:13 $

Author:

David R. Tribble, david@tribble.com.
Copyright ©2001-2003 by David R. Tribble, all rights reserved.
Permission is granted to freely use and distribute this source code provided that the original copyright and authorship notices remain intact.

Field Summary

protected int	m_colNo Source column number.
protected java.lang.String	m_fileName Source filename.
protected int	m_lineNo Source line number.
protected java.lang.String	m_text Token text.
protected int	m_tok Token code.
(package private) static java.lang.String	REV Revision information.

Constructor Summary

TokenAdapter()
Default constructor.

TokenAdapter(int type, java.lang.String text, int ln)
Constructor.

TokenAdapter(int type, java.lang.String text, int ln, int col)
Constructor.

TokenAdapter(int type, java.lang.String text, java.lang.String fname, int ln, int col)
Constructor.

TokenAdapter(java.lang.String text)
Constructor.

Method Summary

void	copyFrom(TokenAdapter tok) Copy the contents of another token into this token.
void	dump(java.io.PrintWriter out) Dump the contents of this token to a (debugging) output stream.
int	getColumnNumber() Retrieve the source column number of this token.
java.lang.String	getFileName() Retrieve the source filename of this token.
int	getLineNumber() Retrieve the source line number of this token.
java.lang.String	getText() Retrieve the textual contents of this token.
int	getType() Retrieve the type code of this token.
void	setColumnNumber(int col) Establish the source column number of this token.
void	setFileName(java.lang.String fname) Establish the source filename of this token.
void	setLineNumber(int ln) Establish the source line number of this token.
void	setText(java.lang.String text) Establish the textual contents of this token.
void	setType(int type) Establish the token type code of this token.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

REV

static final java.lang.String REV

Revision information.

See Also:

Constant Field Values

m_tok

protected int m_tok

Token code.

m_text

protected java.lang.String m_text

Token text.

m_lineNo

protected int m_lineNo

Source line number.

m_colNo

protected int m_colNo

Source column number.

m_fileName

protected java.lang.String m_fileName

Source filename.

Constructor Detail

TokenAdapter

public TokenAdapter()

Default constructor.

Since:

1.1, 2001-05-19

TokenAdapter

public TokenAdapter(java.lang.String text)

Constructor.

Parameters:

text - The string containing the textual contents for this token.

Since:

1.1, 2001-05-19

TokenAdapter

public TokenAdapter(int type,
                    java.lang.String text,
                    int ln)

Constructor.

Parameters:

type - The token type code for this token.

text - The string containing the textual contents for this token.

ln - The source line number for this token.

Since:

1.1, 2001-05-19

TokenAdapter

public TokenAdapter(int type,
                    java.lang.String text,
                    int ln,
                    int col)

Constructor.

Parameters:

type - The token type code for this token.

text - The string containing the textual contents for this token.

ln - The source line number for this token.

col - The source column number for this token.

Since:

1.3, 2003-02-22

TokenAdapter

public TokenAdapter(int type,
                    java.lang.String text,
                    java.lang.String fname,
                    int ln,
                    int col)

Constructor.

Parameters:

type - The token type code for this token.

text - The string containing the textual contents for this token.

fname - The source filename for this token.

ln - The source line number for this token.

col - The source column number for this token.

Since:

1.3, 2003-02-22

Method Detail

setText

public void setText(java.lang.String text)

Establish the textual contents of this token.

Specified by:

setText in interface TokenI

Parameters:

text - The string containing the textual contents for this token.

Since:

1.1, 2001-05-19

getText

public java.lang.String getText()

Retrieve the textual contents of this token.

Specified by:

getText in interface TokenI

Returns:

A string containing the textual contents for this token.

Since:

1.1, 2001-05-19

setType

public void setType(int type)

Establish the token type code of this token.

Specified by:

setType in interface TokenI

Parameters:

type - The token type code for this token.

Since:

1.1, 2001-05-19

getType

public int getType()

Retrieve the type code of this token.

Specified by:

getType in interface TokenI

Returns:

The type code for this token.

Since:

1.1, 2001-05-19

setLineNumber

public void setLineNumber(int ln)

Establish the source line number of this token.

Specified by:

setLineNumber in interface TokenI

Parameters:

ln - The source line number for this token. If this number is negative, the line number setting of this token is not modified.

Since:

1.3, 2003-02-22

getLineNumber

public int getLineNumber()

Retrieve the source line number of this token.

Specified by:

getLineNumber in interface TokenI

Returns:

The source line number for this token.

Since:

1.3, 2003-02-22

setColumnNumber

public void setColumnNumber(int col)

Establish the source column number of this token.

Specified by:

setColumnNumber in interface TokenI

Parameters:

col - The source column number for this token. If this number is negative, the column number setting of this token is not modified.

Since:

1.3, 2003-02-22

getColumnNumber

public int getColumnNumber()

Retrieve the source column number of this token.

Specified by:

getColumnNumber in interface TokenI

Returns:

The source column number for this token.

Since:

1.3, 2003-02-22

setFileName

public void setFileName(java.lang.String fname)

Establish the source filename of this token.

Specified by:

setFileName in interface TokenI

Parameters:

fname - The source filename for this token.

Since:

1.1, 2001-05-19

getFileName

public java.lang.String getFileName()

Retrieve the source filename of this token.

Specified by:

getFileName in interface TokenI

Returns:

The source filename for this token.

Since:

1.1, 2001-05-19

dump

public void dump(java.io.PrintWriter out)

Dump the contents of this token to a (debugging) output stream.

Specified by:

dump in interface TraceableI

Parameters:

out - A (debugging) output stream.

Since:

1.1, 2001-05-19

copyFrom

public void copyFrom(TokenAdapter tok)

Copy the contents of another token into this token.

Parameters:

tok - Another token to copy from.

Since:

1.1, 2001-07-20