tribble.io
Class TokenAdapter

java.lang.Object
  extended by tribble.io.TokenAdapter
All Implemented Interfaces:
TokenI, TraceableI

public class TokenAdapter
extends java.lang.Object
implements TokenI, TraceableI

Default generic lexical token implementation.

This is a simple implementation for any kind of lexical token object, which is constructed by some form of lexical analyzer (lexer) such as any that implements the LexerI interface. A token object represents a single lexical entity from an input source text stream. A lexer provides a stream of tokens read from such an input stream, which can then be used by a syntactical parser (such as any that implements the ParserI interface) for parsing sequences of source tokens into a symbol table or parse tree.

A token object has the following attributes:

text
The textual contents of the token, which is typically the characters it is composed of as it occurs in the source text stream.

type
A code used to indicate what kind of token it is. This is typically a particular value from a predefined set of token code constants.

line number
The source line number, corresponding to the physical text line that the token was read from.

column number
The source column number, corresponding to the physical text column that the token was read from. This can be either the column number at which the token starts or ends.

filename
The source filename, corresponding to the name of the physical file or stream that the token was read from.

Since:
2003-02-22
Version:
$Revision: 1.3 $ $Date: 2003/02/23 03:06:13 $
Author:
David R. Tribble, david@tribble.com.
Copyright 2001-2003 by David R. Tribble, all rights reserved.
Permission is granted to freely use and distribute this source code provided that the original copyright and authorship notices remain intact.

Field Summary
protected  int m_colNo
          Source column number.
protected  java.lang.String m_fileName
          Source filename.
protected  int m_lineNo
          Source line number.
protected  java.lang.String m_text
          Token text.
protected  int m_tok
          Token code.
(package private) static java.lang.String REV
          Revision information.
 
Constructor Summary
TokenAdapter()
          Default constructor.
TokenAdapter(int type, java.lang.String text, int ln)
          Constructor.
TokenAdapter(int type, java.lang.String text, int ln, int col)
          Constructor.
TokenAdapter(int type, java.lang.String text, java.lang.String fname, int ln, int col)
          Constructor.
TokenAdapter(java.lang.String text)
          Constructor.
 
Method Summary
 void copyFrom(TokenAdapter tok)
          Copy the contents of another token into this token.
 void dump(java.io.PrintWriter out)
          Dump the contents of this token to a (debugging) output stream.
 int getColumnNumber()
          Retrieve the source column number of this token.
 java.lang.String getFileName()
          Retrieve the source filename of this token.
 int getLineNumber()
          Retrieve the source line number of this token.
 java.lang.String getText()
          Retrieve the textual contents of this token.
 int getType()
          Retrieve the type code of this token.
 void setColumnNumber(int col)
          Establish the source column number of this token.
 void setFileName(java.lang.String fname)
          Establish the source filename of this token.
 void setLineNumber(int ln)
          Establish the source line number of this token.
 void setText(java.lang.String text)
          Establish the textual contents of this token.
 void setType(int type)
          Establish the token type code of this token.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

REV

static final java.lang.String REV
Revision information.

See Also:
Constant Field Values

m_tok

protected int m_tok
Token code.


m_text

protected java.lang.String m_text
Token text.


m_lineNo

protected int m_lineNo
Source line number.


m_colNo

protected int m_colNo
Source column number.


m_fileName

protected java.lang.String m_fileName
Source filename.

Constructor Detail

TokenAdapter

public TokenAdapter()
Default constructor.

Since:
1.1, 2001-05-19

TokenAdapter

public TokenAdapter(java.lang.String text)
Constructor.

Parameters:
text - The string containing the textual contents for this token.
Since:
1.1, 2001-05-19

TokenAdapter

public TokenAdapter(int type,
                    java.lang.String text,
                    int ln)
Constructor.

Parameters:
type - The token type code for this token.
text - The string containing the textual contents for this token.
ln - The source line number for this token.
Since:
1.1, 2001-05-19

TokenAdapter

public TokenAdapter(int type,
                    java.lang.String text,
                    int ln,
                    int col)
Constructor.

Parameters:
type - The token type code for this token.
text - The string containing the textual contents for this token.
ln - The source line number for this token.
col - The source column number for this token.
Since:
1.3, 2003-02-22

TokenAdapter

public TokenAdapter(int type,
                    java.lang.String text,
                    java.lang.String fname,
                    int ln,
                    int col)
Constructor.

Parameters:
type - The token type code for this token.
text - The string containing the textual contents for this token.
fname - The source filename for this token.
ln - The source line number for this token.
col - The source column number for this token.
Since:
1.3, 2003-02-22
Method Detail

setText

public void setText(java.lang.String text)
Establish the textual contents of this token.

Specified by:
setText in interface TokenI
Parameters:
text - The string containing the textual contents for this token.
Since:
1.1, 2001-05-19

getText

public java.lang.String getText()
Retrieve the textual contents of this token.

Specified by:
getText in interface TokenI
Returns:
A string containing the textual contents for this token.
Since:
1.1, 2001-05-19

setType

public void setType(int type)
Establish the token type code of this token.

Specified by:
setType in interface TokenI
Parameters:
type - The token type code for this token.
Since:
1.1, 2001-05-19

getType

public int getType()
Retrieve the type code of this token.

Specified by:
getType in interface TokenI
Returns:
The type code for this token.
Since:
1.1, 2001-05-19

setLineNumber

public void setLineNumber(int ln)
Establish the source line number of this token.

Specified by:
setLineNumber in interface TokenI
Parameters:
ln - The source line number for this token. If this number is negative, the line number setting of this token is not modified.
Since:
1.3, 2003-02-22

getLineNumber

public int getLineNumber()
Retrieve the source line number of this token.

Specified by:
getLineNumber in interface TokenI
Returns:
The source line number for this token.
Since:
1.3, 2003-02-22

setColumnNumber

public void setColumnNumber(int col)
Establish the source column number of this token.

Specified by:
setColumnNumber in interface TokenI
Parameters:
col - The source column number for this token. If this number is negative, the column number setting of this token is not modified.
Since:
1.3, 2003-02-22

getColumnNumber

public int getColumnNumber()
Retrieve the source column number of this token.

Specified by:
getColumnNumber in interface TokenI
Returns:
The source column number for this token.
Since:
1.3, 2003-02-22

setFileName

public void setFileName(java.lang.String fname)
Establish the source filename of this token.

Specified by:
setFileName in interface TokenI
Parameters:
fname - The source filename for this token.
Since:
1.1, 2001-05-19

getFileName

public java.lang.String getFileName()
Retrieve the source filename of this token.

Specified by:
getFileName in interface TokenI
Returns:
The source filename for this token.
Since:
1.1, 2001-05-19

dump

public void dump(java.io.PrintWriter out)
Dump the contents of this token to a (debugging) output stream.

Specified by:
dump in interface TraceableI
Parameters:
out - A (debugging) output stream.
Since:
1.1, 2001-05-19

copyFrom

public void copyFrom(TokenAdapter tok)
Copy the contents of another token into this token.

Parameters:
tok - Another token to copy from.
Since:
1.1, 2001-07-20