Evaluate the Lexical Analysis Done by Your Chosen Compiler

Study the way lexical analysis is done and evaluate it with respect to the issues discussed in class. Following are some examples of issues that would be interesting in evaluating the analysis. You should add any others that you discover or that you think are relevant.

• Does the analyzer use some sort of finite state automata, regular expression, etc. specification? Is there any formal specification of the tokens in the documentation or language definition?
• Are there non-finite state aspects of the lexical analysis? If so, what are they? Are they required to handle this language or are they used for the convenience of the compiler writer (e.g. for efficiency)? Explain.
• Is there a clean interface between scanner and parser? Between scanner and semantics? If so, how can you tell? If not, why not?
• Is there a sequential interface between the scanner and other phases (i.e. could the scanner be written with minimal change to cause all scanning to be completed and the token stream stored before beginning any other phase)
• What would be involved in changing the lexical definition of the language? How hard would it be? (Examples might be putting keywords in other natural languages besides English, or changing operator symbols).