CSE340 Project 1: Lexical Analysis

Due: Sunday, February 12th, 2023 by 11:59 pm MST The goal of this project is to give you hands-on experience with lexical analysis. You will extend the provided lexical analyzer to support more token types. The next section lists all new token types that you need to implement.

1. Token Types

Modify the lexer to support the following 3 token types:

REALNUM   = NUM DOT digit digit*
BASE08NUM = ((pdigit8 digit8*) + 0) (x) (08) BASE16NUM = ((pdigit16 digit16*) + 0) (x) (16)

Where

digit16  = 0 + 1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 + A + B + C + D + E + F pdigit16 = 1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 + A + B + C + D + E + F digit    = 0 + 1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9
pdigit   = 1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 digit8   = 0 + 1 + 2 + 3 + 4 + 5 + 6 + 7 pdigit8  = 1 + 2 + 3 + 4 + 5 + 6 + 7

Note that NUM and DOT are already defined in the lexer, but here are the regular expressions for the sake of completeness:

NUM = (pdigit digit*) + 0
DOT = '.'

...

2. How-To

Follow these steps:

Download the lexer.cc , lexer.h , inputbuf.cc and inputbuf.h files accompanying this project description. Note that these files might be a little different than the code you've seen in class or elsewhere.

Add your code to the files to support the token types listed in the previous section.

Compile your code using GCC compiler in Ubuntu 19.04 or higher . You will need to use the g++ command to compile your code in a terminal window.

Note that you are required to compile and test your code in Ubuntu 19.04 or higher using the GCC compilers. You are free to use any IDE or text editor on any platform, however, using tools available in Ubuntu 19.04 g++ version 4.9 (or tools that you could install on Ububntu 19.04 could save time in the development/compile/test cycle. See next section for more details on how to compile using GCC.

Test your code to see if it passes the provided test cases. You will need to extract the test cases from the zip file and run the test script test1.sh . More details on this in the next section.

Submit your code in canvas before the deadline:

For this project you need to update lexer.cc and lexer.h. The updates that you need to do in lexer.h are minimal and are already implemented in the submission website. So you do not need to upload the lexer.h file in CANVAS. But you still do need to update lexer.h if you want to compile locally.