diff options
| author | Johnny Richard <johnny@johnnyrichard.com> | 2023-04-18 18:33:36 +0200 |
|---|---|---|
| committer | Carlos Maniero <carlosmaniero@gmail.com> | 2023-04-18 13:40:37 -0300 |
| commit | 8b9899bcf338d4aca95419dfd1006f7f822a16c9 (patch) | |
| tree | fec4c33cbd91f8be121c6f0bbeb36a2dc249814b /src/lexer.h | |
| parent | 3433ac8bd34f302cec30f7d2faeb00a9c77e5ef8 (diff) | |
lexer: Add tokenizer for OP and UNKNOWN tokens
We want to tokenizer arithmetic expressions.
We are handling exceptional cases with UNKNOWN token.
Co-authored-by: Carlos Maniero <carlosmaniero@gmail.com>
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
Diffstat (limited to 'src/lexer.h')
| -rw-r--r-- | src/lexer.h | 4 |
1 files changed, 3 insertions, 1 deletions
diff --git a/src/lexer.h b/src/lexer.h index a091fb7..ae8ab60 100644 --- a/src/lexer.h +++ b/src/lexer.h @@ -31,7 +31,9 @@ typedef enum { TOKEN_OCURLY, TOKEN_CCURLY, TOKEN_NUMBER, - TOKEN_EOF + TOKEN_OP, + TOKEN_EOF, + TOKEN_UNKNOWN } token_kind_t; typedef struct token_t { |