lexer: Split operation tokens into their own token

The +, -, *, and / tokens used to be TOKEN_OP, but the TOKEN_OP has been
removed and a token for each operation has been introduced. Python's
token names were followed: https://docs.python.org/3/library/token.html

Signed-off-by: Carlos Maniero <carlosmaniero@gmail.com>
Reviewed-by: Johnny Richard <johnny@johnnyrichar.com>

1 files changed, 5 insertions, 1 deletions

diff --git a/src/lexer.h b/src/lexer.h
index 8c84745..d4e84e1 100644
--- a/src/lexer.h
+++ b/src/lexer.h
@@ -32,7 +32,11 @@ typedef enum
   TOKEN_OCURLY,
   TOKEN_CCURLY,
   TOKEN_NUMBER,
-  TOKEN_OP,
+  TOKEN_PLUS,
+  TOKEN_MINUS,
+  TOKEN_STAR,
+  TOKEN_SLASH,
+  TOKEN_EQUAL,
   TOKEN_EOF,
   TOKEN_UNKNOWN
 } token_kind_t;