summaryrefslogtreecommitdiff
path: root/src/lexer.h
AgeCommit message (Collapse)Author
2023-05-09parser: parses an if statement no code generationCarlos Maniero
This commit parses a if statement following the grammar bellow: if boolean_expression { n_epressions; } No else neither code generation was implemented. Signed-off-by: Carlos Maniero <carlos@maniero.me>
2023-05-09parser: Add the bool typeCarlos Maniero
This commit introduces a new type for booleans. There is no code generation for this type yet. The intention of this commit is to enable flow control in the near future. Signed-off-by: Carlos Maniero <carlos@maniero.me> Reviewed-by: Johnny Richard <johnny@johnnyrichard.com>
2023-05-06lexer: Tokenize logical and bitwise operatorsCarlos Maniero
The followed logic operators were added to lexer: TOKEN_EQUAL == TOKEN_NOT ! TOKEN_NOT_EQUAL != TOKEN_GT > TOKEN_GT_EQUAL >= TOKEN_LT < TOKEN_LT_EQUAL <= TOKEN_AND && TOKEN_OR || Bitwise operators were also added TOKEN_BITWISE_AND & TOKEN_BITWISE_OR | TOKEN_BITWISE_SHIFT_LEFT << TOKEN_BITWISE_SHIFT_RIGHT >> TOKEN_BITWISE_XOR ^ TOKEN_BITWISE_NOT ~ TOKEN_EQUAL '=' was renamed TOKEN_ASSIGN, and now TOKEN_EQUAL is used for the logical comparator '=='. Signed-off-by: Carlos Maniero <carlos@maniero.me>
2023-05-04parser: Introduce statement keywordsCarlos Maniero
This commit introduces a few changes in pipalang syntax. Now, both functions and variables requires keywords to be defined. before: main(): i32 { a: i32 = 2; return a; } now: fn main(): i32 { let a: i32 = 2; return a; } Signed-off-by: Carlos Maniero <carlos@maniero.me> Reviewed-by: Johnny Richard <johnny@johnnyrichard.com>
2023-05-03parser: Use lookahead instead of consuming tokensCarlos Maniero
Previously, during block declaration, the parser consumed the token which caused some parsers (such as return and variable declaration) to not be self-contained and to depend on the callee to start the parser. In this commit, I've refactored the parser to only look for future tokens using lookahead, and delegate the consumption to child parser functions. This results in a more modular and self-contained parser that improves the overall maintainability and readability of the code. Signed-off-by: Carlos Maniero <carlos@maniero.me>
2023-05-03parser: Refactor return statement to return an ast_nodeCarlos Maniero
During the refactoring process, I identified a memory leak where the return argument was allocated but not freed in case of an error. It also introduces the concept of keyword tokens. Where return is now a keyword simplifying the parser. Signed-off-by: Carlos Maniero <carlos@maniero.me>
2023-05-01parser: Use peek and drop token when parsing expressionsJohnny Richard
2023-05-01lexer: Peek next tokenJohnny Richard
The only way to get the next token was by consuming it. So then, our parser starts to become hard to understand, once sometimes we just want to take a look on the next token to understand what should be the next kind of expression. This commit introduces a new function that will help us to improve our parser implementation. Signed-off-by: Johnny Richard <johnny@johnnyrichard.com> Reviewed-by: Carlos Maniero <carlos@maniero.me>
2023-04-26lexer: Split operation tokens into their own tokenCarlos Maniero
The +, -, *, and / tokens used to be TOKEN_OP, but the TOKEN_OP has been removed and a token for each operation has been introduced. Python's token names were followed: https://docs.python.org/3/library/token.html Signed-off-by: Carlos Maniero <carlosmaniero@gmail.com> Reviewed-by: Johnny Richard <johnny@johnnyrichar.com>
2023-04-25style: Use clang-format as formatter and linter toolJohnny Richard
We want to keep the code style consistent, this first commit adds a .clang-format in order to "document" our style code. This patch also adds a target *linter* to Makefile which will complain if we have any style issue on test and src dirs. I have run the follow command to create the .clang-format file: $ clang-format -style=mozilla -dump-config > .clang-format And I also made some adjusts to .clang-format changing the following properties: PointerAlignment: Right ColumnLimit: 120 Commands executed to fix the current styling: $ find . -name *.h | xargs clang-format -i $ find . -name *.c | xargs clang-format -i Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
2023-04-21parser: Parse integers arithmetic expressionJohnny Richard
This patch implements the AST creation for arithmetic expressions. NOTE: The implementation works only for integer numbers. Signed-off-by: Johnny Richard <johnny@johnnyrichard.com> Reviewed-by: Carlos Maniero <carlosmaniero@gmail.com>
2023-04-18lexer: Add tokenizer for OP and UNKNOWN tokensJohnny Richard
We want to tokenizer arithmetic expressions. We are handling exceptional cases with UNKNOWN token. Co-authored-by: Carlos Maniero <carlosmaniero@gmail.com> Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
2023-04-16Start using string_view on lexer and parserJohnny Richard
This change fixes the memory leak when token got created. Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
2023-04-15parser: Show filepath row and col when parsing failsJohnny Richard
In order to find out where a parsing error occurred, this patch introduces the exactly location following the format 'file:row:col'. Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
2023-04-15parser: Create parser for function with return statementsJohnny Richard
This is a very limited parser implementation which parses a single function with return type i32 and body containing a return number statement. The parser doesn't show the 'filepath:row:col' when it fails, a future improvement would be display it to easy find where the compilation problem is located. The ast_nodes are taking the token.value ownership (which is a really bad design since not all token.value ownership has been taken causing memory leaking) but we never free them. For a future fix we could use a string_view instead since we never change the original source code. The string_view will also improve the performance a lot avoiding unnecessary heap memory allocation. Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
2023-04-14lexer: Extract lexer.c and lexer.h from pipa.cJohnny Richard
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>