summaryrefslogtreecommitdiff
path: root/test/lexer_test.c
AgeCommit message (Collapse)Author
2023-05-04parser: Introduce statement keywordsCarlos Maniero
This commit introduces a few changes in pipalang syntax. Now, both functions and variables requires keywords to be defined. before: main(): i32 { a: i32 = 2; return a; } now: fn main(): i32 { let a: i32 = 2; return a; } Signed-off-by: Carlos Maniero <carlos@maniero.me> Reviewed-by: Johnny Richard <johnny@johnnyrichard.com>
2023-05-01lexer: Peek next tokenJohnny Richard
The only way to get the next token was by consuming it. So then, our parser starts to become hard to understand, once sometimes we just want to take a look on the next token to understand what should be the next kind of expression. This commit introduces a new function that will help us to improve our parser implementation. Signed-off-by: Johnny Richard <johnny@johnnyrichard.com> Reviewed-by: Carlos Maniero <carlos@maniero.me>
2023-04-26lexer: Split operation tokens into their own tokenCarlos Maniero
The +, -, *, and / tokens used to be TOKEN_OP, but the TOKEN_OP has been removed and a token for each operation has been introduced. Python's token names were followed: https://docs.python.org/3/library/token.html Signed-off-by: Carlos Maniero <carlosmaniero@gmail.com> Reviewed-by: Johnny Richard <johnny@johnnyrichar.com>
2023-04-25style: Use clang-format as formatter and linter toolJohnny Richard
We want to keep the code style consistent, this first commit adds a .clang-format in order to "document" our style code. This patch also adds a target *linter* to Makefile which will complain if we have any style issue on test and src dirs. I have run the follow command to create the .clang-format file: $ clang-format -style=mozilla -dump-config > .clang-format And I also made some adjusts to .clang-format changing the following properties: PointerAlignment: Right ColumnLimit: 120 Commands executed to fix the current styling: $ find . -name *.h | xargs clang-format -i $ find . -name *.c | xargs clang-format -i Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
2023-04-25parser: Add support for variables and identifiers in function bodyCarlos Maniero
This commit adds support for variables and identifiers in the function body of the parser, stored as a vector. However, at this point, identifier resolution is not fully implemented, and we currently accept identifiers without checking if they can be resolved. This is a known limitation that will be addressed in a future commit once hash-tables are added to the parser. Signed-off-by: Carlos Maniero <carlosmaniero@gmail.com> Reviewed-by: Johnny Richard <johnny@johnnyrichard.com>
2023-04-18lexer: Include a test function assert_token_atCarlos Maniero
With this function now it is possible to assert a token given an index. Signed-off-by: Carlos Maniero <carlosmaniero@gmail.com> Reviewed-by: Johnny Richard <johnny@johnnyrichard.com> Link: https://lists.sr.ht/~johnnyrichard/pipalang-devel/%3C20230418170136.3949-1-carlosmaniero%40gmail.com%3E
2023-04-18lexer: Add tokenizer for OP and UNKNOWN tokensJohnny Richard
We want to tokenizer arithmetic expressions. We are handling exceptional cases with UNKNOWN token. Co-authored-by: Carlos Maniero <carlosmaniero@gmail.com> Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
2023-04-18tests: Include lexer's number tokenizer testsCarlos Maniero
Signed-off-by: Carlos Maniero <carlosmaniero@gmail.com> Reviewed-by: Johnny Richard <johnny@johnnyrichard.com>