summaryrefslogtreecommitdiff
path: root/src/lexer.c
AgeCommit message (Collapse)Author
2023-04-20parser: Stop exiting on parser errorCarlos Maniero
Previously, when an error occurred during parsing, the application would exit, making it difficult to test the parser and limiting the compiler's extensibility. This commit improves the parser's error handling by allowing for continued execution after an error, enabling easier testing and increased flexibility. The parser is prepared to handle multiples errors, although the current implementation always returns a single error, it may be useful given multiples functions where we can show errors by context. Signed-off-by: Carlos Maniero <carlosmaniero@gmail.com> Reviwed-by: Johnny Richard <johnny@johnnyrichard.com>
2023-04-18style: Fix identation on lexer.cCarlos Maniero
Co-authored-by: Johnny Richard <johnny@johnnyrichard.com> Signed-off-by: Carlos Maniero <carlosmaniero@gmail.com> Link: https://lists.sr.ht/~johnnyrichard/pipalang-devel/%3C20230418165847.3798-1-carlosmaniero%40gmail.com%3E
2023-04-18lexer: Add tokenizer for OP and UNKNOWN tokensJohnny Richard
We want to tokenizer arithmetic expressions. We are handling exceptional cases with UNKNOWN token. Co-authored-by: Carlos Maniero <carlosmaniero@gmail.com> Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
2023-04-18lexer: Extract tokenization functionsCarlos Maniero
make the next token function small by extracting the functions that make tokens. Signed-off-by: Carlos Maniero <carlosmaniero@gmail.com> Reviewed-by: Johnny Richard <johnny@johnnyrichard.com>
2023-04-18lexer: extract the lexer_drop_spacesCarlos Maniero
Extracted logic for skipping empty characters into a separate function. No change in lexer behavior. Signed-off-by: Carlos Maniero <carlosmaniero@gmail.com> Reviewed-by: Johnny Richard <johnny@johnnyrichard.com>
2023-04-16lexer: Extract lexer_define_literal_token_props functionJohnny Richard
This is an attempt of reducing code duplication. Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
2023-04-16Start using string_view on lexer and parserJohnny Richard
This change fixes the memory leak when token got created. Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
2023-04-15parser: Show filepath row and col when parsing failsJohnny Richard
In order to find out where a parsing error occurred, this patch introduces the exactly location following the format 'file:row:col'. Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
2023-04-15parser: Create parser for function with return statementsJohnny Richard
This is a very limited parser implementation which parses a single function with return type i32 and body containing a return number statement. The parser doesn't show the 'filepath:row:col' when it fails, a future improvement would be display it to easy find where the compilation problem is located. The ast_nodes are taking the token.value ownership (which is a really bad design since not all token.value ownership has been taken causing memory leaking) but we never free them. For a future fix we could use a string_view instead since we never change the original source code. The string_view will also improve the performance a lot avoiding unnecessary heap memory allocation. Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
2023-04-15build: Enable warning and debug CFLAGSJohnny Richard
After enabling the warning flags, the compiler was firing the following warnings: warning: implicit declaration of function ‘strdup’; did you mean ‘strcmp’? [-Wimplicit-function-declaration] token->value = strdup("("); ^~~~~~ strcmp warning: assignment to ‘char *’ from ‘int’ makes pointer from integer without a cast [-Wint-conversion] token->value = strdup("("); ^ In order to fix these warnings above, I have decided to replace *strdup* and *strndup* by *strcpy* and *strncpy* functions. Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
2023-04-14lexer: Extract lexer.c and lexer.h from pipa.cJohnny Richard
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>