Age | Commit message (Collapse) | Author |
|
Signed-off-by: Carlos Maniero <carlosmaniero@gmail.com>
Reviewed-by: Johnny Richard <johnny@johnnyrichard.com>
|
|
With this function now it is possible to assert a token
given an index.
Signed-off-by: Carlos Maniero <carlosmaniero@gmail.com>
Reviewed-by: Johnny Richard <johnny@johnnyrichard.com>
Link: https://lists.sr.ht/~johnnyrichard/pipalang-devel/%3C20230418170136.3949-1-carlosmaniero%40gmail.com%3E
|
|
We want to tokenizer arithmetic expressions.
We are handling exceptional cases with UNKNOWN token.
Co-authored-by: Carlos Maniero <carlosmaniero@gmail.com>
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
|
|
Signed-off-by: Carlos Maniero <carlosmaniero@gmail.com>
Reviewed-by: Johnny Richard <johnny@johnnyrichard.com>
|
|
We are allocating heap memory to create tokens value, we can minimize the
number of allocations if we start using string_view.
We have other problems, right now the tokens value ownership are quite
unclear once the AST nodes also share the memory allocation done by
token_get_next_token function.
It's important to clarify we also have memory leaks on the current
implementation. Hence, we are going to start using string_view to make
the memory management easier. :^)
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
|
|
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
|