| Age | Commit message (Collapse) | Author |
|---|
|
This path implements a hashmap data structure using the FNV-1a hashing
function of 32 bits. However, there are a few limitations to consider:
a) The table does not automatically expand when it becomes too large.
b) The capacity of the hashmap must be a power of 2 to optimize
performance, as we utilize bitwise shifting instead of modulus
calculations.
c) Currently, there are no functions available to destroy hashmap
entries.
Collisions are handled using a linked list, which is not the most
optimal solution in terms of performance. However, it serves our current
needs adequately.
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
|
|
Until now the below code was not valid for pipac.
fn main(): i32 {
return fib(13);
}
fn fib(n: i32): i32 {
if n <= 1 {
return n;
}
return fib(n - 1) + fib(n - 2);
}
Pipa's parser was adding a function to scope after they were fully
parsed which means that fib's self-reference would not work.
Also, functions were required to follow the be called in the order they
are declared for the same scope reason so, the main function was
required to be defined after fib.
And how it is working now?
When a TOKEN_NAME is not found in the scope, instead of returning an
error, an unknown token is created as placeholder. The parser stores the
node reference and the token it was trying to parse.
During type checks, if the parser detects an unknown node, instead of
returning an error, it stores in that node what was the expected type.
After the NS is fully parsed a reevaluation is made on those unknown
nodes by setting the lexer back on the node's token position and parsing
the TOKEN_NAME again.
Ps: There is a typo on the unknown token. It will be addressed in
another commit since this issue was not introduced by this change.
Signed-off-by: Carlos Maniero <carlos@maniero.me>
|
|
For now function calls are following the C's calling convention, which
means they are using the following registers to pass functions'
arguments:
rdi, rsi, rdx, rcx, r8, r9
If a function has more then 6 parameters, the compilation will fail.
To enable function with more than 6 parameters we will need to save the
extra arguments on stack.
Naming:
parameters: function parameters are the variables a function receives.
arguments: Arguments are the values passed to a function when calling
it.
Calling mechanism:
When a function is called, all the expressions passed as argument are
evaluated, after the evaluation, the result is stored on the register
that represents its argument position, the first argument will be
stored on rdi, the second on rsi and so on.
Receiving mechanism:
When a function starts, the first thing it does is store all the
registers onto the stack. So rdi will be stored on -8(rbp), rsi on
-16(rbp) and so on. And, a ref_entry is created making the
relationship parameter-stack_offset.
Signed-off-by: Carlos Maniero <carlos@maniero.me>
|
|
It was necessary to test function calls errors.
Signed-off-by: Carlos Maniero <carlos@maniero.me>
|
|
We have been always parsing a single function. Since we want to have
multiple functions in a near future, this patch introduces an namespace
that represents an entire file.
To ensure a function is defined inside a namespace, a helper function
was created. Today our ast_node structure is highly exposed, and this is
something that Johnny and I have been discussed. So then, this is a
first step to try to protected the code generation from our ast tree.
Signed-off-by: Carlos Maniero <carlos@maniero.me>
|
|
The comparators && and || should have precedence over others comparators
(> < >= <= == !=).
Signed-off-by: Carlos Maniero <carlos@maniero.me>
|
|
If statements are now working, the only exception is for the comparators
|| and && that will be addressed in a further commit. Checks tested:
fn main(): i32 {
let n: i32 = 11;
if (n == 11) {
if n != 12 {
if n < 12 {
if n <= 11 {
if n > 10 {
if n >= 11 {
return 42;
}
}
}
}
}
}
return n;
}
To compile the && and || a precedence issue must be addressed: they must
have the highest precedence, witch is not working now:
1 == 2 || 3 != 2
The or should be the higher level of the tree in the example above.
Signed-off-by: Carlos Maniero <carlos@maniero.me>
|
|
This commit parses a if statement following the grammar bellow:
if boolean_expression {
n_epressions;
}
No else neither code generation was implemented.
Signed-off-by: Carlos Maniero <carlos@maniero.me>
|
|
Since the next step to pipa programming language is about having control
flow statements we could benefit ourselves by having a block node to
control scope. Now, functions has a block node, instead of an vector as
body. As you can see through the ast-dump:
FunctionDecl name='main'
└─ body:
└─ Block
└─ ReturnStmt
└─ Literal type=i32 value='69'
This same node kind can be used for parsing if, for and while blocks.
I could use ast_block_t as body for functions but instead, I opted to
use an ast_node_t. This brings the flexibility to, in the future, having
another function body kinds, such as arrow functions if we want to:
fn add(a: i32, b: i32): i32 => a + b;
Signed-off-by: Carlos Maniero <carlos@maniero.me>
|
|
After this commit, this is a valid expression:
1 || 2 && 3 > 4 < 5 >= 6 <= 7 == 8 != 9
Signed-off-by: Carlos Maniero <carlos@maniero.me>
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
|
|
When assign a variable or returning a value it now ensures that the
expression matches the expected type.
To make this possible a %result_type% field was added to ast_node_t and
this field is used whenever to make the comparison.
Signed-off-by: Carlos Maniero <carlos@maniero.me>
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
|
|
This is a simple implementation of a general propose single-linked list.
There is only the *prepend* function implemented at this moment. We can
alway revisit the code and implement new missing functionality on demand.
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
|
|
This commit introduces a new type for booleans. There is no code
generation for this type yet. The intention of this commit is to enable
flow control in the near future.
Signed-off-by: Carlos Maniero <carlos@maniero.me>
Reviewed-by: Johnny Richard <johnny@johnnyrichard.com>
|
|
The followed logic operators were added to lexer:
TOKEN_EQUAL ==
TOKEN_NOT !
TOKEN_NOT_EQUAL !=
TOKEN_GT >
TOKEN_GT_EQUAL >=
TOKEN_LT <
TOKEN_LT_EQUAL <=
TOKEN_AND &&
TOKEN_OR ||
Bitwise operators were also added
TOKEN_BITWISE_AND &
TOKEN_BITWISE_OR |
TOKEN_BITWISE_SHIFT_LEFT <<
TOKEN_BITWISE_SHIFT_RIGHT >>
TOKEN_BITWISE_XOR ^
TOKEN_BITWISE_NOT ~
TOKEN_EQUAL '=' was renamed TOKEN_ASSIGN, and now TOKEN_EQUAL is used
for the logical comparator '=='.
Signed-off-by: Carlos Maniero <carlos@maniero.me>
|
|
Signed-off-by: Carlos Maniero <carlos@maniero.me>
Reviewed-by: Johnny Richard <johnny@johnnyrichard.com>
|
|
This commit introduces a few changes in pipalang syntax. Now, both
functions and variables requires keywords to be defined.
before:
main(): i32 {
a: i32 = 2;
return a;
}
now:
fn main(): i32 {
let a: i32 = 2;
return a;
}
Signed-off-by: Carlos Maniero <carlos@maniero.me>
Reviewed-by: Johnny Richard <johnny@johnnyrichard.com>
|
|
Munit uses the follow format:
Error: file.c:line: error message
But this error format does not works well with vim make program.
This commit changes the error to the follow format:
file.c:line: Error: error message
Signed-off-by: Carlos Maniero <carlos@maniero.me>
|
|
Signed-off-by: Carlos Maniero <carlos@maniero.me>
|
|
Since it is possible to look a future token without consuming it, it was
possible to split the block parser into small chunks of code.
There is the performance drawback, because now the parser makes multiple
lookups to the same token. However IMO that it is not a big concern
given the small computation required to get a token. Also it can be
easily addressed by computing all token in advance.
Memory Leak:
During the refactor I found some extra memory leaks related to not
released scopes. So then, more than just printing a message I introduced
an assert on scope.c to make sure developers will get this feedback asap
because our testing framework suppress messages from stderr when the
test passes.
Signed-off-by: Carlos Maniero <carlos@maniero.me>
|
|
Previously, during block declaration, the parser consumed the token
which caused some parsers (such as return and variable declaration) to
not be self-contained and to depend on the callee to start the parser.
In this commit, I've refactored the parser to only look for future
tokens using lookahead, and delegate the consumption to child parser
functions. This results in a more modular and self-contained parser that
improves the overall maintainability and readability of the code.
Signed-off-by: Carlos Maniero <carlos@maniero.me>
|
|
During the refactoring process, I identified a memory leak where the
return argument was allocated but not freed in case of an error.
It also introduces the concept of keyword tokens. Where return is now a
keyword simplifying the parser.
Signed-off-by: Carlos Maniero <carlos@maniero.me>
|
|
In many situations, the parser is responsible for reserving memory for
nodes, particularly during function body parsing. This commit introduces
a new standard where parser functions not only allocate memory for
ast_nodes, but also return them. In case of a parser error, a NULL
pointer is returned.
This standard will be extended to other parsers in future commits,
ensuring consistency throughout the codebase.
Signed-off-by: Carlos Maniero <carlos@maniero.me>
|
|
This commit introduces variable assignment making it possible to
change a variable value. Example:
myvar: i32 = 1;
myvar = 2;
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
Co-authored-by: Carlos Maniero <carlos@maniero.me>
|
|
The only way to get the next token was by consuming it. So then, our
parser starts to become hard to understand, once sometimes we just
want to take a look on the next token to understand what should be the
next kind of expression.
This commit introduces a new function that will help us to improve our
parser implementation.
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
Reviewed-by: Carlos Maniero <carlos@maniero.me>
|
|
This patch adds the variable compilation and uses a scope (a stack of
map) to lookup for identities.
Today we use a vector + ref_entry structs in order to achieve the scope
implementation. The ref_entry lacks memory management, we are still no
sure who will be the owner of the pointer.
We also want to replace the scope a hashtable_t type as soon as we get
one.
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
Co-authored-by: Carlos Maniero <carlosmaniero@gmail.com>
|
|
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
|
|
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
|
|
We are parsing variables/functions and checking if they are defined on
scope. Otherwise we fail the parsing with a nice message.
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
Co-authored-by: Carlos Maniero <carlosmaniero@gmail.com>
|
|
Prior to this change, ast_variable_declaration_t and
ast_function_declaration_t used a string_view as an identifier. However,
to support scoped identifiers, it is more appropriate to use an
ast_identifier_t as a reference.
Signed-off-by: Carlos Maniero <carlosmaniero@gmail.com>
Co-authored-by: Johnny Richard <johnny@johnnyrichard.com>
|
|
Before accepting an identifier, the parser should check if that
identifier will be available. With this implementation it will be
possible. Take the following code example:
main(): i32 {
return my_exit_code;
}
The parser must return an error informing that *my_exit_code* is not
defined in the example above. The ast scope is a support module for
parser and ast, simplifying identifier resolution.
Once a curly bracket ({) is open the *scope_enter()* is called and when
it is closed (}) we pop the entire stack with *scope_leave()*.
Signed-off-by: Carlos Maniero <carlosmaniero@gmail.com>
|
|
The AST was using a string view to distinguish the operation kind. An
enum was created for this purpose simplifying code generation.
Signed-off-by: Carlos Maniero <carlosmaniero@gmail.com>
Reviewed-by: Johnny Richard <johnny@johnnyrichar.com>
|
|
The +, -, *, and / tokens used to be TOKEN_OP, but the TOKEN_OP has been
removed and a token for each operation has been introduced. Python's
token names were followed: https://docs.python.org/3/library/token.html
Signed-off-by: Carlos Maniero <carlosmaniero@gmail.com>
Reviewed-by: Johnny Richard <johnny@johnnyrichar.com>
|
|
We want to keep the code style consistent, this first commit adds a
.clang-format in order to "document" our style code.
This patch also adds a target *linter* to Makefile which will complain
if we have any style issue on test and src dirs.
I have run the follow command to create the .clang-format file:
$ clang-format -style=mozilla -dump-config > .clang-format
And I also made some adjusts to .clang-format changing the following
properties:
PointerAlignment: Right
ColumnLimit: 120
Commands executed to fix the current styling:
$ find . -name *.h | xargs clang-format -i
$ find . -name *.c | xargs clang-format -i
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
|
|
This commit adds support for variables and identifiers in the function
body of the parser, stored as a vector.
However, at this point, identifier resolution is not fully implemented,
and we currently accept identifiers without checking if they can be
resolved. This is a known limitation that will be addressed in a future
commit once hash-tables are added to the parser.
Signed-off-by: Carlos Maniero <carlosmaniero@gmail.com>
Reviewed-by: Johnny Richard <johnny@johnnyrichard.com>
|
|
This commit introduces a full-featured CLI that allows you to compile a
file, set the gas and linker path, and define the executable output.
Signed-off-by: Carlos Maniero <carlosmaniero@gmail.com>
Reviewed-by: Johnny Richard <johnny@johnnyrichard.com>
Link: https://lists.sr.ht/~johnnyrichard/pipalang-devel/patches/40642
|
|
Previously, we lacked a dynamic array for storing children elements in
our abstract syntax tree (AST). This commit introduces a new
implementation that dynamically adjusts its capacity as elements are
added, using a doubling strategy.
I considered two approaches for managing the vector's memory
allocation: allocating it on the heap, or providing a vector_init
function that allocates only the items array. Ultimately, I decided to
provide a vector_new function for instantiating the vector, as this
aligns with the expected usage pattern when there is a destroy function.
With this new implementation, we can efficiently store and manage AST
children, enabling more flexible and expressive tree structures.
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
|
|
We decided for using push and pop to simplify the implementation, we
want to revisit the approach latter.
Signed-off-by: Carlos Maniero <carlosmaniero@gmail.com>
Co-authored-by: Johnny Richard <johnny@johnnyrichard.com>
|
|
This patch implements the AST creation for arithmetic expressions.
NOTE:
The implementation works only for integer numbers.
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
Reviewed-by: Carlos Maniero <carlosmaniero@gmail.com>
|
|
Since we want to extend our code to support multiple kind of expression
it does not make sense that the return statement always return a number.
For now on, return statement has an ast_node_t as argument, meaning that
it could be anything. The literal_node_t was also implemented in order
to keep the application behavior.
Following the C's calling convention the literal values are stored at
%eax and the return takes this argument to do anything it is needed.
Signed-off-by: Carlos Maniero <carlosmaniero@gmail.com>
Reviewed-by: Johnny Richard <johnny@johnnyrichard.com>
|
|
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
|
|
This tests perform the whole cycle. It takes the output from pipac
compile, execute and check the returned status code.
Signed-off-by: Carlos Maniero <carlosmaniero@gmail.com>
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
|
|
Previously, when an error occurred during parsing, the application
would exit, making it difficult to test the parser and limiting the
compiler's extensibility. This commit improves the parser's error
handling by allowing for continued execution after an error, enabling
easier testing and increased flexibility.
The parser is prepared to handle multiples errors, although the
current implementation always returns a single error, it may be
useful given multiples functions where we can show errors by context.
Signed-off-by: Carlos Maniero <carlosmaniero@gmail.com>
Reviwed-by: Johnny Richard <johnny@johnnyrichard.com>
|
|
Previously, the abstract syntax tree (AST) used static types, meaning
that an ast_function_t would always have a ast_return_stmt_t as
its body. However, this assumption is not always true, as we may have
void functions that do not have a return statement. Additionally, the
ast_return_stmt_t always had a number associated with it, but this too
is not always the case.
To make this possible, I need to perform a few changes in the whole
project. One of the main changes is that there is no longer the
inheritance hack. That mechanism was replaced by composition and
pointers where required for recursive type reference.
It is important to mention that I decided to use union type to implement
the composition. There is two main advantages in this approach:
1. There is only one function to allocate memory for all kind of nodes.
2. There is no need to cast the data.
In summary, this commit introduces changes to support dynamic typing
in the AST, by replacing the inheritance hack with composition and
using union types to simplify memory allocation and type casting.
Signed-off-by: Carlos Maniero <carlosmaniero@gmail.com>
Reviewed-by: Johnny Richard <johnny@johnnyrichard.com>
|
|
Signed-off-by: Carlos Maniero <carlosmaniero@gmail.com>
Reviewed-by: Johnny Richard <johnny@johnnyrichard.com>
|
|
With this function now it is possible to assert a token
given an index.
Signed-off-by: Carlos Maniero <carlosmaniero@gmail.com>
Reviewed-by: Johnny Richard <johnny@johnnyrichard.com>
Link: https://lists.sr.ht/~johnnyrichard/pipalang-devel/%3C20230418170136.3949-1-carlosmaniero%40gmail.com%3E
|
|
We want to tokenizer arithmetic expressions.
We are handling exceptional cases with UNKNOWN token.
Co-authored-by: Carlos Maniero <carlosmaniero@gmail.com>
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
|
|
Signed-off-by: Carlos Maniero <carlosmaniero@gmail.com>
Reviewed-by: Johnny Richard <johnny@johnnyrichard.com>
|
|
We are allocating heap memory to create tokens value, we can minimize the
number of allocations if we start using string_view.
We have other problems, right now the tokens value ownership are quite
unclear once the AST nodes also share the memory allocation done by
token_get_next_token function.
It's important to clarify we also have memory leaks on the current
implementation. Hence, we are going to start using string_view to make
the memory management easier. :^)
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
|
|
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
|
