diff --git a/libraries.md b/libraries.md new file mode 100644 index 0000000..dec5df2 --- /dev/null +++ b/libraries.md @@ -0,0 +1,183 @@ +# Tomo Library/Module Design + +There are two ways to "import" code that is defined elsewhere: local files from +the same project and shared library objects from another project. The first +type of import (local files) is necessary for splitting large projects into +smaller components for ease of understanding and compilation speed. The second +type of import (shared libraries) is to allow you to install third party +libraries or frameworks that can be used across many projects. + +## Local Imports + +To see how local imports work, let's look at a simple file: + +``` +// File: foo.tm +my_variable := 123 +``` + +When this file is compiled to a static object file by `tomo -c foo.tm`, it +produces the following C header file and C source file: + +```c +// File: foo.tm.h +#pragma once +#include + +extern Int_t foo$my_variable; +``` + +```c +// File: foo.tm.c +#include +#include "foo.tm.h" + +Int_t foo$my_variable = I64(123); +``` + +Notice that the symbols defined here (`foo$my_variable`) use a file-based +prefix that includes a dollar sign. C compilers support an extension that +allows dollar signs in identifiers, and this allows us to use guaranteed-unique +prefixes so symbols from one file don't have naming collisions with symbols +in another file. + +The C file is compiled by invoking the C compiler with something like: `cc + -c foo.tm.c -o foo.tm.o` + +Now, what happens if we want to _use_ the compiled object file? + +``` +// File: baz.tm +foo := use ./foo + +func say_stuff(): + say("I got {foo.my_variable} from foo") + +func main(): + say_stuff() +``` + +If I want to run `baz.tm` with `tomo baz.tm` then this transpiles to: + +```c +// File: baz.tm.h +#pragma once +#include +#include "./foo.tm.h" + +void baz$say_stuff(); +void baz$main(); +``` + +```c +// File: baz.tm.c +#include +#include "baz.tm.h" + +public void baz$say_stuff() +{ + say(CORD_all("I got ", Int$as_text(&foo$my_variable, no, &$Int), " from foo")); +} + +public void baz$main() +{ + baz$say_stuff(); +} +``` + +Then `baz.tm.o` is compiled to a static object with `cc -c baz.tm.c +-o baz.tm.o`. + +Next, we need to create an actual executable file that will invoke `baz$main()` +(with any command line arguments). To do that, we create a small wrapper +program: + +```c +// File: /tmp/program.c +#include +#include "baz.tm.h" + +int main(int argc, char *argv[]) +{ + tomo_init(); + if (argc > 1) + errx(1, "This program doesn't take any arguments."); + baz$main(); + return 0; +} +``` + +This program is compiled with the already-built object files to produce an +executable binary called `foo` like this: `cc /tmp/program.c +foo.tm.o baz.tm.o -o baz` + +Finally, the resulting binary can be executed to actually run the program! + + +## Shared Library Imports + +Now, what's the story with shared library imports? The equivalent process for C +is to create a `.so` or `.dll` file. In order to build a shared library, you +run the command `tomo -s=qux.1.2.3 file1.tm file2.tm...`. Each specified file +will have its `.o` static object file compiled, along with its dependencies, +and all of the resulting `.o` files will be linked together by `tomo` with a +command like `cc -Wl,-soname=libqux.1.2.3.so -shared file1.tm.o +file2.tm.o dep1.tm.o ... -o libqux.1.2.3.so`. The specified files must not +define the same public symbols as each other, since `foo` will now be treated +as a single namespace that holds all the symbols from each of the given files. + +### Symbol Uniqueness + +In the future, each of these symbols will be given an extra prefix to prevent +namespace collisions and a standalone header file will be built that defines +every public symbol in the library in a way that a C compiler can understand. +In our example, running `tomo -s=qux.1.2.3 foo.tm baz.tm` would produce a +header file like this: + +```c +#pragma once +#include + +extern Int_t qux$1$2$3$foo$my_variable; +extern void qux$1$2$3$baz$say_stuff(); +``` + +### Installing + +Now, the components necessary to install this shared library on your computer +are these: + +- The `.so` file, installed in a standard location. In our case, we will default + to `~/.local/share/tomo/lib/libqux.so` +- The standalone `.h` file, installed in a standard location. We default to + `~/.local/share/tomo/include/qux.h` +- All of the source `.tm` files (which store type information necessary for tomo + to understand what's being imported. These will be installed to + `~/.local/share/tomo/src/qux/` + +### Using Shared Libraries + +To use a shared library, write a statement like `use qux` with an unqualified +name (i.e. not an absolute or relative path like `/qux` or `./qux`). When a +program uses a shared library, that shared library gets dynamically linked to +the executable when compiling, and all of the necessary symbol information is +read from the source files during compilation. + +### Library Versioning + +In order to accommodate multiple versions of the same shared libraries on a +system, users may specify a library version when compiling, for example: `tomo +-s qux.1.2.3 foo.tm baz.tm` During installation, symlinks are created to map +less specific version numbers to more specific version numbers. For example, +when installing `qux.1.2.3`, links are created: + +- `~/.local/share/lib/tomo/libqux.1.2.so` -> `~/.local/share/lib/tomo/libqux.1.2.3.so` +- `~/.local/share/lib/tomo/libqux.1.so` -> `~/.local/share/lib/tomo/libqux.1.2.3.so` +- `~/.local/share/lib/tomo/libqux.so` -> `~/.local/share/lib/tomo/libqux.1.2.3.so` +- And so on for `include/tomo/libqux.1.2.3.h` and `src/tomo/qux.1.2.3/` + +If there are multiple versions (e.g. `1.2.3` and `1.3.0`), then links point at +the highest-numbered version with the necessary prefix. In this case, `qux -> +qux.1.3.0`, `qux.1 -> qux.1.3.0`, `qux.1.3 -> qux.1.3.0`, and `qux.1.2 -> +qux.1.2.3`. +