Renyi

Extend

Embedding Renyi in a host#

How a Rust host loads a Renyi module under a grant and calls its functions, with the effect system as the sandbox.

On this page
  1. 1. The grant
  2. 2. Loading a module
  3. 3. Calling
  4. 4. The memory budget
  5. 5. The same grant from the command line
  6. 6. What it promises
  7. 7. From C and the languages that call C

A host program loads a Renyi module with a grant and calls its public functions. The module reaches nothing the grant does not name: there is no ambient I/O, no reflection and no native call without foreign, so the effect system is the sandbox, with no process, container or Wasm runtime around it (decisions Q3, AP1 and AP2; design/07-system-design.md section 4). The API is Rust, in the renyi crate; the C API of decision AW46, for C hosts and the languages that call C, is a layer over it in the crate renyi_c (section 7).

1. The grant#

A grant is spelled as a needs clause is, with the same scopes, budgets (at most, decision P2) and guards (only to, decision P3):

console, network.http("api.example.com") at most 60 per minute, filesystem.read("data")

In a file it is a JSON object with the memory budget beside it; both fields are optional, and there are no others:

JSON
{
  "grant": "console, network.http(\"api.example.com\") at most 60 per minute",
  "memory": "256 megabytes"
}

Grant::parse reads the clause, Grant::from_json and Grant::from_file the object; the memory is bytes, kilobytes, megabytes or gigabytes. An empty clause grants nothing: a module loaded with it can compute and nothing else. A program's main may declare a memory budget of its own, memory at most 256 megabytes in its needs (decision AW64); under renyi run --sandbox the smaller of the two is in force.

2. Loading a module#

Rust
use renyi::{Grant, Sandbox, Value};

let grant = Grant::from_file("grant.json")?;
let mut module = Sandbox::load("tools/summarize.ry", grant)?;
for function in module.functions() {
    println!("{}: {}", function.name, function.signature);
}

Sandbox::load reads the file, resolves its imports from its directory and its project (renyi.json, decision AC1) and checks it as renyi check does; a module with errors is refused with the diagnostics, and the warnings are kept (warnings()). Sandbox::load_source(name, text, grant) takes the text from memory, with name the file name its imports resolve from. A bytecode file (.ryc) is refused: it carries no visibility.

functions() lists the public functions of the module in source order: the name, the signature as the project map spells it, the purpose, the number of parameters, the needs clause, for a function with expose as tool the JSON Schema of its parameters as a text, as renyi tools writes it (decision D6), and whether call_json can decode every parameter (callable, section 7).

3. Calling#

Rust
match module.call("summarize", vec![Value::text(report), Value::integer(3)]) {
    Ok(value) => println!("{value:?}"),
    Err(error) => eprintln!("summarize {error}"),
}

A call names a public function and gives its arguments in order, as Values (Value::text, Value::integer, Value::Boolean, lists, maps and the rest of renyi_vm::Value). The value comes back, or a CallError:

The grant's budgets count across the calls of one sandbox as one run: filesystem.read("data") at most 2 per run admits two reads in the sandbox's life, whichever calls make them. Each call runs on a VM of its own, and nothing survives from one call to the next but the budgets: a Renyi module has no mutable state at the top level, so there is nothing else to keep.

A value the host hands in may carry a guard of the grant (decision P3):

Rust
let secret = module.guarded(Value::text(key), "filesystem.read(\"secrets\")")?;
module.call("sign", vec![secret, Value::text(payload)])?;

guarded names a capability of the grant that has only to sinks; the value, and whatever the module computes from it, may leave the module only through those sinks, and any other outgoing primitive fails with Guarded.

A host that keeps its values as JSON calls through call_json instead (decision AW46): the arguments as one JSON array text, the value back as one JSON text, both in the codec of std.json (a record an object by its field names, a variant an object with kind, a list an array, a map an object, Bytes base64, an Instant ISO 8601). Each argument is decoded against the declared type of its parameter before anything runs; one that does not fit is refused with the parameter named (argument 1 (`left`) does not decode as `Integer`: Mismatch(...)). The guards come as a slice with, per argument, the capability whose guard the argument carries, or None.

Rust
let text = module.call_json("shift", r#"[{"across": 1, "down": 2}, 10]"#, &[])?;
assert_eq!(text, r#"{"across":11,"down":12}"#);

4. The memory budget#

memory in the grant bounds the bytes a call holds above the level at its start (a function other than main carries no budget of its own, so the host's is the one in force for a call). The VM reads the count at every call, every primitive and every loop turning, and a call over the budget ends as a whole with OverMemory { limit, used }; the next call starts afresh. The machine code the VM generates for a sandbox with a memory budget keeps the budget as the interpreter does, at a body's entry and at a loop's back edge (decision AW66), and the memory of its compilations is not counted; a sandbox without one pays nothing for the test.

The count comes from the process's allocator, which must be the counting one. The official renyi binary declares it; a host declares it in one line:

Rust
#[global_allocator]
static ALLOCATOR: renyi::Allocator = renyi::Allocator;

renyi::Allocator is mimalloc under the count. A host that keeps an allocator of its own wraps it instead:

Rust
#[global_allocator]
static ALLOCATOR: renyi::Counting<std::alloc::System> = renyi::Counting(std::alloc::System);

Without either, a grant with memory is refused at load with the line to add; a grant without memory needs nothing. The count costs one load per allocation while no budget is in force. One budget is in force at a time in a process, and it counts the process's allocations, so a host that allocates on other threads during a budgeted call sees those bytes counted too.

5. The same grant from the command line#

Shell
renyi run --sandbox grant.json program.ry
renyi record --sandbox grant.json program.ry
renyi serve --watch --sandbox grant.json service.ry

main runs under its declared grant intersected with the file's, the file's budgets and guards added, and its memory budget in force; main must need nothing the grant does not cover, or the command refuses to start naming the capability (the sandbox does not grantnetwork.http, whichmainneeds), as --deny does (decision E3). The run manifest and a recording name the effective grant. A run over the memory budget exits with status 2, as a crash does.

6. What it promises#

The sandbox isolates effects (the grant), calls (the budgets), where data may go (the guards), time (within deadlines inside the module) and memory (the budget). It does not hide timing or scheduling from the module; foreign code and Python, once granted, are outside every guarantee, as extensions.md and python.md say; and the module's console output goes to the process's standard streams in this version. A module's calls run one after the other (decision S2), each on its own VM.

7. From C and the languages that call C#

The crate renyi_c builds the same API as a shared library (librenyi_c.so, librenyi_c.dylib, renyi_c.dll) and a static one (librenyi_c.a), with the header crates/renyi_c/include/renyi.h (decision AW46, stage 40a). The library declares the counting allocator, so a grant with memory works with no declaration of the host's. Values cross as JSON texts, as call_json takes and answers them.

Shell
cargo build --release -p renyi_c
cc host.c -Icrates/renyi_c/include -Ltarget/release -lrenyi_c -o host

Every text crosses as a pointer and a length, UTF-8, not NUL-terminated. A text the library answers belongs to the handle it came from and is valid until the next call on that handle; a handle is for one thread at a time.

crates/renyi_c/examples/host.c is a host of fifty lines over calc.ry beside it; CI compiles it with the system's compiler, runs it and compares its output with host.expected. A test of the crate holds the header's functions equal to the library's exports.

From Python. bindings/python/renyi.py is the binding: one file over ctypes and the standard library, no dependency (stage 40b). It finds the library through RENYI_C_LIB, else beside itself (the release archive's layout), else under target/release and target/debug of the checkout it sits in.

Python
import renyi

with renyi.Sandbox.load("calc.ry", {"grant": "console", "memory": "16 megabytes"}) as calc:
    for function in calc.functions():
        print(function["signature"])
    print(calc.call("add", 2, 3))                          # 5
    print(calc.call("shift", {"across": 1, "down": 2}, 10))  # {'across': 11, 'down': 12}

Sandbox.load(path, grant) and Sandbox.load_source(name, text, grant) take the grant as a dict, as a JSON text or as None, and raise LoadError with the reason; the sandbox is a context manager and close frees the handle. call(name, *arguments, guards=None) takes Python values json.dumps writes and answers what json.loads reads, or raises Failed, Crashed, Exited (with code), Refused or OverMemory (with limit and used), each a RenyiError; module, grant, warnings and functions() describe the module; renyi.version() and renyi.api_version() tell the versions. bindings/python/example.py is the C example in Python, which CI runs and compares with example.expected.

The archive. Every release carries, beside the binary's archive for each target, renyi_c-<version>-<target>.tar.gz (.zip on Windows): the shared and the static library, renyi.h, renyi.py, example.py, host.c, calc.ry, the license and a README (bindings/README.md). The crate renyi_c is on crates.io with the others (docs/RELEASE.md).

View the source of this page