exo/docs/runtime/overview.md

Runtime Overview

Overview

A typical runtime consists of the following parts:

• Compiled
• Allocator
• Program
• Compiler

Compiled

The Compiled class is responsible for initializing and managing a device.

::: tinygrad.device.Compiled

options:
    members: [
        "synchronize"
    ]
    show_source: false

Allocator

The Allocator class is responsible for managing memory on the device. There is also a version called the LRUAllocator, which caches allocated buffers to optimize performance.

::: tinygrad.device.Allocator

options:
    members: true
    show_source: false

::: tinygrad.device.LRUAllocator

options:
    members: true
    show_source: false

Program

The Program class is created for each loaded program. It is responsible for compiling and executing the program on the device. As an exmaple, here is a ClangProgram implmentation which loads program and runs it.

::: tinygrad.runtime.ops_clang.ClangProgram

options:
    members: true

Compiler

The Compiler class compiles the output from the Renderer and produces it in a device-specific format.

::: tinygrad.device.Compiler

options:
    members: true