exo

exo: Run your own AI cluster at home with everyday devices. Maintained by [exo labs](https://exolabs.net). Supports iPhone, iPad, Android, Mac, Linux, pretty much any device!

[Homepage](https://github.com/exo-explore/exo) | [Documentation](https://github.com/exo-explore/exo/docs) | [Discord](TODO)

---

Get Involved

We have a list of bounties in this sheet.

Features

Wide Model Support

exo supports LLaMA and many other popular models.

Dynamic Model Partitioning

exo optimally splits up models based on the current network topology and device resources available.

Automatic Device Discovery

Zero manual configuration. exo will automatically discover other devices using the best method available.

ChatGPT-compatible API

exo provides a ChatGPT-compatible API for running models. This means it's a one-line change in your application to run models on exo.

Inference Engines

exo supports the following inference engines:

• MLX
• tinygrad

Usage

Documentation

Example Usage on Multiple Devices

Device 1:

./run.sh

Device 2:

./run.sh

That's it! No configuration required - exo will automatically discover the other device(s).

A ChatGPT-like web interface will be available on each device on port 8000 http://localhost:8000.

An API endpoint will be available on port 8001. Example usage:

curl -X POST http://localhost:8001/api/v1/chat -H "Content-Type: application/json" -d '{"messages": [{"role": "user", "content": "What is the meaning of life?"}]}'