 Alex Cheema
|
ee5204fbca
readme installation instructions
|
hai 1 ano |
|
Alex Cheema
|
78da11e10b
slightly nicer readme
|
hai 1 ano |
|
Alex Cheema
|
2fc472c8fe
slightly nicer readme
|
hai 1 ano |
|
Alex Cheema
|
8ff3e263a0
slightly nicer readme
|
hai 1 ano |
|
Alex Cheema
|
32f2e36fd3
main rename
|
hai 1 ano |
|
Alex Cheema
|
5bbde22a23
move everything under exo module
|
hai 1 ano |
|
Alex Cheema
|
c851644a43
update requirements, specify exact versions
|
hai 1 ano |
|
Alex Cheema
|
32972033dd
update readme
|
hai 1 ano |
|
Alex Cheema
|
5ef07d41a5
readme
|
hai 1 ano |
|
Alex Cheema
|
490fa102a4
tinygrad inference engine
|
hai 1 ano |
|
Alex Cheema
|
e6f387a690
handle is_finished
|
hai 1 ano |
|
Alex Cheema
|
b01f69bb6b
add support for multiple concurrent requests with request ids
|
hai 1 ano |
|
Alex Cheema
|
7077652c8e
graceful node shutdown
|
hai 1 ano |
|
Alex Cheema
|
ca6095c04d
a generic test for every inference engine
|
hai 1 ano |
|
Alex Cheema
|
850b72d3ea
make StatefulShardedModel callable, add some tests for mlx sharded inference
|
hai 1 ano |
|
Alex Cheema
|
6ee0547eff
fix layer calculation for sharded llama
|
hai 1 ano |
|
Alex Cheema
|
445eda156c
dynamically assign shards to nodes deterministically weighted by memory
|
hai 1 ano |
|
Alex Cheema
|
36b8456798
collect global topology with local peer visibility, ring memory weighted partitioning strategy
|
hai 1 ano |
|
Alex Cheema
|
3a66a0a4a8
add requirements.txt
|
hai 1 ano |
|
Alex Cheema
|
ee96c6b023
add another test for device capabiities on MacBook Air
|
hai 1 ano |
|
Alex Cheema
|
6c8c9ee7b1
topology with partitioning strategy
|
hai 1 ano |
|
Alex Cheema
|
563dcb56b0
mlx sharded implementation with example of distributed inference
|
hai 1 ano |
|
Alex Cheema
|
a21f59ff45
scaffolding for networking, inference and orchestration
|
hai 1 ano |
