 JakobDylanC
|
f2f61ccee6
inference engine selection improvements
|
9 months ago |
 Alex Cheema
|
4e46232364
add simple prometheus metrics collection, with a prometheus / grafana instance for live dashboard. related: #22
|
9 months ago |
 Alex Cheema
|
2e419ba211
Merge pull request #48 from itsknk/intel-mac
|
9 months ago |
 itsknk
|
e934664168
implement dynamic inference engine selection
|
10 months ago |
 Alec Potluri
|
db583a863f
disable tui flag
|
10 months ago |
|
Alex Cheema
|
e49924e1b9
add chatgpt-api-response-timeout-secs flag, set this to 20 mins in test
|
10 months ago |
|
Alex Cheema
|
a342e1abd8
add web url and chatgpt api endpoint to panel (fixes #43), fix a rounding error in the partition to shard mapping implementation
|
10 months ago |
|
Alex Cheema
|
d9484906a3
remove the spammy logs
|
10 months ago |
|
Alex Cheema
|
4b592f9d45
exo topology visualisation that shows the topology of the network, device capabilities and the currently active node using opaque statuses. fixes #36. ready for #33
|
10 months ago |
|
Alex Cheema
|
35177690bd
by default find an ephemeral node port fixes #35, more robust topology updates. both fix #15 and #14
|
10 months ago |
|
Alex Cheema
|
945f90f676
allow overriding inference_engine and separate flag for TINYGRAD_DEBUG
|
10 months ago |
|
Alex Cheema
|
72fe293729
exo text on start and stop
|
10 months ago |
|
Alex Cheema
|
1e1e11cdc6
check if inference_engine has tokenizer before printing with it
|
10 months ago |
|
Alex Cheema
|
eb92da2c3e
cleaner chatgpt api impl with async callbacks
|
10 months ago |
|
Alex Cheema
|
71e00745cc
fix tokenizer inconsistencies
|
10 months ago |
|
Alex Cheema
|
ce46f00059
linux device capabilities
|
10 months ago |
|
Alex Cheema
|
dbbc7be57f
remove hard dependency on MLX fixes #8
|
10 months ago |
|
Alex Cheema
|
dd8d18128c
add an opaque inference_state that inference engines can use to pass around small state to other devices
|
10 months ago |
|
Alex Cheema
|
f2895cbcee
revive the chatgpt api endpoint on :8000
|
10 months ago |
|
Alex Cheema
|
05b9fa497d
initialize node id to uuid4 if not set
|
10 months ago |
|
Alex Cheema
|
32f2e36fd3
main rename
|
10 months ago |
|
Alex Cheema
|
5bbde22a23
move everything under exo module
|
10 months ago |
|
Alex Cheema
|
36b8456798
collect global topology with local peer visibility, ring memory weighted partitioning strategy
|
10 months ago |
|
Alex Cheema
|
563dcb56b0
mlx sharded implementation with example of distributed inference
|
10 months ago |
|
Alex Cheema
|
a21f59ff45
scaffolding for networking, inference and orchestration
|
10 months ago |
