 Alex Cheema
|
6c1bf127b3
add --max-parallel-downloads flag that limits the number of downloads at a time with asyncio.semaphore
|
9 kuukautta sitten |
|
Alex Cheema
|
e6902b2fcf
add --download-quick-check flag to bypass the hf api calls / remote file checks
|
9 kuukautta sitten |
|
Alex Cheema
|
71591d2ebc
display all interfaces web chat and chatgpt api are available on fixes #134
|
9 kuukautta sitten |
|
Alex Cheema
|
6bddb2a9dc
download edge cases
|
9 kuukautta sitten |
|
Alex Cheema
|
f29963f41e
preemptively start downloads when any node starts processing a prompt. this fixes #104
|
9 kuukautta sitten |
|
Alex Cheema
|
476a714bbb
make a separate ShardDownloader abstract class w HFShardDownloader. this opens up plugging in different methods of downloading model shards e.g. #79 / #16
|
9 kuukautta sitten |
|
Alex Cheema
|
d22ed12e7b
bring tinygrad to parity with mlx on llama models, show progress of each download file
|
9 kuukautta sitten |
|
Alex Cheema
|
545a486ed3
separate hf_helpers, make extra dir with download_hf script, unify downloading so tinygrad uses the same method as mlx and interoperable model formats
|
9 kuukautta sitten |
|
Alex Cheema
|
0bfb8e3b6d
sticky node ids #16
|
9 kuukautta sitten |
|
Alex Cheema
|
d6a7e46324
async model downloading with download progress. fixes #102. related: #16 #104
|
9 kuukautta sitten |
|
Alex Cheema
|
57b2f2a4e2
fix ruff lint errors
|
9 kuukautta sitten |
|
Alex Cheema
|
9a373c2bb0
make configurable discovery timeout
|
10 kuukautta sitten |
|
Alex Cheema
|
63a05d5b4f
make configurable discovery timeout
|
10 kuukautta sitten |
 Alex Cheema
|
174cff071e
Merge pull request #58 from jakobdylanc/main
|
10 kuukautta sitten |
|
Alex Cheema
|
b0e7dd9d2d
add max-generate-tokens flag fixes #54
|
10 kuukautta sitten |
 JakobDylanC
|
f2f61ccee6
inference engine selection improvements
|
10 kuukautta sitten |
|
Alex Cheema
|
4e46232364
add simple prometheus metrics collection, with a prometheus / grafana instance for live dashboard. related: #22
|
10 kuukautta sitten |
|
Alex Cheema
|
2e419ba211
Merge pull request #48 from itsknk/intel-mac
|
10 kuukautta sitten |
 itsknk
|
e934664168
implement dynamic inference engine selection
|
10 kuukautta sitten |
 Alec Potluri
|
db583a863f
disable tui flag
|
10 kuukautta sitten |
|
Alex Cheema
|
e49924e1b9
add chatgpt-api-response-timeout-secs flag, set this to 20 mins in test
|
10 kuukautta sitten |
|
Alex Cheema
|
a342e1abd8
add web url and chatgpt api endpoint to panel (fixes #43), fix a rounding error in the partition to shard mapping implementation
|
10 kuukautta sitten |
|
Alex Cheema
|
d9484906a3
remove the spammy logs
|
10 kuukautta sitten |
|
Alex Cheema
|
4b592f9d45
exo topology visualisation that shows the topology of the network, device capabilities and the currently active node using opaque statuses. fixes #36. ready for #33
|
10 kuukautta sitten |
|
Alex Cheema
|
35177690bd
by default find an ephemeral node port fixes #35, more robust topology updates. both fix #15 and #14
|
10 kuukautta sitten |
|
Alex Cheema
|
945f90f676
allow overriding inference_engine and separate flag for TINYGRAD_DEBUG
|
10 kuukautta sitten |
|
Alex Cheema
|
72fe293729
exo text on start and stop
|
10 kuukautta sitten |
|
Alex Cheema
|
1e1e11cdc6
check if inference_engine has tokenizer before printing with it
|
10 kuukautta sitten |
|
Alex Cheema
|
eb92da2c3e
cleaner chatgpt api impl with async callbacks
|
10 kuukautta sitten |
|
Alex Cheema
|
71e00745cc
fix tokenizer inconsistencies
|
10 kuukautta sitten |
