 Nel Nibcord
|
d69a9c4d43
Enabled inference engine intercompatibility
|
hai 8 meses |
 Alex Cheema
|
11c5e9e1af
Merge pull request #449 from blindcrone/remove_inference_state
|
hai 8 meses |
|
Nel Nibcord
|
42172b2c39
Updated unit tests
|
hai 8 meses |
|
Nel Nibcord
|
8b71d57da7
Removed inference state entirely
|
hai 8 meses |
 Alex Cheema
|
4c98108d3e
increase grpc msg limit
|
hai 8 meses |
|
Alex Cheema
|
1946373716
fix debug log
|
hai 8 meses |
|
Alex Cheema
|
42db2ffbc8
Merge pull request #440 from blindcrone/debug-fix
|
hai 8 meses |
|
Nel Nibcord
|
f02e62c9c0
Neglected to backpropagate this debug output fix from my training branch
|
hai 8 meses |
|
Alex Cheema
|
fb8933547a
Merge pull request #437 from blindcrone/unit-tests
|
hai 8 meses |
|
Nel Nibcord
|
03924cf9af
Need tokens. Also, for some reason this gets mad if we have non-integral tokens but this isn't a problem elsewhere?
|
hai 8 meses |
|
Alex Cheema
|
854a7c22ac
Merge pull request #436 from blindcrone/unit-tests
|
hai 8 meses |
|
Nel Nibcord
|
e463cd8196
Ok not sure we're using this but just in case
|
hai 8 meses |
|
Nel Nibcord
|
7e3ad9abc8
Missed a spot
|
hai 8 meses |
|
Alex Cheema
|
b8b4ea3633
Merge pull request #435 from blindcrone/unit-tests
|
hai 8 meses |
|
Nel Nibcord
|
1cd3efbe4c
Fixed unit tests
|
hai 8 meses |
|
Alex Cheema
|
b400a442ee
Merge pull request #420 from blindcrone/refactor-inference
|
hai 8 meses |
|
Nel Nibcord
|
65fdc99ccc
Call no longer needs request_id
|
hai 8 meses |
|
Nel Nibcord
|
90518a3bbe
Hoisted caching to a wrapper class
|
hai 8 meses |
|
Nel Nibcord
|
bf33ffde87
This doesn't need to be a tuple really
|
hai 8 meses |
|
Nel Nibcord
|
10e9f44a10
one-line output buffering
|
hai 8 meses |
|
Nel Nibcord
|
52ef6ee4a3
Made temperature and top_p available to the inference engine sample interfaces
|
hai 8 meses |
|
Nel Nibcord
|
8205a5aebc
Implemented per-request caching in tinygrad
|
hai 8 meses |
|
Nel Nibcord
|
13572e6a40
Some stability improvements for tinygrad inference
|
hai 8 meses |
|
Nel Nibcord
|
aefc0d7c51
I think this is more faithful to how it was originally done
|
hai 8 meses |
|
Nel Nibcord
|
c06b5f3b56
Corrected type annotations
|
hai 8 meses |
|
Nel Nibcord
|
9b66758b59
Make sure they're np arrays
|
hai 8 meses |
|
Nel Nibcord
|
b9d0fb6825
Since infer_prompt is a thin wrapper that works the same for all inference engines, we can de-abstract it
|
hai 8 meses |
|
Nel Nibcord
|
527c7a6e49
Applied new interface to tinygrad and dummy inference engines
|
hai 8 meses |
|
Nel Nibcord
|
52b91de817
Changed model classname due to the sharding being done elsewhere
|
hai 8 meses |
|
Nel Nibcord
|
34019e4608
Forgot an abstractmethod
|
hai 8 meses |
