소스 검색

undo diff

Rory Clear 8 달 전
부모
커밋
3c81845ab7
1개의 변경된 파일1개의 추가작업 그리고 0개의 파일을 삭제
  1. 1 0
      exo/inference/tinygrad/inference.py

+ 1 - 0
exo/inference/tinygrad/inference.py

@@ -51,6 +51,7 @@ def build_transformer(model_path: Path, shard: Shard, model_size="8B", device=No
     weights = load(str(model_path), shard)
   weights = convert_from_huggingface(weights, model, MODEL_PARAMS[model_size]["args"]["n_heads"], MODEL_PARAMS[model_size]["args"]["n_kv_heads"])
   weights = fix_bf16(weights)
+
   with Context(BEAM=0):
     # replace weights in model
     load_state_dict(model, weights, strict=False, consume=False)  # consume=True