Ver Fonte

[ML] Newline delimit PyTorch inference commands (#80408)

Currently PyTorch inference commands are sent as a series
of JSON documents back-to-back with no delimiter in between.
This conflicts with a change to the way
rapidjson::kParseStopWhenDoneFlag works in the newer version
of RapidJSON that we want to upgrade to on the C++ side.
The newer version requires either a character or end-of-file
to follow a JSON document before it considers parsing it to
be complete. Since we don't want to end the stream, we have
to send another character. To minimise data transfer we could
send the opening brace of the next document, not knowing what
the next document will contain yet. But it's nicer to send a
newline character and let the next document be sent in full
when it's ready. Even with the old version of RapidJSON
sending a newline after each command should not cause a
problem, as trailing whitespace is acceptable in JSON.

Relates elastic/ml-cpp#2106
David Roberts há 4 anos atrás
pai
commit
b4d671a6b7

+ 1 - 0
x-pack/plugin/ml/src/main/java/org/elasticsearch/xpack/ml/inference/pytorch/process/NativePyTorchProcess.java

@@ -65,6 +65,7 @@ public class NativePyTorchProcess extends AbstractNativeProcess {
 
     public void writeInferenceRequest(BytesReference jsonRequest) throws IOException {
         processInStream().write(jsonRequest.array(), jsonRequest.arrayOffset(), jsonRequest.length());
+        processInStream().write('\n');
         processInStream().flush();
     }
 }