Function supports consuming the partial results of a prediction as they are made available by the predictor.

​
Consuming a Prediction Stream

Use the fxn.predictions.stream function to consume a prediction stream:

// Create a prediction stream
const stream = await fxn.predictions.stream({
    tag: "@text-co/split-sentence",
    inputs: { text: "Hello world" }
});

// Consume the stream
for await (const prediction of stream) {
    console.log(prediction.results[0]);
}

You can use prediction streaming to implement text generation user interfaces for LLMs.

​
Creating vs. Streaming Predictions

Streaming in Function is designed to be highly intuitive to use. We fully separate how a prediction function is implemented (i.e. eager vs generator functions) from how the function might be consumed (i.e. creating vs. streaming). Consider these two predictors:

def predict ():
    return "hello from Function"

You can choose how to consume a prediction function depending on what works best for your user experience.

Here are the reuslts of creating vs. streaming each function at runtime: