Calls a LLM and streams its response
reasonStream()
literally is reason()
but with a few added bonus, it is necessary to understand reason()
first.reasonStream()
is an async function that calls a LLM and streams back to your app its response. Everything you can do with reason()
you can also do with reasonStream()
.
`reasonStream` response
reasonStream()
reasonStream()
in our POST()
function, however reasonStream()
is an async generator — which means there is much more we can do with it othen than directly return it.
For instance, let’s say we needed to include a picture of the country’s flag in the response. How could we do this?
Well, we could ask the LLM for an URL that contains a picture of the country’s flag but this is not ideal as the URL might be broken, might have expired, etc.
Another way would be to wait for the response from the LLM (that contains the country
property) and then search the web for a picture of the country. This seems like a good approach.
To do this, we need to introduce a few concepts first.
reasonStream()
is a generator
, we need to fully understand what they are and how they work.
Although this is not a feature from RΞASON itself but a native JavaScript feature, it is not widely used so most people are not familiar with them.
async
:
*
after the function
keyword;for (const yieldedValue of generator()) { }
;async generator
you need to add an await
in for await (const yieldedValue of generator()) { }
.reasonStream()
.
reasonStream()
reasonStream()
is an async generator
, you can use all JS generators features. Such as:
description
property was filled overtime.
Import to note that while we specified in our City
interface a single description
property that is a string, reasonStream
returned a object that has done
& value
. Why?
StreamableObjects
reasonStream()
yields are StreamableObjects
.
A StreamableObject
is just a wrapper for your value that has a done: boolean
property to indicate whether the LLM has fully finished processing that particular property. This is to help developers know what state a certain property is while being streamed.
A StreamableObject
has three different states:
When the LLM has not even started returning the value
StreamableObject
will be:When the LLM has started returning the value but not finished
StreamableObject
will be:When the LLM has finished returning the value
StreamableObject
will be:A note on incomplete values
rating
property will be filled like this:reasonStream()
yields like this is because in almost all scenarios that you need to access intermediate values, you also need to know when a certain value has been fully returned from the LLM or not.StreamableObject
in nested propertiesStreamableObjects
. Even nested properties, such as array elements & object properties.
For instance, given the following interface:
StreamableObject
will be:
StreamableObjects
is because to iterate through reasonStream()
you’ll to handle them.
In the example we lay out previously, we had this code:
country
property) and then searching the web for a picture of the country.
Let’s do it:
test
entrypoint in the RΞASON Playground:
`reasonStream` response
StreamableObjects
to the clientreasonStream()
yields StreamableObjects
, the response that was streamed from RΞASON to the client (the Playground in this case) is a regular object and not a StreamableObject
.
Why?
Because all StreamableObjects
yielded from your entrypoints are unwrapped to their original form. This is done because almost never you actually want to return StreamableObjects
to your client.
StreamableObject
country_picture
:
StreamableObject
that reasonStream()
yields to your needs. This can be useful when you want to add some extra information to the object that is not returned from the LLM: add a new property, modify an existing one, etc.
StreamableObjects
even with complex interfaces — such as nested objects inside of arrays.If you don’t feel like going through that now, feel free to skip it.City
interface consists of a complex object that has the points_of_interest
property being an array of objects. Here’s what this entrypoint outputs:
getDistance()
).Output from `POST /hello` with the distance property
StreamableObjects
value
property of the StreamableObject
— the done
property is read-only. For instance, in point_of_interest.value.distance = ...
we are adding a new property to the value
property of the point_of_interest
object.
If we did point_of_interest.distance = ...
, it would not work as we would be trying to modify the StremableObject
itself and its value
.
reasonStream()
in your code. Although there is more to learn about reasonStream()
, this page should be enough to get you started.
Next, we’ll be going in-depth about the concept of agents in RΞASON.