​

What is observability?

Observability refers to the ability to monitor & understand the internals of a software system — it’s crucial for detecting & diagnosing issues.

In the context of LLM app development, observability is even more relevant. That’s because LLMs part of building a great LLM app is, at least, 80% trying new prompts, new RAG pipelines, different agent behaviours, etc.

RΞASON follows the OpenTelemetry standard. The OpenTelemetry is a open standard for obversability that prevents vendor lock-in and interops with a bunch of different tools — allowing you to choose the best tool for you.

​

Why OpenTelemetry is a big deal?

OpenTelemetry is a standard in the industry, having your app data in it, means you can use any tool you want to monitor your app. You are not locked-in into whatever tool RΞASON supports or into depending that RΞASON integrates with a provider you want to use.

If you already have data pipelines configured, you don’t need to change a single thing. Your RΞASON app data will automatically work there too.

​

Using it with RΞASON

There are a bunch of tools that integrate with OpenTelemetry data — both free & paid. You can use whichever you prefer. However, by default RΞASON uses Zipkin: a free open-source tracing tool developed by Twitter.

You just need to start a Zipkin instance to access your observability data — don’t need to change a thing in RΞASON. And the easies way to start Zipkin is through Docker:

docker run -d -p 9411:9411 openzipkin/zipkin:2.23

You can now access Zipkin at http://localhost:9411/zipkin/:

Zipkin dashboard

Let’s make a new request to our MathAgent we created in the Agents page using the RΞASON Playground:

MathAgent responding

Now if hit the RUN QUERY button in Zipkin:

Zipkin dashboard with a trace

Then clicking SHOW will open the trace:

A Zipkin trace

A lot is going on in the screenshot above, it shows the what happens during the call to POST /math — it is an icicle graph.

It shows the time each thing took during your entrypoint execution. It also shows the inputs & outputs.

You can click in any of the spans (the rectangles in the graph) to open more information about it. Let’s open the multiply action:

Multiplity action span

We are able to see the inputs that the LLM sent to the function & also the output the function returned.

​

Using other OpenTelemetry providers

In the previous example you saw how to use Zipkin with RΞASON, however since RΞASON supports the OpenTelemetry standard, you can use many other tools if you prefer.

To do that, you just need to add a new openTelemetryExporter in .reason.config.js. Let’s say you want to use Jaeger:

import { JaegerExporter } from '@opentelemetry/exporter-jaeger';

// Jaeger options
const options = {
  host: 'localhost',
  port: 6832,
  maxPacketSize: 65000
}

/** @type {import('tryreason').ReasonConfig} */
const config = {
  projectName: 'my-reason-app',

  openai: {
    key: 'redacted',
    defaultModel: 'gpt-3.5-turbo'
  },

  openTelemetryExporter: new JaegerExporter(options),
}

export default config