Streaming (Optional)

LLM responses can take several seconds to complete. Streaming allows your UI to start rendering the moment the first piece of the C1 response is available, drastically reducing perceived latency and creating a much more responsive and engaging experience for the end-user.

Overview

Streaming involves your entire application stack. The C1 API sends the response in chunks, your backend forwards these chunks, and your UI progressively renders them as they arrive.

Backend: Enabling the Stream

To enable streaming, you must first set stream: true in your call to the C1 API. Your backend’s primary role is then to efficiently forward this stream to the UI with the correct Content-Type: text/event-stream header. Our server-side SDKs for Python and Node.js provide helpers to simplify this process.

Python (FastAPI)
Node.js (Next.js)

To simplify streaming in FastAPI, we provide the thesys-genui-sdk Python library. You can install it via pip: pip install thesys-genui-sdkThe library provides the @with_c1_response decorator, which automatically handles setting the correct response headers and creating a streaming context. Inside the decorated function, you can use the write_content helper to yield each chunk from the LLM stream.

For framework independent streaming, see the thesys-genui-sdk package on PyPI.

main.py

import os
from openai import OpenAI
from fastapi import FastAPI
from pydantic import BaseModel
from thesys_genui_sdk.fast_api import with_c1_response
from thesys_genui_sdk.context import write_content, get_assistant_message

# --- Setup ---
app = FastAPI()
client = OpenAI(
    api_key=os.environ.get("THESYS_API_KEY"),
    base_url="https://api.thesys.dev/v1/embed"
)

class ChatRequest(BaseModel):
    prompt: str

# --- API Endpoint ---
@app.post("/chat")
@with_c1_response()  # This decorator handles the streaming response
async def chat(request: ChatRequest):
    # 1. Enable streaming from the C1 API
    stream = client.chat.completions.create(
        model="c1-model-name",
        messages=[{"role": "user", "content": request.prompt}],
        stream=True,
    )

    # 2. Yield each chunk using the write_content helper
    for chunk in stream:
        content = chunk.choices.delta.content
        if content:
            await write_content(content)

    # 3. Store the assistant message in the database if required
    assistantMessage = get_assistant_message()
    # messageStore.addMessage(assistantMessage) # replace with your own database login

For Node.js, we provide helpers in the @thesysai/genui-sdk/server package. The makeC1Response function creates an object that manages the response stream.You can then use a utility like transformStream from @crayonai/stream to process the stream from the C1 API and pipe the content chunks into the response object using its writeContent method.

app/api/chat/route.ts

import { NextRequest, NextResponse } from "next/server";
import OpenAI from "openai";
import { transformStream } from "@crayonai/stream";
import { makeC1Response } from "@thesysai/genui-sdk/server";

// --- Setup ---
const client = new OpenAI({
  apiKey: process.env.THESYS_API_KEY,
  baseURL: 'https://api.thesys.dev/v1/embed',
});

// --- API Endpoint ---
export async function POST(req: NextRequest) {
  const { prompt } = (await req.json()) as { prompt: string };
  const c1Response = makeC1Response();

  // 1. Enable streaming from the C1 API
  const llmStream = await client.chat.completions.create({
    model: "c1-model-name",
    messages: [{ role: "user", content: prompt }],
    stream: true,
  });

  // 2. Pipe the LLM stream into our response stream
  transformStream(llmStream,
    (chunk) => {
      const contentDelta = chunk.choices.delta.content;
      if (contentDelta) {
        c1Response.writeContent(contentDelta);
      }
      return null; // We are not transforming, just observing
    },
    { onEnd: () => {
      const assistantMessage = c1Response.getAssistantMessage();
      // store the assistant message in the database if required
      // messageStore.addMessage(assistantMessage); // replace with your own database login

      c1Response.end();
    } }
  );

  // 3. Return the managed response stream
  return new NextResponse(c1Response.responseStream, {
    headers: { "Content-Type": "text/event-stream" },
  });
}

UI: Rendering the Stream

Handling a streaming response on the UI requires manually fetching the data, reading the stream chunk by chunk, and updating your component’s state as new data arrives. While this involves more code than a standard fetch request, it gives you full control over the user experience. This section breaks down a complete, working example.

Manual Stream Handling with `<C1Component>`

Here is a full React component that fetches a streaming C1 DSL response from a backend endpoint and renders it progressively.

app/page.tsx

import { useState } from "react";
import { C1Component, ThemeProvider } from "@thesysai/genui-sdk";
import "@crayonai/react-ui/styles/index.css";

function MyStreamingComponent() {
  const [c1Response, setC1Response] = useState<string>("");
  const [isLoading, setIsLoading] = useState<boolean>(false);

  const handleGenerate = async (prompt: string) => {
    setIsLoading(true);
    setC1Response(""); // Clear previous response

    try {
      const response = await fetch("/api/chat", {
        method: "POST",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify({ prompt }),
      });

      if (!response.body) {
        throw new Error("Response body is empty.");
      }

      const reader = response.body.getReader();
      const decoder = new TextDecoder();
      let accumulatedResponse = "";

      // Read the stream chunk by chunk
      while (true) {
        const { done, value } = await reader.read();
        if (done) break; // Exit loop when stream is finished

        const chunk = decoder.decode(value);
        accumulatedResponse += chunk;

        // Update state to re-render the component with new data
        setC1Response(accumulatedResponse);
      }
    } catch (error) {
      console.error("Error fetching or reading stream:", error);
    } finally {
      setIsLoading(false);
    }
  };

  return (
    <ThemeProvider>
      {/* Your application's input form */}
      <button onClick={() => handleGenerate("Show me sales data")}>
        Generate Report
      </button>

      {/* C1Component renders the streaming DSL */}
      <C1Component
        c1Response={c1Response}
        isStreaming={isLoading}
      />
    </ThemeProvider>
  );
}

Code Breakdown

Let’s walk through the key parts of the code above.

State Management: We use two state variables:
- c1Response: An accumulating string that holds the C1 DSL as it arrives from the stream. It starts empty.
- isLoading: A boolean to track the request status, which is passed to the isStreaming prop.
The Fetch Request: Inside the handleGenerate function, we initiate a standard fetch call to our streaming backend endpoint.
Reading the Stream: This is the core of the logic.
- We get a reader from the response.body.
- The while (true) loop continuously calls reader.read() to get the next chunk of data.
- A TextDecoder converts each raw data chunk into a string.
- We append this string to our accumulatedResponse variable and update the c1Response state with setC1Response(). This state update is what causes the UI to render progressively.
- The loop breaks when the stream sends a done: true signal.
Connecting to <C1Component>:
- The c1Response state variable is passed directly to the <C1Component>. As this state updates with each new chunk, the component re-renders to display the incoming UI.
- The isLoading state is passed to the isStreaming prop, which can be used by the component to display loading indicators.

All-in-One Solution: `<C1Chat>`

For conversational interfaces, <C1Chat> is the simplest solution. It has streaming enabled by default and encapsulates all the complex state and stream-handling logic shown above. As long as the apiUrl you provide points to a streaming backend endpoint, no further UI configuration is required.

app/page.tsx

import { C1Chat } from "@thesysai/genui-sdk";
import "@crayonai/react-ui/styles/index.css";

export default function App() {
  // This will stream automatically if /api/chat is a streaming endpoint
  return <C1Chat apiUrl="/api/chat" />;
}

For guides on using <C1Chat>, please refer to the Conversational UI section.

Getting Started

Basics

Interactivity

Styling

Conversational UI

Artifacts

Advanced

Ecosystem

Overview

Backend: Enabling the Stream

UI: Rendering the Stream

Manual Stream Handling with `<C1Component>`

Code Breakdown

All-in-One Solution: `<C1Chat>`

​Overview

​Backend: Enabling the Stream

​UI: Rendering the Stream

​Manual Stream Handling with <C1Component>

​Code Breakdown

​All-in-One Solution: <C1Chat>

Overview

Backend: Enabling the Stream

UI: Rendering the Stream

Manual Stream Handling with `<C1Component>`

Code Breakdown

All-in-One Solution: `<C1Chat>`