使用JanHQ/Cortex获取结构化输出的技术指南-CSDN博客

本文链接：https://blog.csdn.net/gitblog_00185/article/details/148783799

使用JanHQ/Cortex获取结构化输出的技术指南

前言

在现代人工智能应用中，从语言模型获取结构化输出是一个常见需求。JanHQ/Cortex项目提供了一种高效的方式，让开发者能够从本地运行的开源语言模型（如Llama和Mistral）中获取结构化的JSON输出。本文将详细介绍如何使用这一功能，并通过实际案例展示其强大之处。

准备工作

启动模型

首先需要启动目标语言模型。以Llama3.1 8B模型为例：

cortex run -d llama3.1:8b-gguf-q4-km

启动成功后，你将看到确认信息，表明模型已准备就绪。

基础示例：日历事件提取

定义数据结构

我们使用Pydantic来定义期望的数据结构。Pydantic是一个Python库，用于数据验证和设置管理。

from pydantic import BaseModel
from openai import OpenAI
import json

client = OpenAI(
    base_url="http://localhost:39281/v1",
    api_key="not-needed"
)

class CalendarEvent(BaseModel):
    name: str
    date: str
    participants: list[str]

发送请求并解析结果

completion = client.beta.chat.completions.parse(
    model="llama3.1:8b-gguf-q4-km",
    messages=[
        {"role": "system", "content": "Extract the event info as JSON"},
        {"role": "user", "content": "Alice and Bob are going to a science fair on Friday"}
    ],
    response_format=CalendarEvent,
    stop=["<|eot_id|>"]
)
print(json.dumps(completion.choices[0].message.parsed.dict(), indent=2))

输出结果

{
  "name": "science fair",
  "date": "Friday",
  "participants": ["Alice", "Bob"]
}

这个例子展示了如何从自然语言文本中提取结构化的事件信息，包括事件名称、日期和参与者。

进阶示例：数学问题求解步骤

定义复杂数据结构

对于更复杂的场景，我们可以定义嵌套的数据结构：

class Step(BaseModel):
    explanation: str
    output: str

class MathReasoning(BaseModel):
    steps: list[Step]
    final_answer: str

发送数学问题请求

response = client.beta.chat.completions.parse(
    model="llama3.1:8b-gguf-q4-km",
    messages=[
        {
            "role": "system",
            "content": "Solve this math problem step by step. Output as JSON."
        },
        {
            "role": "user",
            "content": "how can I solve in a lot of detail, the equation 8x + 7 = -23"
        }
    ],
    response_format=MathReasoning,
    stop=["<|eot_id|>"]
)
print(json.dumps(response.choices[0].message.parsed.model_dump(), indent=2))

输出结果

{
  "steps": [
    {
      "explanation": "The given equation is 8x + 7 = -23. To isolate x, we need to get rid of the constant term (+7) on the left side.",
      "output": ""
    },
    {
      "explanation": "We can subtract 7 from both sides of the equation to get: 8x = -30",
      "output": "8x = -30"
    },
    {
      "explanation": "Now, we need to isolate x. To do this, we'll divide both sides of the equation by 8.",
      "output": ""
    },
    {
      "explanation": "Dividing both sides by 8 gives us: x = -3.75",
      "output": "x = -3.75"
    },
    {
      "explanation": "However, looking back at the original problem, we see that it's asking for the value of x in the equation 8x + 7 = -23.",
      "output": ""
    },
    {
      "explanation": "We can simplify this further by converting the decimal to a fraction.",
      "output": ""
    },
    {
      "explanation": "The decimal -3.75 is equivalent to -15/4. Therefore, x = -15/4",
      "output": "x = -15/4"
    }
  ],
  "final_answer": "x = -3"
}

这个例子展示了如何获取复杂的数学解题步骤，包括每一步的解释和中间结果。

简单列表输出

对于只需要简单列表的场景，可以使用更简洁的方式：

completion = client.chat.completions.create(
    model="llama3.1:8b-gguf-q4-km",
    messages=[
        {
            "role": "system",
            "content": "List songs in {song_name: release_year} format"
        },
        {
            "role": "user",
            "content": "List 10 songs"
        }
    ],
    response_format={"type": "json_object"},
    stop=["<|eot_id|>"]
)
print(json.dumps(json.loads(completion.choices[0].message.content), indent=2))

输出结果

{
  "Hotel California": 1976,
  "Stairway to Heaven": 1971,
  "Bohemian Rhapsody": 1975,
  "Smells Like Teen Spirit": 1991,
  "Viva la Vida": 2008,
  "Imagine": 1971,
  "Hotel Yorba": 2001,
  "Mr. Brightside": 2004,
  "Sweet Child O Mine": 1987,
  "Livin on a Prayer": 1986
}