openai-harmony/docs/rust.md

Rust API Reference

The Rust crate provides the core rendering and parsing logic. It is organised into a few modules which are re‑exported at the crate root for convenience.

Crate Setup

Add the crate to your Cargo.toml:

openai-harmony = { git = "https://github.com/openai/harmony" }

and import the items you need:

use openai_harmony::{load_harmony_encoding, HarmonyEncodingName};
use openai_harmony::chat::{Message, Role, Conversation};

Crate Layout

• chat – data structures representing messages and conversations.
• encoding – tokenisation and rendering implementation.
• registry – helper for loading predefined encodings.
• tiktoken_ext – extensions for tiktoken.

Key items are re‑exported so you can simply:

use openai_harmony::{HarmonyEncoding, HarmonyEncodingName, load_harmony_encoding};
use openai_harmony::chat::{Message, Role, Conversation};

chat module

Role

enum Role { User, Assistant, System, Developer, Tool }

Represents the author of a message.

Author

struct Author { role: Role, name: Option<String> }

Helper for identifying the message author. Author::new(role, name) creates a named author and Author::from(role) creates an anonymous one.

TextContent

Simple text payload. Implements Into<Content> so it can be passed directly to message constructors.

SystemContent and DeveloperContent

Structures used for the respective message types. They offer builder‑style methods (with_model_identity, with_instructions, with_tools, …) to configure the message payload.

Message

struct Message { author: Author, recipient: Option<String>, content: Vec<Content>, channel: Option<String>, content_type: Option<String> }

Convenience constructors mirror those exposed in Python (from_role_and_content, adding_content, etc.).

Conversation

struct Conversation { messages: Vec<Message> }

Created via Conversation::from_messages.

encoding module

HarmonyEncoding

Represents an encoding instance. Obtainable through load_harmony_encoding.

Important methods:

• name() – name of the encoding.
• tokenizer_name() – name of the underlying tokenizer.
• max_message_tokens() – maximum number of tokens a single message may use.
• render_conversation_for_completion(conversation, next_role, config) – convert a conversation into tokens ready for inference.
• render_conversation_for_training(conversation, config) – render a conversation for training data.
• render_conversation(conversation, config) – render a conversation without appending a new role.
• render(message) – render a single message into tokens.
• parse_messages_from_completion_tokens(tokens, role) – parse a list of tokens back into messages.
• stop_tokens() and stop_tokens_for_assistant_actions() – sets of stop tokens for sampling.

StreamableParser

Incremental parser that consumes tokens one by one. Create with StreamableParser::new(encoding, role) and feed tokens via process. Access information via getters like current_content, current_role, messages, tokens and state_json.

registry module

load_harmony_encoding

fn load_harmony_encoding(name: HarmonyEncodingName) -> Result<HarmonyEncoding>

Load a predefined encoding by name.

HarmonyEncodingName

enum HarmonyEncodingName { HarmonyGptOss }

Enum of the available encodings. It implements FromStr and Display.

Feature flags

If the python-binding feature is enabled, the crate exposes a Python module via pyo3 (see src/py_module.rs). This module is used by the accompanying Python package but can be ignored when using the crate purely from Rust.

Usage Examples

Below is a minimal program that builds a conversation, renders it using the HarmonyEncoding, and parses the tokens back into messages:

use openai_harmony::chat::{Conversation, Message, Role, SystemContent};
use openai_harmony::{load_harmony_encoding, HarmonyEncodingName};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create some messages
    let convo = Conversation::from_messages([
        Message::from_role_and_content(Role::System, SystemContent::new()),
        Message::from_role_and_content(Role::User, "What is 2 + 2?"),
    ]);

    // Render tokens for completion
    let enc = load_harmony_encoding(HarmonyEncodingName::HarmonyGptOss)?;
    let tokens = enc.render_conversation_for_completion(&convo, Role::Assistant, None)?;
    println!("{:?}", tokens);

    // Decode & parse back
    println!("{}", enc.decode_utf8(&tokens)?);
    let parsed = enc.parse_messages_from_completion_tokens(tokens, Some(Role::Assistant))?;
    for m in parsed { println!("{:?}", m); }
    Ok(())
}

Error Handling

Most functions return anyhow::Result<T> (a type alias for Result<T, anyhow::Error>). Errors may originate from loading encodings (LoadError), rendering problems (RenderFormattingTokenError) or parsing failures when the token stream is malformed.

Typical applications propagate errors with the ? operator:

let tokens = enc.render_conversation_for_completion(&convo, Role::Assistant, None)?;
let parsed = enc.parse_messages_from_completion_tokens(tokens, Some(Role::Assistant))?;

You can also match on the underlying error kinds via anyhow::Error if you need more specific handling.