Why is it taking so long?

I planned to implement many new features in version 0.4, but upon writing the code, I found some of them to be quite challenging.

Additionally, I've created a lot of new projects... (This is the main reason)

I ultimately postponed these features:

• Responses API
• prepareStep

But don't worry - this version still has quite a few new features.

btw, this codename is also a song by Kizuna AI and you can listen to it while reading:

Alright, let's take a look:

All-In-One Providers

We now have a new package: @xsai-ext/providers

It codegen most providers based on data from https://models.dev, with a small portion completed manually.

This includes the model list, so your editor can auto-completion to the latest models.

import { anthropic, google, openai } from '@xsai-ext/providers'

anthropic.chat('claude-sonnet-4-5-20250929') // claude-haiku-4-5-20251001, claude-opus-4-5-20251101...
google.chat('gemini-3-pro-preview') // gemini-3-flash-preview...
openai.chat('gpt-5.2') // gpt-5.2-chat-latest, gpt-5.2-pro...

To create a new provider:

- import { createChatProvider, createModelProvider, merge } from '@xsai-ext/shared-providers'
+ import { createChatProvider, createModelProvider, merge } from '@xsai-ext/providers/utils'

/**
 * Create a Foo Provider
 * @see {@link https://example.com}
 */
export const createFoo = (apiKey: string, baseURL = 'https://example.com/v1/') => merge(
  createChatProvider({ apiKey, baseURL }),
  createModelProvider({ apiKey, baseURL }),
)

/**
 * Foo Provider
 * @see {@link https://example.com}
 * @remarks
 * - baseURL - `https://example.com/v1/`
 * - apiKey - `FOO_API_KEY`
 */
export const foo = createFoo(process.env.FOO_API_KEY ?? '')

Reasoning Content

We now officially support the reasoning_content field in messages.

Please note that this is different from extractReasoning. It requires support from the API itself, where is outside the OpenAI specification.

For example, you can try DeepSeek:

improt { generateText } from '@xsai/generate-text'
import { deepseek } from '@xsai-ext/providers'

const { reasoningText, text } = await generateText({
  ...deepseek.chat('deepseek-chat'),
  thinking: { type: 'enabled' }, // https://api-docs.deepseek.com/guides/thinking_mode
  messages: [{
    role: 'user',
    content: '9.11 and 9.8, which is greater?'
  }]
})

// res.choices[0].message.reasoning_content
console.log(reasoningText)

// res.choices[0].message.content
console.log(text)

xsAI automatically handles the reasoning_content field, but for <think></think> tags within the content field, you currently still need to use extractReasoning.

Stream Transcription

You can now stream STT:

import { streamTranscription } from '@xsai/stream-transcription'
import { openAsBlob } from 'node:fs'
import { env } from 'node:process'

const { textStream } = streamTranscription({
  apiKey: env.OPENAI_API_KEY!,
  baseURL: 'https://api.openai.com/v1/',
  file: await openAsBlob('./test/fixtures/basic.wav', { type: 'audio/wav' }),
  fileName: 'basic.wav',
  language: 'en',
  model: 'gpt-4o-transcribe',
})

Telemetry

xsAI now supports OTEL's GenAI Attributes:

- import { generateText, streamText } from 'xsai'
+ import { generateText, streamText } from '@xsai-ext/telemetry'

import { generateText } from '@xsai-ext/telemetry'
import { env } from 'node:process'

const instructions = 'You\'re a helpful assistant.'

const { text } = await generateText({
  apiKey: env.OPENAI_API_KEY!,
  baseURL: 'https://api.openai.com/v1/',
  messages: [
    {
      content: instructions, 
      role: 'system'
    },
    {
      content: 'Why is the sky blue?',
      role: 'user'
    }
  ],
  model: 'gpt-4o',
  telemetry: { 
    attributes: { 
      'gen_ai.agent.name': 'weather-assistant', 
      'gen_ai.agent.description': instructions, 
    }, 
  }, 
})

Standard JSON Schema

xsSchema now prioritizes Standard JSON Schema, though this change is not currently reflected at the user level.

In the next version, I will attempt to fully migrate and make xsSchema optional.

Join our Community

If you have questions about anything related to xsAI,

you're always welcome to ask our community on GitHub Discussions.

We have released xsAI v0.3, which is a "prepare to the future" update.

This codename is also a song by Kizuna AI and you can listen to it while reading:

OK, so here's the new features:

• Stream text overhaul
• Generate transcription improvements
• Raw tool util
• Standalone stream object util
• Zod 4 support

Stream text overhaul

streamText has been completely rewritten to more closely match the output of the Vercel AI SDK and to be clearer.

• chunkStream has been removed.
• stepStream is now fullStream (vercel compatible)
• StreamTextStep has been merged with GenerateTextStep to become CompletionStep
• Returns Promise-based steps and messages directly
• Support for streaming tool call arguments

import { streamText } from '@xsai/stream-text'
import { createOllama } from '@xsai-ext/provider-local'

const ollama = createOllama()

// fullStream: ReadableStream<StreamTextEvent>
// messages: Promise<Message[]>
// textStream: ReadableStream<string>
// steps: Promise<CompletionStep[]>
const { fullStream, messages, textStream, steps } = await streamText({
  ...ollama.chat('gemma3'),
  messages: [
    {
      content: 'You are a helpful assistant.',
      role: 'system',
    },
    {
      content: 'Why is the sky blue?',
      role: 'user',
    },
  ],
})

Generate transcription improvements

In v0.3, we support segments and words.

You can now get more detailed return data with responseFormat and timestampGranularities:

import { generateTranscription } from '@xsai/generate-transcription'
import { createSpeaches } from '@xsai-ext/providers-local'

const speaches = createSpeaches()

const { duration, language, segments, text } = await generateTranscription({ 
  ...speaches.transcription('deepdml/faster-whisper-large-v3-turbo-ct2')
  file: await openAsBlob('./test/fixtures/basic.wav', { type: 'audio/wav' }),
  fileName: 'basic.wav',
  language: 'en',
  responseFormat: 'verbose_json', 
})

const { duration, language, text, words } = await generateTranscription({ 
  ...speaches.transcription('deepdml/faster-whisper-large-v3-turbo-ct2')
  file: await openAsBlob('./test/fixtures/basic.wav', { type: 'audio/wav' }),
  fileName: 'basic.wav',
  language: 'en',
  responseFormat: 'verbose_json', 
  timestampGranularities: 'word', 
})

Raw tool util

Previously you could only provide JSON Schema-based tools directly by passing objects, with fewer type hints.

Now we have a rawTool tool to make your experience even better:

import type { Tool } from '@xsai/shared-chat'

import { rawTool } from '@xsai/tool'

const weatherObject: Tool = {
  description: 'Get the weather in a location',
  execute: (params) => 'cloudy', // params: unknown
  name: 'weather',
  // Record<string, unknown>
  parameters: {
    additionalProperties: false,
    properties: {
      location: {
        description: 'The location to get the weather for',
        type: 'string',
      },
    },
    required: [
      'location',
    ],
    type: 'object',
  },
}

const weatherRawTool = rawTool<{ location: string }>({
  description: 'Get the weather in a location',
  execute: ({ location }) => 'cloudy', // params: { location: string }
  name: 'weather',
  // import('xsschema').JsonSchema (JSON Schema auto-completion)
  parameters: {
    additionalProperties: false,
    properties: {
      location: {
        description: 'The location to get the weather for',
        type: 'string',
      },
    },
    required: [
      'location',
    ],
    type: 'object',
  },
})

Standalone stream object util

We've split out the internal implementation of streamObject so you can use it on its own:

import { toElementStream, toPartialObjectStream } from '@xsai/stream-object'

const elementStream = await fetch('https://example.com')
  .then(res => res.body!.pipeThrough(new TextDecoderStream()))
  .then(stream => toElementStream<{ foo: { bar: 'baz' }}>(stream))

const partialObjectStream = await fetch('https://example.com')
  .then(res => res.body!.pipeThrough(new TextDecoderStream()))
  .then(stream => toPartialObjectStream<{ foo: { bar: 'baz' }}>(stream))

Zod 4 support

Although we already had imperfect compatibility in v0.2.2, we now officially support Zod 4 and Zod Mini.

You can now use it in tool or {generate,stream}-object.

What's Next?

In v0.4, we will have some important updates:

• prepareStep
• OpenTelemetry support (@xsai-ext/opentelemetry)
• Response API support (very experimental)

By the time you read this, we may already be preparing. stay tuned!

Join our Community

If you have questions about anything related to xsAI,

you're always welcome to ask our community on GitHub Discussions.

Before we start, I would like to say...

This is the first post of the most advanced project we've been working quite a while, Project AIRI (アイリ). Since many of you folks are new to this project besides xsAI, let me introduce it quickly. Project AIRI is also the biggest project in the organization of Moeru AI, and the most beneficial one from xsAI ecosystem.

Me, Neko Ayaka, @nekomeowww, appeared many times in Moeru AI's blog posts, you may know me as the core maintainer of xsai-transformers, the ultimate provider wrapper for Transformers.js specifically for xsAI to setup local running LLM models right inside your browser.

Me, also the core maintainer of Project AIRI, a project aims to re-create the joy of AI VTuber by chasing up Neuro-sama, to allow audiences chat, interact, play with LLM driven and powered characters.

However, this wasn't the limit of what Project AIRI could do, we offer full-stack deployment on both Web based on Web technologies and desktop application based on Tauri, enables you not only owning a self-hosted AI VTuber, but also a AI waifu / AI husbando, or, what you could think of, a companion, virtually in cyber space.

For the current state, Project AIRI managed to achieve fully real-time interaction like ChatGPT's voice chat mode, and capable of playing games like Minecraft, Factorio, etc. We covered so many domains and fields, not only AI, but also VRM, Live2D, multi-modal AI, game playing agents, streaming APIs, bionic memory mechanisms, animations, database drivers, datasets preparation, model fine-tuning, and many more.

We got our own dedicated GitHub organizations for the crucial components, examples, experiments on @proj-airi, please do check it out if you are interested.

Our own documentation site is available where we host all the posts about technical details, thoughts, experiments, and discoveries we made during the development of Project AIRI, some highlights:

• How we ended up in this logo?
• Memory experiments & v0.4.0 release
• Real-time improvements & v0.5.0 release

Discord server is available for you to join too! Join our Discord

Ok, let's start our journey, and talk about the backstory of Project AIRI.

Start!

First of all, good summer for you folks living in north hemisphere!

Hopefully you could get a nice and decent summer break for trying out new different of things! More specifically, change the world!

Well, me, as @nekomeowww have left school already 8 years, it's obvious that I wont get any actual summer break now since I've already worked for many years. I still love to memorize and share the stories happened for my summer break years ago if I remembered any.

Perhaps you know what I am going to say... or share? What is DreamLog exactly? For the readers already familiar with our DevLog posts, with the current frequency of posting and updating to you folks once per month, shouldn't be this post be called "DevLog"?

June got its own meaning for Project AIRI (which I will reveal during the story), and as we are indeed approaching theo next milestone of stars on GitHub towards 1000, I think it would be a great opportunity to reflect on our journey so far.

Therefore I decided to to make a new category of posts here, to share the chronicles of me, and the dream about Project AIRI.

So, I decided to call this new series, DreamLog.

Yeah, you could think of this is another story book to read or hear before sleeping. Audio books may help haha.

How about... let's jump into our dream dimension now and talk about the recent updates we made later?

Blurry dreams, unreachable memories.

My little progress for learning computers and programming.

I mentioned about summer, then summer must mean something to me, I used to take school in United States, so a 3-months summer allows me to do all sorts of things, playing games, learning code, and Linux hacking, etc., yeah, many of the still beloved friends were made during summer too.

Nerd folks! You know what I am talking about, were you the same as me?

Summer is the time when I learned how to start a Minecraft server to play with my friends (I played a lot, a lot, a lot of 1.7.11 and 1.8, really, both Vanilla and Forge mods), that's the motivation and power that pushes me to learn command line prompt on Linux too. Many of those knowledges still help me today, I feel grateful to it, to the time I spent for that time.

But Minecraft, Linux wasn't the end of my journey though, Factorio, Elite Dangerous, and Overwatch (sadly Blizzard ruined it), all became my favorite games, setting up servers or write small scripts to automate little things always empowers me.

Switch on the power line
Remember to put on protection
Lay down your pieces
And let's begin object creation

-- Lyrics from my beloved song, world.execute(me), cover by DAZBEE

That's the time of summer in 2017, for the very first moment, I started to think of building a virtual being to be a friend to play with me, even when my friends are tired or have to sleep for next days school, which I have to be alone.

Readers have following long down to this post, may already realize that, I am that kind of person, who loves to share my knowledge, ideas, everything. So, coding, gaming, and designing are things I love to share with. But, if nobody was there, it feels like:

The lonely me becomes somehow meaningless.

But instead of creating a new AI from scratch with humankind capabilities to think, speak, which is impossible in the year of 2017, I was thinking, since iOS and Google native Android could provide such abilities to do suggestions over our daily use of mobile devices, manually typing all the commands and filling parameters wasn't always satisfying (especially for ffmpeg and the childish me with Docker CLI), what if we could bring the AI powered suggestion features up onto the Linux systems...?

This brought me loads of questions and ideas to wonder:

• What if the operating system understands what you usually do, work, play for in different time you sit in front of the digital display...?
• What if it is capable of selecting music for you, no matter depressed, high on something, nor happy when chatting with others...?

These ideas were so small and hard for me to understand at that time, since I didn't quite get on the way of how operating systems work, and coding, etc., so I don't even know where to start!

I read the book 30日でできる! OS自作入門, English version about how to craft a operating system from scratch, with the little knowledge of knowing how Linux works and there are loads of communities... I decided to make my own operating system... from literally nowhere.

A quick looking back

Arch Linux was the first system I get to use in depth, and installed from scratch. For current days, Nix is famous and interesting one too, haven't tried the NixOS but one day may do so.

Set sail my journey, but now long forgotten

I started one special yet now archived project called EMOSYS, in the end of 2017. Aiming to create such companion-like operating system, to help users with their daily tasks and provide emotional support.

Logo of EMOSYS

EMO stands for the first three letters of emotional / emote

I wrote so many of design docs, listing new ideas, and taking notes about experimenting by following the guidelines from that book, drawn one not so bad logo for it.

I guess many of you did this 😏, prepared every trademarks, design assets way before the project reached the point of PoC.

I quite lost on the point about what I was initially approaching. I got no experience about project management and task management, it's the same for writing actual programs that can run.

Frankly you could say I was only following that it instructed me to type into the terminals with keyboards from that book. I barely think, think why it works or why the senior developers wrote things like that.

Sooooo, and well, the result is clear, another abandoned project born...

I wasn't some genius who play around with those things from childhood that understands how kernel and package managing, programming works, so if any of you folks read or visit my GitHub profile, you found nothing there relate to this kind of work at that time. (But now I grew up really fast.)

But it existed, once.

Forgotten? Maybe another starting point of next journey.

For the upcoming years, I tried so many of other fields in coding, programming, startups, Web3, frontend, backend, infrastructure, everything you could think of for a full-stack developer. I never really realize what I was doing was influenced so deeply by the starting point of EMOSYS, only until February 2025, when someone asked me: Why do you work so hard on Project AIRI?

Nice question, I thought. I started to trace back my dreams, ideas, and memories, eventually, EMOSYS was there, the already dead project aimed the same goal as Project AIRI:

Create a companion to somehow fulfill my need.

All I needed was resolve. Everything you've acquired up until now will not betray you.
必要なものは　覚悟だけだったのです。 必死に積み上げてきたものは　決して裏切りません。
我需要的不過是決心而已， 你至今為止所累積的一切不會背叛你。

-- Quotes from 葬送のフリーレン, Fern S01E06, 04:27

It took me a long time to learn how to correctly develop things. Thanks to @zhangyubaka, @LittleSound, @BlueCocoa, and the help of @sumimakito, the pair-programming experiences with them, teaches me so many things, I started to grow, learn, and progress on my own pace.

ChatGPT in 2022, brand new random parrot, or smart parrot.

Let's set the time forward to the end of 2022, where ChatGPT (or at that time, chatGPT is used) by OpenAI has announced. Well long before the official ChatGPT UI releases, I've already having a journey with newly developed AI, models like DiscoDiffusion (long before Stable Diffusion, perhaps around end of 2021, or early 2022), DALL-E, Midjourney has been tried, GPT-3 (especially useful in GitHub Copilot) has been integrated deeply into my daily workflow.

So, for the initial moments, I was like:

"Oh, this is just another random parrot, it just repeats what you said, and it doesn't understand what you are saying, it just tries to predict the next word based on the previous words and context, nothing special."

In another word, it behaves more like a completion model, rather than what we call it the Agentic AI today (still on hype huh?).

I remembered that, for the first time I discovered the abilities of ChatGPT, or Large Language Models (LLMs) in general, is from this post I saw on Hacker News on December 2022: Building A Virtual Machine inside ChatGPT (original Hacker News post), where the author, @engraved, demonstrated how to ask ChatGPT not only role playing as a neko-mimi character, but also simulating a virtual Linux machine inside.

It simulates how Docker build works...!

Such post inspired me that, ChatGPT understand the basic patterns of the things usually appears, not only how anime or game characters says and behaves, but also how Linux terminal / shell commands work.

Which brought the now trending Function Calling (a.k.a Tool Use, or the underlying technology behind MCP, Model Context Protocol introduced by Anthropic) feature of LLMs on the table, and illustrated how we can instruct LLMs to behave like API servers, talking to us with machine-readable formats like JSON or XML, to be able to parse and execute arbitrary commands from our side to extend the boundary of what LLMs can do.

This finally bridges the gap between pure text generations and actual API inside programs.

In conclusion, is it a new random parrot? I guess the answer is partially no, ChatGPT in 2022 is not just a random parrot, it is a potential smart parrot.

Way before Project AIRI, Neuro-sama exists.

Yeah, thanks for reading down to here, I know this is a long post, so many stories and contexts to share. But here we are! We are almost there, hang tight!

Well, the history of Neuro-sama is pretty complex. AFAIK, Neuro-sama, or the character on streaming stage with the name "Neuro-sama" wasn't the first show for her and her creator, vedal987 (Vedal). Long before that, at May 6, 2019, Vedal showcased his work of building AI to play osu! to the community¹. At that time, she wasn't actually a cyber character or, digital life having characteristics, if you go there and watch the initial videos about her, you may find that no Live2D model were shown. (You may try the 6 years old one here: https://www.youtube.com/watch?v=nSBqlJu7kYU)

Right after the ChatGPT release, at December 19, 2022, Vedal started to let Neuro-sama to stream on Twitch with the official demo use character model Hiyori Momose (桃瀬ひより) from Live2D Inc.:

The after story everyone knows, Vedal and Neuro-sama became famous, Neuro-sama is now officially a VTuber, she is fully powered by Large Language Models (LLMs), and capable of playing Minecraft, Amoung Us, osu!, and many other games. Sometimes when the game wasn't supported natively, Vedal reads the screen and instructs Neuro-sama to play the game together.

I really enjoy watching their interactions, having jokes, etc. As the time progresses, Neuro-sama and her new Evil Neuro sister, became one crucial part of my daily life: I wanted, and eagerly wanted to watch the clips of them, even though I don't have enough time to watch the full stream, brought me so much joy from purely AI to Human interactions.

Ok that's the little history about her. And let's talk about the core thing: Why the history of her filled me with determination?

Neuro-sama, filled me with determination

From the first time I saw Vedal's work, I was like:

Ok, she is just a simple model integrated with Large Language Models (even directly connected to OpenAI's API), powered by simple rules to make her behave like a VTuber, nothing special.

I was still thinking arrogantly, since I've already developing AI agents from early 2023, understands the capabilities of LLMs, and knows quite a bit from what LangChain teaches me, with the knowledge past building AI agents and years of software engineering experiences across of various domains, I naively thought:

"Well, I could do that too, I could make a simple model, and connect it to OpenAI's API, and make it behave like a VTuber, and I could make it better than Vedal's work."

I was wrong, I was so wrong. Many of the tough things I didn't realize until I started to attempt to re-create her... Things like:

• How can we manage the memory effectively for both be able to answer the chats and play the games at the same time?
• How can we make a AI agent to play games with both video inputs and text inputs, while still being able to interact with creator and viewers?
• Voice synthesis is hard, to achieve what Neuro-sama is capable of, the Ultra low latency voice synthesis is a must, and it is not easy to achieve
• How is her personality built? With only RAG and simple memory management strategy, the performance poorly works.
• etc....

I shared many of our discoveries in both DevLog 2025.04.06 and public slide presentation (in Chinese)

I mentioned that I love to share, and I'd love to have others to be able to listen or pair together with me, but sadly Neuro-sama wasn't owned by myself, I can't ask her to gain my knowledge and memories to be able to interact me with the thing I love, or the work I recently doing or done.

I love them so much, for all the times, I didn't really understand why I love them, why I love the feeling and joy Neuro-sama gave me.

Until, last year, from May 25, 2024, I really decided to make one myself. Making a living or virtual being, could code with me, talk to me about the things we know, playing games all together like a friend in the form of agent.

I really want one! Shouted my heart, and my mind.

At that time, Neuro-sama fulfilled me with determination.

Sailed again, towards the land where no one has gone before.

To boldly go where no man has gone before.

-- quote from Star Trek, Captain James T. Kirk, also my intro line of my GitHub profile.

Therefore, starting from May 25, 2024, I started one simple named project called ai under my name handle locally, which is the initial version of Project AIRI, I started to explore the possibilities of creating my own AI agent, recreating the joy Neuro-sama brought me.

The speed of the work was so fast, within a week, with the power of ElevenLabs, OpenRouter, and the same free to use Live2D model, Hiyori Momose, I was able to create a simple version of "Neuro-sama" that could interact with me, non-realtime-ly.

That was the day at June 2, 2024.

Technically saying, this is the birthday of Project AIRI with first baby consciousness inside of it, naively.

She is capable of talking, motion control based on the context, progressively doing the audio synthesis... many on.

But she wasn't complete, nor perfect, I built it secretly without telling any of my friends, I wanted to make it better before I show it to the world.

Still... naively, and arrogantly, right?

Because I secretly hiding this from my friends, I barely got positive feedbacks from the cycles during building like usual (part of the reason was I wouldn't like to admit that the arrogantly thought was wrong, well since I am now writing this to share the experience publicly to everyone, I would say I've already forgiven myself for making naive decisions), and another reason here was, the issues or challenges I faced (which I mentioned above, about memory, personality stability, realtime, and game playing etc.) were so hard to solve with the knowledge I had at that time, and lack of documentations, learning materials of realtime LLMs interactive examples, I put it away, again.

TBH, I didn't give it up, I started to learn many things about multi-model, and voice synthesis, motion control, and Minecraft playing. I did a lot of researches on how other AI VTuber or AI waifu projects work. These researches later on produces this huge awesome list of AI VTuber projects:

Awesome AI VTuber A curated list of AI VTubers and their related projects

Ok, but it's still called ai, where is Project AIRI then?

Reborn, with stronger, and better determination

Someday near the end of 2024, November, @kwaa chatted me about making virtual characters in VR/AR world, with the power of WebXR. When we talked about the motion control and the character emotion detection, I told they I got a project that did exactly what you are looking for, but codebase wasn't organized, nor ready to be published to GitHub.

What to wait for? I started to work on it again, rethink about the structure and design, improved the implementation with much faster and better queueing and multiplexing playback system, and adjustments on the basic WebUI I made randomly, finally, I published it to GitHub on December 2, 2024 with commit d9ae0aa.

Project AIRI was somehow born or reborn, with the name of AIRI (アイリ, formerly Airi).

Did you know?

Interestingly, from the upload 2 years ago, March 25, 2023, https://www.youtube.com/watch?v=Tts-YAdn5Yc, a clip from Twitch stream of Vedal, and Neuro-sama, Vedal mentioned that right before she called the name "Neuro-sama", she was called "Airis AI", the name Airis magically, and coincidentally, matches the name of Project AIRI, which I am working on now. But I wasn't aware of this name until I searches more about their stories long after I open sourced Project AIRI.

In fact, the name AIRI (アイリ) was named by GPT-4o, I asked it about naming this project by referencing other Japanese / or Anime-ish names, it suggested the name Airi.

I failed so many of times on startups and other projects, only the recent ones become known by the public, I tried my best to make it better, with better UI, better code structure, leading technologies to build and code with rapid speed. I put so much effort into it with public slides show, and demonstrate it to others to my friends and during small meetups and conferences.

Many of those experiences was learned from my previous failures.

Glad many trials succeeded, and I am still here, working on Project AIRI.

Perhaps, it's another time that my determination was filled by not only Neuro-sama, but also the most profound, talented contributors, and fans.

Keep going, keep dreaming

New Banner!

When life gives you lemons, you lemon. Or something like that, my point is that this painful obstacle is an opportunity for me go get stronger, baby!

-- quote from Evil Neuro when streaming playing Slay the Spire

Now, Project AIRI is approaching to 1000 stars on GitHub when I am writing this post, while having over 150 Discord members, and 200 Telegram group members.

We covers fields like AI, VRM, Live2D, UI design, multi-modal AI, game playing agents, streaming APIs, bionic memory mechanisms, and many more. She is capable of playing games like Minecraft, Factorio. We got another community member who is researching on integrating her to be able to play and control Kerbal Space Program (KSP), as well as play any arbitrary games.

Many other companies are reaching out to us asking for collaboration, and we are working on it, to make Project AIRI better, and more useful for the community.

There is so much to do, and discover, we haven't reached the singularity of general purpose AI, perhaps Project AIRI will never made that point, but for now, having a companion-like AI agent to talk to, play games with, and share the knowledge and ideas with, is already a great achievement for me, and I hope it is for you too.

This is only the beginning memory address of our dreams, 0x1, the first byte of our journey.

How much memory we could store? It depends on how much we could dream, and how much we could achieve together.

Thanks for reading all the way here! Thanks for reading! Oh, and, Happy Birthday, Project AIRI!

This version codename still corresponds to a song by Kizuna AI and you can listen to it:

btw, v0.1 is "Hello World"

OK, so here's the new features:

• Generate image
• Reasoning utils
• More schema library supported
• More providers
• More integrations

Generate Image

GPT 4o Image Generation is very popular these days, isn't it?

Now you can also use it via API and @xsai/generate-image:

import { generateImage } from '@xsai/generate-image'
import { env } from 'node:process'

const prompt = 'A children\'s book drawing of a veterinarian using a stethoscope to listen to the heartbeat of a baby otter.'

const { image } = await generateImage({
  apiKey: env.OPENAI_API_KEY!,
  baseURL: 'http://api.openai.com/v1/',
  model: 'gpt-image-1',
  prompt,
})

const { images } = await generateImage({
  apiKey: env.OPENAI_API_KEY!,
  baseURL: 'http://api.openai.com/v1/',
  n: 4,
  model: 'gpt-image-1',
  prompt,
})

If this feature is popular, we may introduce editImage later.

Reasoning Utils

We made @xsai/utils-reasoning for models like qwq and deepseek-r1:

import { generateText } from '@xsai/generate-text'
import { streamText } from '@xsai/stream-text'
import { extractReasoning, extractReasoningStream } from '@xsai/utils-reasoning'

const messages = [
  {
    content: 'You\'re a helpful assistant.',
    role: 'system'
  },
  {
    content: 'Why is the sky blue?',
    role: 'user'
  },
]

const { text: rawText } = await generateText({
  baseURL: 'http://localhost:11434/v1/',
  messages,
  model: 'deepseek-r1',
})

const { textStream: rawTextStream } = await streamText({
  baseURL: 'http://localhost:11434/v1/',
  messages,
  model: 'deepseek-r1',
})

// { reasoning: string | undefined, text: string }
const { reasoning, text } = extractReasoning(rawText!)
// { reasoningStream: ReadableStream<string>, textStream: ReadableStream<string> }
const { reasoningStream, textStream } = extractReasoningStream(rawTextStream)

More schema library supported

We have supported Zod 4 Beta before it was officially released. (also includes @zod/mini!)

Effect Schema is supported as well.

More providers

We now support Featherless.

Did you know we've added a lot of providers? view here.

Special providers

New 🤗 Transformer.js provider

Have you dreamed about a possible future where you can use xsAI completely offline without Ollama and other inference server setup? We mentioned a bit in the previous blog post about our roadmap, here we come.

We now get a new dedicated project called xsai-transformers on GitHub, where we wrapped the famous library to work with models and inference, Transformer.js to help you get started on running embedding, speech, transcription, chat completions models with seamlessly designed API that compatible to xsAI, in both browser, WASM supported or WebGPU supported environments.

If you are interested, try it on our live demo.

npm i xsai-transformers

It feels like this when using it:

import { createEmbedProvider } from 'xsai-transformers'
import embedWorkerURL from 'xsai-transformers/embed/worker?worker&url'
import { embed } from 'xsai'

const transformers = createEmbedProvider({ baseURL: `xsai-transformers:///?worker-url=${embedWorkerURL}` })

// [
//   -0.038177140057086945,
//   0.032910916954278946,
//   -0.005459371022880077,
//   // ...
// ]
const { embedding } = await embed({
  ...transformers.embed('Xenova/all-MiniLM-L6-v2'),
  input: 'sunny day at the beach'
})

unSpeech

While you may notice we removed unSpeech from @xsai-ext/providers-local (our written provider to connect speech synthesis services with the style of OpenAI API), this doesn't mean we completely gave up of unSpeech, instead, for the past month, we added support of Alibaba Cloud Model Studio and Volcano Engine to unSpeech.

Therefore, it's time for unSpeech to get its own package, you can still use all the previous provided features by installing unspeech:

npm i unspeech

More Integrations

Did you know? We now have official Agentic and VoltAgent integrations.

Join our Community

If you have questions about anything related to xsAI,

you're always welcome to ask our community on GitHub Discussions.

Vercel AI SDK is way too big, it includes unnecessary dependencies.

For example, Vercel AI SDK shipped with non-optional OpenTelemetry dependencies, and bind the user to use zod (you don't get to choose), and so much more...

This makes it hard to build small and decent AI applications & CLI tools with less bundle size and more controllable and atomic capabilities that user truly needed.

But, it doesn't need to be like this, isn't it?

So how small is xsAI?

Without further ado, let's look:

It's roughly a hundred times smaller than the Vercel AI SDK (*install size) and has most of its features.

Also it is 5.7KB gzipped, so the title is not wrong.

Getting started

You can install the xsai package, which contains all the core utils.

npm i xsai

Or install the corresponding packages separately according to the required features:

npm i @xsai/generate-text @xsai/embed @xsai/model

Generating Text

So let's start with some simple examples.

import { generateText } from '@xsai/generate-text'
import { env } from 'node:process'

const { text } = await generateText({
  apiKey: env.OPENAI_API_KEY!,
  baseURL: 'https://api.openai.com/v1/',
  model: 'gpt-4o'
  messages: [{
    role: 'user',
    content: 'Why is the sky blue?',
  }],
})

xsAI does not use the provider function like Vercel does by default, we simplified them into three shared fields: apiKey, baseURL and model.

• apiKey: Provider API Key
• baseURL: Provider Base URL (will be merged with the path of the corresponding util, e.g. new URL('chat/completions', 'https://api.openai.com/v1/'))
• model: Name of the model to use

Don't worry if you need to support non-OpenAI-compatible API provider, such as Claude, we left the possibilities to override fetch(...) where you can customize how the request is made, and how the response was handled.

This allows xsAI to support any OpenAI-compatible API without having to create provider packages.

Generating Text w/ Tool Calling

Continuing with the example above, we now add the tools.

import { generateText } from '@xsai/generate-text'
import { tool } from '@xsai/tool'
import { env } from 'node:process'
import * as z from 'zod'

const weather = await tool({
  name: 'weather',
  description: 'Get the weather in a location',
  parameters: z.object({
    location: z.string().describe('The location to get the weather for'),
  }),
  execute: async ({ location }) => ({
    location,
    temperature: 72 + Math.floor(Math.random() * 21) - 10,
  }),
})

const { text } = await generateText({
  apiKey: env.OPENAI_API_KEY!,
  baseURL: 'https://api.openai.com/v1/',
  model: 'gpt-4o'
  messages: [{
    role: 'user',
    content: 'What is the weather in San Francisco?',
  }],
  tools: [weather],
})

Wait, zod is not good for tree shaking and annoying. Can we use valibot? Of course!

import { tool } from '@xsai/tool'
import { description, object, pipe, string } from 'valibot'

const weather = await tool({
  name: 'weather',
  description: 'Get the weather in a location',
  parameters: object({
    location: pipe(
      string(),
      description('The location to get the weather for'),
    ),
  }),
  execute: async ({ location }) => ({
    location,
    temperature: 72 + Math.floor(Math.random() * 21) - 10,
  }),
})

We can even use arktype, and the list of compatibility will grow in the future:

import { tool } from '@xsai/tool'
import { type } from 'arktype'

const weather = await tool({
  name: 'weather',
  description: 'Get the weather in a location',
  parameters: type({
    location: 'string',
  }),
  execute: async ({ location }) => ({
    location,
    temperature: 72 + Math.floor(Math.random() * 21) - 10,
  }),
})

xsAI doesn't limit your choices into either zod, valibot, or arktype, with the power of Standard Schema, you can use any schema library it supported you like.

Easy migration

Are you already using the Vercel AI SDK? Let's see how to migrate to xsAI:

- import { openai } from '@ai-sdk/openai'
- import { generateText, tool } from 'ai'
+ import { generateText, tool } from 'xsai'
+ import { env } from 'node:process'
import * as z from 'zod'

const { text } = await generateText({
+ apiKey: env.OPENAI_API_KEY!,
+ baseURL: 'https://api.openai.com/v1/',
- model: openai('gpt-4o')
+ model: 'gpt-4o'
  messages: [{
    role: 'user',
    content: 'What is the weather in San Francisco?',
  }],
- tools: {
+ tools: [
-   weather: tool({
+   await tool({
+     name: 'weather',
      description: 'Get the weather in a location',
      parameters: z.object({
        location: z.string().describe('The location to get the weather for'),
      }),
      execute: async ({ location }) => ({
        location,
        temperature: 72 + Math.floor(Math.random() * 21) - 10,
      }),
    })
- },
+ ],
})

That's it!

Next steps

Big fan of Anthropic's MCP?

We are working on Model Context Protocol support: #84

Don't like any of the cloud provider?

We are working on a 🤗 Transformers.js provider that enables you to directly run LLMs and any 🤗 Transformers.js supported models directly in browser, with the power of WebGPU!

You can track the progress here: #41. It is really cool and playful to run embedding, speech, and transcribing models directly in the browser, so, stay tuned!

Need framework bindings?

We will do this in v0.2. See you next time!

Documentation

Since this is just an introduction article, it only covers generate-text and tool.

xsai has more utils:

export * from '@xsai/embed'
export * from '@xsai/generate-object'
export * from '@xsai/generate-speech'
export * from '@xsai/generate-text'
export * from '@xsai/generate-transcription'
export * from '@xsai/model'
export * from '@xsai/shared-chat'
export * from '@xsai/stream-object'
export * from '@xsai/stream-text'
export * from '@xsai/tool'
export * from '@xsai/utils-chat'
export * from '@xsai/utils-stream'

If you are interested, go to the documentation at https://xsai.js.org/docs to get started!

Besides xsAI, we made loads of other cool stuff too! Check out our moeru-ai GitHub organization!

Join our Community

If you have questions about anything related to xsAI,

you're always welcome to ask our community on GitHub Discussions.