ChatChromeAI

info

This feature is experimental and is subject to change.

note

The Built-in AI Early Preview Program by Google is currently in beta. To apply for access or find more information, please visit this link.

ChatChromeAI leverages the webGPU and Gemini Nano to run LLMs directly in the browser, without the need for an internet connection. This allows for running faster and private models without ever having data leave the consumers device.

Getting started

Once you've been granted access to the program, follow all steps to download the model.

Once downloaded, you can start using ChatChromeAI in the browser as follows:

import { ChatChromeAI } from "langchain/experimental/chat_models/chrome_ai";
import { HumanMessage } from "@langchain/core/messages";

const model = new ChatChromeAI({
  temperature: 0.5, // Optional, defaults to 0.5
  topK: 40, // Optional, defaults to 40
});

const message = new HumanMessage("Write me a short poem please");

const response = await model.invoke([message]);

Streaming

ChatChromeAI also supports streaming chunks:

import { AIMessageChunk } from "@langchain/core/messages";

let fullMessage: AIMessageChunk | undefined = undefined;
for await (const chunk of await model.stream([message])) {
  if (!fullMessage) {
    fullMessage = chunk;
  } else {
    fullMessage = fullMessage.concat(chunk);
  }
  console.log(fullMessage.content);
}

We also have a simple demo application which you can copy to instantly start running ChatChromeAI in your browser. Navigate to the README.md in the ./app directory of the integration for more instructions.