@charisma-ai/sdk
Charisma.ai SDK for Javascript (browser)
Last updated 4 months ago by benhjames .
MIT · Repository · Original npm · Tarball · package.json
$ cnpm install @charisma-ai/sdk 
SYNC missed versions from official npm registry.

Charisma.ai SDK for JavaScript

Usage

yarn add @charisma-ai/sdk
import { Charisma, Microphone, Speaker } from '@charisma.ai/sdk';

async function run() {
  const token = await Charisma.createPlaythroughToken({ storyId: 4 });
  const conversationId = await Charisma.createConversation(token);

  const charisma = new Charisma(token);
  const speaker = new Speaker();
  const microphone = new Microphone();

  const conversation = charisma.joinConversation(conversationId);
  conversation.on('start-typing', () =>
    console.log('Character started typing...'),
  );
  conversation.on('stop-typing', () =>
    console.log('Character stopped typing...'),
  );
  conversation.on('message', message => {
    console.log(message);
    if (message.message.speech) {
      microphone.stopListening();
      speaker.play(message.message.speech.audio.data);
      microphone.startListening();
    }
  });
  conversation.setSpeechConfig({
    encoding: 'mp3',
    output: 'buffer',
  });

  charisma.connect();
  conversation.start();

  microphone.startListening();
  microphone.on('recognise', text => conversation.reply({ text }));
}

API Reference

Create a new Charisma instance to connect to a playthrough and interact with the chat engine.

  • playthroughToken (number): The token generated in Charisma.createPlaythroughToken.

(static) Charisma.createPlaythroughToken

Use this to set up a new playthrough.

  • storyId (number): The id of the story that you want to create a new playthrough for. The story must be published, unless a Charisma.ai user token has been passed and the user matches the owner of the story.
  • version (number, optional): The version of the story that you want to create a new playthrough for. If omitted, it will default to the most recent published version. To get the draft version of a story, pass -1 and a userToken.
  • userToken (string, optional): If the story is unpublished, pass a userToken to be able to access your story.

Returns a promise that resolves with the token.

const token = await Charisma.createPlaythroughToken({
  storyId: 12,
  version: 4,
  userToken: '...',
});

(static) Charisma.createConversation

A playthrough can have many simultaneous conversations. In order to start interacting, a conversation needs to be created, which can then be joined.

  • playthroughToken (string): The token generated with Charisma.createPlaythroughToken.
const conversationId = await Charisma.createConversation(token);

Charisma.joinConversation

This makes the Charisma instance listen out for events for a particular conversation, and returns a Conversation that events can be called on and event listeners attached.

  • conversationId (string): The conversation id generated with Charisma.createConversation.

Returns a Conversation, which can be used to send and receive events bound to that conversation.

Charisma.joinConversation(conversationId);

Charisma.connect

This is what kicks off the connection to the chat engine. Call this once you're ready to start sending and receiving events.

await Charisma.connect();

Events

To interact with the story, events are sent to and from the server that the WebSocket is connected to.

Events sent from client

conversation.start({ ... })

{
  "startNodeId": 12, // Optional, default undefined
  "speech": true // Optional, default false
}

conversation.reply({ ... })

{
  "text": "Please reply to this!",
  "speech": true // Optional, default false
}

Events received by client

conversation.on('reply', (data) => { ... })

{
  "message": {
    "text": "Greetings and good day.",
    "character": {
      "id": 20,
      "name": "Ted Baker",
      "avatar": "https://s3.charisma.ai/..."
    },
    "speech": {
      "duration": 203,
      "audio": /* either a buffer, or a URL */,
    }
    "metadata": {
      "myMetadata": "someValue"
    },
    "media": null
  },
  "endStory": false,
  "path": [{ "id": 1, "type": "edge" }, { "id": 2, "type": "node" }]
}

conversation.on('start-typing', () => { ... })

This event has no additional data.

conversation.on('stop-typing', () => { ... })

This event has no additional data.

Conversation helpers

conversation.setSpeechConfig(config)

This sets the speech configuration to use for all events in the conversation until set otherwise:

{
  "encoding": "mp3",
  "output": "buffer"
}

encoding is the file format of the resulting speech: mp3, ogg or pcm.

output determines whether the speech received back is a buffer (a byte array) or whether it should instead be a url pointing to the audio file.

Microphone

The microphone can be used to provide speech-to-text functionality. This is only available in browsers that support SpeechRecognition, currently Google Chrome only.

const microphone = new Microphone();

microphone.on('recognise', (text) => { ... })

To be used in conjunction with speech recognition (see below).

microphone.on('recognise-interim', (text) => { ... })

To be used in conjunction with speech recognition (see below).

microphone.on('start', () => { ... })

Emitted when the microphone is manually started via startListening.

microphone.on('stop', () => { ... })

Emitted when the microphone is either manually stopped via stopListening or automatically stopped after a timeout.

microphone.on('timeout', () => { ... })

Emitted when the microphone is automatically stopped after a timeout.

microphone.startListening(timeout?: number)

Starts browser speech recognition. The microphone will then emit recognise-interim (player hasn't finished speaking, this is the current best guess) and recognise (player has finished speaking and we're confident about the result) events.

The speech recognition will NOT automatically pause when a character is speaking via charisma.speak.

A timeout can optionally be passed, which will automatically stop the microphone after timeout milliseconds.

microphone.stopListening()

Stops browser speech recognition.

microphone.resetTimeout(timeout: number)

Resets the microphone timeout to timeout milliseconds.

Speaker

The speaker can be used to provide text-to-speech functionality.

const speaker = new Speaker();

speaker.play(data, interrupt)

Typically, you would want to use this in combination with a message conversation handler. You may also wish to pause the microphone while this happens.

Returns a Promise that resolves once the speech has ended.

interrupt is a boolean used to interrupt (stop playing) all currently playing audio before starting the audio passed into play.

conversation.on('message', async data => {
  if (data.message.speech) {
    microphone.stopListening();
    await speaker.play(data.message.speech.audio.data, true);
    microphone.startListening();
  }
});

#### speaker.on('start', () => { ... })

Emitted when the speaker starts playing any audio.

#### speaker.on('stop', () => { ... })

Emitted when the speaker finishes playing all audio.

Questions

For further details or any questions, feel free to get in touch with ben@charisma.ai

Current Tags

  • 2.0.0-alpha.6                                ...           alpha (a day ago)
  • 1.8.1                                ...           latest (4 months ago)
  • 0.7.0-next.1                                ...           next (2 years ago)

73 Versions

  • 2.0.0-alpha.6                                ...           a day ago
  • 2.0.0-alpha.5                                ...           8 days ago
  • 2.0.0-alpha.4                                ...           8 days ago
  • 2.0.0-alpha.3                                ...           a month ago
  • 2.0.0-alpha.2                                ...           a month ago
  • 2.0.0-alpha.1                                ...           a month ago
  • 1.10.0-alpha.1                                ...           a month ago
  • 1.9.1                                ...           4 months ago
  • 1.9.0                                ...           4 months ago
  • 1.8.1                                ...           4 months ago
  • 1.8.0                                ...           8 months ago
  • 1.8.0-alpha.3                                ...           9 months ago
  • 1.8.0-alpha.2                                ...           9 months ago
  • 1.8.0-alpha.1                                ...           9 months ago
  • 1.7.0                                ...           9 months ago
  • 1.7.0-alpha.4                                ...           9 months ago
  • 1.7.0-alpha.3                                ...           9 months ago
  • 1.7.0-alpha.2                                ...           9 months ago
  • 1.7.0-alpha.1                                ...           9 months ago
  • 1.6.1                                ...           10 months ago
  • 1.6.0                                ...           10 months ago
  • 1.5.0                                ...           a year ago
  • 1.5.0-alpha.2                                ...           a year ago
  • 1.5.0-alpha.1                                ...           a year ago
  • 1.4.0                                ...           a year ago
  • 1.3.0                                ...           a year ago
  • 1.2.0                                ...           a year ago
  • 1.2.0-alpha.2                                ...           a year ago
  • 1.2.0-alpha.1                                ...           a year ago
  • 1.1.0                                ...           a year ago
  • 1.0.5                                ...           a year ago
  • 1.1.0-alpha.2                                ...           a year ago
  • 1.1.0-alpha.1                                ...           a year ago
  • 1.0.4                                ...           a year ago
  • 1.0.3                                ...           a year ago
  • 1.0.2                                ...           a year ago
  • 1.0.1                                ...           a year ago
  • 1.0.0                                ...           a year ago
  • 1.0.0-alpha.14                                ...           a year ago
  • 1.0.0-alpha.13                                ...           a year ago
  • 1.0.0-alpha.12                                ...           a year ago
  • 1.0.0-alpha.11                                ...           a year ago
  • 1.0.0-alpha.10                                ...           a year ago
  • 1.0.0-alpha.9                                ...           a year ago
  • 1.0.0-alpha.8                                ...           a year ago
  • 1.0.0-alpha.7                                ...           a year ago
  • 1.0.0-alpha.6                                ...           a year ago
  • 1.0.0-alpha.5                                ...           a year ago
  • 1.0.0-alpha.4                                ...           a year ago
  • 1.0.0-alpha.3                                ...           a year ago
  • 1.0.0-alpha.2                                ...           a year ago
  • 1.0.0-alpha.1                                ...           a year ago
  • 0.9.2                                ...           a year ago
  • 0.9.1                                ...           a year ago
  • 0.9.0                                ...           a year ago
  • 0.8.3                                ...           a year ago
  • 0.8.2                                ...           a year ago
  • 0.8.1                                ...           a year ago
  • 0.8.0                                ...           a year ago
  • 0.7.3                                ...           a year ago
  • 0.7.2                                ...           a year ago
  • 0.7.1                                ...           a year ago
  • 0.7.0                                ...           a year ago
  • 0.7.0-next.1                                ...           2 years ago
  • 0.7.0-next.0                                ...           2 years ago
  • 0.6.0                                ...           2 years ago
  • 0.5.1                                ...           2 years ago
  • 0.5.0                                ...           2 years ago
  • 0.4.2                                ...           2 years ago
  • 0.4.1                                ...           2 years ago
  • 0.4.0                                ...           2 years ago
  • 0.3.1                                ...           2 years ago
  • 0.3.0                                ...           2 years ago
Downloads
Today 0
This Week 19
This Month 50
Last Day 13
Last Week 31
Last Month 41
Dependencies (7)
Dependents (1)

Copyright 2014 - 2017 © taobao.org |