@or-sdk/qna

QnA SDK

The @or-sdk/qna package provides a client for working with the QnA (Question and Answer) system in OneReach.ai a.k.a Lookup. With this client, you can perform operations such as loading documents, creating collections, searching in collections, asking questions to generate answers based on the loaded documents, and managing properties of collections.

Concepts

Collection

A collection is a group of searchable items, also known as passages. It allows you to organize and manage related documents and their passages together under a single entity. When you perform a search or ask a question, the QnA system will search through the passages within the specified collection.

Document

Documents are used to load a bulk of data into a collection. They can be any type of text-based file such as PDFs, text files, or web pages. When you provide a URL for a document, the QnA system downloads the content, splits it into smaller chunks (passages), and adds them to the collection.

Passage

A passage is an atomic piece of data that represents a portion of a document. It is the smallest searchable unit within a collection. When you search or ask a question, the QnA system uses vector similarity to find the most relevant passages from all the documents in the collection.

The search process involves comparing the vector representation of the input query (search term or question) with the vector representations of the passages in the collection. The results are ranked based on the similarity between the query and passages, with the most similar passages being returned as the search results.

Property

A property is an additional piece of information that can be added to a collection to store custom metadata about the collection. Properties can have a name, datatype, and an optional description. You can manage properties by adding, updating, or deleting them from a collection.

Installation

To install the package, run the following command:

$ npm install @or-sdk/qna

Usage

To use the QnA client, you need to create an instance of the class with the appropriate configuration options. You can either provide the direct API URL or the service discovery URL. Here is an example:

import { QnA } from '@or-sdk/qna';

// with direct api url
const qna = new QnA({
  token: 'my-account-token-string',
  serviceUrl: 'http://example.qna/endpoint'
});

// with service discovery(slower)
const qna = new QnA({
  token: 'my-account-token-string',
  discoveryUrl: 'http://example.qna/endpoint'
});

Once you have an instance of the QnA client, you can use its methods to interact with the OneReach.ai QnA system. The available methods are:

loadDocument

Loads a document into a collection by providing a URL to the document. The document can be in PDF, CSV, or HTML format. The QnA system downloads the content, splits it into smaller chunks called passages, and adds them to the collection. For CSV files, the primary data source is the content column, and additional columns can be matched to custom properties in the collection schema. The loading process is asynchronous, and the document status can be used to check if the loading is in progress or if an error occurred.

Params

• collectionId: string - The ID of the collection the document will be loaded into
• params: LoadDocument - Document loading parameters
  • description: string (optional) - Document description
  • url: string - The URL of the document to load
  • name: string - Document name
  • defaultProperties: Record<string, unknown> (optional) - Default custom properties for the document passages. Note that these properties should be listed in the collection properties.

Status

The status field is used to identify the document loading progress and possible errors. The possible status values are:

• NEW - The document has been created, but the loading process has not started yet.
• LOADING - The document loading is in progress.
• READY - The document loading has been completed successfully.
• ERROR - An error occurred during the document loading process due to unknown reasons.
• string - If a specific error occurred during the loading process, the error message will be provided in the status field.

Example

In this example, the loadDocument method downloads a document from the provided URL, splits the content into passages, and adds them to the specified collection. Additionally, the defaultProperties attribute is used to set custom properties for the passages.

const collectionId = 'a1b2c3d4-uuid';
const document = await qna.loadDocument(collectionId, {
  name: 'Sample Document',
  description: 'A sample document',
  url: 'http://example.com/sample-document.pdf',
  defaultProperties: {
    author: 'John Doe',
    publishDate: '2020-12-31'
  }
});

// Example response
{
  id: 'e5f6g7h8-uuid',
  accountId: 'i9j0k1l2-uuid',
  collection: 'a1b2c3d4-uuid',
  name: 'Sample Document',
  description: 'A sample document',
  status: 'LOADING'
}

After the document loading process, the status will update according to the result:

// Example response when the document loading is successful
{
  id: 'e5f6g7h8-uuid',
  accountId: 'i9j0k1l2-uuid',
  collection: 'a1b2c3d4-uuid',
  name: 'Sample Document',
  description: 'A sample document',
  status: 'READY'
}

// Example response when there is an error during the document loading
{
  id: 'e5f6g7h8-uuid',
  accountId: 'i9j0k1l2-uuid',
  collection: 'a1b2c3d4-uuid',
  name: 'Sample Document',
  description: 'A sample document',
  status: 'Could not load the document due to an error...'
}

To handle cases where you need to wait for the document to fully load, you can use polling. Here's an example:

import { sleep } from 'utils';

async function waitForDocumentLoading(collectionId: string, documentId: string) {
  let document;
  let isLoading = true;

  while (isLoading) {
    document = await qna.getDocument(collectionId, documentId);

    if (document.status === 'READY' || typeof document.status === 'string') {
      isLoading = false;
    } else {
      await sleep(5000); // Wait for 5 seconds before checking the status again
    }
  }

  return document;
}

async function loadDocumentAndWait(collectionId: string, documentParams: LoadDocument) {
  const document = await qna.loadDocument(collectionId, documentParams);
  const loadedDocument = await waitForDocumentLoading(collectionId, document.id);
  return loadedDocument;
}

// Example usage
const collectionId = 'a1b2c3d4-uuid';
const documentParams = {
  name: 'Sample Document',
  description: 'A sample document',
  url: 'http://example.com/sample-document.pdf',
  defaultProperties: {
    author: 'John Doe',
    publishDate: '2020-12-31'
  }
};

const loadedDocument = await loadDocumentAndWait(collectionId, documentParams);
console.log('Loaded document:', loadedDocument);

In this example, the loadDocumentAndWait function first calls the loadDocument method to start loading a document and then uses the waitForDocumentLoading function to poll for the document's status until it's either READY or an error message is provided. The sleep function is used to introduce a delay before checking the status again.

createCollection

Creates a new collection with optional properties. A collection is a group of related documents that are searchable together. Additional custom properties can be defined to store metadata associated with the documents within the collection.

Params

• createParams: CreateCollection - An object containing the properties of the collection:
  • name: string - The name of the collection. It can start only with a letter and it must be at least 3 characters long, and cannot exceed 100 characters in length.
  • description: string (optional) - A brief description of the collection. If present, it must be at least 3 characters and no more than 500 characters in length.
  • properties: Property[] (optional) - An array of custom properties to add to the collection. Each property should include a name and datatype. Please see list of supported datatypes.
  • imageUrl: string (optional) - The URL of the thumbnail image for the collection. It must be at least 3 characters and it cannot exceed 2048 characters in length.
  • answerInstruction: string (optional) - An instruction used to generate a correct answer. If present, it cannot be more than 1000 characters long.
  • questionInstruction: string (optional) - An instruction used to generate a proper search query. If present, it cannot be more than 1000 characters long.
  • greetingInstruction: string (optional) - An instruction used to generate a correct first message. If present, it cannot be more than 1000 characters long.
  • temperature: number (optional) - Sampling temperature to use. If present, it must be a number between 0 and 2.
  • maxTokens: number (optional) - The maximum output length from the Large Language Model (LLM). If present, it must be a number between 128 and 2048.
  • frequencyPenalty: number (optional) - Value to penalize new tokens based on their frequency in existing text. If present, it must be a number between -2 and 2.
  • presencePenalty: number (optional) - Value to penalize new tokens based on their presence in existing text. If present, it must be a number between -2 and 2.
  • maxDistance: number (optional) - Value to filter out passages that are further than this value from the question or context. If present, it must be a number between 0 and 1.
  • modelName: string (optional) - Name of the large language model to be used. It can't be more than 64 characters long.

Example

Creating a collection with a name, description, and additional instructions

const collection = await qna.createCollection({
  name: 'Sample Collection',
  description: 'A sample collection for QnA',
  properties: [
    { name: 'author', dataType: 'string' },
    { name: 'year', dataType: 'int' },
  ],
  imageUrl: 'https://example.com/sample-collection.jpg',
  answerInstruction: 'Provide relevant answer from the text',
  questionInstruction: 'Generate a question based on understanding of the text',
  greetingInstruction: 'Initiate conversation providing context about the document',
  temperature: 0.7,
  maxTokens: 200,
  frequencyPenalty: 0.8,
  presencePenalty: 1,
  maxDistance: 0.5,
  modelName: 'text-davinci-002'
});

console.log(collection);

This will print collection information with its ID, name, and other parameters. It will also include any custom properties. Creating a collection with additional properties

const collectionWithProperties = await qna.createCollection({
  name: 'Collection With Properties',
  description: 'A collection with properties',
  properties: [
    {
      name: 'author',
      dataType: 'string',
      description: 'Author of the document',
    },
    {
      name: 'publishDate',
      dataType: 'date',
      description: 'Date when the document was published',
    },
  ],
});

// Example response
{
  id: 'a1b2c3d4-uuid',
  accountId: 'i9j0k1l2-uuid',
  name: 'Collection With Properties',
  description: 'A collection with properties'
  properties: [
    {
      id: 'e5f6g7h8-uuid',
      name: 'author',
      dataType: 'string',
      description: 'Author of the document',
    },
    {
      id: 'i9j0k1l2-uuid',
      name: 'publishDate',
      dataType: 'date',
      description: 'Date when the document was published',
    },
  ],
}

// Example response { id: 'x1y2z3w4-uuid', accountId: 'i9j0k1l2-uuid', name: 'Collection With Properties', description: 'A collection with properties', properties: [ { name: 'author', dataType: 'string', description: 'Author of the document', }, { name: 'publishDate', dataType: 'date', description: 'Date when the document was published', }, ], }

**Note:** The custom properties defined for the collection can be used for filtering, sorting, and selecting specific properties when performing search or ask operations.

#### `addProperty`
Add a custom property to an existing collection. Custom properties can be used to store additional metadata about the collection or its passages. The available data types for properties can be found at https://weaviate.io/developers/weaviate/config-refs/datatypes. Note that there are reserved property names ['accountId', 'collection', 'document', 'content', 'loaderMetadata', 'sourceUrl']

#### Params
- `collectionId` (string): The ID of the collection to which the property will be added.
- `property` (Property): The property to be added, which includes:
  - `name` (string): The name of the property, e.g. `nameOfTheAuthor`. Name should not include any space.
  - `dataType` (string): The data type of the property. See https://weaviate.io/developers/weaviate/config-refs/schema#datatypes for the available types
  - `description` (string, optional): An optional description of the property.

#### Example
```typescript
const collectionId = 'a1b2c3d4-uuid';
const property = {
  name: 'author',
  dataType: 'string',
  description: 'Author of the document',
};

await qna.addProperty(collectionId, property);

// No direct response, the property is added to the collection

You can verify the added property by retrieving the collection:

const collection = await qna.getCollection(collectionId);

// Example response
{
  id: 'a1b2c3d4-uuid',
  accountId: 'i9j0k1l2-uuid',
  name: 'Sample Collection',
  description: 'A sample collection',
  properties: [
    {
      name: 'author',
      dataType: 'string',
      description: 'Author of the document',
    },
  ],
}

deleteCollection

Delete a collection by its ID. This operation will permanently remove the collection along with all the documents, passages, and custom properties associated with the collection.

Params

• collectionId: string - The ID of the collection to delete

Example

const collectionId = 'a1b2c3d4-uuid';
await qna.deleteCollection(collectionId);

Note: Use this operation with caution, as deleting a collection is an irreversible action. Once a collection is deleted, all data associated with it will be lost and cannot be recovered. Always make sure to backup any important data before deleting a collection.

search

Search for documents in a collection based on a query. This method accepts several parameters that can be used to filter, sort, and limit the results.

Params

• term: string - The search term, query, or keywords
• limit: number (optional) - Limit the number of search results
• where: Record<string, unknown> (optional) - Weaviate filter object for advanced filtering. For more details, visit https://weaviate.io/developers/weaviate/api/graphql/filters
• select: string[] (optional) - An array of custom properties to include in the results

Example

const collectionId = 'a1b2c3d4-uuid';

const searchResults = await qna.search(collectionId, {
  term: 'search term',
  limit: 10,
  where: {
    operator: 'And',
    operands: [
      {
        operator: 'GreaterThan',
        path: ['publishDate'],
        valueDate: '2020-01-01T00:00:00Z',
      },
      {
        operator: 'LessThan',
        path: ['publishDate'],
        valueDate: '2022-01-01T00:00:00Z',
      },
    ],
  },
  select: ['author', 'publishDate'],
});

// Example response
[
  {
    id: 'e5f6g7h8-uuid',
    distance: '0.1234',
    content: 'Found passage content',
    sourceUrl: 'http://example.com/source-url',
    loaderMetadata: {
      custom: 'metadata'
    },
    document: {
      id: 'i9j0k1l2-uuid',
      name: 'Document Name'
    }
  }
]

In this example, the search method is called with advanced filtering and property selection options. The where parameter is used to filter the results based on the publishDate property, and the select parameter is used to include the author and publishDate properties of the selected passages in the response. As a result, the search returns a list of passages that meet the specified conditions and include the custom property values. Along with these properties, the result also includes new properties like sourceUrl, loaderMetadata and document.

ask

Asks a question and generates an answer based on the documents in a collection. This method accepts a set of parameters to configure the question-answering process and to fine-tune the generated outputs.

The ask method supports two modes: 'ask' and 'conversation'. Here are the details of each mode:

1. 'ask' mode:

   This mode allows you to ask a direct, standalone question. In this mode, the processing of the question and the generation of the answer are not influenced by any prior context or conversation history.

   This mode is suitable for scenarios where a single question is to be answered independently, such as a search query or a factual question.

2. 'conversation' mode:

   This mode allows you to ask questions in the context of a conversation. This means that prior messages from the conversation will be taken into account while processing the question and generating the answer.

   This mode is suitable for scenarios where a dialogue is happening, and the generation of an answer is dependent on the previous exchanges in the conversation. In this mode, you can supply previous conversation messages in the messages parameter, which accepts an array of message objects, each with a sender role ('assistant' or 'user') and the content of the message.

Both modes use a language model to generate an answer given a question and a collection of documents. How the prior messages are used to influence the retrieval of information and the generation of responses differs based on the mode used.

Params

• question: string - The question to ask
• messages: CompletionRequestMessage[] (optional) - An array of previous chat messages with roles 'user' and 'assistant'. Default is an empty array
• limit: number (optional) - Limit the number of search results. Default is 5. Accepts values between 1 and 100
• where: Record<string, unknown> (optional) - Weaviate filter object for advanced filtering. For more details, visit https://weaviate.io/developers/weaviate/api/graphql/filters
• select: string[] (optional) - Array of the custom properties to select
• mode: AskMode (optional) - Question mode, can be either 'ask' or 'conversation'. Default is 'conversation'
• answerInstruction: string | null (optional) - A summarization instruction used to generate correct answer. Maximum Length: 1000 characters
• questionInstruction: string | null (optional) - A summarization instruction used to generate correct search query. Maximum Length: 1000 characters
• temperature: number (optional) - Adjusts the randomness in the model's output. Default is 1. Accepts values between 0 and 2
• maxTokens: number (optional) - Maximum output length from the language model. Default is 1024. Accepts values between 128 and 2048
• frequencyPenalty: number (optional) - Penalizes new tokens based on their existing frequency in the text so far. Default is 0. Range accepted is -2 to 2
• presencePenalty: number (optional) - Penalizes new tokens based on whether they appear in the text so far. Default is 0. Range accepted is -2 to 2
• maxDistance: number (optional) - Filters out passages that are further than this value from the question or context. Measured by cosine metric. Default is 0. Range accepted is 0 to 1

Example

const collectionId = 'a1b2c3d4-uuid';

const askResults = await qna.ask(collectionId, {
  question: 'What is the meaning of life?',
  messages: [
    { role: 'user', content: 'Hello' },
    { role: 'assistant', content: 'Hi there!' },
  ],
  limit: 10,
  where: {
    operator: 'And',
    operands: [
      {
        operator: 'GreaterThan',
        path: ['publishDate'],
        valueDate: '2020-01-01T00:00:00Z',
      },
      {
        operator: 'LessThan',
        path: ['publishDate'],
        valueDate: '2022-01-01T00:00:00Z',
      },
    ],
  },
  select: ['title', 'author', 'publishDate'],
  mode: 'conversation',
  answerInstruction: 'Generate a brief and concise answer.',
  questionInstruction: 'Find documents related to the meaning of life.',
  temperature: 0.8,
  maxTokens: 1500,
  frequencyPenalty: 0.5;
  presencePenalty: 0.5;
  maxDistance: 0.9,
});

// Example response
{
  result: { role: 'assistant', content: 'The meaning of life is 42.' },
  searchResult: [
    {
      id: 'e5f6g7h8-uuid',
      distance: '0.1234',
      content: 'Found passage content',
      sourceUrl: 'https://example.com/meaning-of-life',
      loaderMetadata: {...},
      document: {
        id: 'i9j0k1l2-uuid',
        name: 'On the Meaning of Life',
      },
      title: 'On the Meaning of Life',
      author: 'John Doe',
      publishDate: '2020-12-31',
    },
    // ... More results if available
  ],
}

updateDocument

Update a document's description in a collection. This method allows you to modify the description of an existing document, providing an updated text to better explain or detail the document's content.

Params

• collectionId: string - The ID of the collection the document belongs to
• documentId: string - The ID of the document to update
• params: UpdateDocument - The update document parameters, which include:
• description: string - The updated description of the document

Example

Updating a document's description in a collection:

const collectionId = 'a1b2c3d4-uuid';
const documentId = 'e5f6g7h8-uuid';
const updatedDocument = await qna.updateDocument(collectionId, documentId, {
  description: 'Updated document description',
});

// Example response
{
  id: 'e5f6g7h8-uuid',
  accountId: 'i9j0k1l2-uuid',
  collection: 'a1b2c3d4-uuid',
  name: 'Sample Document',
  description: 'Updated document description'
}

Note: Only the document description can be updated using this method. Other metadata such as the document name or URL cannot be modified once the document is created and added to the collection.

updateCollection

Update an existing collection's properties. The collection must be previously created using the createCollection method. The updated collection will preserve its existing properties, ID, and name, but its associated documents might be affected based on the new instructions.

Params

• collectionId: string - The ID of the collection you want to update
• updateParams: UpdateCollection - An object containing the properties you want to update:
  • description: string (optional) - The new description for the collection. Minimum length: 3, Maximum length: 500
  • imageUrl: string (optional) - A collection thumbnail image URL. Minimum length: 3, Maximum length: 2048
  • answerInstruction: string (optional) - A summarization instruction used to generate a correct answer. Maximum length: 1000
  • questionInstruction: string (optional) - A summarization instruction used to generate a correct search query. Maximum length: 1000
  • greetingInstruction: string (optional) - A summarization instruction used to generate a correct first message. Maximum length: 1000
  • temperature: number (optional) - What sampling temperature to use. Default value: 1 Range: [0, 2]
  • maxTokens: number (optional) - Max output length from the Large Language Model (LLM). Range: [128, 2048]
  • frequencyPenalty: number (optional) - Positive values penalize new tokens based on their existing frequency in the text. Range: [-2, 2]
  • presencePenalty: number (optional) - Penalize new tokens based on whether they appear in the text. Range: [-2, 2]
  • maxDistance: number (optional) - Filter out passages that are further then maxDistance from the question or context. Range: [0, 1]
  • modelName: string (optional) - Name of the large language model to be used. Maximum length: 64

Example

Updating a collection's description and adding a new question instruction.

const collectionId = 'a1b2c3d4-uuid';
const updatedCollection = await qna.updateCollection(collectionId, {
  description: 'Updated collection description',
  questionInstruction: 'Consider the main theme of the document for the query',
  maxTokens: 200,
  temperature: 1.5,
  modelName: 'gpt-3'
});

// Example response
{
  id: 'a1b2c3d4-uuid',
  accountId: 'i9j0k1l2-uuid',
  name: 'Sample Collection',
  description: 'Updated collection description',
  questionInstruction: 'Consider the main theme of the document for the query',
  maxTokens: 200,
  temperature: 1.5,
  modelName: 'gpt-3'
}

getDocument

Retrieve a single document from a collection by its ID. A document is a text-based file (e.g., PDF, TXT, or HTML) that contains content, which is split into passages for searching within a collection.

Params

• collectionId: string - The ID of the collection the document belongs to.
• documentId: string - The ID of the document to retrieve.

Example

const collectionId = 'a1b2c3d4-uuid';
const documentId = 'e5f6g7h8-uuid';
const document = await qna.getDocument(collectionId, documentId);

// Example response
{
  id: 'e5f6g7h8-uuid',
  accountId: 'i9j0k1l2-uuid',
  collection: 'a1b2c3d4-uuid',
  name: 'Sample Document',
  description: 'A sample document'
}

getCollection

Retrieve a collection by its ID. This method returns a collection object including its ID, account ID, name, description, and any additional properties defined during the collection's creation.

Params

• collectionId: string - The ID of the collection to retrieve

Example

In this example, the getCollection method is called with a collectionId to retrieve the corresponding collection object. The response includes the collection's metadata, such as its ID, name, and description, as well as any custom properties defined for the collection (in this case, author and publishDate).

const collectionId = 'a1b2c3d4-uuid';
const collection = await qna.getCollection(collectionId);

// Example response
{
  id: 'a1b2c3d4-uuid',
  accountId: 'i9j0k1l2-uuid',
  name: 'Sample Collection',
  description: 'A sample collection',
  properties: [
    {
      name: 'author',
      dataType: 'string',
      description: 'Author of the document',
    },
    {
      name: 'publishDate',
      dataType: 'date',
      description: 'Date when the document was published',
    },
  ],
}

listDocuments

List documents in a collection with optional pagination, query, and ordering.

Params

• collectionId: string - The ID of the collection to retrieve documents from.

• find: Find (optional) - Optional find parameters.

  • query: string (optional) - The search query to filter documents based on their name and/or description.
  • mode: SearchMode (optional) - The search mode, either 'bm25' for full-text search or 'vector' for vector similarity search. Default is 'bm25'
  • size: number (optional) - The number of documents to return per request. Defaults to 10.
  • from: number (optional) - The starting index for fetching the documents, used for pagination.
  • orderProperty: string (optional) - The property by which to order the documents, e.g., 'name' or 'createdAt'.
  • orderDirection: OrderDirection (optional) - The direction to order the documents, either 'asc' for ascending or 'desc' for descending.

Returns

• An array of documents with pagination info.

Example

const collectionId = 'a1b2c3d4-uuid';
const documents = await qna.listDocuments(collectionId, {
  query: 'search query',
  mode: 'bm25',
  size: 10,
  from: 0,
  orderProperty: 'name',
  orderDirection: 'asc',
});

// Example response
{
  "items": [
    {
      "id": "e5f6g7h8-uuid",
      "accountId": "i9j0k1l2-uuid",
      "collection": "a1b2c3d4-uuid",
      "name": "Sample Document",
      "description": "A sample document",
      "createdAt": "2022-01-02T03:04:05Z"
    }
  ],
  "total": 1
}

listCollections

List collections with optional pagination, ordering, and query.

Params

• find: Find - Optional find parameters.

  • query: string (optional) - The search query to filter collections based on their name and/or description.
  • mode: SearchMode (optional) - The search mode, either 'bm25' for full-text search or 'vector' for vector similarity search. Default is 'bm25'
  • size: number (optional) - The number of collections to return per request. Defaults to 10.
  • from: number (optional) - The starting index for fetching the collections, used for pagination.
  • orderProperty: string (optional) - The property by which to order the collections, e.g., 'name' or 'createdAt'.
  • orderDirection: OrderDirection (optional) - The direction to order the collections, either 'asc' for ascending or 'desc' for descending.

Returns

• An array of collections with pagination info.

Example

const collections = await qna.listCollections({
  query: 'search query',
  mode: 'bm25',
  size: 10,
  from: 0,
  orderProperty: 'name',
  orderDirection: 'asc',
});

// Example response
{
  "items": [
    {
      "id": "a1b2c3d4-uuid",
      "accountId": "i9j0k1l2-uuid",
      "name": "Sample Collection",
      "description": "A sample collection",
      "createdAt": "2022-01-02T03:04:05Z"
    }
  ],
  "total": 1
}

deleteDocument

Delete a document from a collection by its ID. This will remove the document and all its associated passages from the collection, and the removed data will no longer be searchable.

Params

• collectionId: string - The ID of the collection containing the document
• documentId: string - The ID of the document to delete

Example

const collectionId = 'a1b2c3d4-uuid';
const documentId = 'e5f6g7h8-uuid';
await qna.deleteDocument(collectionId, documentId);

// No direct response, the document is deleted from the collection

Note: Deleting a document is a permanent operation, and the removed data cannot be recovered. Be cautious when using this method and ensure you have backups of your data if necessary.

createPassage

Create a passage and add it to a specific document within a collection. A passage is an atomic piece of data that represents a portion of a document. It is the smallest searchable unit within a collection.

Params

• collectionId: string - The ID of the collection the document belongs to
• params: CreatePassage - An object containing parameters required to create a passage
• content (string): The content of the passage to be created
• documentId (string): The ID of the document the passage will be associated with
• property: value (optional): Custom properties of the passage. Note that these properties should be listed in the collection properties

Example

const collectionId = 'a1b2c3d4-uuid';
const params: CreatePassage = {
  content: 'This is the content of the passage.',
  documentId: 'e5f6g7h8-uuid',
};
const passage = await qna.createPassage(collectionId, params);

// Example response
{
  id: 'x1y2z3w4-uuid',
  content: 'This is the content of the passage.',
  // ...more properties
}

Note: When creating a passage, you can also include additional custom properties if they are defined in the collection schema. These custom properties can later be used for filtering, sorting, and selecting specific properties when performing search or ask operations.

createManyPassages

Create a batch of passages and add it to a specific document within a collection. A passage is an atomic piece of data that represents a portion of a document. It is the smallest searchable unit within a collection.

Params

• collectionId: string - The ID of the collection the document belongs to
• passages: CreatePassage[] - An array of objects containing parameters required to create each passage
• content (string): The content of the passage to be created
• documentId (string): The ID of the document the passage will be associated with
• property: value (optional): Custom properties of the passage. Note that these properties should be listed in the collection properties

Example

const collectionId = 'a1b2c3d4-uuid';
const passages: CreatePassage[] = [
  {
    content: 'This is the content of the first passage.',
    documentId: 'e5f6g7h8-uuid',
  },
  {
    content: 'This is the content of the second passage.',
    documentId: 'e5f6g7h8-uuid',
  },
];
const passage = await qna.createPassage(collectionId, params);

// No direct response, the batch of passages are added to the collection

Note: The number of the passages are not limited but the total amount of the data to create in a single batch is limited to 1MB.

listPassages

List all passages in a collection with optional pagination, search query filtering, and search mode. This method allows you to retrieve passages from a collection and optionally filter them based on a search query, page size, and offset.

Params

• collectionId: string - The ID of the collection to retrieve passages from.
• params: FindPassages (optional) - An object containing the following optional properties:
  • query: string (optional): Search query for filtering passages.
  • from: number (optional): Pagination offset (the index of the first item on the page).
  • size: number (optional): Maximum number of passages to retrieve (page size).
  • mode: SearchMode (optional) - The search mode, either 'bm25' for full-text search or 'vector' for vector similarity search. Default is 'bm25'
  • where: Record<string, unknown> (optional): A weaviate where filter. See Weaviate Filters for more information.

Example

Listing passages with pagination, query filter, and a specific search mode:

const collectionId = 'a1b2c3d4-uuid';
const params: FindPassages = {
  query: 'sample query',
  from: 0,
  size: 10,
  mode: 'vector',
};
const passages = await qna.listPassages(collectionId, params);

// Example response
{
  items: [
    {
      id: 'x1y2z3w4-uuid',
      content: 'This is the content of the passage.',
      // ...other properties
    },
    // ...more passages
  ],
  total: 123
}

This will return a list of passages from the specified collection that match the filter query and are ordered according to their vector similarity. It also supports a 'where' query for advanced filtering.

listPassagesInDocument

List all passages in a document with optional pagination, query, and ordering.

Params

• collectionId: string - The ID of the collection containing the document.

• documentId: string - The ID of the document to retrieve passages from.

• find: Find (optional) - Optional find parameters.

  • query: string (optional) - The search query to filter passages based on their content.
  • mode: SearchMode (optional) - The search mode, either 'bm25' for full-text search or 'vector' for vector similarity search. Default is 'bm25'
  • size: number (optional) - The number of passages to return per request. Defaults to 10.
  • from: number (optional) - The starting index for fetching the passages, used for pagination.
  • orderProperty: string (optional) - The property by which to order the passages, e.g., 'createdAt'.
  • orderDirection: OrderDirection (optional) - The direction to order the passages, either 'asc' for ascending or 'desc' for descending.

Returns

• An array of passages with pagination info.

Example

const collectionId = 'a1b2c3d4-uuid';
const documentId = 'e5f6g7h8-uuid';
const passages = await qna.listPassagesInDocument(collectionId, documentId, {
  query: 'search query',
  mode: 'bm25',
  size: 10,
  from: 0,
  orderProperty: 'createdAt',
  orderDirection: 'asc',
});

// Example response
{
  "items": [
    {
      "id": "m3n4o5p6-uuid",
      "accountId": "i9j0k1l2-uuid",
      "collection": "a1b2c3d4-uuid",
      "documentId": "e5f6g7h8-uuid",
      "content": "This is a sample passage in the document.",
      "createdAt": "2022-01-02T03:04:05Z"
    }
  ],
  "total": 1
}

getPassage

Retrieve a single passage from a collection by its ID. The passage object will include any custom properties specified in the collection schema.

Params

• collectionId: string - The ID of the collection the passage belongs to
• passageId: string - The ID of the passage to retrieve

Example

const collectionId = 'a1b2c3d4-uuid';
const passageId = 'x1y2z3w4-uuid';
const passage = await qna.getPassage(collectionId, passageId);

// Example response
{
  id: 'x1y2z3w4-uuid',
  content: 'This is the content of the passage.',
  // ...more properties based on the collection schema
}

Returns:

A Passage object representing the retrieved passage:

• id (string): The ID of the passage
• content (string): The content of the passage
• The rest of the properties specified in the collection schema

updatePassage

Update a passage in a collection. This method allows you to modify properties of a specific passage, including both content and custom properties.

Params

• collectionId: string - The ID of the collection the passage belongs to
• passageId: string - The ID of the passage to update
• params: UpdatePassage - The update passage parameters, which can include:
• content (string): Updated content of the passage
• You can also include additional properties to update if they exist in the collection schema and have a text datatype

Example

Updating a passage with new content and a custom property

const collectionId = 'a1b2c3d4-uuid';
const passageId = 'x1y2z3w4-uuid';
const params: UpdatePassage = {
  content: 'Updated content of the passage',
  customProperty: 'Updated custom property value',
};
const updatedPassage = await qna.updatePassage(collectionId, passageId, params);

// Example response
{
  id: 'x1y2z3w4-uuid',
  content: 'Updated content of the passage',
  customProperty: 'Updated custom property value',
  // ...more properties if available
}

Returns:

A Passage object representing the updated passage:

• id (string): The ID of the updated passage
• The updated properties, including the new content and any other properties specified in the update parameters

Note that updating text datatype properties will update the passage vector.

deletePassage

Delete a passage from a collection by its ID. This method removes a specific passage from the given collection, based on the passage's ID.

Params

• collectionId: string - The ID of the collection the passage belongs to
• passageId: string - The ID of the passage to delete

Example

In this example, we provide the collectionId and passageId as input parameters to the deletePassage method. The specified passage will be deleted from the collection, with no direct response returned.

const collectionId = 'a1b2c3d4-uuid';
const passageId = 'x1y2z3w4-uuid';
await qna.deletePassage(collectionId, passageId);

// There's no direct response, the passage is deleted from the collection

Note: Deleting a passage is a permanent operation, and the removed data cannot be recovered. Be cautious when using this method and ensure you have backups of your data if necessary.