@or-sdk/qna

QnA SDK

The @or-sdk/qna package provides a client for working with the QnA (Question and Answer) system in OneReach.ai. With this client, you can perform operations such as loading documents, creating collections, searching in collections, asking questions to generate answers based on the loaded documents, and managing properties of collections.

Concepts

Collection

A collection is a group of searchable items, also known as passages. It allows you to organize and manage related documents and their passages together under a single entity. When you perform a search or ask a question, the QnA system will search through the passages within the specified collection.

Document

Documents are used to load a bulk of data into a collection. They can be any type of text-based file such as PDFs, text files, or web pages. When you provide a URL for a document, the QnA system downloads the content, splits it into smaller chunks (passages), and adds them to the collection.

Passage

A passage is an atomic piece of data that represents a portion of a document. It is the smallest searchable unit within a collection. When you search or ask a question, the QnA system uses vector similarity to find the most relevant passages from all the documents in the collection.

The search process involves comparing the vector representation of the input query (search term or question) with the vector representations of the passages in the collection. The results are ranked based on the similarity between the query and passages, with the most similar passages being returned as the search results.

Property

A property is an additional piece of information that can be added to a collection to store custom metadata about the collection. Properties can have a name, datatype, and an optional description. You can manage properties by adding, updating, or deleting them from a collection.

Installation

To install the package, run the following command:

$ npm install @or-sdk/qna

Usage

To use the QnA client, you need to create an instance of the class with the appropriate configuration options. You can either provide the direct API URL or the service discovery URL. Here is an example:

import { QnA } from '@or-sdk/qna';

// with direct api url
const qna = new QnA({
  token: 'my-account-token-string',
  serviceUrl: 'http://example.qna/endpoint'
});

// with service discovery(slower)
const qna = new QnA({
  token: 'my-account-token-string',
  discoveryUrl: 'http://example.qna/endpoint'
});

Once you have an instance of the QnA client, you can use its methods to interact with the OneReach.ai QnA system. The available methods are:

loadDocument

Loads a document into a collection by providing a URL to the document. The document can be in any text-based format, such as PDF, text files, or web pages. The QnA system downloads the content, splits it into smaller chunks called passages, and adds them to the collection.

Params

• collectionId: string - The ID of the collection the document will be loaded into
• params: LoadDocument - Document loading parameters
  • description: string (optional) - Document description
  • url: string - The URL of the document to load
  • name: string - Document name
  • defaultProperties: Record<string, unknown> (optional) - Default custom properties for the document passages. Note that these properties should be listed in the collection properties.

Example

In this example, the loadDocument method downloads a document from the provided URL, splits the content into passages, and adds them to the specified collection. Additionally, the defaultProperties attribute is used to set custom properties for the passages.

const collectionId = 'a1b2c3d4-uuid';
const document = await qna.loadDocument(collectionId, {
  name: 'Sample Document',
  description: 'A sample document',
  url: 'http://example.com/sample-document.pdf',
  defaultProperties: {
    author: 'John Doe',
    publishDate: '2020-12-31',
  },
});

// Example response
{
  id: 'e5f6g7h8-uuid',
  accountId: 'i9j0k1l2-uuid',
  collection: 'a1b2c3d4-uuid',
  name: 'Sample Document',
  description: 'A sample document'
}

createCollection

Create a new collection with optional properties. A collection is a group of related documents that are searchable together. You can define additional properties to store custom metadata associated with the passages within the collection.

Params

• name: string - The name of the collection
• description: string (optional) - A brief description of the collection
• properties: Property[] (optional) - An array of custom properties to add to the collection. Each property should include a name, datatype, and an optional description. For more information on supported datatypes, visit https://weaviate.io/developers/weaviate/config-refs/schema#datatypes

Example

Creating a collection with a name and description

const collection = await qna.createCollection({
  description: 'A sample collection',
  name: 'Sample Collection',
});

// Example response
{
  id: 'a1b2c3d4-uuid',
  accountId: 'i9j0k1l2-uuid',
  name: 'Sample Collection',
  description: 'A sample collection'
}

Creating a collection with additional properties

const collectionWithProperties = await qna.createCollection({
  description: 'A collection with properties',
  name: 'Collection With Properties',
  properties: [
    {
      name: 'author',
      dataType: 'string',
      description: 'Author of the document',
    },
    {
      name: 'publishDate',
      dataType: 'date',
      description: 'Date when the document was published',
    },
  ],
});

// Example response
{
  id: 'x1y2z3w4-uuid',
  accountId: 'i9j0k1l2-uuid',
  name: 'Collection With Properties',
  description: 'A collection with properties',
  properties: [
    {
      name: 'author',
      dataType: 'string',
      description: 'Author of the document',
    },
    {
      name: 'publishDate',
      dataType: 'date',
      description: 'Date when the document was published',
    },
  ],
}

Note: The custom properties defined for the collection can be used for filtering, sorting, and selecting specific properties when performing search or ask operations.

addProperty

Add a custom property to an existing collection. Custom properties can be used to store additional metadata about the collection or its passages. The available data types for properties can be found at https://weaviate.io/developers/weaviate/config-refs/schema#datatypes. Note that there are reserved property names ['accountId', 'collection', 'document', 'content', 'loaderMetadata']

Params

• collectionId (string): The ID of the collection to which the property will be added.
• property (Property): The property to be added, which includes:
  • name (string): The name of the property, e.g. nameOfTheAuthor. Name should not include any space.
  • dataType (string): The data type of the property. See https://weaviate.io/developers/weaviate/config-refs/schema#datatypes for the available types
  • description (string, optional): An optional description of the property.

Example

const collectionId = 'a1b2c3d4-uuid';
const property = {
  name: 'author',
  dataType: 'string',
  description: 'Author of the document',
};

await qna.addProperty(collectionId, property);

// No direct response, the property is added to the collection

You can verify the added property by retrieving the collection:

const collection = await qna.getCollection(collectionId);

// Example response
{
  id: 'a1b2c3d4-uuid',
  accountId: 'i9j0k1l2-uuid',
  name: 'Sample Collection',
  description: 'A sample collection',
  properties: [
    {
      name: 'author',
      dataType: 'string',
      description: 'Author of the document',
    },
  ],
}

deleteCollection

Delete a collection by its ID. This operation will permanently remove the collection along with all the documents, passages, and custom properties associated with the collection.

Params

• collectionId: string - The ID of the collection to delete

Example

const collectionId = 'a1b2c3d4-uuid';
await qna.deleteCollection(collectionId);

Note: Use this operation with caution, as deleting a collection is an irreversible action. Once a collection is deleted, all data associated with it will be lost and cannot be recovered. Always make sure to backup any important data before deleting a collection.

search

Search for documents in a collection based on a query. This method accepts several parameters that can be used to filter, sort, and limit the results.

Params

• term: string - The search term, query, or keywords
• limit: number (optional) - Limit the number of search results
• where: Record<string, unknown> (optional) - Weaviate filter object for advanced filtering. For more details, visit https://weaviate.io/developers/weaviate/api/graphql/filters
• select: string[] (optional) - An array of custom properties to include in the results

Example

const collectionId = 'a1b2c3d4-uuid';

const searchResults = await qna.search(collectionId, {
  term: 'search term',
  limit: 10,
  where: {
    operator: 'And',
    operands: [
      {
        operator: 'GreaterThan',
        path: ['publishDate'],
        valueDate: '2020-01-01T00:00:00Z',
      },
      {
        operator: 'LessThan',
        path: ['publishDate'],
        valueDate: '2022-01-01T00:00:00Z',
      },
    ],
  },
  select: ['author', 'publishDate'],
});

// Example response
[
  {
    id: 'e5f6g7h8-uuid',
    distance: '0.1234',
    content: 'Found passage content',
    author: 'John Doe',
    publishDate: '2020-12-31'
  }
]

In this example, the search method is called with advanced filtering and property selection options. The where parameter is used to filter the results based on the publishDate property, and the select parameter is used to include the author and publishDate properties of the selected passages in the response. As a result, the search returns a list of passages that meet the specified conditions and include the custom property values.

ask

Asks a question and generates an answer based on the documents in a collection. This method accepts several parameters that can be used to filter, sort, and limit the results.

Params

• question: string - The question to ask
• messages: CompletionRequestMessage[] (optional) - An array of previous chat messages with roles 'user' and 'assistant'
• limit: number (optional) - Limit the number of search results
• where: Record<string, unknown> (optional) - Weaviate filter object for advanced filtering. For more details, visit https://weaviate.io/developers/weaviate/api/graphql/filters
• select: string[] (optional) - An array of custom properties to include in the results

Example

The example below demonstrates a query with filtering and property selection options provided. The where parameter is used to filter the results based on the publishDate property, and the select parameter is used to include the author and publishDate properties of the selected passages in the response.

const collectionId = 'a1b2c3d4-uuid';

const askResults = await qna.ask(collectionId, {
  question: 'What is the meaning of life?',
  messages: [
    { role: 'user', content: 'Hello' },
    { role: 'assistant', content: 'Hi there!' },
  ],
  limit: 10,
  where: {
    operator: 'And',
    operands: [
      {
        operator: 'GreaterThan',
        path: ['publishDate'],
        valueDate: '2020-01-01T00:00:00Z',
      },
      {
        operator: 'LessThan',
        path: ['publishDate'],
        valueDate: '2022-01-01T00:00:00Z',
      },
    ],
  },
  select: ['author', 'publishDate'],
});

// Example response
{
  result: { role: 'assistant', content: 'The meaning of life is 42.' },
  searchResult: {
    id: 'e5f6g7h8-uuid',
    distance: '0.1234',
    content: 'Found passage content',
    author: 'John Doe',
    publishDate: '2020-12-31',
  },
}

updateDocument

Update a document's description in a collection. This method allows you to modify the description of an existing document, providing an updated text to better explain or detail the document's content.

Params

• collectionId: string - The ID of the collection the document belongs to
• documentId: string - The ID of the document to update
• params: UpdateDocument - The update document parameters, which include:
• description: string - The updated description of the document

Example

Updating a document's description in a collection:

const collectionId = 'a1b2c3d4-uuid';
const documentId = 'e5f6g7h8-uuid';
const updatedDocument = await qna.updateDocument(collectionId, documentId, {
  description: 'Updated document description',
});

// Example response
{
  id: 'e5f6g7h8-uuid',
  accountId: 'i9j0k1l2-uuid',
  collection: 'a1b2c3d4-uuid',
  name: 'Sample Document',
  description: 'Updated document description'
}

Note: Only the document description can be updated using this method. Other metadata such as the document name or URL cannot be modified once the document is created and added to the collection.

updateCollection

Update an existing collection's description. The collection must be previously created using the createCollection method. The updated collection will preserve its existing properties, ID, name, and associated documents.

Params

• collectionId: string - The ID of the collection you want to update
• updateParams: UpdateCollection - An object containing the properties you want to update, which can include:
• description: string (optional) - The new description for the collection

Example

Updating a collection's description

const collectionId = 'a1b2c3d4-uuid';
const updatedCollection = await qna.updateCollection(collectionId, {
  description: 'Updated collection description',
});

// Example response
{
  id: 'a1b2c3d4-uuid',
  accountId: 'i9j0k1l2-uuid',
  name: 'Sample Collection',
  description: 'Updated collection description'
}

getDocument

Retrieve a single document from a collection by its ID. A document is a text-based file (e.g., PDF, TXT, or HTML) that contains content, which is split into passages for searching within a collection.

Params

• collectionId: string - The ID of the collection the document belongs to.
• documentId: string - The ID of the document to retrieve.

Example

const collectionId = 'a1b2c3d4-uuid';
const documentId = 'e5f6g7h8-uuid';
const document = await qna.getDocument(collectionId, documentId);

// Example response
{
  id: 'e5f6g7h8-uuid',
  accountId: 'i9j0k1l2-uuid',
  collection: 'a1b2c3d4-uuid',
  name: 'Sample Document',
  description: 'A sample document'
}

getCollection

Retrieve a collection by its ID. This method returns a collection object including its ID, account ID, name, description, and any additional properties defined during the collection's creation.

Params

• collectionId: string - The ID of the collection to retrieve

Example

In this example, the getCollection method is called with a collectionId to retrieve the corresponding collection object. The response includes the collection's metadata, such as its ID, name, and description, as well as any custom properties defined for the collection (in this case, author and publishDate).

const collectionId = 'a1b2c3d4-uuid';
const collection = await qna.getCollection(collectionId);

// Example response
{
  id: 'a1b2c3d4-uuid',
  accountId: 'i9j0k1l2-uuid',
  name: 'Sample Collection',
  description: 'A sample collection',
  properties: [
    {
      name: 'author',
      dataType: 'string',
      description: 'Author of the document',
    },
    {
      name: 'publishDate',
      dataType: 'date',
      description: 'Date when the document was published',
    },
  ],
}

listDocuments

List documents in a collection with optional pagination and query.

const collectionId = 'a1b2c3d4-uuid';
const documents = await qna.listDocuments(collectionId, {
  query: 'search query',
  size: 10,
  skip: 0,
});

// Example response
```json
{
  "items": [
    {
      "id": "e5f6g7h8-uuid",
    "accountId": "i9j0k1l2-uuid",
    "collection": "a1b2c3d4-uuid",
    "name": "Sample Document",
    "description": "A sample document"
    }
  ],
  "total": 1
}

listCollections

List collections with optional pagination and query.

const collections = await qna.listCollections({
  query: 'search query',
  size: 10,
  skip: 0,
});

// Example response
{
  "items": [
    {
      "id": "a1b2c3d4-uuid",
      "accountId": "i9j0k1l2-uuid",
      "name": "Sample Collection",
      "description": "A sample collection"
    }
  ],
  "total": 1
}

deleteDocument

Delete a document from a collection by its ID. This will remove the document and all its associated passages from the collection, and the removed data will no longer be searchable.

Params

• collectionId: string - The ID of the collection containing the document
• documentId: string - The ID of the document to delete

Example

const collectionId = 'a1b2c3d4-uuid';
const documentId = 'e5f6g7h8-uuid';
await qna.deleteDocument(collectionId, documentId);

// No direct response, the document is deleted from the collection

Note: Deleting a document is a permanent operation, and the removed data cannot be recovered. Be cautious when using this method and ensure you have backups of your data if necessary.

createPassage

Create a passage and add it to a specific document within a collection. A passage is an atomic piece of data that represents a portion of a document. It is the smallest searchable unit within a collection.

Params

• collectionId: string - The ID of the collection the document belongs to
• params: CreatePassage - An object containing parameters required to create a passage
• content (string): The content of the passage to be created
• documentId (string): The ID of the document the passage will be associated with
• property: value (optional): Custom properties of the passage. Note that these properties should be listed in the collection properties

Example

const collectionId = 'a1b2c3d4-uuid';
const params: CreatePassage = {
  content: 'This is the content of the passage.',
  documentId: 'e5f6g7h8-uuid',
};
const passage = await qna.createPassage(collectionId, params);

// Example response
{
  id: 'x1y2z3w4-uuid',
  content: 'This is the content of the passage.',
  // ...more properties
}

Note: When creating a passage, you can also include additional custom properties if they are defined in the collection schema. These custom properties can later be used for filtering, sorting, and selecting specific properties when performing search or ask operations.

createManyPassages

Create a batch of passages and add it to a specific document within a collection. A passage is an atomic piece of data that represents a portion of a document. It is the smallest searchable unit within a collection.

Params

• collectionId: string - The ID of the collection the document belongs to
• passages: CreatePassage[] - An array of objects containing parameters required to create each passage
• content (string): The content of the passage to be created
• documentId (string): The ID of the document the passage will be associated with
• property: value (optional): Custom properties of the passage. Note that these properties should be listed in the collection properties

Example

const collectionId = 'a1b2c3d4-uuid';
const passages: CreatePassage[] = [
  {
    content: 'This is the content of the first passage.',
    documentId: 'e5f6g7h8-uuid',
  },
  {
    content: 'This is the content of the second passage.',
    documentId: 'e5f6g7h8-uuid',
  },
];
const passage = await qna.createPassage(collectionId, params);

// No direct response, the batch of passages are added to the collection

Note: The number of the passages are not limited but the total amount of the data to create in a single batch is limited to 1MB.

listPassages

List all passages in a collection with optional pagination, search query filtering, and search type. This method allows you to retrieve passages from a collection and optionally filter them based on a search query, page size, and offset.

Params

• collectionId: string - The ID of the collection to retrieve passages from
• params: ListPassages (optional) - An object containing the following optional properties:
• query (string, optional): Search query for filtering passages
• from (number, optional): Pagination offset (index of the first item in the current page)
• size (number, optional): Maximum number of passages to retrieve (page size)
• alpha (number, optional): Weight between sparse and vector search. A value of 0 represents a pure sparse search, while a value of 1 represents a pure vector search. Values between 0 and 1 control the balance between the two search modes.

Example

Listing passages with pagination and query filter:

const collectionId = 'a1b2c3d4-uuid';
const params: ListPassages = {
  query: 'sample query',
  from: 0,
  size: 10,
  alpha: 0.5,
};
const passages = await qna.listPassages(collectionId, params);

// Example response
{
  items: [
    {
      id: 'x1y2z3w4-uuid',
      content: 'This is the content of the passage.',
      // ...more properties
    },
    // ...more passages
  ],
  total: 123
}

getPassage

Retrieve a single passage from a collection by its ID. The passage object will include any custom properties specified in the collection schema.

Params

• collectionId: string - The ID of the collection the passage belongs to
• passageId: string - The ID of the passage to retrieve

Example

const collectionId = 'a1b2c3d4-uuid';
const passageId = 'x1y2z3w4-uuid';
const passage = await qna.getPassage(collectionId, passageId);

// Example response
{
  id: 'x1y2z3w4-uuid',
  content: 'This is the content of the passage.',
  // ...more properties based on the collection schema
}

Returns:

A Passage object representing the retrieved passage:

• id (string): The ID of the passage
• content (string): The content of the passage
• The rest of the properties specified in the collection schema

updatePassage

Update a passage in a collection. This method allows you to modify properties of a specific passage, including both content and custom properties.

Params

• collectionId: string - The ID of the collection the passage belongs to
• passageId: string - The ID of the passage to update
• params: UpdatePassage - The update passage parameters, which can include:
• content (string): Updated content of the passage
• You can also include additional properties to update if they exist in the collection schema and have a text datatype

Example

Updating a passage with new content and a custom property

const collectionId = 'a1b2c3d4-uuid';
const passageId = 'x1y2z3w4-uuid';
const params: UpdatePassage = {
  content: 'Updated content of the passage',
  customProperty: 'Updated custom property value',
};
const updatedPassage = await qna.updatePassage(collectionId, passageId, params);

// Example response
{
  id: 'x1y2z3w4-uuid',
  content: 'Updated content of the passage',
  customProperty: 'Updated custom property value',
  // ...more properties if available
}

Returns:

A Passage object representing the updated passage:

• id (string): The ID of the updated passage
• The updated properties, including the new content and any other properties specified in the update parameters

Note that updating text datatype properties will update the passage vector.

deletePassage

Delete a passage from a collection by its ID. This method removes a specific passage from the given collection, based on the passage's ID.

Params

• collectionId: string - The ID of the collection the passage belongs to
• passageId: string - The ID of the passage to delete

Example

In this example, we provide the collectionId and passageId as input parameters to the deletePassage method. The specified passage will be deleted from the collection, with no direct response returned.

const collectionId = 'a1b2c3d4-uuid';
const passageId = 'x1y2z3w4-uuid';
await qna.deletePassage(collectionId, passageId);

// There's no direct response, the passage is deleted from the collection

Note: Deleting a passage is a permanent operation, and the removed data cannot be recovered. Be cautious when using this method and ensure you have backups of your data if necessary.