@casual-simulation/aux-records - npm Package Versions

Improved the Records API to be able to return errors to allowed HTTP origins.

Improved os.meetCommand() to return a promise.

Added the ability to specify an options object with os.recordData(key, address, data, options) that can specify update and delete policies for the data.

• These policies can be useful to restrict the set of users that can manipulate the recorded data.

• options is an object with the following properties:

  let options: {
      /**
       * The HTTP Endpoint that should be queried.
       */
      endpoint?: string;
  
      /**
       * The policy that should be used for updating the record.
       * - true indicates that the value can be updated by anyone.
       * - An array of strings indicates the list of user IDs that are allowed to update the data.
       */
      updatePolicy?: true | string[];
  
      /**
       * The policy that should be used for deleting the record.
       * - true indicates that the value can be erased by anyone.
       * - An array of strings indicates the list of user IDs that are allowed to delete the data.
       * Note that even if a delete policy is used, the owner of the record can still erase any data in the record.
       */
      deletePolicy?: true | string[];
  };
  

Added the os.tip(message, pixelX?, pixelY?, duration?) and os.hideTips(tipIDs?) functions to make showing tooltips easy.

• os.tip(message, pixelX?, pixelY?, duration?) can be used to show a tooltip and takes the following parameters:
  • message is the message that should be shown.
  • pixelX is optional and is the horizontal pixel position on the screen that the message should be shown at. If omitted, then the tooltip will be shown at the current mouse position or the last touch position. Additionally, omitting the position will cause the tooltip to only be shown when the mouse is near it. Moving the mouse away from the tooltip in this mode will cause the tooltip to be automatically hidden.
  • pixelY is optional and is the vertical pixel position on the screen that the message should be shown at. If omitted, then the tooltip will be shown at the current mouse position or the last touch position. Additionally, omitting the position will cause the tooltip to only be shown when the mouse is near it. Moving the mouse away from the tooltip in this mode will cause the tooltip to be automatically hidden.
  • duration is optional and is the number of seconds that the toast should be visible for.
  • Returns a promise that resolves with the ID of the newly created tooltip.
• os.hideTips(tipIDs?) can be used to hide a tooltip and takes the following parameters:
  • tipIDs is optional and is the ID or array of IDs of tooltips that should be hidden. If omitted, then all tooltips will be hidden.
  • Returns a promise that resolves when the action has been completed.

Improved the menuPortal to use 60% of the screen width on large screens when the screen is taller than it is wide.

Improved the systemPortal to support system tag values that are set to non-string values such as booleans and integers.

Added WebXR hand tracking support.

• Tested and verified on Meta Quest 2 headset.
• Wrist portals have custom offsets for hands to try and minimize the blocking of hands during interaction.
• Air tap interaction only currently. Air taps are when you tap your index finger and thumb together to perform a "tap/click" on what the pointer ray is targeting.

• This is useful for saving or getting records from another CasualOS instance.
• The following functions have been updated to support an optional endpoint parameter:
  • os.recordData(key, address, data, endpoint?)
  • os.getData(recordName, address, endpoint?)
  • os.listData(recordName, startingAddress?, endpoint?)
  • os.eraseData(key, address, endpoint?)
  • os.recordManualApprovalData(key, address, data, endpoint?)
  • os.getManualApprovalData(recordName, address, endpoint?)
  • os.listManualApprovalData(recordName, startingAddress?, endpoint?)
  • os.eraseManualApprovalData(key, address, endpoint?)
  • os.recordFile(key, data, options?, endpoint?)
  • os.eraseFile(key, url, endpoint?)
  • os.recordEvent(key, eventName, endpoint?)
  • os.countEvents(recordName, eventName, endpoint?)

• This makes bot strokes that are scaled appear correct.
• This change also makes lines and strokes appear the same size on screen no matter the zoom level of the camera. This can make it easier to identify bots when zoomed out a lot.

• All record keys are now split into two categories: subjectfull keys and subjectless keys.
• subjectfull keys require login in order to publish data are are the default type of key.
• subjectless keys do not require login in order to publish data.
• When publishing data with a subjectless key, all users are treated as anonymous. In effect, this makes the owner of the record fully responsible for the content that they publish.

• functionName is the name of the function that should be triggered from the Jitsi Meet API.
• args is the list of arguments that should be provided to the function.
• Returns a promise that resolves with the result of the function call.

• Unlike @onMeetLoaded, @onMeetEntered is only triggered after the user clicks the "Join" button from the meeting waiting room.
• See the documentation for more detailed information.

• See the Jitsi FAQ for more information on how to setup a moderator for a meeting: https://developer.8x8.com/jaas/docs/faq#how-can-i-set-a-user-as-moderator-for-a-meeting

• Additionally, the botPortalAnchorPoint and botPortalStyle tags can be set on the botPortalBot similarly to how meetPortalAnchorPoint can be set on the meetPortalBot.

• For example, setting systemTagName to "test" will cause the systemPortal to search for bots that have a test tag instead of a system tag.

Added the os.openImageClassifier(options) and os.closeImageClassifier() functions.

• These functions are useful for applying Machine Learning inside CasualOS to detect categories of things via the camera feed.
• Currently, the image classifier is only able to consume models generated with Teachable Machine.
  1. To create a model, go to https://teachablemachine.withgoogle.com/ and click "Get Started".
  2. Create an "Image Project" and choose "Standard image model".
  3. Add or record photos in each class.
  4. Click "Train".
  5. Once training is done you can get a model URL by clicking "Export Model".
  6. Under "Tensorflow.js", choose "Upload (shareable link)" and click "Upload". You can also optionally save the project to Google Drive.
  7. Once uploaded, copy the shareable link.
  8. Create a bot with an @onClick tag and put the following code in it (replacing MY_MODEL_URL with the shareable link):

     await os.openImageClassifier({
         modelUrl: 'MY_MODEL_URL',
     });
     

• options is an object with the following properties:
  • modelUrl - The sharable link that was generated from Teachable Machine.
  • modelJsonUrl - Is optional and can be used in advanced scenarios where you want to control where the model is stored.
  • modelMetadataUrl - Is optional and can be used in advanced scenarios where you want to control where the model is stored.
  • cameraType - Is optional and is the type of camera that should be preferred. Can be "front" or "rear".

Created the oai-1 appBundle.

• This appBundle is currently a simple ab that can query the OpenAI GPT-3 API via a shout.

• The ab has the following features:

  • A single manager bot in the oai-1 dimension and systemPortal as oai-1.manager.

  • @generateTextResponse is a listener that asks GPT-3 to respond to a given text prompt.

    • It takes the following parameters:

      • apiKey - The API key that should be used to access the API. You can get an API key at https://beta.openai.com/overview.
      • prompt - The text that the AI should respond to. An example is "Write a tagline for an ice cream shop.". Also see this guide: https://beta.openai.com/docs/guides/completion.
      • engine - The engine that should be used to process the prompt. Defaults to "text-davinci-001" if not specified. You can find a list of engines is available here: https://beta.openai.com/docs/engines.
      • options - An object that contains additional options for the request. You can find the documentation for these options here: https://beta.openai.com/docs/api-reference/completions/create.

    • It returns a promise that contains a list of generated choices.

    • Example:

      let oai = getBot('system', 'oai-1.manager');
      const response = await oai.generateTextResponse({
          apiKey: 'myAPIKey',
          prompt: 'Write a tagline for an ice cream shop.',
      });
      
      if (response.choices.length > 0) {
          os.toast('Best choice: ' + response.choices[0]);
      } else {
          os.toast('No choices.');
      }