@casual-simulation/aux-records

Records

A set of helpers and managers for the CasualOS records system.

Changelog

V3.0.0

Date: 2/10/2022

:rocket: Improvements

• Added the os.openImageClassifier(options) and os.closeImageClassifier() functions.

  • These functions are useful for applying Machine Learning inside CasualOS to detect categories of things via the camera feed.
  • Currently, the image classifier is only able to consume models generated with Teachable Machine.
    1. To create a model, go to https://teachablemachine.withgoogle.com/ and click "Get Started".
    2. Create an "Image Project" and choose "Standard image model".
    3. Add or record photos in each class.
    4. Click "Train".
    5. Once training is done you can get a model URL by clicking "Export Model".
    6. Under "Tensorflow.js", choose "Upload (shareable link)" and click "Upload". You can also optionally save the project to Google Drive.
    7. Once uploaded, copy the shareable link.
    8. Create a bot with an @onClick tag and put the following code in it (replacing MY_MODEL_URL with the shareable link):

       await os.openImageClassifier({
           modelUrl: 'MY_MODEL_URL',
       });
       

  • options is an object with the following properties:
    • modelUrl - The sharable link that was generated from Teachable Machine.
    • modelJsonUrl - Is optional and can be used in advanced scenarios where you want to control where the model is stored.
    • modelMetadataUrl - Is optional and can be used in advanced scenarios where you want to control where the model is stored.
    • cameraType - Is optional and is the type of camera that should be preferred. Can be "front" or "rear".

• Created the oai-1 appBundle.

  • This appBundle is currently a simple ab that can query the OpenAI GPT-3 API via a shout.

  • The ab has the following features:

    • A single manager bot in the oai-1 dimension and systemPortal as oai-1.manager.

    • @generateTextResponse is a listener that asks GPT-3 to respond to a given text prompt.

      • It takes the following parameters:

        • apiKey - The API key that should be used to access the API. You can get an API key at https://beta.openai.com/overview.
        • prompt - The text that the AI should respond to. An example is "Write a tagline for an ice cream shop.". Also see this guide: https://beta.openai.com/docs/guides/completion.
        • engine - The engine that should be used to process the prompt. Defaults to "text-davinci-001" if not specified. You can find a list of engines is available here: https://beta.openai.com/docs/engines.
        • options - An object that contains additional options for the request. You can find the documentation for these options here: https://beta.openai.com/docs/api-reference/completions/create.

      • It returns a promise that contains a list of generated choices.

      • Example:

        let oai = getBot('system', 'oai-1.manager');
        const response = await oai.generateTextResponse({
            apiKey: 'myAPIKey',
            prompt: 'Write a tagline for an ice cream shop.',
        });
        
        if (response.choices.length > 0) {
            os.toast('Best choice: ' + response.choices[0]);
        } else {
            os.toast('No choices.');
        }
        

:bug: Bug Fixes

• Fixed an issue with os.listData() where it was impossible to list data items unless a starting address was provided.