Unity AI & Sentis

Unity AI & Sentis

Unity integrates artificial intelligence both inside the Editor for productivity, and at runtime to drive gameplay mechanics.

Unity Muse

Unity Muse is a suite of AI-powered creation tools inside the editor:

• Muse Chat: Answers technical questions and drafts boilerplate scripts.
• Muse Texture: Generates PBR texture maps from text prompts.
• Muse Sprite: Generates 2D concept art and UI sprites.

Unity Sentis (Runtime Neural Networks)

Unity Sentis is an open-source engine that allows you to run trained AI models (deep learning neural networks) locally inside your game on user devices.

• It imports standard ONNX model files.
• It computes inference tasks (running the model) on the CPU or GPU.
• Useful for running local NPC speech models, hand-tracking AI, custom physics simulations, or real-time game difficulty adaptation.

Loading and Running ONNX Models:

using UnityEngine;
using Unity.Sentis;

public class ModelRunner : MonoBehaviour
{
    public ModelAsset modelAsset;
    private Model model;
    private IWorker worker;

    void Start()
    {
        // Load the ONNX model
        model = ModelLoader.Load(modelAsset);
        // Create an inference execution worker
        worker = WorkerFactory.CreateWorker(BackendType.GPUCompute, model);
    }

    void OnDestroy()
    {
        // Clean up memory
        worker.Dispose();
    }
}

TL;DR

• Muse tools help generate code, sprites, and textures inside the editor.
• Unity Sentis runs trained ONNX neural networks locally at runtime.
• Model inference can execute on the GPU or CPU.
• Make sure to dispose of your Sentis worker to prevent memory leaks.