Basic Integration

As mentioned in the setup section, game studios will be required to create 2 files in order for the NRN Agent to "understand" the game world, and for the game world to "understand" the NRN Agent output. The NRN team will provide partner game studios with template files for Javascript, Unity, and Unreal.

Red indicates what is understandable for the game, while blue indicates what is understandable for NRN Agent

After instantiating the NRN Agent, the basic flow for integration is as follows:

• Convert the game world into an state space that the NRN model can use

• Human Controlled

  • Update human inputs

  • Convert executed inputs to an action that the NRN model can understand

  • Add the state and action pair to the dataset

• NRN Agent Controlled

  • Perform inference to select the action

  • Convert the selected action to an input that the game can understand

• Either send data to the trainer platform or train directly in-game

Below we show how to initialize the model and use it at the game manager level:

Javascript

class Game {
    // Instantiate agent
    constructor(agentInputs) {
        this.agent = AgentFactory.createAgent(...agentInputs);
    }
    
    // Game loop update
    update() {
        // Get the state space for the model
        const state = getStateSpace(this);
        
        if (this.human) {
            // If human is controlling, then update their inputs
            this.inputs.p1.update();
            
            // Convert the inputs into something the NRN agent can understand
            const action = getActionOneHot(this.inputs.p1);
            
            // Add the observed state and action to the dataset
            this.agent.collect({ state, action });
        }
        else {
            // If NRN agent is controlling, then perform inference
            const agentAction = this.agent.selectAction(state);
            
            // Execute the action in-game
            convertActionToGame(agentAction);
        }
    }
}

Then at the end of the game, you can send the data you collected to the trainer platform as follows:

await agent.uploadData();

OR

If you have the API access to train in-game, then you can directly train within the game loop as follows:

await agent.train(model.getTrainingData(), trainingConfiguration);

C# - Unity

public class Game : MonoBehaviour 
{
    // Define the agent that will be taking actions
    public IAgentRL Agent { get; set; }

    // Instantiate model
    private async void _loadAgent(modelInputs)
    {
        Agent = AgentFactory.CreateAgent<IAgentRL>("reinforcement", modelInputs);
    }
    
    // Game loop update
    private async void Update() {
        // Create the world representation which we will convert to the state space matrix
        var world = new NrnIntegration.NrnWorld 
        {
            // All world attributes/objects that are relevant for the AI
        };
        
        // Get the state space for the model
        Matrix state = NrnIntegration.NrnStateSpace.GetState(world);
        
        if (human) 
        {
            // Convert the inputs into something the NRN agent can understand
            Dictionary<string, bool> action = NrnActionSpace.ConvertInputToBools();
            
            // Add the observed state and action to the dataset
            Agent.Collect({ state, action });
        }
        else {
            // If NRN agent is controlling, then perform inference
            Dictionary<string, bool> agentAction = Agent.SelectAction(state);
            
            // Execute the action in-game
            NrnActionSpace.ConvertActions(agentAction);
        }
    }
}

Then at the end of the game, you can send the data you collected to the trainer platform as follows:

await Agent.UploadData();

OR

If you have the API access to train in-game, then you can directly train within the game loop as follows:

await AgentArc.Train();

The NRN team will work with each game studio on creating an optimal set of training configurations for their games