Action Space

In order for model's actions to convert to something executable in the game world, we need to map it to controller/keyboard inputs - we will call this the model's "action space".

In order to use the actions that an NRN agent recommends, we need to convert it into a format that the game can understand - this will be different for every game. Below we showcase an example of converting the output from an NRN model to a movement vector which can be used in a pong game:

Javascript

const convertActionToGame = (actions) => {
    const movementInput = new Vector3()
    if (actions.up) {
        movementInput.y = 1
    }
    else if (actions.down) {
        movementInput.y = -1
    }
    return movementInput
}

const getActionOneHot = (inputs) => {
    const action = [0, 0, 0]
    if (this.pressed["KeyW"]) {
        action[0] = 1 // Up
    }
    else if (this.pressed["KeyS"]) {
        action[1] = 1 // Down
    }
    else {
        action[2] = 1 // idle
    }
}

C# - Unity

using UnityEngine;

namespace NrnIntegration
{
    class NrnActionSpace
    {
        public static Vector3 ConvertActions(Dictionary<string, bool> actions)
        {
            Vector3 movementInput = new Vector3();
            if (actions["up"]) movementInput = new Vector3(0,1f,0);
            else if (actions["down"]) movementInput = new Vector3(0,-1f,0);
            else if (actions["idle"]) movementInput = new Vector3(0,0,0);                
            return movementInput;
        }

        public static Dictionary<string, bool> ConvertInputToBools()
        {
            Dictionary<string, bool> actions = new Dictionary<string, bool>();

            actions["up"] = Input.GetKey(KeyCode.W);
            actions["down"] = !actions["up"] && Input.GetKey(KeyCode.S);
            actions["idle"] = !actions["up"] && !actions["down"];
        
            return actions;
        }
    }
}

In the code shown above, we have 2 functions for action conversion:

• Model -> Game: This is used to execute the recommended action in-game

• Game -> Model: This is used to take human inputs and turn it into data that the model can use for training