Action Space

In order for model's actions to convert to something executable in the game world, we need to map it to controller/keyboard inputs - we will call this the model's "action space".

In order to use the actions that an NRN agent recommends, we need to convert it into a format that the game can understand - this will be different for every game. Below we showcase an example of converting the output from an NRN model to a movement vector which can be used in a pong game:

Javascript

const convertActionToGame = (actions) => {
    const movementInput = new Vector3()
    if (actions.up) {
        movementInput.y = 1
    }
    else if (actions.down) {
        movementInput.y = -1
    }
    return movementInput
}

const getActionOneHot = (inputs) => {
    const action = [0, 0, 0]
    if (this.pressed["KeyW"]) {
        action[0] = 1 // Up
    }
    else if (this.pressed["KeyS"]) {
        action[1] = 1 // Down
    }
    else {
        action[2] = 1 // idle
    }
}

C# - Unity

In the code shown above, we have 2 functions for action conversion:

• Model -> Game: This is used to execute the recommended action in-game

• Game -> Model: This is used to take human inputs and turn it into data that the model can use for training