Basic Integration
As mentioned in the setup section, game studios will be required to create 2 files in order for the NRN Agent to "understand" the game world, and for the game world to "understand" the NRN Agent output. The NRN team will provide partner game studios with template files for Javascript, Unity, and Unreal.
After instantiating the NRN Agent, the basic flow for integration is as follows:
Convert the game world into an state space that the NRN model can use
Human Controlled
Update human inputs
Convert executed inputs to an action that the NRN model can understand
Add the state and action pair to the dataset
NRN Agent Controlled
Perform inference to select the action
Convert the selected action to an input that the game can understand
Either send data to the trainer platform or train directly in-game
Below we show how to initialize the model and use it at the game manager level:
class Game {
// Instantiate agent
constructor(agentInputs) {
this.agent = AgentFactory.createAgent(...agentInputs);
}
// Game loop update
update() {
// Get the state space for the model
const state = getStateSpace(this);
if (this.human) {
// If human is controlling, then update their inputs
this.inputs.p1.update();
// Convert the inputs into something the NRN agent can understand
const action = getActionOneHot(this.inputs.p1);
// Add the observed state and action to the dataset
this.agent.collect({ state, action });
}
else {
// If NRN agent is controlling, then perform inference
const agentAction = this.agent.selectAction(state);
// Execute the action in-game
convertActionToGame(agentAction);
}
}
}Then at the end of the game, you can send the data you collected to the trainer platform as follows:
await agent.uploadData();OR
If you have the API access to train in-game, then you can directly train within the game loop as follows:
await agent.train(model.getTrainingData(), trainingConfiguration);public class Game : MonoBehaviour
{
// Define the agent that will be taking actions
public IAgentRL Agent { get; set; }
// Instantiate model
private async void _loadAgent(modelInputs)
{
Agent = AgentFactory.CreateAgent<IAgentRL>("reinforcement", modelInputs);
}
// Game loop update
private async void Update() {
// Create the world representation which we will convert to the state space matrix
var world = new NrnIntegration.NrnWorld
{
// All world attributes/objects that are relevant for the AI
};
// Get the state space for the model
Matrix state = NrnIntegration.NrnStateSpace.GetState(world);
if (human)
{
// Convert the inputs into something the NRN agent can understand
Dictionary<string, bool> action = NrnActionSpace.ConvertInputToBools();
// Add the observed state and action to the dataset
Agent.Collect({ state, action });
}
else {
// If NRN agent is controlling, then perform inference
Dictionary<string, bool> agentAction = Agent.SelectAction(state);
// Execute the action in-game
NrnActionSpace.ConvertActions(agentAction);
}
}
}Then at the end of the game, you can send the data you collected to the trainer platform as follows:
await Agent.UploadData();OR
If you have the API access to train in-game, then you can directly train within the game loop as follows:
await AgentArc.Train();Last updated