Model Architectures

Inductive Bias

We have 2 models currently in production for NRN Agents:

1. Feedforward Neural Network: This model is well-suited for most environments that have a continuous state space. The state space for this model must have the ability to be represented as a vector.

2. Tabular Agent: This model is specifically designed for environments in which we can define the state space as a set of discrete scenarios.

We have 2 other models in the final stages of testing and will be released to production very soon:

1. Hierarchical Neural Network

2. Convolutional Neural Network

Since most games require agents to perform multiple actions simultaneously, our models are all capable of multi-task learning

State Space

When registering a model, we need to know the size of the state space. In the case of a feedforward neural network, this is the number of features in the input vector. While for the tabular agent, this is the number of discrete scenarios.

When starting the process, the NRN team will work collaboratively with the game studio to figure out which features are important for decision making and design a state space around that.

Learn more about state spaces here.

Action Space

As part of the initial research, we have to figure out all the actions a player can take in the game and come up with an action space for the agent. The end result will largely depend on how many actions an agent can take simultaneously and whether the actions are discrete or continuous.

Learn more about action spaces here.

Model Size

The size of the network determines the degrees of freedom that a model has to learn an objective. Generally speaking, the larger the neural network, the higher likelihood it can learn "more". Thus, as the complexity of the games increase, so will the size of the neural networks that we decide to deploy.

Part of the research process will be testing various model sizes to find the perfect fit for the game. We want the model to be large enough to behave intelligently in the game, yet small enough to perform inference very fast and not cause frame drop. A member of the NRN team will be advising each game on the appropriate model size.