About

About

This project is a multi-agent market simulator designed to explore the emergent dynamics of heterogeneous trading agents. Each agent seeks to maximize personal profit while interacting in a sandbox environment. Agents place buy/sell orders against one another, and market prices arise naturally from supply–demand forces.

Notably, no external shocks are injected: the entire trajectory of the market is determined by the initial configuration.

Each agent observes:

• Price vector
• Volume vector
• Short-horizon volatility vector
• Own cash and inventory

Agent types include Reinforcement Learning (RL) Traders, Momentum Traders, Mean-Reversion Traders, and Market Makers. Current RL traders begin naïve, with policies learned entirely from in-simulation historical data—making the environment a genuine RL sandbox.

Research focus: emergent microstructure phenomena such as spread dynamics, depth formation, volatility clustering, and regime shifts induced by heterogeneous interactions. Given determinism, we are particularly interested in the possibility of deterministic non-periodic flow (chaos).

Agent Types

Reinforcement Learning Traders

RL agents use a dueling Q-network with experience replay and soft target updates. They start from naïve policies but learn adaptively from simulation history. Unlike rule-based agents, RL traders explore new behaviors — sometimes converging to profitable niches, sometimes destabilizing the market. This makes them ideal for studying how adaptive strategies interact with fixed heuristics.

Key features: ε-greedy exploration, Huber loss stabilization, and reproducible seeding. They capture the dynamics of “learning under feedback” often seen in real markets.

Momentum Traders

Momentum traders exploit directional trends. They compare short-term and long-term moving averages and act when the difference exceeds a threshold. If (STMA – LTMA) > threshold they buy, and if (LTMA – STMA) > threshold they sell.

While simple, these agents reinforce existing trends, often causing runs of one-sided order flow. They are useful for creating volatility spikes and bubbles within the simulation.

Mean-Reversion Traders

Mean-reverters bet on price returning to a rolling mean, normalized by volatility. A large positive deviation triggers a sell; a large negative deviation triggers a buy.

These agents act as natural stabilizers, counterbalancing momentum flow. They generate realistic microstructure features such as tighter spreads and rapid corrections when prices deviate too far from equilibrium.

Market Makers

Market makers continuously provide two-sided liquidity around the mid-price. Their spreads and sizes adjust with volatility and inventory, ensuring they remain solvent while keeping the market tradable.

They thicken the order book and reduce transaction costs for others. Without them, markets in the simulation quickly dry up or become unstable. By interacting with momentum and mean-reversion flows, market makers are central to generating realistic spread and depth dynamics.

Simulation Flow

1. Agents observe the current state (prices, volumes, volatilities, balances).
2. Each agent decides on an action and places a buy/sell/hold order.
3. Orders enter the limit order book, matched by price–time priority.
4. Trades update balances and inventories; RL agents update policies via replay.