Simulated Exchange - Notes from Ron

Source: Notion | Last edited: 2022-10-26 | ID: c2f50de6-d17...

---

Goals

1. Simulate the exchange/agent interaction at the highest possible speed
2. Simulation must be accurate enough for the results to be trustworthy and usable in real trading.

Assumptions and Constraints

1. Minimize simulator-specific code paths in the agent as much as possible, to preserve confidence and reduce work
2. In particular, the agent should communicate with the simulator using the same protocols it uses with a real exchange, i.e. HTTP JSON API and Websocket connection.
3. Need access to full order book (depth up to 1,000) and trades stream at arbitrary timestamps.
4. An instance of the simulator needs to support only one trading agent at a time.

Approach

Two design approaches have been proposed: mocked timestamps, or accelerated replay.

Mocked timestamps

In this approach, to every message between the agent and simulator we append a simulator-only parameter that specifies the timestamp when the message was supposedly generated. The initial timestamp is user-specified, and has no relation to real world time. Each side uses this value to update an internal mock clock. The clock only changes  when a message is generated or received. The clock only goes forward, never backward.

Advantages:

1. All else being equal, this is the fastest way to run the simulation, because neither side needs to stall waiting for the right time to perform their next action.

2. Agent and simulator are always in agreement about where they are in the simulation. No risk of skew.

3. Runs are reproducible, minimal risk of race conditions or lost messages Disadvantages:

4. Likely requires substantial doctoring of the agent side. Specifically:

• When sending HTTP requests: need to pass a timestamp
• When receiving websocket updates: need to compare their time stamp to the current mock time, and update the mock clock before handling them.
• In theory, HTTP responses and websocket updates could arrive out of order. We may be able to guarantee that this doesn’t happen. Otherwise, messages would need to be sorted and handled according to their mock timestamps.

1. Need to plumb at least the timestamp param and potentially other data via CCXT. There’s some support for this but I don’t know if it’s enough.

Accelerated Replay

In this approach, we define a speedup factor for our simulation to run relative to what its real world pace would have been. For example, the client and simulator agree to run at 100x real time. Requests going back and forth do not carry timestamps beyond what they normally would per the official API. Tick-by-tick updates from the simulator are driven by timed events so that they arrive at the agreed-upon pace

Advantages:

1. Much less intrusive on the client side. Will need some custom code to set the speedup factor. Also any client operations that are time-based (as opposed to event-driven) would need to take the speedup into account Disadvantages:

2. See the advantages section of the mocked timestamps approach above

3. Unclear whether we’d be able to send tick-by-tick update data fast enough. It depends on the types of tick-by-tick data required. See questions regarding this in the Q&A section below.

Q&A

1. Confirm that we need depth 1000 order book [A: yes!]
2. In production how is the order book data updated?

• CCXT listening to tick by tick updates and keeps a local cache always up to date
• CCXT sends HTTP requests when the client actually retrieves order book data
• Something else?

1. Same as above but for recent trades [A: no need for the client to subscribe or retrieve this]
2. Websocket messages: need a list of the types of updates the client currently listens to. For each type, we need to know:

• Is the message handled by our code, or by CCXT behind the scenes
• If it’s handled by CCXT, is it possible to introduce mock timestamp handling?
• If not, is it possible to run the simulation without consuming this type of message at all?