TradeStation API

Source: Notion | Last edited: 2023-10-05 | ID: fcf46a37-4d3...

---

The following outlines the current state of the TradeStation API development.

The API follows an Access Token and Refresh Token pattern. Both access and refresh tokens are generated via a browser-based redirect URL, with the help of API key and Secret issued by TradeStation. Once the Access Token is generated, it’s only valid for 20 minutes, at which point the Refresh tokens need to be used to re-generate Access tokens.

Since we already generated both Access and Refresh tokens, I don’t think we need to ever repeat the initial browser-based (localhost redirect URL) authentication step.

(general note: all futures prices are 10 minutes delayed)

The boilerplate API authentication in python looks like this:

# Constants
TOKEN_ENDPOINT = 'https://signin.tradestation.com/oauth/token'
BARS_ENDPOINT = "https://api.tradestation.com/v3/marketdata/barcharts/"

CLIENT_ID = 'API Key'
CLIENT_SECRET = 'API Secret'
ACCESS_TOKEN = ''
REFRESH_TOKEN = ''

HEADERS = {
    "Authorization": f"Bearer {ACCESS_TOKEN}"
}

And the refreshing of the Access Token, once expired works like this:

def refresh_access_token(refresh_token):
    data = {
        'grant_type': 'refresh_token',
        'refresh_token': refresh_token,
        'client_id': CLIENT_ID,
        'client_secret': CLIENT_SECRET,
    }
    response = requests.post(TOKEN_ENDPOINT, data=data)
    response_data = response.json()

    if response.status_code != 200:
        raise Exception("Failed to refresh token. {}".format(response_data.getca('error_description')))

    return response_data['access_token']

API documentation can be found here:

https://api.tradestation.com/docs/specification

The five contracts that we have so far been interested in (Corn, Soybeans, Wheat, Yen, Light Sweet Crude Oil) are called with the following base names: “C”, “S”, “W”, “JY”, “CL”

but are then appended with Month and Year information depending on the contract we’re interested in. The Month Schema is the following:

# Example contract month/years:
SYMBOLS_DESCRIPTIONS = ['CU23', 'SU23', 'WU23', 'JYU23', 'CLU23']

Additional information about Futures Names can be found here:

https://help.tradestation.com/10_00/eng/tradestationhelp/symbology/futures_symbology.htm?_ga=2.243325379.793626684.1691617517-749679228.1690921689

Example Script for continual fetching of 1-minute data for our 5 contracts:

import requests
import datetime
import pandas as pd
import sqlite3
import time

# Constants
TOKEN_ENDPOINT = 'https://signin.tradestation.com/oauth/token'
BARS_ENDPOINT = "https://api.tradestation.com/v3/marketdata/barcharts/"
SYMBOLS_DESCRIPTIONS = ['CU23', 'SU23', 'WU23', 'JYU23', 'CLU23']
CLIENT_ID = 'API Key'
CLIENT_SECRET = 'API Secret'
ACCESS_TOKEN = ''
REFRESH_TOKEN = ''

HEADERS = {
    "Authorization": f"Bearer {ACCESS_TOKEN}"
}

# Initialize database connection
conn = sqlite3.connect('futures_data.db')

def refresh_access_token(refresh_token):
    data = {
        'grant_type': 'refresh_token',
        'refresh_token': refresh_token,
        'client_id': CLIENT_ID,
        'client_secret': CLIENT_SECRET,
    }
    response = requests.post(TOKEN_ENDPOINT, data=data)
    response_data = response.json()

    if response.status_code != 200:
        raise Exception("Failed to refresh token. {}".format(response_data.get('error_description')))

    return response_data['access_token']

def fetch_bars_for_symbol(symbol):
    # Updated bars_url to use barsback instead of firstdate and lastdate
    bars_url = f"{BARS_ENDPOINT}{symbol}?interval=1&unit=Minute&barsback=1"

    response = requests.get(bars_url, headers=HEADERS)

    if response.status_code == 401:
        print("Access token expired. Refreshing token for bars...")
        global ACCESS_TOKEN
        ACCESS_TOKEN = refresh_access_token(REFRESH_TOKEN)
        HEADERS['Authorization'] = f"Bearer {ACCESS_TOKEN}"
        response = requests.get(bars_url, headers=HEADERS)  # Retry the request

    if response.status_code == 200:
        bar_data = response.json()
        bars = bar_data.get('Bars', [])

        # Convert bars to pandas DataFrame
        df = pd.DataFrame(bars)

        # Save to database
        df.to_sql(symbol, conn, if_exists='append', index=False)

        # Print the dataframe
        print(df)
    else:
        print(f"Couldn't fetch bars for {symbol}")
        print(f"Status Code: {response.status_code}")
        print(f"Response Content: {response.content.decode('utf-8')}")

def main():
    while True:
        for symbol in SYMBOLS_DESCRIPTIONS:
            print(f"Fetching bars for {symbol}:")
            fetch_bars_for_symbol(symbol)
        time.sleep(60)  # Wait for 1 minute before the next fetch

if __name__ == "__main__":
    main()

The above code lacks the following:

1. Error handling like network outage etc.

2. Contract Month expiry and rollover (working on it…)

3. Security in terms of keeping Access Tokens unexposed? (This account belongs to Emil and has $250k in it, which we could trade with this API… so we have to be careful with security)

4. sometimes it fetches the same candle twice in a row, because I think that the script’s time is not synched with the exchange’s time, so the script’s timing moves around the minute, while the exchange times for new minute candles are set.

The format of the data is the following:

Contract Switching and Rollover

Switching to a new contract could be handled like so, for example (this is not fully tested yet):

(not all futures have all months available. Some futures for example, only have certain months, or quarterly contracts). For more information on specific futures contracts, see below.

import calendar

def get_expiry_date(symbol):
    """
    Get the expiration date for a given contract based on its symbol.
    """
    base_contract = symbol[0]
    month_code = symbol[1]
    year = int("20" + symbol[2:4])
    month = month_codes[month_code]

    if base_contract == 'C' or base_contract == 'S' or base_contract == 'W':
        # Trading terminates on the business day prior to the 15th day of the contract month.
        return datetime.date(year, month, 14)
    elif base_contract == 'JY':
        # Trading terminates 2 business days prior to the third Wednesday of the contract month.
        third_wednesday = [week[calendar.WEDNESDAY] for week in calendar.monthcalendar(year, month) if week[calendar.WEDNESDAY] != 0][2]
        return datetime.date(year, month, third_wednesday) - datetime.timedelta(days=2)
    elif base_contract == 'CL':
        # Trading terminates 3 business days before the 25th calendar day of the month prior to the contract month.
        if month == 1:
            month = 12
            year -= 1
        else:
            month -= 1
        if datetime.date(year, month, 25).weekday() >= 5:
            return datetime.date(year, month, 25) - datetime.timedelta(days=4)
        else:
            return datetime.date(year, month, 25) - datetime.timedelta(days=3)

def should_switch_contract(symbol):
    """
    Determine if today is the day to switch to the next contract.
    Arbitrarily set at 1 day before expiry
    """
    expiry_date = get_expiry_date(symbol)
    today = datetime.date.today()
    return today == (expiry_date - datetime.timedelta(days=1))

def get_next_contract(symbol):
    """
    Construct the next contract symbol based on the current contract symbol.
    """
    base_contract = symbol[0]
    month_code = symbol[1]
    year = int("20" + symbol[2:4])
    month = month_codes[month_code]

    if month == 12:
        next_month_code = 'F'
        year += 1
    else:
        next_month_index = list(month_codes.values()).index(month) + 1
        next_month_code = list(month_codes.keys())[next_month_index]

    return f"{base_contract}{next_month_code}{str(year)[2:]}"

month_codes = {
    'F': 1,
    'G': 2,
    'H': 3,
    'J': 4,
    'K': 5,
    'M': 6,
    'N': 7,
    'Q': 8,
    'U': 9,
    'V': 10,
    'X': 11,
    'Z': 12
}

# ...

def main():
    while True:
        for symbol in SYMBOLS_DESCRIPTIONS:
            if should_switch_contract(symbol):
                print(f"Switching from {symbol} due to contract expiry.")
                symbol = get_next_contract(symbol)
                print(f"Fetching bars for new contract: {symbol}")
            else:
                print(f"Fetching bars for {symbol}:")

            fetch_bars_for_symbol(symbol)
        time.sleep(60)  # Wait for 1 minute before the next fetch

1. TimeStamp: The date and time marking the end of the period for the bar data.
2. TotalVolume: The total number of contracts traded during the specified time period.
3. DownTicks: The number of times the price moved downward during the specified time period.
4. DownVolume: The volume of contracts traded during the periods of downward price movement.
5. OpenInterest: The number of futures contracts that have not yet been exercised, expired, or fulfilled by delivery. Open interest is used as an indicator of the strength of the price trend.
6. IsRealtime: Indicates whether the bar data is real-time or not. “TRUE” for real-time data and “FALSE” for delayed or historical data.
7. IsEndOfHistory: Indicates if this bar data represents the last available data point in the series. Often useful for understanding if more data is to be expected or not.
8. TotalTicks: The total number of price changes or ticks that occurred during the specified time period.
9. UnchangedTicks: The number of times the price remained unchanged during the specified time period.
10. UnchangedVolume: The volume of contracts traded during the periods when the price remained unchanged.
11. UpTicks: The number of times the price moved upward during the specified time period.
12. UpVolume: The volume of contracts traded during the periods of upward price movement.
13. Epoch: Represents the time in a machine-readable format, typically seconds since 1970-01-01 (the Unix epoch). It’s useful for algorithms or systems that work with time in a numeric format rather than a human-readable one.
14. BarStatus: Indicates the status of the bar. “Closed” means the time period for this data point has ended, while other potential statuses (not shown in your example) might be “Open” or “In-progress”, depending on the system providing the data.

Further Notes on Specific Futures Contracts

Oil / CL

Corn

Soybeans

Wheat

Japanese Yen

Trading Hours

Continued API work:

Here is a better script that continually fetches 1-minute bars for all commodities, plus BTC and ETH futures. It also handles monthly roll-over (although this might not be optimal), and saves everything to a database. Very minimal error-handling.

TODO:

• line up the script time with the exchange time, so that bars are fetched at the right time.
• perhaps fetch 2 or 3 months of each contract at all times, so that we have parallel data for several expiry months for each instrument Notes: continual fetching with TradeStation has some quirks: 1-minute bars are sometimes missing, which is due to the fact that no trades occurred during that time.

import requests
import datetime
import pandas as pd
import sqlite3
import time
import calendar
import re

# Constants
TOKEN_ENDPOINT = 'https://signin.tradestation.com/oauth/token'
BARS_ENDPOINT = "https://api.tradestation.com/v3/marketdata/barcharts/"
SYMBOLS_DESCRIPTIONS = ['C', 'S', 'W', 'JY', 'CL', 'BTC', 'ETH']
CLIENT_ID = 'A77K5Vk37dCJRdXQlaWALLRGIpP45mp0'
CLIENT_SECRET = ''
ACCESS_TOKEN = ''
REFRESH_TOKEN = ''

HEADERS = {
    "Authorization": f"Bearer {ACCESS_TOKEN}"
}

# Initialize database connection
conn = sqlite3.connect('futures_data.db')

# Initial month and year for each base symbol
current_contracts = {
    'C': {'month': 'U', 'year': '23'},
    'S': {'month': 'U', 'year': '23'},
    'W': {'month': 'U', 'year': '23'},
    'JY': {'month': 'U', 'year': '23'},
    'CL': {'month': 'U', 'year': '23'},
    'BTC': {'month': 'U', 'year': '23'},
    'ETH': {'month': 'U', 'year': '23'},  # Add the initialization for TBT
}

# Month Codes for Contract Symbols
month_codes = {
    'F': 1,
    'G': 2,
    'H': 3,
    'J': 4,
    'K': 5,
    'M': 6,
    'N': 7,
    'Q': 8,
    'U': 9,
    'V': 10,
    'X': 11,
    'Z': 12
}

def refresh_access_token(refresh_token):
    data = {
        'grant_type': 'refresh_token',
        'refresh_token': refresh_token,
        'client_id': CLIENT_ID,
        'client_secret': CLIENT_SECRET,
    }
    response = requests.post(TOKEN_ENDPOINT, data=data)
    response_data = response.json()

    if response.status_code != 200:
        raise Exception("Failed to refresh token. {}".format(response_data.get('error_description')))

    return response_data['access_token']

def get_contract_symbol(base_symbol):
    """
    Constructs the full contract symbol based on the base symbol and the current month/year mapping.
    """
    return base_symbol + current_contracts[base_symbol]['month'] + current_contracts[base_symbol]['year']

def fetch_bars_for_symbol(symbol):
    # Updated bars_url to use barsback instead of firstdate and lastdate
    bars_url = f"{BARS_ENDPOINT}{symbol}?interval=1&unit=Minute&barsback=1"

    response = requests.get(bars_url, headers=HEADERS)

    if response.status_code == 401:
        print("Access token expired. Refreshing token for bars...")
        global ACCESS_TOKEN
        ACCESS_TOKEN = refresh_access_token(REFRESH_TOKEN)
        HEADERS['Authorization'] = f"Bearer {ACCESS_TOKEN}"
        response = requests.get(bars_url, headers=HEADERS)  # Retry the request

    if response.status_code == 200:
        bar_data = response.json()
        bars = bar_data.get('Bars', [])

        # Convert bars to pandas DataFrame
        df = pd.DataFrame(bars)

        # Save to database
        df.to_sql(symbol, conn, if_exists='append', index=False)

        # Print the dataframe
        #print(df)
    else:
        print(f"Couldn't fetch bars for {symbol}")
        print(f"Status Code: {response.status_code}")
        print(f"Response Content: {response.content.decode('utf-8')}")

def get_expiry_date(symbol):
    """
    Get the expiration date for a given contract based on its symbol.
    """
    # Extract month code and year using regex
    match = re.search(r'([A-Z])(\d+)$', symbol)
    if not match:
        raise ValueError(f"Invalid symbol format: {symbol}")

    month_code = match.group(1)
    year = int("20" + match.group(2))

    # Base contract is the remaining part of the symbol after removing month code and year
    base_contract = symbol[:match.start()]

    month = month_codes[month_code]

    if base_contract in ['C', 'S', 'W', 'BTC', 'ETH']: #BTC and ETH might need separate rules
        # Trading terminates on the business day prior to the 15th day of the contract month.
        return datetime.date(year, month, 14)
    elif base_contract == 'JY':
        # Trading terminates 2 business days prior to the third Wednesday of the contract month.
        third_wednesday = [week[calendar.WEDNESDAY] for week in calendar.monthcalendar(year, month) if week[calendar.WEDNESDAY] != 0][2]
        return datetime.date(year, month, third_wednesday) - datetime.timedelta(days=2)
    elif base_contract == 'CL':
        # Trading terminates 3 business days before the 25th calendar day of the month prior to the contract month.
        if month == 1:
            month = 12
            year -= 1
        else:
            month -= 1
        if datetime.date(year, month, 25).weekday() >= 5:
            return datetime.date(year, month, 25) - datetime.timedelta(days=4)
        else:
            return datetime.date(year, month, 25) - datetime.timedelta(days=3)

def should_switch_contract(base_symbol):
    """
    Determine if today is the day to switch to the next contract.
    Arbitrarily set at 1 day before expiry.
    """
    symbol = get_contract_symbol(base_symbol)
    expiry_date = get_expiry_date(symbol)
    today = datetime.date.today()
    return today == (expiry_date - datetime.timedelta(days=1))

def get_next_contract(base_symbol):
    """
    Update the current month/year mapping for the base symbol to the next contract's month/year.
    """
    current_month_code = current_contracts[base_symbol]['month']
    current_year = int(current_contracts[base_symbol]['year'])

    # Check if current_month_code is the last in the month_codes list
    if current_month_code == 'Z':
        next_month_code = 'F'  # Reset to the first month code
        current_year += 1
    else:
        next_month_index = list(month_codes.keys()).index(current_month_code) + 1
        next_month_code = list(month_codes.keys())[next_month_index]

    # Ensure that the year is always two digits
    year_str = str(current_year)[-2:]

    current_contracts[base_symbol]['month'] = next_month_code
    current_contracts[base_symbol]['year'] = year_str

def main():
    while True:
        for base_symbol in SYMBOLS_DESCRIPTIONS:
            contract_symbol = get_contract_symbol(base_symbol)
            if should_switch_contract(base_symbol):
                print(f"Switching from {contract_symbol} due to contract expiry.")
                get_next_contract(base_symbol)
                contract_symbol = get_contract_symbol(base_symbol)
                print(f"Fetching bars for new contract: {contract_symbol}")
            else:
                print(f"Fetching bars for {contract_symbol}:")

            fetch_bars_for_symbol(contract_symbol)
        time.sleep(60)  # Wait for 1 minute before the next fetch

if __name__ == "__main__":
    main()

occasional server errors with this script:

Fetching bars for BTCU23:
Couldn't fetch bars for BTCU23
Status Code: 500
Response Content: {"Error":"InternalServerError","Message":"ERROR - The operation has timed out."}

Fetching bars for ETHU23:
Couldn't fetch bars for ETHU23
Status Code: 500
Response Content: {"Error":"InternalServerError","Message":"ERROR - The operation has timed out."}

Getting Historical Data with Individual Contract Logic

The below script seems to work for fetching historical data, and allows for logic adjustment for each contract individually, if required (i.e. when the rollover date is, etc.). This script throws a lot of errors but I think that is when it tries to query weekends or holidays for which there is no data. It just continues to run.

TradeStation’s Symbology can be found here: https://www.tradestation.com/learn/market-basics/futures-options/symbology/futures-plus-symbology/

import requests
import pandas as pd
import datetime
import os

# Constants
TOKEN_ENDPOINT = 'https://signin.tradestation.com/oauth/token'
BARS_ENDPOINT = "https://api.tradestation.com/v3/marketdata/barcharts/"
SYMBOLS_DESCRIPTIONS = ['C', 'S', 'W', 'JY', 'CL']
CLIENT_ID = 'A77K5Vk37dCJRdXQlaWALLRGIpP45mp0'
CLIENT_SECRET = ''
ACCESS_TOKEN = ''
REFRESH_TOKEN = ''

HEADERS = {
    "Authorization": f"Bearer {ACCESS_TOKEN}"
}

month_codes = {
    'F': 1,
    'G': 2,
    'H': 3,
    'J': 4,
    'K': 5,
    'M': 6,
    'N': 7,
    'Q': 8,
    'U': 9,
    'V': 10,
    'X': 11,
    'Z': 12
}

# Functions from the TradeStation script
def refresh_access_token(refresh_token):
    data = {
        'grant_type': 'refresh_token',
        'refresh_token': refresh_token,
        'client_id': CLIENT_ID,
        'client_secret': CLIENT_SECRET,
    }
    response = requests.post(TOKEN_ENDPOINT, data=data)
    response_data = response.json()
    if response.status_code != 200:
        raise Exception("Failed to refresh token. {}".format(response_data.get('error_description')))
    return response_data['access_token']

def fetch_bars_for_symbol(base_symbol, symbol, start_date, end_date, contract_month):
    first_date_str = start_date.strftime('%Y-%m-%dT%H:%M:%SZ')
    last_date_str = end_date.strftime('%Y-%m-%dT%H:%M:%SZ')
    bars_url = f"{BARS_ENDPOINT}{symbol}?interval=5&unit=Minute&firstdate={first_date_str}&lastdate={last_date_str}&sessiontemplate=USEQPreAndPost"
    response = requests.get(bars_url, headers=HEADERS)

    # Handle token expiration and refresh
    if response.status_code == 401:
        print("Access token expired. Refreshing token for bars...")
        global ACCESS_TOKEN
        ACCESS_TOKEN = refresh_access_token(REFRESH_TOKEN)
        HEADERS['Authorization'] = f"Bearer {ACCESS_TOKEN}"
        response = requests.get(bars_url, headers=HEADERS)  # Retry the request

    if response.status_code == 200:
        bar_data = response.json()
        bars = bar_data.get('Bars', [])
        df = pd.DataFrame(bars)
        df['ContractMonth'] = contract_month
        filename = f"{base_symbol}_continuous_contract_TS_2022_August_2023.csv"
        filepath = os.path.join("./TS2YearOutput_V3/", filename)

        # Check if the file exists and has headers
        header = not os.path.exists(filepath)

        df.to_csv(filepath, mode='a', header=header, index=False)
    else:
        print(f"Couldn't fetch bars for {symbol}")
        print(f"Status Code: {response.status_code}")
        print(f"Response Content: {response.content.decode('utf-8')}")

# Define the logic for each symbol
# Individual functions for each symbol

#Japanese Yen
def get_JY_contract(today):
    if today.month < 2 or (today.month == 2 and today.day <= 10):
        return 3, today.year
    elif today.month < 5 or (today.month == 5 and today.day <= 10):
        return 6, today.year
    elif today.month < 8 or (today.month == 8 and today.day <= 10):
        return 9, today.year
    else:
        return 12, today.year

# Corn
def get_C_contract(today):
    return get_generic_contract(today, [3, 5, 7, 9, 12])

# Soybeans
def get_S_contract(today):
    return get_generic_contract(today, [3, 5, 7, 9, 12])

# Wheat
def get_W_contract(today):
    return get_generic_contract(today, [3, 5, 7, 9, 12])

# Oil
def get_CL_contract(today):
    if today.day < 10:
        contract_month = (today.month % 12) + 1
        contract_year = today.year if today.month != 12 else today.year + 1
    else:
        contract_month = ((today.month + 1) % 12) + 1
        contract_year = today.year if today.month != 11 else today.year + 1
    return contract_month, contract_year

def get_generic_contract(today, traded_months):
    if today.day <= 10:
        contract_month = today.month
    else:
        contract_month = (today.month % 12) + 1

    while contract_month not in traded_months:
        contract_month = (contract_month % 12) + 1

    contract_year = today.year
    if contract_month < today.month:
        contract_year += 1

    return contract_month, contract_year

# Main function
def main():
    current_date = START_DATE
    while current_date <= END_DATE:
        for base_symbol in SYMBOLS_DESCRIPTIONS:
            contract_month, contract_year = globals()[f"get_{base_symbol}_contract"](current_date)

            month_code = [k for k, v in month_codes.items() if v == contract_month][0]
            contract_symbol = f"{base_symbol}{month_code}{str(contract_year)[-2:]}"
            fetch_bars_for_symbol(base_symbol, contract_symbol, current_date, current_date + datetime.timedelta(days=1), contract_month)
        current_date += datetime.timedelta(days=1)

# Main execution
if __name__ == "__main__":
    main()

Order Execution

The below script implements a Class that handles account information and order execution. Importantly, one must be careful to set the correct base-url to be either SIM (paper) or Live trading. In the code below, we have the SIM trading url in place.

This class can:

1. get accounts available with the given API

2. get account balances

3. get positions

4. get orders (this fetches orders of the day, both executed and pending)

5. confirm orders: This estimates cost and price of an oder, without actually executing it.

6. place orders (market and limit implemented so far, stoplimit, and stopmarket not yet…)

7. replace orders (not yet implemented)

8. cancel orders

9. cancel all unfilled orders

see API specifications for more details

import requests
import re
import time

class ExecutionClass:
    def __init__(self, client_id, client_secret, refresh_token):
        self.TOKEN_ENDPOINT = 'https://signin.tradestation.com/oauth/token'
        self.ACCOUNT_ENDPOINT = "https://sim-api.tradestation.com/v3/brokerage/accounts"
        self.EXECUTION_ENDPOINT = "https://sim-api.tradestation.com/v3/orderexecution"
        self.MARKETDATA_ENDPOINT = "https://sim-api.tradestation.com/v3/marketdata"
        self.CLIENT_ID = ''
        self.CLIENT_SECRET = ''
        self.ACCESS_TOKEN = ''
        self.REFRESH_TOKEN = ''
        self.ACCOUNTS = ['SIM1530338X', 'SIM1530337F', 'SIM1530336M'] # account types: Forex, Futures, Margin
        self.HEADERS = {
            "Authorization": f"Bearer {self.ACCESS_TOKEN}"
        }
        self.refresh_access_token()

    def refresh_access_token(self):
        data = {
            'grant_type': 'refresh_token',
            'refresh_token': self.REFRESH_TOKEN,
            'client_id': self.CLIENT_ID,
            'client_secret': self.CLIENT_SECRET,
        }
        response = requests.post(self.TOKEN_ENDPOINT, data=data)
        response_data = response.json()

        if response.status_code != 200:
            raise Exception("Failed to refresh token. {}".format(response_data.get('error_description')))

        self.ACCESS_TOKEN = response_data['access_token']
        self.HEADERS['Authorization'] = f"Bearer {self.ACCESS_TOKEN}"

    #generic request function
    def make_request(self, method, url, json):
        response = requests.request(method, url, json=json, headers=self.HEADERS)

        if response.status_code == 401:
            print("Access token expired. Refreshing token...")
            self.refresh_access_token()
            response = requests.request(method, url, json=json, headers=self.HEADERS)

        if response.status_code == 200:
            return response.json()
        else:
            return None

    def get_account(self):
        data = self.make_request("GET", self.ACCOUNT_ENDPOINT, None)
        if data:
            #print(data)
            return data

    def get_balances(self, account):
        url = f"{self.ACCOUNT_ENDPOINT}/{account}/balances"
        data = self.make_request("GET", url, None)
        if data:
            #print(f"{account} Current Balances:")
            #print(data)
            return data

    def get_positions(self, account):
        url = f"{self.ACCOUNT_ENDPOINT}/{account}/positions"
        data = self.make_request("GET", url, None)
        if data:
            #print(f"{account} Current Positions:")
            #print(data)
            return data

    # Function to get the quantity for a given symbol from the positions_data delived by get_positions()
    def get_quantity_for_symbol(self, positions_data, target_symbol):
        for position in positions_data.get("Positions", []):
            if position["Symbol"] == target_symbol:
                return int(position["Quantity"])
        print(f"No position found for symbol: {target_symbol}")
        return 0

    def get_orders(self, account):
        url = f"{self.ACCOUNT_ENDPOINT}/{account}/orders"
        data = self.make_request("GET", url, None)
        if data:
            #print(f"{account} Today's Orders:")
            #print(data)
            return data

    # Returns estimated cost and commission information for an order without the order actually being placed.
    def confirm_order(self, account, symbol, quantity, order_type, trade_action, duration, limit_price=None):
        url = f"{self.EXECUTION_ENDPOINT}/orderconfirm"

        payload = {
            "AccountID": f"{account}",
            "Symbol": f"{symbol}",
            "Quantity": f"{quantity}",
            "OrderType": f"{order_type}", # Valid Types: Market, Limit, Stop Market, Stop Limit, Options, and Order Sends Order (OSO)
            "TradeAction": f"{trade_action}",
            "TimeInForce": {"Duration": f"{duration}"},
            "Route": "Intelligent"
        }

        # Include LimitPrice only if it is not None (i.e., for Limit orders)
        if limit_price is not None:
            payload["LimitPrice"] = f"{limit_price}"

        headers = {
            "content-type": "application/json",
            "Authorization": "Bearer TOKEN"
        }

        data = self.make_request("POST", url, payload)
        if data:
            #print(f"{account} Order Confirm:")
            #print(data)
            return data

    ## !!! Place and Order !!! (Make sure to check if you are in SIM or LIVE Environment!)
    def place_order(self, account, symbol, quantity, order_type, trade_action, duration, limit_price=None):
        url = f"{self.EXECUTION_ENDPOINT}/orders"

        payload = {
            "AccountID": f"{account}",
            "Symbol": f"{symbol}",
            "Quantity": f"{quantity}",
            "OrderType": f"{order_type}", # Valid Types: Market, Limit, Stop Market, Stop Limit, Options, and Order Sends Order (OSO)
            "TradeAction": f"{trade_action}", #BUY, SELL, ...
            "TimeInForce": {"Duration": f"{duration}"}, # DAY, ...
            "Route": "Intelligent"
        }

        # Include LimitPrice only if it is not None (i.e., for Limit orders)
        if limit_price is not None:
            payload["LimitPrice"] = f"{limit_price}"

        headers = {
            "content-type": "application/json",
            "Authorization": "Bearer TOKEN"
        }

        data = self.make_request("POST", url, payload)
        if data:
            print(f"{account} Order Execution (!!!):")
            print(data)
            return data

    # replace order
    def replace_oder(self, order_ID):
        return

    # cancel an order given its order ID and account
    def cancel_order(self, order_ID, account):
        url = f"{self.EXECUTION_ENDPOINT}/orders/{order_ID}"
        data = self.make_request("DELETE", url, None)
        if data:
            print(f"{account} Order Cancellation:")
            print(data)
            return data

    def close_all_positions(self, account):
        # Get all positions
        positions_data = self.get_positions(account)

        if not positions_data or 'Positions' not in positions_data:
            print(f"{account} No open positions to close.")
            return

        # Loop through each position
        for pos in positions_data['Positions']:
            symbol = pos['Symbol']
            quantity = abs(int(pos['Quantity']))  # Make sure the quantity is positive
            long_short = pos['LongShort']

            # Determine the trade action (Buy if short, Sell if long)
            trade_action = "BUY" if long_short == "Short" else "SELL"

            # Place order to close the position
            print(f"Attempting to close position: {symbol}, {quantity}, {trade_action}")
            order_response = self.place_order(account, symbol, quantity, "Market", trade_action, "DAY")

            print("Order response:")
            print(order_response)

    # Function to cancel all unfilled orders
    def cancel_unfilled_orders(self, orders, account):
        for order in orders.get('Orders', []):
            legs = order.get('Legs', [])
            for leg in legs:
                if int(leg['QuantityRemaining']) > 0:
                    symbol = leg['Symbol']
                    order_ID = order['OrderID']
                    print(f"Cancelling unfilled order for {symbol} with Order ID: {order_ID}")

                    # Cancel the order
                    self.cancel_order(order_ID, account)

    ## function that calculates and returns initial margin requirements
    ## there doesnt seem to be a direct API method for this so we have to calculate ourselves
    def calculate_initial_margin_requirements(self, symbol_full):
        # Define the multipliers for each contract
        multipliers = {
            "BTC": 5,
            "ETH": 50,
            "CL": 1000,
            "C": 5000,
            "S": 5000,
            "JY": 12500000/100,
            "W": 5000
        }

        # Extract the base symbol by removing the last three characters
        base_symbol = symbol_full[:-3]

        # Get the multiplier based on the base symbol
        multiplier = multipliers.get(base_symbol, None)
        if multiplier is None:
            print(f"Unknown base symbol: {base_symbol}")
            return

        # Make the API call to get the estimated price and cost
        order_data = self.confirm_order(account="SIM1530337F", symbol=symbol_full,
                                        quantity=1, order_type="Market",
                                        trade_action="BUY", duration="DAY")

        estimated_price = float(order_data['Confirmations'][0]['EstimatedPrice'])
        estimated_cost = float(order_data['Confirmations'][0]['EstimatedCost'].replace(',', ''))

        # Special handling for futures quoted with fractional prices and in cents
        if base_symbol in ["C", "S", "W"]:
            estimated_price = estimated_price / 100  # Convert from cents to dollars

        # Calculate the initial margin requirement
        initial_margin_requirement = (estimated_cost / (estimated_price * multiplier))
        print(f"Initial Margin Requiremnt for {symbol_full}: {initial_margin_requirement * 100}%")
        return initial_margin_requirement

    #get quote snapshot, bid, ask, etc.
    def get_quote_snapshot(self,symbol_full):
        url = f"{self.MARKETDATA_ENDPOINT}/quotes/{symbol_full}"
        data = self.make_request("GET", url, None)
        if data:
            # Extract the required fields from the data
            last_price = float(data['Quotes'][0]['Last'])
            ask_price = float(data['Quotes'][0]['Ask'])
            ask_size = int(data['Quotes'][0]['AskSize'])
            bid_price = float(data['Quotes'][0]['Bid'])
            bid_size = int(data['Quotes'][0]['BidSize'])
            # Extract market flags
            is_delayed = data['Quotes'][0]['MarketFlags']['IsDelayed']
            is_halted = data['Quotes'][0]['MarketFlags']['IsHalted']

            return {
                'Last': last_price,
                'Ask': ask_price,
                'AskSize': ask_size,
                'Bid': bid_price,
                'BidSize': bid_size,
                'IsDelayed': is_delayed,
                'IsHalted': is_halted # indictates if Market is Open or not
            }

    #returns true if if market is currently open
    def is_market_open(self, symbol_full):
        quote_snapshot_data = self.get_quote_snapshot(symbol_full)
        #print(not quote_snapshot_data['IsHalted'])
        return not quote_snapshot_data['IsHalted']

    def main(self):
        self.get_account()
        #for account in self.ACCOUNTS:
            #self.get_balances(account)
            #self.get_positions(account)
            #self.get_orders(account)

## Test all of the important methods of the Execution Class:
def run_trade_simulation(account, symbol_full):
    # check account balance
    print("Getting Account Balances: ")
    execution.get_balances(account)
    print()

    # check order history
    print("Check Order History: ")
    execution.get_orders(account)
    print()

    # get last quotes
    print("Last Quotes:")
    quote_snapshot = execution.get_quote_snapshot(symbol_full)
    print()

    # place limit order
    print("Placing Limit order at Ask Price")
    execution.place_order(account, symbol_full, 1, "Limit", "BUY", "DAY", limit_price=quote_snapshot["Ask"])
    print()

    print("Waiting for a few seconds so that the order can execute... ")
    time.sleep(2)

    # check orders
    print("Checking Order History Again:")
    orders = execution.get_orders(account)
    print()

    # get positions
    print("Get Positions")
    positions_data = execution.get_positions(account)
    print()

    # get position quanity for our symbol
    print(f"Position Quantity for {symbol_full}:")
    quantity = execution.get_quantity_for_symbol(positions_data, symbol_full)
    print(quantity)
    print()

    # if there is a position, sell it.
    if quantity > 0:
        print("Placing market sell order...")
        execution.place_order(account, symbol_full, quantity, "Market", "SELL", "DAY")

    # check account balance
    print("Getting Account Balances Again: ")
    execution.get_balances(account)
    print()

    time.sleep(2)
    # finally, if there are unfilled orders, cancel them:
    # check orders
    print("Checking Order History Again:")
    orders = execution.get_orders(account)
    print()

    ## cancel unfilled orders:
    print("cancelling unfilled orders...")
    execution.cancel_unfilled_orders(orders, account)

    return

#testing the close all positions function
#also testing is_market_open() function
def run_trade_simulation_v2(account, symbol_list):
    for s in symbol_list:
        print(f"Market for {s} is open? {execution.is_market_open(s)}")

    if execution.is_market_open(symbol_list[0]):
        execution.place_order(account, symbol_list[0], 1, "Market", "BUY", "DAY")

    if execution.is_market_open(symbol_list[1]):
        execution.place_order(account, symbol_list[1], 2, "Market", "SELL", "DAY")

    positions = execution.get_positions(account)
    #print(positions)
    orders = execution.get_orders(account)
    #print(f"Orders: {orders}")

    print("cancelling unfilled orders...")
    execution.cancel_unfilled_orders(orders, account)

    print("Attempting to close all positions..... ")
    execution.close_all_positions(account)
    positions = execution.get_positions(account)
    print(positions)
    print(execution.get_balances(account))
    return

if __name__ == '__main__':
    execution = ExecutionClass('API Key', 'API Secret', 'Refresh Token')
    execution.main()
    #run_trade_simulation('SIM1530337F','BTCU23')
    run_trade_simulation_v2('SIM1530337F',['BTCU23', 'ETHU23'])

    #execution.get_quote_snapshot("BTCU23")
    #execution.confirm_order('SIM1530337F', "BTCU23",1,"Market", "BUY", "DAY")
    #execution.calculate_initial_margin_requirements("BTCU23")
    #execution.calculate_initial_margin_requirements("ETHU23")
    #execution.calculate_initial_margin_requirements("JYU23")
    #execution.calculate_initial_margin_requirements("CLV23")
    #execution.calculate_initial_margin_requirements("CZ23")
    #execution.calculate_initial_margin_requirements("WZ23")
    #execution.calculate_initial_margin_requirements("SU23")