Skip to content
Eon-Labs KB

Quantiphi Questionnaire

Source: Notion | Last edited: 2022-12-31 | ID: 6b0a7b97-f82...

  1. Which technique is used for forecasting of the base model?
  2. LTSM
  3. What is the current accuracy observed with this technique?
  4. We use a few simple objective functions, such as Calmar Ratio and maximum drawdown.
  5. How many variables are considered in this technique and what more do we need to consider with this revised model?
  6. Open High Low Close Volume (OHLCV) are currently used. Some sentiment indicators are being investigated.
  7. We believe OHLCV are the main inputs. Perhaps OHLCV of related instruments can help somehow but we haven’t tried.
  8. How frequently are models trained?
  9. New models are trained monthly but we don’t easily switch to new ones.
  10. What is the source of the data for model training and inference?
  11. Binance OHLC minute-based data, e.g. 15-min, 30-min, etc
  12. What is the type of data being used - images/text/tabular?
  13. time series price data
  14. Is the data pre-processed? Any EDA, Feature Engineering is to be done?
  15. basic technical analysis such as moving averages are used as features
  16. How much data do we need for model inference per model?
  17. This varies, the recent one we use is 5-8 years for training, 6 months for verification, and 6 months for testing
  18. How are model objects currently saved?
  19. As model files in AWS S3
  20. What is the destination of the output of model training and inference?
  21. Database
  22. How much historical data do we need for model training per model?
  23. 5-8 years
  24. Where does the model training currently take place? Is there any need to provide scope for training models / retraining models in the required ML Ops platform
  25. Local servers.
  26. Not at the moment.
  27. Are there any dependencies on any system like hadoop?
  28. Not much dependency, mainly aws services such as DynamoDB, Parameter Store and S3

Others:

Section titled “Others:”
  1. What typical use cases are solved with the existing / upcoming models (Classification, Regression, NLP, etc)
  2. Regression
  3. Which consumption layer components do we have on top of this or are required (CLI, UI , Dashboards)?
  4. N/A
  5. What are the expectations regarding orchestration and lifecycle management?
  6. We don’t have much orhestration and lifecycle to manage, and here is a introduction of it. As of today, we train models on our local machines. Once trained, it will be evaluated against a test dataset. Evaluation result are saved to Database, and models are saved to AWS S3. The evaluation results are read by the team (human). Once we decide to deploy some of them, we will manually run a ever-running script on the prediction servers that will download the model from s3, continuously fetching new data and make predictions every minute. The prediction servers are AWS EC2 instances, and each model on the prediction server is controlled through a tmux session.
  7. When we want to deprecate a model, we will kill the corresponding tmux session. Since we don’t train too many models, we leave the database records and models in s3 as is.
  8. This is a intro of the orchestration and lifecycle management we have. Please let us know if you have any suggestion.