Repository implemented to develop my final dual degree project:
- COMPUTER SCIENCE: Data Engineer Project at KDB+/q.
- BUSINESS ADMINISTRATION: ETF risk study model on their volatilities and drawdowns.
The main idea is create an entire data engineer process on Jupyter Notebook, using the main tool kernel: KDB+/Q.
The entire project are created over environment called "kdb" created using Miniconda 24.3.0 and kdb+/q kernel 4.1 released on February 2024 (4.1 info).
This it's opcional, but I did to have in different envs the kernel that I usually used.
-
Create conda env with python version (at least 3.6 to use kdb+ 4.X kernel version):
conda activate
conda create --name kdb python=3.10
- Activate env:
conda activate kdb
Now you should see something like:
(kdb) [your machine]$:
- Installing kdb+:
conda install -c kx kdb
- Now execute q and add the license(Get Free License).
q
- To use KDB+/Q kernel on Jupyter Notebook you have to added JupyterQ:
conda install -c kx jupyterq
- Execute a jupyter session and start to managed the info:
jupyter notebook
If your license are correct install the JupyterQ will be able to manage. If your jupyter cannot reconize your license or whatever other problem, you can check this blog where they confiurate the kernel manually.
Manual KDB+/q jupyter kernel configuration.
- All you have to use for generate date enrichment → dataETL.ipynb.
- For visualizate and agrupation of volatilities followings performance → dataAnalytics.ipynb.
We created an ETL project structure, which involves learning: kdb+, q language, Python, Python graphical libraries, integrations, data processing... in short, a complete project.
- The main objective is to create a study environment for Blackrock's main ETFs (in US dollars) to try to valorate the situation of these ETF.
- Secondly, create an estimate based on the past, taking the premise that the market is cyclical.
Therefore we will study 2 types of critical points:
- What happens to drawdowns and volatilities when we reach a cumulative % loss?
- What happens to drawdowns and volatilities when we reach a % of accumulated profits?
Then → we will be able to group and categorise the variables in order to make decisions on when to buy and when to sell.
Subsequently we will group the volatilities and drawdowns on sell/buy alerts moments and we will place the different ETFs according to their situation assessment. We will be able to determine the quartiles and the maximum and minimum drawdowns at those times, which is key to the estimates. For this we will use Matplotlib for visualization added to Q language calculations done in the ETL proccess.
We will take advantage of the structure and capabilities offered by Q and qSQL to manage and manipulate this data.
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
