Skip to content

Commit

Permalink
Update tutorials
Browse files Browse the repository at this point in the history
  • Loading branch information
@nkaz001
nkaz001 committed May 19, 2023
1 parent ee21475 commit 8eb436d
Show file tree
Hide file tree
Showing 2 changed files with 314 additions and 92 deletions.
203 changes: 157 additions & 46 deletions docs/tutorials/Integrating Custom Data.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,88 +12,117 @@
},
{
"cell_type": "markdown",
"id": "e962b0d2",
"id": "47864a60",
"metadata": {},
"source": [
"## Quick Example for Accessing Funding Rate\n",
"## Accessing Spot Price\n",
"\n",
"In this example, we'll combine the spot BTCUSDT mid-price with the USDM-Futures BTCUSDT feed data. This will enable you to estimate the fair value price, taking the underlying price into consideration.\n",
"\n",
"First, let's demonstrate how to access your custom data. If you're using data that has been converted from raw feed by the built-in utility, you can effortlessly incorporate `markPrice` stream data. Find out more details [here](https://hftbacktest.readthedocs.io/en/latest/reference/hftbacktest.data.utils.binancefutures.html)."
"The spot data is used only in the local-side, and thus, should come with a local timestamp. Following this, in your backtesting logic, your task is to identify the most recent data that predates the current timestamp."
]
},
{
"cell_type": "markdown",
"id": "3bd2277d",
"metadata": {},
"source": [
"The raw spot feed is processed to create spot data, which includes both a local timestamp and the spot mid price."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "d1e04ecc",
"id": "b5b2a572",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"local_timestamp is ahead of exch_timestamp by 26932.0\n",
"found 6555 rows that exch_timestamp is ahead of the previous exch_timestamp\n",
"Correction is done.\n"
]
}
],
"outputs": [],
"source": [
"import numpy as np\n",
"import gzip\n",
"import json\n",
"\n",
"from hftbacktest.data.utils import binancefutures\n",
"spot = np.full((100_000, 2), np.nan, np.float64)\n",
"i = 0\n",
"\n",
"data = binancefutures.convert('usdm/btcusdt_20230405.dat.gz', opt='m')\n",
"np.savez('btcusdt_20230405_m', data=data)"
"with gzip.open('spot/btcusdt_20230405.dat.gz', 'r') as f:\n",
" while True:\n",
" line = f.readline()\n",
" if line is None or line == b'':\n",
" break\n",
"\n",
" line = line.decode().strip()\n",
" local_timestamp = int(line[:16])\n",
"\n",
" obj = json.loads(line[17:])\n",
" if obj['stream'] == 'btcusdt@bookTicker':\n",
" data = obj['data']\n",
" mid = (float(data['b']) + float(data['a'])) / 2.0\n",
" # Sets the event ID to 110 and assign an invalid exchange timestamp, \n",
" # as it's not utilized in the exchange simulation. \n",
" # And stores the mid-price in the price column.\n",
" spot[i] = [local_timestamp, mid]\n",
" i += 1\n",
" \n",
"spot = spot[:i]"
]
},
{
"cell_type": "markdown",
"id": "d998ceb3",
"id": "09180f41",
"metadata": {},
"source": [
"You can obtain the funding rate by using `get_user_data` function along with event id 102."
"It displays the basis and spot mid price as it identifies the latest Point-in-Time data that falls before the current timestamp."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "c47802b1",
"id": "d9bab818",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Load btcusdt_20230405_m.npz\n",
"current_timestamp: 1680652860032116 futures_mid: 28150.75 funding_rate: 2.76e-05\n",
"current_timestamp: 1680652920032116 futures_mid: 28144.15 funding_rate: 2.813e-05\n",
"current_timestamp: 1680652980032116 futures_mid: 28149.95 funding_rate: 2.826e-05\n",
"current_timestamp: 1680653040032116 futures_mid: 28145.75 funding_rate: 2.826e-05\n",
"current_timestamp: 1680653100032116 futures_mid: 28140.55 funding_rate: 2.841e-05\n",
"current_timestamp: 1680653160032116 futures_mid: 28143.85 funding_rate: 2.85e-05\n"
"current_timestamp: 1680652860032116 futures_mid: 28150.75 , spot_mid: 28164.42 , basis: -13.67\n",
"current_timestamp: 1680652920032116 futures_mid: 28144.15 , spot_mid: 28155.82 , basis: -11.67\n",
"current_timestamp: 1680652980032116 futures_mid: 28149.95 , spot_mid: 28163.48 , basis: -13.53\n",
"current_timestamp: 1680653040032116 futures_mid: 28145.75 , spot_mid: 28158.88 , basis: -13.12\n",
"current_timestamp: 1680653100032116 futures_mid: 28140.55 , spot_mid: 28156.06 , basis: -15.51\n",
"current_timestamp: 1680653160032116 futures_mid: 28143.85 , spot_mid: 28155.82 , basis: -11.97\n"
]
}
],
"source": [
"from numba import njit\n",
"from hftbacktest import HftBacktest, FeedLatency, Linear, COL_PRICE\n",
"from hftbacktest import HftBacktest, FeedLatency, Linear\n",
"\n",
"@njit\n",
"def print_funding_rate(hbt):\n",
"def print_basis(hbt, spot):\n",
" spot_row = 0\n",
" \n",
" # Checks every 60-sec (in microseconds)\n",
" while hbt.elapse(60_000_000):\n",
" # funding_rate data is stored with event id 102.\n",
" funding_rate = hbt.get_user_data(102)\n",
" # Finds the latest spot mid value.\n",
" while spot_row < len(spot) and spot[spot_row, 0] <= hbt.current_timestamp:\n",
" spot_row += 1\n",
" spot_mid_price = spot[spot_row - 1, 1] if spot_row > 0 else np.nan\n",
" \n",
" mid_price = (hbt.best_bid + hbt.best_ask) / 2.0\n",
" basis = mid_price - spot_mid_price\n",
" \n",
" print(\n",
" 'current_timestamp:',\n",
" hbt.current_timestamp,\n",
" 'futures_mid:',\n",
" round(mid_price, 2),\n",
" 'funding_rate:',\n",
" funding_rate[COL_PRICE]\n",
" ', spot_mid:',\n",
" round(spot_mid_price, 2),\n",
" ', basis:',\n",
" round(basis, 2)\n",
" )\n",
" \n",
"\n",
"hbt = HftBacktest(\n",
" [\n",
" 'btcusdt_20230405_m.npz'\n",
Expand All @@ -107,7 +136,7 @@
" snapshot='btcusdt_20230404_eod.npz'\n",
")\n",
"\n",
"print_funding_rate(hbt)"
"print_basis(hbt, spot)"
]
},
{
Expand All @@ -117,21 +146,16 @@
"source": [
"## Combining Spot Price\n",
"\n",
"We've shown you how to access user data within the data processed by the built-in data utility. Now, let's explore how to merge your custom data with the feed data.\n",
"\n",
"In this example, we'll combine the spot BTCUSDT mid-price with the USDM-Futures BTCUSDT feed data. This will enable you to estimate the fair value price, taking the underlying price into consideration."
"While integrating custom data with feed data might be more challenging than simply accessing the data demonstrated in the first example, this process could be necessary if you're intending to develop your own custom exchange model. Viewing the custom data from the exchange-side could indeed provide a more comprehensive approach to backtesting, such as when considering funding."
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "b5b2a572",
"id": "e8372f9c",
"metadata": {},
"outputs": [],
"source": [
"import gzip\n",
"import json\n",
"\n",
"tmp = np.full((100_000, 6), np.nan, np.float64)\n",
"i = 0\n",
"\n",
Expand Down Expand Up @@ -220,8 +244,7 @@
}
],
"source": [
"from numba import njit\n",
"from hftbacktest import COL_PRICE\n",
"from hftbacktest import reset, COL_PRICE\n",
"\n",
"@njit\n",
"def print_basis(hbt):\n",
Expand All @@ -244,8 +267,6 @@
" ', basis:',\n",
" round(basis, 2)\n",
" )\n",
" \n",
"from hftbacktest import reset\n",
"\n",
"reset(\n",
" hbt,\n",
Expand All @@ -257,6 +278,96 @@
"\n",
"print_basis(hbt)"
]
},
{
"cell_type": "markdown",
"id": "e962b0d2",
"metadata": {},
"source": [
"## Combining Funding Rate by Using Built-in Data Utility\n",
"\n",
"If you're using data that has been converted from raw feed by the built-in utility, you can effortlessly incorporate `markPrice` stream data. Find out more details [here](https://hftbacktest.readthedocs.io/en/latest/reference/hftbacktest.data.utils.binancefutures.html)."
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "d1e04ecc",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"local_timestamp is ahead of exch_timestamp by 26932.0\n",
"found 6555 rows that exch_timestamp is ahead of the previous exch_timestamp\n",
"Correction is done.\n"
]
}
],
"source": [
"from hftbacktest.data.utils import binancefutures\n",
"\n",
"data = binancefutures.convert('usdm/btcusdt_20230405.dat.gz', opt='m')\n",
"np.savez('btcusdt_20230405_m', data=data)"
]
},
{
"cell_type": "markdown",
"id": "d998ceb3",
"metadata": {},
"source": [
"You can obtain the funding rate by using `get_user_data` function along with event id 102."
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "c47802b1",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Load btcusdt_20230405_m.npz\n",
"current_timestamp: 1680652860032116 futures_mid: 28150.75 funding_rate: 2.76e-05\n",
"current_timestamp: 1680652920032116 futures_mid: 28144.15 funding_rate: 2.813e-05\n",
"current_timestamp: 1680652980032116 futures_mid: 28149.95 funding_rate: 2.826e-05\n",
"current_timestamp: 1680653040032116 futures_mid: 28145.75 funding_rate: 2.826e-05\n",
"current_timestamp: 1680653100032116 futures_mid: 28140.55 funding_rate: 2.841e-05\n",
"current_timestamp: 1680653160032116 futures_mid: 28143.85 funding_rate: 2.85e-05\n"
]
}
],
"source": [
"@njit\n",
"def print_funding_rate(hbt):\n",
" # Checks every 60-sec (in microseconds)\n",
" while hbt.elapse(60_000_000):\n",
" # funding_rate data is stored with event id 102.\n",
" funding_rate = hbt.get_user_data(102)\n",
" mid_price = (hbt.best_bid + hbt.best_ask) / 2.0\n",
" \n",
" print(\n",
" 'current_timestamp:',\n",
" hbt.current_timestamp,\n",
" 'futures_mid:',\n",
" round(mid_price, 2),\n",
" 'funding_rate:',\n",
" funding_rate[COL_PRICE]\n",
" )\n",
" \n",
"reset(\n",
" hbt,\n",
" [\n",
" 'btcusdt_20230405_m.npz'\n",
" ],\n",
" snapshot='btcusdt_20230404_eod.npz'\n",
")\n",
"\n",
"print_funding_rate(hbt)"
]
}
],
"metadata": {
Expand Down
Loading

0 comments on commit 8eb436d

Please sign in to comment.