Refactor FinancialDatasetAPI integration and the memory cache

[anhquan075]

This pull request focuses on refactoring the codebase to use a new FinancialDatasetAPI class for accessing financial data, replacing individual API functions. The changes involve using this new class, removing the old API functions, and implementing a base API client class for better structure and error handling.

Refactoring to use FinancialDatasetAPI:

• src/agents/market_data.py: Replaced individual API function calls with methods from the new FinancialDatasetAPI class. Adjusted the import statements accordingly.
• src/agents/risk_manager.py, src/agents/technicals.py, src/backtester.py: Updated to use FinancialDatasetAPI.prices_to_df for converting price data to a DataFrame.

Removal of old API functions:

• src/tools/api.py: Removed all old API functions such as get_financial_metrics, search_line_items, get_insider_trades, get_market_cap, get_prices, and prices_to_df.

Implementation of base API client:

• src/tools/api/base.py: Made an abstract base class BaseAPIClient to handle common API client functionality such as making requests and handling responses.
• src/tools/api/config.py: Added configuration classes BaseAPIConfig and FinancialDatasetAPIConfig for managing API configuration.
• src/tools/api/financial_dataset.py: Created the FinancialDatasetAPI class extending BaseAPIClient, implementing methods for fetching financial metrics, insider trades, market cap, and price data.

[hoainam-nguyen]

Awesome

[marc-at-brightnight]

Nice work 👏🏼 Would probably just recommend some typing improvements

[mripani]

Heads up for this PR!

This will allow to easily integrate with other API data providers which is needed for the project.

[anhquan075]

Hi team, thanks for your patience. I'm just back from the public holiday and have now reviewed and addressed all comments, as well as merged with the latest branch. I appreciate your attention to this PR and hope you can take some time to help me review it.

[anhquan075]

Hi @virattt , any chance for me to test the API behavior without using the API key? 🫠 I’m not sure if the minimum top-up credit will be enough for my needs.

[dtiukalov]

since you introduced _process_price_data(), calling prices_to_df() is redundant (in technicals.py and risk_manager.py)
and prices_df shall be replaced by prices

[dtiukalov]

in risk_manager.py and technicals.py shall be from tools.api import FinancialDatasetAPI

[dtiukalov]

in technicals.py instead of prices = get_prices() shall be prices = financial_api.get_prices()

[dtiukalov]

check everywhare: instead of if not prices: -> if prices.empty:

[dtiukalov]

using import fireducks.pandas as pd instead of import pandas as pd may improve performance

[anhquan075]

using import fireducks.pandas as pd instead of import pandas as pd may improve performance

It seems the fireducks isn't supported for Mac ARM rn @dtiukalov, so I can't test with this framework.