Live Contract Trading Data Harvesting

The goal of this project was to build an ETL system collecting and computing data about trades and trading positions for various cryptocurrencies. The data about trades and trading positions were integrated on the fly from multiple trading platforms via REST APIs and web socket.

The data about trades and trading positions were fetched via multiple REST APIs and web sockets (depending on the trading platform). The data was then normalized, and some elements were aggregated, computed, and estimated. Finally, the normalized data were loaded into a common data model in a relational database (Exasol DB). 

The system architecture combined Apache NiFi native data processors and several custom data processors in Python.

The final data was used for financial analysis by an advisory company focusing on investments in crypto assets, in order to create investment recommendations for their clients.

Data Scientist | Machine Learning Engineer | AI Advisor

20 years of experience in data processing from BigData to AI