Realtime Algorithmic Trading with Apache Flink - End-to-End Data Engineering Project

Overview

Develop an end-to-end data engineering project for real-time algorithmic trading using Apache Flink, Apache Kafka, Redpanda, and Python. Learn the fundamentals of algorithmic trading, explore Apache Flink's stream processing capabilities, and integrate Kafka for efficient data ingestion. Process and transform stock price data in real-time using Flink SQL and DataStream API, manage event time and watermarks, and build and deploy Flink jobs for continuous data processing. Create and consume Kafka topics for stock price data, implement real-time analytics, and execute buy/sell orders based on custom trading algorithms. Gain hands-on experience with tools like Docker and SQL while following a comprehensive workflow from data ingestion to trade execution. Suitable for data engineers, software developers, financial analysts, and anyone interested in real-time data processing and trading systems.

Syllabus

Introduction
Stock Trading Strategies
Process of Buying and Selling Stocks in the Market
System Architecture
Installations and Dependencies
Setting up the Architecture
Retrieving Historical News Producer
Setting up Custom LLM for Sentiment Analysis
Setting up Historical Prices Producer
Implementing Custom Trading Algorithm