Imane.BENZEGUNINE
← All projects
Data engineering

Real-Time Market Data Engineering Platform on Google Cloud (GCP)

FlowTrade · Ecole Nationale des Sciences Appliquées de Berrechid · Oct 2025 – Dec 2025

TOOLS
Google Cloud Platform (GCP) · Python · Pub/Sub · Apache Beam / Dataflow · BigQuery
CONTRIBUTORS
ISSAM, Aymen and Salmane Zid
PROJECT DESCRIPTION

In today’s financial landscape, the ability to process and analyze market data in real-time is a game-changer. FlowTrade was designed to handle high-velocity data streams, ensuring seamless ingestion, processing, and visualization for actionable insights.

The Architecture:

Leveraging the power of GCP, we built a robust pipeline that includes:

  • Data Ingestion: Using Cloud Pub/Sub for reliable, real-time messaging.
  • Processing: Stream processing via Apache Beam/Cloud Dataflow.
  • Storage: Scalable warehousing with BigQuery.
  • Interface: A dynamic dashboard providing a clear, real-time view of market fluctuations and trends.
DATA FLOW

A view of the system.

  1. 01Cloud Pub/Sub
  2. 02Apache Beam / Cloud Dataflow
  3. 03BigQuery
  4. 04Dashboard
Simplified diagram based on the project description.
THE NEXT CONVERSATION

Good work starts
with a conversation.

Let’s talk

Have a role in mind, or a data challenge to work through?