The RaftLib C++ library, streaming/dataflow concurrency via C++ iostream-like operators

The easiest way to use Machine Learning. Mix and match underlying ML libraries and data set sources. Generate new datasets or modify existing ones with ease.

[ICLR 2025 Spotlight] LayerDAG: A Layerwise Autoregressive Diffusion Model of Directed Acyclic Graphs

Big Data CONFERENCE Europe 2025 🇱🇹 AI, Cloud and Data Conference - Build Your First End-to-End Lakehouse Solution (aka.ms/fabconlake)

This repository serves as a personal collection of Power BI Dashboard and Report sample files, along with the datasets used in the samples, tutorials, and other related content.

A set of libraries for rapidly developing RabbitMQ workflows/pipelines.

Templated modern C++ library for implementing DataFlow programming

How flows of data reveal the ecosystems of our planet

Easy and lightweight parallelized dataflows in Python

An end-to-end data pipeline in Microsoft Fabric to process and analyze COVID-19 data.

COVID 19 Data Pipeline - Azure Data Engineering: A complete end-to-end ETL pipeline to fetch COVID19 daily and weekly data from API, transform and load to SQL database in Azure using Data Factory, Databricks, Key-Vaults and Data Lake

This project demonstrates the development of a robust batch processing pipeline designed to handle large-scale data ingestion and transformation.

Custom agentic AI Generominos cards

Three Microsoft Fabric projects collected in one place, showcasing practical end-to-end data engineering workflows. Built primarily with Microsoft Fabric services like dataflowgen2, datafactory, pyspark-notebooks , covering ingestion, transformation, and orchestration patterns in Fabric.

Incremental computation engine for Go with live what-if worlds: fork the graph into N scenarios that pin some inputs and track the rest live, all updated by one stabilization. Data-shaped topology (Bind), per-key diff collections, O(divergence) scenario cost.

small ETL library to convert MRASCO & SPAA UK energy industry dataflows into csv

🏗️ Terraform module to deploy and manage Dataflow jobs, templates, staging configuration, worker settings, and service accounts.

COVID19 Data Pipeline: An end-to-end ETL data pipeline to fetch daily and weekly Covid data from API, transform and load it into SQL database using some of the Azure services.

Serverless Power BI connector for Smokeball — built with Azure Functions, Blob Storage & OAuth2 token rotation. Enables automated scheduled refreshes in Power BI Service via Dataflows.

This repo contains details about real time mock uber ride data publishing into Pub Sub and leveraging custom data flow template building with Apache beam and loading the raw data and alerts data into Big Query table in real time, Thanks