SDE (Data Product)

◦ KPI Automation for Finance Domains: Automated KPI reporting across critical finance domains Order to Cash

(OTC), Procure to Pay (PTP), and Record to Report (RTR) by orchestrating data ingestion and processing pipelines.

◦ API-based Data Ingestion: Fetched and processed data from SAP and HRC systems using REST APIs via Azure

Data Factory (ADF), enabling real-time integration of enterprise data.

◦ Scalable ETL Pipelines: Designed and developed scalable ETL pipelines using Azure Data Factory, Azure Databricks,

and PySpark to clean, transform, and load KPI metrics into reporting layers.

◦ Pipeline Optimization: Implemented efficient transformation logic using SQL and Python, reducing data latency and

enhancing pipeline performance.

◦ Legacy to Modern Stack Migration: Converted legacy stored procedures to DBT (Data Build Tool) models with the

help of ChatGPT API, reducing manual effort and enhancing code maintainability by 50%.

◦ Stakeholder Integration: Collaborated with business stakeholders to integrate KPIs into dashboards, supporting

strategic decision-making through data-driven insights.

◦ Technologies Used: Azure Data Factory, Azure Databricks, PySpark, SQL, Python, SAP, HRC, REST

APIs, Git, DBT, ChatGPT API.

Azure Data Engineer

• Optimized data processing: Converted SAS scripts to PySpark and automated the data processing pipeline, incorporating metadata scripts and orchestration JSON, achieving a 50% reduction in data processing time.
• Data management: Organized and managed over 10 million records using Spark libraries (PySpark SQL and PySpark MLlib), improving data accessibility.
• Testing and orchestration: Implemented testing environments with Azure DataBricks and Azure Data Factory, increasing code testing efficiency by 50% and reducing debugging time by 35%. 
• Pipeline construction: Built a data pipeline to process 100 million raw records from 14 data sources, accelerating reporting and analytics capabilities.