Retail Domain
Project 1: Omnichannel Sales Data Lake
I developed an omnichannel retail data lake for a leading global retailer to consolidate data from in-store purchases, e-commerce platforms, mobile apps, and third-party vendors.
I used AWS Glue and S3 to ingest and store raw transactional data from systems like Salesforce, Shopify, and SAP.
I built PySpark ETL jobs that cleansed and enriched product, inventory, and sales data.
I handled deduplication and currency normalization across regions and wrote business logic to compute KPIs like sales per SKU, stock-to-sales ratios, and seasonal trends.
I stored the cleaned data in Amazon Redshift and applied dbt for semantic modeling and metrics standardization.
The team used Looker and Tableau for visual analytics. This project enabled stakeholders to get a single source of truth for product performance, allowing for 20% better inventory distribution and a 30% improvement in forecasting accuracy.
Project 2: Customer Behavior & Recommendation Engine
I worked on a customer behavior analytics platform using Databricks, Delta Lake, and Azure Data Factory to gather insights into customer preferences.
Data from loyalty programs, web logs, and mobile app activity was streamed via Kafka and processed using Spark Streaming.
I created sessionization logic using window functions in PySpark and applied collaborative filtering techniques for recommendations.
I integrated MLflow for model tracking and Azure Synapse Analytics for aggregating results by demographic segments.
I also used Azure Key Vault to manage access securely.
The output of these analytics fed into our recommendation engine that boosted cross-selling and up-selling conversions by 18%.
The marketing team was able to launch personalized campaigns directly based on the user clusters we identified.
Project 3: Supply Chain Optimization and Stock Movement
I built a real-time inventory tracking and supply chain pipeline for a chain of 1,500 stores across North America.
I ingested RFID, barcode scans, and logistics updates via Apache Kafka and stored them in HDFS using Apache Hudi for incremental updates and time-travel capabilities.
I used Spark Structured Streaming to correlate warehouse dispatches with store deliveries and real-time shelf replenishment data.
I enriched this with product master data from SAP BW and weather forecasts to anticipate logistics delays.
The output of this pipeline was visualized in Grafana and also exposed as APIs for real-time mobile inventory lookup. This drastically reduced stockouts and improved warehouse-to-shelf time by 25%.
5. Payroll Domain
Project 1: Payroll Processing & Tax Compliance Engine
I worked on a payroll engine for a multinational enterprise that processed payments for over 80,000 employees across 10 countries.
The raw input data included employee time logs, benefits info, and tax rules, which I ingested using Apache NiFi and stored in Amazon S3.
I wrote ETL logic in Scala using Apache Spark to process time sheets, calculate gross and net pay, deduct taxes (including TDS, PF, 401k, etc.), and apply bonuses and leave encashments.
Tax brackets were managed via DynamoDB, and results were stored in PostgreSQL and Snowflake for compliance auditing.
The processed results were integrated with SAP SuccessFactors and reported using Power BI.
My automation eliminated manual intervention and cut payroll cycle time from 3 days to 6 hours while ensuring statutory compliance.
Project 2: Pay Stub Generation and Archival Platform
I developed a scalable system to generate and store over 1 million digital pay stubs monthly for a leading payroll processing firm.
I ingested structured and semi-structured employee data using Azure Data Factory and stored it in Azure Data Lake Gen2.
Using PySpark on Azure Databricks, I implemented transformation logic and created templates for PDF generation using Apache PDFBox.
I orchestrated the entire pipeline using Azure Data Factory pipelines and triggered alerts for failures using Azure Monitor.
Generated pay stubs were encrypted and archived in Azure Blob Storage, and access was provided through a secure React-based web portal with OAuth2.0 authentication.
This reduced operational burden and also enhanced employee satisfaction through easy access to historical payslips.
I led the development of a predictive analytics engine for salary forecasting and workforce trend analysis.
I ingested HRMS data from Workday, employee profiles, and project allocations into a GCP BigQuery warehouse.
Using Python with Scikit-learn, I developed salary forecasting models based on tenure, skills, past appraisals, and market trends.
I scheduled model training and batch scoring using Vertex AI Pipelines and integrated model predictions with Looker dashboards.
This tool helped HR teams anticipate payroll budget hikes and identify attrition-prone employee segments.
The forecasts had a 91% accuracy rate, helping to streamline resource planning and financial forecasting.
6. Finance Domain
Project 1: Credit Risk Data Platform
I built a data platform for a finance company to assess credit risk across personal loans, credit cards, and mortgages.
I ingested transactional data, credit bureau scores, and behavioral signals using Kafka Connect, AWS Lambda, and S3.
Using EMR Spark clusters, I wrote ETL jobs to merge borrower data, apply risk rules (like DTI ratios, late payment history), and prepare features for model training.
The risk scoring models were developed in SageMaker and predictions were pushed back into Redshift for access by underwriting systems.
I also created lineage tracking using AWS Glue Data Catalog and access control using Lake Formation.
This platform enabled real-time risk analysis and significantly reduced loan default rates.
Project 2: Investment Portfolio Aggregation Platform
I developed a data pipeline to unify investment portfolios of customers spread across brokerage accounts, mutual funds, 401(k), and crypto wallets.
I used Apache Airflow to orchestrate daily batch jobs fetching data from APIs like Plaid, Alpaca, and custodians like Fidelity.
I used Pandas and PySpark to normalize asset types, calculate returns, risk-adjusted metrics (Sharpe, Alpha), and fees.
All processed data was stored in Google BigQuery for interactive queries and portfolio rebalancing analysis.
I designed interactive dashboards in Tableau where wealth advisors could track their clients’ net worth, diversification, and risk profile.
The platform drove personalized advisory and added a new revenue channel for the firm.
Project 3: Financial Forecasting & Budgeting System
I worked on a budgeting platform that supported financial planning for a global manufacturing company.
I ingested ERP, CRM, and historical financial data into Snowflake using Fivetran and Kafka.
I designed and developed ETL logic in dbt to calculate OPEX, CAPEX, cash flow trends, and revenue forecasts.
I integrated Prophet for time-series forecasting and used Streamlit to build internal tools for finance teams to adjust assumptions and simulate scenarios.
This solution gave CFOs real-time control over budgeting cycles and enabled them to cut planning cycles by over 40%.
It also empowered teams to plan based on actuals, seasonal adjustments, and market variables.