I

Irfan Shah

About

Detail

United States

Timeline

work
Job
folder
Project

Résumé

Jobs verified_user 0% verified
  • Rockset
    Senior Data Architect
    Rockset
    Oct 2020 - Current (6 years 1 month)
    • Designed and implemented a scalable enterprise data warehouse using Snowflake and Databricks, enabling seamless analytics across multi terabyte datasets. • Led the migration of on-prem databases to AWS, leveraging Glue, Redshift, and Kinesis, reducing infrastructure costs by 30% and improving data accessibility. • Architected real-time and batch data pipelines using Apache Airflow, DBT, and Kafka, ensuring efficient data processing for business-critical applications. • Established data governance frameworks, implementing RBAC, encryption, and data lineage tracking to ensure compliance with GDPR and SOC 2. • Mentored a team of data engineers, fostering best practices in data modeling, ETL optimization, and cloud data management.
  • RudderStack
    Lead Data Engineer
    RudderStack
    Mar 2019 - Sep 2020 (1 year 7 months)
    • Developed high-performance ETL pipelines using Apache Spark and Hadoop, processing billions of records daily for advanced analytics and reporting.
    • Spearheaded the adoption of CI/CD for data pipelines with GitHub Actions and Terraform, accelerating deployment cycles by 40%.
    • Designed a hybrid data architecture integrating Azure Synapse, PostgreSQL, and Cassandra, supporting both OLAP and OLTP workloads.
    • Optimized SQL queries and indexing strategies, improving query performance by 60% and reducing data processing time from hours to minutes.
    • Led cross-functional collaborations with ML Engineers and Business Analysts to deliver data-driven insights and predictive analytics solutions. Additionally, utilized M
  • D
    Big Data Engineer
    DataForge Labs
    Jun 2016 - Feb 2019 (2 years 9 months)
    • Engineered real-time data streaming solutions using Apache Kafka and Flink, enabling near-instantaneous data processing for fraud detection.
    • Implemented Change Data Capture (CDC) techniques with Debezium and Kinesis, ensuring seamless data synchronization across distributed systems.
    • Designed and deployed containerized data workflows using Docker and Kubernetes, improving scalability and resource utilization.
    • Automated data security and compliance workflows, including encryption, masking, and IAM policies, ensuring adherence to HIPAA and CCPA regulations.
    • Conducted strategic tool evaluations and vendor selections, optimizing technology investments and enhancing overall data infrastructure resilience.
Education verified_user 0% verified
  • COMSATS University
    Bachelor's Degree in
    COMSATS University
Projects (professional or personal) verified_user 0% verified
  • RudderStack
    Data Governance / Compliance Projects
    RudderStack
    Mar 2019 - Aug 2020 (1 year 6 months)