Awais Irshad

U

Lead Data Engineer

Unifonic,

Feb 2024 - Current (2 years 8 months)

• Designed and implemented real-time data ingestion pipelines using Apache Kafka and PySpark, enabling processing of 5TB+ daily transactional data. • Deployed and integrated DataHub with the company's Data Warehouse (DWH) and Data Lake, streamlining data governance and discovery processes. • Planned and oversaw data integration strategies, including ETL processes, data migration, and synchronization, ensuring 100% data consistency and integrity across systems. • Automated ETL workflows with Apache Airflow, enhancing efficiency and reducing manual intervention by 60%. • Delivered analytics dashboards using Power BI, improving operational insights for over 100,000 end users. Technologies used: Airflow | Vertica | Oracle Cloud Storage | A

O

Solution Architect / Lead Data Engineering Department

Odyssey Solutions,

Apr 2022 - Feb 2024 (1 year 11 months)

Project: Multiverse DataLake - A data lake tailored for power, energy, and agriculture sectors, improving operational efficiency. • Designed a flexible data lake leveraging AWS S3, Lambda, and SQS. • Automated data ingestion with PySpark crawlers, processing 1TB+ data daily, improving data accessibility by 40%. • Designed materialized views for advanced analytics, reducing query processing times by 35%. • Streamlined real-time data workflows with Apache Kafka, enhancing system reliability by 99.9% uptime. • Led a team to deliver scalable data solutions, adhering to best practices, and achieving a 25% faster project delivery rate. Technologies used: AWS S3 | Lambda | SQS | PySpark | Greenplum | Kafka | CI/CD Pipelines. Project: - • Mig

T

Senior Data Engineer

TRG (Ibex Pvt Ltd),

Mar 2018 - Feb 2020 (2 years)

Project: - and for an • Streamlined AWS-based data pipelines for efficient storage and processing. • Automated data orchestration using Airflow and Dockerized applications, improving data synchronization efficiency by 50%. • Developed and deployed scalable ETL workflows with PySpark, processing 2+ million property records daily. • Built monitoring dashboards with CloudWatch, improving pipeline observability and reducing debugging time by 40%. Technologies used: AWS | Airflow | Docker

N

Senior Software Engineer

NetSol Technologies,

Mar 2015 - Jan 2018 (2 years 11 months)

Project: - Designed and optimized complex procedures and databases for asset financing workflows. Automated business-critical reports, reducing processing time by 40%. Technologies used: MSSQL Server | Python | Pentaho

T

Critical Services Engineer

Techlogix Pvt Ltd,

Jul 2012 - Feb 2015 (2 years 8 months)

Project: - Technologies used: Oracle BI | MSSQL Server | Oracle PeopleSoft

About

Detail

Timeline

Résumé

Awais Irshad

About

Detail

Behavioral traits