Sohail Anjum

• Overall, 12 years of IT experience across Big Data. ETL\ELT, SQL, Python, Scala, C#, Java. Interested and passionate about working in Big Data environment and data analytics • Expertise in AWS Services such as Amazon Glue, AWS Lambda, Amazon RDS, Amazon Redshift, Amazon DynamoDB, Amazon S3, AWS CloudWatch, AWS IAM, AWS IoT Analytics etc. • Designed and developed ETL/ELT data pipelines with AWS Glue/Lambda. • Integrated AWS Redshift with Microsoft Power BI enhancing data-driven insights. • Data ingestion workflows for Snowflake/Redshift using AWS Glue/Lambda, • Utilized AWS Glue Data Catalog to maintain a centralized metadata repository, enabling efficient data discovery and cataloging. • Implemented and optimized complex data transformations on unstructured, semi-structured and structure data using AWS Glue/ PySpark data frames. • Strong Analytical and data aggregation skills using PySpark, Advanced SQL, duckdb. Pandas. • Developed data pipelines using Databricks Delta, a data lake house platform built on Apache Spark that provides ACID transactions and scalable analytics. • Configured Auto workload management in AWS Redshift • Implemented PySpark streaming to pick up data from Kafka topic and send to Data pipeline. • Design and Implemented Database, Tables, Relationships and Store Procedures. • Developed custom scripts and applications using AWS Wrangler API to automate data ingestion, cleaning, and transformation tasks, resulting in faster data processing times and improved data quality. • Designed interactive dashboards and reports in Microsoft Power BI, enhancing data-driven insights. • Implemented DAX/Measure/Relationship for custom KPIs to track key business metrics.