Waqas Mian

L

Data Engineer – III

LexisNexis IPlytics GmbH

Mar 2023 - Current (3 years 4 months)

• Building a modern centralised data platform to incorporate more than 30+ data sources • Transforming a legacy C# data system to a modern, cloud-agnostic architecture, decreasing system downtime by 99% and improving data pipeline resilience • Building and managing ETL production ready data pipelines in databricks • Designing, implementing and optimising scalable, high-performance data pipelines • Working on Spark streaming data workflows to handle extremely large volume of data • Leading development of multiple important data features of project • Orchestrating and scheduling jobs on Kubernetes using Apache Airflow • Streamline Agile workflows to boost productivity, enhance collaboration, and ensure timely project delivery

S

Principal Consultant – Software & Data

Systems Limited (Visionet Inc)

Jan 2022 - Sep 2023 (1 year 9 months)

• Worked on data framework of a product which incorporates data from different customers to create custom plans and perform forecasting for their businesses • Created delta lake solution using managed spark jobs (Azure Databricks) • Created internal data pipelines (Azure Data Factory) • Orchestrated different modules of framework (Apache Airflow) • Build faceted search functionality for platform (Elasticsearch) • Exposed main modules of data platform to web via RestFul APIs (Django) • Led a Pre-Sales POC Activity for a customer and mentored junior resource

Software Engineer

Sendoso

Oct 2020 - Oct 2021 (1 year 1 month)

• Worked as backend software engineer on core Sendoso product • Reviewing PRD's (Product Requirement Documents) with the product team and creating Tech Recs (Technical Recommendation Documents) with engineering team • Writing RESTFul APIs in Ruby on Rails • Designed Postgres SQL Database for product • Keep an eye on customer-facing bugs and get them resolved on a priority basis.

N

Data Engineer

Northbay Solution

Jun 2019 - Oct 2020 (1 year 5 months)

• Built an in-house data lake project direct managed by the VP of Data Science team • Built a datalake accelerator that creates a 80% generilised ready to use and 20% customisable data lake for any customer usecase • Worked on a datalake solution on AWS Cloud and exposed it via web through python django framework • Assigned on different big data Customer proof of concept projects

About

Detail

Timeline

Résumé