We are seeking a highly skilled Data Engineer to design, build, and maintain scalable data pipelines and infrastructure. The ideal candidate will work closely with data analysts, data scientists, and business teams to ensure high-quality, reliable, and accessible data across the organization.
Key Responsibilities:
- Design, develop, and maintain ETL/ELT pipelines using modern data engineering tools.
- Build and optimize data warehouses, data lakes, and data marts.
- Integrate data from multiple structured and unstructured sources.
- Develop scalable and high-performance data processing workflows.
- Collaborate with engineering and analytics teams to ensure data availability and quality.
- Implement data quality checks, monitoring, and validation frameworks.
- Optimize database performance, query execution, and storage solutions.
- Work with cloud platforms such as AWS, Azure, or GCP to deploy and manage data infrastructure.
- Ensure compliance with data governance, security, and privacy policies.
- Create and maintain documentation related to pipelines, schemas, and processes.
Required Skills and Qualifications:
- Master’s degree in Computer Science, IT, Data Engineering, or a related field.
- Strong programming skills in Python, SQL, Scala, or Java.
- Experience with ETL tools such as Airflow, DBT, Glue, Informatica, or similar.
- Hands-on experience with big data technologies such as Hadoop, Spark, Kafka, Hive, and HBase.
- Experience with cloud data services including AWS (Redshift, S3, Glue), Azure (Data Factory, Synapse), and GCP (BigQuery, Dataflow).
- Solid understanding of data modeling and database design.
- Experience working with CI/CD pipelines and version control tools such as Git.
- Understanding of API integrations and data streaming pipelines.
Preferred Qualifications:
- Knowledge of ML pipeline workflows (ML Ops).
- Experience with real-time data processing systems.
- Familiarity with tools such as Snowflake, Databricks, Tableau, or Power BI.
