You have opted out of job matches in .
To undo this, go to the 'Skills and Interests' section of your preferences.
Posted 4 days ago
Requirements and responsibilities
We are seeking a Senior Data Engineer to design, build, and maintain scalable data pipelines and processing systems. This role is heavily focused on Python and involves working across modern data platforms, streaming systems, and cloud-based infrastructure.
You will play a key role in enabling analytics, reporting, and data-driven decision-making by ensuring reliable and efficient data flows.
Key Responsibilities:
- Design, develop, and maintain batch and streaming data pipelines using Python.
- Implement scalable data architectures for analytics, automation, and reporting.
- Build and optimize ETL/ELT workflows using Apache Spark and Databricks.
- Orchestrate pipelines using Apache Airflow.
- Integrate data from multiple sources via streaming and messaging systems.
- Ensure high performance, reliability, and scalability of data systems.
- Collaborate with analytics and engineering teams to deliver high-quality data models.
- Implement data quality checks, monitoring, and automation processes.
- Support downstream applications and reporting use cases.
Required Qualifications:
- 8+ years of experience in Data Engineering.
- Strong proficiency in Python.
- Hands-on experience with data pipelines and ETL/ELT design.
- Hands-on experience with Apache Spark.
- Hands-on experience with Databricks.
- Hands-on experience with Apache Airflow.
- Experience with databases and storage systems: PostgreSQL, MongoDB, Elasticsearch.
- Experience with messaging/streaming systems: Kafka, RabbitMQ, AWS SQS.
- Solid understanding of data modeling and processing patterns.
- Experience supporting analytics and reporting workloads.
Nice to Have:
- Experience with Java-based data systems.
- Hands-on experience with AWS cloud services.
- Familiarity with CI/CD pipelines for data workflows.
Optionally, you can add more information later (benefits, pre-screening questions, etc.)