Senior Data Engineer - Data Integration & ETL Development
Position Overview
We are seeking an experienced Senior Data Engineer specializing in data integration and ETL/ELT development to join our dynamic team. This role focuses on building robust data pipelines, managing complex data transformations, and implementing scalable data solutions across multiple platforms and technologies.
Key Responsibilities
Data Integration & ETL Development
Design, develop, and maintain ETL/ELT processes using modern tools including Azure Data Factory, Apache Airflow, dbt, SSIS, and Databricks
Build and optimize data transformation workflows to ensure efficient data processing and quality
Implement data integration solutions across various source systems and target platforms
Develop automated data pipelines that support business intelligence and analytics initiatives
Database Management & Development
Work with multiple database platforms including SQL Server, Oracle, PostgreSQL, MySQL, SQLite, and MongoDB
Design and implement stored procedures, functions, and complex queries for data processing
Perform database optimization and performance tuning
Manage Snowflake environments including querying, data masking, and Cortex functionality
Team Leadership & Project Management
Lead data integration development teams and mentor junior developers
Manage end-to-end data integration projects from requirements gathering to deployment
Define and implement company-wide standards for ETL development and data management
Create comprehensive technical and functional requirements documentation
Data Governance & Quality
Implement Master Data Management (MDM) solutions
Establish data quality standards and validation processes
Ensure data security and compliance with organizational policies
Design and implement data masking and privacy protection measures
Reporting & Analytics
Build reports and dashboards using Excel, Power BI, Hyperion, and Cognos
Support business intelligence initiatives with reliable data delivery
Collaborate with stakeholders to understand reporting requirements
Required Technical Skills
Programming & Scripting
Python: Advanced proficiency with pandas, NumPy, LangChain, Selenium, BeautifulSoup
SQL: Expert-level SQL development and optimization
Jinja: Template engine for dynamic SQL generation
Shell scripting for automation tasks
Data Platforms & Tools
Cloud Platforms: Azure (Data Factory, DevOps), Databricks, Snowflake
ETL Tools: SSIS, Apache Airflow, dbt (data build tool)
Databases: SQL Server, Oracle, PostgreSQL, MySQL, MongoDB
Data Formats: JSON, CSV, XML, YAML, Parquet, Excel/TXT files
DevOps & Version Control
Git & GitHub for source code management
Azure DevOps/TFS for project management and CI/CD
Master Data Services (MDS)
Linux and Windows operating systems
Required Experience
5+ years of experience in data engineering and ETL development
3+ years of hands-on experience with cloud-based data platforms (Azure, Snowflake, Databricks)
2+ years of team leadership experience in data integration projects
Proven track record of managing complex data integration initiatives
Experience with Master Data Management implementation
Preferred Qualifications
Bachelor's degree in Computer Science, Information Systems, or related field
Experience in financial services, insurance, or similar regulated industries
Certification in Azure Data Engineering or Snowflake
Experience with modern data architecture patterns (data mesh, lakehouse, etc.)
Knowledge of data governance frameworks and best practices
What We Offer
Opportunity to work with cutting-edge data technologies
Leadership role in defining data engineering standards
Remote work flexibility
Professional development opportunities
Collaborative team environment focused on innovation
Location
Remote work available with preference for candidates in the Americas time zone for team collaboration.
