Lead Data Engineer at ARK Infotech Spectrum | Torre

Lead Data Engineer

You'll lead a critical large-scale data platform migration to Databricks, shaping future-state architecture.
Emma highlights
This highlight was written by Emma’s AI. Ask Emma to edit it.
Full-time

Legal agreement: Employment

Provide your expected compensation while applying
location_on
Remote (for Canada residents)
Match
skeleton-gauges
You have opted out of job matches in .
To undo this, go to the 'Skills and Interests' section of your preferences.
Review preferences
Shared by
Gus García
about 1 month ago

Requirements and responsibilities


We are looking for a Lead Data Engineer to drive a large-scale migration from AWS Glue & Amazon Redshift to Databricks Lakehouse. If you have strong experience in Spark, Databricks, and AWS, along with leading migration projects end-to-end — this role is for you!Key ResponsibilitiesLead end-to-end migration (Glue/Redshift → Databricks)Design and implement Lakehouse architecture (Unity Catalog, Medallion layers)Re-platform pipelines using PySpark, Spark SQL, Delta Live TablesDefine data validation, reconciliation, and testing strategiesManage cutover for BI, ML, and downstream systemsEstablish CI/CD, governance, and data quality frameworksOptimize performance and cost (clusters, autoscaling, Photon, SQL warehouses)Required SkillsStrong experience with Databricks (Delta Lake, Unity Catalog, DLT)Expertise in Apache Spark (PySpark, Spark SQL)Hands-on with AWS Glue & Amazon RedshiftProven experience in data platform migrationsStrong SQL & Python skillsKnowledge of AWS services (S3, IAM, VPC, etc.)What We’re Looking ForStrong technical leadership and ownershipAbility to work with stakeholders and engineering teamsExpertise in building scalable, high-performance data platforms📩 Interested candidates, share your resume at: Srikanth@arkinfotechspectrum.com
Optionally, you can add more information later (benefits, pre-screening questions, etc.)
check_circle

Payment confirmed

A member of the Torre team will contact you shortly

In the meantime, continue adding information to your job opening.