Lead Data Engineer – PySpark & SQL at AB2 Consulting, Inc. and DATAMAXIS, Inc. | Torre
Lead Data Engineer – PySpark & SQL
Report
AB2 Consulting, Inc. and DATAMAXIS, Inc.

Lead Data Engineer – PySpark & SQL

AB2 Consulting, Inc. and DATAMAXIS, Inc.
Emma
Emma
Emma highlights
This highlight was written by Emma’s AI. Ask Emma to edit it.
Full-time

Legal agreement: To be defined

Provide your expected compensation while applying
location_on
Remote (anywhere)
Shared by
Jose Aldave
 6 days ago

Responsibilities

Key Responsibilities:Design, develop, and optimize scalable ETL/ELT data pipelines.Build high-performance data processing solutions using PySpark and Spark DataFrame API.Develop complex SQL queries, stored procedures, joins, window functions, and perform query optimization.Design data models for analytics, reporting, and ML workloads.Work with Azure Databricks, Azure Data Lake, Azure Synapse, and Azure Data Factory.Implement data quality, validation, monitoring, governance, and security practices.Troubleshoot production data pipelines and ensure reliable data delivery.Lead code reviews, enforce development standards, and mentor junior data engineers.Collaborate with Data Scientists, Analysts, Product Managers, and business stakeholders.Mandatory Skills:PySpark – ExpertAdvanced SQLPythonAzure DatabricksBig Data ArchitectureETL/ELT Pipeline DevelopmentModern Data Lake / Data Warehouse ConceptsActive Azure Databricks CertificationGood to Have:Azure Data FactoryAzure SynapseAzure Key VaultApache AirflowKafka / Event HubsSpark Structured StreamingData Governance & Data CatalogingCI/CD & Git
Closes in:
0
days
0
hours
0
min
0
sec
tune NOT FOR YOU? IMPROVE YOUR RESULTS