Are you an experienced Lead Data Engineer with expertise in Microsoft Azure and exposure to Generative AI? We're looking for a technology leader to build scalable data platforms while contributing to AI-driven data solutions.Key Responsibilities:Lead the design and implementation of enterprise-scale data engineering solutions on Microsoft Azure.Develop scalable ETL/ELT pipelines using Azure Data Factory (ADF), Azure Databricks, and Azure Synapse Analytics.Build and optimize data lakes and modern data warehouse solutions using ADLS Gen2.Develop high-performance data processing frameworks using Python, PySpark, and SQL.Collaborate with Data Science and AI teams to enable data pipelines for Generative AI and machine learning use cases.Work with Azure OpenAI, vector databases, or LLM-based applications (preferred).Implement CI/CD pipelines using Azure DevOps and follow best engineering practices.Mentor engineering teams and provide technical leadership.Ensure data quality, governance, security, and performance across enterprise data platforms.Required Skills:10+ years of experience in Data Engineering.Strong hands-on experience with Microsoft Azure.Expertise in Azure Data Factory (ADF), Azure Databricks, Azure Synapse Analytics, and Azure Data Lake Storage (ADLS Gen2).Strong programming skills in Python, PySpark, and SQL.Experience with Delta Lake, Spark, and large-scale data processing.Experience with Azure DevOps, Git, and CI/CD pipelines.Basic to intermediate experience in Generative AI, such as working with Azure OpenAI, OpenAI APIs, LangChain, Prompt Engineering, RAG, or integrating LLMs into enterprise applications.Knowledge of vector databases (e.g., Pinecone, FAISS, or Azure AI Search) is a plus.
