For companies:
For professionals:
For everyone:
For developers:
Responsibilities
In your day-to-day, you will:
* Discover, assess, and catalog Product Engineering and network data sources across the organization.
* Ingest and migrate historical and ongoing data into Databricks using Azure Blob Storage, Auto Loader, mounts, and orchestrated workflows.
* Enrich and maintain metadata in Alation, including lineage, business glossaries, ownership, and data quality profiling.
* Design, build, test, and productionize ETL/ELT pipelines in Databricks (Spark, Delta Lake) to deliver high-quality datasets for AI/ML use cases.
* Optimize pipelines through performance tuning, partitioning, and incremental processing to support scalable analytics and ML workloads.
* Collaborate closely with Product Engineering, AI/ML, and network teams to resolve data discrepancies and incorporate network-specific context.
* Establish data quality standards, monitoring, documentation, and governance processes to ensure long-term sustainability.
Skills you have:
* 3–7 years of experience in data analysis, data engineering, or related roles.
* Strong experience with Databricks, Spark, Delta Lake, and ETL/ELT pipeline development.
* Proficiency in SQL and Python for data transformation and analysis (Scala is a plus).
* Experience working with cloud storage solutions such as Azure Blob Storage or equivalent.
* Basic understanding of network or telecommunications data (e.g., circuits, connectivity, performance metrics).
* Strong analytical mindset, attention to detail, and commitment to data quality and governance.
* Excellent communication skills and the ability to work independently in a fast-paced environment.
Skills that will make you stand out:
* Hands-on experience with Alation or other data catalog and governance tools.
* Experience supporting AI/ML data pipelines and feature engineering workflows.
* Experience with n8n or similar orchestration tools.
* Familiarity with advanced Databricks features such as Unity Catalog, Photon, or streaming pipelines.
* Prior exposure to large-scale data migration or network and product engineering datasets.
This job post is closed.