Senior IT Data Engineer at Smithfieldfoods | Torre

Senior IT Data Engineer

Emma highlights
This highlight was written by Emma’s AI. Ask Emma to edit it.
Full-time

Legal agreement: To be defined

Base compensation
USD85.5k - 127k/year

+ Health insurance

location_on
Hybrid (United States)
Posted over 2 years ago

Responsibilities


- $18 billion U.S. food company with nearly 60,000 employees worldwide. - Competitive Pay: $85,500– $127,050 annually - Comprehensive Health Insurance, Retirement Benefits and More - Education benefit available to full and part time Smithfield team members on their first day of employment. - Opportunities for career growth, professional development, and tuition assistance. We are looking for a Senior IT Data Engineer to join our team and work on developing and supporting a wide variety of data engineering pipelines utilizing tools including Azure Databricks, Azure Data Factory, Azure Data Lake Store, and Azure Synapse. The successful candidate will work hand-in-hand within a dedicated team of business analysts, data analysts, business SMEs, data engineers, and data architects to develop and support the next generation of Smithfield’s data engineering and analytics solutions. This is a great opportunity for someone with advanced knowledge of data management, data integration, and data platform development techniques, as well as experience with Microsoft Azure technologies. Responsibilities: - Implement complex production scheduling logic utilizing a combination of SQL and Python on the Apache Spark/Databricks platform - Design, develop, deploy and support data pipelines using a combination of SQL and Python functions, programs, routines and stored procedures to export, transform and load data - Create and maintain code module design documents and run books - Follow standard processed for management of pipeline orchestration, source code control, error/event handling, CI/CD, access control - Partner with data architects and analysts to adopt reusable data pipeline design patterns - Work with IT, business and data management teams to understand data needs and develop solutions - Develop and execute test plans and scenarios including data design, tool design, data extract/transform, networks and hardware - Work with Business Analysts and other team members to support business case creation - Work with Business Analysts and other team members to understand requirements for data use, analysis, and quality - Work with Data Analysts, Data Engineers and Data Architects to design data repositories and pipelines - Develop solutions using architectural standards and consistency - Assist in the development of coding and documentation standards - Document code according to standards - Assist in assessing solution option risks and develop mitigation plans - Promote continuous improvement of operational solutions - Provide mentoring to junior team members