Now hiring!
You have opted out of job matches in .
To undo this, go to the 'Skills and Interests' section of your preferences.
Posted over 2 years ago
Powered by
Requirements and responsibilities
🚀 Shift4 offer re-location benefits.
We are looking for an AWS Engineer to join our team of world-class in-house engineers. As an AWS Engineer III, you will be responsible for designing, developing, and maintaining robust data infrastructure solutions to support the organization's data-driven initiatives. We are looking for self-driven and motivated individuals who take ownership of their projects. You will work closely with cross-functional teams, including project managers, business analysts, and software engineers, to ensure the availability, reliability, and scalability of our systems. This role requires expertise in AWS data engineering. We are looking for individuals that are extremely self-sufficient, available to work flexible hours and hold themselves to the highest standards of professionalism.
Your responsibilities should be:
- Develop and maintain automated data pipelines and ETL processes to ingest, transform, and load large volumes of data from diverse sources into relational databases utilizing AWS architecture, S3, Glue, Lamba, Step Functions, RDS, Kinesis, DMS, Redshift, - PostgreSQL, and other database technologies to move data.
- Ensure the quality, integrity, and security of data by implementing data validation, data cleansing, and data governance processes.
- Work on necessary new system developments, enhancements and bugs utilizing an agile software development lifecycle (SDLC).
- Optimize and fine-tune data processes and systems for performance, scalability, and reliability.
- Monitor and troubleshoot data pipelines, data processing jobs, and database performance issues, including off hours support when necessary.
- Stay up-to-date with emerging data engineering technologies, tools, and best practices, and recommend improvements to existing data infrastructure.
- Collaborate with software engineers to integrate data engineering solutions into applications and software products.
- Document data engineering processes, data models, and system configurations for knowledge sharing and future reference.
Qualifications:
- Bachelor's or Master's degree in Computer Science, Information Systems, or a related field.
- Minimum 5 years of professional experience in AWS data development or a related field.
- Advanced experience working with AWS: S3, Glue, Lamba, Step Functions, RDS, Kinesis, DMS, Redshift.
- Advanced experience working with various data sources such as AWS S3 and Redshift, Postgres, SQL Server, Amazon Athena, Excel, Flat Files, etc.
- Advanced experience working with large data sets from sources such as AWS S3 and Redshift, Postgres, SQL Server, etc.
- Strong PySpark, Python, CLI proficiency.
- Advanced experience with relational databases like MSSQL, PostgreSQL, or AWS Redshift.
- Strong problem-solving and analytical skills, with a keen attention to detail.
- Experience working with an Enterprise Data Warehouse and Dimensional Databases is a plus.
- Experience working in an agile software development lifecycle (SDLC).
- Experience with Jira and Confluence is a plus.
- Ability to prioritize multiple tasks and easily adjust to changing priorities.
- Ability to identify problems, initiate solutions, and effectively collaborate and communicate with team members.
- Have excellent verbal and written communication skills.
*However, please note that our availability to sponsor visas or work permits for this position is limited. You must have the legal authorization to have a long-term full-time employee contract in the US.*
Optionally, you can add more information later (benefits, pre-screening questions, etc.)