Job SummaryNRTC System Engineers handle the installation, maintenance, and troubleshooting of servers and applications used by NRTC members to support their ISP networks, as well as developing, maintaining, and implementing tools to import member end user data into NRTC’s systems. The System Engineers works with a team to remotely administer NRTC servers on-site at member locations, providing services such as caching DNS, DHCP, network speed tests, and radius authentication.ResponsibilitiesDevelop, maintain, and implement tools and scripts to import new member end user customer data into NRTC systems in a secure and efficient mannerManage NRTC’s remote Network Utility Servers deployed in member network to support ISP servicesConfigure and manage DHCP IPv4 and IPv6 IP systems for ISP network, including configuration of DHCP options for advanced logging and analysisConfigure and manage radius authentication infrastructure for member ISP services, including BNG and PPPoE configurationsProvide members with best practices for supported systemsOnboarding and management of member authoritative and caching DNS servicesOnboarding of member email hosting, email client configuration, email platform managementProvide support for engineering and professional servicesMonitor and maintain system uptime, security, virtual environments, and disaster recoverySupport team members with the installation and configuration of Linux systems, including updating and maintaining automatic installation toolsImplement and support Linux environments for NRTC members, including troubleshooting and problem resolution for operating systems, application configuration, and hardware issuesTroubleshoot and diagnose issues reported by members and internal technical support teamsWork independently or as part of a team to diagnose and troubleshoot issuesExercise good judgment when seeking assistance to solve or troubleshoot most issuesSupport NRTC member change requests and conversionsCreate scripts and processes for automation of repetitive tasksCreate and maintain up to date documentation on specific member configuration and troubleshooting stepsPerform change management activities, including code upgrades to production systemsMentor teammates on current tools, processes, and changes to member system configurationsStay current on new technologiesOther duties and responsibilities as assignedQualificationsKnowledge, Skills, and AbilitiesAdvanced level knowledge of all aspects of administration for Linux operating systems including installation, management, application deployment, database management, and securityThorough understanding of DHCP, authoritative and caching DNS, AAA IP management (radius)Thorough understanding of email delivery, protocols, troubleshooting, and server applications (Postfix, Dovecot, Zimbra)Thorough understanding of container management (Docker, Kubernetes, Podman, etc.)Thorough understanding and experience with version control software (GIT, SVN, CVS, etc.)Strong experience troubleshooting, debugging and root cause analysis of system outages and slowdownsExperience managing virtualization technologies such as VMware, Xen and Hyper-VExperience integrating with API interfacesExperience with major incident managementAble to automate regular tasks using common scripting languages (Bash, Python, PHP, or Perl)Able to understand and update existing scripts in common scripting languages (Bash, Python, PHP, or Perl)Ability to communicate and solve problems efficiently and effectively with identified customers and IT personnel to ensure the implementation of accurate processesExperience administrating authoritative and caching DNS servers, including advanced configurations such as split DNSExperience obtaining and deploying SSL certificates using automated tools such as Let’s Encrypt certbotExperience with monitoring software (Zabbix, Nagios, PRTG)Experience with automation tools (Ansible, Chef, Salt, etc.)Advanced ability to be proactive, detail-oriented, and self-motivatedStrong teamwork skillsMinimum Education and ExperienceBachelor's degree in computer science or related fieldFive (5) years of experience managing Linux server infrastructure, including troubleshooting, physical installations, modifications, replacements and configuration changes and log analysisLinux Foundation Certified System Administrator or equivalent desiredAn equivalent combination of education and experience may be consideredPhysical DemandsWork is typically performed in an office and/or remote settingParticipation in a weekly on-call rotation to provide 24x7 support for ISP services managed by NRTCThis role will require the ability to team lift at least 80 poundsMinimum domestic travel may be possible, approximately 5%