Mainframe Systems Programmer - Senior - AS400 System Administrator

Essential Duties and Responsibilities:

IBM Power 9 I Series (AS400) Systems Programming (System 36 and Native Mode)

• Maintain, upgrade, monitor and report on all IBM iSeries systems and HMC infrastructure, including PTFs and firmware upgrades, errors, failures, and security aspects
• Maintain and monitor daily and system backups along with Replication/DR processes
• Maintain system journaling of databases
• Research and resolve systems issues as needed
• Manages installation, and configuration of AS400 software applications, including third party software and system management tools. 
• Engage with third party vendors and support as needed
• Assist with client emulation configuration and connectivity
• Manages AS400 user and device configuration
• Manages and helps update AS400 security policies
• Support interfaces between DLT and other agencies.  Manages security and networking interconnectivity between systems.
• Assist with developing, maintaining and executing disaster recovery protocols for the agency devices and integration with Enterprise teams as well as perform periodic testing
• Assist with technical and business process with development of Business Continuity Plan (BCP) and Continuity of Operations (COOP) plans
• Assist EOC operators and programmers with runtime errors
• Check and maintain network configurations, routing, host tables and DNS.
• Performance Monitoring: Utilize performance tools to analyze system performance metrics. Identify any performance bottlenecks and take corrective actions, including but not limited to performance, critical messages, errors.
• System Values: Maintain and review system values to ensure they are configured for security and performance.
• Subsystem Monitoring: Review subsystems to ensure they are running optimally. Check for any subsystem-related errors or performance issues.
• Disk Space: Review disk space utilization and run disk reports to address issues.
• Job Queues and Job Scheduler:  Deploy new jobs into the schedule, examine job queue configuration. Review the job scheduler to ensure scheduled jobs are running as expected and report on success/failure and triage exceptions for all scheduled items including but not limited to scheduled jobs, file transfers, file locks, message waiting, email alert and report notifications.
• Security: Maintain and review user profiles and system values to ensure proper user system access and security.  Check object authorities to sensitive objects and data.
• Backup and Recovery:  Confirm that regular backups are scheduled and completed successfully. Verify the integrity of backup media and perform a test restore if requested. Currently leveraging legacy tapes in a tape library, LaserVault.
• Network Configuration: Check network configurations, routing, host tables and DNS.
• Performance Monitoring: Utilize performance tools like Performance Navigator or IBM Navigator to analyze system performance metrics. Identify any performance bottlenecks and take corrective actions. 
• Journaling and Logging: Review journals and receivers. Check system logs (DSPLOG) for any unusual or critical messages.
• Printing and Output Queues: Verify that printers and output queues are configured correctly. Monitor for any spool file issues or printer errors.
• Backup and Recovery Plan: Ensure there is a well-documented backup and recovery plan is in place.  Test and maintain plan.
• Monitors and provides system health check documentation 
• Develop and maintain up-to-date documentation on the system's configuration, settings, and document any changes
• Assist in developing technical documentation and Standard Operating Procedures
• Assist with technical documentation, audits, reviews, and processes
• Provide technical expertise to implement best practices
• Provide operational, status, and other reports as requested
• Engage with third part vendors and support teams as required
• Performs other duties as assigned

Assist Maintain, Support, and Manage support of IBM Mainframe systems:

• Assist EOC operators with runtime errors to address programming, OS, and hardware issues.
• Monitor and report on System Status including but not limited to performance, critical messages, errors.  Triage and address issues.
• Maintain and review system values to ensure they are configured for security and performance.
• Review partitions to ensure they are running optimally. Check for any partition-related errors or performance issues.
• Patches and Updates:  Verify that the system is running the latest cumulative PTF (Program Temporary Fix) package. Check for the latest operating system updates and patches. 
• Disk Space: Review disk space utilization and run disk reports to address issues
• Job Queues and Job Scheduler:  Deploy new jobs into the schedule, examine job queue configuration. Review the job scheduler to ensure all scheduled jobs are running as expected and report on success/failure and triage exceptions including but not limited to scheduled jobs, file transfers, file locks, message waiting, email alert and report notifications.
• Security: Maintain and review user profiles and system values to ensure proper user system access and security.  Check object authorities to sensitive objects and data.
• Backup and Recovery:  Confirm that regular backups are scheduled and completed successfully. Verify the integrity of backup media and perform a test restore if requested.
• Network Configuration: Check network configurations, routing, host tables and DNS.
• Performance Monitoring: Utilize performance tools to analyze system performance metrics. Identify any performance bottlenecks and take corrective actions. 
• Logging:  Check system logs (MLOG) for any unusual or critical messages and report and take action as directed and required.
• Printing and Output Queues: Verify that printers and output queues are configured correctly. Monitor for any spool file issues or printer errors.
• Backup and Recovery Plan: Ensure there is a well-documented backup and recovery plan is in place.  Test and maintain plan.
• System Health Check Documentation:  Maintain up-to-date documentation on the system's configuration, settings, and document any changes.  Assist in developing technical documentation and Standard Operating Procedures
• Assist with developing, maintaining and executing disaster recovery for the agency devices and integration with Enterprise teams.
• Assist with technical and business process with development of BCP and COOP plans.
• Assist with technical documentation, audits, reviews, and processes
• Provide technical expertise to implement best practices.
• Assist in migration efforts to modernize the legacy applications.
• Provide operational, status, and other reports as requested.

EXPERIENCE & SKILLS

Experience and skills to support and perform the duties and responsibilities described above. 

• Minimum 3-5 years’ experience
• Ability to communicate effectively with staff and engage business, technical resources, and support vendors
• COBOL programming experience
• AS400 Administration experience
• Go Anywhere Integration, ODBC, SFTP, FTP, Mocha, Rumba, Blue Zone emulation software, and HIS system experience
• Expertise in creating tables, dropping tables, adding fields, indexes, maintaining and resolving database issues, and database recovery processes (various databases Ex: DB2, SQL, SQL/DS, VSAM databases)
• Familiarity with IBM OS utilities, MVS JCL, VSE JCL, Easytrieve Plus, Message Log (MLOG) and intrinsic functions
• Experience with XL Print Paris (forms & programming), Visio, MS Office, Teams, Jira and other common tools
• Other experience as necessary to perform the system administration and programming tasks required.
• Database recovery
• Troubleshoot and work with affected agencies to resolve problems.
• Experience in application conversion to outside vendors