Test Engineer

Pensacola, FL

At Criterion Systems, we developed a different kind of business—a company whose real value is a reputation for excellence built upon the collective skills, talents, perspectives, and backgrounds of its people. By accepting a position with Criterion Systems, you will join a group of professionals with a collaborative mindset where we share ideas and foster professional development to accomplish our goals. In addition to our great culture, we also offer competitive compensation and benefit packages, company-sponsored team building events, and advancement opportunities. To find out more about how Criterion can help you take your career to the next level please visit our website: www.criterion-sys.com.

We are looking for candidates to help deliver full lifecycle development and sustainment for the Department of Homeland Security's (DHS) National Cybersecurity Protection System (NCPS). The NCPS assists more than 100 federal civilian government agencies with the protection of their networks against advanced cyber threats. Under this contract, you will support the government in the design, development, operations and maintenance services for the NCPS. Together we will build the mission critical cybersecurity solutions that will enable the timely dissemination of threat warnings and improve information sharing across DHS's stakeholder community.

The Transition to Operations Test Engineer will be engaged in the early conception of products, assisting with development oversight, and directly supporting the transition of products from conception & development into the production environment. They will write custom functions tests for the product or system to ensure it functions properly and meets the customer business needs. The job responsibilities will include but are not limited to the following: Testing all aspects of the product/system like function/ component, system, performance, regression and service. As a member of the Transition to Operations team, will participate in developing, enhancing, and facilitating policies, processes, procedures, and techniques with the goal of successfully transitioning new capabilities from design and development to a production environment.

Minimum Requirements

  • Four (4) or more years of related experience with a Bachelor’s degree in Math, Science, Engineering, Computer Science or related discipline
  • Professional experience may be considered in lieu of a degree.
  • Clearance Level: Must be a US Citizen with a current Secret clearance with the ability to obtain and maintain a Top Secret clearance with SCI access
  • Experience in Linux system administration and/or Windows server 2008, 2012 & 2016; supporting production systems and core utility services in an enterprise environment including Red Hat Enterprise 6-7 Linux (RHEL), Red Hat Virtualization, system and configuration management, and orchestration tools (i.e., Satellite Puppet, Ansible by Red Hat).
  • Demonstrated experience participating in projects and coordinating the efforts of associates in a globally distributed team
  • Positive approach toward customers; ability to cooperate and communicate well within a geographically distributed team
  • Full Chain Certificates & Java Key Store/Trust Stores: should have the knowledge of how certificates work both self-signed & full chain. They also need to know how to create and maintain Java key / trust stores.
  • Ability to work well both independently and in a team environment

Preferred Skills

  • Ability to write custom functions tests, create executive level reports with results, and brief leadership.
  • Experience with RedSeal
  • Know how to use Microsoft Project, Visio, Excel, and PowerPoint.
  • Experience developing scripts and automating tasks with shell and scripting languages like Bash, Perl, PowerShell & Python.
  • Ability to multi-task in a dynamic environment with changing priorities
  • Experience with the following commercial technologies: ArcSight, Nagios, Sourcefire, EADB | Aginaty, E-Mail/MEG, PGP, Tableau, Red Hat IDM, Logstash, Kafka Connect, Open Nebula, Free IPA, Who is, Ambaris Hadoop/Spark, Ansible, Docker, Open Shift, WSUS, Tenable Security Scanner & Security Center, RedSeal
  • Certification(s): Red Hat Certified System Administrator (RHCSA) OR Red Hat Certified Engineer (RHCE) certification & RHEL 6; 7 / Windows server 2008, 2012 & 2016

Criterion Systems is committed to hiring and retaining a diverse workforce. We are proud to be an Equal Opportunity/Affirmative Action Employer, making decisions without regard to race, color, religion, creed, sex, sexual orientation, gender identity, marital status, national origin, age, veteran status, disability, or any other protected class. For our complete EEO/AA and Pay Transparency statement, please visit https://criterion-sys.com/careers.



Criterion offers comprehensive health benefits including medical, dental, vision, life and disability insurance. Most of our plans are available at no cost for employee only coverage.

Time Off

Employees begin accruing PTO at 15 days per year and acquire more based on seniority. In addition to PTO, Criterion provides 10 holidays and bereavement, military, jury duty, and family medical leave.


  • Roth and Traditional 401(k) Plans with company matching contributions
  • Health Care and Dependent Care Flexible Spending Accounts
  • Health Savings Accounts
  • Commuter Benefits


All employees are eligible to use up to $3,000 annually for approved professional development, including trainings, memberships, seminars, and degree programs.

Employee Testimonials

Pets of Criterion

We love our furry friends!

Equal Employment Opportunity and Affirmative Action Employer

Criterion Systems, Inc. is committed to equal employment opportunity and non-discrimination at all levels of our organization. We believe in treating all applicants and employees fairly and make decisions without regard to an individual’s protected status: race/ethnicity, color, national origin, ancestry, sex/gender, gender identity/expression, sexual orientation, marital/parental status, pregnancy/childbirth or related conditions, religion, creed, age, disability, genetic information, veteran status, or any other protected status.

Know Your Rights

Applicants have rights under Federal Employment Laws: Family and Medical Leave Act | Equal Employment Opportunity | Employee Polygraph Protection Act. Criterion participates in E-Verify. Review Right to Work information.

Need an Accommodation?

Criterion is committed to Equal Employment Opportunity and providing reasonable accommodations to applicants with physical and/or mental disabilities. If you are interested in applying for a position with Criterion and need special assistance or an accommodation to apply, please send an email with your request to recruiting@criterion-sys.com or call us at 703-942-5800. Determination on requests for reasonable accommodation are made on a case-by-case basis.