Onyx Point works with Exascale Computing Project to bring CI to supercomputing centers.

Back to Listing

Jessica Sauers

Hanover, MD, 18 June 2018

Energy science hinges on High Performance Computing (HPC), and U.S. Department of Energy (DOE) scientists write millions of lines of scientific simulation code every year to drive new discoveries. Simulations need to be tested and kept up to date, and the demand among scientists for robust continuous integration (CI) is growing. While HPC centers at the U.S. national laboratories are home to many of the world’s most powerful supercomputers, existing CI solutions could not meet the security and compliance requirements for users of these unique multi-user systems. While much of the DOE codebase is open source, it is critical for HPC centers to protect access to sensitive codes and data, and to prevent unauthorized use of expensive and highly customized hardware platforms. Existing CI systems are not up to the task of true multi-tenant security.

Onyx Point, LLC., with support from the DOE’s Exascale Computing Project (ECP), has been given the opportunity to deliver a secure, compliant, and easy-to-use CI solution for DOE HPC facilities. With their extensive technical expertise and drive to provide secure, compliant, automated, and efficient DevOps solutions, Onyx Point, LLC. will play a critical role in enhancing the workflow of DOE computational scientists by building consolidated testing systems that will provide users with easy access to automated builds at HPC facilities. The solution will allow scientists to test open source projects and internal projects automatically and securely on some of the world’s largest and fastest machines. This is intended to dramatically increase the reliability, quality, and performance of the HPC software ecosystem – one of the key goals of the ECP.

Partnering with the ECP, Onyx Point, LLC. will provide a policy-compliant security standardization for the agency’s HPC infrastructure by leveraging a comprehensive code-based solution using the open source GitLab tool. Onyx Point will add new features to GitLab that allow it to run jobs with the appropriate security credentials, and integrate with the batch systems, like the open source SLURM tool, that are used to run massively parallel HPC jobs. This CI solution will provide consistent management of CI job execution across both the public and private portions of the DOE’s HPC infrastructure. Instances can be deployed by individual DOE facilities for local users, or by the agency for all of its HPC centers. The solution is intended to meet the security requirements of a shared-use environment like those found at DOE’s large-scale HPC facilities and leverage GitLab’s well-defined framework to extend new capabilities to a larger environment more efficiently.

The solution will enable developers using DOE HPC facilities to leverage automated systems to run tests on their codebase while preserving security and access levels to those automated systems. By default, the automated system running CI must only have access to resources for which the user running those tests also has access. This ensures that test code cannot interact with shared resources without proper credentials. By isolating each process that is created, the data remains protected from processes created by other users on the same multi-tenant node. In addition, Onyx Point, LLC. is extending the auditing capabilities of GitLab’s automated testing solutions.

Onyx Point, LLC., a firm committed to DevOps best practices, believes this solution will increase the efficiency of the research performed at HPC facilities by providing faster feedback, streamlining the development processes, and allowing more effective use of the laboratories’ HPC resources. As an official GitLab partner, we are excited to work side by side with the DOE to provide a protected security layer for the GitLab CI Runners utilized throughout their national laboratories. Onyx Point will also be working with GitLab to add the new features to the mainline GitLab product, so that other agencies and organizations can leverage the new security capabilities. Onyx Point, LLC. and the ECP have recognized how powerful these tools are in aiding the DevOps workflow. Our security code will help standardize lab processes, create a baseline for continuous collaboration, and enhance existing workflows. This solution will allow DOE researchers and scientists to focus on managing their critical projects with confidence that their data is protected.

At Onyx Point, our engineers focus on Security, System Administration, Automation, Dataflow, and DevOps consulting for government and commercial clients. We offer professional services for Puppet, RedHat, SIMP, NiFi, GitLab, and the other solutions in place that keep your systems running securely and efficiently. We offer Open Source Software support and Engineering and Consulting services through GSA IT Schedule 70. As Open Source contributors and advocates, we encourage the use of FOSS products in Government as part of an overarching IT Efficiencies plan to reduce ongoing IT expenditures attributed to software licensing. Our support and contributions to Open Source, are just one of our many guiding principles

  • Customer First.
  • Security in All We Do.
  • Pursue Innovation with Integrity.
  • Communicate Openly and Respectfully.
  • Offer Your Talents, and Appreciate the Talents of Others

gitlab, exascale computing project, ci, devops, DOE, HPC

Share this story

We work with these Technologies + Partners