VM Migration

Updated 4th February 2020

Introduction

This article provides users with information about work currently underway to migrate the JASMIN infrastructure from RedHat Enterprise 6 (RHEL6) to CentOS7 and preparing for the replacement of the LSF batch scheduler by SLURM.

As stated in these previous announcements, we are in the process of implementing a gradual transition of all "sci" servers and the LOTUS cluster to CentOS7.

Additionally, we will replace the existing batch scheduler (Platform LSF) with SLURM. We have chosen SLURM because it is a cost-effective alternative which is widely-used by other academic/scientific institutions (such as the UK Met Office).

Replacing the JASMIN Analysis Platform with JASPY

Work is underway to provide a new delivery method for the “JASMIN Analysis Platform” software, which is the stack of software currently maintained and deployed across the scientific analysis servers, the LOTUS batch processing clusterand a number of bespoke VMs which have been deployed.

The new system, known as “JASPY”, will be based around the "conda" system for creating the core Python environments in Python 2.7 and Python 3.7. JASPY is documented  here.

The JAP replacement takes the form of 2 new software components:

  • a new Conda-based set of Python environments: known as "Jaspy"
  • a Software Collections Library (SCL) which encapsulates the non-Python which were also part of the JAP: known as "jasmin-sci"

We have drafted documentation about the changes to software on JASMIN. The documentation is currently provided as a single web-based document (PDF). Please review the documentation here:

Draft documentation

Please let us know about your experience of using these CentOS7 machines, by e-mailing CEDA Support support@ceda.ac.uk including "centos7 support" in the subject line. Please send us both your positive and negative feedback so that we can gauge the response and react to problems. If you have any specific feedback regarding the documentation then please also let us know.

Migration of virtual machines used for hosting JASMIN and CEDA services, and project VMs

The CEDA and JASMIN team is busy working on this activity which should be largely transparent to users. Where individual project VMs are affected, projects have been contacted and plans are in preparation for the redeployment of those hosts with CentOS7 equivalents.

Next steps

The first scheduled changes are as follows:

  1. Two new sci servers, sci[1,2].jasmin.ac.uk, will be created with the CentOS7 systems and the new software environments. These two VMs will be available and listed under the Message Of The Day on jasmin-login1, from mid-February. [Please note that the domain name has changed from .ceda.ac.uk to .jasmin.ac.uk.
  2. The centos7 queue (currently used for testing) will be closed. A sub-cluster managed by SLURM will be deployed on CentOS7 nodes. We will update JASMIN users with details and a timeline soon.
  3. The transition period from LSF to SLURM is provisionally 6 months (February-31st July 2020). During this period, users are encouraged to migrate their workflows to the CentOS7 cluster which is managed by SLURM.

Migration of the transfer, login and other general purpose servers will be communicated in future announcements.

Thank you for your attention and please look out for further updates as this work progresses.

CEDA and JASMIN Team

This website and others run by CEDA use cookies. By continuing to use this website you are agreeing to our use of cookies.