Migrating old data from Hamilton7

Our previous HPC system, Hamilton7, was superseded by Hamilton8 in early 2022 and has been retired from service. If you were a user of Hamilton7 and still hold data there, it is important that you do the following before the end of June:

• copy any data you wish to keep, either to Hamilton8 or another service
• check that your data have been copied successfully
• delete your remaining data from Hamilton7, especially from /ddn/data.

This page contains information about storing your data on Hamilton8.  For secure backups of your data, you may also wish to make use of the University's research storage facilities (PRS, SRS), which are managed by CIS.

Storage options on Hamilton8

Like Hamilton7, Hamilton8 is not suitable for processing sensitive data. The University library offers advice on research data management during a project.

You can migrate Hamilton7 data to the following areas:

Home directories - $HOME - Your Hamilton8 home directory has a quota of 10GB. As with Hamilton7, this directory is backed up.

Data directories - $NOBACKUP - /ddn/data/<username> has been replaced by /nobackup/<username>, which is accessible from all nodes and is intended for large datasets. The data directories reside on a 1.9PB Lustre filesystem and are not backed up. All users have been given a default quota of 600GB; if you require a quota above this then please get in touch with the Hamilton team.

To migrate your data from Hamilton7:

• Arrange to backup any files from /ddn/data that you wish to keep long-term.  The PRS and SRS service offered by CIS provides secure storage suitable for this.  If you do not already have an allocation, you will need to request either PRS or SRS space, and you will also need to request an account on the CIS linux service, mira.  Once you have these, follow the Hamilton8 instructions on how to transfer data to PSR/SRS.
• Copy your files to Hamilton8 using a command such as scp or rsync. You may find this process faster if you 'tar' directories and/or compress your files first. For a simple transfer of all a directory called <directory_name> from /ddn/data, for example, type the following  at a Hamilton7 prompt:

scp -pr /ddn/data/$USER/<directory_name>  hamilton8.dur.ac.uk:/nobackup/<your_username>

scp -pr $HOME/<directory_name>  hamilton8.dur.ac.uk:/home/<your_username>

• Once your files have been copied successfully and you have migrated all your work to Hamilton8, please remove the original files from Hamilton7, especially from /ddn/data. This will help us assess the number of accounts still to be transferred.

Please contact the Hamilton team at arc-rcp@durham.ac.uk if you need help to transfer your files.