What is /scratch volume?

The /scratch is a temporary workspace that you can use to copy your input data and write your calculation outputs. The pros of working in the /scratch are the following:

  • offers a bigger volume than the volume available in your /home space (currently 2 /scratch space of ~ 35To are available)
  • increase processing speed of reads/writes
  • increase the simultaneous I/O on the volume.

Two types of /scratch are available:

• The global /scratch

Subdivided into 2 /scratch space :

  • E5 scratch , common to every E5-* queues

This /scratch is accessible to all Équip@méso nodes and to the front nodes e5-2670comp1 & 2, e5-2667v4comp1 & 2,
as well as the visualization server r730visu.

It contains the empty verification file: /scratch/e5-2670-gfs-scratch.

  • X5 scratch , common to every x55/x56* queues

This /scratch is accessible to all x55/x56 nodes and to the front nodes x5570comp1 & 2.

It contains the empty verification file: /scratch/x5570-gfs-scratch.

The diagram below represents the integration of these two / scratch spaces in the PSMN infrastructure.

Cliquez pour afficher ⇲

Cliquez pour masquer ⇱

• Locals /scratch:

These locals /scratch are only avaiable in some nodes of the cluster see more complete description .

How to use the scratch

Using the /scratch: manual copy

For the global /scratch, before submitting your job, copy your inputs into a folder on the /scratch (ie /scratch/login/whatever/). The simplest is to reproduce the same hierarchy of folders and files by changing /home by /scratch (copy with cp -r ).

Do not forget to modify your submitting files consequently.

At the end of the run, retrieve your results manually, in the / scratch, from any of the corresponding front nodes.

Using the /scratch : automatic copy

For local /scratch you don't know which node will be assigned to your job, it is also valid for the global scratch, but less annoying). Thus, You cannot copy your data to a particular node in advance. You must integrate the copy /scratch/home in the submitted script.

Examples are available here.

We don't have a sample script that cleans up automatically. Similarly, if you don't feel comfortable with these scripts, do not hesitate to come and talk about during a Tuesday coffee session.

Technological transfer

The global /scratch (glusterfs) were deployed thanks to the upstream work, recipes and integrations made on the CBP experimental platform.

en/documentation/tutorials/scratch.txt · Dernière modification: 2018/05/25 15:23 par cpetit