| Les deux révisions précédentesRévision précédenteProchaine révision | Révision précédenteDernière révisionLes deux révisions suivantes |
| en:documentation:tutorials:scratch [2020/02/10 15:37] – [What is /scratch volume?] ltaulell | en:documentation:tutorials:scratch [2021/11/18 15:20] – [Network topology of PSMN] ltaulell |
|---|
| Two types of ''/scratch'' are available: | Two types of ''/scratch'' are available: |
| |
| | ===== Global scratch of a cluster ===== |
| |
| ==== • The global /scratch ==== | As we have various clusters of different generations of processors (Lake, the newest, E5 and X5, the former), we have several global scratches. |
| |
| Subdivided into 2 ''/scratch'' space : | ==== Cluster Lake ==== |
| |
| * <wrap hi> E5 scratch </wrap>, common to every ** E5-* ** queues | The ''/scratch/Lake/'', common to ** CLG* and SLG* ** queues is available for all [[documentation:clusters:hardware&#cluster_lake|cluster Lake nodes]]. |
| |
| This ''/scratch'' is accessible to all [[documentation:clusters:hardware&#cluster_equip_meso_ou_e5|Équip@méso nodes]] and to the front nodes **[[documentation:clusters:services|e5-2670comp1 & 2]]**, **[[documentation:clusters:services|e5-2667v4comp1 & 2]]**, | It contains ''/scratch/Lake/lake-gfs-scratch'', as a empty check file. |
| \\ as well as the visualization server **[[documentation:tutorials:visualisation|r730visu]]**. | |
| |
| It contains the empty verification file: ''/scratch/e5-2670-gfs-scratch''. | It is accessible from **[[documentation:clusters:services|cluster Lake front machines]]** |
| |
| | <note tip> |
| | **-At last-, ''/scratch/Lake'' is available** |
| | </note> |
| |
| * <wrap hi> X5 scratch </wrap>, common to every ** x55/x56* ** queues | <code> |
| | /scratch/ |
| | ├── Lake/ (common to Lake cluster) |
| | ├── disk/ (local to some servers) |
| | ├── Chimie/ (reserved to chimie usage) |
| | ├── Bio/ (reserved to biologie usage) |
| | ... |
| | └── Project_name/ (reserved to some servers, |
| | with dedicated hardware) |
| |
| This ''/scratch'' is accessible to all [[documentation:clusters:hardware&#cluster_x55_x56|x55/x56 nodes]] and to the front nodes **[[documentation:clusters:services|x5570comp1 & 2]]**. | </code> |
| |
| It contains the empty verification file: ''/scratch/x5570-gfs-scratch''. | ==== Cluster E5 ==== |
| | |
| | The ''/scratch/E5N/'', common to ** E5-* ** queues is available for all [[documentation:clusters:hardware&#cluster_e5|cluster E5 nodes]]. |
| | |
| | It contains ''/scratch/E5N/E5N-gfs-scratch'', as a empty check file. |
| | |
| | It is accessible from **[[documentation:clusters:services|cluster E5 front machines]]** |
| | |
| | |
| | <code> |
| | /scratch/ |
| | ├── E5N/ (E5 scratch, common to E5 cluster) |
| | ├── nvme/ (local to some servers) |
| | ├── ssd/ (local to some servers) |
| | ... |
| | └── Project_name/ (local to some servers, |
| | with dedicated hardware) |
| | |
| | </code> |
| | ==== Cluster X5 ==== |
| | |
| | The ''/scratch/X5/'', common to ** X5-* ** queues is available for all [[documentation:clusters:hardware&#cluster_x55_x56|cluster x55/x56 nodes]]. |
| | |
| | It contains ''/scratch/X5/x5570-gfs-scratch'', as a empty check file. |
| | |
| | It is accessible from **[[documentation:clusters:services|cluster x55/x56 front machines]]** |
| | |
| | <code> |
| | /scratch/ |
| | └── X5/ (X5 scratch, common to X5 cluster) |
| | </code> |
| | |
| | |
| | ==== Scratch local to some nodes ==== |
| | |
| | Some local ''/scratch'' are available only on some nodes. [[documentation:clusters:hardware|See detailled description]] or ask staff. |
| | |
| | |
| | ==== Network topology of PSMN ==== |
| |
| |
| \\ | \\ |
| The diagram below represents the integration of these two ''/ scratch'' spaces in the PSMN infrastructure. | The diagram below represents the integration of these ''/scratch'' spaces in the PSMN infrastructure. |
| \\ | \\ |
| |
| {{ :documentation:tutorials:psmn_network_synoptic.png?direct&660 |}} | {{ :documentation:tutorials:psmn_network_synoptic_a3_2021.png?direct&660 |}} |
| |
| <hidden> | |
| {{ :documentation:tutorials:psmn-network-synoptic.png?direct&640 |}} | |
| </hidden> | |
| \\ | \\ |
| |
| ===== Using the /scratch: manual copy ===== | ===== Using the /scratch: manual copy ===== |
| |
| For the global ''/scratch'', before submitting your job, copy your inputs into a folder on the ''/scratch'' (//ie// ''/scratch/login/whatever/''). | For global ''/scratch'', before submitting your job, copy your inputs into a folder on the ''/scratch/'' (//ie// ''/scratch/<cluster>/<login>/whatever/''). |
| The simplest is to reproduce the same hierarchy of folders and files by changing '' /home'' by ''/scratch'' (copy with '' cp -r ''). | |
| | The simplest is to reproduce the same hierarchy of folders and files by changing '' /home'' by ''/scratch/<cluster>''. |
| |
| Do not forget to modify your submitting files consequently. | Do not forget to modify your submitting files consequently. |
| ===== Using the /scratch : automatic copy ===== | ===== Using the /scratch : automatic copy ===== |
| |
| For local ''/scratch'' you don't know which node will be assigned to your job, it is also valid for the global scratch, but less annoying). Thus, You cannot copy your data to a particular node in advance. You must integrate the copy ''/scratch'' <-> ''/home'' in the submitted script. | For local ''/scratch'' you don't know which node will be assigned to your job (it is also valid for the global scratch, but less annoying). Thus, You cannot copy your data to a particular node in advance. You must integrate the copy ''/scratch'' <-> ''/home'' in the submitted script. |
| |
| |
