Différences

Ci-dessous, les différences entre deux révisions de la page.

Lien vers cette vue comparative

Les deux révisions précédentesRévision précédente
Prochaine révision
Révision précédente
en:documentation:tutorials:scratch [2020/02/10 15:37] – [What is /scratch volume?] ltaulellen:documentation:tutorials:scratch [2023/12/12 12:57] (Version actuelle) – supprimée ltaulell
Ligne 1: Ligne 1:
-====== What is /scratch volume? ====== 
- 
- 
-The ''/scratch'' is a **//temporary//** workspace that you can use to copy your input data and write your calculation outputs. 
-The pros of working in the ''/scratch'' are the following: 
-  * offers a bigger volume than the volume available in your ''/home'' space (many ''/scratch'' workspaces are available), 
-   * increase processing speed of reads/writes, 
-   * increase the  simultaneous I/O. 
-\\ 
- 
-Scratches are **shared and common storages**, their performances depends on your proper use. Hence, they must not contain: 
- 
-  * documentation (lots of useless small files) ''=> /home'', 
-  * symbolic links (tiny files), 
-  * programs or exe (useless I/O) ''=> /home''. 
- 
-<note tip>**Exclusively input, temporary or output data**</note> 
- 
- 
-Two  types of ''/scratch'' are available: 
- 
- 
-==== • The global /scratch  ==== 
- 
-Subdivided into 2 ''/scratch'' space : 
- 
-  * <wrap hi> E5 scratch </wrap>, common to every ** E5-* ** queues 
- 
-This ''/scratch'' is accessible to all [[documentation:clusters:hardware&#cluster_equip_meso_ou_e5|Équip@méso nodes]] and to the front nodes **[[documentation:clusters:services|e5-2670comp1 & 2]]**,   **[[documentation:clusters:services|e5-2667v4comp1 & 2]]**, 
-\\ as well as the visualization server  **[[documentation:tutorials:visualisation|r730visu]]**. 
- 
-It contains the empty verification file: ''/scratch/e5-2670-gfs-scratch''. 
- 
- 
-  *  <wrap hi> X5 scratch </wrap>,  common to every ** x55/x56* ** queues  
- 
-This ''/scratch'' is accessible to all [[documentation:clusters:hardware&#cluster_x55_x56|x55/x56 nodes]] and to the front nodes **[[documentation:clusters:services|x5570comp1 & 2]]**. 
- 
-It contains the empty verification file: ''/scratch/x5570-gfs-scratch''. 
- 
- 
-\\ 
-The diagram below represents the integration of these two ''/ scratch'' spaces in the PSMN infrastructure. 
-\\ 
- 
-{{ :documentation:tutorials:psmn_network_synoptic.png?direct&660 |}} 
- 
-<hidden> 
-{{ :documentation:tutorials:psmn-network-synoptic.png?direct&640 |}} 
-</hidden> 
-\\ 
- 
- 
-==== • Locals /scratch: ==== 
- 
-These locals ''/scratch'' are only avaiable in some nodes of the cluster [[documentation:clusters:hardware&#clusters_mutualises|see more complete description ]]. 
- 
- 
-====== How to use the scratch ====== 
-===== Using the /scratch: manual copy ===== 
- 
-For the global ''/scratch'', before submitting your job, copy your inputs into a folder on the ''/scratch'' (//ie// ''/scratch/login/whatever/''). 
-The simplest is to reproduce the same hierarchy of folders and files by changing '' /home'' by ''/scratch'' (copy with '' cp -r ''). 
- 
-Do not forget to modify your submitting files consequently.  
- 
-At the end of the run, retrieve your results manually, in the ''/ scratch'', from any of the corresponding front nodes. 
- 
-===== Using the /scratch : automatic copy ===== 
- 
-For local ''/scratch'' you don't know which node will be assigned to your job, it is also valid for the global scratch, but less annoying). Thus, You cannot copy your data to a particular node in advance. You must integrate the copy ''/scratch'' <-> ''/home'' in the submitted script. 
- 
- 
- 
-**Examples are available  [[documentation:examples:submit_scripts&#job_parallele_variantes|here]].** 
- 
- 
-We don't have a sample script that cleans up automatically. Similarly, if you don't feel comfortable with these scripts, do not hesitate to come and talk about during a Tuesday coffee session. 
- 
- 
-===== Technological transfer ===== 
- 
-<WRAP info> The global /scratch (glusterfs) were deployed thanks to  the upstream work, recipes and integrations made on the CBP experimental platform.</WRAP> 
  
en/documentation/tutorials/scratch.1581349041.txt.gz · Dernière modification : 2020/08/25 15:58 (modification externe)