| Les deux révisions précédentesRévision précédenteProchaine révision | Révision précédente |
| en:documentation:tutorials:scratch [2018/05/25 12:58] – cpetit | en:documentation:tutorials:scratch [2023/12/12 12:57] (Version actuelle) – supprimée ltaulell |
|---|
| ====== What is /scratch volume? ====== | |
| |
| |
| The ''/scratch'' is a **//temporary//** workspace that you can use to copy your input data and write your calculation outputs. | |
| The pros of working in the ''/scratch'' are the following: | |
| * offers a bigger volume than the volume available in your ''/home'' space (currently 2 ''/scratch'' space of ~ 35To are available) | |
| * increase processing speed of reads/writes | |
| * increase the simultaneous I/O on the volume. | |
| |
| \\ | |
| Two types of ''/scratch'' are available: | |
| |
| |
| ==== • The global /scratch ==== | |
| |
| Subdivided into 2 ''/scratch'' space : | |
| |
| * <wrap hi> E5 scratch </wrap>, common to every ** E5-* ** queues | |
| |
| This ''/scratch'' is accessible to all [[documentation:clusters:hardware&#cluster_equip_meso_ou_e5|Équip@méso nodes]] and to the front nodes **[[documentation:clusters:services|e5-2670comp1 & 2]]**, **[[documentation:clusters:services|e5-2667v4comp1 & 2]]**, | |
| \\ as well as the visualization server **[[documentation:tutorials:visualisation|r730visu]]**. | |
| |
| It contains the empty verification file: ''/scratch/e5-2670-gfs-scratch''. | |
| |
| |
| * <wrap hi> X5 scratch </wrap>, common to every ** x55/x56* ** queues | |
| |
| This ''/scratch'' is accessible to all [[documentation:clusters:hardware&#cluster_x55_x56|x55/x56 nodes]] and to the front nodes **[[documentation:clusters:services|x5570comp1 & 2]]**. | |
| |
| It contains the empty verification file: ''/scratch/x5570-gfs-scratch''. | |
| |
| |
| \\ | |
| The diagram below represents the integration of these two ''/ scratch'' spaces in the PSMN infrastructure. | |
| \\ | |
| |
| {{ :documentation:tutorials:psmn_network_synoptic.png?direct&660 |}} | |
| |
| <hidden> | |
| {{ :documentation:tutorials:psmn-network-synoptic.png?direct&640 |}} | |
| </hidden> | |
| \\ | |
| |
| |
| ==== • Locals /scratch: ==== | |
| |
| These locals ''/scratch'' are only avaiable in some nodes of the cluster [[documentation:clusters:hardware&#clusters_mutualises|see more complete description ]]. | |
| |
| |
| ====== How to use the scratch ====== | |
| ===== Using the /scratch: manual copy ===== | |
| |
| For the global ''/scratch'', before submitting your job, copy your inputs into a folder on the ''/scratch'' (//ie// ''/scratch/login/whatever/''). | |
| The simplest is to reproduce the same hierarchy of folders and files by changing '' /home'' by ''/scratch'' (copy with '' cp -r ''). | |
| |
| Do not forget to modify your submitting files consequently. | |
| |
| At the end of the run, retrieve your results manually, in the ''/ scratch'', from any of the corresponding front nodes. | |
| |
| ===== Using the /scratch : automatic copy ===== | |
| |
| Pour les ''/scratch'' locaux, vous ne savez pas quel noeud vous sera attribué (c'est valable aussi pour le scratch global, mais moins génant). Vous ne pouvez donc pas copier vos données vers un noeud en particulier à l'avance. Il faut intégrer la copie ''/scratch''<->''/home'' dans le script soumis. | |
| |
| **Vous trouverez [[documentation:examples:submit_scripts&#job_parallele_variantes|des exemples sur cette page]].** | |
| |
| |
| Nous n'avons pas d'exemple de script qui fasse le ménage de manière automatique. De même, si vous ne vous sentez pas à l'aise avec ces scripts, n'hésitez pas à venir en discuter autour d'un café, le lundi après-midi. | |
| |
| |
| ===== Technological transfer ===== | |
| |
| <WRAP info> The global /scratch (glusterfs) were deployed thanks to the upstream work, recipes and integrations made on the CBP experimental platform.</WRAP> | |
| |
