Changes between Version 31 and Version 32 of HPC_deploy

Timestamp:

2015-11-19T02:40:13+01:00 (10 years ago)

Author:

Pieter Neerincx

Comment:

--

HPC_deploy

@@ -4 +4 @@
 
 == Deploying (reference) data sets ==
+
+=== Where to put reference data ===
+
+Reference data sets available to all (Hence not group specific data) can be deployed ''as-is'' in:
+{{{
+/apps/data/${provider}/${data_set}/$version/
+}}}
+When reference data must be modified for example because it must be indexed / reformatted for use with specific version of software, you must put the derived version in a sub dir to indicate it is not the original. When it was modified for a specific version of an app you could for example create additional sub dirs like this:
+{{{
+/apps/data/${provider}/${data_set}/$version/${app}/${version}/    
+}}}
+Always add a {{{/apps/data/${provider}/${data_set}/$version/README}}} with at least details on:
+* What the source location of the data was.
+* When it was download. 
+* If a derived flavor was created: how the data was modified (link to eLabjournal and/or code in our GitHub repos) and for what purpose.
+
+{{{#!comment
+TODO: add example for GRCh38
+
+/apps/data/GRC/GRCh/38/ * data gedownload "as is"; hooguit uitgepakt
+/apps/data/GRC/GRCh/38/BWA/0.7.12-goolf-1.7.20/ * een setje relatieve symlinks naar de referentie sequenties: ../../uitgepakte referentie fasta seqs
+                                          * bwa indices voor deze referentie
+}}}
+
+=== Syncing deployed reference data to nodes === #SyncRefData
+
+Before you can use reference data on cluster nodes it needs to be synced to various places.
+* Switch to the ''envsync'' user:
+{{{ 
+$> sudo -u umcg-envsync bash 
+}}}
+* Now sync the reference data by specifying the path to the data set relative to /apps/data/ (or specify the complete absolute path if you like to type). The sync will work recursively.
+{{{
+$> hpc-environment-sync.bash -r ReferenceData/
+}}}
+or
+{{{
+$> hpc-environment-sync.bash -r /apps/data/ReferenceData/
+}}}
+* For a full list of options use the commandline help:
+{{{
+hpc-environment-sync.bash -h
+}}}
 
 == Deploying software ==