Changes between Initial Version and Version 1 of TransparentMultiHopSSHNewLobby

Timestamp:

2021-07-06T22:03:47+02:00 (4 years ago)

Author:

Pieter Neerincx

Comment:

--

TransparentMultiHopSSHNewLobby

@@ +1 @@
+= Transparent multi-hop SSH=
+
+== Connecting from outside/home via a proxy server a.k.a. jump host ==
+
+[[TOC()]]
+
+Our servers can only be accessed using a jumphost (a.k.a. proxy) server. This requires port 22 to be open on all machines between your computer and our jumphost server. [[BR]]
+The existing {{{lobby.hpc.rug.nl}}} will shortly be replaced by a new machine with the same name. This new machine currently has no DNS entry yet; the example config below is for the new {{{lobby}}} jumphost and is based on its IP address only.
+
+Note that previously there was also a jumphost named lobby.hpc.rug.nl, but this one has been deprecated. Furthermore there many other machines specified in the example config like {{{flexo}}} and {{{bender}}}. Please remove all entries for these machines from your {{{~/.ssh/config}}} file:
+ * The only line that should remain in your ~/.ssh/config for our machines is 
+ * Additional lines that are required to access our machines are added to a separate {{{~/.ssh/conf.d/calculon}}} file now.
+
+== Users on Linux / Unix / Mac OS X ==
+
+To setup transparent multi-hop SSH:
+ * On the machine from which you want to connect
+   * Make sure you have a **''~/.ssh''** folder with generated SSH keys linked to your account. See [wiki:RequestAccount] for instructions.
+   * Create **''~/.ssh/tmp''** and **''~/.ssh/conf.d''** folders. Open a terminal and type the following command:
+{{{
+mkdir -p ~/.ssh/tmp
+mkdir -p ~/.ssh/conf.d
+}}}
+   * Create a **''~/.ssh/config''** file if it does not exist yet. 
+   * Add to your **''~/.ssh/config''** something like the following:
+{{{
+#
+# Generic stuff: only for MacOS clients.
+#
+IgnoreUnknown UseKeychain
+        UseKeychain yes
+IgnoreUnknown AddKeysToAgent
+        AddKeysToAgent yes
+
+#
+# Generic stuff: prevent timeouts 
+#
+Host *
+        ServerAliveInterval 60
+        ServerAliveCountMax 5
+
+#
+# Generic stuff: share existing connections to reduce lag when logging into the same host in a second shell
+#
+ControlMaster auto
+ControlPath ~/.ssh/tmp/%h_%p_%r
+
+#
+##
+### RUG HPC v2 hosts in *.hpc.rug.nl domain with DNS.
+##
+#
+Host *peregrine pg-interactive !*.hpc.rug.nl
+        HostName %h.hpc.rug.nl
+        User prefix-youraccount
+
+#
+##
+### UMCG Research IT HPC v2 hosts in *.hpc.rug.nl domain.
+##
+#
+#  A. With DNS entry.
+#
+Host foyer lobby *calculon *cher-ami !*.hpc.rug.nl
+        HostName %h.hpc.rug.nl
+        User prefix-youraccount
+
+#
+##
+### GCC HPC v2 hosts in *.gcc.rug.nl domain.
+##
+#
+Host *flexo *bender *gattaca* !*.gcc.rug.nl
+        HostName %h.gcc.rug.nl
+        User prefix-youraccount
+
+#
+##
+### Proxy settings for multi-hop SSH.
+##
+# 
+# The syntax in all the ProxyCommand rules below assumes your private key is in the default location.
+# The default location is:
+#  ~/.ssh/id_rsa for keys generated with the RSA algorithm.
+#  ~/.ssh/id_dsa for keys generated with the DSA algorithm.
+# In case your private key file is NOT in the default location you must:
+#  1. Specify the path to your private key file on the command line when logging in with SSH.
+#     For example:
+#         $> ssh -i ~/.ssh/some_other_key.file prefix-youraccount@proxy_server+destination_server
+#  2. Add the path to your private key file in the ProxyCommand rules below.
+#     For example:
+#         Host proxy_server+*
+#             PasswordAuthentication No
+#             ProxyCommand ssh -X -q -i ~/.ssh/some_other_key.file prefix-youraccount@$(echo %h | sed 's/+[^+]*$//').some.sub.domain -W $(echo %h | sed 's/^[^+]*+//'):%p
+#
+
+#
+# Universal proxy settings for triple-hop SSH.
+#
+Host *+*+*
+        ProxyCommand ssh -X -q $(echo %h | sed 's/+[^+]*$//') -W $(echo %h | sed 's/^[^+]*+[^+]*+//'):%p
+
+#
+# Double-hop proxy settings for HPC V2 & V3 environment servers in *.hpc.rug.nl or *.umcg.nl domain.
+#
+Host lobby+* foyer+*
+        PasswordAuthentication No
+        ProxyCommand ssh -X -q prefix-youraccount@$(echo %h | sed 's/+[^+]*$//').hpc.rug.nl -W $(echo %h | sed 's/^[^+]*+//'):%p
+Host passage+* gate+*
+        PasswordAuthentication No
+        ProxyCommand ssh -X -q prefix-youraccount@$(echo %h | sed 's/+[^+]*$//').umcg.nl -W $(echo %h | sed 's/^[^+]*+//'):%p
+#
+# Sometimes port 22 for the SSH protocol is blocked by firewalls; in that case you can try to use SSH on port 80 as fall-back.
+# Do not use port 80 by default for SSH as it officially assigned to HTTP traffic and some firewalls will cause problems when trying to route SSH over port 80.
+#
+Host lobby80+* foyer80+*
+        PasswordAuthentication No
+        ProxyCommand ssh -X -q prefix-youraccount@$(echo %h | sed 's/+[^+]*$//').hpc.rug.nl -W $(echo %h | sed 's/^[^+]*+//'):%p -p 80
+
+}}}
+   Replace all occurences of '''prefix-youraccount''' with:[[BR]]
+   '''prefix''' = based on your organization. Usually either '''umcg''' or '''lifelines''' [[BR]] 
+   '''youraccount''' = your account on calculon.hpc.rug.nl = your account on umcg.hpc.rug.nl = etc.[[BR]][[BR]]
+   If you are **not** on a Mac or on a very old one you may have to comment the ''# Generic stuff: only for MacOS clients'' section at the top of example **''~/.ssh/config''**[[BR]][[BR]]
+   * Make sure you are the only one who can access your ~/.ssh folder. Type the following command in a terminal:
+{{{
+chmod -R go-rwx ~/.ssh
+}}}
+ * You can now for example connect to ''calculon.hpc.rug.nl'' with the account as specified by ''User'' via for example proxy server ''lobby.hpc.rug.nl'' using the alias lobby+calculon. Type the following command in a terminal:
+{{{
+ssh lobby+calculon
+}}}
+   In order to override the accountname specified in your ''~/.ssh/config'' you can use:
+{{{
+ssh prefix-youraccount@lobby+calculon
+}}}
+   You can also transfer data with scp (secure copy) to copy files to your home dir on the cluster like this:
+{{{
+scp some_file lobby+calculon:
+}}}
+   or even better use rsync to copy data to your home dir on the cluster like this:
+{{{
+rsync -av some_directory lobby+calculon:
+}}}
+   Note the colon at the end of the ''scp'' and ''rsync'' commands. Without the colon you would copy to a local file named ''lobby+calculon''.
+   You can add additional ''proxy+*'' aliases for other User Interface (UI) servers and other proxy servers depending on your computing preferences or authorization. If you also added the {{{Host *+*+*}}} code block you can do tripple hops starting with a proxy like this:
+{{{
+ssh proxy+intermediate_server+destination_server
+}}}
+   In case you are on a network where the default port for SSH (22) is blocked by a firewall you can try to setup SSH over port 80 using an alias like this:
+{{{
+ssh lobby80+calculon
+}}}
+
+Frequent Asked Questions:
+ * Q: Why do I get the error {{{muxserver_listen bind(): No such file or directory.}}}? [[BR]]
+   A: You may have failed to create the **''~/.ssh/tmp''** folder in the first step or the permissions on this folder are wrong. [[BR]][[BR]]
+ * Q: Why do I get the error {{{ControlPath too long}}}? [[BR]]
+   A: The {{{ControlPath ~/.ssh/tmp/%h_%p_%r}}} line in your ''~/.ssh/config'' file expands to a path that is too long. Change the ''!ControlPath'' line in your ''~/.ssh/config'' file to create a shorter path for the automagically created sockets. You can use for example {{{ControlPath /tmp/%h_%p_%r}}} to create sockets in /tmp. [[BR]][[BR]]
+ * Q: Why do I get the error {{{nc: getaddrinfo: Name or service not known. ssh_exchange_identification: Connection closed by remote host}}}? [[BR]]
+   A: This server does not exist (anymore); You have a typo in the name of the server you are trying to connect to. Check both the command you typed as well as your ''~/.ssh/config'' for typos in server names. [[BR]][[BR]]
+ * Q: Why do I get the error {{{Permission denied (publickey).}}}? [[BR]]
+   A: This error can be caused by various configuration issues: [[BR]]
+         Either you are using the wrong account name [[BR]]
+         or you are using the wrong private key file [[BR]]
+         or the permissions on your home **''~/.ssh/''** dir and/or on its content are wrong [[BR]]
+         or your account is misconfigured on our account server. [[BR]]
+      Check your account name, private key and permissions. If that did not resolve the issue, then increase the verbosity to debug connection problems (see below). [[BR]][[BR]]
+ * Q: Can I increase the verbosity to debug connection problems? [[BR]]
+   A: Yes try adding ''-vvv'' like this
+      {{{ssh -vvv prefix-youraccount@lobby+calculon}}} [[BR]][[BR]]
+   If that does not help to figure out what is wrong please [wiki:Contact send an email to the HPC helpdesk] and include: [[BR]]
+    1 The command you used for your failed login attempt [[BR]]
+    2 The output of that failed login attempt with ''-vvv'' debugging enabled [[BR]]
+    3 A copy of your ~/.ssh/config file. [[BR]]
+   (Never ever send us your private key; It does not help to debug your connection problems, but will render the key useless as it is no longer private.)
+
+== Users on Windows ==
+
+ * On the machine from which you want to connect
+   1. Make sure you have generated SSH keys linked to your account. See [wiki:RequestAccount RequestAccount] for instructions.
+   1. Download and install **''[http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html Putty]''**. Make sure to also download the key manager **''Pageant''**.
+   1. Start **''Pageant''**
+   1. Load your private key into **''Pageant''**
+   1. Start **''Putty''**
+   1. Go to **''Connection''** -> **''Ssh''** -> **''Auth''** and select **''Allow agent forwarding''**
+   1. Go to **''Connection''** -> **''Data''** and fill in your user name in the **auto-login username''** option
+
+ * You can now connect to for example ''calculon.hpc.rug.nl'' via for example proxy server ''lobby.hpc.rug.nl'' using a double hop like this
+   * In **''Putty''** connect to **''lobby.hpc.rug.nl''** (or choose another proxy from the list)
+   * Once the connection is established type the following command in a terminal:
+{{{
+ssh prefix-youraccount@calculon.hpc.rug.nl
+}}}
+   Change the name of the destination host to connect to other User Interface (UI) servers.
+On order to automate this double hop on Windows please have a look at these instructions: http://mikelococo.com/2008/01/multihop-ssh/