Bulk data tranfsfer#
To make massive data transfers, there are two transfer nodes that allow large amounts of data to be copied and downloaded to the user space by directly accessing the data backbone network.
We have two nodes enabled for this purpose that provide greater bandwidth between the nodes themselves and the shared storage system. These nodes are accessible on the addresses:
| Name | IP Address |
|---|---|
| tn-t01.hpc.iter.es | 193.146.150.177 |
| tn-t02.hpc.iter.es | 193.146.150.178 |
The transfer of information through these nodes is through the use of secure protocols over SSH such as SFTP (SecureSHell File Transfer Protocol or Secure File Transfer Protocol). SFTP allows you to easily download and upload files, while providing us with confidentiality of the transmitted data. In contrast, on an FTP server we do not have any type of security, since user credentials and all data traffic are sent unencrypted. For this, it is necessary to have a client for this protocol on the local machine, such as sftp on linux or psftp on windows .
Remember to disconnect from the VPN when you want to transfer massive data. Otherwise, your computer will route all traffic through the VPN with a lower performance.
sFTP for Linux users#
You can open the connection to the server through IP public:
Most used commands#
Once the session is open, the help command will display the list of commands that can be used. The most used ones are described below and they are valid for Linux users such as Windows:
sftp> help -- Muestra la ayuda.
sftp> cd dir -- Cambia el directorio de trabajo remoto.
sftp> lcd dir -- Cambia el directorio de trabajo local.
sftp> pwd -- Muestra el directorio de trabajo actual.
sftp> lpwd -- Muestra el directorio de trabajo actual local.
sftp> put file1.zip ... -- Sube un fichero desde el directorio de trabajo local al directorio de trabajo remoto
sftp> get file1.zip ... -- Descarga un fichero desde el directorio de trabajo remoto al directorio de trabajo local
sftp> put -r directory -- Sube un directorio desde el directorio de trabajo local al directorio de trabajo remoto
sftp> get -r directory -- Descarga un directorio desde el directorio de trabajo remoto al directorio de trabajo local
sFTP for Windows users#
For Windows users there are many applications both with the command line and with a graphical interface. Among them the best known are: psftp, Filezilla y WinSPC.
psftp#
PSFTP is the SFTP command line client that is installed with the PuTTY (a very popular SSH client for windows).
There are three ways to run PSFTP:
- Click on the Windows Start button and go to All Programs. From the list of program, Click on PuTTY and then PSFTP.
- To enter the following path in the address bar C:\Program Files (x86)\PuTTY and then double-click on psftp.exe.
- From Windows command-line interpreter Command Prompt or PowerShell.
However, to open it from the command prompt, It is recommended to include the directory in which PSFTP has been installed in the user's PATH here
You can open the connection to the server through its public IP.
Filezilla#
FileZilla is one of the most used programs to be used as an FTP / FTPS and FTPES client, but it also incorporates the possibility of connecting to an SFTP server. You just have to enter the following syntax Server: sftp: // IP in the address bar and the username and password that have been provied in its corresponding place.
WinSCP#
WinSPC is a popular free download client available for Windows. It supports the SFTP, SCP, WebDAV and FTP protocols and is mainly focused on the transfer of files, the use of scripts and basic functionalities of a file manager.
Otros#
sFTP for MAC users#
The MAC user can use the terminal in the same way as Linux users or install an application with a graphical interface like the following: