How to setup a remote computer/code#

The computational resources required for a calculation will often surpass those available on a local machine. In such cases, a user may want to leverage the computing power of a supercomputing cluster. The following guide will show you how to set up the AiiDAlab environment to run calculations on such remote machines.

Note

You may notice on this page dropdown sections pertaining to “CSCS”

These collapsible sections provide instructions specific to machines housed at the Swiss Supercomputer Centre (CSCS), such as Daint and Eiger. However, if you do not have an account with CSCS, you may still find these instructions useful as an example for your machine.

Please expand Step 3: Choose computational resources.

Under the Codes section, you can set up executables for the three relevant Quantum ESPRESSO codes used in the app:

pw.x - used in planewave (PW) self-consistent calculations
dos.x - used to compute the density-of-states (DOS)
projwfc.x - used to compute projected density-of-states (PDOS)

Note

Code selection for density-of-states (DOS/PDOS) will be disabled unless Projected density of states is selected as a property of instance in step 2.

Codes are represented as code-label@computer-label, where a computer refers to a collection of configurations relating to the remote machine. So we first need to define a computer (on the remote machine), then a code (an executable) on the remote machine attached to that computer.

To begin, click on Setup new code for pw.x. This will open the setup widget explained in the following section.

Setup Widget#

The Quantum ESPRESSO app includes a built-in widget meant to assist with setting up new codes/computers. We will show you how to use it here to set up the AiiDAlab environment with executable codes located on a remote machine.

Important

If you’re using a CSCS machine, please check out the dropdown below. Otherwise, click on the Detailed Setup tab and proceed to the next section.

Connection#

We need to set up a connection to the remote host. In Set up password-less SSH connection, you can set up an SSH connection including a proxy jump and authentication options specific to your supercomputing center of choice.

Important

You will only need to do this once per host, i.e. computers set up on this host in the following section will all have access to the connection defined here.

Note

If you are unsure of the details of the connection, you may consider reaching out to your institution’s IT department or supercomputing IT service desk.

Once you’ve entered all the necessary details, click Setup ssh.

Tip

You can test your connection by entering your password in the passwordless enabling log box above and clicking Continue. After a few second, you’ll be asked to reenter your password. Please do so and press Continue again. If you see the message 30s timeout, please click Setup ssh once more and retry your password.

Computer#

Codes are attached to a computer, which is a set of configurations associated with a remote host machine. You can provide these configurations in Set up a computer in AiiDA.

Transport & Scheduler#

For Transport and Scheduler, the core. prefix is an AiiDA artifact.

The core.local transport type is set when using a local machine. As our task here pertains to a remote machine, we will be using core.ssh.

Similarly, the core.direct scheduler is used for local machines. Please choose the relevant scheduler for your supercomputing center.

Prepend/Append Text#

AiiDA will construct the job execution command for you. However, your scheduler will need additional information regarding the job. You may add these as Prepend text. AiiDA will add it before the execution command.

For example, you may choose to provide the following to a Slurm scheduler:

#SBATCH --partition=...
#SBATCH --constraint=...
#SBATCH --cpus-per-task=...

export OMP_NUM_THREADS=...
source ...
ulimit -s unlimited

<execution command>

You may also add commands to be run post-submission as Append text. AiiDA will add it after the execution command.

Tip

The computer’s appended text is not meant to be used for calculation-specific post-processing, etc., as codes come with their own appended text option (see next section). You may consider using the computer’s appended text for machine-related tasks, such as clean up, file management, etc.

Once you’ve entered all the necessary details, click Setup computer to complete the setup. To test the new computer, click Test computer and wait about a minute. If all is well, you should see something similar to the following:

Report: Testing computer<daint-mc> for user<aiida@localhost>...
* Opening connection... [OK]
* Checking for spurious output... [OK]
* Getting number of jobs from scheduler... [OK]: 2255 jobs found in the queue
* Determining remote user name... [OK]: xingwang
* Creating and deleting temporary file... [OK]
Success: all 5 tests succeeded

Code#

Once a computer is defined, we may proceed to set up our codes in Set up a code in AiiDA.

For each of our codes (pw, dos, projwfc), you need to:

Provide a label (used in the AiiDA database)
Select the computer (defined in the previous section)
Select a code plugin (e.g. quantumespresso.pw for pw.x)
(optional) Provide a description
Set the absolute path to the executable on the remote machine

You may choose to prepend/append commands to the execution command that are code-specific. You may do so here. AiiDA will add it around the execution command, between the prepend/append commands that may have been added in the previous section. The final submission script would have the following form:

<computer-prepend>
<code-prepend>
<execution command>
<code-append>
<computer-append>

Tip

As mentioned in the previous section, computer prepend/append text are machine-specific commands required for all jobs, while code prepend/append texts are calculation-specific. Use these texts, for example, to load necessary modules prior to execution, and to perform post-processing post-execution.

Once you are finished, click Setup code to complete the setup.

Important

AiiDA can only make use of executables if they exist at their respective provided absolute paths. Please make sure that the codes are pre-compiled on the remote machine and are present before using them.

Once a code is created, you can close the setup widget by clicking on Setup new code. The newly created code is now available in its respective dropdown menu.

Repeat the above process for each of the codes required for your calculation. You can skip the connection setup, as one has already been established. Similarly, you don’t need to set up a new computer. But do make sure to select the same computer for each of the codes.