ASCI: Australian Synchrotron Compute Infrastructure

Welcome to the user’s guide to the Australian Synchrotron’s interactivate computing environment, ASCI.

What is ASCI?

ASCI is a high-performance computing platform to allow researchers who have collected data at the Australian Synchrotron to process and visualise their data.

To begin using ASCI, following the Getting Started link below.

Getting Started

Gaining access to ASCI

If you have performed an experiment on the XFM or IMBL beamlines after June, 2017 you will automatically have access to ASCI. If you don’t fall into this category please contact a Beamline Scientist to request access.

To log in to ASCI:

  1. Open https://asci.synchrotron.org.au/ with an up to date version of Chrome or Firefox.
  2. Log in with your Australian Synchrotron Portal account.

If you have forgotten your password you can reset it at https://portal.synchrotron.org.au/.

Launching a session

Once you have logged you will be presented with a number of Environments to choose from. These are templates for creating your computing desktop and define what software is available inside your desktop.

To launch a session simply click on an Environment according to the type of data you wish to process.

Online vs offline

If you are currently performing an experiment on the XFM or IMBL beamlines you will have access to an “Online” environment. This environment will run on a dedicated compute node so that you won’t be competing for system resources with other users.

_images/launch.gif

Connecting to a session

Once you have created a session it will appear at the top of the page. Click on this session to connect to it.

_images/connect.gif

Sharing a sesson

Sharing your session will allow other ASCI users to connect to it. They will see the same desktop as you and can also control the mouse cursor, launch applications etc.

To share the desktop:

  1. On the ASCI home page click the Edit Session icon in the top right corner of the session summary box.
  2. In the “Invite User” box, enter the name or email address of the person you wish to share the session with.
  3. Click Search.
  4. Click the name of the user and they will be added to the session.
_images/share.gif

Locating your data

Inside your session you will find the data located at:

/data/<beamline>/<epn>

To browse to your data, you can use the File Explorer which can be launched via the Computer shortcut on the desktop, or the Caja icon on the menu bar at the top of the screen. From there, open File System and then the data folder:

_images/data.gif

Managing your data

Downloading

SFTP Clients

You can download your data from the Australian Synchrotron SFTP service.

  1. Install and run a SFTP client such as Filezilla, Cyberduck or WinSCP.

  2. Configure your client with the following settings:

    Host / Server: sftp.synchrotron.org.au
    Port: 22
    Protocol: SFTP
    Username: Your Portal email address
    Password: Your Portal password

  3. When you connect you will be presented with a list of folders for each experiment you have attended.

For example, the Filezilla configuration would look like:

_images/filezilla.jpg

To download files or folders, simply drag them from the right pane to the left pane. You can see the progress of the transfer under “Queued files” at the bottom of the window.

Note: Data produced on ASCI will only be available for download if it has been saved in the experiment folder.

rsync

Rather than downloading your data with a graphical client you can use the command-line tool rsync. This has advantages including:

  • You can run rsync on a folder that is already partially downloaded and it will only copy new or changed files
  • rsync can resume interrupted download
  • rsync downloading can be automated
Installing rsync
macOS / Linux

rsync comes pre-installed with macOS and many Linux distributions (try typing “rsync” into your terminal). If it is not installed, you can get it with:

# Debian / Ubuntu:
sudo apt-get install rsync

# CentOS / Fedora:
sudo yum install rsync
Windows

rsync can be installed with Cygwin.

Using rsync

Run the following command in your terminal:

rsync -rtzP <your_email_address>@sftp.synchrotron.org.au:/data/<epn>/<path_to_folder> .

For example:

rsync -rtzP alice@example.edu@sftp.synchrotron.org.au:/data/1234/frames/angles/scan1 .

The flags that are used here are:

  • -r : copy folders are as well as files
  • -t : copy timestamps on files (makes re-syncing faster)
  • -z : compress files before transferring
  • -P : enable resuming partially downloaded files

For more options available to rsync, run man rsync.

Using SSH keys to avoid typing password to rsync

rsync allows you to download your data without being prompted for a password by using SSH. This is especially useful if you want to set up automated data syncing.

To install keys, first generate them if you haven’t already:

ssh-keygen

Then copy the key with:

ssh-copy-id -i ~/.ssh/id_rsa <your_email_address>@sftp.synchrotron.org.au

From then on you won’t be prompted for a password when you run rsync.

If you are on macOS you will need to install ssh-copy-id by:

  1. Installing homebrew.

  2. Running:

    brew install ssh-copy-id

Uploading

We currently do not support direct uploads to ASCI. However, from within an ASCI session you can launch a browser and connect to file hosting services such as Dropbox or Google Drive. Filezilla is also installed inside the ASCI environments and can be used for downloading data from external file servers.

Support

If you need help with ASCI, would like to report an issue or have requests or suggestions please email ascidev@synchrotron.org.au.