Data Synchronisation

Synchronising data between the dashboard and a Python/Jupyter client

Your projects, snapshots etc. can all be sent up to the dashboard for better analysis. Whether you're running a Python script or a Jupyter notebook, you can easily link your project with your dashboard.

This is good if;

  • You use different machines that don't share storage.

  • You are creating results on a cloud system.

  • You are creating results on multiple systems.

  • You want to see your results centrally on the Etiq Dashboard.

Creating Access Tokens

To do this, you need to log into the dashboard and create an "Access Token". This is a secret key used to submit content to the dashboard;

  1. Enter a useful name for the token in the "Token Name" box. This will help identify it later on when it comes to removing it.

  2. Click "Add"

  3. The token is created. Copy it safely to a file somewhere.

You can't come back to this page to view the full token. So be sure to copy the token somewhere!

Using Access Tokens

Within your Python code, we should call this directly before we open any projects or snapshots;

etiq.login("http://dashboard.example.org", "my-token-text")

We use the pair of "dashboard url" and "token text" to identify where and to whom the projects are sent. That is, the user which created the token will own projects created using this token.

From hereon, all records created will be synchronised with the dashboard until the script finishes or etiq.logout() is called.

Don't keep access tokens in source code!

For security reasons, and to aid in convenience if the token changes, you should either keep the auth key in a file and read that file at runtime;

with open("key.txt") as f:
    etiq.login("https://dashboard.example.org", f.read().strip())

Or as an environment variable;

import os
# These environment variables need to be set yourself outside the script!
etiq.login(os.environ["ETIQ_DASHBOARD_URL"], os.environ["ETIQ_TOKEN"])

If you log in above and you're in an interactive session, you'll be notified that the login was a success;

'Connection successful. Projects and pipelines will be displayed in the dashboard. 😀'

If this token is invalid, a traceback will be raised.

Login Scope

You must login each session to continue persisting, it is only kept in memory.

If you want to stop sharing but have already logged in, calling etiq.logout() will "forget" the token and work will only be available locally.

Accessing Existing Projects

This feature is added in Etiq library 1.6.0

You can also access projects that you've created previously or have been shared with you using the Python API;

# Either you know what it's called:
project = etiq.projects.open("My Existing Project", create_if_missing=False)

# Or you can pick and choose:
all_projects = etiq.projects.get_all_projects()

Note the use of the parameter create_if_missing above. By default if you open a project and it's not there, we will just quietly create it for you. But if you know you want to add to an existing project, then it's best to use this so we can be sure we've got an existing project.

Adding To Existing Shared Projects

When you have an existing project you can add snapshots to that as though it were your own. This way you can collaborate on projects with team members.

Deleting Access Tokens

Simply click the "Delete" button within the token management page. Note that this makes the token invalid and will not work anymore. The etiq.login method will fail if we try to use it from this point onwards.

PreviousProject SharingNextAirflow

Last updated