How to access Project data in Guacamole VM using VRE CLT

VRE users are able to access to own projects' data in VRE workbench tool for further analysis, and push the outputs back to VRE projects by using VRE Command Line Tool. 

1. Access to Command Line Tool within Guacamole VM

Please see VRE User Guide/Analyzing Data/Gucamole for additional details regarding: 

• What is the Guacamole VM workbench tool?
• How can I request access to my Project's Guacamole VM workbench tool?
• How can I access the Guacamole VM and launch terminal?

To decide which VM you wish to enter, please be advised:

Within Guacamole VM, users can then launch VRE Command Line Tool to access to Project data for further analysis. The VRE Command Line Tool (CLI) is a binary executable program that provides a convenient way for users to perform file actions and platform-related tasks programmatically. 

By typing vrecli in Guacamole VM terminal, users can launch the latest version of Command Line Tool:

2. User login through Command Line Tool

The commands users can input are categorized into groups of File, Project, and Users as shown in the above screenshot.

Users need to login to the VRE by their Charite AD credentials before performing any tasks through Command Line Tool, by typing vrecli user login, users are asked to provide user name and password:

3. Download Project data through Command Line Tool

a. The commands supporting the file related tasks are grouped within file , users can print help screen to preview which file related commands they are input by typing vrecli file --help:

b. From the above screenshot, the command users can input to download Project data is file sync, let's take a look what options users need to provide in order to download a specific Project file/folder by typing vrecli file sync --help :

As part of the file downloading command, users are required to provide

• the target file/folder's path within the project, and
• the local path you want to save the downloaded file/folder

c. Example: a file of TEST_csv.csv can be downloaded from Indoc Test Project Core zone within the folder uname to user's Home Directory by typing vrecli file sync indoctestproject/uname/TEST_csv.csv . -z core 

d. TEST_csv.csv is successfully downloaded to the Home Directory:

e. Trouble shooting. If downloading failed, please ensure:

• Whether your session expired for being inactive for an hour.
• Whether the target file/folder exists under correct path you provided from the portal.
• Whether you have access to the target file/folder:
  • you need to be a member of the Project where your target file/folder is within, and
  • your Project role is Project Administrator or Project Collaborator, who have the access to all files/folders within the Project Core zone. 

4. Upload Project data through Command Line Tool

After grabbing Project data, users are then able to run containerized pipelines in Workbench Tools to analyze the data. The generated outputs can be uploaded back into the Project by the Command Line Tool. 

a. Let's take a look what options users need to provide in order to upload a file or folder by typing vrecli file upload --help :

As part of the file uploading command, users are required to provide the options of:

• the target file/folder's path on your local, and
• the Project directory you want to upload the file/folder to
• the Project zone. By default, data should be uploaded to Green Room for pseudonymisation. However, data can also be uploaded directly into Core zone if the data is output of source data from Core zone, if this's the case, you need to confirm on the VRE Terms of Use to proceed the uploading. 
• the upload message if upload to Core zone. Users are required to provide a message to explain why data is uploaded directly to the Core zone. 

b.  Example: a file of TEST_csv.csv can be uploaded from user's Home Directory into a  folder of uname  within Project of Indoc Test Project Core zone by typing vrecli file upload ./TEST_csv.csv -p indoctestproject/uname -z core -m "workbench output" 

 c. Example 02:  a file of TEST_csv.csv can be uploaded from user's Home Directory into a  folder of uname  within Project of Indoc Test Project Core zone, and build a data lineage with a source file testing_data.png in  a  folder of uname  within same Project Core zone and with a pipeline name of testing_pipeline  by typing vrecli file upload ./TEST_csv.csv -p indoctestproject/uname -z core -m "testing" --source-file uname/testing_data.png  --pipeline "test_pipeline"   

The Data Lineage Graph in Properties of file TEST_csv.csv or testing_data.png will display the lineage between testing_data.png as a Upstream file, testing_pipeline as a Process, and TEST_csv.csv as a Downstream file linked within the lineage graph: