Skip to Content
Last modified by publicadmin on 2025/12/16 13:04
Show last authors
1 VRE users are able to access to own projects' data in VRE workbench tool of Jupyterhub for further analysis, and push the outputs back to VRE projects by using VRE Command Line Tool.
2
3 === 1. Access to Jupyterhub Terminal ===
4
5 Jupyterhub is integrated with VRE user account with Single Sign-On, so users can launch JupyterHub from the VRE Portal within the Project canvas.
6
7 Please visit this page: [[VRE User Guide/Analyzing Data/J>>url:https://vre.charite.de/xwiki/wiki/adminsubwiki/view/Main/user_guide/Analyzing%20Data/Guacamole/]][[upyterhub>>doc:Main.user_guide.Analyzing Data.JupyterHub.WebHome]] for details on
8
9 * What is Jupyterhub workbench tool?
10 * How to enter Jupyterhub and launch Terminal?
11 * How to install Python packages, create a Python Virtual Environment and register a Kernel?
12
13 === 2. Access to Command Line Tool within Jupyterhub Terminal ===
14
15 Within Jupyterhub Terminal, users can then launch VRE Command Line Tool to access to Project data for further analysis. The VRE Command Line Tool (CLI) is a binary executable program that provides a convenient way for users to perform file actions and platform-related tasks programmatically. For more technical details, please find: [[Developer Guide/Command Line Tool>>doc:Main.user_guide.Managing Data.Command Line Tool.WebHome]]
16
17 By typing {{code language="none"}}vrecli{{/code}} in Jupyterhub Terminal, users can launch the latest version of Command Line Tool:
18
19 {{code language="none"}}
20 swang@jupyter-swang:~$ vrecli
21 Usage: vrecli [OPTIONS] COMMAND [ARGS]...
22
23 What's new (Version 1.4.0):
24
25 1. Support file annotation when uploading folder
26
27 2. Support PACS data integration with VRE
28
29
30
31 Options:
32 --help Show this message and exit.
33
34 Commands:
35 file File Actions
36 project Project Actions
37 user User Actions
38 {{/code}}
39
40 === 3. User login through Command Line Tool ===
41
42 The commands users can input are categorized into groups of File, Project, and Users as shown in the above screenshot.
43
44 Users need to login to the VRE by their Charite AD credentials before performing any tasks through Command Line Tool, by typing {{code language="none"}}vrecli user login{{/code}}, users are asked to provide user name and password:
45
46 {{code language="none"}}
47 swang@jupyter-swang:~$ vrecli user login
48 Username: swang
49 Password:
50 Welcome to the VRE Command Line Tool!
51 {{/code}}
52
53 === 4. Download Project data through Command Line Tool ===
54
55 a. The commands supporting the file related tasks are grouped within {{code language="none"}}file{{/code}} , users can print help screen to preview which file related commands they are input by typing {{code language="none"}}vrecli file --help{{/code}}:
56
57 {{code language="none"}}
58 swang@jupyter-swang:~$ vrecli file
59 Usage: vrecli file [OPTIONS] COMMAND [ARGS]...
60
61 Options:
62 --help Show this message and exit.
63
64 Commands:
65 attribute-export export attribute template from a given project
66 attribute-list check attributes of a given project
67 list List files and folders inside a given project/folder
68 sync Download file from a given project/folder/file
69 upload upload file to a given project file upload <file-
70 path>...
71 {{/code}}
72
73 b. From the above file related commands, the one users can input to download Project data is {{code language="none"}}file sync{{/code}}, let's take a look at what options users need to provide in order to download a specific Project file/folder by typing {{code language="none"}}vrecli file sync --help{{/code}} :
74
75 {{code language="none"}}
76 swang@jupyter-swang:~$ vrecli file sync --help
77 Usage: vrecli file sync [OPTIONS] [PATHS]... OUTPUT_PATH
78
79 Download file from a given project/folder/file
80
81 Options:
82 -z, --zone TEXT Target Zone, core/greenroom [default: greenroom]
83 --help Show this message and exit.
84 {{/code}}
85
86 As part of the file downloading command, users are required to provide
87
88 * the target file/folder's path within the project, and
89 * the local path you want to save the downloaded file/folder
90 * the Project zone (Green Room or Core) where contains the target file/folder
91
92 c.** Example**: a file of {{code language="none"}}TEST_csv.csv{{/code}} can be downloaded from {{code language="none"}}Indoc Test Project{{/code}} Core zone within the folder {{code language="none"}}swang{{/code}} to user's Home Directory by typing {{code language="none"}}vrecli file sync indoctestproject/swang/TEST_csv.csv . -z core{{/code}}
93
94 {{code language="none"}}
95 swang@jupyter-swang:~$ vrecli file sync indoctestproject/swang/Test_csv.csv . -z core
96 Preparing status: READY_FOR_DOWNLOADING
97 start downloading...
98 Downloading TEST_csv.csv |██████████████████████████████ 100% 00:00
99 File has been downloaded successfully and saved to: ./TEST_csv.csv
100 {{/code}}
101
102 d. {{code language="none"}}TEST_csv.csv{{/code}} is successfully downloaded to the Home Directory:
103
104 {{code language="none"}}
105 swang@jupyter-swang:~$ ls
106 'List Core data.ipynb' test test_folder.zip Untitled2.ipynb
107 'Mental Health Survey Analysis .ipynb' TEST_csv.csv Untitled1.ipynb Untitled.ipynb
108 {{/code}}
109
110 e. Trouble shooting. If downloading failed, please ensure:
111
112 * Whether your session expired for being inactive for an hour.
113 * Whether the target file/folder exists under correct path from the portal.
114 * Whether you have access to the target file/folder:
115 ** you need to be a member of the Project where your target file/folder is within, and
116 ** your Project role is Project Administrator or Project Collaborator if you are trying to download files/folders from the Project Core zone.
117
118 === 4. Upload Project data through Command Line Tool ===
119
120 After grabbing Project data, users are then able to run containerized pipelines in Workbench Tools to analyze the data. The generated outputs can be uploaded back into the Project by the Command Line Tool.
121
122 a. Let's take a look what options users need to provide in order to upload a file or folder by typing {{code language="none"}}vrecli file upload --help{{/code}} :
123
124 {{code language="none"}}
125 swang@jupyter-swang:~$ vrecli file upload --help
126 Usage: vrecli file upload [OPTIONS] [PATHS]...
127
128 upload file to a given project
129
130 file upload <file-path> -g <generate-id> -a <attribute> -p <project-code/folder> -t <tag>
131 -z <zone> -m <upload-message> -s <folder/source-file> --pipeline <pipeline> --zip --pacs
132
133 Options:
134 -p, --project-code TEXT Project Code
135 -g, --generate-id TEXT Generate ID (Only for project Generate)
136 -a, --attribute TEXT File Attribute Template
137 -t, --tag TEXT Add tag to the uploaded file
138 -z, --zone TEXT Target Zone, core/greenroom [default:
139 greenroom]
140
141 -m, --upload-message TEXT When upload processed files, must commit a
142 message to explain the purpose.
143
144 -s, --source-file TEXT The name of the source file of your processed
145 files.
146
147 --pipeline TEXT The processed pipeline of your processed files.
148 optional for source.
149
150 --zip Upload folder as a compressed zip file
151 --pacs Identify uploading data as pacs
152 --help Show this message and exit.
153 {{/code}}
154
155 As part of the file uploading command, users are required to provide the options of:
156
157 * the target file/folder's path on your local, and
158 * the Project directory you want to upload the file/folder to
159 * the Project zone. By default, data should be uploaded to Green Room for pseudonymisation. However, data can also be uploaded directly into Core zone if the data is output of source data from Core zone, if this's the case, you need to confirm on the VRE Terms of Use to proceed the uploading.
160 * the upload message if upload to Core zone. Users are required to provide a message to explain why data is uploaded directly to the Core zone.
161
162 b. ** Example**: a file of {{code language="none"}}TEST_csv.csv{{/code}} can be uploaded from user's Home Directory into a folder of {{code language="none"}}swang{{/code}}  within Project of {{code language="none"}}Indoc Test Project{{/code}} Core zone by typing {{code language="none"}}vrecli file upload ./TEST_csv.csv -p indoctestproject/swang -z core -m "workbench output"{{/code}}
163
164 {{code language="none"}}
165 swang@jupyter-swang:~$ vrecli file upload ./TEST_csv.csv -p indoctestproject/swang -z core -m "workbench output"
166 You are about to transfer data directly to the VRE Core! In accordance with the VRE Terms of Use, please confirm that you have made your best efforts to pseudonymize or anonymize the data and that you have
167 the legal authority to transfer and make this data available for dissemination and use within the VRE .To review the Terms of Use, please visit https://vre.charite.de .If you need to process the data to remove
168 sensitive identifiers, please cancel this transfer and upload the data to the Green Room to perform these actions.
169 To cancel this transfer, enter [n/No]
170 To confirm and proceed with the data transfer, enter [y/Yes]
171 [y/N]: y
172 Starting upload of: ./TEST_csv.csv
173 Pre-upload complete.
174 uploader: swang container_id: 37 total_size: 303684 total_chunks: 1 uniq_identifier: f45fbb58-c135-4b94-a224-9bcd469a08ee-1631558977 resumable_relative_path swang/
175 Upload job is finalizing, please wait...
176 Upload job complete.
177 All files uploaded successfully.
178
179 {{/code}}
180
181 ----
182
183 {{box}}
184 [[image:https://i.creativecommons.org/l/by-sa/4.0/88x31.png||data-xwiki-image-style-alignment="center"]]
185
186 (% style="text-align: center;" %)
187 Copyright © 2022, Indoc Research. This work is licensed under a [[Creative Commons Attribution-ShareAlike 4.0>>http://creativecommons.org/licenses/by-sa/4.0/ ||rel="noopener noreferrer" target="blank"]] International License.
188 {{/box}}