Wiki source code of 4.3 File Processing - Green Room
Last modified by publicadmin on 2025/12/16 13:04
Hide last authors
| author | version | line-number | content |
|---|---|---|---|
| |
1.1 | 1 | {{toc start="3"/}} |
| 2 | |||
| 3 | ---- | ||
| 4 | |||
| 5 | === Overview === | ||
| 6 | |||
| 7 | The Green Room is a restricted storage zone where all files enter a VRE Project. Access is highly restricted because research files landing here may contain sensitive personal information that could identify an individual research data subject, such as their first and last name, address, or facial features on MR scans. | ||
| 8 | |||
| 9 | === Access === | ||
| 10 | |||
| 11 | Files uploaded by a project member to the Green Room can only be viewed by that project member and by the Project Administrators (PIs of the project). | ||
| 12 | |||
| 13 | Files can also be accessed by VRE Administrators who are bound by confidentiality agreements and trained on the technical and organizational measures implemented by the Charité. This access is needed for them to help VRE users to set up their requested custom configurations and to troubleshoot technical issues. | ||
| 14 | |||
| 15 | === File Upload Methods === | ||
| 16 | |||
| 17 | Data can be added to a VRE Project using one of several different methods: | ||
| 18 | |||
| 19 | * Files can be [[uploaded>>doc:Main.user_guide.Managing Data.File Upload.WebHome]] by project members using the [[File Explorer>>doc:Main.user_guide.Navigating the Project Canvas.File Explorer.WebHome]] or the [[Command Line Tool>>doc:Main.user_guide.Managing Data.Command Line Tool.WebHome]]. | ||
| 20 | * Files can be transferred into the Project by an approved automated pipeline*. | ||
| 21 | * Data can be ingested from an integrated service on the Charité Health Data Platform such as the [[Phönix PACS System>>doc:Main.user_guide.Integrations.PACS.WebHome]] or [[REDCap>>doc:Main.user_guide.Integrations.REDCap.WebHome]] or from a Charité local file network drive.* | ||
| 22 | |||
| 23 | //*Automated pipelines and ingestion services require the assistance of VRE Administrators.// | ||
| 24 | |||
| 25 | === Pseudonymization === | ||
| 26 | |||
| 27 | As Data Controller, the Project Administrator specifies the methods needed (if any) for pseudonymization of all incoming Project data files to ensure sensitive personal information is removed before the data reaches the VRE Core for broader research collaboration and analysis. For example, incoming files may contain no personal information and no pseudonymization is needed. Alternatively, it may be necessary to perform manual, interactive steps or run a preconfigured pipeline to remove sensitive information from pre-defined headers or fields or images. VRE Administrators can assist Project Administrators in deploying customized automated pseudonymizations or interactive workbench tools to perform pseudonymization according to their Project’s needs. Please contact the VRE Administration team ([[vre-support@charite.de>>mailto:vre-support@charite.de]]) to learn more. | ||
| 28 | |||
| 29 | {{info}} | ||
| 30 | **NOTE:** Remember that research data files from fully anonymous data or non-human experiments may still contain personal information of researchers in data fields or file metadata. All data sources and content should be evaluated as part of the data protection impact assessment and pseudonymized as needed. | ||
| 31 | {{/info}} | ||
| 32 | |||
| 33 | The Table below illustrates different example scenarios for automated or interactive data pseudonymization in the Green Room: | ||
| 34 | |||
| 35 | (% border="1" style="margin-right:auto" %) | ||
| 36 | |=(% style="background-color: rgb(0, 50, 97); color:white" %)Example|=(% style="background-color: rgb(0, 50, 97); color:white" %)Options for Data Processing | ||
| 37 | |**Example 1**: Project does not involve the collection of any personal data|((( | ||
| 38 | 1. Project Administrator manually copies all files/folders to the Core using the VRE Portal. | ||
| 39 | 1. Project Administrator requests setup of automated pipeline to copy all incoming data directly to the VRE Core | ||
| 40 | ))) | ||
| 41 | |**Example 2**: Project data contains identifiable personal information in predictable file fields or headers|((( | ||
| 42 | 1. Project Administrator requests setup of automated pipeline*, triggered on file upload, that replaces identifiable information in the pre-defined fields/headers with pseudonymous values. Output files remain in the Green Room for preview and approval or the pipeline can copy data directly to the Core. | ||
| 43 | 1. Project Administrator requests setup of a Green Room workbench VM and uses interactive workbench tools to inspect and manipulate fields and headers to remove sensitive personal information. The outputs are saved back to the Green Room storage and can be manually copied to the Core. | ||
| 44 | ))) | ||
| 45 | |**Example 3**: Project data contains identifiable information in facial features of MR scans|((( | ||
| 46 | 1. Project Administrator requests setup of a Green Room workbench VM and uses interactive graphical user interface-based software tools (e.g., Freesurfer) to inspect and de-face the images. The outputs are saved back to the Green Room storage and can be manually copied to the Core. | ||
| 47 | ))) | ||
| 48 | |||
| 49 | //*Automated pipelines and ingestion services require the assistance of VRE Administrators.// | ||
| 50 | |||
| 51 | |||
| 52 | |||
| 53 | === Copying Data from Green Room to the VRE Core === | ||
| 54 | |||
| 55 | Project data analysis activities occur in the VRE Core. This is the storage zone where you and other members of your Project team can view and access all the Project files, create Datasets and file collections, and use the workbench tools to perform complex data analysis. But first, data that has been pseudonymized to the extent possible (without removing scientific value) must be copied to the Core to separate it from the Green Room where the highly sensitive information is stored. Only a Project Administrator can carry out this crucial step. | ||
| 56 | |||
| 57 | The VRE offers different possibilities to approve data to be copied to the VRE Core: | ||
| 58 | |||
| 59 | {{toc start="4"/}} | ||
| 60 | |||
| 61 | |||
| 62 | ==== Copy to Core initiated by a Project Administrator ==== | ||
| 63 | |||
| 64 | Project Administrators can manually approve and copy data from a Project’s Green Room to the VRE Core using the following steps: | ||
| 65 | |||
| 66 | 1. Sign into the VRE. From the Project Landing page, locate and click your Project to open it. | ||
| 67 | 1. From the Project Canvas File Explorer, navigate to the Green Room directory. | ||
| 68 | 1. Use the checkboxes to select the file(s) and/or folder(s) that you wish to copy to the VRE Core, then click **Copy to Core**. The files table view now changes to reveal the tag “copied-to-core” beside files that have already been copied to the Core (this does not prevent you from copying the file to the Core again). | ||
| 69 | 1. Click **Copy to Core **again. A popup confirmation modal appears. Review the information and enter the requested random code. The purpose of this code is to add an extra step to help you avoid accidental or unintentional file copy operations. | ||
| 70 | 1. Select a destination folder in the VRE Core where the file should be copied to, then click Confirm. Your files will begin copying to the VRE Core and the [[File Status>>doc:Main.user_guide.Navigating the Project Canvas.File Status.WebHome]] window will open automatically to show the status of your file as it is processed. | ||
| 71 | 1. When finished, use the File Explorer to view the files in the destination folder in the Core. Use the Refresh button to if the files don’t appear right away. | ||
| 72 | 1. A system tag //copied-to-core //will be added to your file automatically once processing is complete - this tag cannot be removed. Files copied to the Core will be visible to all Project Administrators and [[Collaborators>>doc:Main.user_guide.Managing User Accounts.User Roles.WebHome]]. | ||
| 73 | |||
| 74 | ==== ==== | ||
| 75 | |||
| 76 | ==== Copy to Core initiated by a Project Collaborator ==== | ||
| 77 | |||
| 78 | While only Project Administrators can copy data into the Project Core zone, the VRE provides a “**Request to Core”** feature to allow Project Collaborators to submit data copy to Core requests that will be reviewed and approved by a Project Administrator. | ||
| 79 | |||
| 80 | **Submitting a Request to Core (Project Collaborators)** | ||
| 81 | |||
| 82 | 1. Sign into the VRE and navigate to your Project landing page. | ||
| 83 | 1. From the Project Canvas File Explorer, navigate to the folder or files your wish to copy in the Green Room directory. | ||
| 84 | 1. Use the checkboxes to select the file(s) and/or folder(s) that you wish to copy to the VRE Core. When at least one file or folder is selected, the top menu of the File explorer displays the option **Request to Core**. | ||
| 85 | 1. Click **Request to Core**. | ||
| 86 | 1. Select a destination folder in the VRE Core where the file should be copied to. | ||
| 87 | 1. Enter //Request notes, //these will be visible to the Project Administrator to help them evaluate the request. | ||
| 88 | 1. Click **Confirm**. An email containing your request will be sent to the Project Administrator, and they can review the pending request in the VRE Portal. | ||
| 89 | |||
| 90 | **Viewing pending Requests to Core (Project Collaborators)** | ||
| 91 | |||
| 92 | 1. From the left menu bar of the Project Canvas, select the Requests icon. | ||
| 93 | 1. In the Copy Data to Core Requests panel, click //New// to view your new requests that have not been fulfilled, or //Completed// to view your requests that have been fulfilled. | ||
| 94 | 1. Requests are identified by the username and the date/time stamp when the request was submitted. | ||
| 95 | |||
| 96 | **Approving or Denying Requests to Core (Project Administrators)** | ||
| 97 | |||
| 98 | {{info}} | ||
| 99 | **NOTE: **As Project Administrator, it is your responsibility to only approve copy of files and folders from the Green Room to Core that you are certain contain no sensitive information. If in doubt, please check with the requester. | ||
| 100 | {{/info}} | ||
| 101 | |||
| 102 | 1. Sign in to the VRE. From the Project Landing page, locate your project and click to open it. | ||
| 103 | 1. From the left menu bar of the Project Canvas, select the Requests icon. Click New to see the list of pending requests. Individual requests are identified by the username and the date/time stamp when the request was submitted. | ||
| 104 | 1. Click on a request. The header displays the information about the request and the files table displays the files and folders included in the request. | ||
| 105 | 1. From the Files table, use the checkboxes to select the file(s) that you approve to copy to the Core. If the request contains folders, you have the option to select an entire folder or individual files / subfolders inside. | ||
| 106 | 1. When at least one file or folder is selected, the top menu of the Files table displays the options “Approve Selected” or “Deny Selected”. Choose an action for the selected files/folders. “Approve Selected” applies to files/folders that you approve to be copied to the Core; “Deny Selected” applies to selections that you don’t approve, or you are uncertain of and need more information about. | ||
| 107 | 1. After selecting an action (//Approve Selected //or //Deny Selected//), a popup confirmation modal appears. Review the information and enter the requested random code. The purpose of this code is to add an extra step to help you avoid accidental or unintentional file copy operations. | ||
| 108 | 1. Depending on the action you chose, click Approve or Deny to confirm and close the modal. | ||
| 109 | 1. In the Files table, status is indicated by different colours. Approved files will appear in green with a checkmark and Denied files in red, with an X. All files in the request must be Approved or Denied before the request can be closed. Folders do not display a status - click the folder name to view the status of the files inside. | ||
| 110 | |||
| 111 | **Closing a Request to Core (Project Administrators)** | ||
| 112 | |||
| 113 | 1. After completing Approve/Deny actions for all the files/folders in the request, it’s time to close the request and notify the user. Click the button **Close Request & Notify User**. | ||
| 114 | 1. If you //have not// completed actions for all the files/folders in the request, a popup will appear reminding you to complete the remaining requests. | ||
| 115 | 1. If you //have// completed actions for all the files/folders in the request, a popup confirmation modal appears. Confirm that you wish to complete this request, and enter optional //Review notes// that will be seen by the requester. | ||
| 116 | 1. Click Confirm. An email notification for your approval will be sent to the requester, and they can review the completed request in the Project Canvas Requests panel. | ||
| 117 | |||
| 118 | ==== ==== | ||
| 119 | |||
| 120 | ==== Custom automated copy-to-core pipeline ==== | ||
| 121 | |||
| 122 | For large bulk operations where the pseudonymization methods are predictable and straightforward or not required at all, VRE Project Administrators can request assistance to set up an automated pipeline to copy their Project data from the Green Room to Core without manual intervention. Please contact the VRE Administration at [[vre-support@charite.de>>mailto:vre-support@charite.de]] team to learn more. | ||
| 123 | |||
| 124 | ---- | ||
| 125 | |||
| 126 | **See Also:** | ||
| 127 | |||
| 128 | [[Platform Architecture>>doc:Main.user_guide.About the VRE.Introduction]] | ||
| 129 | |||
| 130 | [[Definitions>>doc:Main.user_guide.Managing Data.Data Lineage.WebHome]] | ||
| 131 | |||
| 132 | [[File Status>>doc:Main.user_guide.Navigating the Project Canvas.File Status.WebHome]] | ||
| 133 | |||
| 134 | [[User Roles>>doc:Main.user_guide.Managing User Accounts.User Roles.WebHome]] | ||
| 135 | |||
| 136 | ---- | ||
| 137 | |||
| 138 | {{box}} | ||
| 139 | (% style="text-align:center" %) | ||
| 140 | image:https://i.creativecommons.org/l/by-sa/4.0/88x31.png | ||
| 141 | |||
| 142 | (% style="text-align: center;" %) | ||
| 143 | Copyright © 2022, Indoc Research. This work is licensed under a [[Creative Commons Attribution-ShareAlike 4.0>>http://creativecommons.org/licenses/by-sa/4.0/ ||rel="noopener noreferrer" target="blank"]] International License. | ||
| 144 | {{/box}} |