Cloud Data Delivery

The Imputation Service allows for the delivery of output files from successful jobs to a Google Cloud Storage (GCS) bucket of the user's choice. Output files are available for 14 days after job completion, after which the files will be deleted. Therefore, it's strongly recommended that the user download or deliver their outputs to a GCS bucket for long-term safekeeping.

Users must grant the service write access to their cloud destination. This is done by granting the following entities write access (the Storage Object Creator role is sufficient) to the bucket:

• [e-mail] - this grants the Service access to your bucket
• your Terra proxy group, which looks like PROXY_{user-specific-id}@firecloud.org and can be found on your account page - this allows the Service to confirm that you have access to the bucket you're delivering to (this protects you against others attempting to write data to your bucket)

The service will check that both it and you have write access to the bucket before starting the delivery process. 

Output files will be delivered to a subdirectory at the destination, named using the job ID. For example, if you are delivering the outputs for job ID a8373d2e-cb94-4371-80cd-dcf459ea49bb to destination gs://my-bucket/my-path, each of the files will be written to gs://my-bucket/my-path/a8373d2e-cb94-4371-80cd-dcf459ea49bb/{outputFileName}

Helpful Hints

• The write access described above can be revoked after the delivery is complete. Access is only needed for the duration of the delivery process, which typically takes under one minute.
• Once data is delivered to a GCS destination, it can no longer be downloaded directly from the service. If you wish to download outputs after delivery has been completed, please download them from your delivery destination instead.
• If your delivery location is a Terra workspace bucket, you only need to grant [e-mail] write access to your workspace; you can skip the proxy group step.