Skip to main content

Pipeline and Service Updates

Morgan Aster
Updated

Instructions for citing this service can now be found at How to cite the service.
 

June 16, 2026

Service updates:

  • Each pipeline has a maximum number of quota that can be run in a single job. This value is now displayed in the web portal with the other quota information, after selecting a pipeline.

This change corresponds to version 5.0.5 of the service.

 

June 5, 2026

Service updates:

  • In the web portal, the "Account & Quotas" page has been separated into two pages: "Profile" and "Quotas", which can be accessed using the hamburger menu on the left. 
  • The hamburger menu has been updated with improved navigation, allowing you to more easily access job and account pages.

 

May 25, 2026

Service updates:

  • The web portal now requires users to agree to the Terms of Service when submitting a job.

 

May 22, 2026

CLI updates:

  • When submitting jobs through the CLI via terralab submit, users must now include the --agreeToTerms flag to indicate that they have read and agree to the Terms of Service.

This change corresponds to version 4.0.0 of the terralab cli.

 

May 20, 2026

Service updates:

  • The deprecated api/pipelineruns/v1/prepare endpoint has been removed. Users should be using the v3 endpoint (api/pipelineruns/v3/prepare).
  • Users submitting jobs from the new v3 api/pipelineruns/v3/prepare endpoint must agree to the Terms of Service when submitting a new job. In a future update, the v2 endpoint will be removed.
  • The service returns a clear error message if users attempt to submit data in a Requester Pays bucket, which is not supported.

This update corresponds with version 4.0.2 of the service.

 

May 15, 2026

Service updates:

  • The Terms of Service and Acceptable Use Policy can now be viewed in the web portal. In a future update, users submitting jobs through the web portal will be required to accept these terms before submission.

 

May 08, 2026

Service updates:

  • The v1 result endpoint (/api/pipelineruns/v1/result/{jobId}) is no longer available. Users should use the latest endpoint (/api/pipelineruns/v3/result/{jobId}) instead. The latest version of the CLI and the web portal use the v3 endpoint.

This update corresponds with version 3.0.0 of the service.

 

April 29, 2026

Service updates:

  • The web portal now allows users to deliver the outputs of their successful jobs to a Google Cloud Storage bucket. To deliver outputs, go to the job details view for your job and enter a destination bucket in the Data Delivery section.

Pipeline Updates

  • The All of Us + AnVIL Array Imputation v2 pipeline is now available to run. This version will be the default version run unless v1 is specified. A description of v2 can be found in the Pipeline Versions documentation.

These changes correspond with the ImputationBeagle_v3.0.1 tool version.

CLI updates:

  • Users of the CLI can now deliver the outputs of their jobs to a cloud destination using the terralab deliver command.

This change corresponds to version 3.3.0 of the terralab cli.

 

April 14, 2026

Service updates:

  • Users are now able to deliver the outputs of their job to a bucket in Google Cloud Storage using the /api/pipelineruns/v1/result/{jobId}/output/deliver-to-cloud API endpoint. For more information about using cloud data delivery, see the user documentation.

This change corresponds to version 2.3.14 of the service.

CLI updates:

  • The terralab jobs details command returns pipeline outputs and their file sizes, when available. This change was part of updating the CLI to use the new service API version api/pipelineruns/v3/result/{jobId}, as api/pipelineruns/v2/result/{jobId} is now deprecated.

This change corresponds to version 3.2.0 of the terralab cli.

 

April 2, 2026

Service updates:

  • The api/pipelineruns/v3/result/{jobId} API now returns output file size (when available) in the outputs field in the response.

This change corresponds to version 2.3.4 of the service.

CLI updates:

  • Users can now run terralab account to retrieve information about their account, as well as view cloud resource sharing instructions for using cloud-based inputs with the service.

This change corresponds to version 3.1.5 of the terralab cli.

 

March 20, 2026

Service updates:

  • The v2 getPipelineRunResult endpoint api/pipelineruns/v2/result/{jobId} has been deprecated.

  • In the v3 getPipelineRunResult endpoint api/pipelineruns/v3/result/{jobId}, the structure of the outputs field has changed:

    Previous format:

    {
  "outputName1": "outputValue1",
  "outputName2": "outputValue2"
}

    New format:

    {
  "outputName1": { "value": "outputValue1" },
  "outputName2": { "value": "outputValue2" }
}

These changes correspond to version 2.3.0 of the service.

 

March 6, 2026

Service updates:

  • In the web portal, a new Account & Quotas has been added, which allows you to view your account information and manage pipeline quotas. You can visit this new page by clicking on the hamburger menu in the top left, then clicking on your name, and finally clicking "Account & Quotas".

 

February 27, 2026

CLI updates:

  • terralab submit now accepts either local or Google Cloud Storage hosted input files. 

This change corresponds to version 3.1.0 of the terralab cli.

Service updates:

  • In the web portal, when submitting a new job, you can now choose between uploading an input file from your computer or using a file hosted in Google Cloud Storage. Previously, only locally uploaded files were supported.

 

February 18, 2026

Service updates:

  • The service APIs now support cloud-based user input files (Google Cloud Storage only) as well as local uploads.

This change corresponds to version 2.2.0 of the service.

 

February 11, 2026

Service updates:

  • Users will now receive an email notification when their quota limit has changed.

This change corresponds to version 2.1.11 of the service.

 

February 6, 2026

Pipeline updates:

For the All of Us + AnVIL array imputation pipeline:

  • The All of Us + AnVIL array imputation pipeline now removes trailing zeros from the AF and DR2 annotations in the output VCF. This change, in some cases, can reduce output file size and result in downstream storage cost savings.
     
  • The pipeline now performs a check on the number of variants in the user's input VCF and will fail if there is more than 3 million variants.  This is to help prevent WGS inputs being submitted inadvertently.

These changes correspond with the ImputationBeagle_v2.5.0 and ImputationBeagle_v2.5.1 tool versions.

 

February 5, 2026

Service update:

  • The service no longer allows users to submit jobs if the user's remaining quota is less than the minimum quota for the pipeline.

This change corresponds to version 2.1.2 of the service.

 

January 30, 2026

Service update:

  • In the web portal, the Job Details page now displays the total number of samples that were submitted for the job.

 

January 16, 2026

Service update:

  • In the web portal, users may now click on an individual job in the Job History table to view additional details about the job.

CLI update:

  • terralab download error messages for non-successful runs and runs with expired outputs have changed. This change was part of updating the CLI to use new service API versions api/pipelineruns/v2/result/{jobId} and api/pipelineruns/v2/result/{jobId}/output/signed-urls, as api/pipelineruns/v1/result/{jobId} is now deprecated.

This change corresponds with version 3.0.0 of the terralab cli.

 

January 13, 2026

Service update:

  • The v1 pipeline run result API endpoint (api/pipelineruns/v1/result/{jobId}) has been deprecated. 
  • For v2 results, the job details and output signed URLs have been split out into two separate endpoints: 
    • The result endpoint (api/pipelineruns/v2/result/{jobId}) contains the same information as v1 except only the output filenames rather than signed URLs
    • The output/signed-urls endpoint (api/pipelineruns/v2/result/{jobId}/output/signed-urls) returns the signed URLs needed to download file outputs for the job.

This change corresponds with version 2.1.0 of the service.

 

January 9, 2026

Pipeline updates:

For the All of Us + AnVIL array imputation pipeline:

  • The docker image containing Beagle jar used in Phase and Impute tasks is now generated from a version-controlled GitHub repository.
  • The pipeline now performs additional validation checks on the user's input VCF and returns a relevant error message to the user if these checks fail. The new checks include:
    • detecting contig headers with missing length attribute
    • detecting indels that are improperly coded as REF or ALT “D/I”

These changes correspond with the ImputationBeagle_v2.4.2 tool version.

 

January 7, 2026

Service updates:

  • The pipeline run details response's inputs field now includes the default values used for any optional inputs not provided by the user.
  • Deprecated API /api/pipelineruns/v1/pipelineruns has been removed from the service.

CLI updates:

  • terralab jobs details command will now return additional metadata about the job: the input values specified by the user, and the detected input size.

These updates correspond with version 2.0.2 of the service and version 2.1.0 of the terralab cli.

 

December 15, 2025

Service updates:

  • To help users more easily track their jobs, the service API now returns additional metadata about the user's job in the pipelineruns/result endpoint: the inputs that the user submitted for the job, and the detected input size and its unit (for array imputation, this is samples). These changes will be incorporated into the CLI and web portal soon.

These updates correspond with version 1.7.0 of the service.

 

November 21, 2025

Pipeline updates:

  • The All of Us + AnVIL array imputation pipeline now produces an additional output file containing contig-wide metrics reflecting the number of variants in the VCF input in each contig, as well as overlap with the reference panel. More information about the pipeline outputs can be found in our Pipeline Documentation.

Service updates:

  • In the web portal Job History view, users can now search and filter for jobs based on job ID, status, description, and pipeline name.

CLI updates:

  • terralab jobs list command output improvements:
    • New Output Expires column which will display outputs expiration date (when available)
    • Completed timestamp column has been removed
    • Pipeline Name and Version columns combined into a single Pipeline column
    • All timestamps are now simplified - seconds and explicit timezone information is no longer available. (note: all timestamps will continue to reflect the system’s local timezone)
    • Timestamps in terralab jobs details output will no longer show seconds information.

These updates correspond with the ImputationBeagle_v2.3.0 tool version, version 1.6.1 of the service, and version 2.0.0 of the terralab cli.

 

November 18, 2025

Service updates:

  • The backend service API now allows for searching/filtering of the getAllPipelineRunsV2 endpoint. Users may specify one or more of the following filter criteria: job ID, status, description, and pipeline name.

These updates correspond with version 1.6.0 of the service.

 

November 17, 2025

Service updates:

  • The backend service API now includes output expiration date for succeeded jobs in getAllPipelineRunsV2 response.

These updates correspond with version 1.5.0 of the service.

 

October 28, 2025

Pipeline updates:

  • The All of Us + AnVIL array imputation pipeline returns more helpful error messages in the user input QC checks, and corresponding troubleshooting documentation has been added. 

These changes correspond with the ImputationBeagle_v2.2.4 tool version.

 

October 21, 2025

Pipeline updates:

  • The All of Us + AnVIL array imputation pipeline now runs additional checks on the user's input VCF to ensure it is sorted, is block gzipped, and contains fewer than 10 million sites. The service returns a relevant error message to the user if any of these checks fail.

These changes correspond with the ImputationBeagle_v2.2.3 tool version.

 

October 17, 2025

Service updates:

  • The backend service API now includes pipeline input and output descriptions and some other metadata in the getPipelineDetails response. 
  • (bug fix) The service now enforces that compressed file outputs (e.g. .vcf.gz files) maintain the correct extensions when downloaded.

These updates correspond with version 1.4.0 of the service.

CLI updates:

  • The terralab pipelines details command now includes pipeline input and output descriptions.
  • If you are running an outdated version of the terralab CLI, you will now be prompted to update to the latest version.

These updates correspond with version 1.2.0 of the CLI.

 

October 9, 2025

Pipeline updates:

  • The All of Us + AnVIL array imputation pipeline now accepts as input a subset of autosomes, rather than requiring all chromosomes 1-22 in the input data. All data must be submitted in one input VCF file.
  • The All of Us + AnVIL array imputation pipeline now allows users to filter by DR2. Acceptable values are between 0 and 1, with 0 being the default.

These changes correspond with the ImputationBeagle_v2.2.0 tool version.

Service updates:

  • File names and string inputs are now checked for invalid characters when submitting new jobs.

This change corresponds with version 1.3.8 of the service.

CLI updates:

  • If you are running an outdated version of the terralab CLI, you will now be prompted to update to the latest version.

This change corresponds with version 1.1.2 of the CLI.

 

October 3, 2025

The following release notes correspond to the dates September 29 to October 3, 2025.

User interface updates:

  • The expected outputs for a successful job are now displayed on the Run Job page.
  • The file uploader now displays an estimated time to upload completion.
  • The "View Outputs" button in the Job History table is now disabled for jobs older than 14 days, as outputs are automatically deleted after 14 days.

Other:

  • Job completion emails have been updated to be more mobile-friendly.

 

September 17, 2025

Service updates:

  • Pipeline details now include the outputs that users can expect to receive after their job completes.
  • Job details now include the effective quota consumed by the job.

These updates correspond with version 1.3.0 of the service and are reflected in version 1.1.0 of the terralab cli.

 

September 12, 2025

Service updates:

  • The getAllPipelineRuns endpoint has been updated to v2, bringing improved pagination controls that allow users to select the page number and sort on certain PipelineRun fields ('created', 'updated', and 'quotaConsumed').

These updates correspond to version 1.2.0 of the service.

 

September 5, 2025

Service updates:

  • The service now runs a quality control check on user-submitted input files. If the input does not pass the quality control checks, the pipeline fails and returns an actionable message to the user to help diagnose the problem. See the documentation for more information on input requirements.
  • The Array Imputation pipeline will now add a VCF header line with the pipeline and pipeline version that produced the imputed output. The header line will look like ##ScientificServicesPipeline=ArrayImputation_v1

These updates correspond to version 1.1.0 of the service.

 

August 25, 2025

The All of Us + AnVIL Imputation Service Beta Release is now available to the public. 

Users can submit up to 2,500 samples to the service at no cost, thanks to funding from the NIH. 

The beta release supports imputation of array data against a 515,000+ genome reference panel. 

Related articles

Comments

0 comments

Please sign in to leave a comment.

Powered by Zendesk