Google Dataform API

Name: Dataform API
Brand: Dataform API
Availability: InStock

✓ Official Vendor SpecAnalyticsData Pipelinesoauth243 EndpointsREST

For Agents

Programmatically manage Dataform repositories, workspaces, and SQL workflow invocations in BigQuery. Lets agents author, compile, and trigger SQL pipelines without touching the Dataform UI.

Use for: I need to trigger a Dataform workflow invocation against BigQuery, Create a new Dataform repository and connect it to a Git remote, Compile the latest commit of a Dataform workspace into an execution graph, List all workflow invocations that failed in the last release

Not supported: Does not run SQL queries directly, manage BigQuery tables, or schedule non-SQL tasks - use for Git-backed Dataform repository, workspace, and workflow-invocation management only.

The Dataform API lets you develop, version-control, and operationalize SQL pipelines that run inside BigQuery. It manages repositories, workspaces, release configurations, compilation results, and workflow invocations so teams can ship reproducible SQL transformations through Git-backed workflows. The API exposes 43 endpoints covering Git operations (commit, push, pull, fetch history), workspace file editing, and scheduled workflow execution against BigQuery.

Install Jentic One Beta

Connect the Dataform API to your agent

Jentic One is a self-hosted execution layer for AI agents. It lets your agent call the Dataform API, or any other public or private API you need. You set the rules, the agent never sees your credentials, and every call is logged.

Two steps, two machines. Install the instance in a safe environment, then register your agent from wherever it runs.

Step 1: Jentic One Host machine

# On the machine that will host your Jentic One instance:
curl -fsSL "https://jentic.com/install.sh?src=apis&api=%2Fapis%2Fgoogleapis.com%2Fdataform" | sh

Step 2: Agent machine

# On the machine where your agent runs (keep this separate from the instance):
curl -fsSL "https://jentic.com/install.sh?src=apis&api=%2Fapis%2Fgoogleapis.com%2Fdataform" | sh
jentic register       # connects your agent to your Jentic One instance

Jentic One is in public beta. The setup above keeps your agent separate from the instance, which is what you want before using real credentials: an agent running as the same OS user as Jentic One can read its stored keys directly. Just evaluating? A single local install is fine to start. See the secure deployment guide for the tiers.

Capabilities

What an agent can do with Dataform API.

Create and manage Dataform repositories backed by Git remotes

Edit SQLX files inside workspaces and commit changes through the API

Compile workflow definitions into validated BigQuery execution graphs

Trigger workflow invocations on demand or on a release schedule

Inspect Git status, ahead/behind counts, and file history for a workspace

GET STARTED

Start building with Dataform API

Explore with Jentic One

View OpenAPI Document

Use Cases

Patterns agents use Dataform API for, with concrete tasks.

★ Scheduled BigQuery Transformation Pipelines

Operationalize SQL transformations in BigQuery by defining release configurations that compile a Git ref and run on a schedule. Dataform handles dependency resolution between models, incremental table builds, and assertions, so analytics teams ship production SQL pipelines without standing up Airflow. A typical setup completes in a few hours by pointing the API at an existing GitHub repository.

Create a release configuration on repository 'analytics-prod' that compiles the main branch and triggers a workflow invocation every night at 02:00 UTC

Git-Backed SQL Development Workflow

Edit SQLX models inside an isolated workspace, commit changes, push to a Git remote, and merge through pull requests before promoting to production. The API exposes fetchFileGitStatuses, commit, push, and pull operations so agents and CI bots can drive the full development loop without the Dataform console. Most repository-and-workspace setups take under an hour.

Create a workspace 'feature-revenue-model' from repository 'analytics-prod', write a new SQLX file, commit it with message 'add daily revenue model', and push to origin

Pre-Production Compilation Validation

Before a SQL change reaches production, compile the workflow against BigQuery to catch reference errors, circular dependencies, and assertion failures. The API returns a CompilationResult with the resolved DAG and any compilation errors, so CI pipelines can fail fast on broken SQL. This adds compile-time safety to a workflow that would otherwise only catch errors at runtime in BigQuery.

Create a compilation result for the pull-request commit SHA on repository 'analytics-prod' and report any compilation errors back to the PR

AI Agent SQL Pipeline Operator

An AI agent can drive Dataform end-to-end through Jentic: search for the right operation by intent, load the input schema, and execute repository, workspace, compilation, and workflow-invocation calls without browsing reference docs. Credentials stay in your Jentic One instance, so the agent receives only scoped access tokens. Setup through Jentic takes under an hour versus several days for a direct integration with Google's OAuth flow.

Use Jentic to search 'trigger a dataform workflow', load the createWorkflowInvocation schema, and execute it against repository 'analytics-prod' with the latest compilation result

Key Endpoints

43 endpoints — the dataform api lets you develop, version-control, and operationalize sql pipelines that run inside bigquery.

METHOD

PATH

DESCRIPTION

POST

/v1beta1/{+parent}/repositories

Create a Dataform repository

POST

/v1beta1/{+parent}/workspaces

Create a workspace inside a repository

POST

/v1beta1/{+name}:commit

Commit changes in a workspace

POST

/v1beta1/{+parent}/compilationResults

Compile a Git ref into an execution graph

POST

/v1beta1/{+parent}/workflowInvocations

Trigger a workflow invocation against BigQuery

GET

/v1beta1/{+name}:fetchFileGitStatuses

Fetch Git status of files in a workspace

POST

/v1beta1/{+parent}/repositories

Create a Dataform repository

POST

/v1beta1/{+parent}/workspaces

Create a workspace inside a repository

POST

/v1beta1/{+name}:commit

Commit changes in a workspace

POST

/v1beta1/{+parent}/compilationResults

Compile a Git ref into an execution graph

POST

/v1beta1/{+parent}/workflowInvocations

Trigger a workflow invocation against BigQuery

GET

/v1beta1/{+name}:fetchFileGitStatuses

Fetch Git status of files in a workspace

Why Jentic?

What agents get from Jentic-routed access to this vendor.

Setup

Wiring the Dataform API by hand means setting up Google OAuth, refreshing short-lived scoped tokens rather than holding client secrets in code, and addressing repository and workspace resources on dataform.googleapis.com. Through Jentic you install once, import the Dataform API from the API Directory, store the Google credential once, and your agent calls it.

Permission scoping

The API carries the parent and resource name in the URL path (/v1beta1/{parent}/repositories and /v1beta1/{name}:commit), so a rule can pin your agent to one repository or workspace. You choose the operations it may call, so it can create workspaces and trigger workflow invocations while anything you leave out stays unavailable.

Credential isolation

Your Google OAuth credential is stored once, encrypted, by your own Jentic One instance and injected at execution time. It never enters the agent's prompt, logs, or context.

Intent-based discovery

Agents search Jentic by intent such as 'trigger a Dataform workflow', and Jentic returns the matching Dataform operation with its input schema so the agent calls the right endpoint without browsing the reference docs.

FAQs

Specific to using Dataform API through Jentic.

What authentication does the Dataform API use?

The Dataform API uses OAuth 2.0 with Google scopes (cloud-platform). Through Jentic the OAuth client and refresh tokens are stored in your Jentic One instance and the agent receives a short-lived scoped access token, so raw Google credentials never enter the agent context.

Can I trigger BigQuery SQL pipelines with the Dataform API?

Yes. POST /v1beta1/{+parent}/workflowInvocations triggers a compiled workflow against BigQuery, and POST /v1beta1/{+parent}/compilationResults produces the compilation that the invocation runs from. Together they give you full programmatic execution of a Dataform release.

What are the rate limits for the Dataform API?

Google enforces standard Cloud quotas on Dataform: per-project read/write quotas on repositories, workspaces, and workflow invocations, plus the underlying BigQuery quotas on jobs and slot usage. Quotas are visible in the Cloud Console under IAM and admin, quotas, filtered to dataform.googleapis.com.

How do I commit a SQLX file to a Dataform workspace through Jentic?

Search Jentic for 'commit changes to a dataform workspace', load the schema for POST /v1beta1/{+name}:commit, and execute with the workspace name, commit message, and file actions array. Jentic returns the request body shape and handles the OAuth token exchange.

Is the Dataform API free?

Dataform itself has no per-call charge, but the BigQuery jobs that workflow invocations run are billed under standard BigQuery on-demand or reservation pricing. Storage of repositories and workspaces is also free; you pay for the compute the SQL consumes.

How do I check the Git status of a workspace?

Call GET /v1beta1/{+name}:fetchFileGitStatuses with the workspace resource name. It returns the per-file status (added, modified, deleted, conflicted) so a CI bot or agent can decide whether to commit, reset, or surface conflicts before pushing.

Can I limit what my agent is allowed to do with the Dataform API?

Yes. Because Jentic One is self-hosted, you set the rules that decide which Dataform operations and credentials your agent may use. Since the API carries the parent and resource name in the URL path, such as /v1beta1/{parent}/repositories and /v1beta1/{name}:commit, you can pin the agent to a single repository or workspace. You also choose the exact operations it may call, so it can create workspaces and trigger workflow invocations while any operation you leave out stays unavailable.

Google Dataform API

Connect the Dataform API to your agent

Capabilities

Start building with Dataform API

Use Cases

Key Endpoints

Why Jentic?

Related APIs

FAQs