Authenticate BigQuery

Ensure that DataSyncs has correct permissions to access your BigQuery instance

  1. Go to Google Cloud Console → IAM & Admin → Service Accounts.

  2. Click Create Service Account, name it, and click Create.

  3. Assign Permissions

    • Grant BigQuery Data Viewer (roles/bigquery.dataViewer) for read access or Big Query Data Editor (roles/bigquery.dataEditor) for write access.

    • Grant BigQuery Job User (roles/bigquery.jobUser) to allow query execution.

      • This permission is necessary to ensure that we can actually pull information

  4. After service account user creation, create a new JSON key and download it securely. This key will be used as part of the authentication for BigQuery

Gather Connection Information for DataSyncs

  • The JSON key you downloaded will have connection information that you can directly upload when configuring a sync

  • Dataset ID: the specific dataset that contains the table you are importing

  • Table Name: the specific table that you are importing

Setting a limit on service account user (optional)

Larger imports (>100k rows) can increase compute costs. Setting a quota ensure no unexpected compute costs:

  • Refer to BigQuery's Quota and Limits documentation to see which options are available to as limits you can set on the service account you are providing to DataSyncs

Last updated