Import from BigQuery
Ensure that DataSyncs has correct permissions to access your BigQuery instance
Go to Google Cloud Console → IAM & Admin → Service Accounts.
Click Create Service Account, name it, and click Create.
Assign Permissions
Grant BigQuery Data Viewer (
roles/bigquery.dataViewer
) for read access.Grant BigQuery Job User (
roles/bigquery.jobUser
) to allow query execution.This permission is necessary to ensure that we can actually pull information
After service account user creation, create a new JSON key and download it securely. This key will be used as part of the authentication for BigQuery
Gather Connection Information for DataSyncs
The JSON key you downloaded will have connection information that you can directly upload when configuring a sync
Dataset ID: the specific dataset that contains the table you are importing
Table Name: the specific table that you are importing
Setting a limit on service account user (optional)
Larger imports (>100k rows) can increase compute costs. Setting a quota ensure no unexpected compute costs:
Refer to BigQuery's Quota and Limits documentation to see which options are available to as limits you can set on the service account you are providing to DataSyncs
Last updated