User Tools

Site Tools


google_bigquery

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
google_bigquery [2021/04/02 04:31]
windsor_ai
google_bigquery [2023/10/24 11:39] (current)
windsor_ai
Line 1: Line 1:
 ====== Getting your data into Google BigQuery ====== ====== Getting your data into Google BigQuery ======
 +
 +
  
 ==== Frequently asked questions ==== ==== Frequently asked questions ====
  
-  * Do I need to create a table or will Windsor.ai create a table?+ 
 +=== How do I setup the transfer to Google Big Query? === 
 + 
 + 
 +  - You go to https://onboard.windsor.ai/ 
 +  - On destination page you select Google Big Query 
 +  - You fill in the fields 
 +  - As connector URL you can use any URL providing a JSON. Either from the connectors or for example a URL with cached and transformed data from charts.windsor.ai 
 + 
 +{{ :mk_2023-10-24_12-35-45_2x.png?800 |}} 
 + 
 + 
 + 
 + 
 + 
 + 
 +=== Do I need to create a table or will Windsor.ai create a table? ===
  
 Windsor.ai will create a new table in your [[https://console.cloud.google.com/bigquery/|BigQuery Data Set]] with the name specified in the BigQuery configuration on [[https://onboard.windsor.ai|Windsor.ai]] Windsor.ai will create a new table in your [[https://console.cloud.google.com/bigquery/|BigQuery Data Set]] with the name specified in the BigQuery configuration on [[https://onboard.windsor.ai|Windsor.ai]]
  
-  * How do I find my data set ID?+=== How do I find my data set ID? ===
  
 Head to your [[https://console.cloud.google.com/bigquery/|BigQuery Console]] and click on the data set you would like to use. You can now see the data set ID. Head to your [[https://console.cloud.google.com/bigquery/|BigQuery Console]] and click on the data set you would like to use. You can now see the data set ID.
Line 14: Line 32:
  
  
-   What user rights do I need to grant Windsor.ai?+=== What user rights do I need to grant Windsor.ai? ===
  
 Please grant access to the user **bq-upload@windsor-ai-bigquery.iam.gserviceaccount.com** in your [[https://console.cloud.google.com/iam-admin/|IAM & Admin Console]] and assign the roles **BigQuery Admin** and **Storage Admin** Please grant access to the user **bq-upload@windsor-ai-bigquery.iam.gserviceaccount.com** in your [[https://console.cloud.google.com/iam-admin/|IAM & Admin Console]] and assign the roles **BigQuery Admin** and **Storage Admin**
-   How do I know if my data was processed successfully?+ 
 +=== How do I know if my data was processed successfully? ===
  
 After you set up the configuration and click **Save**, it can take up to a few minutes for the data to load. A message will appear below **Last processing status**. If it is successful you can query your database with <code> select * from [your table name] </code> After you set up the configuration and click **Save**, it can take up to a few minutes for the data to load. A message will appear below **Last processing status**. If it is successful you can query your database with <code> select * from [your table name] </code>
  
-  * Why is access to the cloud storage required?+=== Why is access to the cloud storage required? ===
  
 Windsor.ai creates a file in JSON format for the data it streams into your BigQuery table. Windsor.ai creates a file in JSON format for the data it streams into your BigQuery table.
  
-  * How do I set up the cloud storage bucket?+=== How do I set up the cloud storage bucket? ===
  
 Head to [[https://console.cloud.google.com/storage|Cloud Storage]] and create a new bucket (unless you want to reuse an existing one.  Head to [[https://console.cloud.google.com/storage|Cloud Storage]] and create a new bucket (unless you want to reuse an existing one. 
Line 37: Line 56:
 Once you click finish add the name (marketing-data-demo in this sample) in Google Cloud Storage Bucket on [[https://onboard.windsor.ai|Windsor.ai]] Once you click finish add the name (marketing-data-demo in this sample) in Google Cloud Storage Bucket on [[https://onboard.windsor.ai|Windsor.ai]]
  
-  * How do I set up a cloud storage folder?+=== How do I set up a cloud storage folder? ===
  
 In your [[https://console.cloud.google.com/storage|Cloud Storage]] bucket, click on create new folder. Use this folder name on [[https://onboard.windsor.ai|Windsor.ai]] In your [[https://console.cloud.google.com/storage|Cloud Storage]] bucket, click on create new folder. Use this folder name on [[https://onboard.windsor.ai|Windsor.ai]]
Line 44: Line 63:
  
  
-   I'm having problems with my connector URL. How do I test if it is working correctly?+=== I'm having problems with my connector URL. How do I test if it is working correctly? ===
  
 If you have a complex query it makes sense to test your connector URL in a tool like [[https://www.postman.com/|Postman]] or [[https://insomnia.rest/|Insomnia]] to make sure that it is working as expected. If you have a complex query it makes sense to test your connector URL in a tool like [[https://www.postman.com/|Postman]] or [[https://insomnia.rest/|Insomnia]] to make sure that it is working as expected.
google_bigquery.1617334270.txt.gz · Last modified: 2021/04/02 04:31 by windsor_ai