Using Components: Google BigQuery Source

Google BigQuery source is used to read data from a table in a BigQuery dataset.

source settings
  1. Define the parameters for connecting to your table as follows (for more information, see Allowing Xplenty access to my Google BigQuery dataset):
    • google bigquery connection - either click the drop-down arrow and select an existing connection, or click (create new) to create a new connection.
    • access mode - select table to extract an entire table or query to execute a user query.
    • source table - the table from which the data will be imported. You can also refer to the table with dataset.table notation. Table names can also contain a date based partition using the notation tablename$partition (e.g. events$20160405).
    • or query - type in a BigQuery (legacy) SQL query.  
    • Use Legacy SQL - check to use legacy SQL syntax or uncheck to use standard SQL.

    Then click the Test Connection button to help check that the connection is good and that the source table exists.

column mapping

  1. After defining the source settings you can use the green Auto-detect schema button to get the field names and data types or the Preview button to preview the data and fill in the fields manually.
  2. Define the columns you want to extract from the table as follows:
    • Define the column name in the table.
    • Define the alias you will use for the column as a field in the data-flow.
    • Define the data type for the field. Use the following table when matching Google BigQuery data types to Xplenty data types.
Google BigQuery Xplenty
String String
Integer Long
Float Double
Timestamp Datetime
Record Map
Record (repeated) Bag

Creating packages

  1. Creating a new package
  2. Create a package from a template
  3. Working in the package designer
  4. Using Components: Facebook Ads Insights Source (Beta)
  5. Using components: File Storage Source
  6. Using components: Database Source
  7. Using components: Google AdWords Source
  8. Using components: NetSuite Source
  9. Using Components: Google Analytics Source
  10. Using Components: Google BigQuery Source
  11. Using components: Google Cloud Spanner Source
  12. Using Components: Bing Ads Source
  13. Using components: MongoDB Source
  14. Using components: Amazon Redshift Source
  15. Using Components: Rest API Source
  16. Using Components: Salesforce Source
  17. Using components: Select
  18. Using components: Sort
  19. Using components: Rank
  20. Using components: Limit
  21. Using components: Sample
  22. Using components: Join
  23. Using components: Cross Join
  24. Using components: Clone
  25. Using components: Cube and Rollup
  26. Using components: Union
  27. Using components: Filter
  28. Using Components: Window
  29. Using components: Assert
  30. Using components: Aggregate
  31. Using components: Distinct
  32. Using components: File Storage Destination
  33. Using components: Amazon Redshift Destination
  34. Using Components: Salesforce Destination (Beta)
  35. Using components: Google BigQuery Destination
  36. Using components: Google Cloud Spanner Destination
  37. Using components: Database Destination
  38. Using components: MongoDB Destination
  39. Using and setting variables in your packages
  40. Validating a package
  41. Using pattern-matching in source component paths
  42. Using ISO 8601 string functions
  43. Using Expressions in Xplenty
  44. Xplenty Functions

Feedback and Knowledge Base