Using Components: Google BigQuery Source

Google BigQuery source is used to read data from a table in a BigQuery dataset.

source settings
  1. Define the parameters for connecting to your table as follows (for more information, see Allowing Xplenty access to my Google BigQuery dataset):
    • google bigquery connection - either click the drop-down arrow and select an existing connection, or click (create new) to create a new connection.
    • access mode - select table to extract an entire table or query to execute a user query.
    • source table - the table from which the data will be imported. You can also refer to the table with dataset.table notation. Table names can also contain a date based partition using the notation tablename$partition (e.g. events$20160405).
    • or query - type in a BigQuery (legacy) SQL query.  When using regular expressions, make sure to escape \characters.
      • example: \\ gets transformed to \ and \\\\ gets
        transformed to \\
    • Use Legacy SQL - check to use legacy SQL syntax or uncheck to use standard SQL.

    Then click the Test Connection button to help check that the connection is good and that the source table exists.

column mapping

  1. After defining the source settings you can use the green Auto-detect schema button to get the field names and data types or the Preview button to preview the data and fill in the fields manually.
  2. Define the columns you want to extract from the table as follows:
    • Define the column name in the table.
    • Define the alias you will use for the column as a field in the data-flow.
    • Define the data type for the field. Use the following table when matching Google BigQuery data types to Xplenty data types.
Google BigQuery Xplenty
String String
Integer Long
Float Double
Timestamp Datetime
Record Map
Record (repeated) Bag

Feedback and Knowledge Base