How to Warehouse Your Data
From a New Connection
You can warehouse data from a new connection by selecting the “+ Connection” button on the Connections Overview Page from the Data tab in Grow. This will take you to the New Connection Page, where you can select the data source you would like to warehouse. This will open the warehouse table selection flow, where you will need to authenticate. Once authenticated, you can select which tables of data you want to warehouse.
From an Existing Connection
You can begin warehousing data from an existing connection by selecting a connection from the Connections Overview Page. From the Connection Details page, select the “Manage Connection” button inside the Connection Details section. This will open the warehouse table selection flow. Your authentication is already connected, so you can immediately select which tables of data you want to warehouse.
Picking what to store
Once inside the Warehouse Table Selection flow, you will be presented with a list of tables you can warehouse within Grow.
Each table has a corresponding toggle. Turning the toggle on indicates that the table is active and will be kept up-to-date on a designated refresh interval (see the Refresh Rates section below for more details). Turning the toggle off indicates that the table is no longer actively being refreshed; however, the data you have already synced up to this point will still be accessible throughout Grow.
Additionally, these tables are broken into three categories (Suggested Tables, Other Tables, and Custom Tables) described below:
Suggested Tables
For each data source (with the exception of databases), we have curated the most used tables of data that our customers use and put them at the top of the list. These are the tables we suggest you warehouse based on what we see hundreds of other customers using within Grow for that particular data source.
When warehousing a new connection, suggested tables are automatically toggled on for your convenience; however, you can toggle any of them off before beginning to store any data.
Other Tables
For some data sources, particularly CRMs with custom objects, we list off all other predefined tables of data in this section.
Other tables are not automatically turned on and must be toggled on individually for these tables to be stored.
For Databases, all public schemas and their corresponding tables will appear in this section.
Custom Tables
For some data sources, there are alternative ways to pull data from that data source. Popular custom tables include:
- SQL queries against a database or data source that supports this kind of method (e.g., Salesforce SOQL)
- Customizable endpoints. (e.g., HubSpot Analytics for Sources which support multiple groupings and drill-down options)
Begin Storing the Data
When you have finished selecting the data that you want stored in Grow’s Data Warehouse, click the “Sync & Store” button at the bottom.
This will begin the initial population of your warehouse with the tables you defined. Depending on the data source, populating your data warehouse for the first time may take a while, even up to several hours.
How to Manage Your Warehouse Connection
Editing the Warehouse Table Selections
Once you’ve set up a warehouse, you can edit what tables are actively refreshing. Find the connection that is actively warehousing data and select the “Manage Connection” from the Connection Details section on the Connection Details Page. This will reopen the Warehouse Table Selection flow where you can change which tables are toggled on or off and edit any custom tables you have set up.
Interpreting the Warehouse Table
For some data sources, there is a single toggle in the warehouse table selection flow that populates several tables in the warehouse. For instance, the Shopify “Orders” toggle populates several tables including “Refunds”, “Order Tax Lines”, and “Order Line Items”. These appear in the “Warehoused Tables” section on the Connections Detail page.
The Warehoused Tables section shows the Warehoused Parent Tables and each of their derived Warehouse Tables. We also indicate the # of Rows of data we are storing, the storage size of that data, when it was last updated, and its current refresh interval.
Using Warehoused Data in the Dataset & Metric Builders
Selecting a warehoused table
To select your warehoused data in the Metric or Dataset Builder, add a new report and go to the Connection tab in the Add Data modal. Then select which data source your warehouse is built from.
Now you will be in the Metric or Dataset Builder, and you will finish selecting what data you want to use. If you have more than one connection to the chosen data source, select the connection from the dropdown list that your warehouse is built from.
In the Data section of the Data Settings, toggle over to the "Warehouse" option instead of "Direct Query". From the dropdown menu select the warehoused report that you want to use.
Managing your connection from within the builder
Select the text “Manage Warehoused Reports”. This will open the Warehouse Table selection flow. You can toggle on new warehoused tables from this flow. Once you toggle on a new connection, the initial population will begin for that warehoused table. In the drop down menu it will indicate once it begins processing. Once we have some data in the warehouse, you can pull it into the builder while the rest of the warehouse finishes populating.
Note: If you sync more than one new table, you may have to wait until the first table is finished populating before the second and subsequent tables begin populating. We suggest syncing and storing only one table at a time while in the building flow.
How to Adjust Refresh Rates
The default is 12 hours per table. The minimum refresh frequency will differ depending on the amount of data you have, the data source’s API limitations, the number of tables you are warehousing for that particular data source, and the refresh strategies available to each data source.
You can update the refresh rate on the Warehoused Table details page through the Warehouse section of the Data Library section. There is a dropdown menu that shows how often that specific table can be refreshed.
Currently Supported Data Sources
Currently data warehousing is available only for the following data sources:
- ActiveCampaign
- Airtable
- Amazon Redshift
- Asana
- BigCommerce
- Chargebee
- Copper
- Freshbooks
- Google Ads
- Google BigQuery
- HubSpot
- Infusionsoft
- MySQL
- Pipedrive
- Postgres
- Salesforce
- Shopify
- SQL Server
- Square
- TSheets
- WooCommerce