We’re excited to announce the arrival of our Data Transform Pipeline feature! Previously, data transforms had to be performed in a specific order. Now you can do them in any order!
Given this flexibility, here are some tips on using the Data Transform Pipeline.
Note: If you want to see what transforms are available, check out this article for more details.
How to Use
To add a transform, simply click Add Transform. This will bring up the list of transforms. You can add whatever transform you need, and an icon for that transform will appear under the Data Settings as the second step in the pipeline.
To remove a transform, click on the transform. Within that transform in the lower right, click the Remove button.
To rearrange the transform order, hover over the transform you want to move. Two dotted lines will appear on the left side of the transform box. Click and drag the two dotted lines to move the transform to whatever order you want. But be careful---if you move transforms into a different order, you could break the logic that your data is relying on.
Tips, Tricks, & Best Practices
Here are some tips and best practices with using the new transform pipeline.
The filter transform now has the operators
OR. When using the filter transform, all of the settings on that transform will either be
OR. You can’t mix and match within one filter transform, though you can add multiple filter transforms in a row to accomplish a combination of
Moving a transform may break the logic after it. If you move a transform that created a column based on another transform’s column, the logic will break. Let’s say you create an aggregation that was based on a calculated column. If you move the aggregation before the calculated column, the aggregation will show up red, because it doesn’t work to aggregate a column that doesn’t exist in the logic. You have to change the configurations to make sure the logic makes sense.
Add aggregations after filters. If you plan on using filters and aggregations, the logical (not required, but logical) flow is to do aggregations last. This allows you to still see everything that you’ve set up with filters.
Disabled Add Transforms button? There’s a reason why. If you have columns with a duplicate title, the Add Transforms button will be grayed out and unusable. Be sure your data is formatted correctly, which includes unique column headers.
As a tip, the logic between each transform defaults to AND logic.
Pivot Data & Calculated Columns
One major difference is that you can pivot your data before using calculated columns. While it was not possible previously, it is now!
Pretend you track website sessions by source over time and you’d like to get the percentage breakdown of each source as it pertains to the total. You could first use the pivot table function to get the data broken out over time by source. Afterwards, you can use calculated columns to add the two columns together and take the percentage of each.
Complex Filter Logic
Another handy feature is being able to use complex logic. When you use the filter transform, it gives you the option to use
Let’s say you wanted to look only at direct traffic and traffic from a specific source such as Capterra over the last week. To filter down to these results you could use two filters: the first filter would be
Source = "direct" OR
Source = "Capterra" the second filter would be
Date = "last 7 days" .
Questions about the data transform pipeline? Email us at firstname.lastname@example.org or chat in from your dashboards.