We’re excited to announce the arrival of our Data Transform Pipeline feature! Previously, data transforms had to be performed in a specific order. Now you can do them in any order!
Given this flexibility, here are some tips on using the Data Transform Pipeline.
Note: If you want to see what transforms are available, check out this article for more details.
How to Use
To add a transform, simply click Add Transform. This will bring up the list of transforms. You can add whatever transform you need, and an icon for that transform will appear next to the Add Transform button.
To remove a transform, click on the transform icon. This will bring up the window for that transform. In the lower left-hand corner, click the Remove button.
To rearrange the transform order, hover over the icon you want to move. Two dotted lines will appear on the left side of the icon. Click and drag the two dotted lines to move the transform to whatever order you want. But be careful---if you move transforms into a different order, you could break the logic that your data is relying on.
Tips, Tricks, & Best Practices
Here are some tips and best practices with using the new transform pipeline.
The filter transform now has the operators
OR. When using the filter transform, all of the settings on that transform will either be
OR. You can’t mix and match.
Moving a transform may break the logic after it. If you move a transform that created a column based on another transform’s column, the logic will break. Let’s say you create an aggregation that was based on a calculated column. If you move the aggregation before the calculated column, the aggregation will show up red, because it doesn’t work to aggregate a column that doesn’t exist in the logic. You have to change the configurations to make sure the logic makes sense.
Add aggregations after filters. If you plan on using filters and aggregations, the logical (not required, but logical) flow is to do aggregations last. This allows you to still see everything that you’ve set up with filters.
Don’t add a ton of transforms. We currently don’t have a way to scroll over (yet) through a long list of transforms, so in order to see the transforms on the end, you’ll need to remove some of the earlier ones.
Disabled Add Transforms button? There’s a reason why. If you have columns with a duplicate title, the Add Transforms button will be grayed out and unusable. Be sure your data is formatted correctly, which includes unique column headers.
As a tip, the logic between each transform is AND logic.
Pivot Data & Calculated Columns
One major difference is that you can pivot your data before using calculated columns. While it was not possible previously, it is now!
Pretend you track website sessions by source over time and you’d like to get the percentage breakdown of each source as it pertains to the total. You could first use the pivot table function to get the data broken out over time by source. Afterwards, you can use calculated columns to add the two columns together and take the percentage of each. See it in action here.
Complex Filter Logic
Another handy feature is being able to use complex logic. When you use the filter transform, it gives you the option to use
Let’s say you wanted to look only at direct traffic and traffic from a specific source such as Capterra over the last week. To filter down to these results you could use two filters: the first filter would be
Source = "direct" OR
Source = "Capterra" the second filter would be
Date = "last 7 days" . See it in action here.
Questions about the data transform pipeline? Email us at firstname.lastname@example.org or chat in on the site.