In order to protect servers from being overloaded and to maintain a high quality of service, most APIs set a limit on how many calls can be made per second (or minute, or other time interval). These are called rate limits.
Rate Limits
Rate limits differ based on the API. When you pull data through Grow, this counts toward your rate limits for that API.
A visual example of rate limits is to think of them like eating a steak. If you try eating a full steak in one bite, you'll choke (or get a 500 internal server error, for example). If you try eating the steak one small bite at a time you will be more likely to get through the whole steak (and get all of your data), but you might still get full halfway through the steak (and hit the hourly or daily rate limit for that data source).
Most of Grow's popular data sources have fairly robust APIs and may not run into these types of issues. We recommend checking the help center for the specific data source you are using for detailed documentation.
Grow Data Warehouse
The Grow Data Warehouse tries to automatically adhere to the best practices of each data source while pulling the data into the warehouse. We have rules in place to try a certain number of calls per time period, with cooling down periods if the APIs return errors.
Direct Query
With Direct Query, we don't not enforce any API rate limit best practices. Setting up a lot of metrics, all pulling data as fast as they can from a slow API could cause errors. We recommend using the Warehouse, but if that is not possible, using a common dataset so only 1 dataset is pulling frequently, instead of 100 individual metrics.
API Rate Limit Terms
- Metric Refresh Rate: In the metric builder you can specify the frequency of the data refresh rate. This means every 5 minutes, 30 minutes, or hour the metric will ping the data source API and update the data. For data sources with strict (tight) rate limits, we recommend users set the metric refresh rate to be 1 hour or something that is compatible with that specific API.
- Account Level/Platform Package: Some platforms increase rate limits based on your account level with them. As you increase the package size with them, they will increase your account's API call limit. Some data sources that do this include Zoho CRM, Pardot, RingCentral, and MOZ, among a couple others. If you have questions, we suggest looking at the specific data source's API documentation.
-
API Documentation: Most data sources have really well-documented information about their specific API. You can often find specifics about a data source (rate limits and other endpoint information) by first reading through its API documentation. An easy way to find API docs is to google
[Platform name] API Documentation.