Custom REST API feature helps you integrate with your own APIs or connectors that are not supported by Grow. This is a generic tool and provides a flexible way to retrieve data from any API endpoint. Then, you can warehouse the data for further use.
You can set health endpoints for testing, authentication modes, paging strategy, query parameters, etc. and Grow gets the data and puts it in the Warehouse, which gets synced periodically.
Note that only JSON format is supported for API requests and responses.
Please go through the API docs of the API that you want to connect to Grow. The APIs can have multiple parameters and constraints set which might affect the connection and data retrieval. Having a thorough knowledge of the API is essential for setting this connection up.
Connecting to Your Custom APIs
Before you start configuring APIs with Grow, you will need API URLs and authentication details.
When you have those details,
Click on the Data tab at the top left of the global navigation. Then select Connections in the Data Library section.
Click on the blue Connect button and search for Custom REST API.
Select the Auth method, and enter the Base URL and Health endpoint details the respective fields.
Based on the authentication method, enter the required details for authentication. To know more about authentication, go to next section. Select None if you do not use any authentication method.
Health endpoint is optional and works only with Basic and Authorization Header authentication methods. If you fill in this field, make sure that you start with '/' character.
Select Connect to connect to the API.
Once connected, you will see the tables that are warehoused. Add a table to the warehouse by configuring an API endpoint. Toggle the button next to the table, to either enable or disable the table from being warehoused.
Following authentication methods are available for use:
- Authorization Header: Enter a token to be used for authentication.
- Basic Auth: Enter Username and Password for basic authentication.
- OAuth2: This is a more advanced authentication method. It is used by most known companies to secure their APIsIn this case, you need to enter the Client ID and secret, the authorization URL to validate this information, as well as the token URL to update the authentication token automatically.
To add endpoints for the custom API,
Go to the Custom API connection, select Manage Connections.
Select Add Custom Table.
In the Add a Custom REST API Endpoint pop-up, enter the necessary details:
Enter the Name and Description for the endpoint.
Enter the Endpoint URL. This must start with a '/' character.
Select the Http Method used to retrieve data.
Enter the JSON Path to Data. Use dot notation to describe the path to the custom table data. If the data is present directly in the response, omit this field.
Select the Paging Strategy. This improves the performance of the API. To know more, go to the Pagination section.
Add custom headers or parameters for the API, if necessary, by selecting Add Header/ Add Parameter. Enter the Key and Value.
Select Save to save the details.
Once an endpoint is created, the data will start syncing in the warehouse.
The API can return huge amount of data. This can lead to network traffic issues. To deal with this, pagination comes into play. Pagination breaks the response data into multiple segments, thereby reducing the stress on the network.
Following pagination options are available in Grow:
Page Parameter: Use this to specify the parameters to be used for paging.
Next Page URL: Use this option if the API response contains the next page URL. Enter the path to the field containing the next page URL, using dot notation for nested values.
Cursor: Use this strategy if the response has a cursor value. Specify the Cursor Key and the Next Cursor Path in the respective fields.
Offset Limit: Use this to specify the Limit and Offset, to fetch the results. The limit is set to 100 records, even if not included in the request call. To access results beyond the first 100 records, mention "offset" parameter in the request.
For example, if you want to access results from 201 to 300, select Add Parameter, then enter the Parameter Key as Offset, and enter the Parameter Value as 200. Note that the offset can only be set in multiples of 100, or left blank (for the first 100 records).
The default maximum paging limit is 1000 pages.