1. Grow
  2. Connecting to Data Sources
  3. Development

Custom REST API

Article last updated:

Custom REST API feature helps you integrate with your own APIs or connectors that are not supported by Grow. This is a generic tool and provides a flexible way to retrieve data from any API endpoint. Then, you can warehouse the data for further use.

You can set health endpoints for testing, authentication modes, paging strategy, query parameters, etc. and Grow gets the data and puts it in the Warehouse, which gets synced periodically.

Note that Grow supports JSON, CSV, and XML response types.

Please go through the API docs of the API that you want to connect to Grow. The APIs can have multiple parameters and constraints set which might affect the connection and data retrieval. Having a thorough knowledge of the API is essential for setting this connection up.

Connecting to Your Custom APIs

Before you start configuring APIs with Grow, you will need API URLs and authentication details.

When you have those details,

  1. Click on the Data tab at the top left of the global navigation. Then select Connections in the Data Library section.

  2. Click on the blue Connect button and search for Custom REST API.

  3. Select the Auth method, and enter the Base URL and Health endpoint details the respective fields.

    Based on the authentication method, enter the required details for authentication. To know more about authentication, go to next section. Select None if you do not use any authentication method.

    Health endpoint is optional and works only with Basic and Authorization Header authentication methods. If you fill in this field, make sure that you start with '/' character.

  4. Select Connect to connect to the API.

  5. Once connected, you will see the tables that are warehoused. Add a table to the warehouse by configuring an API endpoint. Toggle the button next to the table, to either enable or disable the table from being warehoused.

Authenticating Custom APIs

Following authentication methods are available for use:

  • Authorization Header: Enter a token to be used for authentication.
  • Basic Auth: Enter Username and Password for basic authentication.
  • OAuth2: This is a more advanced authentication method. It is used by most known companies to secure their APIsIn this case, you need to enter the Client ID and secret, the authorization URL to validate this information, as well as the token URL to update the authentication token automatically.

Adding Endpoints

To add endpoints for the custom API,

  1. Go to the Custom API connection, select Manage Connections.

  2. Select Add Custom Table.
    Custom Rest API only provides Custom Tables to warehouse your data.

  3. In the Add a Custom REST API Endpoint pop-up, enter the necessary details:

    • Enter the Name and Description for the endpoint.

    • Enter the Endpoint URL. This must start with a '/' character.

    • Select the Http Method used to retrieve data.

    • Enter the JSON Path to Data. Use dot notation to describe the path to the custom table data. If the data is present directly in the response, omit this field.

    • Select the Paging Strategy. This improves the performance of the API. To know more, go to the Paging Strategy section.

    • Add custom headers or parameters for the API, if necessary, by selecting Add Header/ Add Parameter. Enter the Key and Value.

  4. Select Save to save the details.

Once an endpoint is created, the data will start syncing in the warehouse.

Paging Strategy

The API can return huge amount of data. This can lead to network traffic issues. To deal with this, paging strategy comes into play. This breaks the response data into multiple segments, thereby reducing the stress on the network.

The followingPaging Strategyoptions are available inGrow:

  • Page Parameter: Use this to specify the parameters to be used for paging.

  • Next Page URL: Use this option if the API response contains the next page URL. Enter the path to the field containing the next page URL, using dot notation for nested values.

  • Cursor: Use this strategy if the response has a cursor value. Specify the Cursor Key and the Next Cursor Path in the respective fields.

  • Offset Limit: Use this to specify the Limit and Offset, to fetch the results. The limit is set to 100 records, even if not included in the request call. To access results beyond the first 100 records, mention "offset" parameter in the request.

    For example, if you want to access results from 201 to 300, select Add Parameter, then enter the Parameter Key as Offset, and enter the Parameter Value as 200. Note that the offset can only be set in multiples of 100, or left blank (for the first 100 records).

The default maximum paging limit is 1000 pages.

Modifying Endpoints

Custom Rest API/Grow integration allows you to iterate an endpoint with data from a different endpoint.

Before using this feature, ensure the following:

  • You have an existing source table in the warehouse.
  • You have identified the column in the source table that will be used for the endpoint setup.

Setup

  1. Click Add Custom table to set a new endpoint.
  2. Enable the Use parent stream as source option.
  3. Select the warehouse table you want to use as the parent source from the dropdown menu.
  4. Select the column that needs to be the source of the new endpoint.
    This column is referred to as{{<column_name>}}.
  5. Define the endpoint pattern using{{<column_name>}}.
    This pattern dynamically incorporates values from the selected source column.
    Example:
    /surveys/{{<column_name>}}
  6. Expand Test Settings > Test to check the setup.
  7. Click Save to complete the process.
Was this article helpful?