Tray Platform / Standard / Best Practices / Pagination / Batch processing / Updating missing customer info

Updating missing customer info

(Pagination Tutorial 1)

Introduction

This article is adapted from a Tray blogpost titled Are Non-Paying Customers Burning Out Your Support Team? Here’s How to Fix It (note that the original blogpost was written before the 'loop forever' operation was available - so you should definitely use this article as your guide!)

The idea here is that you are making use of Salesforce as a CRM to manage accounts and contracts, alongside Stripe as a payment system, and you are wanting Tray to take care of the task of automatically checking if any of your contracts are not yet linked to Stripe accounts - thus making sure that non-paying customers aren't taking up your valuable time!

The tutorial below will take you through setting up a workflow which runs periodically and uses a 'pagination' system which can loop through dozens of records and stop when the job is done.

A basic breakdown of what we will be achieving is:

  1. Use the Stripe List customers operation to get all customer data

  2. Use Loop Connectors to create batches of customers which are then sub-processed one-by-one, grabbing their Stripe ID and email address

  3. For each customer presented by the loop, use the SalesForce Find Record operation (by email) to get the SalesForce ID and Stripe ID as stored in SalesForce

  4. Use a Boolean Connector to check if the Stripe ID for that Salesforce Contract is empty

  5. If so then update the Contract with the Stripe ID

This tutorial will be divided into two stages - setting up the system to paginate through the Stripe list of customers, followed by adding the steps to find each customer in Salesforce and updating their Stripe ID if needed.

Set up the pagination system

When returning records, the Stripe API returns a maximum of 100 records at a time, in a list sorted by ID. If there are more than 100 records to be returned, Stripe returns a has_more property so you know whether to make another request. It also lets you pass in a value called Starting After so that you can enter the ID of the last customer in the previous list of 100; thus Stripe will know to start the next batch of 100 from the customer which comes after this ID.

Because we can't get a count of the total records to be returned by Stripe, we will have to set our main loop as a continuous loop which breaks once the has_more property returns True; and within this main loop we will create the sub loop which processes the customers from the main loop batch one-by-one.

Start your workflow with a scheduled trigger set to run e.g. every Monday at 6am

You can then move on to setting up the pagination:

Pagination - Set the main and sub loops

The steps involved here are:

1 - Start a continuous loop

We first add the Main Loop connector:

main-loop

and set the operation to 'Loop Forever':

main-loop-operation

Get last ID is a (/platform/connectors/docs/core/data-storage/)[Data Storage] step:

get-last-id-1

It is set as follows:

get-last-id-operation-1

We are naming the key which is set in the Data Storage connector stripe_last_customer

Note - the 'Default Value' must be set to 'empty string'. This will allow for the fact that no ID will be stored in the first run, and that Stripe will not accept a null value.

List Stripe customers is a Stripe Connector:

The Operation is set as 'List Customers' and the Starting After value set as a $.steps.storage-1.value jsonpath to pick up the Last Customer ID:

stripe-start-after

Note that you can set the limit of customers returned as anything up to 100.

We can then move on to looping through the batches of customers - described in tab 2 above.

2 - Create batch loop

Loop batches is

loop-batches

The operation is set as 'Loop List' and the List value is set as a $.steps.stripe-1.data to grab each customer object from Stripe.

The first step in the batch loop is a boolean:

last-customer-in-batch

It checks the second loop connector to see if it is the last customer in the batch:

last-customer-in-batch-operation

If it is the last customer in the batch then Set Last ID saves the ID of the customer:

set-last-id

It uses the 'Set Value' operation and sets the stripe_last_customer value as the $.steps.loop-2.value.id json path:

set-last-id-operation

We can then move on to checking Stripe if there is another batch of customers - described in above.

3 - Check Stripe for more customers

Outside of the batch loop (back in the main loop) we add the More customers? boolean:

more-customers

This checks the List Stripe customers step to see if there are more customers to list. It does so using the Stripe has_more property, via the $.steps.stripe-1.has_more jsonpath:

more-customers-operation

If the result is True then the main loop will continue to produce another batch.

If it is False then the Break Loop step:

break-loop

is set to break the main loop (loop-1) and end the workflow run:

break-loop-operation

Once you have been through all the above tabs which explain the pagination setup, we can move on!

Find and update Salesforce contracts with missing Stripe ID

Now we just need to add the steps which will find the relevant Salesforce contracts and update their Stripe ID, if it is not found.

This can be done by adding some steps to the beginning of the customer loop, just before the Last customer in batch? step:

Click through the tabs for the details of each of the added steps:

1 - Find contract by email

Find contract by email is a Salesforce step:

find-contract-by-email-1

It uses the 'Find records' operation and specifies the Record type and fields to be returned:

find-contract-by-email-operation-1

The key part here is to scroll down and set the Conditions so that we find the Contract which matches the email address that was stored in Stripe and has been fed to the loop connector:

find-contract-by-email-operation-2

Then we can check to see if this contract is missing a Stripe ID - explained in tab 2 above.

2 - Check for missing Stripe ID

Missing Stripe ID? is a boolean step:

missing-stripe-id

It checks to see if the Stripe ID for this contract is null:

missing-stripe-id-operation

Note that the quite complicated $.steps.salesforce-1.records[0].Stripe_ID__c jsonpath value above can be obtained by dragging the 'connector snake' from 1st Value to the Find contract by email step and then selecting the Stripe ID output:

grab-stripe-id

If this is true then we can update the contract - explained in Tab 3.

3 - Update contract with Stripe ID

Update contract is a Salesforce step

update-contract-1

It updates the contract with the missing Stripe ID as follows:

update-contract-operation

'Record ID' uses the $.steps.salesforce-1.records[0].Id jsonpath to pick up the ID from the Find contract by email step, to specify the Salesforce ID of the contract which needs updated.

While 'Stripe ID' uses the $.steps.loop-1.value.id jsonpath to get the Stripe ID which was fed by the loop connector.

The complete workflow

Having completed the steps set out in all the tabs above, your complete workflow should now look like this:

Note on testing

In order to test a setup such as this, it would be standard to use a manual trigger instead of a scheduled trigger, so you can click to run at any time.

When a run of the workflow has been triggered you can click on the 'Debug' tab to check if it was successful and view the input/output of each step in order to troubleshoot.