Survey Diagnostics Funnel

Understand the estimated population of end users at particular steps in Sprig’s survey selection and contact engine

Survey Diagnostics Funnel Overview

The Survey Diagnostics Funnel allows you to understand the estimated population of end users at particular steps in Sprig’s survey selection and contact engine. The funnel will enable study owners to self diagnose why they may not be getting as many responses as expected and troubleshoot survey design.

Sprig uses an event-driven model. Each time a user sends an event to our backend, we check to see if there is a potential survey or set of surveys to present to the user. This process is documented in a surveys engagement funnel located in the Responses widget. Here’s an example of what a survey engagement funnel may look like:

It’s important to note that not all of these steps will be present in each funnel.

In particular, the following steps may not always be shown, as these will be dependent on the configuration of the study in question as well as other studies on the account:

• Matched Filters
• Included in Sampling
• Selected for this Study

Survey selection works in the following steps:

1. A single event from a single visitor is received by the Sprig backend
2. We select the entire universe of in-progress surveys for that environment
3. The survey selector code begins to filter this universe down by filtering out surveys that:
   1. Have the wrong SDK version
      Step: “Had required SDK”
   2. Do not match resurvey criteria
      Step: “Met resurvey criteria”
   3. Have a different event trigger than the one the user triggered
      Step: “Triggered event”
   4. That the user has already seen (based on resurvey criteria)
      Step: “Hadn’t seen this study yet”
   5. That the user doesn’t match filters and attributes for
      Step: “Matched Filters”
   6. Need to be sampled or throttled
      Step: “Included in sampling”
      Step: “Selected for this study”
4. The result may be:
   1. A single survey that is then served to the user
   2. A randomized array of surveys, from which the first will be served to the user

The last four steps in the engagement funnel are:

1. Received Study
2. Viewed Study
3. Started Responding
4. Completed Study

These steps happen after the universe of surveys has been filtered down to the one particular survey this user should see, and that survey has been sent to their SDK instance.

Using the Diagnostics Funnel

1. For any active or completed Web or Mobile Survey, navigate to the Survey Results page.
2. Select Inspect to launch the Inspect Response Rate modal.

1. The debugger will show you a list of checkpoints and the estimated population of end users at each checkpoint.

Note on Limitations:

• Survey Diagnostics Funnel will be unavailable for surveys completed before October 2023.
• Audience population estimations are limited to the last 30 days of an active survey.
• If any edit is made to an active study, including copy updates or targeting criteria, the diagnostics funnel will reset and begin collecting data from time last updated.

Checkpoint Steps

Checkpoint steps and corrective measures are detailed below:

*For mobile studies, if SDK update is bundled with an app update, end users will need to update your native app in order to qualify for a study with a minimum SDK requirement.

FAQs

How is overall response rate calculated?

The response rate graph is calculated by dividing the number of responded surveys by the number of seen surveys within a day based in Universal Time Coordinated (UTC).

How do event properties get applied?

When targeting select users based on event properties, these event properties are applied during step 5, “Matched filters”. The “Triggered Event” step will evaluate the event, and then “Matched filters” will evaluate the event property filters applied.