Understanding training options for Spotter

The reference questions are designed in a way to provide the exact same answer when users ask the same question. We recommend adding the most commonly asked questions to the reference questions so that business users get a verified answer that is curated by an analyst.
For the questions specified under reference questions, Spotter also remembers the visualization settings set by the analyst. Spotter remembers the chart type selected and the axis settings of the visualizations (other visualization settings may not be retained).
Hence, one of the direct benefits of providing reference questions is to make sure Spotter provides an accurate response for commonly asked questions.

An indirect benefit of providing reference questions is that it helps in warming up our usage-based ranking systems. The reference questions help our system learn which columns must be selected in answering questions.
We have observed that for newly created datasets where the usage-based ranking system isn’t warmed up, Spotter might struggle to pick the correct columns if there are similar-sounding column names. Adding reference questions trains Spotter to understand which columns should be used for answering questions.
We recommend that you add reference questions that use the majority of the columns present in your Model/Worksheet. Ensuring a good column coverage ensures that Spotter understands when and how to use the columns available in your Model/Worksheet.

Spotter provides the answer you have curated when a user asks the same question. Hence, we recommend that you add the most commonly-asked questions by the business user as reference questions so that users get a verified answer every time they ask these questions.

You should think of reference questions as trained question templates. For example, once you have provided “What is the sales for east this month ?” as a reference question, then Spotter has the ability to generalize the learnings from this question to answer similar questions like “What is the sales for west this month ?” , “What are the sales for north this year ?” , “What is the quantity sold for east this week?”, etc. (Please note that the visualization settings provided in query feedback may not extend to these questions)

As mentioned above, Spotter learns how to use columns for analysis when they are used in reference questions. Hence, we recommend that you provide enough reference questions so that all the important columns in your Model/Worksheets are represented in the reference questions.
For newly created datasets with no Liveboards, the reference questions play an extremely critical role in training Spotter.

In order to answer questions, you sometimes need to generate calculated fields or metrics. You can use the business term to teach the Spotter on how to calculate specific metrics for your business. Spotter learns how you calculate specific metrics and it has the ability to extrapolate these learnings to different scenarios.

Often, the same value might be referred to with different names. We recommend the use of business terms to define the synonyms for value. While Spotter has the ability to apply semantic matches based on publicly-known information, there are always some cases where you want to define synonyms or acronyms which are only applicable to your own business and cannot be guessed by a large language model.

Business terms can be used to define filters that should be applied for specific cases. For example, the active clusters example demonstrated how you can use the business term to define specific filters to be applied while calculating things.

The first thing you should evaluate is whether adding the business terms will lead to a meaningful addition to Spotter knowledge about your data. Anything which is obvious or common knowledge can already be guessed by the LLM models as they are trained on large corpus of public datasets. Hence, try to avoid adding the definition for commonly available terms in business terms, as the LLM will already provide correct responses.

Business terms, once defined, are considered to apply uniformly across all the question contexts. You must only use business terms for adding analytical definitions for the terms which hold the same meaning in all the different contexts it can be used for that Model/Worksheet.
You should avoid addition of business terms just to train Spotter on handling date intents better because in most datasets there may be more than one date column, and once you have used business terms to define date intent (e.g. this year → created_date.this year), then Spotter might start using the same definition when this year is used in a different context.

You should only accept the business terms whose analytical definition (represented by ThoughtSpot keyword-based search tokens) is correct.