Unexpected answers in Spotter

What to do when you don’t get an expected answer

Spotter may not generate the answers you are expecting in the first attempt because of the following reasons: Natural language inputs can be ambiguous and the output generated can be subjected to interpretation bias Spotter is powered by LLM models which are probabilistic

However, we have designed the Spotter experience to ensure that you can verify the answers generated by Spotter and, if needed, you can also guide the system in the correct direction whenever it doesn’t provide the expected response.

Let’s take a look at some of the options available to you to get to the expected answers.

Provide instructions in follow-up prompts

Let’s take the example: “What are the monthly sales for jackets in the southern region this year?”

spotter verify query

Spotter didn’t include the ‘southwest’ region while generating the answer. You can fix this easily by providing the instructions to “include the southwest region” in the next message. Providing instructions in the follow-up prompts is one of the easiest ways to reach the expected answer.

spotter verify southwest

A few common scenarios where you can use this are: You can nudge Spotter to use a different column or value while generating the answer (for example, use transaction date instead of delivery date, or include southwest region in southern regions, or give me the average sales instead of total sale, etc.). You can provide more details on how you want Spotter to perform a mathematical calculation (for example, calculate the percentage contribution in terms of units sold instead of sales). You can also ask Spotter to remember context from chat history (for example, retain the filer for item types).

Please note that all follow-up questions in Spotter are assumed to be a follow-up on the latest answer generated in the conversation. If you want to ask a fresh question, you must use the option to reset the chat.

Interactive charts to apply modifications (such as Drill down)

Spotter’s answers are interactive, and you can drill down on a specific point in the chart to continue exploring. For example, let’s say you ask “sales by category this month”, and Spotter generates the following answer:

spotter bar chart

You can select “Jackets” on the chart and use Drill down to further break down the sales for jackets by region. Spotter updates the answer to show the results. Now you can continue to explore this by asking follow-up questions on this newly-generated answer.

spotter bar drill down

You can use a mix of natural language instructions and point and click to ask questions from the data and generate insights.

Interactive chart functionality is only available on the most recent answer in the conversation.

Use the Edit Answer flow to take control

Spotter allows you to take control and edit the answer yourself using the Search Data experience. You can click on the Edit button beside the query tokens to enter edit mode. The Search Data experience provides a simple and intuitive way for you to build a query.

Understand errors in Spotter

Below are a list of some common errors in Spotter:

Spotter sometimes treats a follow-up question as a new question

Spotter is designed to treat all new questions as a follow-up to the previous question. Consider the following example, where you first ask a question: What is the sales by item type this year?, and then ask a follow-up: What is the quantity sold for the east region?
In most cases, Spotter retains the context from the previous question (filter on this year and group by item type) and will also perform the modifications asked (show quantity sold instead of sales, and add a filter on `east region).

spotter follow up

Note that the follow-up question here can also be treated as an independent question which shows the total quantity sold for the east region (ignoring the grouping on item type and filter for this year). In natural language questions, there can be ambiguity on whether you are asking a new question or whether it is a follow-up question. Spotter has a bias in favor of treating questions as a follow-up to the previous answer, but there are some phrasings where Spotter may treat the next input as a new question. Work-arounds::: Phrase the follow-up question to help Spotter understand it as a follow-up;; Instead of asking “show for east region”, you can ask “show this for east region” to eliminate ambiguity. Instead of asking “give me quantity sold”, you can say “show the above answer with quantity sold”. Specify which parts of the answer to retain;; Ask a follow-up question like “Retain the grouping by item type and filter on date column”. You can also use the edit user input option to edit the answer.

Spotter did not generate a complete answer

For certain complex questions, Spotter may not generate a complete answer. In such cases, we suggest you break the complex question into multiple steps. For example, Spotter may have difficulty answering the question: “What percentage of this year’s sales is contributed by top 5 most selling products of last year?”

spotter complex

The answer generated does not look complete. To get the answer you’re searching for, you can first ask “What were the top 5 most selling products of last year?” Then you can ask a follow-up question: “What percentage of this year’s sale is contributed by these products?”

“There was a problem generating this Answer” error

Note that if you receive an error message saying there was a problem generating the answer, it’s likely that the LLM took longer than expected to generate a response, which triggered a timeout. In most cases, using the Try Again option should generate an answer. If a large group on the same cluster is trying Spotter at once, you may hit the rate limit for the model provider. We recommend trying Spotter at different times.

“No data found for this query” error

You receive this message when the filters in your query are such that no row in the table matches the results. For example, in the sample table shown, if you ask “What are the products with the highest sales in January 2010?”, you receive this error as there are no rows in the table for data in 2010. The problem can arise because of two reasons:

  • The filter conditions in the question asked didn’t have any matches.

  • One of the filter conditions generated by Spotter was incorrect. You can use the Edit answer option to review the filters applied in the answer and modify them to get the correct answer.

Was this page helpful?
×