Search Everything

search_everything finds records in the Provider Pool by what their text says: calls where a customer raised pricing, issues about an SSO login loop. search_available_data finds where data lives, the tables and columns. Use that first when you do not yet know the table. Use search_everything when the question is about the contents of records. It searches only the Provider Pool, and only text fields a Provider has flagged for this search. It returns one row per matching source record, never one per passage. A record a Provider Policy hides from you is left out, and its text never appears in a snippet. A session that cannot query the Provider Pool does not see its records here. An empty result is not proof that nothing matches. A missing model policy, a policy that admits no embedding model for this search, or an index that is missing or still building all return no rows, the same as a search that found nothing. Try the question again later, or use search_available_data and read the table directly. Each matching record is read again from its source table before it is returned. When one source table takes too long to read, its records are left out of that search, and records from the other tables still return. When no source table can be read in time, the search fails with DEADLINE_EXCEEDED and the error code QL-TO-004 (ROW_SEARCH_REREAD_TIMEOUT). The message names the time limit. Run the search again. If it fails again, narrow the scope to fewer tables, such as one Provider. Preview a Plot answers it as HTTP 504 with the same code and message:

search_everything: no source table returned its matching records within the 2s re-read deadline. Run the search again. If it fails again, narrow the scope to fewer tables, for example providers.jiracloud.*

The function runs only in interactive queries, such as Preview a Plot. A Plot cannot call it: saving one records an error, and the Plot cannot publish until the call is removed. Search, then write the Plot’s SQL against the source table.

SELECT table_ref, row_key, column_name, snippet, score, matched_chunks
FROM search_everything('customer raised pricing concerns')
ORDER BY score DESC
LIMIT 20

Arguments

The function takes up to three arguments, by position only: search_everything(query [, scope [, limit]]). Named arguments (query => ...) are refused.

Argument	Meaning
`query`	Plain text describing the records you want.
`scope`	Optional pattern over Provider tables, the same shape `search_available_data` uses, and always starting with `providers`. `NULL` or `''` searches every Provider table the session can read.
`limit`	Optional number of records to return, from 1 to 100. `NULL` returns 20.

A scope such as providers.jiracloud.* keeps the search to one Provider. providers.jiracloud.issues.body keeps it to one field. The Workspace Pool is not searched; a scope starting with workspace is refused. Arguments the function cannot search are refused before anything runs: a scope outside the Provider Pool, a limit outside 1 to 100 or not a whole number, an empty query, or a query or scope that is not a text literal. The refusal is INVALID_ARGUMENT with the error code QL-PL-018, as in search_available_data, and its message names the argument and what to use instead. Preview a Plot answers it as HTTP 400 with the same code and message:

search_everything: scope must start with providers: search_everything searches the Provider Pool only, for example providers.jiracloud.*

Results

Each row is one source record, best passage first. score is that passage’s similarity, from 0 to 1. It is not a sum across passages, so a long record does not outrank a short one that matches more closely. matched_chunks is how many of the record’s passages were among the candidates and still match the text stored for them. A passage matches when the record, read now, still yields the same passage. Some fields are searched with context from the same record, such as an issue’s title before its body, a call transcript is searched as windows of speaker turns, and a Notion page is searched as its blocks under its title. A record whose field, context or turns changed after it was indexed is left out until the index catches up.

Column	Meaning
`table_ref`	The source table, such as `providers.jiracloud.issues`.
`row_key`	The record’s key as JSON. Join back to the source table with it.
`column_name`	The flagged field of the best passage, such as `body`. For a Notion page it is `blocks`: the page’s text is its blocks.
`span`	Where that passage sits in the field, as JSON character offsets. For a list such as a call transcript it also names the first and last item (`items`) and their times (`time_start`, `time_end`), in the source’s unit.
`snippet`	The start of the best passage, at most a few hundred characters, taken from the record as you are allowed to read it. It holds the passage itself, not the context searched with it. The snippet and the search leave out the HTML comments of a Markdown or HTML field: the Provider does not show them.
`score`	The best passage’s similarity.
`matched_chunks`	How many of the record’s passages matched.
`provider`	The Provider’s name in the catalog, the schema of `table_ref`. `NULL` when the table name has no schema.

The snippet is the beginning of the best passage, not the whole record. span names the rest. Read the full record from the source table.

Join back

row_key is a JSON object of the source table’s key columns. A single text key joins like this:

WITH found AS (
  SELECT row_key, snippet, score
  FROM search_everything('SSO login loop', 'providers.jiracloud.*', 20)
)
SELECT issues.key, issues.summary, found.snippet, found.score
FROM found
JOIN providers.jiracloud.issues AS issues
  ON issues.key = json_get_str(found.row_key, 'key')
ORDER BY found.score DESC

row_key is JSON text. json_get_str reads a text key; json_get_int reads a numeric one. Use the names inside row_key. A record with more than one key column matches each of them. The join reads the source table under the same Provider Policies as any other query, so a record the search omitted stays omitted.

Admin

​Arguments

​Results

​Join back

Arguments

Results

Join back