Plots

​

Sources

• PV_STMT_005: the plan is too deep to decode. This error blocks publication, and Prepare refuses the same plan with plot_plan_failed.
• PV_STMT_006: the plan nests 80 levels or more. This advisory names the depth and the limit. It does not block publication.

​

Structure

import type { BasePerception, PlotContract } from 'parable/foundation';
import type { Generic } from '@psgen/scalar-lib';

export interface Activity {
  readonly category: Generic.Text | null;
  readonly hours: Generic.Float64;
}

export interface Perception extends BasePerception<Activity | null, {
  activityFacts: PlotContract<readonly [], readonly Activity[]>;
}> {
  readonly panes: {};
}

​

Optional Artifact inputs

import type { BasePerception, PlotContract } from 'parable/foundation';
import type { Generic } from '@psgen/scalar-lib';

export interface CensusRow {
	readonly email: Generic.Text | null;
	readonly department: Generic.Text | null;
}

export interface Perception extends BasePerception<CensusRow | null, {
	census: PlotContract<readonly [], readonly CensusRow[], {
		readonly 'artifacts.public.census': CensusRow;
	}>;
}> {
	readonly panes: {};
}

import type { BasePerception, PlotContract } from 'parable/foundation';
import type { Generic, Temporal } from '@psgen/scalar-lib';

export interface ActorFileRow {
	readonly 'Full Name': Generic.Text | null;
	readonly Title: Generic.Text | null;
	readonly Team: Generic.Text | null;
	readonly 'Start Date': Temporal.Date | null;
	readonly Active: Generic.Boolean | null;
	readonly Score: Generic.Float64 | null;
}

export interface Actor {
	readonly name: Generic.Text | null;
	readonly team: Generic.Text | null;
	readonly score: Generic.Float64 | null;
}

export interface Perception extends BasePerception<Actor | null, {
	actors: PlotContract<readonly [], readonly Actor[], {
		readonly 'artifacts.parable_perceptions.actors': ActorFileRow;
	}>;
}> {
	readonly panes: {};
}

SELECT "Full Name" AS name, "Team" AS team, "Score" AS score
FROM artifacts.parable_perceptions.actors
WHERE "Active"

​

Collect nested values before sorting

SELECT
  meeting_key,
  array_sort(
    flatten(array_agg(provider_refs)),
    'ASC',
    'NULLS LAST'
  ) AS provider_refs
FROM meeting_rows
GROUP BY meeting_key

WITH numbered AS (
  SELECT
    meeting_key,
    provider_refs,
    row_number() OVER (
      PARTITION BY meeting_key
      ORDER BY provider_rank ASC NULLS LAST, row_key ASC
    ) AS position
  FROM meeting_rows
)
SELECT
  meeting_key,
  array_sort(array_agg(named_struct(
    'position', position,
    'provider_refs', provider_refs
  ))) AS ordered_records
FROM numbered
GROUP BY meeting_key

​

Plot results in the browser

• One reader, one access state. Each time a Page resolves its Panes, the server returns a key that it derives from the reader’s whole access state on the Parable: the reader, their seat, the Policies of the Parable’s current Release, the switch rows of its filter Policies, the reader’s Workspace Roles and their permissions, and the Workspace’s Provider Policies and source permissions switches. The browser encrypts each stored result under this key and holds the key in page memory only. When any of these inputs changes, the key changes, and a result stored under the old key can neither answer nor be decrypted.
• What is never stored. A View as preview, an author’s SQL preview, a read that failed or was refused, and a read through a Platform Parable stay in tab memory only. A read through a Platform Parable answers there only by the catch-up rule below: the key covers this Parable’s access, not the access to the Parables it reads. A reader with All data access, a staff member, an impersonation, a Workspace that enforces source permissions, and a Parable whose filter Policy reads data through in_plot get no key, so nothing of theirs is stored.
• A chosen Refresh. Its rows never change, so its result never reads again. Each Refresh in the menu keeps its own results: a return to yesterday’s Refresh after today’s arrived is instant. When a Refresh leaves the menu, its results go.
• Latest, and a Plot that reads a stored output. A Pipeline, or a Plot that reads one, answers from the store until Latest moves to a newer Refresh. Then its results go and it reads from the server.
• Latest, and a live Plot. It answers at once. When its rows are older than a minute, one read goes to the server in the background, after the Page is drawn. When that read finds other rows, the Pane takes them in place: no reload, no remount, and the chart’s marks move to their new values. The same rows in another order are no change.
• A slow store. On Latest, a read waits for the store while the store looks for the result. When the store has not found it after 60 ms, as on a page that is still starting, the read also goes to the server, and the first answer shows. When the store found the result, the read waits for the store and sends nothing to the server. A chosen Refresh never reads from the server beside the store.
• A Page that opens again. While the Page resolves its Panes, the browser reads the results that the Page used the last time. When the key arrives, it decrypts them at once, so the Page’s first reads find them ready.
• Sign-out. A logout deletes the store on that Workspace’s host: the logout page opens on that host and deletes the store before it goes to the sign-in page. A session that only expired keeps the store locked: no key, so nothing answers, until the same person signs in again.
• Another person in the same browser. When a page of another person opens on the host, on any route, every open tab of the host forgets the keys and closes the store. The store empties itself before its next first read.
• A tab that stays open. The server checks the reader’s access state again when the tab becomes visible, and in the background on a read when the last check is a minute old. When the last check is five minutes old, the read waits for a new check before it answers. A background read that the server refuses as signed out (401) locks the store in every tab and clears the Pages. A background read that the server refuses for access (403) deletes that result, and its Pane reads again. A Pane resolution that the server refuses deletes every result kept under that access state and clears the Page.

• When Latest moves to a newer Refresh, every result that read a stored output on Latest is dropped, whether the move shows while the Pages read Latest, while they read a chosen Refresh, or on a return to a Parable that was closed when it moved. A live Plot’s result stays and reads again in the background.
• When anyone saves or deletes a row of a Pane, the reader included, in another tab, another session or through Ponder, the open Pages update that Pane’s rows in place, and only the results of the Plots that read the Pane by name, or read such a Plot, are dropped and read again, in tab memory and in the store. A chosen Refresh’s results of live Plots stay: they read rows as of their run. Where the platform cannot tell which Panes a Plot reads, every result is dropped.
• When a seat on the Parable, a row that a Policy reads, a Provider Policy or a source permissions switch changes, the reader’s access is decided again: the tab drops what it kept in memory, and every open Page resolves its Panes again before it reads. A narrowed Policy returns fewer rows rather than refusing the read. The stored results are not deleted: the change gives a new key, so they never answer again.
• When a Plot read is refused, the Page resolves its Panes again. The Panes read again with no rows only when the reader’s access changed. Any other refusal drops every result the tab kept in memory.
• A tab hears row, seat, Provider Policy and source permissions changes for a Parable only while that Parable is open in it. A result with a key answers at once on a return, because the Page’s Pane resolution confirmed the reader’s access state again. A result with no key answers only once the tab has heard every change it missed meanwhile. A Pane that asks before then waits for it, up to two seconds, and then reads from the server. When too much changed to catch up on, every result kept in that Workspace is dropped, and the stored Latest results of that Workspace too.

​

Plots read as views

• Columns. A view serves its output schema, the columns a stored table of the Plot would serve: the schema publication planned for its SQL, or, for SQL a draft added or changed, the schema the draft’s last save derived, with every column nullable. If its SQL no longer returns those columns with those types, the read refuses until the Plot is planned again, or for a draft, until a save derives its schema again. A value of the declared type in another storage is served as declared: text that SQL functions return as Utf8View or LargeUtf8 (CAST(… AS VARCHAR), string_agg) reads as Utf8, and a timestamp in another unit in the declared unit. A finer timestamp, such as the nanoseconds CAST(x AS TIMESTAMP) returns, is rounded down, so an instant never moves into the next second or day; inside a list or struct it refuses the read.
• Definition. A Pane read and a Plot preview read each view from the definition they show: a draft reads the draft’s SQL, so an author who previews, in a Patch, a Plot that unions mapping Plots sees their draft mapping Plots, including Plots only the draft has, and a Release reads that Release’s SQL. SQL outside a Parable reads the Release the Workspace serves.
• Views of views. A view can read another view. Views that read each other are refused: reading one answers Plot views form a cycle: … reads …. Publication refuses them with a 409 that names the cycle, such as Plot a reads Plot b reads Plot a. Change one Plot so these Plots do not read each other. Publish and automatic Delivery find the cycle after planning fails: they resolve the current SQL and name only the Plots in a closed cycle. A Plot that only reads a cycle is not named. Old saved schemas, reference pins, SQL comments, and string values cannot prove a cycle. Successful publication adds no validation request for this diagnosis.
• Report runs. A Pipeline may read views of its own Parable. The run computes each view inside its own plan, so the run’s pinned inputs are the Provider tables and stored Plots under the views. A view of the run’s Parable must come from the Release the run captured; a run captured before a later publication of the view is refused, and running the Report again uses the new Release. A run whose view the Workspace catalog serves from an earlier Release is refused with that cause instead: the catalog has not caught up with the captured Release, or that Release was retracted. A run a few minutes later reads the Release the Workspace serves.

• plots.activityFacts() is a hook exposing status, rows, refreshing, rerun and breakCache. Pane code draws three states: reading (status is running with no rows yet), empty (ready with no rows) and rows. A read that fails never reaches it: the host shows the failure in the Pane’s place, with Retry (below).
• plotApi.run('activityFacts') runs from an action handler.
• result.meta identifies the data that answered the request, including a resolved Report when relevant.

const facts = props.plots.activityFacts();
if (facts.status === 'running' && facts.rows.length === 0) return <p>Reading activity…</p>;
if (facts.rows.length === 0) return <p>No activity in this reading.</p>;

​

Plots of a Platform Parable’s tab read the Parable beside it

SELECT change, kind, title FROM subject.changes ORDER BY kind, title

​

Incremental Plots

​

Split an interval

• through a GROUP BY that includes the key,
• through a window whose PARTITION BY includes the key,
• through a join, to each side whose rows the join does not fill with NULL.

​

Foreign keys and graph edges

1. A column has one foreign key or none.
2. references splits at its first dot. A catalog slug never holds a dot, and a column name can. The part before the dot is the catalog slug of a Plot of the same Parable. The part after it is an output column of that Plot.
3. The foreign-key column and the column it references have the same type: the same primitive and the same scalar. Nullability does not count.
4. graphEdge.from and graphEdge.to are two different columns, and each one has a foreign key.
5. Names compare exactly as the catalog spells them: the catalog slug, and the column name as the Plot’s output schema holds it.

​

Where a Pipeline keeps its output

​

Services

await plotApi.preview('activityFacts', { sql: compiled.sql, parameters: [] });

​

Save a query for the Workspace

​

Nullable Conditions

WITH input AS (
  SELECT g, NULLIF(x > 0, true) AS enabled
  FROM (VALUES ('a', 1), ('a', 2)) AS t(g, x)
)
SELECT
  g,
  count(*) FILTER (WHERE enabled) AS bare_count,
  count(*) FILTER (WHERE enabled IS TRUE) AS safe_count
FROM input
GROUP BY g;

​

Large results

SELECT subcategory_name AS category, SUM(hours_estimated) AS hours
FROM parables.teamtimespend.time_spend_facts
WHERE date_day >= $1
GROUP BY subcategory_name
ORDER BY hours DESC

SELECT COUNT(*) AS actions, SUM(hours) AS hours
FROM parables.teamtimespend.action_hours
WHERE person_key = $1

-- One page of a person's Actions, newest first, after the last row shown.
-- $2 and $3 are that row's day and action_key, both NULL for the first page.
SELECT action_key, day, title, hours
FROM parables.teamtimespend.action_hours
WHERE person_key = $1
  AND ($2 IS NULL OR day < $2 OR (day = $2 AND action_key < $3))
ORDER BY day DESC, action_key DESC
LIMIT 200

SELECT action_key, day, title, hours
FROM parables.teamtimespend.action_hours
WHERE person_key = $1
ORDER BY day DESC, action_key DESC
LIMIT 50 OFFSET $2

​

Storage

​

SQL checks and failures

​

Incremental Plot tables

​

Epochs

​

Maintenance

​

Dependency graph

• Cycles. The check finds a Plot that reads itself and Plots that read each other in a cycle. Publication refuses either: parable: plot dependency cycle: Plot … depends on itself, or parable: plot dependency cycle through Plot … in Parable ….
• Report members. For each member, capture lists the members it reads, directly or through views. A member that reads itself, directly or through a view, is refused first, then members that read each other in a cycle. The refusal names the members: parable: plot dependency cycle: Hours by team reads Logged hours reads Hours by team, or parable: plot dependency cycle: Hours by team reads itself. A scheduled run refused this way records the refusal and its reason on the plan, and the occurrence is not tried again.
• Report runs. A Report run runs a member after every member it reads.
• Downstream. The Plots downstream of a set of Plots are all the Plots that read one of them, directly or through other Plots. A Plot of the set is among them only when it reads another Plot of the set.

​

Output schema diff

​

Its Panes’ Plots

​

Run history and status

• Results shows the selected reading. On Latest it holds Preview’s rows with their count and time. Above them it lists what Prepare would block this Plot for, if anything: Preview runs Prepare’s own checks on the edited SQL, judged by how the Plot runs (a Pane or a Report), and still shows the rows. Preview in the Refresh menu is a different thing: a stored Report run on a sample of the data (Reports), read like any other run. On a stored Report run it holds that run’s own rows, read without running the Plot. A run that is queued, running, paused at its budget, failed, cancelled or skipped shows that status instead of rows, and a failed run shows its message. A Plot that finished inside a run that is still moving, or that did not complete, has no readable rows either: only a complete run is readable. When the selected run does not include the Plot, because the Plot is not a Pipeline or the run has no result for it, Results says so and reads nothing. Preview runs on Latest only, so it is unavailable while a stored run is selected.
• History lists every Refresh (a stored Report run), newest first. Each row is named by when the run happened, in your time zone and clock: 12:31pm ET today, Sep 29, 9:00am ET before, with the whole moment on hover. Results names a stored run the same way, as in Refresh Sep 29, 9:00am ET · 1,204 rows. Each row shows this Plot’s status, whether the run was Manual, Scheduled, fired after its sources (After Okta synced) or a Refresh by Parable (Parable started it to repair data after a platform update), Refresh everything when it was forced, and for which Report, Started by (the person who pressed Refresh, or Schedule, Sources or Parable), when the Plot started and last changed, how long it ran, how long it waited to start, its row count, its spend in US dollars and paid calls against the caps captured when the run started, and the catalog address with the run’s short id. A row count the reader may not see reads Hidden, never 0. Choosing a row selects that Refresh in the Refresh menu and the address (refresh); choosing one in the menu selects its row. See Run a Report now.
• On a Workspace’s line, the History of an incremental Pipeline starts with Intervals (PARABLE-8221). Each epoch of its stable table shows how many of its intervals are Pending, Running, Complete and Failed. A Building epoch also shows how many are complete, because Reports read the Current epoch until every interval of the new one is complete (Epochs). Below the epochs, the newest 200 intervals each show their state, when the last run started and how long it took, the rows it wrote, when a waiting interval runs next, the attempt, and the message and code of a failed run. The list reads again every 15 seconds while an interval waits or runs or an epoch builds. A row count reads Hidden when a filter Policy or a Provider Policy narrows what you read, because the interval runs read as the line’s Accountable seat.
• When every interval of an incremental Pipeline waits to run again (a restate, PARABLE-8260), Every interval runs again shows above the epochs. It says why: a source the Pipeline does not read by time changed, a shared setting or Pane row it reads changed, or both. It shows the estimated cost of the paid AI calls against the Workspace’s Refresh cost threshold, and when the restate was held. Estimating means the estimate still runs. Held means the estimate is above the threshold, or it ended without an answer. Approve shows only to a person with the budget override permission and the Primary seat, as Raise budget does. It asks you to confirm the estimated cost first. Approved shows until the next sweep, within about a minute, runs every interval again. If the restate changed before your approval, the list reads again and shows it as it stands.
• Backfill, beside Intervals, runs every interval of a range of days again, such as the history from before the Pipeline became incremental (PARABLE-8691). It spends the Refresh budget, so it shows only to a person with the budget override permission and the Primary seat, as Raise budget does. You choose From and To in your own time zone, and the dialog counts the intervals before you confirm. It also estimates the paid AI calls of those intervals and shows the cost against the Workspace’s Refresh cost threshold (PARABLE-8741). Backfill waits for the estimate while it counts. No answer means that the estimate ended without an answer, and Try again starts it again. The real cost can be a little higher than the estimate, because intervals that run at the same time can each pay for one call. The intervals run in parallel, up to half of the Workspace’s query-run slots, and their commits take turns, so readers see part of the history until every interval completes. A range ends by now, and one backfill marks at most 100,000 interval runs. One Pipeline takes at most 20 backfills in 24 hours (PARABLE-8871). After that, the dialog says when you can backfill it again, before you confirm. Ponder suggests the range and points you to Backfill. It does not start one.
• Details lists the catalog address parables.{parable}.{plot}, whether Report runs include the Plot, its published caps and its output columns.

​

Security

• A Report run writes a Pipeline’s output. The run holds the Plot’s contract and checks every row against it before it writes. A run whose rows break the contract writes nothing and fails with QR-QU-011, and the Plot keeps the output of its last successful run.
• The read of those rows makes the same check, whoever wrote them. It checks every value in each column the query reads whose contract says more than the column’s type, such as a literal, a scalar format, a bound, a required value, or a Date’s precision. This includes columns used by Policies, filters, joins, grouping and aggregation. A column the query never reads is not read from storage or checked, so COUNT(*) reads no column.

​

Stability