Temporal types
Contract-derived temporal scalars use native Arrow types:
A
Temporal.DateTime value is an instant written as its UTC wall-clock time.
The Arrow type carries no timezone because the Arrow Flight SQL JDBC driver
cannot read a timezone-aware timestamp. Every catalog serves it this way:
Provider tables, workspace.* tables, and Plot tables.
String-like semantic scalars remain Arrow Utf8 and carry optional field
metadata. Other tables can use the Arrow types of their stored data.
JSON values
AGeneric.JSON column holds JSON text. Read inside it, or inside any other
JSON text, with the JSON functions: json_get_str, json_get_int,
json_get_float, json_get_bool, json_get_json, json_get_array, json_get,
json_as_text, json_contains, json_length and json_object_keys. Each
takes the JSON value, then the keys or array indexes of the path, for example
json_get_str(payload, 'owner', 'email'). A missing path reads NULL.
A Provider’s custom fields are also served as their own columns and through
the custom_fields and raw_unknown_fields maps. A filter on a JSON value
runs row by row, so it cannot skip data the way a filter on a typed column
can.
Semantic scalar metadata
When a field has a known semantic scalar, its Arrow metadata can include:
Provider tables and the
workspace.{connector}_ingestion_config and
workspace.{connector}_authentication_config tables put the same keys on a
nested field as on a top-level column. A nested field is a field inside a
struct, list, or map column, for example description inside the fields
struct of Jira Data Center issues. A nested Text.Long field therefore has
parable.scalar.large_text set to true, the same as a top-level Text.Long
column. A statement that selects the struct, list, or map column keeps these
keys on its nested fields. Plot and artifact tables put only
parable.scalar.canonical_name, and a declared parable.temporal.format, on
their fields.
For example, columns carrying Contact.Email can be joined across Providers
and to workspace.users.email when they represent the same person.
Treat these keys as optional. An omitted key means unknown or not computed; it
does not mean an empty value, false, or zero.
Table annotations
Table schemas fromGetTables can also describe logical identity and current
annotations. A statement’s result schema does not carry them: its columns keep
their field metadata, and the schema itself carries only the
statement warnings.
A query can name a table of a Provider the Workspace has not connected, or of
a stream it has not enabled. That table has no rows, its
health_state is
not_connected, and its health_reason names the Provider or the stream. See
Provider Pool.
A cold GetTables waits up to one second for the first annotation load, so
watermarks, health, and row counts that are already known are on that
response. A slower load does not hold the response: it omits annotations, the
load finishes in the background, and the next response includes them. While
Parable refreshes annotations, a response omits them. The next response
includes the new values. Queries do not wait for annotations.
Field metadata can mark internal columns with parable.annotation.hidden,
mark the row-state columns Parable adds to Provider tables (dt and the
_parable_* load and deletion columns) with parable.annotation.system, or
warn that a representation limits data skipping with
parable.annotation.poor_pruning. System columns stay queryable.
Statement warnings
A statement’s result schema can carry warnings about the statement itself:
The schema a prepared statement returns before it runs names only the
Providers and streams its SQL names directly. The schema of the result stream
also names those read through views.
Generic clients can ignore Parable metadata and still decode the Arrow schema.
Applications that use it should read the contract version and tolerate
additive keys.