Artifact Pool - Parable

The Artifact Pool contains files published as queryable tables. Each table has a namespace within its Workspace:

artifacts.{namespace}.{table}

For example:

SELECT *
FROM artifacts.public.headcount
LIMIT 10

Existing uploads use the reserved public namespace. The shorter artifacts.headcount remains an alias for artifacts.public.headcount; it never searches other namespaces. Separate publications can use the same table name, such as artifacts.september.classifications and artifacts.october.classifications, without sharing data. Namespaces are stable SQL identities. Folders remain display organization and do not change a table’s SQL address. The System Workspace does not accept Artifact Pool publications.

Publication revisions

Ordinary SQL, prepared statements, Flight metadata, and catalog discovery read each namespace’s active revision. A correction is a new revision of the same namespace. Queries keep reading the previous active revision until every table of the correction is ingested and someone activates it, so a failed replacement leaves the previous complete dataset in place. A Flight SQL request can read another revision instead, such as a staged correction waiting for QA, the verified tables of a failed revision, the previous revision, or one revision pinned across several queries. Name it in request metadata:

x-parable-artifact-revisions: {namespaceId}:{revisionId}

Separate several namespaces with commas. For that request, the named namespace’s tables resolve to the selected revision; other namespaces keep their active revision. A selection grants nothing: it resolves only among the revisions of the request’s Workspace, and the caller still needs access to the Artifact Pool. A revision the Workspace does not list fails as not found. A prepared statement runs only under the selection it was prepared with. A removed revision, and every revision of a deleted namespace, is no longer listed. A namespace keeps its active revision, the revision that was active before it, and every staged or failed revision until someone removes it. There is no expiry. When a revision is removed, its storage is deleted, except tables a kept revision still shares. A producer may publish its own context table, such as CM3’s report_context; the producer owns that table’s contract. Table schemas, counts, hashes, and storage verification stay in the publication inventory. Report and QA tools read the tables with ordinary SQL. Producers upload Arrow as their libraries write it. The publication worker normalizes every batch before it validates or writes a table: UTC or naive timestamps become the existing microsecond serving representation, UInt32 counts become Int64, and large/view strings become UTF-8. Values that would overflow or lose sub-microsecond precision, other timezone labels, and types the shared mapper cannot represent fail the upload as an invalid schema. ANALYSE derives each table’s schema from the normalized stream. Report readers restore UTC interpretation from shared Temporal.DateTime metadata. The JDBC 18.3.0 client requires a UTC JVM default for these timezone-free values; UTC-aware serving remains blocked by its general value reader. See the accepted publication amendment for the decision.

Publish a complete dataset

Publication producers use the experimental generated SDK operations to make an entire dataset available together:

Call artifacts.reserveArtifactPublication with a namespace, revision ID, expected active revision, and every table’s name, row count, source size, format, and hash. Tables declare no schema: the worker derives it from the upload. A CSV table gets all-String columns, so send Arrow for typed columns. A table whose name, hash, and row count match the expected active revision’s table reuses its stored data: the receipt marks it reused, and it gets no upload URL.
Upload each remaining CSV or Arrow IPC stream directly to its returned storage URL with the required headers.
Call artifacts.finalizeArtifactPublication and poll artifacts.artifactPublicationReceipt. Each table’s outputSchema appears in the receipt once analysis derives it.
Once every table is ingested, call artifacts.activateArtifactPublication. Until then, or until you activate it, the revision stays staged and QA can read it by selecting it. Activation checks the caller’s current permissions and switches the namespace to the complete revision in one transaction; it does not query the tables.

A failed replacement leaves the previous complete dataset available. Retry unchanged failed inputs with retryFailed; corrected inputs require a new revision ID in the selected namespace. Activation refuses a replacement whose expected active revision has changed. Activating a revision that is already READY returns its status and changes nothing, unless the request sets expectedActiveRevisionId to the current active revision: that switches back to it, which is how you roll back to the previous revision. artifacts.removeArtifactPublicationRevision removes a staged or failed revision that is not active and queues cleanup of its storage. artifacts.deleteArtifactPublication deletes the namespace: queries stop serving it, its slug becomes free for a new namespace, and durable cleanup of all its stored data is queued. Receipts of a removed revision or deleted namespace are no longer readable. Cleanup waits for issued upload URLs to expire.

Publish a table

An uploaded file becomes queryable in public after the Workspace:

Reserves the upload with artifacts.prepareArtifactUpload, then sends the file directly to the returned signed storage URL using its required headers.
Calls artifacts.completeArtifactUpload and waits for uploadStatus: ready. The worker verifies the file and saves a bounded sample and an all-String schema.
Reviews that schema, optionally using artifacts.runArtifactSchemaInference or artifacts.updateArtifactDataSchema.
Calls artifacts.publishArtifact with a new request ID, then polls the Artifact’s publishWork. The table becomes queryable only when that work succeeds. Values that cannot be converted to the reviewed schema fail publication; correct the schema or source and retry. Invalid typed values are no longer silently replaced with nulls.

The app performs these steps and keeps processing status across page reloads. A repeated publication request with the same request ID and captured input returns its existing work. Unpublish hides the table and retains storage. Delete hides the Artifact and queues cleanup; outstanding upload grants must expire before its final cleanup sweep. See the Workspace HTTP reference for the exact requests and generated SDK examples.

Schema inference and the model policy

runArtifactSchemaInference sends the CSV’s header row and up to five data rows to the model the Workspace’s model policy chooses for schema inference, on the AI processor that policy allows, and saves the column types it proposes. Every environment’s policy runs it on Gemini (gemini-2.5-flash) in Vertex AI, in the Workspace’s own Vertex project when it has one and the environment’s shared project otherwise. The result’s inference names the model and the processor that ran. When the policy allows no processor for schema inference, for example because a Workspace administrator switched VERTEX off, nothing is sent to any processor. The call still succeeds: the Artifact’s schemaInferenceStatus becomes skipped_by_policy, the result’s inferenceRefusal names the reason, and the Artifact keeps the all-String schema from upload. Edit that schema with artifacts.updateArtifactDataSchema, then publish it as usual.

Upload limits

Each uploaded file can be up to 1 GB. Accepted formats are .csv, .xlsx, .xls, .numbers, .ods, and .zip. Only CSV supports preview, schema inference, and publication. Other formats remain storage-only uploads. The browser and upload reservation enforce the size limit. The signed request binds its content type, exact byte length, and create-only condition; the worker checks stored size and content before making the Artifact ready. Expired URLs require a new reservation. File bytes do not pass through the Workspace API or browser proxy. The publication’s table name becomes {table} in that namespace. Artifacts can join other pools. For example, a published headcount plan can join Provider directory data on a compatible email column:

SELECT
  plan.email,
  plan.target_department,
  directory.orgunitpath,
  directory.isadmin
FROM artifacts.headcount AS plan
LEFT JOIN providers.google.users AS directory
  ON directory.primaryemail = plan.email

Use catalog discovery to inspect the published schema rather than inferring it from the source file.

In Report runs

A Report run reads an artifact table as it reads a Provider table: at the last version published before the run’s AS OF time, so a publish while the run is queued is not read. Artifact tables have no Data Policies, so a Report run’s recorded source authorities do not list them. See Reports.

Admin

​Publication revisions

​Publish a complete dataset

​Publish a table

​Schema inference and the model policy

​Upload limits

​In Report runs