Content, collections and memory

Synapse separates source files from reusable memory facts. A collection groups approved source versions. A memory fact records a value with its sources, namespace and validity dates.

These APIs require a content store. Document bodies and generated run bodies go to the store; control records retain references, hashes, access metadata and lifecycle state.

Choose a store

A store is either Katara storage or a bucket you connect. Both are created by an organization owner or administrator, both belong to one environment, and every content operation below works the same way with either. Connecting a bucket needs a person’s signed-in session rather than an API key, and so does deleting Katara storage, because that destroys its files.

• Katara storage. Call createHostedStore with a name. Katara keeps the files in its own storage for that environment, encrypted under a key used for nothing else, and nothing has to be set up in an AWS account. Each organization may keep 5 GB of Katara storage per environment: getStorageUsage reports the quota and what each store holds, and an upload that would pass the quota is refused with storage_quota_exceeded. Katara staff reach these files only through audited support access.
• Your own bucket. Prepare a storage connection with prepareContentConnection. Apply the returned trust, permissions and browser-upload settings to the bucket, then call verifyContentConnection.

listContentStores returns both kinds; kind is hosted or customer, and a hosted store carries its name.

Upload a source

1. Discover the active store with listContentStores. Hash the file bytes with SHA-256 and call beginContentUpload with its object ID, previous version, media type and exact byte count.
2. Call issueContentUpload and pass the signed grant, returned content reference and bytes to uploadContent in TypeScript or upload_content in Python. The helper sends the storage grant without your Katara credential.
3. Call completeContentUpload. Completion makes the upload a candidate; it does not approve its contents.
4. A permitted person reads the candidate with reviewContent and approves the exact reference with approveContent.

A content reference contains org_id, environment_id, id, version, integrity, media_type and size_bytes. Preserve every field. A filename or storage URL cannot replace it.

The content API accepts files up to 100 MiB within its supported media types. Parsing and agent inputs have smaller limits. Upload grants expire; after an uncertain upload, check or complete the original intent before creating another version.

Delete, export and hold a source

deleteContent / delete_content deletes a source in stages. Reads stop and the records derived from it, such as parsed text, index entries and graph facts, are retired as soon as the call returns. A worker then destroys the stored objects, every version of the source included, in Katara storage or in your bucket; the target for that stage is fifteen minutes and the limit is seven days.

Revoking a connection to your own bucket leaves its files in the bucket. Deleting Katara storage, closing the organization or resetting Test destroys every file in that store within seven days.

exportContent / export_content returns the record of a source: every version it has had, who it was shared with, and how far its deletion has got, stage by stage. A sha256 digest covers that state and leaves out the export time, so two exports of an unchanged source carry the same digest and a record kept after the source is gone can be compared with a later one. People who manage a source can export it while it exists; once it is deleted, the person who could have deleted it or an administrator can. Each export is recorded in the audit log, because it copies a customer’s record out of the system.

An owner or administrator places a legal hold with placeContentHold / place_content_hold, giving the reason and the reference that authorizes the exception, and lifts it with releaseContentHold / release_content_hold. Both need a person’s signed-in session, and both take a UUID as the idempotency key. A hold can be placed before or after the source is deleted. It does not bring access back: deleting a held source still stops reads and retires what was derived from it. What it stops is destruction. The stored objects and everything derived from them are kept until the hold is lifted, and an export shows held with no date for the record’s removal. listContentHolds / list_content_holds lists a source’s holds, newest first, with who authorized each and who lifted it; lifting one takes the id it lists.

A deleted or blocked source leaves the content list. listBlockedContent / list_blocked_content lists the ones whose records are still kept, most recently first, up to 25 a page, with whether a hold is in force on each, so a hold can be placed or lifted without the ID at hand. It needs an owner or administrator and synapse:memory:read.

Parse an approved document

parseContent / parse_content reads an approved source and returns text sections and chunks linked to that source. The call does not add the source to a collection, create a memory fact or persist an index.

Parsing accepts inputs up to 16 MiB and refuses archives. Its bounds include 8 MiB of normalized text, 32 MiB of response data, 8,192 sections, 500 PDF pages and 50 pages requiring OCR. A PDF with more pages that need OCR is refused whole with document_limit, before any page is read, and the message gives the number of pages: a document read in part would give citations that look complete. Fifty scanned pages take a minute or two to read.

OCR reads English, Spanish, French, German, Portuguese and Italian, all six on every page. A page in another script, such as Cyrillic, Greek, Arabic or Chinese, is refused with unsupported_language rather than read into letters that spell nothing. Another language written in the Latin script is read with the same six and can lose letters they do not have, so check such a result before you rely on it.

Every chunk read by OCR says where its words sit: locator.region is a box with left, top, width and height, measured from the top left corner of the page or image as it is displayed, in points (1/72 inch) on a PDF page and in pixels on an image, as its unit says. A chunk spanning several lines gets the box around all of them. Chunks that were not read by OCR have no region. Character offsets refer to normalized text rather than byte offsets in the original file.

OCR returns the characters on a page, not what a picture means: a diagram with no words in it parses to nothing. Table borders and box outlines are removed before a page is read, so a scanned table comes back one row per line and the labels inside a diagram’s boxes are read. A table without ruling lines is read by layout, which can join neighbouring columns; a typed PDF, DOCX or CSV keeps its structure exactly. French and Portuguese accents are sometimes read as a different accent, such as dû as dú, so check a quoted word before you rely on its spelling. For what an image shows, see Describe an image.

Access and the current content version are checked again after extraction. The SDK checks the returned source and chunk references. A withdrawn or changed source can therefore fail even if parsing started successfully.

Describe an image

A managed run can ask what an approved image, or one page of an approved PDF, shows. It names the collection it reads, the source at its approved version, the page for a PDF, optionally a region of it in the units OCR uses, and a description number from 1 to 64. The picture goes to the model the model catalog offers for images, katara/glm-4.6v@1 today, and the call is model work on the run’s budget, priced per token from the credit catalog like any other call to that model. The run needs synapse:memory:read on the collection and the source, and cortex:inference with locus:actions:execute to pay for the call.

The description comes back with what it describes: the source, the page or image, the region that was shown and the model that looked, with the model’s own statement of how sure it is, low, medium or high, and what it could not make out. A description is a record of its own in the run, kept apart from the OCR text of the same page, so cite the two separately. Text inside an image is described, never followed as an instruction.

Asking again under the same number returns the same description and pays nothing; a different picture under a number already used is refused. When no model in the catalog accepts images, the request is refused before anything is stored or paid; the preflight reports that in advance as its image_description workflow. A run whose budget cannot hold the price of the call is refused before the model is called. Deleting the source stops its descriptions being read, as it does its parsed text.

Build a collection

Create a collection with a client-generated UUID and a name. Fetch its current revision before adding or removing a source, renaming it, or changing access. Each mutation must use the current revision.

Adding a source requires access to both the collection and the approved source. Collection membership does not give another person permission to read the source. If source access changes, collection reads continue to enforce that source’s access rules.

A collection keeps the exact version each source had when it was added. When a document gets a newer version, for example a web source that synced again, resyncCollectionSources / resync_collection_sources moves each source you can read to its document’s current approved version, drops a source whose document was deleted or whose newest version is not approved, and leaves a source you cannot read as it is. One call handles up to 100 sources and returns what it changed; pass next_cursor back as after, with the returned revision, until no cursor comes back.

Source pagination carries a collection revision. Keep the response’s revision with its cursor; restart the listing if the collection changes. The SDK examples include an exact-version source addition.

Assemble context for your own runtime

An agent you run yourself can ask for the passages that best answer a question, packed to fit its model. Store the question as content, then call assembleContext / assemble_context with up to five collections at the revisions you read, the model the context is for, a token budget, how many passages to take from each collection, and the grant and budget that pay for the search.

Each collection is searched once, with the question embedded and charged as a retrieval. The best passages across all of them are packed into the budget as that model counts tokens; every passage names its source document, its collection and the chunk it came from, and over_budget counts those left out. The passages come from what managed runs of your agents have indexed; see indexing when a run starts.

Katara keeps each search’s result for an hour, for you alone: repeat a request under the same id within that hour and you get the same paid searches back, not new ones. A collection that changed since you read it is refused before anything is searched or charged; read it again first. When the search model is busy with other work for longer than the request can wait, the request is refused with 429 retrieval_busy and nothing is charged; try again.

Attach a repository

A repository attachment lets an environment answer from a GitHub repository. Katara reads it through the Katara GitHub App, which an owner or administrator installs on the GitHub account that owns the repository, with read access to contents and metadata only.

1. Install the App. On the Repositories page, an owner or administrator chooses Install the GitHub App, picks the GitHub account and the repositories it may read, and authorizes the App. GitHub returns them to the page, and Katara links the installation to the organization once GitHub confirms that they can reach it. In the API this is beginGitHubAuthorization with purpose set to install, then completeGitHubAuthorization with the code, state and installation_id GitHub returned. An installation is linked to one organization at a time; listGitHubInstallations lists the organization’s.
2. Link your GitHub login. Each person links theirs once, with purpose set to link. Katara keeps the login and its GitHub id, and revokes the token GitHub issued to prove them as soon as it has done so.
3. Attach. attachRepository takes a client-generated UUID, the provider, the owner and the repository exactly as GitHub names them, and the content store the documents go to. listGitHubRepositories lists the repositories an installation reaches. Attaching the same repository twice attaches it once. Attaching approves the repository’s default branch for 30 days, so it needs a person’s signed-in session, not an API key.
4. Sync. syncRepository reads the repository at the newest commit on its default branch and keeps reading that commit until it is read completely, at most 20 documents a call, so call it again until complete is true. Anyone who may attach can ask for a sync while the approval lasts. Each document becomes content in the store owned by the person who attached the repository.

Each file becomes a document: Markdown, plain text, CSV and JSON keep their types, and YAML, HTML, reStructuredText and AsciiDoc are stored as plain text. A changed file becomes a new version of the same document, and a file that left the repository is deleted. listRepositoryFiles names each document with the path, the blob and the commit it was read at, so a passage cited from a document version can be traced to the commit it came from.

Some files are seen and not read, and listRepositorySkips says which and why: a credential-shaped name or directory (.env, *.pem, secrets/, .ssh/ and the like), a file over 1 MiB, a file that is not UTF-8 text, or a file whose contents hold a secret such as a private key or an access token. A secret is named by the rule that matched it; the text that matched is never stored. A repository whose file list GitHub reports as truncated, or that holds more than 2,000 documents, is refused with repository_too_large rather than partly read.

Pushes and the approval

While the approval lasts, a push to the default branch syncs automatically. GitHub tells Katara about the push, and Katara reads the new commit, only the files whose blobs changed, with nobody signed in. One sync of a repository runs at a time, and several pushes in quick succession are read as one sync of whatever the head is when it starts. A push to another branch is not read.

Katara approves each new document version it reads under the approval and records that it did: listRepositoryFiles gives each version’s approver (system), when it approved it, and the approval it rested on, with approval_given_by naming the person who attached or renewed the repository. Katara approves versions of that repository’s documents and nothing else.

The attachment says where its approval stands: approval_given_by, approval_given_at, approval_expires_at and approval_state, active or expired. Six days before the approval ends, the person who attached the repository and the organization’s owners and admins get a reminder; see Notifications.

Once the approval has ended, a push only sets sync_state to behind with the new head_commit, and nothing is read: a sync asked for then is refused with repository_approval_expired. renewRepository approves the repository again for 30 days from then. It needs the same authority and signed-in session as attaching; the person who renews need not be the one who attached it. Renewing reads the newest commit if the repository is behind.

Who may read what

An attachment records which repository, never who may read it. Every read of a repository document asks GitHub, at that moment, whether the person asking may read the repository, as the GitHub login they linked. Removing someone’s access at GitHub stops their reads on the next question. A person who has not linked a login is refused with github_link_required until they do. For an agent’s run, the person asking is the person who started the run; an API key asks as the person it belongs to, and a service account reads nothing from a repository. Permissions are per repository, not per file. The documents are also content in the store, so who in the organization may use them follows the store’s sharing, as for any other document.

Detach or disconnect

detachRepository stops reads at once and deletes every document read from the repository, which retires the parsed text, index entries and facts derived from them. The same happens when the App is uninstalled at GitHub, when the repository is removed from the installation, and when an owner or administrator unlinks the installation with unlinkGitHubInstallation. listRepositories returns attachments in pages, most recent first, and includes the detached ones: what an agent could read last month is a question about the past. Carry the response’s next_cursor to the next call rather than an offset.

Approve a web source

A web source lets an environment fetch documents from one https origin, under one path, for a period you choose. Approving it is a person’s decision, made in a signed-in session. Call registerSource (register_source in Python) with kind set to https, the content store the documents go to, the origin, a path prefix that begins and ends with a slash, and how long the approval lasts, from an hour to a year. Web sources are the only kind served; any other kind is refused.

syncSource fetches one document under the approved path and says whether it changed since the last fetch. The first fetch of an address goes into a content object you name; later fetches of the same address add a version to that object when the document has changed, and the answer names it. A document outside the path, or of a type the content store does not take, is refused rather than fetched. HTML is one of those types: turning a page into text is an extraction decision, not a side effect of fetching it.

reconnectSource approves the source again, for the same path or a narrower one, and needs a person’s signed-in session. revokeSource withdraws the source and every document fetched through it, and deleteSourceDocument withdraws one. getSource and listSources show each source’s origin, path, state and document count, and when its approval ends. Approving, syncing, reconnecting and revoking take a UUID as the idempotency key.

Review memory facts

A fact belongs to an organization, user, agent or session namespace. Create a candidate with proposeMemory, supplying its content reference, 1–31 source references and dates. Extracted facts also require the approved extraction-recipe reference.

Use inspectMemory and reviewMemory to inspect a candidate. approveMemory requires a human session and the fact’s exact ID, version and integrity hash. Approved reads use getMemory and readMemory; they still check current source authority and validity. A new proposal does not silently replace an approved fact.

Use deleteMemory to withdraw a fact. Content deletion and collection-source removal are separate operations with different effects. Generated output remains untrusted; storing or approving a fact does not turn its text into execution authority.

Whether managed runs are served where you call is a separate question.

What managed runs do with knowledge

A managed run reads the collections its agent version lists in knowledge_collections, and the approved facts its grants name.

Indexing when a run starts

Before a run makes its first planning decision, it brings its knowledge up to date. Each document version a listed collection pins is parsed into passages of about two hundred words, and each passage is embedded for search, unless an earlier run already did that under the same parsing and embedding settings and the result can still be read. Each approved fact the run’s grants name with synapse:memory:read and synapse:memory:index is placed in the fact graph that recall reads, unless that exact version is already there.

A run whose collections are already indexed plans at once and pays nothing for its knowledge. An agent version that lists no collection does no indexing. Two runs of the same agent that start together index once: one does the work, and the other waits for it and then uses it.

What it costs

Parsing and embedding are service work charged to the run’s own budget: parsing by the size of each document, embedding by the tokens in its passages. Placing facts in the graph is not charged. The charges are made before the run’s first model call and appear in what the run has spent and in its trace. A budget whose service allowance cannot hold the price ends the run with reason budget_exhausted before it plans; the run does not go on without its knowledge. Indexing counts toward the run’s active time.

What it needs

A run indexes with the agent’s own access. Its grant must allow synapse:memory:read and synapse:memory:index on each collection and each document in it, and the agent must be able to write the collection’s index. enableContentAccess / enable_content_access, and allowing the agent to run in the platform, set up both for a version that lists collections. They also name the approved facts of the organization and of the agent, with the documents those facts cite, so the agent’s runs place them in the graph. A fact approved later needs the same call again, as a document added to a collection does. A run without that authority ends with authority_denied before it plans, and so does a run started by a person who has not linked the GitHub login a repository document is read as.

A document the agent cannot read is left out and the run goes on: a deleted document, one no longer shared with the agent, one the person who started the run can no longer read on GitHub, or one the parser refuses, such as a scan in an unsupported script. The run’s search never returns it.

When it indexes again

Only what changed is indexed again. When a document gets a new version and resyncCollectionSources moves the collection to it, the next run parses and embeds that version alone and reuses the rest. A new approved version of a fact is placed in the graph by the next run.

Some changes take what was indexed with them, and the next run that needs it indexes it again:

• a document removed from a collection or deleted, whose passages stop being searchable at once;
• a change to who may read a collection, which retires the whole collection’s index;
• the end of the run that did the indexing, if it failed, was cancelled or awaits reconciliation, which retires that run’s work with the rest of what it produced;
• the grant or consent that paid for it expiring or being revoked, its budget closing, or the end of the retention the run kept its output for.

What search and recall return

A run’s search and its indexing take turns at the search model: each waits for the other, briefly, rather than being refused. If the model stays busy past that wait, the search step fails with nothing charged and the indexing tries again on the run’s next turn.

A run’s search ranks indexed passages by meaning and by their words and returns the best ones, each naming its document version and the passage it came from. Before a passage is returned, its document is checked again: it must still be in the collection at that version and readable by the agent, and a repository document readable on GitHub by the person who started the run. Graph recall returns approved facts with the sources they cite, checked again the same way.

Facts are not extracted from documents. Propose a fact with proposeMemory; a person approves each one.