Skip to main content

Artifact

Blob storage with metadata. The right block for generated files: images, audio, structured outputs, anything that's bigger than a record and that you want to fetch back later by id.

API

service Artifact {
rpc Put (stream PutRequest) returns (PutResponse); // client-stream
rpc Get (GetRequest) returns (stream GetResponse);
rpc Stat (StatRequest) returns (StatResponse);
rpc Delete(DeleteRequest) returns (DeleteResponse);
rpc List (ListRequest) returns (stream ArtifactMeta);
}

message PutRequest {
oneof body {
PutInit init = 1; // first message
PutChunk chunk = 2; // every subsequent message
}
}

Put is client-streaming: the first message carries PutInit (namespace, optional id, content type, user metadata); every subsequent message carries PutChunk bytes. The service streams those bytes through a TeeReader → sha256 so the response carries the server-computed SHA-256 and byte count of the upload.

Get is server-streaming: the first message carries GetHeader (metadata); every subsequent message carries GetChunk data. Range reads via offset / length work on every driver.

List enumerates a namespace's artifacts as a stream of ArtifactMeta (metadata only, no bytes) in ascending id order. Artifact is otherwise addressable only by known id, so List lets an agent discover contents without an external index. It takes an exact-match metadata filter, a start_after resume cursor (the last id from the previous page), and a limit.

Drivers

DriverNotes
memoryIn-memory map. Lossy on restart; fine for tests.
fsLocal filesystem with <base>/<ns>/<2-char-shard>/<id> layout and atomic temp-file-then-rename writes. Metadata in a sibling .json.
s3minio-go. Works against AWS S3, MinIO, Cloudflare R2, any S3-compatible store. Per-object metadata in x-amz-meta-* headers.

The service supports an optional metaPatcher interface so the FS and memory drivers persist the post-stream SHA-256 + size into their metadata record. S3 doesn't implement it today; a future slice can do a CopyObject self-copy with updated user-metadata to close that gap.

Configuration

backends:
- name: blob-local
driver: fs
options:
base_dir: /var/lib/mindd/blobs
- name: blob-s3
driver: s3
options:
endpoint: s3.amazonaws.com
bucket: my-bucket
use_ssl: true
access_key_env: AWS_ACCESS_KEY_ID
secret_key_env: AWS_SECRET_ACCESS_KEY

namespaces:
- { block: artifact, name: blobs, backend: blob-local }

gRPC example (HTTP/JSON gateway can't do client-stream)

# Encode the payload once for the multi-message JSON stream.
PAYLOAD_B64=$(base64 < ./generated.png | tr -d '\n')

printf '{"init":{"namespace":"blobs","id":"render-1","content_type":"image/png"}}\n{"chunk":{"data":"%s"}}\n' "$PAYLOAD_B64" | \
grpcurl -plaintext -H "x-mindd-capability: Bearer $TOKEN" -d @ \
127.0.0.1:7777 mindd.artifact.v1.Artifact/Put

grpcurl -plaintext -H "x-mindd-capability: Bearer $TOKEN" \
-d '{"namespace":"blobs","id":"render-1"}' \
127.0.0.1:7777 mindd.artifact.v1.Artifact/Stat

The Put response carries the SHA-256 the server computed while streaming. Use it to verify integrity client-side.

Python example

The Python SDK wraps the streaming into a bytes-in / bytes-out helper that chunks at 64 KiB internally:

ref = m.artifact.put("blobs", open("render.png","rb").read(),
id="render-1", content_type="image/png")
assert ref.size == ...
data = m.artifact.get("blobs", "render-1")

# Enumerate a namespace (metadata only), optionally filtered and paged.
for meta in m.artifact.list("blobs", filter={"kind": "render"}, limit=100):
print(meta.id, meta.size, meta.content_type)

Op names

OpMethod
artifact.putArtifact/Put
artifact.getArtifact/Get
artifact.statArtifact/Stat
artifact.deleteArtifact/Delete
artifact.listArtifact/List

Notes

  • The HTTP/JSON gateway can't surface client-streaming RPCs. Put is gRPC-only. Stat / Get / Delete work over HTTP fine (server-stream Get emits NDJSON envelopes).
  • Capability scope checks on Put happen after the first message arrives (since that's where the namespace is); a token without scope is rejected before any payload bytes are written to the driver.