Photos, videos, documents, and JSON files can be treated as sequences of bytes that an application uploads and retrieves. Object storage organizes that content as identifiable objects, with metadata and operations on their representation.
The architectural decision is to separate data that needs frequent queries and transactions from content read or written as files. That separation can offload the application and database, but introduces permissions, intermediate states, and recovery across different systems.
Bytes, metadata, and tokens
An application can keep a file’s owner, status, original name, and reference in PostgreSQL, while storing its bytes in object storage. Filters, relationships, and permissions use that queryable data model. The object can also have its own metadata, such as content type or checksum; separating responsibilities is a decision, rather than a prohibition on storing metadata alongside bytes.
| Representation | What it preserves | What needs evaluation |
|---|---|---|
| BLOB in PostgreSQL | Bytes inside the database, for example in a bytea column or through Large Objects. | Access, transactions, sizes, backups, and database load. |
| Object | Bytes and metadata associated with a key in a bucket. | Service operations, permissions, versions, durability, and transfer cost. |
| ERC-721 NFT | A token’s identity and ownership in a contract. | Which content it references, how that reference resolves, and what guarantees it offers. |
In PostgreSQL, bytea and its storage through TOAST still keep bytes inside PostgreSQL. The Large Objects interface uses a different API and storage in dedicated tables; it is not interchangeable without considering its access patterns and lifecycle. Storing an external URL in a row preserves a reference, rather than the BLOB it points to. The choice depends on size, frequency, and the transactional unit; object storage is not automatically better for every binary value.
A BLOB and an NFT are not equivalent. In ERC-721, the metadata extension and tokenURI are optional. That URI can resolve to a JSON document referencing an image or another resource; this does not imply that the file is inside the token. Token ownership does not guarantee file availability or rights to its content. Token identity, references, and byte storage are distinct layers.
Buckets, keys, and versions
A common model uses a bucket as a container and a key as the object’s identifier within it. A key such as tenant-42/uploads/upload-123/photo.jpg organizes names by prefixes. The appearance of folders does not imply POSIX directories, inherited permissions, or transactional renames between paths. The service’s contract defines those operations.
The database should store a stable locator: bucket, key, and, where applicable, version or generation. A signed URL is temporary and does not serve as a permanent identifier. The original filename can remain presentation data; it does not need to determine the key or tenant by itself.
In many object APIs, writing means uploading a complete object or replacing the one associated with a key. As a concrete example, S3 PutObject stores the submitted object; it does not turn a key into a row that can be edited byte by byte. Reading a byte range, when supported, does not imply equivalent partial writes either.
Object storage does not mean automatic immutability. Overwrites, versioning, write conditions, and retention are different capabilities. Versioning can preserve revisions, but is not equivalent to preventing their deletion through a retention or locking policy. When published content must remain fixed, the application needs an explicit strategy for keys, versions, and permissions.
Durability and availability
Durability describes the ability to preserve data against failures covered by the service. Availability describes whether it can be read or written at a particular moment. An object can remain preserved while being temporarily inaccessible.
Replication maintains copies in defined failure domains. Erasure coding distributes data and redundancy fragments that allow content to be reconstructed while enough fragments remain. Distribution, fault tolerance, and recovery behavior depend on the design and storage class; not every object is replicated to every region.
Those mechanisms do not replace permissions, backups, or recovery policies for logical deletions. An authorized deletion can propagate to copies. The metadata used to find the object also needs preservation and recovery: available bytes with a lost reference still pose a problem for the application.
Direct uploads and signed URLs
The application can authorize an upload and let the client send bytes directly to storage. This avoids routing every file through API servers. Those servers retain control of business rules.
--- config: look: handDrawn --- flowchart TD client["Client"] -->|"Request upload"| api["API<br/>Authenticate and authorize"] api -->|"Record pending attempt"| metadata[("Metadata DB<br/>Owner, key, and status")] api -->|"Signed upload URL"| upload["Client uploads<br/>and completes the upload"] upload --> storage[("Object storage<br/>Bytes and version")] upload -.->|"Request finalization"| verify["Server verifies<br/>object, version, and conditions"] verify -->|"Check actual object"| storage verify -->|"Mark ready after verification"| metadata metadata -->|"Ready status"| read["API authorizes read"] read -->|"Signed read URL"| download["Client downloads bytes"] download --> storage
The API authenticates the user, checks whether they may attach that file, and creates an upload_id with an owner, assigned key, and pending status. After committing that metadata, it issues limited authorization for the upload. The client cannot freely choose someone else’s key or make its declared ownerId a trusted identity.
A signed URL acts as a bearer capability: whoever possesses it can perform the permitted operation while it remains valid. In the S3 example, it is bound to a bucket, key, method, validity period, and the signer’s permissions. It can be reused and expire earlier than its stated deadline if the associated credentials expire. It must not be treated as a one-time link or as an application permission check on each access.
The signature can restrict conditions or headers according to the available mechanism. Size, format, and quota limits need controls that are actually supported and server-side checks; declaring Content-Type does not prove that a file has that format. URLs should be treated as sensitive capabilities, with suitable lifetimes and without unnecessary exposure in logs.
Multipart and upload recovery
For large files, multipart divides an upload into parts that can be sent in parallel and retried separately. This can use the connection effectively and avoid resending the entire file, but concurrency should match real memory, bandwidth, limits, and costs.
In the S3 multipart flow, an upload is initiated, its parts are sent, and it is completed with the required references. The final object materializes after completion; uploading some parts does not mean publishing the file. Incomplete parts can continue consuming storage until completion, abort, or configured cleanup.
The upload identifier and confirmed parts allow progress to be recovered under the implementation’s contract. A timeout requires checking what was accepted before repeating the operation or starting another attempt. Completion can also have an uncertain outcome: the application must be able to query and reconcile the object associated with the same attempt.
Checksums and ETag must not be universally conflated. Their meaning depends on the service and upload mode; a multipart ETag is not necessarily the hash of the whole file. Verification should use the documented integrity mechanism for that operation.
Consistent finalization and reads
The client can request finalization, but its claim that an upload finished is not sufficient evidence. The server authorizes the upload_id and verifies the actual object at the expected key, its version, size, and required conditions. Only then does it mark the metadata as ready. If additional content validation is required, the status can remain pending until that completes.
Verification should be bound to the bytes that will be served. If upload authorization allows overwriting the same key, a later write could change already-validated content. Keys isolated by attempt, write conditions, and a specific version for reads allow that contract to be defined; enabling versioning alone does not prevent overwrites or deletions.
The database and object storage do not automatically form a shared atomic transaction. An object can remain without ready metadata, or a pending attempt’s file may never arrive. Idempotent finalization and durable states allow those situations to be recovered, consistent with transaction boundaries and recovery across components.
On reads, the API checks permissions and ready status, retrieves the stable locator, and issues a limited read URL. The client downloads bytes directly. An application permission change does not necessarily revoke an already-issued URL immediately; its lifetime and the service’s capabilities should fit the access contract.
Operations and selection criteria
Cleanup should distinguish active uploads, abandoned parts, and orphaned objects, using deadlines and verified references to avoid deleting valid content. It is also useful to reconcile ready metadata with missing objects and record finalization failures for recovery.
Repeated reads can benefit from a CDN or cache, with explicit freshness and authorization decisions. A mutable key may need invalidation; a fixed version makes it easier to identify which content is served, without resolving every permission by itself.
Evaluation includes size and growth, upload and read latency, failures and retries, transfers, requests, and storage for versions or incomplete parts. There is no universal savings or throughput figure. For small byte values participating in a local transactional unit, PostgreSQL can remain appropriate; for files and direct uploads, object storage may better fit the expected access patterns.
The System Design path connects this choice to data models, system limits, and recovery. The useful question is what each component preserves and how to verify that a file and its status still correspond.