"Where do we put the data?" has three basic answers in the cloud, and they are not interchangeable. They differ in how an application reaches the data, which in turn decides performance, scale and what the data is good for.
The three, briefly
Object storage keeps data as objects in a flat namespace of buckets. Each object is a blob of any size with a key (its name) and metadata. You reach it over an HTTP API: put, get, list, delete. Objects are replaced as a whole, not edited in place. Amazon S3, Google Cloud Storage and Azure Blob Storage are well-known examples.
Block storage is a raw disk. The provider gives a virtual machine a volume made of fixed-size blocks, which the operating system formats with a file system and uses like a local drive. It is fast and has low latency, and is usually attached to one machine at a time.
File storage is a shared folder. Many machines mount the same hierarchical file system over a network protocol (NFS or SMB), and see the usual directories, permissions and file operations.
Compared
| Object storage | Block storage | File storage | |
|---|---|---|---|
| Access | HTTP API (get / put objects) | Attached as a disk to a machine | Network file share (NFS, SMB) |
| Structure | Flat: buckets and keys, with metadata | Blocks under a file system you create | Hierarchical folders and files |
| Editing | Replace the whole object | Change any block in place | Change any part of a file |
| Latency | Higher per request | Lowest | Moderate |
| Throughput and scale | Very large capacity; scales with parallel requests | Limited to volume size and the machine's attachment | Large, shared among many clients |
| Shared by many machines | Yes, naturally | Usually one at a time | Yes, by design |
| Typical uses | Backups, images and video, logs, data lakes, static websites | Databases, system disks, applications that need fast random access | Shared content, home directories, applications that expect a normal file system |
Choosing
Ask what the application needs from the data.
- Does it need a file system and fast random reads and writes? (A database, an operating system.) → Block.
- Do several servers need to read and write the same files as ordinary folders? → File.
- Is it large amounts of data written once and read many times, reached by programs, possibly by many clients at once? → Object.
Many systems use all three: servers boot from block volumes, shared uploads sit on file storage, and backups, media and analytical data live in object storage.
Things that catch people out
- Object storage is not a drive. There are no real folders (a slash in a key is just a character), renaming means copying, and you cannot append to an object in place. Applications must be written for it.
- Block volumes belong somewhere. They are usually tied to one availability zone and one attached machine. Back them up with snapshots.
- Shared file systems need care with many writers. Locking and performance behave differently from a local disk.
- Charges beyond capacity. Requests, data leaving the cloud and performance tiers can outweigh the per-gigabyte price.
- Durability is not backup. Providers store data redundantly, but a deletion or a bad overwrite is copied too. Use versioning, snapshots and separate backups.
What we compared
- How it is accessed
- Structure and metadata
- Performance profile
- Scale and sharing
- Typical uses
- What it costs to get wrong
Recommendation
Use object storage for large, mostly-read data, backups, media and data lakes. Use block storage for the disks of a server or a database that needs low latency. Use file storage when several machines need a shared folder with normal file semantics.
Caveats
- Providers' products differ in detail: limits, consistency rules, performance tiers and pricing. Read the current documentation of the one you use.
- Some workloads fit more than one. Databases run on block storage but may back up to object storage.
- Moving between types later means changing the application, so decide with the access pattern in mind.
Frequently asked questions
Can I mount object storage like a drive?
Adapters exist, but object storage is not a file system: no true folders, no editing a file in place, and different performance. Use it through its API where you can.
Which is cheapest?
Per gigabyte, object storage is usually the cheapest, then file, then block. But the right comparison includes request charges, data transfer and performance tiers, so price your real workload.
References
From Grokem: https://www.grokem.com/cloud/storage/object-block-file-storage-comparison
