How Google Drive Syncs Files and Resolves Concurrent Edits

High-level overview: why one upload becomes a distributed systems problem

To a user, a cloud drive is folders and files. Behind it are two systems: a metadata system for names, folders, permissions, versions, and sync positions; and a byte system for large, slow-to-copy file objects. Combining them in one database or request path makes both a bottleneck.

Several devices and collaborators make failures normal: transfers interrupt, devices edit offline, and regions fail. A safe design turns each into a retry or recoverable conflict. It needs one authoritative result for metadata changes, resumable byte transfer, and a durable history for devices that were offline longer than a folder listing can explain.

Durability is more than replicas: checksums detect corruption, independent copies survive hardware loss, history and delayed deletion recover from mistakes, and restore drills validate the whole chain. Each covers a different failure.

Scope and assumptions

The design covers files, folders, sharing, history, resumable transfer, multi-device sync, and machine, zone, and regional failure recovery. Authentication, search, media processing, antivirus, billing, legal retention, and live Google Docs-style editing are out of scope. “No data loss” means an acknowledged version survives the failures designed for.

Functional requirements

A client can create a folder and upload, download, rename, move, trash, restore, or permanently delete a file.
An interrupted upload resumes without retransmitting confirmed chunks.
A committed file version downloads byte-for-byte identically to the uploaded input.
A user can list prior versions and restore one as the current version.
Owners can grant and revoke viewer or editor access to users and groups.
Each device can request all changes after its last cursor without rescanning the full tree.
Desktop and mobile clients detect local changes, queue them offline, and resume sync after reconnecting.
The service stores any binary file type and enforces configurable account, version, and retention limits.

Non-functional requirements

Property	Design target
Metadata latency	p99 below 150 ms in the user’s home region, assuming nearby regions
Transfer setup latency	p99 below 300 ms; byte transfer remains network-bound
Consistency	Strictly serializable metadata mutations within a drive shard; eventual notification delivery
Durability	99.999999999% annual object durability target; acknowledge a version only after its bytes and metadata are durable in another region inside the same residency boundary
Scale	1 billion accounts, 100 million daily active users, 102,000 peak metadata QPS
Cost	At most 0.81 stored bytes per logical byte before replication, assuming 30% data reduction and 15% storage overhead
Transport security	TLS 1.3 externally and mutual TLS between internal services
Storage security	AES-256 at rest with envelope encryption; key rotation within 90 days
Audit	Permission changes and file access enter an append-only log within 5 seconds

Strict serializability combines serializable multi-record transactions with linearizable real-time ordering: if commit A finishes before commit B begins, B observes A. If replicas cannot safely agree, the system delays or rejects the write rather than claim an order it cannot keep.

Back-of-the-envelope estimation

Assume 1 billion registered accounts, 100 million daily active users (DAU), 20 metadata reads, two file-version commits, and ten downloads per active user per day. The metadata-read count includes listing, permission, and download-setup calls. Assume an 8 MiB average upload, a 4 MiB average download, a four-times peak factor, 30% savings from compression and deduplication, three replicas, and 15% storage overhead.

Figure	Visible arithmetic	Result
Metadata reads	`100M × 20 ÷ 86,400`	23,148 average QPS
File-version commits	`100M × 2 ÷ 86,400`	2,315 average QPS
Read-to-file-commit ratio	`20 ÷ 2`	10:1
Peak metadata load	`(23,148 + 2,315) × 4`	101,852 QPS
New logical bytes	`100M × 2 × 8 MiB`	1.49 PiB/day
Yearly logical growth	`1.49 PiB × 365`	544 PiB/year
Physical yearly growth	`544 PiB × 0.70 × 3 × 1.15`	1,314 PiB/year
Average ingress	`1.49 PiB × 8 ÷ 86,400`	155 Gbit/s
Average egress	`100M × 10 × 4 MiB × 8 ÷ 86,400`	388 Gbit/s
Peak edge bandwidth	`(155 + 388) Gbit/s × 4`	2.17 Tbit/s
API nodes	`101,852 ÷ 2,000 QPS/node × 2`	102 nodes with redundancy
Storage nodes for one year	`1,314 PiB × 1,024 ÷ 20 TiB/node × 1.25`	about 84,100 nodes

The per-node figures are planning assumptions. Real benchmarking decides the final count. The calculation exposes the governing fact: storage fleet size and repair traffic dominate the API tier.

Choosing the upload path

There are three common ways to move file bytes. Only one keeps the application tier out of the expensive data path.

Approach	How it works	Benefit	Cost
Store on the API server	Client uploads to local server disk	Minimal prototype	Server loss can lose data; disk and scaling limits arrive quickly
Proxy through the API server	Server forwards the body to the chunk store	Central policy enforcement	Every byte crosses the application fleet twice
Direct upload	Metadata service issues limited-time URLs; client writes through the chunk-store gateway	Lowest server bandwidth and independent transfer scaling	Requires secure tokens, commit validation, and abandoned-upload cleanup

Choose direct upload as the default. Small files may use a separately scaled ingest pool; larger or unreliable transfers use a resumable session and go directly to the chunk store. The chunk store has a flat key space, so folders live in metadata and renames never move bytes.

The chunk-store gateway verifies the authorized length and cryptographic digest against the received bytes before issuing a durable receipt; it never trusts a client-declared chunk ID alone.

Core concepts

Follow one file from a laptop to another device: split it, store its pieces, publish their manifest atomically, reject accidental overwrites, and let other devices replay the durable change.

1. Split a file into chunks

Restarting a 5 TB upload after one timeout is unacceptable. Re-uploading an entire virtual-machine image after a small edit is wasteful. Chunk boundaries decide how much work a retry or edit creates.

A chunk is an immutable piece of a file. A manifest records its ordered chunk references, sizes, checksum algorithm/version, whole-file checksum, and encryption metadata, so it can rebuild one exact version. Edits create new chunks only where bytes change; failed-upload chunks remain invisible until later cleanup proves no retained manifest needs them.

2. Keep file details separate from file bytes

The metadata service handles names, folders, owners, access rules, versions, and sync positions. The chunk store holds immutable file chunks, and its chunk-store gateway verifies direct uploads and downloads. A client asks the metadata service for a temporary URL, then sends bytes directly to the chunk-store gateway.

This keeps large file traffic away from API servers. The temporary URL works only for one upload session or file version, expires quickly, and cannot be used for another file.

3. Choose where chunks begin

A SHA-256 hash identifies a chunk’s plaintext bytes, but is not its storage identity. A reusable chunk is identified by (dedupeScopeId, hash, byteLength, encryptionDomain): the scope limits reuse to one tenant or trusted organization, and the encryption domain prevents references across incompatible keys.

Fixed-size chunking cuts every N bytes. It is simple and predictable. Content-defined chunking (CDC) uses the file’s contents to choose boundaries. After a small insertion, later boundaries can line up again, so more old chunks can be reused. CDC costs more CPU and produces variable-sized chunks; it is useful for large files that change often, not required for every upload.

4. Make a complete version visible

Uploading chunks does not make a file visible. The publish transaction verifies durable receipts, saves the manifest, changes currentVersionId, and records the sync event. Readers see the old complete file or the new one, never a partial file.

5. Stop one device from overwriting another

Each update includes the version the client started from, often as an ETag (an opaque version token) or version number. The server accepts the update only when that version is still current. If another device changed the file first, the server reports a conflict instead of silently overwriting newer bytes.

6. Help every device catch up

A notification is a doorbell, not a history book. It tells a device to check for changes, but it can arrive late or more than once. The change feed is the saved, ordered history. Each device stores a cursor per drive, meaning “I have applied everything up to here.” It saves the next cursor only after it has safely applied those changes locally.

Quick path: the model to remember

With the core concepts in place, newer system-design readers can next read the Write path and Handle offline edits and conflicts; the estimates and deep dives are optional on a first pass.

Keep names, permissions, and version pointers in the metadata service; keep large bytes in immutable chunks.
Upload chunks directly, then publish one manifest in a metadata transaction. Until that commit, no reader sees the new file.
Treat notifications as a wake-up signal. The durable change feed, replayed from a saved cursor, is what keeps devices correct.
Send the base version with every edit. A stale binary edit becomes a conflict copy rather than overwriting work.
Delete cautiously: retained manifests are the source of truth for garbage collection.

Detailed design

Components and ownership

The architecture separates two jobs: moving bytes and deciding which version users should see. The most important step is the single metadata commit between those jobs.

The gateway authenticates metadata requests; the token service grants limited file-data access.
The transactional store owns file names, current versions, permissions, upload sessions, group-membership versions, and change history. A consensus quorum orders its changes; an acknowledged version also requires durable chunk copies in at least two regions. If either guarantee cannot be met, the service rejects the write.
The chunk store owns chunks that never change. It keeps the required copies in separate zones, including another region inside the same residency boundary before it confirms a write. A background job later moves old versions to cheaper storage without changing their manifests.
Notification fanout consumes the committed metadata change log through its transactional outbox. Devices recover missed signals from the change log.

The metadata service groups records by driveId. This keeps most operations for one personal drive or shared workspace on the same database shard. As a result, ordering changes and running transactions is simpler.

The transaction and ordering guarantees hold within one drive shard: moves, cycle prevention, placement, ACL inheritance, and optional name uniqueness are strictly serializable multi-record operations. Cross-drive moves and quota changes for another owner need a distributed transaction or reserve/confirm step and are out of scope.

A very large shared drive can split folder listings and event storage, while a compact per-drive sequencer—the logical writer that assigns the next cursor only when a metadata transaction commits—assigns one ordered changeCursor to each committed change. Clients retain that one cursor; event partitions are an implementation detail behind it. This single strict drive cursor is a deliberate teaching simplification: on a huge shared drive it becomes a hot writer. The scale-out path partitions the feed and uses snapshot watermarks plus composite partition cursors; file operations still serialize at their owning shard. Stable IDs are storage keys; paths are metadata.

The cache stores file summaries, folder pages, and permission results. Every cached result includes the version it came from. Effective ACLs may be cached or materialized with the relevant ancestor-ACL and group-membership versions; authorization validates those versions before granting access. Revocations advance or invalidate those versions, and a previously minted download URL can remain useful only until its short expiry.

Final logical quota is checked atomically at commit, not per chunk, because transfers can be abandoned or deduplicated. To prevent rejected uploads from consuming unlimited resources, session creation also enforces per-tenant byte/session limits and can reserve quota for large files. A background job calculates physical storage after deduplication.

Each metadata shard has one chosen write leader and several copies, including the remote copy required for durability. If the leader fails, the copies choose a new one. This gives versions and permissions one clear order without relying on one machine. If too few copies can agree, the system rejects writes instead of claiming they are safe.

API surface

Start with the large-file path: create a session, upload each chunk, then commit. The other rows manage folders, downloads, syncing, and sharing. Every metadata-service call uses TLS, authentication, and authorization. A mutation includes an idempotency key, a client-supplied retry token that returns the original result instead of creating a duplicate.

Important failures are explicit: stale base versions return 409 Conflict, expired upload sessions or sync cursors require a new session or snapshot, invalid manifests/checksums are rejected, and throttled clients back off.

Goal	URL or URL sequence	Input	Output
Create, rename, move, or trash a file or folder	`POST /v1/items` or `PATCH /v1/items/{itemId}`	Create: `{ parentId, name, kind }`. Update: `{ baseMetadataVersion, name?, parentId?, trashed? }`.	`{ itemId, metadataVersion, createdAt }` or `{ itemId, metadataVersion, changeCursor }`. A stale metadata version or a move that creates a loop is rejected.
Upload a small file in one request	`POST /v1/files/{fileId}/content`	Raw file bytes. Headers include `Content-Type`, `X-Base-Version-Id`, `X-File-Checksum`, and `Idempotency-Key`.	`{ fileId, versionId, version }` — the new file version.
Upload a large file in resumable chunks	1. `POST /v1/files/{fileId}/upload-sessions` 2. `POST /v1/upload-sessions/{sessionId}/chunks` 3. `PUT {transferUrl}`	1. File size, type, base version, checksum, and chunking mode. 2. Chunk number, length, and checksum. 3. Chunk bytes.	1. `{ sessionId, chunkingProfile, expiresAt }`. 2. `{ transferUrl, expiresAt }`. 3. `{ chunkId, receivedBytes }`. The client can retry a chunk safely.
Find already stored chunks	`POST /v1/upload-sessions/{sessionId}/chunks:check`	`{ chunks[] }`, with chunk IDs and lengths	`{ present[{ chunkId, leaseReceipt }], missing[] }`
Resume an upload	`GET /v1/upload-sessions/{sessionId}`	Session ID	`{ state, receivedChunks[], expiresAt }`
Make the uploaded file visible	`POST /v1/upload-sessions/{sessionId}/commit`	`{ chunks[], fileChecksum }`, with chunk IDs and lengths in order.	`{ fileId, versionId, version, changeCursor }`. The service checks every chunk before publishing the version; a stale base returns `409 Conflict`.
Create a conflict copy	`POST /v1/upload-sessions/{sessionId}/conflict-copy`	parent and a client-chosen conflict name	The client explicitly commits the retained stale upload as a sibling item.
Download a file	`GET /v1/files/{fileId}/download?version={versionId}`	`fileId`; optional `versionId`.	`{ manifest, transferUrls[], expiresAt }`. The client downloads the chunks, rebuilds the file, and checks its checksum.
Catch up after being offline	`GET /v1/drives/{driveId}/changes?cursor={cursor}&limit={n}`	Drive ID, saved cursor, and optional page size.	`{ changes[], nextCursor, hasMore }`. Save `nextCursor` only after applying the changes locally.
Share a file or folder	`PUT /v1/items/{itemId}/permissions/{principalId}`	`{ role, baseAclVersion }`, where the role is viewer, commenter, editor, or owner.	`{ aclVersion, changeCursor }`. An access-control list (ACL) version refreshes permission caches.

baseVersionId, baseMetadataVersion, and baseAclVersion are safety checks for file contents, file details, and permissions. The server accepts a change only when the relevant value has not changed since the client read it. changeCursor is a private saved position in the change history, not a clock time.

A lease receipt is a short-lived, signed proof that one upload session may reference one verified chunk under stated size and durability constraints. The batch check returns a lease receipt for each reusable chunk, and commit accepts it only while that receipt remains valid. Garbage collection deletes a chunk only when it is unreferenced by retained manifests and has no valid upload lease.

Data model

Record	Important fields	Invariant
`User`	`userId`, subscription tier, home region, `logicalQuotaBytes`, `usedLogicalBytes`	One user owns the quota for personal files
`Device`	`deviceId`, `userId`, platform, `lastAppliedCursorByDrive`, `lastSeenAt`	Each drive cursor moves forward only after local changes are saved
`Workspace`	`driveId`, `ownerId`, sharing mode, home region	Defines who shares data and where its updates are ordered
`Item`	`itemId`, `driveId`, `parentId`, name, file type, `sizeBytes`, owner, creator, timestamps, `currentVersionId`, `metadataVersion`, `aclId`	Stable ID and parent define placement; duplicate names are a product policy
`FileVersion`	`versionId`, `fileId`, `manifestId`, `baseVersionId`, `sizeBytes`, file checksum, creator	Never changes after commit
`Manifest`	`manifestId`, ordered chunk references, total bytes, file checksum	Rebuilds one exact file
`Chunk`	`dedupeScopeId`, SHA-256 hash, byte length, encryption domain, key reference, copy status	The scoped tuple, not hash alone, identifies immutable bytes
`UploadSession`	`sessionId`, expected file details, received chunks, expiry, state	Publishes at most once
`Permission`	`aclId`, user or group, role, inherited-from field, `aclVersion`	Removing access changes its version in one step
`GroupMembership`	`groupId`, principal, `membershipVersion`	Authorization evaluates current membership, not a stale ACL cache
`Change`	feed position, item, operation, resulting version	Saved in order and safe to replay

The subscription tier is the user’s plan, such as free or business. Logical quota is the amount of file data the user is allowed to store, counted by file size—not by how many internal copies the service keeps. usedLogicalBytes is how much of that allowance is currently used.

Client-side sync engine

The client needs a watcher, local database, indexer, chunker, and durable transfer queue. It uses operating-system events for efficiency, but treats them as hints.

Debouncing combines repeated auto-save events before expensive chunking begins.
The local index stores file identity, versions, hashes, and queued work across restarts.
A durable queue retries uploads, while the change feed supplies edits from other devices.

The local-change algorithm debounces rapid events until size and modification time settle, compares stable file identity and metadata with the local index, then classifies create, content update, rename, move, or delete. For a content update it records chunk work and the future metadata commit in a durable queue before network work begins. After commit it saves the new base version, continues from its old cursor, and safely skips its echoed change. At startup, watcher overflow, or suspected loss, it reconciles only the affected area against the local index.

Platform	Change mechanism	Recovery concern
Windows	USN Journal with `ReadDirectoryChangesW` as a live hint	The journal supports durable recovery; the live API can overflow and needs a targeted rescan
macOS	FSEvents	Rescan a changed subtree
Linux/Android	`inotify` / `FileObserver`	Watch limits and suspended-app wake-up
iOS/iPadOS	File Provider / `NSFilePresenter`	Sandbox and background limits

The client limits parallel chunks, reduces concurrency on weak links, resumes confirmed ranges, and reuses connections. Compression is conditional and layer-specific: Gzip/Brotli applies to client-to-gateway transfer of compressible text; Zstandard/Snappy applies to speed-sensitive internal replication or metadata storage; LZMA applies only to cold internal archives. Skip recompression for already-compressed or encrypted bytes.

Names need platform-independent rules too. The metadata service canonicalizes Unicode names to NFC, while macOS clients tolerate filesystem-provided NFD and compare the canonical form. Windows-style case-insensitive folders reject names that collide after case folding. Finally, many editors save by writing a temporary sibling and then renaming it over the original; debouncing plus stable file identity lets the client treat that sequence as one replacement instead of syncing a transient file.

Write path

The write flow waits to publish the file until its chunks and metadata are durable. A storage event alone never changes the current file version.

Chunk retries are safe because their identities and checksums remain stable.
The commit validates durable receipts before changing the visible version pointer.
Change fanout starts from the committed log entry, never from a chunk-store callback.

The client hashes chunks and the whole file, batch-checks eligible chunks, uploads only misses in parallel, and retries safely. The chunk-store gateway verifies each received digest and length before confirming the required copies across separate zones and the remote region. Commit checks ordered chunk receipts, quota, and the base version. Success requires both a metadata consensus quorum to order the transaction and receipts proving full-data chunks in at least two regions; consensus alone is not a cross-region byte-durability acknowledgement. The transaction then saves the manifest, advances the current-version pointer, and writes the change record before success returns. Notifications are later, never part of commit.

An upload starts as pending. While the service checks and publishes it, it is committing. After the database transaction succeeds, it is committed. A session that waits too long becomes expired.

The idempotency key identifies one upload attempt. If the client retries the same request, the service returns the existing session or its saved result instead of starting another upload. Only one commit can publish the manifest. If a worker fails before the database transaction, a retry can safely finish the work. If it fails after the transaction, a retry returns the committed version. A timeout therefore never creates two versions or leaves the client guessing what happened.

Direct upload does not give the client general access to storage. Each temporary URL allows one operation on one expected chunk or byte range. It limits the size, checksum, account, and expiry time. For CDC, the client gets the next URL only after it knows that chunk’s boundary. Storage rejects anything outside those rules, so a stolen URL cannot read arbitrary files or write unrelated objects.

Read and sync paths

For a download, the service checks permission and loads the requested Item and FileVersion. It reads the manifest and returns short-lived chunk URLs. The client downloads chunks in parallel, verifies each hash, rebuilds the file, and checks the whole-file checksum. If one replica returns bad bytes, storage tries another copy and schedules a repair.

The server can stream small files through an edge cache, but large downloads should remain direct. Range requests map byte offsets to manifest entries, allowing video seeking or partial recovery without assembling the whole file in a server. Popular public files need a separate abuse and content-delivery policy so one shared link cannot exhaust the owner’s metadata shard. The authorization decision can mint a cacheable signed response while the bytes come from a content delivery network.

For sync, push or a long-held connection wakes the client; it applies GET /drives/{driveId}/changes pages in cursor order, then persists the new cursor. If retention has expired, the server chooses one history position, returns a snapshot valid at that position, and returns the exact cursor from which replay resumes. Without that shared boundary, a change made during snapshot transfer could be missing from both the snapshot and the later feed. Clients can safely replay already-applied changes, but must never advance their cursor past unapplied changes.

Per-drive cursors cannot discover a drive that was newly shared with a user. A separate per-user access feed records drive grants, revocations, and individually shared items. On a grant, the client creates that drive’s first cursor and syncs it; on a revocation, it removes local access. Long polling suits one-way hints, WebSockets suit frequent two-way messages, and mobile push saves battery—but every option falls back to periodic feed polling.

Every metadata read and download authorization evaluates the ACL, its ancestor versions, and current group membership. Permissions can target users, groups, domains, or links; roles are viewer, commenter, editor, and owner. Children inherit folder ACLs unless overridden. Moves are therefore security operations: caches include ACL and membership versions, and any minted URL lasts only until short expiry. Permission and membership changes enter the audit log.

Security layer	Design
External transport	TLS 1.3
Internal transport	Mutual TLS with short-lived service identities
File chunks and metadata	AES-256 envelope encryption with keys in a managed key service
Enterprise keys	Optional customer-managed encryption key reference per workspace
Audit	Append-only records for reads, writes, sharing changes, deletes, and restores

Caching follows the cost of being stale rather than one universal rule.

Cached data	Freshness policy	Reason
Metadata and ACLs	Versioned entries; compute ACLs from ancestors at authorization time	Cached transfer authorization can remain stale only for the short-lived URL
Thumbnails and previews	30–60 second time to live	Regeneration is expensive, while brief visual staleness is acceptable
Client sync state	Event-driven invalidation plus cursor polling	Push gives speed; the durable feed prevents missed changes

Versions, quota, and retention

Every committed edit creates an immutable FileVersion; restoring one creates a new current version pointing to its manifest. Retain the newest 100 versions or 30 days, whichever preserves more, plus explicitly pinned versions. Retained versions count toward the owner’s logical quota, never viewers’ accounts or deduplicated physical usage. These are product policies, not Google measurements.

Deep dive: three hard sub-problems

1. Publishing bytes without partial files or acknowledged loss

The dangerous gap sits between storing bytes and changing metadata. If metadata points to missing chunks, the file is corrupt. If chunks exist but metadata commit fails, capacity leaks but user data remains safe. The design must prefer the second outcome.

Candidate	Strength	Cost or failure mode
Write bytes through one database transaction	Simple conceptual commit	Databases are poor large-blob transfer engines; locks and logs become enormous
Upload mutable object, then rename it	Familiar file-system model	Object stores do not generally provide a cross-system atomic rename with metadata
Immutable chunks plus transactional manifest publish	Readers see one complete version; retries are safe	Requires orphan collection, reference accounting, and two storage systems

Choose immutable chunks and publish a manifest only after receipts prove the required durable copies. The transaction checks receipts, saves the manifest, advances the version pointer, and writes the change record. Failed sessions leave only invisible data for later cleanup.

Garbage collection uses two passes: mark from retained manifests and quarantine apparent orphans; then, after the longest replication and backup delay, recheck references and upload leases before deletion. A counter alone is unsafe after missed events or a restored backup. Background scrubbing verifies hashes, repairs bad copies, and periodically rebuilds sample files from independent replicas to compare their whole-file checksums. Track detected damage, successful repairs, unrecoverable objects, and the age of the last successful check separately. Acknowledge only after the required zone copies, remote regional copy, and metadata transaction are durable; this deliberately adds write latency. The versioned whole-file checksum catches valid-but-misordered, duplicated, or missing chunks.

2. Chunking strategies: moving only the bytes that changed

Candidate	Strength	Cost or failure mode
Whole-file upload	Few requests and minimal client CPU	Terrible retry cost; no delta transfer
Fixed 4–16 MiB chunks	Predictable memory, parallelism, and offsets	Insertion near the front shifts later chunks
Content-defined chunks with min/target/max sizes	Preserves boundaries around insertions; better deduplication	More CPU, variable parts, and abuse controls needed

Fixed-size chunking uses predictable 4–16 MiB offsets: boundary choice is O(1), though hashing the file remains O(n). It supports parallel upload and range reads, but an early insertion shifts later chunks. CDC cuts when a rolling fingerprint matches a mask, so boundaries usually realign after an insertion; it suits frequently edited VM images, snapshots, and similar binaries.

Use fixed chunks for videos, encrypted archives, and other low-reuse formats; use CDC for large, frequently edited files. Minimum/target/maximum sizes prevent tiny-chunk abuse. A Rabin fingerprint can find boundaries in O(1) work per window, but SHA-256 identifies the final chunk. Version and constrain server-approved chunking profiles in the manifest so upgrades cannot break resume or trigger millions of lookups.

for each byte window:
    rolling_hash = remove(old_byte) + add(new_byte)
    if chunk_size >= minimum and rolling_hash & mask == 0:
        cut chunk
    if chunk_size == maximum:
        force cut

The manifest records which chunking algorithm and settings were used. Without that information, a client upgrade might split the same file differently and break upload resume or deduplication. The server should support a few versioned profiles instead of accepting any client-supplied setting. It can reject an unreasonable manifest before performing millions of chunk lookups.

The upload decision stays observable and deterministic. This flow avoids pretending that one chunker fits every format.

Small files avoid multipart overhead, but a failed one-request upload starts over.
Fixed chunks favor predictable throughput; content-defined chunks favor reuse after edits.
Every path converges on cryptographic hashing and upload of only missing chunks.

Compress before encryption only when sampling predicts a benefit; JPEG, MP4, ZIP, and many office files are already compressed. Deduplicate only inside an allowed tenant boundary and encryption domain, using the scoped chunk identity defined above. With server-side encryption, the service can deduplicate validated plaintext references before storing one encrypted representation. Randomized ciphertext cannot itself deduplicate; convergent encryption can, but creates confirmation-of-file risks. End-to-end-encrypted clients therefore normally forgo server-side deduplication. Never expose cross-tenant chunk existence through timing, quota, billing, or a client-visible lookup.

3. Handle offline edits and conflicts

An offline device must replay missed changes and never silently erase its own edit. The same transaction that changes metadata writes the ordered feed entry; notifications only prompt a read. A client persists its cursor only after applying a page, and cursor_expired triggers a snapshot plus fresh cursor.

Edits carry their base version. A match commits; a mismatch returns 409 Conflict with the current version and leaves the upload session available. The client fetches the current content, merges when it can, or explicitly creates a sibling conflict copy from the retained session; the server never silently creates one. Timestamps are display data, not ordering. A single ordered metadata writer does not need version vectors for server-side ordering, though version vectors can still help client-side causality or conflict diagnosis. For structured text, merge only with domain-aware rules; collaborative documents use OT/CRDT rather than attempting to merge arbitrary ZIP, image, or video bytes.

A matching base version takes the normal commit path.
A collaborative document goes to its collaboration service.
A binary conflict or delete-versus-edit race keeps recoverable content instead of discarding bytes.

For binary files, the first accepted edit keeps the original name. After 409 Conflict, the client can explicitly commit the retained stale upload as a sibling item, for example Design (Asha's conflicted copy 2026-09-20).pptx, linked to its original item for cleanup and user-visible resolution. Renames, moves, edits, and deletes use separate version checks because they change different parts of a file’s record. A delete becomes a temporary marker, called a tombstone, so an offline edit can still be recovered.

Concurrent operations	Policy
Content edit and content edit	The first successful edit stays primary; the stale commit returns `409`; the client can create a conflict copy
Rename and content edit	Check the rename and the content edit separately. Apply both when both checks pass; reject only the stale change
Delete and content edit	Keep a tombstone for the delete. Return `409`; the client can preserve its stale bytes as recoverable conflicted content instead of silently restoring the file
Folder move and child update	Check the folder move and child update separately. Apply both when valid; reject only an invalid destination or a cycle
Two folder moves	The first move chooses the parent; return a conflict for the later stale move

Moving a folder changes the visible path of every file below it. Rewriting every child record would be slow and risky. Instead, store stable parent IDs, reject moves that create a cycle, and write one subtree-move event. New clients can understand that event directly. A compatibility worker can expand it into individual changes for older clients.

Full high-level design

Now that the client sync engine, chunking, safe publication, and conflict handling are established, this end-to-end view connects them to the metadata authority. Solid paths carry synchronous requests; the dashed notification is only a wake-up signal, so every client still catches up from the durable change feed.

Bottlenecks and failure modes

Situation	What breaks first	Mitigation
Normal growth	Hot metadata partitions for huge shared folders or celebrity links	Hash-shard memberships, paginate stable snapshots, and isolate download authorization from listing
Upload burst	Session creation, token signing, and chunk-store ingress saturate	Preallocate upload-session capacity, rate-limit per tenant, apply backpressure, and let clients use exponential retry
Notification outage	Connected devices stop learning about changes quickly	Clients poll the durable feed with jitter; rebuild fanout from retained log positions
Metadata cache serves stale ACL	Revoked users could retain apparent access	Compute ACLs from ancestors at authorization time; any already minted transfer URL remains usable only until its short expiry
Zone loss during upload	Some chunk writes lose replicas or stall	Quorum across zones, retry another endpoint, and refuse commit until every receipt meets durability policy
Region isolation	Strong cross-region metadata writes may stop	Route home-region traffic to a surviving quorum or enter read-only mode; never accept unreplicated writes as durable
Silent corruption	A replica returns incorrect bytes	End-to-end checksums, background scrubbing, alternate-replica reads, and automatic repair
Ten-times traffic	Metadata leaders, change partitions, repair bandwidth, and storage fleet all face nonlinear pressure	Split by drive, separate foreground and repair budgets, add regions, and move cold versions to erasure-coded tiers

Treat garbage collection as a data-loss risk: mark from committed manifests, quarantine apparent orphans, wait through replication and backup delay, then recheck before deletion. Uneven traffic hurts first—large workspaces, popular files, and retry storms need tenant limits and separate user, repair, and cleanup pools. At 10× load, shard metadata and feeds, add storage/regions, and reserve disk and network for repair. During an outage, degrade features rather than guarantees: serve safe reads without metadata majority, limit commits after zone loss, and let clients poll through notification failures.

Reference technology choices

These are replaceable examples, not claims about Google Drive’s implementation.

Concern	Options	Why
Metadata store	PostgreSQL/MySQL for a shard; Spanner, CockroachDB, or a Vitess-backed relational service at larger scale	Standard SQL is a strong starting point; partition by `driveId` and accept application-managed sharding, failover, and cross-shard trade-offs as scale grows
Change feed and outbox	Transactional outbox to Kafka or Pub/Sub	An outbox is a change record written with metadata, then delivered independently and replayably
Cache	Redis or Memcached	Fast, versioned metadata and effective-ACL cache entries
Chunk store	S3, GCS, or a distributed file store	Durable immutable chunks and direct, signed transfers
Wake-up channel	WebSockets plus FCM/APNs	Low-latency hints for connected and mobile clients; the feed remains authoritative

Technologies and techniques worth exploring

Rabin fingerprints and FastCDC — understand how content-defined boundaries localize edits without using a cryptographic hash at every byte.
Merkle trees — compare large local and remote manifests and narrow integrity checks to mismatched subtrees.
Raft and consensus-backed transactions — see why metadata leaders provide ordered, durable state changes and what partitions cost.
Erasure coding — reduce cold-storage overhead while quantifying repair bandwidth and read amplification.
Transactional outbox pattern — publish change events without a fragile dual write between a database and a message broker.
End-to-end data-integrity checking — follow checksums through memory, network, storage, replication, and restore rather than trusting one layer.

Key takeaways

Immutable chunks make retries, replication, and repair tractable.
A transactional manifest commit is the visibility boundary.
Direct, resumable transfer keeps large file bytes out of the general API path.
Checksums detect corruption, while independent replicas and tested repair recover from it.
The storage and repair fleet, not metadata QPS, determines the physical scale.

The governing contract is simple to state and expensive to honor: once a version is acknowledged, every later layer must preserve enough verified information to find, reconstruct, authorize, and restore those exact bytes.

Quiz

Drive design challenge

Can you keep every byte safe?

Five failure scenarios. Make the call before you reveal the answer.

1 / 5

Question 1 of 5

All chunks reached storage, but the client disconnected before commit. What should readers see?