Designing Enterprise Data-Source Connectors

A practical structure for building and operating connectors across Azure Blob, Salesforce, Box, Dropbox, and other enterprise sources.

Dec 18, 2025 1 min read Data Security

Problem

Enterprise connectors look simple from the outside: authenticate, list objects, fetch metadata, and emit records. In production, the hard work is making each connector resumable, observable, rate-limit aware, and consistent with the rest of the platform.

Connector Responsibilities

Discover roots, containers, folders, users, groups, and objects.
Persist traversal cursors so scans can resume after failures.
Normalize provider-specific metadata into a stable internal model.
Emit changes for downstream governance, analytics, and response workflows.
Surface enough operational detail to debug tenant-specific failures.

Connector Shape

scan scheduler -> source connector -> cursor store
                         |              |
                         v              v
                 normalized metadata -> analytics / governance

Production Notes

Connectors need source-specific adapters, but the rest of the platform should see consistent records. That boundary keeps downstream analytics, governance workflows, and incident response features from inheriting every provider’s API quirks.