Data Governance, Quality Checks, and Retention

Every delivery passes through automated quality checks, governance rules that lock in canonical identifiers, and retention policies that define how long your files and logs are kept. You do not need to operate any of this, but knowing it exists helps when planning your own archival and compliance workflows.

Data quality checks

VidScore runs 17 declarative data quality rules on every batch before delivery. The rules cover completeness (required fields populated), validity (values in expected ranges and formats), uniqueness (no duplicate video_ids within a delivery), freshness (analysis timestamp within the delivery window), consistency (cross-field sanity checks), volume (batch size within expected range), and distribution (no obvious skew toward a single creator or industry). A weekly automated quality report runs every Sunday at 9 AM UTC and is emailed to the VidScore alerts inbox. A data observability log records check results over time so regressions are detectable, and hourly health checks monitor scan failures, Stripe events, Redis, queue backlog, and LLM cost anomalies. You benefit from the result: no malformed records reach your feed.

Governance rules

Every analysis is versioned. The schema_version field on every record identifies which version of the pipeline produced it, so you can tell old records from new ones when comparing across time. Canonical names are locked at first detection and subsequent mentions normalize back to the canonical form, so "iPhone 15 Pro", "iphone 15 pro", and "iPhone15Pro" all resolve to a single product entity. Slugs on published entities are locked once the entity goes public; if a name changes, the old slug becomes an alternate_slug that redirects. Quality scores are recalculated when new mentions are added to an entity and are never manually set. Every API request (on tiers where the API is available) is logged for billing and audit.

Retention policy

Delivery files are kept for 90 days from delivery date and then purged from storage. Archive to your own storage if you need longer retention. Portal activity logs are retained for 365 days per organization and auto-pruned; admins can export to CSV at any time. Team member data is retained while the membership is active, and when a member is removed their identity is detached from the membership row. Analysis cache (analyzed video data) is retained indefinitely so repeat scans are free, this is shared infrastructure rather than per-customer data. SSO configuration metadata is retained while SSO is active for your organization and is deleted on tier downgrade or manual removal. Audit events are insert-only at the database level; they cannot be modified or deleted even by an admin.