FDA builds secure AI data platform on Databricks

The U.S. Food and Drug Administration (FDA) has transitioned its data operations to Databricks on AWS GovCloud to launch its Harmonized AI and Lifecycle Operations for Data (HALO) platform. This enterprise-grade system uses Unity Catalog to establish a unified, secure, and governed foundation for AI and analytics. The agency successfully migrated more than 5,000 users, 8,000 jobs and pipelines, 1,000 data pipelines, and 4,000 notebooks with zero downtime. The platform has already scaled from 500 users in 2020 to over 6,000 today, with expectations to exceed 10,000 users by 2028.

By consolidating more than 40 data sources across eight centers and 30 programs, the FDA achieved significant operational improvements. The agency reported a 30% increase in SQL query speeds for business intelligence workloads and a 20% reduction in compute costs. Additionally, the time required for data provisioning, permissioning, and data sharing dropped by 75%, while overall operational overhead fell by more than 35%. The system leverages Databricks serverless compute, model serving, and Genie to convert hundreds of legacy dashboards into interactive, AI-powered tools.

For AI practitioners, this migration demonstrates how to deploy highly regulated workloads within FedRAMP High and DoD IL5 environments. The FDA integrated HALO with Elsa, its enterprise AI platform, to support initiatives like MARS (Modernizing and Accelerating Regulatory Submissions). MARS assists reviewers in analyzing massive volumes of structured and unstructured data from drug and device applications. By utilizing Unity Catalog as an open governance layer, practitioners can securely manage and share assets across different regions and formats without losing auditability. This setup proves that strict federal compliance and rapid AI deployment can coexist.