Risk Assessment Worksheets

Checks that a model meets clinical safety standards and keeps producing validated outcomes once it is in front of patients. Questions press on whether the system was validated against ground truth representative of your actual population, whether the distinction between advisory and autonomous use is written down, and whether clinicians ever lose the ability to override a recommendation.

• Validation against representative clinical ground truth
• Advisory vs autonomous decision authority
• Confidence scores and uncertainty shown to clinicians
• Human override on every recommendation
• Edge-case testing for rare and underrepresented cohorts
• Outcome drift monitoring and decommissioning criteria

Walks the full lifecycle of protected health information, from how it is encrypted and classified to how it leaves the building. It asks whether keys are organization-managed rather than provider-default, whether de-identification happens before data reaches a training pipeline, and whether model outputs that point to an identifiable patient are themselves treated as PHI.

• Encryption at rest with organization-managed keys
• TLS 1.2+ between every AI component
• De-identification before model training
• Business Associate Agreements with all PHI processors
• Data residency and retention enforcement
• Patient access, correction, and deletion handling

Looks at who actually owns AI decisions inside the organization. It covers whether a body with real authority signs off on deployments, whether a model registry tracks version, owner, and approval history, and whether retraining runs through change management instead of landing in production unannounced. Patient notification and a working escalation path are part of the same picture.

• Governance committee with deployment authority
• Mandatory pre-production risk assessment
• Model registry with version and approval history
• Defined roles across the model lifecycle
• Change management for updates and retraining
• Escalation path for raised concerns

Tests whether clinical work keeps moving when the AI does not. It asks for an availability target tied to the clinical workflow, documented manual fallback for when the system is down, and an architecture without a single point of failure. Deployment safety shows up here too, through canary or blue-green rollouts and an automated rollback when a new model drops below threshold.

• Availability SLA/SLO aligned to clinical need
• Documented manual fallback procedures
• No single point of failure in the architecture
• Health, latency, and error-rate alerting
• Tested disaster recovery for models and data
• Automated rollback on performance degradation

Focuses on the attack surface that traditional security reviews tend to miss. It covers adversarial testing for evasion, poisoning, and model inversion, prompt injection and jailbreak defense for LLM-based systems, segmentation between training, inference, and clinical data, and verification that the model supply chain has not been tampered with before anything ships.

• Authentication, authorization, and rate limiting on endpoints
• Adversarial testing for evasion, poisoning, inversion
• Prompt injection and jailbreak defense
• Segmentation of training, inference, and data stores
• Image and dependency vulnerability scanning
• Supply chain integrity verification

Examines whether the system performs evenly across the people it serves and whether it could quietly widen existing disparities. It asks for performance broken out by race, ethnicity, gender, and age, checks for proxy discrimination where correlated features stand in for protected attributes, and looks for the fairness metrics your clinical stakeholders actually agreed to, such as equalized odds or calibration.

• Performance disparity testing across demographics
• Training data representative of the served population
• Proxy discrimination evaluation
• Accessibility for patients with disabilities
• Language diversity in the patient population
• Agreed fairness metrics and emergent-bias monitoring

Asks whether the people relying on the system can actually understand and trace it. It covers explanations clinicians can act on, a complete audit trail from input through inference to output, and honest documentation of purpose and failure modes. It also reaches into third-party and foundation model risk, staff training, and the energy footprint of training and inference.

• Decision logic explainable to clinicians
• End-to-end auditable inference trail
• Documented purpose, scope, and failure modes
• Third-party and foundation model monitoring
• Staff training on capabilities and limits
• Environmental impact assessment