Generative AI Security Scoping Matrix

From a legal perspective, it’s important to understand both the service provider’s end-user license agreement (EULA), terms of services (TOS), and any other contractual agreements necessary to use their service across Scopes 1 through 4. For Scope 5, your legal teams should provide their own contractual terms of service for any external use of your models. Also, for Scope 3 and Scope 4, be sure to validate both the service provider’s legal terms for the use of their service, as well as the model provider’s legal terms for the use of their model within that service.

Additionally, consider the privacy concerns if the European Union’s General Data Protection Regulation (GDPR) “right to erasure” or “right to be forgotten” requirements are applicable to your business. Carefully consider the impact of training or fine-tuning your models with data that you might need to delete upon request. The only fully effective way to remove data from a model is to delete the data from the training set and train a new version of the model. This isn’t practical when the data deletion is a fraction of the total training data and can be very costly depending on the size of your model.

While AI-enabled applications resemble traditional ones, their interactive nature with large language models (LLMs) demands extra vigilance and specific guardrails. It's crucial to identify risks associated with generative AI workloads and start mitigating them.

Risk identification can generally be done through risk assessments and threat modeling. For Scopes 1 and 2, evaluate the risks stemming from third-party providers and understand their risk mitigation strategies. You must also recognize your own risk management responsibilities as a service consumer.

For Scopes 3, 4, and 5, implement threat modeling that addresses both AI safety and data security risks, with a focus on unique LLM threats like prompt injection. This involves crafting inputs that can manipulate LLM responses, potentially leading to data breaches or unauthorized access. Recent guidance from NIST, MITRE, and OWASP identifies prompt injection as a primary threat, comparable to traditional injection attacks like SQL. Mitigate this risk by applying fine-grained authorization and data filtering before it reaches the LLM, and use Bedrock Guardrails for additional protection.

Emerging threats in generative AI require adapting traditional cybersecurity measures and collaborating closely with development teams to effectively model threats and establish tailored best practices.

Implementing robust controls is crucial for enforcing compliance, policy, and security requirements, thereby mitigating risks associated with generative AI workloads. One of the key considerations is managing identity and access to your models. Unlike traditional databases that offer fine-grained security controls, foundation models have no concept of access control to data stored within the model or data provided at inference time. This makes it essential to implement least privilege access control to data before it’s added as context into the inference request.

To achieve this, you can leverage application layers that interact with the model through endpoints like Amazon Bedrock. These layers should incorporate identity solutions such as Amazon Cognito or AWS IAM Identity Center to authenticate and authorize users. By tailoring access based on roles, attributes, and user communities, you can limit specific actions and help ensure that sensitive data is protected.

Additionally, as your AI workloads evolve, it’s important to continually assess and update your controls to adapt to new threats and changes in data sensitivity. Incorporating advanced techniques such as Retrieval-Augmented Generation (RAG) can allow you to provide real-time data without compromising the integrity of the model. These strategies, combined with ongoing threat modeling, help to maintain a secure and compliant environment for your generative AI applications. 

Availability is a key component of security as called out in the C.I.A. triad. Building resilient applications is critical to meeting your organization’s availability and business continuity requirements. For Scope 1 and 2, you should understand how the provider’s availability aligns to your organization’s needs and expectations. Carefully consider how disruptions might impact your business should the underlying model, API, or presentation layer become unavailable. Additionally, consider how complex prompts and completions might impact usage quotas, or what billing impacts the application might have.

For Scopes 3, 4, and 5, make sure that you set appropriate timeouts to account for complex prompts and completions. You might also want to look at prompt input size for allocated character limits defined by your model. Also consider existing best practices for resilient designs such as backoff and retries and circuit breaker patterns to achieve the desired user experience. When using vector databases, having a high availability configuration and disaster recovery plan is recommended to be resilient against different failure modes.

Instance flexibility for both inference and training model pipelines are important architectural considerations in addition to potentially reserving or pre-provisioning compute for highly critical workloads. When using managed services like Amazon Bedrock or SageMaker, you must validate AWS Region availability and feature parity when implementing a multi-Region deployment strategy. Similarly, for multi-Region support of Scope 4 and 5 workloads, you must account for the availability of your fine-tuning or training data across Regions. If you use SageMaker to train a model in Scope 5, use checkpoints to save progress as you train your model. This will allow you to resume training from the last saved checkpoint if necessary.

Be sure to review and implement existing application resilience best practices established in the AWS Resilience Hub and within the Reliability Pillar and Operational Excellence Pillar of the Well Architected Framework.