Ai Application Security

We are excited to announce the release of the OWASP AI Security Center of Excellence (CoE) Guide! Developed by the dedicated OWASP Top 10 for LLMs and Generative AI Security Project team, this guide is designed to help organizations address the complex challenges of adopting and securing generative AI applications and workloads.

As AI technologies evolve and are increasingly integrated into business operations, there is an urgent need for organizations to establish governance structures that ensure secure and ethical AI deployments. The OWASP AI Security CoE Guide provides a comprehensive business framework to enhance existing Centers of Excellence, bringing together cross-functional leadership from cybersecurity, legal, data science, operations, and end-user engagement teams.

Key benefits of the guide include:

• Enhancing Security Frameworks: Provides a roadmap to develop security protocols tailored to generative AI technologies, ensuring that your organization can confidently meet regulatory and compliance requirements.

• Cross-Departmental Collaboration: Helps bridge communication across business units, fostering collaboration that ensures generative AI technologies are securely and ethically deployed.

• Risk Management and Policy Development: Equips teams with strategies to identify, assess, and mitigate AI-related risks while aligning with broader business goals.

• Training and Awareness: Offers guidance for building internal training programs to educate teams on the secure use of AI technologies.

This guide, built by a global community of cybersecurity professionals and CISOs, is essential for any organization looking to harness the power of AI responsibly. It addresses both the technical and strategic aspects of generative AI adoption, ensuring alignment across various lines of business.

We invite cybersecurity leaders, cross-functional leadership teams, and business groups to leverage this guide as part of their AI initiatives to stay ahead of emerging security threats and foster innovation in a secure environment.

# Recommendation(s)

Large Language Model (LLM) Security Misconfiguration

Overview of the Vulnerability

Misconfigurations can occur across Large Language Model (LLM) within the setup, deployment, or usage of the LLM, leading to security weaknesses or vulnerabilities. These misconfigurations can allow an attacker to compromise confidentiality, integrity, or availability of data and services. Misconfigurations may stem from inadequate access controls, insecure default settings, or improper configuration of fine-tuning parameters.

Business Impact

This vulnerability can lead to reputational and financial damage of the company due an attacker gaining access to unauthorized data or compromising the decision-making of the LLM, which would also impact customers' trust. The severity of the impact to the business is dependent on the sensitivity of the accessible data being transmitted by the application.

Steps to Reproduce

1. Navigate to the following URL:

2. Inject the following prompt into the LLM:

  {malicious prompt}

1. Observe that the LLM returns sensitive data

Proof of Concept (PoC)

The screenshot(s) below demonstrate(s) the vulnerability:

{{screenshot}}