* [Blog](https://origin-researchcenter.paloaltonetworks.com/blog) * [Cloud Security](https://origin-researchcenter.paloaltonetworks.com/blog/cloud-security/) * [AppSec](https://origin-researchcenter.paloaltonetworks.com/blog/cloud-security/category/appsec/) * Security Theater: Your Ap... # Security Theater: Your AppSec Success Metrics Are Misleading [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Forigin-researchcenter.paloaltonetworks.com%2Fblog%2Fcloud-security%2Fsucess-measurements-security-theater%2F) [](https://twitter.com/share?text=Security+Theater%3A+Your+AppSec+Success+Metrics+Are+Misleading&url=https%3A%2F%2Forigin-researchcenter.paloaltonetworks.com%2Fblog%2Fcloud-security%2Fsucess-measurements-security-theater%2F) [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Forigin-researchcenter.paloaltonetworks.com%2Fblog%2Fcloud-security%2Fsucess-measurements-security-theater%2F&title=Security+Theater%3A+Your+AppSec+Success+Metrics+Are+Misleading&summary=&source=) [](https://www.paloaltonetworks.com//www.reddit.com/submit?url=https://origin-researchcenter.paloaltonetworks.com/blog/cloud-security/sucess-measurements-security-theater/&ts=markdown) \[\](mailto:?subject=Security Theater: Your AppSec Success Metrics Are Misleading) Link copied By [Cameron Hyde](https://www.paloaltonetworks.com/blog/author/cameron-hyde/?ts=markdown "Posts by Cameron Hyde") Aug 20, 2024 7 minutes [AppSec](https://www.paloaltonetworks.com/blog/cloud-security/category/appsec/?ts=markdown) [Cloud Security](https://www.paloaltonetworks.com/blog/category/cloud-security/?ts=markdown) [AppSec](https://www.paloaltonetworks.com/blog/tag/appsec/?ts=markdown) [Security Theater](https://www.paloaltonetworks.com/blog/tag/security-theater/?ts=markdown) Welcome to Act 3 of our security theater blog series, where we seek to shed light on illusions in AppSec. In our previous post in this series, [Security Theater: Who Cares About Your AppSec Findings?](https://www.paloaltonetworks.com/blog/prisma-cloud/vulerability-findings-security-theater), we explored how to get the most from your AppSec findings. Today we look at how measurements of success can be misleading. ## **Act 3, Scene 1:** The Metrics Game Different types of metrics lend themselves to being gamified. After all, when held to a certain metric, human nature wants to tailor our performance to meet or exceed expectations. But what happens if the metrics used to measure success don't align with business goals? Could they sabotage business outcomes? In a word, yes. Imagine you own a bakery, for example. You measure your bakers by how many pies they make per day, rather than how many pies they sell. Your bakers naturally focus on producing pies, regardless of sales, and you end up with unsold inventory. To prevent your bakery from eventually going under, you might want to pick a better metric, one that incentivizes the number of pies sold per day. ## Pitfalls of Measuring Success in Cloud Security We can all agree that how you measure the success of tools is essential to the security posture of your organization. Let's say, for instance, you activate a new security tool and immediately receive alerts about numerous vulnerabilities, misconfigurations and risks in your cloud environment. That's good, right? Maybe. Identifying security risks is critical, but a larger number of findings doesn't always equate with better security. Zero in on the tool's results. Consider asking the following questions: * Do we have false positives? * Could we have false negatives and are missing vital information? * Do we have so many findings that we need a new team to address them? * Are the insights provided actionable? * How is risk prioritized? * Are the insights correlated? * Is business criticality identified? * Are we fixing issues at their source? How you answer these questions is key to evaluating the success of the security tool, as each answer provides an insight into the tool's accuracy, efficiency and value. ## Commonly Gamified Metrics ### Total Alerts and Number of Issues Remediated The idea of focusing on the total amount of findings or alerts and measuring success by the number remediated epitomizes what we've been calling *security theater* --- something that might look good on the surface but is almost irrelevant in practice. The rate of remediation is too easily gamed. Put yourself in the developers' shoes. If you measure their success by the number of issues fixed --- when their job is to build and ship code as fast as possible --- they're going to grab the low-hanging fruit and pass over complex issues. Will that give you the most effective security resolution? Probably not, given that your developers are likely fixing code weaknesses in nonreachable functions or similar issues that don't ultimately matter. How much better it would be to measure the number of critical vulnerabilities or attack paths (interconnected risks) remediated. Ask yourself, does this metric align with our business goals? Is it the best metric to help us achieve our business goals? ### Mean Time to Remediate (MTTR) MTTR is a commonly used success metric, but it too can be misleading. On average, [it takes 145 hours to remediate an alert](https://www.paloaltonetworks.com/resources/research/unit-42-cloud-threat-report-volume-7). Organizations must consider what defines a remediated or fixed issue. Security teams might define success by how quickly they can ship an alert off to a developer. Alert resolved, right? But does your organization track whether the developer actually fixes the issue? Does it get immediate attention or is it added to the next sprint? Is a [web application firewall (WAF)](https://www.paloaltonetworks.com/cyberpedia/what-is-a-web-application-firewall) deployed or is the issue remediated in code? How do you track issues sent to developers but never resolved? Keep in mind that development and security teams typically have distinct success metrics. For MTTR to be effective, it must involve both security and development to ensure that success is based on the total time from the alert to the actual fix. ### Securing the OWASP Top 10 You should absolutely secure your applications against the OWASP Top 10 Security Risks. You shouldn't, however, try to retrofit the OWASP Top 10 into a success metric. First, OWASP could easily have composed a list of 100 security risks. Think of the Top 10, in other words, as a set of benchmarks (i.e,. CIS benchmarks), in that it's designed for automation. And refer back to [Security Theater Act 1](https://www.paloaltonetworks.com/blog/prisma-cloud/compliance-security-theater/) where we talked about compliance standards and how they tend to create a false sense of security. Approaching the OWASP Top 10 --- or any guideline --- with a checklist mentality is to mistake a guardrail for the goal. When you evaluate a tool, ensure that it's highly accurate and combines insights accounting for multiple factors, including business criticality. Doing so will help you prioritize risk and focus remediation efforts on the most impactful issues. It will also allow you to base your organization's success on efficiently remediating high-priority risks. Even better, aim to confidently answer this question: If I had only 15 minutes, what can I do that will have the greatest impact? ## **Act 3, Scene 2:** What Defines a Fix? Let's get this out of the way. A patch is not a remediation, nor does a WAF solve your underlying problem. Yes, both are effective security measures in the immediate. But neither solves the underlying issue with your application. Make no mistake --- how you define a fix or remediation is critical to successful [AppSec](https://www.paloaltonetworks.com/cyberpedia/appsec-application-security). You should not consider MTTR complete with the installation of a patch or WAF. You also shouldn't consider it complete once security teams open a Jira ticket and lob it over to a developer. A fix should encompass a developer fixing the issue at its source, obtaining approval for the fix and redeploying the application. Tracking MTTR effectively means tracking every stage. * Alert discovered * Security sends PR or ticket to developer * Developer sees PR * Developer fixes issue * Fix approved * Application redeployed In those stages is your definition of MTTR. How long did it take? That's the metric to benchmark and track. ## \*\*Act 3, Scene 3:\*\*Learning from Our Mistakes Are we learning from these security tools and metrics? With new-found knowledge, teams should focus on preventing risk and educating developers about security. Preventing risk from the beginning is the only way to stay on top of AppSec. Three key metrics to track that will amplify your AppSec program are: 1. Are we shifting left and seeing fewer issues in runtime? 2. When we find problems, where in the pipeline are we finding them? 3. Are developers adopting IDE and VCS integrations to fix issues immediately at their source? By focusing on these three, you can increasingly improve security outcomes for your organization. Empowered by information, teams can determine if they're learning from errors and reducing the occurrence of certain mistakes. If, however, they see no improvement, consider designing a security champions program to improve future performance. ## **Act 3, Scene 4:** Closing Remarks Psychology tells us that when success metrics are known, teams will adjust their efforts to meet them. By aligning success metrics to AppSec and development, organizations can ensure their focus remains on what is effectively most impactful. AppSec platform consolidation is at its tipping point. The future of [AppSec](https://www.paloaltonetworks.com/cyberpedia/appsec-application-security) lies in a platform that provides both AppSec and development teams with tools to accelerate security workflows. **--- End of Act 3 ---** ## **Interlude:** What's Next? If you'd like to learn more about how Prisma Cloud accelerates AppSec workflows with Code to Cloud^TM^context, [join an upcoming shift-left bootcamp](https://www.paloaltonetworks.com/prisma/cloud-interactive.html#shift-left). And check back as we explore more security theater. Act 4, coming soon. *** ** * ** *** ## Related Blogs ### [Application Security](https://www.paloaltonetworks.com/blog/cloud-security/category/application-security/?ts=markdown), [Cloud Security](https://www.paloaltonetworks.com/blog/category/cloud-security/?ts=markdown), [Compliance](https://www.paloaltonetworks.com/blog/cloud-security/category/compliance/?ts=markdown) [#### Security Theater: Don't Hang your Hat on Compliance](https://origin-researchcenter.paloaltonetworks.com/blog/cloud-security/compliance-security-theater/) ### [AppSec](https://www.paloaltonetworks.com/blog/cloud-security/category/appsec/?ts=markdown), [ASPM](https://www.paloaltonetworks.com/blog/cloud-security/category/aspm/?ts=markdown), [Cloud Security](https://www.paloaltonetworks.com/blog/category/cloud-security/?ts=markdown), [Code Security](https://www.paloaltonetworks.com/blog/cloud-security/category/code-security/?ts=markdown), [DevSecOps](https://www.paloaltonetworks.com/blog/cloud-security/category/devsecops/?ts=markdown), [Research](https://www.paloaltonetworks.com/blog/cloud-security/category/research/?ts=markdown) [#### An Inside Look into ASPM: Five Findings from New Industry Research](https://origin-researchcenter.paloaltonetworks.com/blog/cloud-security/aspm-research-omdia/) ### [Application Security](https://www.paloaltonetworks.com/blog/cloud-security/category/application-security/?ts=markdown), [AppSec](https://www.paloaltonetworks.com/blog/cloud-security/category/appsec/?ts=markdown), [ASPM](https://www.paloaltonetworks.com/blog/cloud-security/category/aspm/?ts=markdown), [Cloud Security](https://www.paloaltonetworks.com/blog/category/cloud-security/?ts=markdown), [Partners](https://www.paloaltonetworks.com/blog/cloud-security/category/partners/?ts=markdown) [#### Palo Alto Networks and Veracode: Unifying Application Security from Code to Cloud](https://origin-researchcenter.paloaltonetworks.com/blog/cloud-security/application-security-veracode-partnership/) ### [AppSec](https://www.paloaltonetworks.com/blog/cloud-security/category/appsec/?ts=markdown), [Cloud Detection and Response](https://www.paloaltonetworks.com/blog/cloud-security/category/cloud-detection-and-response/?ts=markdown), [Cloud Runtime Security](https://www.paloaltonetworks.com/blog/cloud-security/category/cloud-runtime-security/?ts=markdown), [Cloud Security](https://www.paloaltonetworks.com/blog/category/cloud-security/?ts=markdown), [Supply Chain Security](https://www.paloaltonetworks.com/blog/cloud-security/category/supply-chain-security/?ts=markdown) [#### Shai-Hulud 2.0: How Cortex Helps Protect Against the Resurgent npm Worm](https://origin-researchcenter.paloaltonetworks.com/blog/cloud-security/shai-hulud-2-0-npm-worm-detection-blocking/) ### [Application Security](https://www.paloaltonetworks.com/blog/cloud-security/category/application-security/?ts=markdown), [AppSec](https://www.paloaltonetworks.com/blog/cloud-security/category/appsec/?ts=markdown), [Cloud Security](https://www.paloaltonetworks.com/blog/category/cloud-security/?ts=markdown), [Partners](https://www.paloaltonetworks.com/blog/cloud-security/category/partners/?ts=markdown) [#### How Cortex Cloud and Semgrep Are Redefining AI-Driven Application Security](https://origin-researchcenter.paloaltonetworks.com/blog/cloud-security/application-security-semgrep-partnership/) ### [AppSec](https://www.paloaltonetworks.com/blog/cloud-security/category/appsec/?ts=markdown), [ASPM](https://www.paloaltonetworks.com/blog/cloud-security/category/aspm/?ts=markdown), [Cloud Security](https://www.paloaltonetworks.com/blog/category/cloud-security/?ts=markdown) [#### Breakdown: Widespread npm Supply Chain Attack Puts Billions of Weekly Downloads at Risk](https://origin-researchcenter.paloaltonetworks.com/blog/cloud-security/npm-supply-chain-attack/) ### Subscribe to Cloud Security Blogs! Sign up to receive must-read articles, Playbooks of the Week, new feature announcements, and more. ![spinner](https://origin-researchcenter.paloaltonetworks.com/blog/wp-content/themes/panwblog2023/dist/images/ajax-loader.gif) Sign up Please enter a valid email. By submitting this form, you agree to our [Terms of Use](https://www.paloaltonetworks.com/legal-notices/terms-of-use?ts=markdown) and acknowledge our [Privacy Statement](https://www.paloaltonetworks.com/legal-notices/privacy?ts=markdown). Please look for a confirmation email from us. If you don't receive it in the next 10 minutes, please check your spam folder. This site is protected by reCAPTCHA and the Google [Privacy Policy](https://policies.google.com/privacy) and [Terms of Service](https://policies.google.com/terms) apply. {#footer} {#footer} ## Products and Services * [AI-Powered Network Security Platform](https://www.paloaltonetworks.com/network-security?ts=markdown) * [Secure AI by Design](https://www.paloaltonetworks.com/precision-ai-security/secure-ai-by-design?ts=markdown) * [Prisma AIRS](https://www.paloaltonetworks.com/prisma/prisma-ai-runtime-security?ts=markdown) * [AI Access Security](https://www.paloaltonetworks.com/sase/ai-access-security?ts=markdown) * [Cloud Delivered Security Services](https://www.paloaltonetworks.com/network-security/security-subscriptions?ts=markdown) * [Advanced Threat Prevention](https://www.paloaltonetworks.com/network-security/advanced-threat-prevention?ts=markdown) * [Advanced URL Filtering](https://www.paloaltonetworks.com/network-security/advanced-url-filtering?ts=markdown) * [Advanced WildFire](https://www.paloaltonetworks.com/network-security/advanced-wildfire?ts=markdown) * [Advanced DNS Security](https://www.paloaltonetworks.com/network-security/advanced-dns-security?ts=markdown) * [Enterprise Data Loss Prevention](https://www.paloaltonetworks.com/sase/enterprise-data-loss-prevention?ts=markdown) * [Enterprise IoT Security](https://www.paloaltonetworks.com/network-security/enterprise-device-security?ts=markdown) * [Medical IoT Security](https://www.paloaltonetworks.com/network-security/medical-device-security?ts=markdown) * [Industrial OT Security](https://www.paloaltonetworks.com/network-security/medical-device-security?ts=markdown) * [SaaS Security](https://www.paloaltonetworks.com/sase/saas-security?ts=markdown) * [Next-Generation Firewalls](https://www.paloaltonetworks.com/network-security/next-generation-firewall?ts=markdown) * [Hardware Firewalls](https://www.paloaltonetworks.com/network-security/hardware-firewall-innovations?ts=markdown) * [Software Firewalls](https://www.paloaltonetworks.com/network-security/software-firewalls?ts=markdown) * [Strata Cloud Manager](https://www.paloaltonetworks.com/network-security/strata-cloud-manager?ts=markdown) * [SD-WAN for NGFW](https://www.paloaltonetworks.com/network-security/sd-wan-subscription?ts=markdown) * [PAN-OS](https://www.paloaltonetworks.com/network-security/pan-os?ts=markdown) * [Panorama](https://www.paloaltonetworks.com/network-security/panorama?ts=markdown) * [Secure Access Service Edge](https://www.paloaltonetworks.com/sase?ts=markdown) * [Prisma SASE](https://www.paloaltonetworks.com/sase?ts=markdown) * [Application Acceleration](https://www.paloaltonetworks.com/sase/app-acceleration?ts=markdown) * [Autonomous Digital Experience Management](https://www.paloaltonetworks.com/sase/adem?ts=markdown) * [Enterprise DLP](https://www.paloaltonetworks.com/sase/enterprise-data-loss-prevention?ts=markdown) * [Prisma Access](https://www.paloaltonetworks.com/sase/access?ts=markdown) * [Prisma Browser](https://www.paloaltonetworks.com/sase/prisma-browser?ts=markdown) * [Prisma SD-WAN](https://www.paloaltonetworks.com/sase/sd-wan?ts=markdown) * [Remote Browser Isolation](https://www.paloaltonetworks.com/sase/remote-browser-isolation?ts=markdown) * [SaaS Security](https://www.paloaltonetworks.com/sase/saas-security?ts=markdown) * [AI-Driven Security Operations Platform](https://www.paloaltonetworks.com/cortex?ts=markdown) * [Cloud Security](https://www.paloaltonetworks.com/cortex/cloud?ts=markdown) * [Cortex Cloud](https://www.paloaltonetworks.com/cortex/cloud?ts=markdown) * [Application Security](https://www.paloaltonetworks.com/cortex/cloud/application-security?ts=markdown) * [Cloud Posture Security](https://www.paloaltonetworks.com/cortex/cloud/cloud-posture-security?ts=markdown) * [Cloud Runtime Security](https://www.paloaltonetworks.com/cortex/cloud/runtime-security?ts=markdown) * [Prisma Cloud](https://www.paloaltonetworks.com/prisma/cloud?ts=markdown) * [AI-Driven SOC](https://www.paloaltonetworks.com/cortex?ts=markdown) * [Cortex XSIAM](https://www.paloaltonetworks.com/cortex/cortex-xsiam?ts=markdown) * [Cortex XDR](https://www.paloaltonetworks.com/cortex/cortex-xdr?ts=markdown) * [Cortex XSOAR](https://www.paloaltonetworks.com/cortex/cortex-xsoar?ts=markdown) * [Cortex Xpanse](https://www.paloaltonetworks.com/cortex/cortex-xpanse?ts=markdown) * [Unit 42 Managed Detection \& Response](https://www.paloaltonetworks.com/cortex/managed-detection-and-response?ts=markdown) * [Managed XSIAM](https://www.paloaltonetworks.com/cortex/managed-xsiam?ts=markdown) * [Threat Intel and Incident Response Services](https://www.paloaltonetworks.com/unit42?ts=markdown) * [Proactive Assessments](https://www.paloaltonetworks.com/unit42/assess?ts=markdown) * [Incident Response](https://www.paloaltonetworks.com/unit42/respond?ts=markdown) * [Transform Your Security Strategy](https://www.paloaltonetworks.com/unit42/transform?ts=markdown) * [Discover Threat Intelligence](https://www.paloaltonetworks.com/unit42/threat-intelligence-partners?ts=markdown) ## Company * [About Us](https://www.paloaltonetworks.com/about-us?ts=markdown) * [Careers](https://jobs.paloaltonetworks.com/en/) * [Contact Us](https://www.paloaltonetworks.com/company/contact-sales?ts=markdown) * [Corporate Responsibility](https://www.paloaltonetworks.com/about-us/corporate-responsibility?ts=markdown) * [Customers](https://www.paloaltonetworks.com/customers?ts=markdown) * [Investor Relations](https://investors.paloaltonetworks.com/) * [Location](https://www.paloaltonetworks.com/about-us/locations?ts=markdown) * [Newsroom](https://www.paloaltonetworks.com/company/newsroom?ts=markdown) ## Popular Links * [Blog](https://www.paloaltonetworks.com/blog/?ts=markdown) * [Communities](https://www.paloaltonetworks.com/communities?ts=markdown) * [Content Library](https://www.paloaltonetworks.com/resources?ts=markdown) * [Cyberpedia](https://www.paloaltonetworks.com/cyberpedia?ts=markdown) * [Event Center](https://events.paloaltonetworks.com/) * [Manage Email Preferences](https://start.paloaltonetworks.com/preference-center) * [Products A-Z](https://www.paloaltonetworks.com/products/products-a-z?ts=markdown) * [Product Certifications](https://www.paloaltonetworks.com/legal-notices/trust-center/compliance?ts=markdown) * [Report a Vulnerability](https://www.paloaltonetworks.com/security-disclosure?ts=markdown) * [Sitemap](https://www.paloaltonetworks.com/sitemap?ts=markdown) * [Tech Docs](https://docs.paloaltonetworks.com/) * [Unit 42](https://unit42.paloaltonetworks.com/) * [Do Not Sell or Share My Personal Information](https://panwedd.exterro.net/portal/dsar.htm?target=panwedd) ![PAN logo](https://www.paloaltonetworks.com/etc/clientlibs/clean/imgs/pan-logo-dark.svg) * [Privacy](https://www.paloaltonetworks.com/legal-notices/privacy?ts=markdown) * [Trust Center](https://www.paloaltonetworks.com/legal-notices/trust-center?ts=markdown) * [Terms of Use](https://www.paloaltonetworks.com/legal-notices/terms-of-use?ts=markdown) * [Documents](https://www.paloaltonetworks.com/legal?ts=markdown) Copyright © 2026 Palo Alto Networks. All Rights Reserved * [![Youtube](https://www.paloaltonetworks.com/etc/clientlibs/clean/imgs/social/youtube-black.svg)](https://www.youtube.com/user/paloaltonetworks) * [![Podcast](https://www.paloaltonetworks.com/content/dam/pan/en_US/images/icons/podcast.svg)](https://www.paloaltonetworks.com/podcasts/threat-vector?ts=markdown) * [![Facebook](https://www.paloaltonetworks.com/etc/clientlibs/clean/imgs/social/facebook-black.svg)](https://www.facebook.com/PaloAltoNetworks/) * [![LinkedIn](https://www.paloaltonetworks.com/etc/clientlibs/clean/imgs/social/linkedin-black.svg)](https://www.linkedin.com/company/palo-alto-networks) * [![Twitter](https://www.paloaltonetworks.com/etc/clientlibs/clean/imgs/social/twitter-x-black.svg)](https://twitter.com/PaloAltoNtwks) * EN Select your language