{"id":105016,"date":"2020-01-14T06:00:00","date_gmt":"2020-01-14T14:00:00","guid":{"rendered":"https:\/\/www.paloaltonetworks.com\/blog\/?p=105016"},"modified":"2020-07-30T20:05:43","modified_gmt":"2020-07-31T03:05:43","slug":"cortex-soc-metrics","status":"publish","type":"post","link":"https:\/\/origin-researchcenter.paloaltonetworks.com\/blog\/2020\/01\/cortex-soc-metrics\/","title":{"rendered":"Do Your SOC Metrics Incentivize Bad Behavior?"},"content":{"rendered":"<p><img loading=\"lazy\" decoding=\"async\"  class=\"size-full wp-image-105030 alignleft lozad\"  data-src=\"https:\/\/www.paloaltonetworks.com\/blog\/wp-content\/uploads\/2020\/01\/image2-2.png\" alt=\"SOC Metrics are one of the key elements to get right in a security operations center. This image displays the element in the style of the periodic table of the elements.\" width=\"360\" height=\"354\" srcset=\"https:\/\/origin-researchcenter.paloaltonetworks.com\/blog\/wp-content\/uploads\/2020\/01\/image2-2.png 360w, https:\/\/origin-researchcenter.paloaltonetworks.com\/blog\/wp-content\/uploads\/2020\/01\/image2-2-230x226.png 230w, https:\/\/origin-researchcenter.paloaltonetworks.com\/blog\/wp-content\/uploads\/2020\/01\/image2-2-305x300.png 305w, https:\/\/origin-researchcenter.paloaltonetworks.com\/blog\/wp-content\/uploads\/2020\/01\/image2-2-41x40.png 41w\" sizes=\"auto, (max-width: 360px) 100vw, 360px\" \/><\/p>\n<p><i><span style=\"font-weight: 400;\">The following post on SOC metrics is adapted from the book, \u201cElements of Security Operations,\u201d a guide to building and optimizing effective and scalable security operations. <\/span><\/i><a href=\"https:\/\/start.paloaltonetworks.com\/elements-of-security-operations.html\"><i><span style=\"font-weight: 400;\">Download a free copy today<\/span><\/i><\/a><i><span style=\"font-weight: 400;\">.<\/span><\/i><\/p>\n<p>Some metrics that security operations centers (SOCs) widely use to evaluate their performance have the potential to drive poor behavior.<\/p>\n<p><span style=\"font-weight: 400;\">One example is mean time to resolution (MTTR). This is a fine metric when used in a network operations center (where uptime is key) but it can be detrimental when used in a SOC. Holding security analysts accountable for MTTR incentivizes them to rush to close incidents rather than rewarding full investigations that feed learning back into the controls to prevent future attacks. Similarly, ranking performance by number of incidents handled may lead to analysts \u201ccherry picking\u201d incidents that they know are fast to resolve. This will not produce better outcomes or reduced risk for the business.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Another poor metric is counting the number of firewall rules deployed. 10,000 firewall rules can be in place, but if the first bypasses the rest (e.g., any-any), then they are useless. This is similar to measuring the number of data feeds into a security information and event management platform (SIEM). If there are 15 data feeds into a SIEM but only one use case, then the data feeds aren\u2019t being utilized and are a potentially expensive waste.\u00a0<\/span><\/p>\n<h2><span style=\"font-weight: 400;\"><br \/>\n<\/span><span style=\"font-weight: 400;\">SOC Metrics That Matter<\/span><\/h2>\n<p><span style=\"font-weight: 400;\">When determining good metrics for your business, always keep in mind the mission of the SOC and the value it provides to the business. The business wants confidence that the SOC can prevent attacks and that if\/when a breach does occur, then the team is able to handle it quickly, limit the impact and learn from it. Good metrics should provide insight into whether the business should have confidence or not. There are two types of confidence to focus on: configuration confidence and operational confidence.<\/span><\/p>\n<p><b>Configuration confidence<\/b><span style=\"font-weight: 400;\"> is knowing that your technology is properly configured to prevent an attack, that you can automatically remediate it and\/or that the proper intelligence can be gathered for analysis by a human. Example questions to answer are:\u00a0<\/span><\/p>\n<ul>\n<li style=\"font-weight: 400;\"><b>Are the security controls running?<\/b><span style=\"font-weight: 400;\"> Oftentimes a \u201ctemporary\u201d change is made to controls and is inadvertently left in place. A developer may need a specific port to be opened to perform a test and that port remains open after the test is completed, providing an access point for an attack.<\/span><\/li>\n<li style=\"font-weight: 400;\"><b>How many changes are occurring outside of the change control policy? <\/b><span style=\"font-weight: 400;\">The change control policy should be followed in every change without any exceptions. Any deviation to the defined process should be noted as it is relevant to the business\u2019s confidence in the configuration of security controls.<\/span><\/li>\n<li style=\"font-weight: 400;\"><b>Are the technologies in place configured to best practices? <\/b><span style=\"font-weight: 400;\">Once a technology is in place, it is rarely a \u201cset it and forget it\u201d situation. Care must be taken to continually evaluate the configuration against best practices. If the measurement of controls against best practices is low, this can drive a plan to increase adherence. If the metrics drop, then a look into why this is happening is warranted.<\/span><\/li>\n<li style=\"font-weight: 400;\"><b>What percent of features and capabilities are being utilized? <\/b><span style=\"font-weight: 400;\">The plethora of security technologies is overwhelming security operations. Many of these technologies are poorly utilized, resulting in a false understanding by the business regarding the actual coverage in place. It can also lead to the purchasing of duplicate features, which exacerbates the issue of too many technologies. Measuring the percentage of feature use can provide the business with a simple understanding of actual value being provided by tools vs. perceived value. For example, what percentage of traffic flowing is visible to analysts? Estimates state that 70-80% of traffic is encrypted. The business should know how much traffic is being analyzed in a SOC and if SSL decryption technology is being used.<\/span><\/li>\n<\/ul>\n<p><b>Operational confidence <\/b><span style=\"font-weight: 400;\">is knowing that the right people and processes are in place to handle <\/span><span style=\"font-weight: 400;\">a breach if\/when it occurs. Example questions to answer are:<\/span><\/p>\n<ul>\n<li style=\"font-weight: 400;\"><b>How many events are analysts handling per hour?<\/b><span style=\"font-weight: 400;\"> This is known as events per analyst hour (EPAH). A reasonable EPAH is 8-13. If the EPAH is too high, say 100, then this indicates that analysts are overwhelmed. They will rush investigations, ignore events and not be set up to properly protect the business. Also, note that it is important to measure per hour and not per day, as an analyst\u2019s tasks should shift throughout the day and shift lengths can vary, causing this number to skew. This metric should not be gathered to compare employees but rather to show the effectiveness of an entire security operations organization.<\/span><\/li>\n<li style=\"font-weight: 400;\"><b>Are there repeat incidents flowing into the SOC?<\/b><span style=\"font-weight: 400;\"> If threats are properly investigated, then the outcome should feed back into a centralized set of controls that synchronize your various tools for future protection. Repeat incidents flowing into a SOC indicate a failure in this feedback and sync of controls.<\/span><\/li>\n<li style=\"font-weight: 400;\"><b>Is the SOC handling alerts for known threats? <\/b><span style=\"font-weight: 400;\">This also indicates a failure in the controls because all known threats should be blocked prior to affecting the business and being passed to the SOC to investigate.<\/span><\/li>\n<li style=\"font-weight: 400;\"><b>How often are there deviations in SOC procedures? <\/b><span style=\"font-weight: 400;\">This metric can indicate the need for employee training on the procedures. It may also illuminate out-of-date procedures that need to be updated.<\/span><\/li>\n<\/ul>\n<figure id=\"attachment_105148\" aria-describedby=\"caption-attachment-105148\" style=\"width: 500px\" class=\"wp-caption alignright\"><div style=\"max-width:100%\" data-width=\"500\"><span class=\"ar-custom\" style=\"padding-bottom:72.4%;\"><img loading=\"lazy\" decoding=\"async\"  class=\"wp-image-105148 size-large lozad\"  data-src=\"https:\/\/www.paloaltonetworks.com\/blog\/wp-content\/uploads\/2020\/01\/image1-5-500x362.png\" alt=\"This image shows the elements of security operations in the style of the periodic table of the elements.\" width=\"500\" height=\"362\" srcset=\"https:\/\/origin-researchcenter.paloaltonetworks.com\/blog\/wp-content\/uploads\/2020\/01\/image1-5-500x362.png 500w, https:\/\/origin-researchcenter.paloaltonetworks.com\/blog\/wp-content\/uploads\/2020\/01\/image1-5-230x167.png 230w, https:\/\/origin-researchcenter.paloaltonetworks.com\/blog\/wp-content\/uploads\/2020\/01\/image1-5-414x300.png 414w, https:\/\/origin-researchcenter.paloaltonetworks.com\/blog\/wp-content\/uploads\/2020\/01\/image1-5-55x40.png 55w, https:\/\/origin-researchcenter.paloaltonetworks.com\/blog\/wp-content\/uploads\/2020\/01\/image1-5.png 602w\" sizes=\"auto, (max-width: 500px) 100vw, 500px\" \/><\/span><\/div><figcaption id=\"caption-attachment-105148\" class=\"wp-caption-text\">This blog is the second in a series based off our book, \"Elements of Security Operations.\" Read the first post, \"<a href=\"https:\/\/www.paloaltonetworks.com\/blog\/2020\/01\/cortex-security-operations\/\">The Six Pillars of Effective Security Operations<\/a>.\"<\/figcaption><\/figure>\n<p><span style=\"font-weight: 400;\">Metrics should be used to improve protections and provide confidence to the business that the security operations organization is executing on its mission \u2013 which requires measuring quality, not just volume. Each metric has specific and limited value; no one metric tells the whole story, but together, they can help drive continued improvement and confidence that the business is properly set up to prevent and contain a breach.\u00a0<\/span><span style=\"font-weight: 400;\">\u00a0To learn more best practices for building effective secu<\/span><span style=\"font-weight: 400;\">rity operations, <\/span><a href=\"https:\/\/start.paloaltonetworks.com\/elements-of-security-operations.html\"><span style=\"font-weight: 400;\">download a free copy of our book, \u201cElements of Security Operations<\/span><\/a><span style=\"font-weight: 400;\">.\u201d<\/span><\/p>\n<p><em>Watch for future posts in\u00a0<a href=\"https:\/\/www.paloaltonetworks.com\/blog\/tag\/elements-of-security-operations\/\">Kerry Matre\u2019s series<\/a> on \u201cElements of Security Operations.\u201d Next up: \"<a href=\"https:\/\/www.paloaltonetworks.com\/blog\/2020\/01\/cortex-secops-strategies\/\">3 SecOps Strategies to Enable Your Smart People to Focus on Smart Things<\/a>.\"<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Many security operations center (SOC) metrics \u2013 even some of the most popular ones \u2013 have the potential to drive poor behavior.<\/p>\n","protected":false},"author":432,"featured_media":104830,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[6770],"tags":[6980,297,673,635],"coauthors":[4695],"class_list":["post-105016","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-secure-the-future","tag-elements-of-security-operations","tag-security-metrics","tag-security-operations-center","tag-soc","sec_ops_category-must-read-articles"],"jetpack_featured_media_url":"https:\/\/origin-researchcenter.paloaltonetworks.com\/blog\/wp-content\/uploads\/2020\/01\/image1.png","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/origin-researchcenter.paloaltonetworks.com\/blog\/wp-json\/wp\/v2\/posts\/105016","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/origin-researchcenter.paloaltonetworks.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/origin-researchcenter.paloaltonetworks.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/origin-researchcenter.paloaltonetworks.com\/blog\/wp-json\/wp\/v2\/users\/432"}],"replies":[{"embeddable":true,"href":"https:\/\/origin-researchcenter.paloaltonetworks.com\/blog\/wp-json\/wp\/v2\/comments?post=105016"}],"version-history":[{"count":9,"href":"https:\/\/origin-researchcenter.paloaltonetworks.com\/blog\/wp-json\/wp\/v2\/posts\/105016\/revisions"}],"predecessor-version":[{"id":105691,"href":"https:\/\/origin-researchcenter.paloaltonetworks.com\/blog\/wp-json\/wp\/v2\/posts\/105016\/revisions\/105691"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/origin-researchcenter.paloaltonetworks.com\/blog\/wp-json\/wp\/v2\/media\/104830"}],"wp:attachment":[{"href":"https:\/\/origin-researchcenter.paloaltonetworks.com\/blog\/wp-json\/wp\/v2\/media?parent=105016"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/origin-researchcenter.paloaltonetworks.com\/blog\/wp-json\/wp\/v2\/categories?post=105016"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/origin-researchcenter.paloaltonetworks.com\/blog\/wp-json\/wp\/v2\/tags?post=105016"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/origin-researchcenter.paloaltonetworks.com\/blog\/wp-json\/wp\/v2\/coauthors?post=105016"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}