--- title: "Cyber Threat Intelligence: The Firehose of Noise and How We Got Here" description: Threat intelligence feeds have become popular, and a company's ability to track threats outside of its own environment is better than ever. image: https://blog.nisos.com/hubfs/graphics-7.jpg --- - [Home](https://www.nisos.com/) - Solutions - [Manage Human Risk](https://nisos.com/solutions/human-risk/) - [Protect Online Platforms](https://www.nisos.com/solutions/trust-safety/) - [Manage Cybersecurity Risks](https://www.nisos.com/solutions/cybersecurity/) - [Protect Executives, Assets, and Reputation](https://www.nisos.com/solutions/corporate-security/) - [Secure the Employee Lifecycle](https://nisos.com/solutions/workforce-risk/) - Services - [Human Risk](https://nisos.com/services/human-risk/) - [Employment Shield](https://nisos.com/services/employment-shield/) - [Executive Shield](https://nisos.com/services/executive-shield/) - [Third-Party Intelligence Solutions](https://nisos.com/services/third-party-intelligence-solutions/) - [Insider Threat Intelligence Solutions](https://nisos.com/services/executive-shield/) - [Risk Assessment](https://www.nisos.com/services/risk-assessments/) - [Threat Landscape](https://www.nisos.com/services/threat-landscape-assessment/) - [Threat Monitoring](https://www.nisos.com/services/threat-monitoring/) - [OSINT Monitoring & Analysis](https://www.nisos.com/services/osint-monitoring-and-analysis/) - [Investigations](https://www.nisos.com/services/investigations/) - [Adversary Investigation](https://www.nisos.com/services/adversary-insights/) - Company - [About Us](https://nisos.com/company/about-us/) - [The Nisos Difference](https://nisos.com/company/the-nisos-difference/) - [Client Success](https://nisos.com/company/client-success/) - [Careers](https://www.nisos.com/company/about-the-company/careers/) - Resources - [News](https://www.nisos.com/newsroom/) - [Blog](https://www.nisos.com/nisos-blog/) - [Events](https://www.nisos.com/events/) - [Resource Library](https://www.nisos.com/resources/library/) - [Podcasts](https://www.nisos.com/podcasts/) - [Contact](https://www.nisos.com/contact/) [![Nisos-Logo-Blue](https://blog.nisos.com/hubfs/Nisos-Logo-Blue.png) ![Nisos-Logo-White](https://blog.nisos.com/hubfs/Nisos-Logo-White.png) ![Nisos-Logo-Blue](https://blog.nisos.com/hubfs/Nisos-Logo-Blue.png)](https://www.nisos.com/) - [Default HubSpot Blog](https://blog.nisos.com/blog) - [Home](https://www.nisos.com/) - Solutions - [Manage Human Risk](https://nisos.com/solutions/human-risk/) - [Protect Online Platforms](https://www.nisos.com/solutions/trust-safety/) - [Manage Cybersecurity Risks](https://www.nisos.com/solutions/cybersecurity/) - [Protect Executives, Assets, and Reputation](https://www.nisos.com/solutions/corporate-security/) - [Secure the Employee Lifecycle](https://nisos.com/solutions/workforce-risk/) - Services - [Human Risk](https://nisos.com/services/human-risk/) - [Employment Shield](https://nisos.com/services/employment-shield/) - [Executive Shield](https://nisos.com/services/executive-shield/) - [Third-Party Intelligence Solutions](https://nisos.com/services/third-party-intelligence-solutions/) - [Insider Threat Intelligence Solutions](https://nisos.com/services/executive-shield/) - [Risk Assessment](https://www.nisos.com/services/risk-assessments/) - [Threat Landscape](https://www.nisos.com/services/threat-landscape-assessment/) - [Threat Monitoring](https://www.nisos.com/services/threat-monitoring/) - [OSINT Monitoring & Analysis](https://www.nisos.com/services/osint-monitoring-and-analysis/) - [Investigations](https://www.nisos.com/services/investigations/) - [Adversary Investigation](https://www.nisos.com/services/adversary-insights/) - Company - [About Us](https://nisos.com/company/about-us/) - [The Nisos Difference](https://nisos.com/company/the-nisos-difference/) - [Client Success](https://nisos.com/company/client-success/) - [Careers](https://www.nisos.com/company/about-the-company/careers/) - Resources - [News](https://www.nisos.com/newsroom/) - [Blog](https://www.nisos.com/nisos-blog/) - [Events](https://www.nisos.com/events/) - [Resource Library](https://www.nisos.com/resources/library/) - [Podcasts](https://www.nisos.com/podcasts/) - [Contact](https://www.nisos.com/contact/) ### Solutions Managed Intelligence solutions are designed to address your needs in an evolving risk landscape. Empower your organization with methodologies and actionable results tailored to the innovative ways adversaries are targeting you. ### Mitigate Advanced Threat Actors Inform your security operations with greater context answering the “how” and “why” of the threat actors you face. ### Counter Disinformation Enable informed business responses with the “who” and “why” behind coordinated campaigns. ### Mitigate Insider Threats Improve your program’s ability to get to “who” and “what” with technical program building and investigations. ### Stop Platform Abuse Take action against nefarious use of your platforms with the “who” and “how” of threat actors targeting them. ### Cyber Diligence for Mergers and Acquisitions Facilitate a smooth transition with the “what” and “why” of the security risks to your acquisition. ### Third Party Risk Dilligence Increase the effectiveness of your security evaluations with specifics on the “where” and “what” of your third party risk profile. ### Services Access a world-class intelligence capability tailored to your specific needs. Control a multi-million dollar program without the time or expense and solve problems both lasting and acute. ### Managed Intelligence Ongoing risks, whether from one threat actor or many, require persistent focus. Leverage skills, data, and technology to harden your defenses. ### Security Investigations Gain the context needed to make informed decisions during periods of heightened risk. ### What is Managed Intelligence? ## Technical Blogs ###### 5 min read # [Cyber Threat Intelligence: The Firehose of Noise and How We Got Here](https://blog.nisos.com/technical-blogs/firehose_noise_how_we_got_here) Jun 23, 2020 12:18:09 PM - [Tweet](https://twitter.com/share) Threat intelligence feeds have become popular, and a company's ability to track threats outside of its own environment is better than ever. With these improvements though, has come an increasing demand on security professionals to select and manage the right combination of tools to achieve their desired outcomes. Here is a brief look at the history of the cyber threat intelligence industry, and where we might go from here. **Turning on the Firehose** When organizations first began setting up security operations centers, they typically did so by collecting a patchwork of data from host based monitoring (like anti-virus) applications, logs for client facing services, and network appliances. Though these setups were effective at detecting previously catalogued malware and unusual network activity, it became clear that even moderately sophisticated attackers could avoid detection. These indicators of compromise (IOCs) were not routinely shared between security vendors or enterprise organizations, so attackers could move from one target to the next without worrying about being recognized. To combat this, access to a stream of the most updated information on host and network IOCs would be needed. Thus, the threat intelligence feed was born. Threat intelligence feeds arose from a surge of data that security companies began collecting as monitoring became standard in home and enterprise environments. This supply fits perfectly with the demand for up to date information about IOCs and the rise of large scale search and analysis tools like Splunk and ELK. The feeds could be integrated into a system that regularly monitored network traffic, files on hosts, and user activity by writing simple code to access feed APIs. The solution to network defense appeared to be straightforward. Based on the belief that more data meant a wider net and thus a better system, CISOs began ingesting feeds and throwing the information into their monitoring software. **Drinking Problems** The feeds certainly worked as advertised and provided data; a lot of it. Tens of thousands of data points a day, loosely categorized, sometimes inaccurate, and often irrelevant. It wasn’t long before front-line practitioners noticed major problems. First, even moderately sophisticated attackers adapted to this monitoring regime by developing techniques to evade basic detection mechanisms. To avoid detection based on simple IOCs, it is easy for an attacker to change their server address or modify their code every time they target a new organization. Simple IOCs such as IP addresses, domains, and file hashes are necessary, but not sufficient. Sophisticated host and network monitoring solutions can perform some level of behavioral monitoring and context-aware detection, [but even these methods can be subverted through observation and mimicry.](https://blog.nisos.com/blog/threat_intelligence_through_eyes_of_adversaries) It also became clear that wrangling the data and making sense of it was more difficult than anticipated. Security teams still spend an excessive amount of time automating threat information feeds and parsing data in order to operationalize information within their security and event management systems. Some feeds are already unwieldy due to formatting differences, lack of integration, ambiguous relevance, and the time-sensitive nature of the data. Additionally, the wide nets cast by multiple data feeds began to generate more and more false positives, especially when IOCs like server addresses were recycled for legitimate purposes. Threat hunters had to deal with hundreds of false alarms a day, a task that quickly numbed the senses and made it even easier to overlook real attacks that used new techniques.With a multitude of potential attack vectors and a high rate of false positives, security professionals needed to know how and where to direct their attention. In order to stay one step ahead of threats, organizations needed to anticipate malevolent behavior and be on the lookout for attacks before they happened. This could only be achieved by understanding their organization in the context of the world in general and the internet in particular. To fulfill this need, a second generation of threat intelligence feeds arose. Security teams began integrating information about APTs collected by major research and cybersecurity firms. These data sets required understanding of more than the traditional technical sources. Social media, breach data, external network telemetry, and geodata are all finding a place within the toolset of advanced threat programs. The prevalence of sophisticated attackers and rich data sources also moves intelligence programs into an entirely different realm of importance. **Widening The Spray** While the technical side of the house was dealing with the challenge of making data feeds palatable, [strategic planners began to learn that threat intelligence could be used to inform business decisions.](https://blog.nisos.com/case-studies/data_market_entry) Varied threats like [disinformation](https://blog.nisos.com/disinformation-investigations), [platform abuse](https://blog.nisos.com/platform_abuse), brand dilution, and [strategic breach campaigns](https://blog.nisos.com/mitigate_threat_actors) became more prevalent. More and more of these threats lived far outside the traditional environment of analysts investigating potential intrusions on their dashboards. Advanced programs began adding new groups that looked more and more like marketing and business intelligence units. This posed another difficult problem: who would be responsible for dealing with these existential threats and how could an accurate picture of the world be gleaned from all of this knowledge? **To administer an effective threat management program, plans made at the strategic, tactical, and technical levels would need to work in concert with one another.** **How? Let’s look at an example, Third Party Risk Management:** Beyond the primary risks associated with a direct compromise of an organization’s assets, there is also [an indirect risk associated with the compromise of a vendor or partner.](https://blog.nisos.com/third-party-risk-diligence) Conversely, a company may find themselves being targeted by a threat actor whose primary goal is compromising one of their clients, and is only targeting the company as means to achieve their ultimate objective. In both of these scenarios, the organization is exposed to a risk that can be managed strategically by breaking off the partnership, tactically by segregating interaction from the partner organization, or technically by allocating more assets to the virtual border between the two organizations. [Choosing the best option (or mix of options) requires decision makers to have accurate intelligence on both the technical profile of the threat actor and their ultimate goal.](https://blog.nisos.com/white-papers/zero_touch_diligence) The next generation of threat intelligence products certainly helps, but difficult issues remain unresolved. **Still Looking For a Straw** The current generation of solutions only solves one of the big problems within threat programs. These additional intelligence feeds augment the data that analysts have at their fingertips, potentially allowing the organization to anticipate attacks before they happen. However, they do not solve the second problem of tractability. If a security program is already drowning in noise, adding more data to the mix is just going to confuse things even more. If the analysts don’t have experience or training on how to use the multitude of intelligence sources, they won’t be able to draw the insights needed to make the most of their tools.. Next generation feeds with a high level of volume and specificity are only useful if there exists a high level of technical expertise, analytical capability, and situational awareness on the part of the organization purchasing them. Unfortunately, very few organizations can afford security personnel with that level of expertise. Instead they spend millions of dollars on technical solutions that replace practitioners instead of facilitating them. Human analysis will always be necessary, but it will always be expensive and in short supply. **Replacing the Firehose with a Straw** The challenge, of course, is for the industry to recognize less can be more. Many threat intelligence programs, and thus feed providers, continue to be measured against metrics that align with [“how much” rather than true impact](https://blog.nisos.com/white-papers/real_cyber_intel_tells_soc_what_security_stack_cannot_detect). [One solution to this problem is a feed or service which packages, filters, and enriches existing data sources by using analysts familiar with the needs of client organizations.](https://blog.nisos.com/why-nisos) In essence, this provider would manage intelligence by providing a layer of automation and human analysis between data feeds and security teams. This would guarantee that customers are only getting information which is relevant and focused, effectively replacing the firehose with a straw. [![Learn More](https://no-cache.hubspot.com/cta/default/6068438/acc4abce-77ee-4faf-8c6e-ac14589fb5a1.png)](https://cta-redirect.hubspot.com/cta/redirect/6068438/acc4abce-77ee-4faf-8c6e-ac14589fb5a1) Topics: [white paper](https://blog.nisos.com/technical-blogs/topic/white-paper) [threat intelligence](https://blog.nisos.com/technical-blogs/topic/threat-intelligence) [noise](https://blog.nisos.com/technical-blogs/topic/noise) ##### Written by [Zachary Henson](https://blog.nisos.com/technical-blogs/author/zachary-henson) ###### [Previous Post](https://blog.nisos.com/technical-blogs/leveraging_technical_expertise_combat_disinformation) [![](https://blog.nisos.com/hubfs/shutterstock_1441871069.jpg)](https://blog.nisos.com/technical-blogs/leveraging_technical_expertise_combat_disinformation) ###### [Leveraging Technical Expertise & Data Partnerships to Combat Disinformation](https://blog.nisos.com/technical-blogs/leveraging_technical_expertise_combat_disinformation) ###### [Next Post](https://blog.nisos.com/technical-blogs/inside_look_advanced_attacker_ttps) [![](https://blog.nisos.com/hubfs/techstack3-01.png)](https://blog.nisos.com/technical-blogs/inside_look_advanced_attacker_ttps) ###### [An Inside Look at Advanced Attacker TTPs and the Danger of Relying on Industry-based Threat Intelligence](https://blog.nisos.com/technical-blogs/inside_look_advanced_attacker_ttps) ### Post a Comment ### Sign up for updates ### Featured ![](https://blog.nisos.com/hubfs/Nisos-Logo-White.png) Responsive strategies and intelligent defenses for protecting your enterprise. - Services - Managed Intelligence - Security Investigations - Why Nisos - Company - Resources - [Blog](https://blog.nisos.com/blog) - [Case Studies](https://blog.nisos.com/case-studies) - [White Papers](https://blog.nisos.com/technical-blogs) - [Podcast](https://blog.nisos.com/podcast) - [News](https://blog.nisos.com/news) - Contact ###### Contact Us [1-703-382-8400](tel:1-703-382-8400) [info@nisos.com](mailto:info@nisos.com) 603 King Street, Suite 300 Alexandria, VA 22314 [![Learn More](https://no-cache.hubspot.com/cta/default/6068438/043dbe58-e265-499f-8574-2e0d1b4d410e.png)](https://cta-redirect.hubspot.com/cta/redirect/6068438/043dbe58-e265-499f-8574-2e0d1b4d410e) ©2020 Nisos. All rights reserved. - Terms - Cookie Policy - Privacy Policy ![](https://px.ads.linkedin.com/collect/?pid=1593266&fmt=gif)