Forschungsgruppe Dezentrale Systeme und Netzdienste - TM

The Intelligence Framework Update

In the endeavor of securing networks, Threat Intelligence (TI) has become a key component. TI can be roughly described as data that allows the identification of threats to your IT system respectively network. This data is available on strategic, tactical and operational levels. The Bro Network Security Monitor supports operational integration of TI with the Intelligence Framework.

In 2016, the Intelligence Framework was refactored. In this context, a couple of new features like removal and expiration of intelligence items (also referred to as Indicators of Compromise, IoCs) as well as a new way of extending the framework have been introduced. Nevertheless, to exploit the Intelligence Framework’s full potential, it is essential to understand the underlying concepts. Last year’s blog post has been a first attempt to shed some light on the framework’s data model and has introduced the new features. The proposed talk will revisit these foundations. In particular, the three different representations/manifestations of intelligence data will be elaborated, i.e. items for ingestion, their internal representation and seen data for matching. Furthermore, common pitfalls that have been encountered in the past will be discussed including developments regarding input reader resilience.

In addition, the talk will present the intel-extensions as an example on how the Intelligence Framework can be customized. Focusing on per-item-expiration, the Framework’s extension mechanisms will be described. To complete the picture, a use case will be introduced that employs per-item-expiration to ingest feeds, which contain domains generated by Domain Generation Algorithms (DGA). This includes experiences gathered during a test deployment at the Karlsruhe Institute of Technology, a German research and education institution.

As Bro basically serves as an in-memory database for intelligence data, it is easy to lose track on the actual working set of indicators, because the files usually used for ingestion not necessarily represent the internal state of Bro. Another problem are the limited possibilities to interact directly with the system in case a feed contains incorrect information and the “intel.log” is flooded with hits. The talk will present small Python scripts that can be used from the command line to insert, delete or query intelligence data managed by Bro. Making use of Bro’s new communication library broker, these scripts represent only one way of dynamically interacting with the Intelligence Framework.

Finally, there will be an outlook on future work that is concerned with the integration of Cuckoo Filters into the Intelligence Framework as suggested by Matthias Vallentin. Overall, the talk aims at providing a thorough overview on the current state of Bro’s Intelligence Framework as well as ideas on how to use it in practice.