Mining The Dark Web For Cyber Intelligence

Search social media and delve into the dark web and you can find all sorts of useful data. James Parry, Technical Manager for Auriga, looks at how businesses can tap this mine of information to learn of impending attacks.

Like any social construct, the Internet has its dark and seedy side. From social media to encrypted chat rooms and the black markets of the deep web, there’s a whole world out there lying just beneath the shiny surface of the Internet and it’s here that many cyber attacks are born.

The majority of successful attacks carried out against businesses are preceded by chatter over social media or underground chat forums, depending on the sophistication of the attacker. Many attacks are also openly publicised to earn the attacker kudos. And yet very few businesses are monitoring either legitimate or underground sites for this kind of noise.

This seems nonsensical given that even a very basic level of monitoring can prevent reputational compromise. For example, monitoring search engines and social media sites such as Facebook, github, Google+, LinkedIn, twitter and Reddit can ensure the organisation is informed of any compromise of the brand.

There are numerous incidents of hackers hijacking slogans or trademarks and impersonating legitimate businesses in order to convince victims the link they are clicking on is bona fide. Monitoring these sites using key words, phrases and search terms is the bare minimum that a business should be doing.

Going beyond that involves the processing of much higher data volumes. In addition to social media posts, there are forums to monitor and videos, and that’s just what we can see. Delve a little deeper and you enter the murky but well-established world of the dark web.

Here you’ll find Tor IRC anonymising software, shady versions of social media platforms and community sites, websites called onions as well as black markets such as the notorious Silk Road. There are, for example, dark versions of twitter and Snapchat, dark wikis, and libraries documenting hacks and exploits. ‘Onions’ usually require some form of invitation or access code and may well be encrypted. Here there’s all sorts of data up for grabs and with the right know-how you can scan for data leakages and compromised data such as emails, domain names or intellectual property.

Yet extrapolating this information into a meaningful form that can be used for threat intelligence is no mean feat. The complexity of accessing the dark web combined with the sheer amount of data involved, correlation of events, and interpretation of patterns is an enormous undertaking, particularly when you then consider that time is the determining factor here. Processing needs to be done fast and in real-time. Algorithms also need to be used which are able to identify and flag threats and vulnerabilities. Therefore, automated event collection and interrogation is required and for that you need the services of a Security Operations Centre (SOC).

The next generation SOC is able to perform this type of processing and detect patterns, from disparate data sources, real-time, historical data etc. These events can then be threat assessed and interpreted by security analysts to determine the level of risk posed to the enterprise. Forewarned, the enterprise can then align resources to reduce the impact of the attack. For instance, in the event of an emerging DoS attack, protection mechanisms can be switched from monitoring to mitigation mode and network capacity adjusted to weather the attack.

But it doesn’t stop there. This type of cyber response provides insights into future threats and attacks and that type of advance warning system can provide the business with real insights that can guide future strategy and security spend, helping focus security investment. For example, knowledge of sector specific emerging threats such as spear phishing campaigns can help steer staff training.

The problem to date is that such SOC services have typically been the preserve of big business. Thankfully, the threat intelligence sector is now maturing and the commoditisation of services is seeing this kind of deep threat intelligence become available to mainstream business. Tiered SOC services provide entry level options which can scale with the business, ensuring that security budgets are no longer frittered away on numerous point solutions, or the endpoint, but are spent where they are needed.

Going forward, legitimate Internet services will continue to be misapplied and used to attack the enterprise. There’s already evidence of spoof social media profiles being created to target high worth individuals, for instance. And the Internet’s underbelly, the dark web, will continue to be used to plan and coordinate organised attacks. The issue remains whether organisations will continue to do business in blissful ignorance or arm themselves with this ready stream of threat intelligence. We ignore these dark domains at our peril.