In this series of posts, Victoria Willis explores how OSINT (Open Source Intelligence) can be applied in the areas of Cyber Threat Intelligence, IT Asset Discovery, Security Assessments and Attack Surface Monitoring. In this second post of the series, the focus is on the relevance of OSINT when performing asset discovery. Check out the previous post on using OSINT for Cyber Threat Intelligence.

Asset Discovery and Shadow IT

Shadow IT refers to IT systems that are managed outside of the IT department, often without their knowledge. Cloud services, such as enterprise-level SaaS applications, file sharing applications, collaboration tools and social media, are key drivers in the expanding web of shadow IT. Estimates suggest around 80 percent of employees are using SaaS applications at work, and in many cases, these apps are being used without approval from IT. Survey data further reveals that 40 percent of all IT spending occurs outside of IT departments. 

Shadow IT is here to stay, but how does an IT department (let alone a Security department) gain visibility of it and monitor it over time?

These trends are representative of the challenges IT professionals face when it comes to discovering and managing shadow IT infrastructure and applications. It is common for workers to use applications outside of the IT department’s purview and today, the average organization is using 1,427 different cloud services. When IT departments examine their cloud usage, they are finding shadow IT is up to ten times more prevalent than they initially believed. 

In a case study, researchers found that leaving shadow IT unchecked can be quite risky. In just two months, a single organization’s cloud service usage expanded by 15 percent — and employee use of high-risk cloud services went up by 34 percent. During that same two month period, the amount of data uploaded to and downloaded from risky services increased from 6.75 GB to 48.79 GB — an increase of over 600 percent. These are significant changes that may go unnoticed without a consistent asset discovery routine. 

Attackers can use OSINT (Open Source Intelligence) to identify assets and collect other pertinent information prior to launching an attack. Three publicly accessible S3 buckets were tied to major data leaks for multiple Fortune 100 companies in 2019. Some of the information exposed included sensitive employee data, system passwords and other critical information that could be used maliciously. Asset discovery takes a proactive approach to managing shadow IT, allowing organizations to identify areas of risk before they become a problem. 

Controlling Infrastructure and Managing Footprints

Large organizations may find that controlling infrastructure and managing their Internet footprint can come with substantial challenges. The average organization is already home to thousands of cloud-based services, many of which are being used ad-hoc. For larger businesses, there may be many thousands of undocumented or unrecognized assets. Any software, hardware or technology that gets used without IT’s knowledge or management can be considered part of shadow IT. 

A larger workforce also presents more opportunities for shadow IT to grow and expand; for example, if one employee starts using an unapproved service, others might do the same. There may be multiple servers that have long since been forgotten about, or other rogue assets that are still in use but have otherwise managed to escape IT’s purview. And with the recent rise of remote work, controlling infrastructure and discovering new assets is more important than ever. Employees that are working remotely may be using unapproved services or running unverified software while they are outside the office. These assets can be difficult to track, and may be overlooked during inventory and assessment. However, they can still pose a risk to an organization’s security. 

Organizations can’t monitor what they don’t know they have, and in a large network, manually completing tasks associated with identifying new assets can be overwhelming. 

Identifying Assets with OSINT

When it comes to uncovering shadow IT and performing asset discovery, OSINT is an invaluable resource. OSINT in this context refers to publicly available information about an organization, their infrastructure and their employees. There are many different sources of OSINT, and organizations can leverage the collection of this information to identify rogue assets and other publicly available information. OSINT collection is a key element of a successful passive reconnaissance strategy. Gathering intelligence on what devices, services and applications have gone rogue provides security teams and IT with invaluable information about what kinds of exposures their organization may be facing and how many assets may be publicly visible. 

Notably, public AWS S3 buckets are a common choice for exploitation; attackers can abuse a poorly permissioned S3 bucket and use it to obtain user data, credentials and other private information about an organization. Many organizations struggle with maintaining proper S3 storage procedures and this can become a real problem when publicly visible buckets are going undetected. S3 bucket mining is becoming more popular, especially now that more dedicated OSINT tools for combing S3 buckets are available. 

OSINT tools can also be used to identify public code repositories (such as Github) that may contain confidential information about an organization or their staff, compromised social media accounts, forgotten servers and much more. These are all assets that attackers can use for gaining information on an organization while preparing to mount an attack. However, these same tools can be employed by organizations and used to assist with asset discovery. But for many organizations, there are too many rogue assets and too many different tools and data sources for manual OSINT discovery to be conducive to efficiency in security operations. 

Platforms that offer automated scanning can help make the process of using multiple sources of OSINT for data collection and asset identification less arduous and more efficient.

With OSINT automation through SpiderFoot HX, you can configure what data sources to query without having to invest effort in integrating with APIs or OSINT tools.

Continuous Monitoring with OSINT

Asset discovery is the first step in gaining a better view of an organization’s digital footprint. Once new assets have been identified, IT and security operations will want to continue to monitor how their footprint evolves. 

Additionally, continuous monitoring provides insights as to how these assets may change over time and what new exposures their organization may be facing. SpiderFoot is an automated OSINT platform that provides continuous monitoring and automatic notifications when a new concern arises. Whether employee emails suddenly appear in a data breach, or an IP address from your network ends up on a botnet tracker, continuous monitoring services can help organizations ensure that new blips on the radar are being detected. 

Performing regular scans for identifying new assets is good; pairing that practice with continuous monitoring is better. We’ll dig further into that in a future post about Attack Surface Monitoring.