We’re proud to announce that a new SpiderFoot HX release was recently deployed, meaning a bunch of new features designed to better automate your OSINT are available, giving you more time to focus on what matters; analysis.

Screenshotting

One of the most common OSINT analysis patterns when spotting a relevant social media profile, dark website or similar domain is to take a screenshot, so we decided to automate that during a scan so that you have a point-in-time view of what was on the website at the time of the scan. This is integrated into all parts of SpiderFoot – Scanning, Investigations and Monitoring.

Please note that this feature is available to Professional and Enterprise subscribers only.

In the image below, you can see an example of a screenshot taken of a similar domain to the target, visible when browsing data from the scan:

Investigations is now out of beta!

It’s been months in the making but Investigations is finally out of beta, plus in combination with the Screenshots functionality, we have made it so that you can view screenshots in the mouse-over info box for data elements, as well as browse and visualise data from the investigation just as you would a scan – see below.

Two New Modules

Two new modules have been shipped – one for identifying SHA and MD5 hashes in web content, which may indicate hard-coded passwords or other sensitive material. The other module is for identifying Google cloud storage buckets, which compliments existing Amazon, Azure and Digital Ocean bucket finders.

Smaller but useful improvements

  • The targets and modules for a Monitor can now be updated and change detection will not flag any changes on the next scan run if the target list has changed.
  • A new API endpoint is available for obtaining the list of scanner IP addresses. This is reflected in the API documentation to be found within the top right menu, under “API key”.
  • The export list now includes more details about exports, so that you can distinguish one export from another when you have multiple of them.
  • Exporting is now consistent throughout SpiderFoot HX, since before it was only possible to export certain formats in specific parts of the UI.

Higher data retention limits for Hobby, Basic and Standard users

Hobby, Basic and Standard subscribers have all had data retention increased significantly to 90 days, 90 days and 6 months, respectively.

Bug fixes

It’s an ongoing process, but it’s worth highlighting that much work has been invested in further improving the quality of data found during scans and eliminating false positives.

Specifically a few modules had improvements implemented for handling “bad data” from the API provider. One further particularly important fix was around the identification of affiliate e-mail addresses which were previously treated as target e-mail addresses, massively bloating some scans.

As usual, if you spot any issues or have any general feedback, please contact support@spiderfoot.net so we can help!