With this release, we’ve overhauled the interfaces for creating new Scans and Monitors so that you can control the depth (iteration) of your scanning and monitoring. We’ve also introduced new modules contributed from the open source community and also scrapped some modules where their data sources are longer active; reducing errors and speeding up scans overall. This, in addition to a bunch of UI/UX improvements! Read on for more…

Iteration control

SpiderFoot has always been iterative in nature. But what does that mean?

Iteration is when SpiderFoot identifies a new entity (e.g. an IP address) related to your target during a scan. When identified, SpiderFoot will trigger all relevant modules for that new entity. This process automatically continues until there are no further entities related to your target to explore.

While this can reveal a tremendous amount of information about a target and saves a ton of time while performing reconnaissance, it can also be a distraction for those only seeking to find information strictly about their target and nothing else. And so, we have updated the New Scan screen to give you the ability to control that behavior:

This new interface gives you the ability to enable/disable iteration generally for the scan, or enable it and then tune specifically where you want to enable iteration. Enabling all iteration is equivalent to how scans have been conducted historically, so if you simply want the same behavior as the past, there is no action to be taken.

Module Changes

Thanks to contributions by @_bcoles to the Open Source version, we have three new modules in this release:

  • DNSGrep
  • Open Passive DNS database
  • NetworksDB

A number of dead data sources have also been removed:

  • SSL tools
  • ThreatExpert.com
  • Hosts-file.net
  • Numpi
  • Nothink (stale honeypot data)

Coming soon is Dehashed, SpySe, Bad Packets and Viper Data. If you have requests for integrations, please contact support.

Tons of UI improvements

We’re big believers in details and creating a great user experience, so we’ve implemented a number of small but meaningful improvements to the SpiderFoot HX user experience:

  • The New Scan and New Monitor interfaces show API key and subscription plan information
  • Correlations and Screenshotting can be enabled/disabled on a per-scan basis
  • Module lists in the New Scan, New Monitor and Scan Profile interfaces enable searching and selecting/de-selecting based on module attributes.
  • Alerts are now pop-ups, not invasive dialog boxes
  • Modules are shown everywhere as names, not as their internal module name
  • When exports complete, you will receive an email notification
  • A “Latest Updates” section has been added so you can find the latest updates in the UI
  • API keys are masked in module configuration and the API key manager
  • Scheduled scans can now be aborted
  • Mousing over nodes on a Discovery Path visualisation presents a link to view details about that node