
Description: StormCrawler is an open source web crawler designed to crawl large websites efficiently by scaling horizontally through Apache Storm. It is fault-tolerant and allows integration with other Storm components like machine learning pipelines.
Type: Open Source Test Automation Framework
Founded: 2011
Primary Use: Mobile app testing automation
Supported Platforms: iOS, Android, Windows

Description: Apache Nutch is an open source web crawler software project written in Java. It is used to build web search engines and web archiving systems. Nutch can crawl websites and index page content and metadata.
Type: Cloud-based Test Automation Platform
Founded: 2015
Primary Use: Web, mobile, and API testing
Supported Platforms: Web, iOS, Android, API