Percolator
Percolator is an open-source web crawler framework written in Java. It allows developers to easily create customized crawlers for gathering and processing web content.
crawler
web-content
java
Percolator: Open-Source Web Crawler Framework
Percolator is an open-source web crawler framework written in Java, allowing developers to create customized crawlers for gathering and processing web content.
What is Percolator?
Percolator is an open-source Java framework designed for building custom web crawlers. It provides a set of APIs and components that handle common crawling functionality like site scraping, data extraction, URL scheduling, and content processing.
Key features of Percolator include:
- Modular architecture - makes it easy to plug in custom components like parsers, processors, and datastores
- Flexible configuration - supports defining crawler scopes, extraction rules, workflows, and policies
- High performance - optimized for scalability across multiple threads and machines
- Resilient crawling - automatically handles errors, retries, timeouts, and politeness
- Built-in components - comes with commonly-used implementations for fetching, parsing, databasing that can be swapped out
- APIs for extension - developer-friendly for adding custom plugins and functionality
Percolator can be used to create vertical web crawlers for domains like e-commerce, news, search engines, and more. Its flexible design makes it well-suited for teams needing customized scraping solutions.
Percolator Features
Features
- Distributed architecture
- Plugin-based extensibility
- Built-in web crawler
- HTML parsing and processing
- URL filtering
- Data storage
Pricing
- Open Source
Pros
Open source and free
Highly scalable
Easy to customize
Good documentation
Active community support
Cons
Steep learning curve
Requires Java knowledge
Not beginner friendly
Limited out-of-the-box functionality
Official Links
The Best Percolator Alternatives
Top Development and Web Crawling and other similar apps like Percolator
Here are some alternatives to Percolator:
Suggest an alternative ❐
MacOSaiX
MacOSaiX is an open-source operating system designed as a free, community-driven alternative to Apple's macOS. It is based on Darwin, the BSD Unix-based operating system that forms the core of macOS.Like macOS, MacOSaiX features an elegant and intuitive graphical user interface designed for ease of use. It includes a dock,...
GeometriCam
GeometriCam is a photogrammetry software application designed specifically for efficient and intuitive dimensional measurements and 3D modeling using nothing but digital images as input. It utilizes advanced computer vision algorithms to detect features and match them across multiple photographs or video frames in order to reconstruct detailed 3D models.Some key...
Aerograph
Aerograph is a powerful vector graphics editor developed by Escape Motions that is aimed at creative professionals. It provides an intuitive and streamlined workflow for creating high-quality vector illustrations, technical drawings, diagrams, logos, concept art, comic art and more.Some of the standout features in Aerograph include:An advanced brush engine with...
Mosaic Creator
Mosaic Creator is software designed specifically for creating photo mosaics. A photo mosaic is an image that is recreated using hundreds or thousands of smaller images, akin to the pieces of a mosaic. Mosaic Creator allows users to choose a source photo they want to recreate, like a portrait or...
AndreaMosaic
AndreaMosaic is a powerful photo mosaic creation software for Windows. It lets you create stunning photo mosaics made up of thousands of images. The software analyzes the colors, details, and textures of a target photo and breaks it down into regions. It then searches through its library of images to...
Repix by Sumoing Ltd
Repix is a graphic design and image editing software developed and offered by Sumoing Ltd. It provides an extensive set of tools for creating various types of visual content ranging from logos, banners, posters, illustrations to marketing materials, presentations, infographics, web graphics, and more.With Repix, users have access to a...
Trigraphy
Trigraphy is a comprehensive yet easy-to-use diagramming and wireframing application for Windows. It allows users to create a wide variety of diagrams and charts such as flowcharts, organizational charts, UML diagrams, network diagrams, UI mockups, and more.With an intuitive drag-and-drop interface and numerous premade templates, Trigraphy makes it simple for...
WidsMob Montage
WidsMob Montage is a user-friendly Windows software designed specifically for creating impressive photo montages and collages with advanced editing capabilities. It comes packed with a wide range of beautiful templates and effects to help users easily turn their photos into eye-catching montages.Key features include:Intuitive interface and drag-and-drop functionality for quick...
Tipix
Tipix is a visual customer engagement software designed to help businesses drive more traffic, leads, and sales through visual interactivity. It allows anyone to create interactive images like quizzes, calculators, configurators, and more without coding.With Tipix, you can quickly turn static product images into engaging shopping experiences with hover effects,...
PXL
PXL is a feature-rich pixel art and sprite creation program designed specifically for artists and game developers. It runs natively on Windows and provides a streamlined workflow for creating detailed pixel artwork, animated sprites, tilesets, and more.Some key features of PXL include:An intuitive layer-based interfacePowerful animation tools with frame manipulation...
BokashiMaru
BokashiMaru is an open-source knowledge management and documentation platform built as an alternative to tools like Notion, Confluence, and Wiki.js. It allows teams to create wikis, documents, and databases to collect organizational knowledge in one place.Some key features of BokashiMaru include:Intuitive editing interface with WYSIWYG editingReal-time collaboration allowing multiple users...
XnShape
XnShape is a 2D vector graphics and diagramming software for Windows. It can be used to create a variety of graphical designs including illustrations, diagrams, charts, animations, icons, logos, maps, and more.XnShape provides an intuitive and easy-to-use interface for both basic and advanced drawing functionalities. It has various drawing tools...
Deco Sketch
Deco Sketch is a versatile vector graphics software designed for illustrators, designers, and creatives. With its easy-to-use interface and powerful drawing tools, Deco Sketch makes it simple to create stunning vector artworks from scratch.It comes packed with multiple pen and brush tools like the pattern brush, scatter brush, art brush...
PixelWakker
PixelWakker is an open-source application designed for macOS to help prevent burn-in on OLED screens. It works by randomly triggering individual pixels across the entire screen area, shifting where light and dark pixels appear over time.This "pixel exercising" helps prevent static imagery from permanently etching onto OLED displays. PixelWakker runs...
