Crawler4j is an open source web crawler for Java, available from GitHub. This means everyone can download and employ the crawler. GitHub describes the crawler as follows: crawler4j is an open source web crawler for Java which provides a simple interface for crawling the Web. Using it, you can setup a multi-threaded web crawler in few minutes. Since everyone can use it, it is impossible to know what is done with the data.
The Internet of Bots has evaluated Crawler4j against 50 different checkpoints, of which 18 have been confirmed as being positive. These checkpoints evaluate the transparency and occurrence of a bot and don´t necessarily say something about its quality. The BotRank is calculated as [18*2=] 36%. The details of the bot and the BotRank can be seen below. For more information on how the BotRank is made, you can visit the page Botrank.