Proxy Server Crawler
When we are doing security tests, we always change our IP address to bypass some security strategies. The easiest way to change IP is using a proxy.
Some websites can provide proxy IPs, but none of them can ensure the healthy of those proxy hosts. It’s a horrible thing to check them one by one by hand when you wanna got one. So we can crawl these websites and test every proxy IP automatically.
The project is hosted at https://github.com/xelzmm/proxy_server_crawler.
Proxy Server Crawler is a tool used to crawl public proxy servers from proxy websites. When crawled a proxy server(ip::port::type), it will test the functionality of the server automatically.
Currently supported websites:
Currently supported testing(for http proxy)
- ssl support
- post support
- speed (tested with 10 frequently used sites)
- Python >= 2.7
- Scrapy 1.3.0 (not tested for lower version)
1 2 3 4 5 6 7 8 9 10
The MIT License (MIT)