Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@

ProxyPool 爬虫代理IP池
=======
[![Tests](https://github.com/jhao104/proxy_pool/actions/workflows/test.yml/badge.svg?branch=master)](https://github.com/jhao104/proxy_pool/actions/workflows/test.yml)
[![Tests](https://github.com/jhao104/proxy_pool/actions/workflows/test.yml/badge.svg)](https://github.com/jhao104/proxy_pool/actions/workflows/test.yml)
[![codecov](https://codecov.io/gh/jhao104/proxy_pool/graph/badge.svg?token=8WHGkrQA6E)](https://codecov.io/gh/jhao104/proxy_pool)
[![](https://img.shields.io/badge/Powered%20by-@j_hao104-green.svg)](http://www.spiderpy.cn/blog/)
[![Packagist](https://img.shields.io/packagist/l/doctrine/orm.svg)](https://github.com/jhao104/proxy_pool/blob/master/LICENSE)
[![GitHub contributors](https://img.shields.io/github/contributors/jhao104/proxy_pool.svg)](https://github.com/jhao104/proxy_pool/graphs/contributors)
Expand Down Expand Up @@ -197,7 +198,7 @@ class MyProxyFetcher(BaseFetcher):
|---------------| ---- | -------- | ------ | ----- |------------------------------------------------|
| 66代理 | ✔ | ★ | * | [地址](http://www.66ip.cn/) | [`ip66.py`](/fetcher/sources/ip66.py) |
| 开心代理 | ✔ | ★ | * | [地址](http://www.kxdaili.com/) | [`kxdaili.py`](/fetcher/sources/kxdaili.py) |
| FreeProxyList | ✔ | ★ | * | [地址](https://www.freeproxylists.net/zh/) | [`freeproxylist.py`](/fetcher/sources/freeproxylist.py) |

| 快代理 | ✔ | ★ | * | [地址](https://www.kuaidaili.com/) | [`kuaidaili.py`](/fetcher/sources/kuaidaili.py) |
| 云代理 | ✔ | ★ | * | [地址](http://www.ip3366.net/) | [`ip3366.py`](/fetcher/sources/ip3366.py) |
| 小幻代理 | ✔ | ★★ | * | [地址](https://ip.ihuan.me/) | [`ihuan.py`](/fetcher/sources/ihuan.py) |
Expand Down
1 change: 1 addition & 0 deletions docs/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
- 新增 `PROXY_FETCHER_EXCLUDE` 黑名单配置
- 新增 `proxyPool.py fetcher` 命令查看启用的代理源
8. 新增代理源 **Proxifly**; (2026-06-01)
9. 移除失效代理源 **FreeProxyList**; (2026-06-02)


## 2.4.2 (2024-01-18)
Expand Down
47 changes: 0 additions & 47 deletions fetcher/sources/freeproxylist.py

This file was deleted.

2 changes: 1 addition & 1 deletion requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,4 @@ APScheduler==3.10.0;python_version>="3.10"
APScheduler==3.2.0;python_version<"3.10"
click==8.0.1
Flask==2.1.1
werkzeug==3.1.6
werkzeug>=2.0,<2.2
18 changes: 1 addition & 17 deletions tests/unit/test_fetcher_sources.py
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ class TestFetcherInterface(object):
("fetcher.sources.docip", "DocipFetcher"),
("fetcher.sources.goodips", "GoodipsFetcher"),
("fetcher.sources.geonode", "GeonodeFetcher"),
("fetcher.sources.freeproxylist", "FreeProxyListFetcher"),

("fetcher.sources.kuaidaili", "KuaidailiFetcher"),
("fetcher.sources.freevpnnode", "FreeVPNNodeFetcher"),
("fetcher.sources.scdn", "ScdnFetcher"),
Expand Down Expand Up @@ -201,22 +201,6 @@ def test_fetch_text_fallback(self, mock_wr):
assert "1.2.3.4:8080" in result


class TestFreeProxyListFetcher(object):

@patch("fetcher.sources.freeproxylist.WebRequest")
def test_fetch(self, mock_wr):
from fetcher.sources.freeproxylist import FreeProxyListFetcher
# FreeProxyList 使用 JS 混淆 IP,这里模拟 script 标签中包含编码后的 IP
script_content = "document.write('%31%2E%32%2E%33%2E%34')"
html = '<table><tr class="Odd"><td><script>%s</script></td><td>8080</td></tr></table>' % script_content
tree = etree.HTML(html)
mock_wr.return_value.get.return_value = _make_response(tree=tree)
result = list(FreeProxyListFetcher().fetch())
# 注意:实际 JS 解码逻辑可能需要更复杂的 mock
# 这里主要验证 fetch() 不报错且返回列表
assert isinstance(result, list)


class TestKuaidailiFetcher(object):

@patch("fetcher.sources.kuaidaili.WebRequest")
Expand Down
Loading