Skip to content

Commit 0ddfb94

Browse files
AgbAccountAgbAccount
committed
fix: fix bugs and feature list
*remove oss code and docs *fix async http request bug *format code
1 parent 48843bc commit 0ddfb94

81 files changed

Lines changed: 1892 additions & 5847 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.aoneci/README.md‎

Lines changed: 144 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,144 @@
1+
# AGB SDK CI/CD 配置说明
2+
3+
## 文件结构
4+
5+
```
6+
.aoneci/
7+
├── pipeline.yml # 主要的CI/CD流水线配置
8+
└── README.md # 本说明文档
9+
10+
# 项目根目录下的相关配置文件
11+
.bandit # Bandit安全扫描配置
12+
```
13+
14+
## CI/CD 流水线概览
15+
16+
### 触发条件
17+
- **合并请求 (MR)**: 针对所有分支的开启和更新
18+
- **推送 (Push)**: 针对 `master` 分支和所有 `release_v*` 分支
19+
20+
### 流水线任务
21+
22+
#### 1. 环境准备 (setup)
23+
- 代码统计分析
24+
- Python环境设置 (默认Python 3.10)
25+
26+
#### 2. 代码质量扫描 (code-quality-scan)
27+
- **代码格式检查**: Black, isort
28+
- **代码质量检查**: Flake8
29+
- **类型检查**: MyPy
30+
31+
#### 3. 安全扫描 (security-scan)
32+
- **Bandit**: Python安全漏洞静态分析
33+
- **pip-audit**: 依赖包漏洞扫描
34+
- **Safety**: 已知安全漏洞检查
35+
36+
#### 4. 构建测试 (build-test)
37+
- Python包构建
38+
- 包完整性检查
39+
40+
#### 5. 单元测试 (unit-tests) - **暂时注释**
41+
- 待添加测试用例后启用
42+
- 包含测试覆盖率报告
43+
44+
## 配置参数
45+
46+
### 主要参数
47+
- `python_enabled`: 启用Python代码扫描 (默认: true)
48+
- `python_version`: Python版本 (默认: "3.10")
49+
- `security_scan_enabled`: 启用安全扫描 (默认: true)
50+
- `unit_test_enabled`: 启用单元测试 (默认: false)
51+
52+
### 资源配置
53+
- `runs_on_resources`: 运行时容器资源规格 (默认: "4-16Gi")
54+
55+
## 通知配置
56+
57+
### 钉钉通知
58+
- **触发条件**: 流水线失败时
59+
- **通知用户**: 指定用户ID列表
60+
- **群组通知**: 指定钉钉群组ID
61+
62+
## 环境要求
63+
64+
### 必需工具
65+
- Python 3.10+
66+
- pip
67+
- Git
68+
69+
### 开发依赖
70+
```bash
71+
pip install -e .[dev,test]
72+
```
73+
74+
### 安全扫描工具
75+
- bandit[toml]
76+
- pip-audit
77+
- safety
78+
79+
### 构建工具
80+
- build
81+
- twine
82+
83+
84+
## 安全扫描配置
85+
86+
### Bandit
87+
- 配置文件: `.bandit`
88+
- 扫描目录: `agb/`
89+
- 报告格式: JSON
90+
- 置信度: MEDIUM
91+
92+
### pip-audit
93+
- 扫描已安装的依赖包
94+
- 输出格式: JSON, CycloneDX
95+
96+
### Safety
97+
- 检查已知安全漏洞
98+
- 基于安全数据库
99+
100+
## 启用单元测试
101+
102+
当添加测试用例后，需要：
103+
104+
1. **取消注释** `pipeline.yml` 中的 `unit-tests` 任务
105+
2. 设置 `unit_test_enabled: true`
106+
107+
## 报告和产物
108+
109+
### 保留时间
110+
- **代码质量报告**: 7天
111+
- **安全扫描报告**: 30天
112+
- **构建产物**: 7天
113+
114+
### 产物路径
115+
- Bandit报告: `bandit-report.json`, `bandit-report.txt`
116+
- pip-audit报告: `pip-audit-report.json`
117+
- Safety报告: `safety-report.json`, `safety-report.txt`
118+
- 构建产物: `dist/**/*`
119+
120+
## 故障排除
121+
122+
### 常见问题
123+
1. **依赖安装失败**: 检查 `pyproject.toml` 中的依赖配置
124+
2. **代码格式检查失败**: 运行 `black .` 和 `isort .` 修复
125+
3. **类型检查失败**: 检查MyPy配置和类型注解
126+
4. **安全扫描误报**: 在 `.bandit` 配置中添加跳过规则
127+
128+
### 调试建议
129+
- 检查具体任务的日志输出
130+
- 下载相关的报告文件进行分析
131+
- 确保本地环境与CI环境一致
132+
133+
## 维护
134+
135+
### 定期更新
136+
- Python版本
137+
- 依赖包版本
138+
- 安全扫描工具版本
139+
140+
### 监控指标
141+
- 代码质量得分
142+
- 安全漏洞数量
143+
- 构建成功率
144+
- 测试覆盖率（待启用）

‎.aoneci/pipeline.yml‎

Lines changed: 218 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,218 @@
1+
# AGB Python SDK CI/CD Pipeline
2+
name: "AGB SDK Quality Assurance"
3+
4+
triggers:
5+
merge_request:
6+
target-branches:
7+
- "**"
8+
types:
9+
- "opened"
10+
push:
11+
branches:
12+
- "master"
13+
- "release_v*"
14+
15+
traits:
16+
- type: "notification"
17+
properties:
18+
types:
19+
- "dingtalk"
20+
when:
21+
- "fail"
22+
users:
23+
- "68125" # 替换为实际用户ID
24+
- "187788" # 替换为实际用户ID
25+
webhooks:
26+
- "https://oapi.dingtalk.com/robot/send?access_token=eea6bd5473a4542870d3a33ebd678b8ba214f2f99a756ce04db1c3da8ba32280"
27+
dingtalkGroupIds:
28+
- "cid4XzFY99C8o2VZU/LaPj6BA=="
29+
30+
params:
31+
python_enabled:
32+
name: 启用Python代码扫描
33+
type: boolean
34+
default: true
35+
python_version:
36+
name: Python版本
37+
type: string
38+
default: "3.10"
39+
security_scan_enabled:
40+
name: 启用安全扫描
41+
type: boolean
42+
default: true
43+
unit_test_enabled:
44+
name: 启用单元测试
45+
type: boolean
46+
default: false
47+
format_code_enabled:
48+
name: 启用代码格式检查
49+
type: boolean
50+
default: true
51+
compare_to:
52+
name: 对比目标
53+
description: "默认在PUSH场景下为前一个提交，CR场景下为目标分支"
54+
type: string
55+
advanced: true
56+
default: "${{(git.merge_request != null ? git.merge_request.targetBranch : (git.push != null ? git.push.beforeSha256 : params.before_commit_id)) ?: git.defaultBranch}}"
57+
runs_on_resources:
58+
name: 资源规格
59+
description: "运行时容器资源规格"
60+
default: "4-16Gi"
61+
options:
62+
- "4-16Gi"
63+
- "8-32Gi"
64+
- "16-64Gi"
65+
66+
jobs:
67+
setup:
68+
name: "环境准备"
69+
runs-on:
70+
- "${{params.runs_on_resources}}"
71+
outputs:
72+
python_code: '${{steps.cloc.outputs.Python_code ?: "0"}}'
73+
python_code_inc: '${{steps.cloc.outputs.Python_added_code ?: "0"}}'
74+
image: alios-8u
75+
steps:
76+
- uses: checkout
77+
- uses: cloc
78+
id: cloc
79+
continue-on-error: false
80+
inputs:
81+
compare_to: "${{params.compare_to}}"
82+
write_outputs: true
83+
- uses: setup-env
84+
inputs:
85+
python-version: "${{params.python_version}}"
86+
87+
88+
89+
code-quality-scan:
90+
name: "代码质量扫描"
91+
needs: ["setup"]
92+
when: "${{params.python_enabled}}"
93+
timeout: "30m"
94+
runs-on:
95+
- "${{params.runs_on_resources}}"
96+
steps:
97+
- uses: checkout
98+
- uses: setup-env
99+
inputs:
100+
python-version: "${{params.python_version}}"
101+
102+
103+
# 代码格式检查
104+
- id: format-check
105+
when: "${{params.format_code_enabled}}"
106+
continue-on-error: true
107+
run: |
108+
echo "检查代码格式，不修改文件..."
109+
python -m pip install --upgrade pip
110+
pip install black && black agb tests --exclude "agb/modules/browser/eval" --check --diff
111+
pip install isort && isort --check-only --diff --verbose agb tests --skip agb/modules/browser/eval
112+
echo "格式检查完成"
113+
114+
# 代码质量检查
115+
- id: lint-check
116+
run: |
117+
echo "执行代码质量检查..."
118+
python -m pip install --upgrade pip
119+
pip install flake8 && flake8 agb tests --count --select=E9,F63,F7,F82 --show-source --statistics --exclude=agb/modules/browser/eval
120+
echo "质量检查完成"
121+
122+
# 类型检查
123+
- id: type-check
124+
continue-on-error: false
125+
run: |
126+
echo "执行类型检查..."
127+
python -m pip install --upgrade pip
128+
pip install mypy
129+
pip install types-requests types-aiohttp types-Pillow types-setuptools types-pydantic || true
130+
mypy agb --install-types --no-error-summary --exclude 'agb/modules/browser/eval' --install-types --non-interactive
131+
echo "类型检查完成"
132+
133+
security-scan:
134+
name: "安全扫描"
135+
needs: ["setup"]
136+
when: "${{params.security_scan_enabled}}"
137+
timeout: "20m"
138+
runs-on:
139+
- "${{params.runs_on_resources}}"
140+
steps:
141+
- uses: checkout
142+
- uses: setup-env
143+
inputs:
144+
python-version: "${{params.python_version}}"
145+
146+
147+
# Bandit安全扫描
148+
- id: bandit-scan
149+
continue-on-error: false
150+
run: |
151+
echo "执行Bandit安全扫描..."
152+
python -m pip install --upgrade pip
153+
pip install bandit
154+
python -m bandit -r agb/ -v --skip B105,B106,B107 --exclude agb/modules/browser/eval
155+
echo "Bandit扫描完成"
156+
157+
# Running security scans
158+
- id: pip-audit-scan
159+
continue-on-error: false
160+
run: |
161+
echo "执行pip-audit依赖漏洞扫描..."
162+
python -m pip install --upgrade pip
163+
pip install pip-audit
164+
pip-audit --desc
165+
echo "依赖漏洞扫描完成"
166+
167+
# 单元测试（暂时注释，待添加测试用例后启用）
168+
# unit-tests:
169+
# name: "单元测试"
170+
# needs: ["setup"]
171+
# when: "${{params.unit_test_enabled && #number(jobs.setup.outputs.python_code) > 0}}"
172+
# timeout: "20m"
173+
# runs-on:
174+
# - "${{params.runs_on_resources}}"
175+
# steps:
176+
# - uses: checkout
177+
# - uses: setup-env
178+
# inputs:
179+
# python-version: "${{params.python_version}}"
180+
#
181+
# - id: install-deps
182+
# run: |
183+
# echo "安装测试依赖..."
184+
# pip install -e .[test]
185+
# echo "依赖安装完成"
186+
#
187+
# - id: run-tests
188+
# run: |
189+
# echo "执行单元测试..."
190+
# pytest tests/ -v --cov=agb --cov-report=term-missing --cov-report=xml:coverage.xml --cov-report=html:htmlcov/
191+
# echo "单元测试完成"
192+
#
193+
# - uses: upload-artifact
194+
# inputs:
195+
# retention-days: 30
196+
# path: |
197+
# coverage.xml
198+
# htmlcov/**/*
199+
# .coverage
200+
201+
summary:
202+
name: "质量检查总结"
203+
needs: ["code-quality-scan", "security-scan"]
204+
runs-on:
205+
- "4-16Gi"
206+
steps:
207+
- id: generate-summary
208+
run: |
209+
echo "=========================================="
210+
echo " AGB SDK 质量检查完成"
211+
echo "=========================================="
212+
echo "✅ 代码质量扫描: 已完成"
213+
echo "✅ 安全扫描: 已完成"
214+
215+
echo "📝 单元测试: 待添加测试用例后启用"
216+
echo "=========================================="
217+
echo "请查看各个任务的详细报告获取更多信息"
218+
echo "=========================================="

0 commit comments

Comments
 (0)