AI辅助自动化Web漏洞探测：从信号提取到智能路由决策 暨 DRXAent开发日志 - 其一一、背景：传统自动化扫描的局限在渗透测试和SRC漏洞挖掘的日常工作中，传统的自动化扫描工具（如AWVS、Nessus）虽然能覆盖大量常见漏洞，但面临几个核心问题：上下文缺失：扫描器无法根据目标技术栈动态调整检测策略误报率高：基于静态规则的检测难以处理业务逻辑漏洞无法处理复杂链路：如需要先登录→获取Token→才能测试的IDOR漏洞缺乏反馈闭环：失败时不会分析原因并调整策略基于DRXAgent项目的实践经验，本文分享一种AI辅助+自动化执行+智能路由的漏洞探测架构。二、核心架构设计2.1 整体闭环流程这个设计的关键在于每一轮都会产生可消费的状态，而非简单的"执行-结束"模式。2.2 信号驱动的动态路由不同于传统扫描器的固定流程，我们的路由层基于运行时信号进行动态决策：# app/skills/router.py 核心逻辑 def route(self, state: CybersecurityAgentState) -> tuple[SkillDSL | None, str | None, list[str]]: # 1. 检查是否有活跃的进行中技能 active = self.registry.get(state.active_skill_id) if state.active_skill_id else None if active and self._next_experiment(active, state): return active, self._next_experiment(active, state), [active.skill_id] # 2. Web目标的首步必须是表面探测 first_web_skill = self._first_web_surface_recon(state) if first_web_skill: return first_web_skill, experiment_id, [first_web_skill.skill_id] # 3. Web链式技能候选（基于证据推进） chained_skill = self._next_web_chain_skill(state) if chained_skill: return chained_skill, experiment_id, [chained_skill.skill_id] # 4. Agentic智能决策（多Agent规划） agentic_skill = self._agentic_next_step_skill(state) if agentic_skill: return agentic_skill, experiment_id, [agentic_skill.skill_id] # 5. 基于信号评分的候选排序 scored = [(self._score(skill, state), skill) for skill in self.registry.all()] # LLM Tie-breaker：当分数接近时让LLM做最终决策 selected = self._llm_tie_break(state, scored)关键洞察：路由不是一次性决策，而是"条件触发→证据采集→再决策"的螺旋推进过程。2.3 运行时上下文推导在执行链中，每一步都会从证据中推导新的上下文键，供后续步骤消费：# app/skills/executor.py 中的上下文推导 def _derive_context(self, state: CybersecurityAgentState) -> dict[str, Any]: context: dict[str, Any] = {} # 从历史证据中提取 for ev in state.evidence: tool_data = ev.metadata.get("tool_data", {}) # 提取JWT Token for token in tool_data.get("jwt_tokens", []) or []: if token and token not in jwt_tokens: jwt_tokens.append(token) # 从localStorage/sessionStorage提取凭证 for area in ("local_storage", "session_storage"): for key, value in tool_data.get("storage", {}).get(area, {}).items(): if re.search(r"(jwt|token|bearer|auth)", str(key), re.IGNORECASE): jwt_tokens.append(str(value)) # 提取表单中的文件上传字段 for form in tool_data.get("forms", []) or []: for item in form.get("inputs", []) or []: if (item.get("type") or "").lower() == "file": file_field_names.append(item.get("name")) # 提取数字ID路径（IDOR候选） for item in tool_data.get("object_id_candidates", []) or []: if item and item not in object_id_urls: object_id_urls.append(item) return context实践技巧：上下文推导函数是扩展探测能力的关键钩子。添加新的正则提取模式即可自动增强整个执行链。三、Web技能链的实战示例3.1 完整的授权测试链路对于提供授权凭据的目标，系统会自动执行以下链路：3.2 链式转换的亲和性评分路由层通过_chain_transition_bonus方法为链式执行增加额外权重：def _chain_transition_bonus(self, skill: SkillDSL, experiment_id: str, state: CybersecurityAgentState) -> float: # 获取当前聚焦的信号 focused_signals = signal_names.intersection({ "web-reflection", "sqli", "ssrf", "jwt", "upload-surface" }) # 计算关键词匹配度 keyword_hits = self._detector_focus_hits(skill, focus_text) if keyword_hits: bonus += min(6.0, 1.35 * keyword_hits) # 信号对齐奖励 signal_hits = self._signal_alignment_hits(skill, signal_names) if signal_hits: bonus += min(8.0, 2.1 * signal_hits) # 聚焦信号技能匹配（如"sqli"信号匹配"br-web-sqli-probe"） focus_skill_hits = self._focused_signal_skill_hits(skill, focused_signals) if focus_skill_hits: bonus += min(9.0, 4.6 * focus_skill_hits) # 亲和性规则匹配 bonus += self._apply_affinity_rules(skill, affinity) # 实验路由提示匹配 bonus += self._apply_experiment_routing_hints(skill, experiment_id, affinity) return bonus实战效果：当br-web-js-api-extract提取到包含/api/users/{id}的fetch调用时，系统会：生成web-route和js-api信号为br-web-route-discovery增加8分奖励为br-web-idor-bola-probe增加10分奖励LLM tie-breaker确认后进入IDOR探测3.3 浏览器运行时证据采集# app/browser/extractor.py 中的表面提取 class BrowserSurfaceExtractor: def capture(self, url: str, timeout: int = 8) -> BrowserSnapshot: root = self.session.fetch(url, timeout=timeout) html = root.body.decode("utf-8", errors="replace") parser = _BrowserHTMLParser() parser.feed(html) # 同源链接去重 links = self._dedupe( self._absolute_same_origin(base_url, base_url, item) for item in parser.links ) # 从inline JS提取fetch意图 fetch_intents: list[BrowserFetchIntent] = [] for body in parser.inline_scripts: fetch_intents.extend(self._extract_fetch_intents(body, base_url, base_url)) # 从JS提取localStorage/sessionStorage storage = self._extract_storage(body) # 提取前端路由（React Router/Vue Router等） frontend_routes = self._extract_frontend_routes(body, base_url, base_url) # 识别登录候选路径 login_candidates = [ item for item in [*api_candidates, *frontend_routes] if re.search(r"(login|signin|auth|session|token|oauth|admin)", item, re.IGNORECASE) ] return BrowserSnapshot(...)一个真实案例：在某次测试中，br-web-js-api-extract从Vue路由中发现了/admin/users/:id，从localStorage中提取到了JWT Token，随后br-web-jwt-probe发现该Token未验证签名，最终生成了越权访问管理后台的复现脚本。四、失败处理与自适应调整4.1 失败反射机制当实验失败时，系统不会简单跳过，而是分析失败原因并调整策略：# app/skills/reflector.py def reflect(self, skill: SkillDSL, result: ExperimentResult, verification: VerificationResult) -> ReflectionResult | None: # 基于失败类型生成纠正措施 if reason == FailureReason.INPUT_CONSTRUCTION_ERROR: actions.extend([ "Normalize generated parameters before retry.", "Prefer a simpler single-purpose probe before expanding patterns." ]) elif reason == FailureReason.TOOL_FAILURE: actions.extend([ "Verify tool availability and argument schema.", "Capture the failing tool call in trace before changing skill strategy." ]) elif reason == FailureReason.TARGET_BEHAVIOR_MISMATCH: actions.extend([ "Re-check the parameter context against live evidence.", "Switch to adjacent detector or collect a broader response sample." ]) # 排除相同参数组合避免无限重试 excluded.append(f"{skill.skill_id}:{result.experiment_id}:same_params") elif reason == FailureReason.VERIFICATION_INSUFFICIENT: actions.extend([ "Add a stronger verifier or collect a second evidence source.", "Do not mark the branch complete from this evidence alone." ]) # LLM增强：获取更精细的失败分析 advice = self._llm_advice(skill, result, verification, reason, actions)4.2 LLM辅助的决策增强# 路由阶段的Tie-breaker决策 choice = self.llm.structured( system=( "You are the BR Red Agent skill router tie-breaker. " "Choose only from provided candidates. " "Prefer evidence-grounded, least-assumptive next experiments. " "Return JSON with selected_skill_id, selected_experiment_id, rationale, risk_notes." ), user=compact_json({ "objective": state.objective, "signals": [signal.model_dump(mode="json") for signal in state.skill_signals], "memory": state.memory.model_dump(mode="json"), "knowledge_graph_summary": self.graph_summarizer.summarize(state.knowledge_graph), "candidates": [...] # 高分候选技能 }), schema=RouterChoice, )关键设计原则：LLM只参与决策，不直接操作目标提供结构化上下文（知识图谱摘要、信号列表、执行历史）输出被约束为JSON Schema，确保可解析失败时自动回退到确定性逻辑五、实战配置示例5.1 启动授权Web测试curl -s -X POST http://127.0.0.1:8088/campaigns/run \ -H 'Content-Type: application/json' \ -d '{ "objective": "对授权网站进行安全研究，先做被动Web面探测", "target": { "target_url": "http://target.example.com/", "login_username": "testuser", "login_password": "testpass" }, "authorized_scope": true, "max_loops": 5 }'5.2 日志分析任务curl -s -X POST http://127.0.0.1:8088/campaigns/run \ -H 'Content-Type: application/json' \ -d '{ "objective": "Investigate API gateway logs for BOLA and injection attempts", "target": { "file_path": "/path/to/api_gateway_logs.jsonl" }, "max_loops": 3 }'5.3 环境变量配置# LLM配置（用于智能决策） BR_RED_LLM_BASE_URL=https://api.lkeap.cloud.tencent.com/coding/v3 BR_RED_LLM_API_KEY=your_key_here BR_RED_LLM_MODEL=your_model_name BR_RED_LLM_TIMEOUT_SECONDS=8 # 存储配置 BR_RED_SQLITE_PATH=./data/br_red_agent.sqlite3 # 生成脚本目录（避免热重载干扰） BR_RED_GENERATED_SCRIPT_DIR=/tmp/br_red_scripts/六、开发心得与最佳实践6.1 关于自动化与AI的分工任务类型实现方式原因信号提取确定性正则+JS解析精确、可控、可审计路由评分确定性规则+加权可解释、快速、稳定分数接近时选择LLM tie-breaker需要语义理解失败分析确定性分类+LLM增强保证基础功能，增强边缘情况验证结果解释LLM增强自然语言解释更友好6.2 可复现性与安全边界生成可执行脚本：每个高风险探测生成独立的Python脚本，可人工审计后执行策略门控：command_exec和python_exec默认只用于授权验证和生成的replay脚本凭据脱敏：login_password在trace中自动替换为***REDACTED***执行预算：限制工具调用次数、高风险操作次数、单实验重复次数6.3 扩展新技能的模板# SKILL.md 示例 --- skill_id: br-web-custom-probe name: Custom Vulnerability Probe description: >- Detect custom vulnerability pattern based on specific indicators. detectors: - tags: [custom, api] keywords: ["vulnerable_pattern", "indicator"] experiments: - experiment_id: probe_custom_vuln objective: Probe for custom vulnerability with bounded requests steps: - tool_name: http_probe arguments: url: "{target_url}" param: "{web_param_names}" success_followups: - br-web-custom-exploit failure_followups: - br-web-route-discovery verifiers: - check_id: custom_indicator_present kind: regex target: evidence.content value: "custom_vuln_signal" weight: 1.0 confidence_policy: success_threshold: 0.8 failure_threshold: 0.2 --- ## When to Use When the target shows indicators of custom vulnerability. ## Workflow 1. Identify candidate parameters from runtime evidence 2. Send bounded probe requests 3. Check response for vulnerability signals ## Verification - Response contains `custom_vuln_signal` - No false positive patterns present七、总结与展望现在项目探索了一条"确定性骨架+AI增强"的安全自动化路径：信号驱动：从运行时证据动态推导上下文，而非预设扫描模板智能路由：结合确定性评分与LLM决策，兼顾效率与灵活性链式执行：基于证据推进的自动链路，减少人工编排失败适应：分析失败原因并动态调整策略可审计：生成可复现的脚本和完整的执行轨迹开发日志：2025-04-28