Add wildcard certificate detection in JSON output #1665
Conversation
This commit implements wildcard certificate detection for subdomains that are covered by wildcard certificates (e.g., *.example.com). When sources return results containing wildcard patterns, the subdomain is now marked with a wildcard_certificate field in JSON output mode. Key changes: - Added WildcardCertificate field to HostEntry and Result structs - Detection logic: checks if result.Value contains "*.subdomain" pattern - Propagates wildcard flag through resolution pipeline - JSON output includes wildcard_certificate field (omitted if false) - Updates version to v2.9.1-dev - Fix .goreleaser.yml 386 architecture quoting - Add /subfinder to .gitignore Note: wildcard_certificate field is not included when using -cs flag to avoid breaking changes to the library API. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
WalkthroughAdds a boolean WildcardCertificate field propagated from enumeration through resolution to JSON outputs; bumps internal version to v2.9.1-dev; updates .gitignore to ignore Changes
Sequence Diagram(s)sequenceDiagram
participant Enum as Enumerate (pkg/runner/enumerate.go)
participant Res as Resolve (pkg/resolve/resolve.go)
participant Out as Output (pkg/runner/outputter.go)
Enum->>Enum: compute isWildcard for subdomain
Enum->>Res: create HostEntry with WildcardCertificate
Note right of Res: WildcardCertificate propagated\ninside Result emissions
Res->>Out: emit Result with WildcardCertificate to callback
Out->>Out: populate JSON result structs\nwith wildcard_certificate
Out->>Client: write JSON outputs
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~35 minutes
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
pkg/runner/enumerate.go (2)
70-73: Wildcard detection heuristic is brittle; prefer normalized, anchored checkstrings.Contains(result.Value, "*."+subdomain) may miss/false-positive across varied source formats. Normalize once and check a left-most wildcard prefix. Also fix typo “furthur” → “further”.
- // check if this subdomain is actually a wildcard subdomain - // that may have furthur subdomains associated with it - isWildcard := strings.Contains(result.Value, "*."+subdomain) + // check if this subdomain came from a wildcard certificate entry + // that may have further subdomains associated with it + isWildcard := isWildcardCert(result.Value, subdomain) + +// add near the top of this file (after replacer) or as a small helper: +func isWildcardCert(value, subdomain string) bool { + raw := strings.TrimSpace(value) + raw = strings.TrimPrefix(raw, "http://") + raw = strings.TrimPrefix(raw, "https://") + // tolerate bullet prefixes some sources use + raw = strings.TrimPrefix(raw, "•.") + raw = strings.TrimPrefix(raw, "•") + // prefer anchored wildcard at left-most label; keep legacy contains as fallback + return strings.HasPrefix(raw, "*.") || strings.Contains(value, "*."+subdomain) +}
96-103: Good: duplicate entries upgrade to wildcardUpgrading existing uniqueMap entries to WildcardCertificate=true avoids losing metadata. Note: ResultCallback is not re-fired on this upgrade; if downstream consumers rely on callback for real-time metadata, consider re-emitting or documenting this behavior.
Would you like a guarded re-emit (e.g., only when flag flips from false→true) to keep callbacks in sync?
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
.goreleaser.ymlis excluded by!**/*.yml
📒 Files selected for processing (5)
.gitignore(1 hunks)pkg/resolve/resolve.go(3 hunks)pkg/runner/banners.go(1 hunks)pkg/runner/enumerate.go(4 hunks)pkg/runner/outputter.go(4 hunks)
🧰 Additional context used
🧬 Code graph analysis (2)
pkg/runner/enumerate.go (1)
pkg/resolve/resolve.go (1)
HostEntry(27-32)
pkg/runner/outputter.go (1)
pkg/resolve/resolve.go (1)
HostEntry(27-32)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: Analyze (go)
- GitHub Check: Lint Test
- GitHub Check: release-test
🔇 Additional comments (11)
pkg/runner/banners.go (1)
20-20: Version bump looks goodNo functional changes; aligns banner/update callback usage.
.gitignore (1)
11-12: Ignore additions are sensibleIgnoring dist and the top-level subfinder binary reduces accidental commits.
pkg/resolve/resolve.go (3)
94-123: WildcardCertificate propagation through all resolve paths: LGTMFlag is carried on pass-through, error, and resolved-subdomain emissions.
28-32: No breaking changes found; all composite literals already use keyed field syntaxAll instances of
resolve.HostEntryconstruction in the codebase (outputter.go:136, enumerate.go:106, enumerate.go:143) already use keyed field syntax. Adding new fields likeWildcardCertificateposes no compilation risk since keyed literals don't depend on field order. Noresolve.Resultunkeyed literals were detected.Likely an incorrect or invalid review comment.
64-71: Range-over-int is supported; no compatibility issue existsThe go.mod specifies
go 1.24.0, which fully supports range-over-int (introduced in Go 1.22). Since the minimum supported Go version is 1.24.0, the code is compatible with the toolchain. No changes required.pkg/runner/enumerate.go (3)
106-118: Propagation into HostEntry and initial callback: LGTMHostEntry carries WildcardCertificate; callback gated on !RemoveWildcard is consistent with existing flow.
143-144: Callback on resolved results includes wildcard flag: LGTMKeeps parity between non-resolve and resolve paths.
122-124: Task channel closure timing: LGTMClose after draining passive results to let workers terminate cleanly; paired with pool’s close(Results) after wg.Wait.
pkg/runner/outputter.go (3)
22-26: JSON schema extension is soundwildcard_certificate added with ,omitempty across host/IP and source variants. For -cs path, field stays omitted (no assignment), preserving backward compatibility.
Please confirm docs/examples for JSON (-oJ) mention the new field.
Also applies to: 29-34, 37-41
123-124: HostIP JSON propagation: LGTMFlag wired into jsonSourceIPResult.
181-182: Host JSON propagation: LGTMFlag wired into jsonSourceResult.
Fixed a bug where the wildcard_certificate field was being lost when using the -nW flag (DNS resolution with wildcard filtering). The issue occurred because when multiple sources find the same subdomain, only the first occurrence is sent to the resolution pool. If a later source marks the subdomain as having a wildcard certificate, that information was stored in uniqueMap but never propagated to foundResults. Solution: After resolution completes, merge wildcard certificate information from uniqueMap into foundResults. This ensures that if any source marked a subdomain as having a wildcard certificate, that flag is preserved in the final output. This ensures consistent behavior - wildcard_certificate field appears in JSON output regardless of whether -nW flag is used. Validation: - Without -nW: api.nuclei.sh shows wildcard_certificate:true ✓ - With -nW: api.nuclei.sh shows wildcard_certificate:true ✓ 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
pkg/runner/enumerate.go(4 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
pkg/runner/enumerate.go (1)
pkg/resolve/resolve.go (1)
HostEntry(27-32)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: Test Builds (macOS-latest)
- GitHub Check: Test Builds (ubuntu-latest)
- GitHub Check: Test Builds (windows-latest)
- GitHub Check: Analyze (go)
🔇 Additional comments (4)
pkg/runner/enumerate.go (4)
96-102: Good handling of duplicate wildcard status updates.The logic correctly updates an existing subdomain entry to mark it as a wildcard when a later source identifies it as such, even if it was initially discovered without the wildcard flag. The conditional check prevents unnecessary updates when the entry is already marked as a wildcard.
106-106: LGTM!The wildcard status is correctly propagated into the HostEntry structure at creation time, consistent with the enhanced struct definition.
143-143: LGTM!The wildcard certificate status is correctly propagated through the ResultCallback in RemoveWildcard mode, ensuring consistency across both enumeration paths.
149-157: Good defensive merge of wildcard status.This merge correctly handles the edge case where a subdomain is sent to the resolution pool before a subsequent source marks it as a wildcard. The iteration over
foundResultsis safe here because it occurs afterwg.Wait()ensures the enumeration goroutine has completed, preventing race conditions.
Summary
This PR implements wildcard certificate detection for subdomain enumeration results. When a subdomain is covered by a wildcard certificate (e.g.,
*.example.com), it is now identified and marked in the JSON output with awildcard_certificatefield.Functional Testing
Test Environment
-oJ -nW(JSON output with DNS resolution and wildcard filtering)Test Results
Command:
Sample Output:
{"host":"api.nuclei.sh","input":"nuclei.sh","source":"crtsh","wildcard_certificate":true} {"host":"retest-prod.api.nuclei.sh","input":"nuclei.sh","source":"crtsh"} {"host":"identity.nuclei.sh","input":"nuclei.sh","source":"crtsh"} {"host":"templates-api.nuclei.sh","input":"nuclei.sh","source":"virustotal"} {"host":"retest-dev.api.nuclei.sh","input":"nuclei.sh","source":"securitytrails"} {"host":"docs.nuclei.sh","input":"nuclei.sh","source":"virustotal"} {"host":"testing-takeover-ec2.nuclei.sh","input":"nuclei.sh","source":"securitytrails"} {"host":"template.nuclei.sh","input":"nuclei.sh","source":"crtsh"} {"host":"templates.nuclei.sh","input":"nuclei.sh","source":"virustotal"} {"host":"scan-prod.api.nuclei.sh","input":"nuclei.sh","source":"securitytrails"} {"host":"app.nuclei.sh","input":"nuclei.sh","source":"virustotal"} {"host":"pdcp-dev.metabase.nuclei.sh","input":"nuclei.sh","source":"securitytrails"} {"host":"scan-dev.api.nuclei.sh","input":"nuclei.sh","source":"securitytrails"} {"host":"www.nuclei.sh","input":"nuclei.sh","source":"crtsh"} {"host":"tm.nuclei.sh","input":"nuclei.sh","source":"crtsh"} {"host":"cve-dev.nuclei.sh","input":"nuclei.sh","source":"crtsh"} {"host":"form.nuclei.sh","input":"nuclei.sh","source":"crtsh"} {"host":"pdcp.metabase.nuclei.sh","input":"nuclei.sh","source":"securitytrails"}Results Summary:
Wildcard Certificate Detection
When a source returns a subdomain with a wildcard certificate pattern (e.g.,
*.api.nuclei.sh), the JSON output will include:{"host":"api.nuclei.sh","input":"nuclei.sh","source":"virustotal","wildcard_certificate":true}The
wildcard_certificatefield is omitted whenfalse(usingomitemptytag), keeping output clean for standard subdomains.Behavior Validation
Standard Mode (without -cs flag)
✅ JSON output includes
wildcard_certificatefield when applicable✅ Field is omitted when value is false (clean output)
✅ Works correctly with
-nWflag (DNS resolution + wildcard filtering)✅ Works correctly with
-oJflag (JSON output)Capture Sources Mode (-cs flag)
-cs(capture sources) flag, thewildcard_certificatefield is not included in the JSON output to avoid breaking changes to the library API. This ensures backward compatibility for existing integrations that rely on the current output format when capturing multiple sources.Technical Implementation
Detection Logic
*.subdomainWildcardCertificate: trueData Flow
*.example.comWildcardCertificate: truewildcard_certificatefieldFiles Modified
pkg/resolve/resolve.go- Added WildcardCertificate to HostEntry and Result structspkg/runner/enumerate.go- Implemented wildcard detection logicpkg/runner/outputter.go- Added wildcard_certificate to JSON output structspkg/runner/banners.go- Updated version to v2.9.1-dev.goreleaser.yml- Fixed 386 architecture quoting.gitignore- Added /subfinder binaryTest Plan
-nWflag functions correctlyCloses
#1656
🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Chores