Why ChatGPT fails for security research
OpenAI trains ChatGPT with a heavy refusal layer. Ask it to generate an XSS payload for an authorized bug bounty target and you get a lecture. Ask it to walk through a privilege escalation chain and it hedges. Ask it to look up a CVE with real detail — it hallucinates because its training data is stale.
For a professional pentester or bug bounty hunter, this isn't inconvenience — it's a broken workflow. You spend more time prompt-engineering around the refusal than doing the actual research.