Brute-forcing Web Content

Fuzzing tools such as wfuzz can be used to discover web content by trying different paths, with URIs taken from giant wordlists, then analyzing the HTTP status codes of the responses to discover hidden directories and files. wfuzz is versatile and can do both content-discovery and form-manipulation. It's easy to get started with, and because wfuzz supports plugins, recipes, and other advanced features, it can be extended and customized into other workflows.

The quality of the wordlists you're using to brute-force-discover hidden content is important. After installing wfuzz, clone the SecLists GitHub repository (a curated collection of fuzz lists, SQLi injection scripts, XSS snippets, and other generally malicious input) at https://github.com/danielmiessler/SecLists. We can start a scan of the target site simply be replacing the part of the URL we'd like to replace with the wordlist with the FUZZ string:

wfuzz -w ~/Code/SecLists/Discovery/Web-Content/SVNDigger/all.txt --hc 404 http://webscantest.com/FUZZ

As you can tell from the command, we passed in the web-content discovery list from SVNDigger with the -w flag, -hc tells the scan to ignore 404 status codes (hide code), and then the final argument is the URL we want to target:

You can see some interesting points to explore. While the effectiveness of brute-force tools is dictated by their wordlists, you can find effective jumping-off points as long as you do your research.

Keep in mind that brute-forcers are very noisy. Only use them against isolated staging/QA environments, and only with permission. If your brute-forcer overwhelms a production server, it's really no different from a DoS attack.