cewl (Custom Word List generator) is a command-line tool used to generate custom wordlists from web pages. It’s primarily used in penetration testing or password-cracking tasks by scraping words from websites that can then be used in brute-force attacks.
Common cewl Options
- -d (Depth): Defines the depth for following links on the target site. For example, a depth of 2 will allow the tool to follow two levels of links from the initial URL.
- -m (Minimum Word Length): Specifies the minimum length for words to be included in the wordlist. This is useful for excluding short or irrelevant words.
- -o (Output File): Saves the scraped wordlist to a specified file.
- –email: Enables scraping for email addresses found on the web page.
- -v (Verbose Mode): Displays detailed output of the scraping process.
- -ua (User-Agent): Allows specifying a custom User-Agent string to mimic different browsers or devices when making requests to the target site.
Additional Options
- –meta: Scrapes metadata such as author or keywords from the web pages.
- –auth_type / –auth_user / –auth_pass: These options allow specifying authentication credentials if the target site is behind HTTP Basic, Digest, or NTLM authentication.
- -w (Custom Wordlist): Uses an existing wordlist to compare against the scraped data.
- –count: Displays the word count from the resulting scrape.
Examples of cewl Commands
Generate a wordlist from the example.com site:
> cewl https://example.comGenerate a wordlist from the example.com site with a minimum word length of 5:
> cewl https://example.com -m 5Scrape emails from the example.com site:
> cewl https://example.com --emailGenerate a wordlist from example.com, saving the output to words.txt:
> cewl https://example.com -o words.txtGenerate a wordlist from example.com using a custom User-Agent string:
> cewl https://example.com -ua "Mozilla/5.0 (Windows NT 10.0; Win64; x64)"