Dnstwist returns non-registered domain even though -r flag is used.

[xapax]

Hey!

I have an issue where a DNS server responds with ServFail. When that happens the string !ServFail is added to dns_a, dns_aaaa, dns_ns, and dns_mx. This in turn has the effect that the flag -r will return these domains, even though they aren't registered, which I think would be considered a false positive.

So for example, if I run this I expect that only registered domains are returned.

python3 dnstwist.py -r somedomain.tld -f json

But if it finds a domain that has a misconfigured domain server, which returns ServFail the result of the above command is the following:

    {
        "dns_a": [
            "!ServFail"
        ],
        "dns_aaaa": [
            "!ServFail"
        ],
        "dns_ns": [
            "!ServFail"
        ],
        "domain": "somedomain.tld",
        "fuzzer": "subdomain"
    }

The issue can be solved by not adding the string "!ServFail" to dns_a, dns_aaaa, dns_ns, and dns_mx. In the code below I have commented out those lines, this way the ServFail domain is not returned with the -r flag.

# line 991
				if nxdomain is False:
					try:
						task['dns_a'] = _answer_to_list(resolve(domain, rdtype=dns.rdatatype.A))
						dns_a = True
					except NoNameservers:
						pass
						# task['dns_a'] = ['!ServFail']
					except DNSException as e:
						_debug(e)

					try:
						task['dns_aaaa'] = _answer_to_list(resolve(domain, rdtype=dns.rdatatype.AAAA))
						dns_aaaa = True
					except NoNameservers:
						pass
						# task['dns_aaaa'] = ['!ServFail']
					except DNSException as e:
						_debug(e)

				if nxdomain is False and dns_ns is True:
					try:
						task['dns_mx'] = _answer_to_list(resolve(domain, rdtype=dns.rdatatype.MX))
						dns_mx = True
					except NoNameservers:
						pass
						# task['dns_mx'] = ['!ServFail']
					except DNSException as e:
						_debug(e)
			else:
				try:
					addrinfo = socket.getaddrinfo(domain, None, proto=socket.IPPROTO_TCP)
				except socket.gaierror as e:
					if e.errno == -3:
						pass
						# task['dns_a'] = ['!ServFail']

I'm not sure if setting those values serve some other purpose that I am unaware of.

Thanks for a great project!

[elceef]

When DNS resolver fails to obtain a valid response from the authoritative DNS server, !ServFail is set to indicate that. Did you try using a different DNS resolver with --nameservers argument?

[CrypticSentinel]

Hi Elceef,
I also encounter the same problem. What do you mean by using the --resolver flag? It is not among those present in the Help:

-a, --all Print all DNS records instead of the first ones
-b, --banners Determine HTTP and SMTP service banners
-d FILE, --dictionary FILE Generate more domains using dictionary FILE
-f FORMAT, --format FORMAT Output format: cli, csv, json, list (default: cli)
--fuzzers LIST Use only selected fuzzing algorithms (separated with commas)
-g, --geoip Lookup for GeoIP location
--lsh [LSH] Evaluate web page similarity with LSH algorithm: ssdeep, tlsh (default: ssdeep)
--lsh-url URL Override URL to fetch the original web page from
-m, --mxcheck Check if MX host can be used to intercept emails
-o FILE, --output FILE Save output to FILE
-r, --registered Show only registered domain names
-u, --unregistered Show only unregistered domain names
-p, --phash Render web pages and evaluate visual similarity
--phash-url URL Override URL to render the original web page from
--screenshots DIR Save web page screenshots into DIR
-t NUM, --threads NUM Start specified NUM of threads (default: 6)
-w, --whois Lookup WHOIS database for creation date and registrar
--tld FILE Swap TLD for the original domain from FILE
--nameservers LIST DNS or DoH servers to query (separated with commas)
--useragent STRING Set User-Agent STRING (default: Mozilla/5.0 (linux 64-bit) dnstwist/20240116)

and even trying to type it, the following error message appears

dnstwist --registered --whois domain.com --resolver
usage: /usr/local/bin/dnstwist [OPTION]... DOMAIN
dnstwist: error: unrecognized arguments: --resolver

Thanks

[elceef]

Sorry, I meant --nameservers argument, for example dnstwist.py --nameservers 1.1.1.1.

[CrypticSentinel]

The results are quite different from each other

dnstwist --registered --whois --nameservers 9.9.9.9 domain.com -> permutations: 100.00% of 11303 | found: 5609 | eta: 0m 00s | speed: 4 qps

dnstwist --registered --whois --nameservers 1.1.1.1 domain.com -> permutations: 100.00% of 11303 | found: 1193 | eta: 0m 00s | speed: 7 qps

the same domain, searched via https://dnstwist.it, returns "Scanned [11436] permutations. Found 5 registered"

[elceef]

That many ServFail responses indicates something is interfering your DNS traffic. Can you change your ISP?

[CrypticSentinel]

I confirm this is a problem caused by the DNS filter. Thank you very much.