Subdomain Takeovers, 0Day Exploits for CVEs, Perceptual Analysis…. A Methodology for conducting Vulnerability Assessments at Scale

7 min readOct 21, 2022

In this blog, I’m going to put emphasis on how, we at ScanFactory, have managed to report at least 10+ critical vulnerabilities to every client of ours & how anyone else can utilize our methodology to conduct Vulnerability Assessments at a larger scale. Unlike my last blog, I’ll try to keep this one short & straightforward.

Tools of the Trade

cvemon — exploit search utility
bug-bounty-recon-dataset — recon data for public bug bounty programs
shottie — screenshot capturing utility
perceptic — screenshot hashing utility
sdto — subdomain takeover utility
rescro — regex identification utility
scanny — premium telegram bot for recon automation

Tricks of the Trade

CVEMON

This year alone, 20k CVEs were assigned. On average, at least 1k get assigned every month. And around 40–50% of these vulnerabilities are for Web2 applications & frameworks. These are some very rough estimates but these numbers are gradually increasing as CVEs, as an standard, are getting more widely adopted.

Before I started the CVEMON, these initial estimates were my motivation. And now, as I am beginning to dive deep into these numbers, being among the first ones to have 0day exploits for widely-adopted web frameworks feels like some real power.

Organizations like Tenable & MITRE are the ones which benefit the most from the CVE standard. Although the disclosure procedure is quite regulated, there are no strict laws governing the exploits market & Platforms like Zerodium & Detectify are known to be among the biggest players. Exploits for CVEs get more actively shared once IDs get assigned. And that’s what we at ScanFactory have been monitoring since the beginning of this decade.

CVEMON is a raw dataset produced & catalogued from the freely available data on the surface web. From Medium articles to YouTube videos, It has got every resource related to a CVE. There are a lot more private & public sources whose existence we haven’t publicly acknowledged yet, but this is where it all gets collected eventually. Timing is the real edge that we have got over the adversaries & our competitors.

This raw dataset is further processed by our proprietary algorithms & analysts to produce intelligence that gets pushed as plugins & signatures onto our platform once they are verified against real-world systems. Our public repository on GitHub is entirely an unprocessed raw dataset that can be used by researchers to monitor Proof-of-Concept Exploits/Signatures/Remediations for CVEs. It’s entirely open for community contributions.

Apart from detecting newly authored 0day exploits, utilizing tools & custom scripts like xlocate, an individual can triangulate CVEs for which exploits are public but rare or unpopular. A lot more can be done which, unfortunately, can’t be revealed yet by us.

BUG BOUNTY RECON DATASET

We monitor around 500+ Private & Public Bug Bounty Programs on multiple platforms including Hackerone. Programs on these platforms are known to have a WhiteHat Security Policy in place which gives Safe Harbor to security researchers for testing and reporting vulnerabilities.

Although I, personally feel present landscape needs stricter laws & policies, I do support the security researchers trying to do a good deed. That’s why, we started this project where we monitor the scopes of public bug bounty programs & giveaway the results at no cost. We started with hosting this dataset on GitHub but due to extreme abuse via automated tools & requests from multiple threat intelligence teams, we moved it on Mega.

PERCEPTUAL ANALYSIS

When things are getting done at large scale, the ability to perceive true nature of subjects can sometimes become difficult. Especially when you can’t afford to ignore the details, the overview of things start getting blurry. For example, even an expert sniper can miss with a heavy machine gun. But if the scope is bigger & accuracy doesn’t matter much then coverage becomes the significant component & even an amateur shooter can inflict some real damage with the same heavy machine gun.

But Vulnerability Assessment at Scale require both Coverage & Accuracy. Stealth is what truly matters in conducting vulnerability assessments at scale.

For that purpose, we recently released alpha versions of two components — A Capturer & A Hasher. Shottie captures screenshots while bypassing firewalls & Perceptic clusters them by their similar looks which is determined by a 14 character alphanumeric hash digest. This helps us in automating fingerprinting of technologies by just the way they look so that we run only contextual exploits & wordlists against it.

The version we released for these tools are pretty basic & easy to use. Not so long ago, Bishop Fox, got a patent around similar technology & Shodan has also integrated the same in their platform. As I already stated earlier, these tools can be utilized to fingerprint & cluster web applications on basis of how they get rendered. It has been useful to us under all circumstances where time was limited and yet the scope was unlimited.

SUBDOMAIN TAKEOVERS & API CREDENTIAL LEAKS

Probably the most easy & widespread vulnerability to find is a Subdomain Takeover. Every year, around a dozen good blogs get released around exploiting it. They definitely need no introduction. But yeah, very few will ever think about non-traditional ways of detecting it.

The ability of cloud-based services offering custom subdomain integration is increasing but very few of them provide a secure way of integrating it. Finding new services that are prone to such attacks can result in greater coverage in case they have got wider client-base.

We have got our ways for finding such services, but the most obvious yet less adopted one is large scale DNS + Perceptual Analysis. Our Bug Bounty Recon Dataset & Perceptual Analysis Tools will prove to be very handy for finding such services.

We also have got another tool we call — SDTO, where we have included regular expressions for identifying such services. The best part about this tool is that it can be easily integrated as a library. One can easily train a M.L. algorithm around the methodology I previously mentioned, then integrate this library in that algorithm to create & identify signature all in a single Python script in an automated manner. I understand if it sounds like a lot of work for just a subdomain takeover, but believe me, it’ll be worth all the time it consumes.

That’s the primary reason why we didn’t use tools like Nuclei & Jaeles. Neither they aren’t as integrable as we needed them to be nor they are compatible with Python Regexes.

Similar methodology can be adopted for identifying API Credentials that are getting leaked in HTML/JS Source Codes utilizing — Rescro. Many tools have already incorporated entropy analysis to identify such instances and they too have to be proven extremely effective. Presence of false positives is pretty trivial in either cases. Please note that our tools are riddled with bugs. Some are obvious, others not so much. Good Luck finding and fixing them. And feel free to submit PRs. Great ones will be rewarded by time.

TELEGRAM RECON BOT

To keep things simpler & our methodologies more accessible, we have released — Scanny. This Telegram Recon Bot is extremely straightforward in terms of usage & output. Its recon data includes, but is not limited to, frequently updated subdomains, open ports, web servers, fingerprints & screenshots.

Our bot has proven to be extremely cost-effective for long term. Tools & Services like Shodan, SecurityTrails, Censys etc. are like Machine Guns. But Scanny is just like a Sniper Rifle. They are also pretty much incomplete when it comes to conducting a recon. In long term, they turn out to be very late and unreliable. Time & Coverage is of crucial importance in such scenarios.

Unfortunately, our bot is only meant for targets having a WhiteHat Security Policy & our team verifies its presence before issuing the subscription.

We are running a promotion for Scanny from now until the end of this month. Each subscription will automatically cost 50% less than its standard cost.

Follow ScanFactory on Twitter — https://twitter.com/scanfactory_io , Share this blog & don’t forget to mention our handle. We randomly select from among our followers for giveaways of vouchers & exploits.

That concludes my blog. Ping me on social media in case you got any queries. Have a great time!

We do not live in an ideal world. Good, Bad, Right, Wrong…. all these ideological, philosophical & self-explanatory terms are relative that might or might not make universal sense to entire mankind but power does for sure. Power is absolute & its definition is universal. Power brings control & elevates inherent ethical tendencies.