Unexpected Web Application Attacks: When Not to Trust Your Search Engine

In today’s complex cybersecurity landscape, it is very apparent that threats can and will originate from virtually anywhere and hardly ever follow predictable patterns. We consistently share this knowledge with our customers and use these insights to drive our product updates and roadmap.

This fact alone makes the task of properly securing your applications even more challenging. To effectively secure your applications, it is critical to implement security controls using the zero-trust model. That means you should not rely solely on external reputation sources, threat feeds, or patches. In fact, we have found that many of the threats we identify using behavioral analysis methods go undetected when using the more traditional methods. More accurate and reliable detection is critical to effective application security, especially when attacks come from the most unsuspecting of sources.

“To effectively secure your applications, it is critical to implement security controls using the zero-trust model.”

On a daily basis, we observe thousands of attacks from all over the world, a large majority of which don’t trigger concern. But occasionally, we come across unusual patterns that pique our interest. Several months ago, we noticed a series of attacks of particular interest. These attacks were originating from a somewhat unexpected place here in the U.S., from Yahoo’s web crawlers! While plenty of attacks originate in the U.S., they hardly ever originate from a major Internet property. Yahoo may not be the industry leader they were in their heyday, but they’re still widely leveraged. 

So why was this pattern of attacks so concerning? Many web application owners will explicitly whitelist Yahoo crawlers so as not to impact search results or SEO efforts. This leaves the door wide open for attacks sourced from Yahoo directly.

Let’s take a look at a few requests originating from the attacking IP addresses. Request traffic from this netblock uses the Yahoo Slurp crawler user-agent: 

ThreatX whitelists a couple of known good crawlers: Googlebot and Bing (a decision that we’ll revisit). We do not, however, explicitly whitelist Yahoo Slurp. Because the ThreatX iWAF Sensor easily identifies suspicious payloads, we observed attacks that were mixed in with normal crawl traffic originating from Yahoo and used the Slurp user-agent for attack requests.

We won’t share the full IP addresses, but as you can see below, they originate from this Inktomi netblock, which is owned by Yahoo.

Here are a few examples of the attacks we identified:

1. Directory Traversal attempt

We see these types of attacks frequently. If the web server is vulnerable to this type of attack, the attacker will leverage the exfiltrated usernames to continue compromising the server:

This attack request is a bit more interesting, but it still doesn’t tell us much about the attacking entity. To mitigate the attack, we assign a high-risk score to the entity action and either block or blacklist them:

Our attack logs indicate that multiple IPs from the same Yahoo/Inktomi address range launched other directory traversal and SQL Injection attacks. That normally signals a compromised network rather than a single entity. Even with this insight, we don’t have enough information to identify the entity tool. This next attack provides us with an important clue:

3. Remote code execution attempt

A quick google search on the string reveals that this is most likely SQLi Dumper: “an excellent, advanced, automatic SQL injection tool for testing links that may contain SQL injection problems in Windows.” SQLi Dumper offers a laundry list of attack methods for script kiddies or targeted attackers. The tool can be used to launch exploitation attacks on a particular site and loads other target sites to attack.

What have we learned?

We now know that our customers are being attacked through the Yahoo Slurp netblock. We also know with reasonable certainty, that SQLi Dumper is the tool being used. Because it’s a Windows application, it is being run by a person – a script-kiddie or targeted attacker.

This information allows us to develop a few theories about how this attack is occurring

1. The attacker has discovered an open proxy in Yahoo’s infrastructure and is using it to launch the SQLi Dumper attacks. This is the least likely scenario since we don’t see the SQLi Dumper user agent, though it could be masked.

2.  One or more servers in Yahoo’s Slurp infrastructure have been compromised and are being used to proxy attacks. Perhaps the admins responsible for Yahoo Slurp are not patching their infrastructure and it got Pwned?

3. The attacker has discovered and exploited a vulnerability in the Yahoo Slurp API. Yahoo’s “current” search API seems to use a different engine than Slurp, so we are unable to determine how the attacker is accessing Slurp other than to state that it doesn’t seem to go through the developer search API. Slurp may not publish a public API, or it is only available to partners, one of which has had their credentials compromised.

We don’t know yet which one of these is the case, but our ability to detect this attack reaffirms our advice to implement zero-trust security controls. Application owners that whitelist known ‘high reputation’ crawlers increase their risk of compromise from ‘reputable’ locations, including one portion of Yahoo’s infrastructure.

Because ThreatX can easily detect and block obfuscated attack payloads, we protect our customers against threats, no matter where they originate. 

(Full disclosure: Well in advance of posting this article, we notified Yahoo about their possible compromised network, both via the email associated with the crawler, and the abuse email from whois.)