With web data extraction algorithms, using PHP HTML parsers, we harvest URLs and contents of search engines like Google, allowing us to apply web data mining techniques with great value for SEO and business intelligence.

Here I’m gonna write how I did to break Google captcha or “automatically bypass” the Google captcha to let one of my online tools (Google Parser) run with a lot of requests and without my intervention.

You probably know about the Google Sorry error page 503, next I’m going to write how to solve and bypass it.

.
First: What’s the problem to solve?

I have an online tool that does requests to Google and gets the search results. When it does too much requests Google ban it and I need to write the letters in the Google captcha to can continue to doing the requests.

.

Second: How Google captcha ban works?

In a few words, when Google receive a lot of requests(there are a lot of another variables) from the same IP, it supposes that the requests are being done by an automatic script or spyware. Then Google ban that IP at least you write the letters of the captcha. If you write the correct letters Google returns a cookie to you that means “I’m a human, give me the search results” and then you can continue doing Google requests.

.

Third: Programming the solution…

The solution to “break Google captcha” is nothing difficult nor brilliant, just showing the captcha to the user who’s using the tool, letting him to write the letters, sending this to Google and saving the cookie to continue with the requests.

This is the final solution, very simple, but the process wasn’t like that. To do this I had to be very carefully in the details of the HTTP requests and beat some Google tricks.

.

Fourth: The results…

Now the script is running, it can manage any amount of requests, there’s no time or number limit and the Google captcha isn’t a problem.

.

The phrase “break Google captcha” isn’t the most accurate for this, but I used it because this post is part of my SEO research too…

Awesome! A lot of servers have their JBoss Management Console open to the world, without any authentication, no password, no security! A huge and silly vulnerability!

Any remote user can completely control the server, having full control to a lot of server configurations and internal network and infrastructure information disclosure, you can change the web service listening port (I test this with one of them, then I put back the original port), view internal IPs and start connections to a client, a lot of server absolute paths, you can change security configurations… too much power with almost no knowledge needed.

This vulnerable JBoss servers let open access to anybody to jmx-console and web-console, these are the online administration tools of JBoss.

There still are a lot of this kind of silly vulnerabilities in the Internet… theres not a JBoss vulnerability, theres a people vulnerability!

Oh, I almost forgot it… you can find all the vulnerable servers using my online Google Parser tool who I wrote a couple of weeks ago. With it you can get a clean list of all the vulnerable sites searching for:

intitle:”jboss management console” “application server” version inurl:”web-console”

or

intitle:”JBoss Management Console – Server Information” “application server” inurl:”web-console” OR inurl:”jmx-console”

You can try different Google search strings and get a clean list of URLs of the Google search results with my Google Parser online tool.

It’s amazing how developers and network administrators still doesn’t pay real attention to security!

Another nice tool, I do penetration tests, sometimes I have to know if a service(for example http) over different IPs was served by different hosts or if it’s really served by the same host.

How can I know this?

That IPs could have another services, different between each IP, for example that hosts could be doing NAT.

So, doing a port scanning, for example with nmap or using any automated tool doesn’t work. If you rest in the results of that tools, you will get wrong results trying to know if are really the same host.

Then, how can I know if this services over different IPs are really served by the same host???

The answer is analyzing the IP header of packets received from this IPs.

(I don’t gonna write a tutorial about networking and the network layer here. You can read the RFCs and read a lot of books, papers and tutorials over this subjects.)

The key is the identification field(32 to 47 bits) in the IP header and the fact that this number is sequential.

Then, if you receive continuous packets from one host, they will have sequential identification numbers.

It’s easy to receive a continue flow of packets from any host and if that host isn’t sending a lot of packets to another IPs, you will have packets with almost sequential identification numbers.

Now, let’s play! A couple of years ago I did a tool to receive packets from different hosts, get and compare the identification numbers received. Now I put it online and you can use it! it’s in Tools section.

If you have any question, you can post it in a comment.

I wrote a perl script to perform certain search in Google, parse the results and save all the harvested URLs in a text file. After a few improvements, I finally made a PHP Google scrapper that allows us, with an HTML parser in their core, to get unlimited Google results to then apply data mining techniques and obtain valuable information for SEO and business intelligence.

This is extremely useful for a lot of things, for example, with e-marketing and SEO related purposes, you can get huge amount of Google results from different keywords to then analyze PageRank, SERP positions, competition companies/domains and much more.

One time I prepare a search string for Google to find sites that have a security vulnerability, then I run an exploit to all this sites and I founded all the vulnerable sites, only with research purposes, another interesting use of this online SEO tool.

This are only a couple of examples… if you use your imagination… you will see a lot of things you can do…

It’s basically a parser of the Google results, so I can get Google results in any format.

Now I write the algorithm(a google parser) in PHP and publish online, you can use it under online SEO tools section and see other interesting tools… or go directly to the Google parser online tool (GooParser).