HTTP Post command; Google search

Hi all.

Straight to the question:

On google, when you type a search and click "search", What exactly gets sent to the server?

I mean, it uses the POST command- but what else?

I've had a fantastic idea and will be using google to prototype it.

You mean the contents of the address bar in the search result such as:

http://www.google.co.nz/search?hl=en&q=hello&btnG=Google+Search&meta=

Haha, So obvious, thanks Skid! :D

And it uses the GET method, not POST.

plus all the other gubbins which are part of the html protocol. There's loads of docs on this though, search for HTML POST GET Protocol and you'll find a gazzillion and 32 pages telling you how to send a browser request. your browser will send what browser you're using, what version of html you can accept, and a blank line at the end etc.

If ever you want to reverse engineer something the hosts file is your friend. Simply setup a Blitz coded TCPIP server on port 80 that echoes inbound traffic to screen and when you get to the bit you want to reverse engineer modify your hosts file to redirect (in this case google.com) to localhost.

eg. a variation of this...

Hi Guys, I was having an empty-head moment :) I know how to handle a server (Wrote simple ones before to allow visualisation of networked games over the browser).... but I didn't know how to make google know what i'd typed when you clicked "send"- however, once Skid showed me, It all came rushing back- a simple modification of the Get command :)

For some reason i thought POST was sent to google detailing the search, Which of course, is as backwards thinking as you can get! :D

Thanks for your comments, and James, Thats the exact code i wrote my old server in :) Thanks! :D

What i'm doing isd reading in Google results and analyzing them. For my product to work i'd need my own search engine but meh, no chance i'm going to do that one anytime soon :P

Cool! You could check Google API, though it might not be useable from Blitz (?). I don't know the first thing about SOAP, WDSL, etc.

Starting to be a bit of a grey area here...
From the Google 'terms of service': http://www.google.com/terms_of_service.html

Some assorted bits:
"The Google Services are made available for your personal, non-commercial use only"
"You may not take the results from a Google search and reformat and display them"
"If you want to make commercial use of the Google Services, you must enter into an agreement with Google to do so in advance."

They seem to explicitely forbit you for harvesting & reformatting their results. Now, they also offer the Google API service which does enable your program to query Google directly. There are a bunch of limitations to that service as well, though:
It requires you to register an account with Google, requires a license key, and limits you to 1,000 queries a day.

They also have some bits about how it is not OK to send automated queries of any sort to Google's system without express permission in advance by Google, and how it's not OK to perform meta-searches either.

Now, technically you can make all of it work, and depending on the scale of the # of queries they may not care a whole lot about what you're doing, just be aware that they have legal options on their side if you do show up on their radar. Other search engines like Altavista appear to have similar restrictions.

Thanks for bringing it up xlsior, i assumed it would be OK for personal use. Its actually for a product aimed for google- Basically a crap filter- Show only real information, not crappy freeweb sites full of misinformation.


I will contact Google as I don't intend on doing anything to harm them, and more than likely will result in a product that would help them.

James, Google API will make the job way easier :) Thanks again!

I'm currently receiving the results, modifying them, detecting each new result, finding the target, and determining if it's a valid result, before adding it to a recomposed webpage. As its been said, Thats certainly against TOS. Using the API- 1000 queries a day is plenty. Might just allow me to do what I want.