Can web bots click buttons?

Miscellaneous Forums/General Discussion/Can web bots click buttons?

Basically my site has download and buy buttons and the php scrip attached to the buttons tracks the number of clicks. It keeps going up each day which is cool, but there are an awful lot of buy buttons clicks. I wondered if web bots roving around sites "click" buttons i.e. follow the link and if my php code would then therefore log the click. Is this possible?

Would the same be true of links to the php counter that I post in the showcase forum do you think?

Thanks for any answers.

H&K

Well... Ive always assumed that things could auto click, otherwise why have those "Type the code in" things as part of the continue links

yoko

Bots can make a "submit" action to trick the script, which is used by some surfing bots and fake ad clicker...

FlameDuck

Is this possible?

It is not only possible, but any bot that didn't wouldn't be very useful.

Perturbatio

The way to get around most of the bots (but makes your site slightly less accessible) is by using javascript to dynamically insert your button.
Most bots won't parse and evaluate javascript.

I just knocked up a simple example but you can expand on it easily enough:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
        "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
<head>
<title></title>
	<meta http-equiv="Content-Language" content="en-uk" />
	<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1" />
	<meta name="robots" content="all" />
	<meta http-equiv="imagetoolbar" content="false" />
	<meta name="MSSmartTagsPreventParsing" content="true" />
<script language="javascript" type="text/javascript">
<!--

function makeForm(){
	var formTarget = document.getElementById("formTarget");
	var myForm = createForm("http://www.google.com/search", "get");
	var myInput = createInput("hidden", "h1", "", "en");
		myForm.appendChild(myInput);
	var myBR = document.createElement("BR");
		myForm.appendChild(myBR);
		myInput = createInput("text", "q", "sf", "");
		myForm.appendChild(myInput);
		myInput = createInput("submit", "btnG", "", "Google Search");
		myForm.appendChild(myInput);
		formTarget.appendChild(myForm);
}


function createForm(sAction, sMethod){
	var frm = document.createElement('FORM');
		frm.action = sAction;
		frm.method = sMethod;
	return frm;
}

function createInput(sType, sName, sID, sValue){
	var inpt = document.createElement('INPUT');
		inpt.type = sType;
		inpt.name = sName;
		inpt.id = sID;
		inpt.value = sValue;
	return inpt;
}


window.onload = makeForm;
//-->
</script>
</head>
<body>
<div id="formTarget"></div>
</body>
</html>

Grey Alien

hmm, I guess Ireally mean crawler bots (is that what they are called) by google and msn search engines etc...

Sledge

I doubt any bots from 'reputable' sources will be poking around like that, but even if they are partial to 'that sort of thing' they'll also be the ones that will adhere to your robots.txt file.

Person

I just clicked it now out of curiousity, maybe it's due to people getting put off by BMTmicros order page (somehow??), or just potential buyers 'dipping their toes' to see the process.

If you're on the comprehensive plan with BMT (which I assume you are) you can ask them to change it to suit your needs. Then again I could be completely wrong. Just a thought.

Grey Alien

your robots.txt file.

Yeah I really ought make one of those...

Cierco: Yeah it could just be toe dippers, or people not liking the page but maybe it's bots too...

jfk EO-11110

#10

BTW you can also use the following trick esp. to prevent bots from auto-submit, required oftenly to prevent the abuse of mailer forms:

Add some input fields that should remain blank. you may hide them visually, eg. by using absolute offscreen coords with css style or so. Then the php script will process the request only if the said fields are blank. Because most bots will fill in some crap in every input field automaticly. This way they can be recoginzed.

Grey Alien

#11

good tip thanks

Avon

#12

or just potential buyers 'dipping their toes' to see the process.

I'm guilty of doing that, on stuff that I have a vague interest in. I just want to see what the checkout procedure is like in case I do decide to purchase later on.

andy_mc

#13

yeah I do this too

andy_mc

#14

actuallly I take it to extremes on ebuyer.com by filling my basket up with a killer PC spec, then wondering if I'll ever be able to afford to click the checkout button

Curtastic

#15

I've made some bots in blitzmax and I don't really know what I'm doing but I use something as simple as this to click a button and it works:

HtmlViewGo myhtmlview, "javascript:document.forms['aspnetForm'].submit()"

xlsior

#16

Interesting read about unintended side-effects of poor web design combined with bots:
http://thedailywtf.com/Articles/The_Spider_of_Doom.aspx

Perturbatio

#17

Interesting read about unintended side-effects of poor web design combined with bots

They had it coming, anyone not using sessions for authentication needs shot.

Hujiklo

#18

I find you have to click the damn 'Buy' button just to find the price in a lot of cases...annoys the hell out of me.

Grey Alien

#19

Hey wait, I've just realised something. You lot think I mean FORM BUTTONS, but I don't. I mean an image with a link. Will bots follow an image with a link and thus activate my php download script? I guess they will...

Can I tell the bots to not do this somehow otherwise they cock up my counters!

xlsior

#20

Will bots follow an image with a link and thus activate my php download script? I guess they will...

a link is a link is a link.

All they see and care about if that you have an "A HREF" tag, they don't know or particularly care that it contains an image.

The whole point of those bots is that they eventually crawl the entire internet, discovering new sites by following links from other places and the likes. It doesn't know what's behind a link until it follows it, after all.

You can prevent that behaviour by using the 'nofollow' tag, which keeps the bots from following the links on your page.

for more info on how to do that, see: http://www.robotstxt.org

You could just put the nofollow on any of your pages that contain the shopping links, but leave the others open to be indexed. Another way is to explicitely disallow certain files to be indexed through the robots.txt file. Unfortunately not all bots recognize and honor those restrictions, but the 'big' ones do and those probably account for the majority of the bot-realted hits anyway.

Now, keep in mind: while it may be an inconvenience to have the bots index parts of your site, in general the bots are a GOOD thing since they increase your exposure, and search engine ratings. The more of your pages are indexed and listed in a search engine, the bigger the chance that one of them will be a match for whatever people are searching for.

Another thought: since you have a PHP page that does the actual counting, you could even implement a 'smarter' counter by not incrementing your counters if the user-agent of the visitor indicates a bot...
(Pretty much all bots identify themself that way, e.g. 'googlebot')

either you can only count hits from 'recognized' browsers like Netscape, Internet Explorer, firefox, Opera, safari and the likes, or you can find a list of common bots and specifically tell your PHP page to exclude those from the counter.

alternatively, you could add some PHP code to count each specific user-agent type, and that way you know exactly what browser your visitors are using, AND which bots have seen crawled your site.

Grey Alien

#21

xlsior: very thorough explanation. I'm very grateful thanks. Yeah I'd rather have the page index so the way to do is some kind of tracking of the source and either exclude or separate as you say. I'm going to learn MySQL and PHP (properly) soon so that should give me some good options for storing data. Thanks again.

Malice

#22

crawler bots are able to identify hyperlinks and visit those pages too, so they dont necessarily need to click buttons to access/view any non-secure pages within a directory