|
|
Friday 9th July 2010
The mystery ends
So, anyway, not that anyone cares or anything but I've worked out why I couldn't get on RightMove. One of my PCs (the main one) is configured as a Tor relay. What's a Tor relay, I here thousands of people cry out . . . well, all is explained here. The technical people at RightMove don't like Tor relays, as is explained by Luke:
I have checked your IP address and it is a TOR node
(http://www.torproject.org/). We do not allow TOR nodes access to our site
as we have had so many cases of automatic data gathering by people, and they
are impossible to stop.
Your IP appears in the list from this request which is one of the external
IP's we use.
https://check.torproject.org/cgi-bin/TorBulkExitList.py?ip=213.161.94.30
However, as it turns out, RightMove have two other IPs that they like to protect. Hence, they also take advice from:
https://check.torproject.org/cgi-bin/TorBulkExitList.py?ip=80.64.55.62
and
https://check.torproject.org/cgi-bin/TorBulkExitList.py?ip=146.101.201.254
Now, as my IP address, 81.168.73.40, was listed in all three of these lists, I was banned from RightMove. How weird is that? So why are Tor relay's banned anyway? Luke? Well, it's apparently all to do with 'automatic data gathering':
"Automatic data gathering" is when someone uses CURL or wget to take just e.g. property details page data, no images, nor page framework. There is no usual pattern, but one type I see is to track price changes for a (sometimes huge) number of properties on a daily basis. This is not what our site is for, we are passionate about the quality of our data and the people we serve it to, so we have to protect ourselves from this abuse of our data.
So then I'm like, "Okay. Fair enough." So, I add this line to my Tor config file:
ExitPolicy reject 213.161.94.30:*,reject 80.64.55.62:*,reject 146.101.201.254:*
. . . and then restart Tor (thereby making the above changes come into effect) but then also set up two cron jobs to stop Tor at 9am and then restart at 11pm, every day (I was getting too much traffic and the bandwidth limits weren't sufficient). And to cut another story short, with the above changes made, I can now go to RightMove. Thanks, Luke!
|
|
|
|