Is website data extraction legal?

Question

Is it illegal to use software to automatically extract data from several websites and then use it to form a website with a larger database.

If it is, would it be legal if there was a reference to the site that the information was sourced from?

I need reliably references (pref websites).

Answer 1

I suppose in the UK you would have to ensure you complied with the data protection act with any information you retrieve and decide to use, but I don't guess it would be illegal.

First of all - taking data from a website that isn't OFFERING that data is pretty much hacking.  Don't do that.  Screen scraping is one thing but trying to access their databases etc is not something you, or anyone should be doing.

Websites that want you to use their data are generally the ones that offer API's or feeds, either RSS or in the case of commercial entities, their IT people will normally help to provide information to customers (like stock listings or prices) because it helps you to sell more, and thus buy more from them.

Then there are websites that don't provide feeds, but you can access data via a login or whatever.  Writing an automated script to do the login and get the data out is not (as far as I know) illegal - but do it a lot and you use up their bandwidth and they might not be overly impressed and ask you to stop doing it.

And finally there are websites which will expressly forbid you take their data under their terms and conditions.  Legally, I dont know what they can do - but they will almost certainly terminate your access.

In short - hacking is VERY bad, automatic extraction of info you can get anyway is questionable depending on your methods, and using APIs and RSS (as long as you do so within the sites terms) is fine - if not welcomed; usually at the cost of a back link to the provider.

Answer 2

No, not illegal. The term is "data mining" and it is only the government which is forbidden from doing this through the Privacy Act. Credit agencies do this with financial information, they collect it and sell it. I can collect information of any kind from any place and use it any way I want to. All it has to be is true information. I can not collect false information and use that to slander or libel for example. There are no limits on private business about data mining. Look at the phone companies. They keep a record of EVERY call you make or receive, and recently the shrub, Dubya, signed a bill which makes the phone companies immune from prosecution for releasing that information to Dubya and Co without a search warrant.

Answer 3

Data mining is not illegal, however you may have a problem using the data in another site if the data is copywrited. If you are using the data to sell items from those companies as an affiliate, they would probably be willing to help you. (if that is what you are planning)