No way I'm going to pick a best answer on this one...?

Question

If you discovered the way you collected data had a error margin of 100-300%,and that it supported your theory.Would you still use it as sciencetific fact?

Answer 1

Depends.  Suppose data had a wide range of error, BUT ANYWHERE within that range of error, it still supported the theory.

Then it's no problem.  Kyle M is on to something, let me try to make it clear.

Example.  I drop two balls off the Tower of Pisa.  One 10 lbs, one 30 lbs.  They take a few seconds to fall.  I measure the difference in time between when they hit the ground.  The heavy one hits 0.1 second sooner +/- 200%, ie somewhere between .1 seconds later and .3 seconds sooner.  My theory is that light objects fall just as fast as heavy objects.  I'm pretty comfortable I've "proved" it, with data with a large error margin.

EDIT - PAPPY - That's exactly what the theory of gravity says.  The force with which the Earth attracts the heavy sphere is exactly large enough to accelerate it it equally as fast as the lighter sphere.  It's a famous experiment, and a legend exists that it was actually done by Galileo.

http://www.endex.com/gf/buildings/ltpisa...

I rather imagine the experiments which proved Einstein right by showing starlight bending near the Sun during a total eclipse, had a fairly large error margin back in the early 1900s.  It wasn't necessary to show the exact amount of bending.  Just showing that it existed was surprising enough.

EDIT2 - PAPPY - Of course if the error range was such as to throw the theory into question (as would likely be the case), it's not support.  I was just pointing out that that didn't have to be the case, using some famous examples.  It's a minority of times that data that uncertain is still "good" but it's not that unusual.  So it really isn't a simple question, which was my point.

Care to share exactly what you're talking about?

Answer 2

Maybe...for example:  My theory seeks to prove/disprove a correlation.

Let's say my theory is "Exposure to radiation is harmful".  If I know the margin of error is 300%, I know the upper limit of a group of radiation sources registering 20 rem would actually be 80 rem.  I can measure the worst reactions in my subjects and assume they come from the range 0-80 rem.  I then calibrate a group of radiation sources to 40 rem (knowing the upper limit to actually be 160 rem).  Effects worst than the first group can be labeled as "Effects of exposure from 80-160 rem".  By using progressively stronger sources of radiation, I can show my theory is sound, dispite the limitations of my instrumentation.  

This is, in essence, how substances are determined to be carcenogenic today.

Using this tried and true method of deductive causality, one would have to suspect (not assume) global warming causes increased atmospheric concentrations of CO2 (not the other way around), based on observational data from ice core samples.

Edit (Re:  Kyle M):  "Data itself does not have margins of error- your conclusions from the data or your predictions based on the data have margins of error.".......a handicap in golf would represent a conclusion/prediction with a margin of error....a score card often contains a "margin of error" as well, as it often varies greatly from the actual number of strokes taken. j/k  

If data didn't have margins of error, there would be no need to learn the rules of significant digits/figures.

Answer 3

Yes, but I'd try to find more evidence with a smaller margin of error.  I would also acknowledge the margin of error and come up with a solid arguement as to why it is still useable and what it means.

Answer 4

I would if my agenda was the be-all and end-all, and facts were irrelevant.

Answer 5

I actually deal with this a lot.   I'm an engineer, and I'm often called upon by our sales people to see if various projects that we might do are cost effective.  I do a very quick calculation - the savings are sometimes only accurate to plus or minus 100% or more.

If calculations show that a project pays for itself in one month, then the data could be off by 500% and it would still be a good project.    Whether the project pays for itself in one week, one month, or six months - it's all good.

If calculations show a project never pays back - or will take 100 years to pay for itself - then that's enough information for me to know that it is not a good project.  

Sometimes, very inaccurate (order of magnitude) data is still useful.   If I have no idea how long a radioactive material takes to decay, and you tell me it takes 1 billion years, then I know that it basically "never" decays.  Whether its a billion or a trillion or 100 million years.   It's all just a super long time.   On the other hand, if it take a nanosecond to decay, I know that it decays immediately.  Whether it's a nanosecond or a millisecond or maybe even a whole second.  It's all just a super short time.  

But you need to be careful about how you use data like this.  You can't treat the final answer as anything other than a thumbs up or a thumbs down - and not always even that.  

Very interesting question - and very smart to start the conversation without mentioning the topic.

Answer 6

Data itself does not have margins of error- your conclusions from the data or your predictions based on the data have margins of error.

I don't know what you're getting at (actually I do, but it's still a crass question) so there is no more to say other than the above demonstration.

Answer 7

That's why I always say, see who funded the study.

j

Answer 8

Only if it affect my pay check.

Answer 9

Ends justify means, right?  If believers want to end individual rights, if believers want to force everyone to drive small and impractical cars, use specific light bulbs, have gvmt control your thermostat, then +/- 300% would be in the relm of saying it "could" happen.

Answer 10

Of course not.

Answer 11

no.

Answer 12

No   it wouldnt be sientific fact with an error margine of that much now would it?

Answer 13

If the variance on any point were a factor of 2 to 4 but the actual range of values spanned something larger than that, maybe an order of magnitude or two, then yes, I would use it as evidence supporting my theory.  Here's an example of this:

http://www.met.wau.nl/projects/flevo/com...

I know nothing about this paper in particular, but I know that wind stress measurements are particularly noisy, showing huge scatter at a particular value of u*.  But when measured over a large range of u*, you can see clear trends in the data, enough to support models of how u* varies.  So the variance on a particular point alone is not enough to determine the overall quality of the data.

Edit:

Better example of what I mean:

http://dataserv.cetp.ipsl.fr/FLUX/IMAGES...

I don't understand what you mean by if you keep increasing the values the data become pointless.  If the thing you are measuring is stationary in the statistical sense, then even if the point-to-point variance is large, the mean will converge to +/- 20% of the correct value (assuming a factor of three variance in the original data) in only 100 samples.  

Things that are noisy are measured with high reliability all the time in environmental systems.  You just need to be careful about how many samples you need to beat down the noise.  Aerosol size spectra (number of particles per volume per increment of radius) are another good example, where the data are even noiser because there is measurement error in the dependent (number of particles) and independent (radius) variables.  Yet people still are able to say something sensible about aerosol populations.  

You clearly have something specific in mind.  You would get better answers if you weren't so coy about framing your question.  As you posed it, your question is naive in terms of fundamentals of data sampling.  What are you talking about specifically?

Answer 14

If you discovered the way you collected data had a error, then the moral and ethical thing to do it correct the data collection methods and then re-run the experiment.

If you discovered the way you collected data had a error - and if your employer was a corporation that wanted to backup their claims of greatness for their product - then your only decision is how much do you really need to keep this job....  If the answer is "very badly" then do what works best for your employer and stay off the job market.

If your answer is "I can afford to go look for another job", then try to be very careful of their feelings while at the same time getting your bosses to see the value of following my first advice above: "correct the data collection methods and then re-run the experiment".

... that's my two cents worth...

Answer 15

Not if I am honest.

Answer 16

huh???

Answer 17

no not if it had an error margin like that..that could hardly be considered a fact.