tag:blogger.com,1999:blog-192445222306631874.post6038622255323191735..comments2022-03-30T02:22:30.689-07:00Comments on Xenia Schmalz's blog: What’s wrong with p-values, and (why) are Bayes Factors better?Xenia Schmalzhttp://www.blogger.com/profile/02238923475669435076noreply@blogger.comBlogger3125tag:blogger.com,1999:blog-192445222306631874.post-21991889247047042022015-08-10T03:40:18.336-07:002015-08-10T03:40:18.336-07:00Dear Stephen,
Thanks for your comment, and for the...Dear Stephen,<br />Thanks for your comment, and for the interesting link! <br />I was trying to be provocative with that statement. :) I would be reluctant to bet on a coin being fair after 100000 tosses and with a tiny p-value. Having a large n-size does tell us something, but I think it's fair to say that this is not how p-values are actually used by many researchers in psychology. Often, conclusions about the presence or absence of an effect are drawn only from the p-value of a single underpowered study, which is kind of like drawing conclusions about the fairness of a coin after tossing it once.Xenia Schmalzhttps://www.blogger.com/profile/02238923475669435076noreply@blogger.comtag:blogger.com,1999:blog-192445222306631874.post-91246167405041004212015-08-10T03:26:57.002-07:002015-08-10T03:26:57.002-07:00This comment has been removed by the author.Xenia Schmalzhttps://www.blogger.com/profile/02238923475669435076noreply@blogger.comtag:blogger.com,1999:blog-192445222306631874.post-28803946883050767992015-08-10T00:45:55.827-07:002015-08-10T00:45:55.827-07:00Thanks for posting this. I do have one comment, a...Thanks for posting this. I do have one comment, and it may be that I've misread you. The idea that a single P value is a single datum and can never tell you anything seems odd to me - it seems to ignore the sample size on which that P-value was based. The "problem" with the single P=0.03 in the example isn't that it's a single P-value; it's that its value is in the region where we need to think carefully about the rate and consequences of false positives. If I flipped a coin 100,000 times and got a P-value of 0.0000001 for a test that it was fair, would you keep betting on it at 1:1 because there was only a single datum and we couldn't conclude anything about its fairness? I'll be waiting to take your bet :-)<br /><br />P values come under a lot of unfair attack, as you acknowledge in your post - it's mostly not about P values, but about the way they are used and (behind that) some surprisingly frequent misunderstandings about what they do. More about this here: http://wp.me/p5x2kS-Y.Stephen Heardhttps://www.blogger.com/profile/01447261122370883924noreply@blogger.com