Skip to content
ContentLora

    Tip: press / anywhere to search.

    Explainer

    P-values and statistical significance, explained

    A p-value is the probability of getting a result at least as extreme as the one observed if the null hypothesis, typically "no effect", were true.[1] Results with p below 0.05 are often called "statistically significant", but that threshold is arbitrary, and a p-value does not give the probability that a hypothesis is true or measure how large or important an effect is.[2][3][4] The American Statistical Association's 2016 statement says decisions should not rest only on whether a p-value passes a threshold, and Cochrane asks researchers to report the effect estimate and confidence interval instead.[5][6]

    Editor reviewedUpdated Explainer17 claims · 100% primaryStatistics and evidence, Science
    Reading level: 1 passage differs; the rest is the same for everyone.

    “Statistically significant” appears in research papers, press releases and news stories as if it settled whether a finding is real. It rests on a single number, the p-value, which is widely misunderstood. This page explains what a p-value measures, using the definitions in the NIST/SEMATECH e-Handbook of Statistical Methods and the Cochrane Handbook, and the six principles the American Statistical Association (ASA) published in 2016.[1][7]

    What a p-value is

    A statistical test starts from a null hypothesis, typically that there is no effect or no difference between groups. The p-value is the probability of getting a test result at least as extreme as the one observed if that null hypothesis were true.[1] In the ASA’s words, p-values can indicate how incompatible the data are with a specified statistical model.[3]

    Imagine testing whether a coin is fair. The null hypothesis is “the coin is fair”. If you get a run of heads that would be very unusual for a fair coin, the p-value is small: the result fits poorly with the idea that the coin is fair. A small p-value does not tell you how biased the coin is, or prove that it is biased.[1][4]

    Significance levels and the 0.05 rule

    Before a test, researchers can choose a significance level, called alpha. An alpha of 0.05 means a true null hypothesis will be rejected 5% of the time; NIST calls the choice somewhat arbitrary and notes that 0.1, 0.05 and 0.01 are common in practice.[8] NIST says it is good practice to decide in advance how small a p-value is needed.[9]

    P-values below 0.05 are often reported as “statistically significant”. The Cochrane Handbook says the 0.05 threshold is arbitrary and became common in medical and psychological research largely because test statistics used to be compared against printed tables.[2] Cochrane now tells review authors not to describe results as “statistically significant” or “non-significant”, and to report the confidence interval with the exact p-value.[6]

    A p-value depends on both the size of an effect and its precision. Because precision rises with sample size, an effect of the same size will usually have a smaller p-value in a larger study than in a smaller one.[10] Sample size affects a poll’s margin of error in a similar way: the larger the sample, the smaller the sampling error.[11]

    Common misreadings

    “p = 0.05 means a 5% chance the result is a fluke.” The ASA says p-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone.[3]

    “Significant means important.” A p-value, or statistical significance, does not measure the size of an effect or the importance of a result.[4] The p-value addresses whether an effect is exactly zero, and in a large study a small p-value may reflect a trivial effect.[12]

    “Not significant means no effect.” A p-value above 0.05 is often misread as evidence that an intervention has no effect. Cochrane tells authors to look at the effect estimate and its confidence interval instead.[13]

    The ASA’s six principles

    On 7 March 2016 the ASA released a statement setting out six principles for using and interpreting p-values. ASA president Jessica Utts said it was the first time the community of statisticians, as represented by the ASA board, had issued a statement on these issues.[7]

    1. P-values can indicate how incompatible the data are with a specified statistical model.[3]
    2. They do not measure the probability that the studied hypothesis is true, or that the data were produced by random chance alone.[3]
    3. Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold.[5]
    4. Proper inference requires full reporting and transparency.[5]
    5. A p-value, or statistical significance, does not measure the size of an effect or the importance of a result.[4]
    6. By itself, a p-value is not a good measure of evidence about a model or hypothesis.[4]

    The statement notes that statisticians often supplement or replace p-values with other approaches, including confidence, credibility or prediction intervals, Bayesian methods, likelihood ratios or Bayes factors, and false discovery rates.[14] ASA executive director Ron Wasserstein said the p-value “was never intended to be a substitute for scientific reasoning”.[15]

    Publication bias and p-hacking

    Announcing the statement, then ASA president Jessica Utts said the p-value appeared to have become a gatekeeper for publication in some fields. She described a “file-drawer effect”, in which statistically significant results are much more likely to be published, and practices called p-hacking and data dredging that put the search for small p-values ahead of other reasoning.[16] The ASA’s fourth principle is that proper inference requires full reporting and transparency.[5]

    When you read that a result is “significant”, look for the size of the effect and its confidence interval, and for the absolute numbers behind it; relative vs absolute risk shows how. A significant link between two things also does not by itself show that one causes the other; see correlation vs causation.[6][17]

    Next in the crash course

    Questions readers ask

    What is a p-value in simple terms?

    It is the probability of getting a test result at least as extreme as the one observed if the null hypothesis, typically that there is no effect or no difference, were true. A smaller p-value means the data fit less well with that assumption.[1][3]

    Does p = 0.05 mean there is a 95% chance the finding is true?

    No. The American Statistical Association says p-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone.[3]

    Why is 0.05 the cut-off for statistical significance?

    Mostly convention. The Cochrane Handbook says the 0.05 threshold is arbitrary and became common largely because test statistics used to be compared against printed tables. NIST notes that 0.1, 0.05 and 0.01 are all used in practice.[2][8]

    Does "statistically significant" mean the effect is important?

    No. A p-value does not measure the size of an effect or the importance of a result, and in a large study a small p-value may reflect a trivial effect.[4][12]

    Does a result that is "not significant" prove there is no effect?

    No. The Cochrane Handbook warns that a p-value above 0.05 is often misread as evidence of no effect, and tells authors to look at the effect estimate and its confidence interval instead.[13]

    What is p-hacking?

    Announcing the ASA statement, its then president Jessica Utts said the p-value had become a gatekeeper for publication in some fields, encouraging practices called p-hacking and data dredging that put the search for small p-values ahead of other statistical and scientific reasoning.[16]

    Sources

    · 17 claims from 5 publishers

    Each numbered claim is a statement we checked against the sources listed with it. Tap a number in the text to see its sources in place.

    1. [1]

      A p-value is the probability of getting a test result at least as extreme as the one observed if the null hypothesis (typically, no effect or no difference) is true. confirmedas of 2026-11-14

    2. [2]

      P-values below 0.05 are often reported as "statistically significant", but the Cochrane Handbook says the 0.05 threshold is arbitrary and became common largely because test statistics used to be compared against printed tables. confirmedas of 2026-11-14

    3. [3]

      The ASA's first two principles are that p-values can indicate how incompatible the data are with a specified statistical model, and that they do not measure the probability that the studied hypothesis is true or that the data were produced by random chance alone. confirmedas of 2026-11-14

    4. [4]

      The ASA's fifth and sixth principles are that a p-value, or statistical significance, does not measure the size of an effect or the importance of a result, and that a p-value by itself is not a good measure of evidence about a model or hypothesis. confirmedas of 2026-11-14

    5. [5]

      The ASA's third and fourth principles are that scientific conclusions and business or policy decisions should not rest only on whether a p-value passes a threshold, and that proper inference requires full reporting and transparency. confirmedas of 2026-11-14

    6. [6]

      Cochrane guidance tells review authors not to lean on P-value thresholds or the label "statistically significant", and to report the confidence interval with the exact P value instead. confirmedas of 2026-11-07

    7. [7]

      On 7 March 2016 the American Statistical Association released a statement on statistical significance and p-values setting out six principles for their proper use and interpretation; ASA president Jessica Utts said it was the first time the community of statisticians, as represented by the ASA Board, had issued a statement on these issues. confirmedas of 2026-11-14

    8. [8]

      The significance level, alpha, is the cut-off chosen for a test; an alpha of 0.05 means a true null hypothesis will be rejected 5% of the time, and NIST notes the choice is somewhat arbitrary, with 0.1, 0.05 and 0.01 common in practice. confirmedas of 2026-11-14

    9. [9]

      NIST says it is good practice to decide before a test how small a p-value is needed to reject the null hypothesis. confirmedas of 2026-11-14

    10. [10]

      An effect of the same size will usually have a smaller p-value in a larger study than in a smaller one, because precision increases with sample size. confirmedas of 2026-11-14

    11. [11]

      Larger samples have smaller sampling error, but with diminishing returns; doubling a sample from 1,000 to 2,000 reduces the margin of sampling error by only about one percentage point. confirmedas of 2026-11-14

    12. [12]

      The Cochrane Handbook also warns against assuming a small p-value means an important benefit; the p-value addresses whether an effect is exactly zero, and in a large study a small p-value may reflect a trivial effect. confirmedas of 2026-11-14

    13. [13]

      The Cochrane Handbook warns that a p-value above 0.05 is often misread as evidence that an intervention has no effect, and tells authors to look at the effect estimate and its confidence interval instead. confirmedas of 2026-11-14

    14. [14]

      The ASA statement notes that statisticians often supplement or replace p-values with other approaches, including confidence, credibility or prediction intervals, Bayesian methods, likelihood ratios or Bayes factors, and false discovery rates. confirmedas of 2026-11-14

    15. [15]

      ASA executive director Ron Wasserstein said the p-value was never intended to be a substitute for scientific reasoning. confirmedas of 2026-11-14

    16. [16]

      Announcing the statement, then ASA president Jessica Utts said the p-value appeared to have become a gatekeeper for publication in some fields, producing a "file-drawer effect" in which significant results are much more likely to be published, and practices called p-hacking and data dredging. confirmedas of 2026-11-14

    17. [17]

      A correlation between two variables does not automatically mean that a change in one causes the change in the other. confirmedas of 2026-11-14

    Revision history (1)
    1. Page created.

    Created Nov 14, 2026. Last reviewed by an editor on Nov 14, 2026. Next scheduled review: Nov 14, 2027.

    Cite this page

    "P-values and statistical significance, explained." ContentLora, updated Nov 14, 2026. https://contentlora.com/explain/p-values-and-statistical-significance-explained

    Spotted an error? Suggest a correction or emailcorrections@contentlora.com.