The American Heritage Dictionary defines the significance level as "the probability of a false rejection of the null hypothesis in a statistical test." Statisticians compare statistical information to this threshold to either refute or uphold a hypothesis. Technically, the statistician does not calculate this probability; he chooses it. A high significance level means there is a large chance that the experiment proves something that is not true. A very small significance level assures the statistician that there is little room to doubt the results.
Define the alternate and the null hypotheses. The alternate hypothesis is the relationship that you hope to prove in an experiment, and the null hypothesis is the relationship that exists if the alternate hypothesis is false. For example, if the alternate hypothesis is "fertiliser makes grass green," then the null hypothesis is "something other than fertiliser makes the grass green."
Choose a significance level for your experiment. A common choice is 0.05 or 5%. At this significance level, there is a 5% chance that the experiment finds the alternate hypothesis to be viable when it is actually not. (
Conduct the experiment and gather data. Scientifically sound experiments are a complicated undertaking, requiring you to test both control and test groups, change only one variable between those two groups, and ensure that other researchers can duplicate your results.
Determine the type of statistic you'll use. Examples of statistical tests are correlation to measure a linear relationship, a t-test to measure the association between two means and a chi-square to measure proportions. Your choice will depend on your hypothesis and significance level.
Input the data into a statistical software program. You can find diverse programs on the market that will help you make sense of your data by running the many complex statistical calculations.
Compare the statistic to the critical value. The particular critical value you'll use will depend on your chosen significance level and on the type of statistic test you've used. If the statistic is lower than the critical value, the finding is not significant and the alternate hypothesis is not viable. If the statistic is higher, the finding is significant, and the alternate hypothesis is viable. (
Run at least three tests of your experiment to ensure that results are valid.
After determining if a relationship is statistically significant, you'll have to identify whether that relationship is strong, moderate or weak. (ref 6)