TTests.ppt

Editor's Notes

• #4: We can use information about distributions to decide how probable it is that the results of an experiment looking at variable x support a particular hypothesis about the distribution of variable y in the population. = central aim of experimental science This is how statistical tests work: test a sample distribution (our experimental results) against a hypothesised distribution, resulting in a ‘p’ value for how likely it is that we would obtain our results under the null hypothesis (null hypothesis = there is no effect or difference between conditions) – i.e. how likely it is that our results were a fluke!
• #5: Normal distribution The x-axis represents the values of a particular variable The y-axis represents the proportion of members of the population that have each value of the variable The area under the curve represents probability Mean and standard deviation tell you the basic features of a distribution mean = average value of all members of the group standard deviation = a measure of how much the values of individual members vary in relation to the mean The normal distribution is symmetrical about the mean 68% of the normal distribution lies within 1 s.d. of the mean Now, it’s important to remember that not all data has this distribution and if you data doesn’t fit this normal distribution, you will have to use another type of test. Normal distribution of data is an assumption of T-tests. If your data doesn’t look like this, you have to try another statistical test (like Chi Squared)
• #6: A hypothesis is a prediction that you have about a specific group. H1 = the experimental hypothesis – significant difference (or between samples) H0 = the null hypothesis states that your experimental group is no different from the rest of the population. statistically significant difference between sample & population (or between samples) To get a statistically significant results, you need to show that your experimental group falls on the higher end of probability. One-sided test One sided test is like it sounds. The value for the null hypothesis is an entire section of the distribution. You would use this if you wanted to know if your experimental group is only significantly greater than another group. A one-sided test is a statistical hypothesis test in which the values for which we can reject the null hypothesis, H0 are located entirely in one tail of the probability distribution. In other words, the critical region for a one-sided test is the set of values less than the critical value of the test, or the set of values greater than the critical value of the test. A one-sided test is also referred to as a one-tailed test of significance. Two-sided test A two-sided test is a statistical hypothesis test in which the values for which we can reject the null hypothesis, H0 are located in both tails of the probability distribution. You would basically use this if you wanted to know if your experimental group is greater than or less than another group. In other words, the critical region for a two-sided test is the set of values less than a first critical value of the test and the set of values greater than a second critical value of the test. A two-sided test is also referred to as a two-tailed test of significance. The choice between a one-sided and a two-sided test is determined by the purpose of the investigation or prior reasons for using a one-sided test.
• #7: P values = the probability that the observed result was obtained by chance and it stands for power. The power of a statistical hypothesis test measures the test's ability to reject the null hypothesis when it is actually false - that is, to make a correct decision. α level is set a priori (Usually 0.05). With this P value, we can be 95% certain that our experimental effect is genuine. If p < result is less than the set P value then you reject the null hypothesis and accept the experimental hypothesis If however, p > α level then we reject the experimental hypothesis and accept the null hypothesis
• #8: Type I error = false positive. Where we incorrectly accept the alternative/ experimental hypothesis. α level of 0.05 means that there is 5% risk that a type I error will be encountered Type II error = false negative. Reject the alternative/ experimental hypothesis, when it should be accepted. Scientists care more about accepting a false results than rejecting a true one. So Power = 1 – probability of a type I error. The maximum power a test can have is 1, the minimum is 0. Ideally we want a test to have high power, close to 1. Beware of errors α level of 0.05 means that there is 5% risk that a type I error will be encountered. The other type of error is… - type II error = false negative Where we incorrectly reject the exp hypothesis
• #9: Here is an example of how these errors work. Let’s say you start dating someone. Imagine, I say, a you just started dating someone. You on their first date he or she mentioned that her birthday was coming up this week. But the you can't remember the exact day. It might be today. Or maybe not. Embarrassed to admit it you decides to make a guess. You have two choices. When you see him or her today you can say "Happy Birthday!" Or he can say nothing, hoping that today isn't her birthday. The reality behind the situation is pretty simple: Either today is her birthday or it isn't. Saying "Happy Birthday!" when it is not her birthday is like a Type 1 error. It is a false positive: I'm saying it is your birthday when, in fact, it isn't.Conversely, staying quiet when today is her birthday is like a Type 2 error. It is a false negative: Today is my birthday and you said nothing to me, you missed it.
• #10: Example of a repeated measures T-test. Take same group and test brain volume before birth and after birth.
• #11: It’s important to understand the difference between these two. Population is the entire group with everyone included. An example of this would be the US census. If already know the variance of the general population, you can use the Z-test. Realistically researchers can’t sample everyone, so they have the use the T-test when you do not know the variance of a general population and you only have a sub group. However, I should point out here that the strength of this test is dependant on the number of participants.
• #12: This is the formula for a 1 sample z-test. Basically, it is a formula that is set up to compare unit groups in a standard way through this linear formula. Again, you use this formula when you have the variance of the entire population. Variance describes how far values lie from the mean. So, you‘ll plub in your mean for the group you are looking at, the mean for the population and the standard variance for the population to get a Z score. One way to make distributions directly comparable, is to standardise them by computing a linear transformation The standardised normal distribution does exactly that This can be thought of as expressing your data in the same ‘units’. Therefore, if you remember from the previous slide, the range of 2 standard deviations around the mean covers approximatley 95%; because the standard deviation of a standardised normal distribution is 1, a z-score of +2 or –2, i.e. 2 std, gives the boundary for our confidence interval Only for 2-tailed tests! See distr. around mean versus area from –infinity to z=2.0
• #14: Often you don’t know the s.d. of the hypothesised or comparison population, and so you use a t-test. This uses the sample s.d. or variance instead. This test introduces a source of error, which decreases as your sample size increases. Therefore, the t statistic is distributed differently depending on the size of the sample, like a family of normal curves. The degrees of freedom (df = sample size – 1) represents which of these curves you are relating your t-value to. There are different tables of p-values for different degrees of freedom. larger sample = more ‘squashed’ t-statistic distribution = easier to get significance
• #15: So, you would use a two sampled t-test if you wanted to determine if two samples are different. So, we’ll look at our previous hypothesis. Does having a baby increase your brain size.
• #16: Because one wants to know if the brain size after giving birth is increase or lessoned than the pre-birth group. This will be a two tailed T-test. If you only wanted to find out if brain size is greater after giving birth, this would be a 1 sample T-test.
• #17: This is our sample, taken directly from a paper.
• #19: This is particularly important because when using SPSS you will have to specify which type of statistical test you are using. I’m putting this up to show that one can take a group and try to find out if it is different from a hypothesized population. Here in this formula, you have the mean – hypothesized population/ Standard error. This is when you compare the mean of 1 sample to a given value. You might uses this if you were testing sleep behaviour and you thought the sample that you had was not the norm. You could hypothesize that the population sleeps 10 hours and night to determine if your sample was significantly different to this. A one sample t-test is a hypothesis test for answering questions about the mean where the data are a random sample of independent observations from an underlying normal distribution N(µ, sigma^2), where sigma^2 is unknown.
• #20: So, let’s say you want to compare more than 2 groups. Lets say you want to look at normal pregnancies vs. Preeclamptic groups. Or if you want to compare several different times a brain scan was taken.
• #21: In an experiment with more than 2 samples or more than 2 tasks (or 2 samples and 2 tasks), one could do lots of t-tests and compare all the different groups with each other this way but actually you increase the possibility of accepting the experimental hypothesis when it’s wrong. You’ll remember this as the false negative situation. (this is referred to as familywise/ experimentwise error rate). It is much better to use ANOVA. In its simplest form ANOVA provides a statistical test of whether or not the means of several groups are all equal, and therefore generalizes t-test to more than two groups It looks to see if the variation between groups is different ANOVA is concerned with differences between means of groups, not differences between variances. The name analysis of variance comes from the way the analysis uses variances to decide whether the means are different. The way it works is simple: the statistical proceedure looks to see what the variation (variance) is within the groups, then works out how that variation would translate into variation (i.e. differences) between the groups, taking into account how many subjects there are in the groups. If the observed differences are a lot bigger than what you'd expect by chance, you have statistical significance. (so if the patterns of data spread are similar in your different samples, then the mean won’t be much different, ie the samples are probably from the same population; reversely, if the pattern of variance differs between groups, so will the mean, thus the samples are likely to be drawn from different populations) Terminology: Factors: the overall “things” being compared (type of pregnancy) Levels: the different elements of a factor (young vs old, time to pregnancy) ANOVA tests for one overall effect only so it can tell us if experimental manipulation was generally successful but it doesn’t provide specific information about which specific groups were affected. need for post-hoc testing! ANOVA produces F-statistic or F-ratio which is similar to t-score as it compares the amount of systematic variance in the data to the amount of unsystematic variance. As such, it is the ratio of the experimental effect to the individual differences in performance. If the F=ratio’s value is less than 1, it must represent a non-significant event (so you always want a F-ratio greater than 1, indicating that experimental manipulation had some effect above and beyond the effect of individual differences in performance). To test for significance, compare obtained F-ratio against maximum value one would expect to get by chance alone in an F-distribution with the same degrees of freedom. p-value associated with F is probability that differences between groups could occur by chance if null-hypothesis is correct.
• #22: The type that I have described is referred to as a one-way ANOVA because it has one factor which = cartoon characters (with more than 2 levels) Can also have two-way, three-way ANOVAs These = factorial ANOVAs Allow for possible interactions between factors as well as main effects For example you could have 2 factors with 2 levels each This would = a 2 x 2 factorial design Can also have related or independent designs or a mixture
• #23: There is a significant difference between the groups NOT where this difference lies Finding exactly where the differences lie requires further statistical analyses. So when you are running a a particular statistical test, you’ll specify that you want to have post hoc values listed. But you’ll need to make sure the overall value is significant.
• #25: T-tests assess if two group means differ significantly Can compare two samples or one sample to a given value ANOVAs compare more than two groups or more complicated scenarios