A marketing study conducts 60 significance tests about means and proportions for several groups. Of them, 3 tests are statistically significant at the 0.05 level. The study's final report stresses only the tests with significant results, not mentioning the other 57 tests. What is misleading about this?

One would expect about 5% of tests to be significant just by chance if the null hypothesis is true, and for 60 tests, this is 0.05(60) = 3 tests. This could explain why these tests are statistically significant.

Step-by-step explanation:

What is misleading about this is the study's final report because this marketing study was carried out with a level of significance of 5% (0.05) which means that if we carry the same study with varying sample data we will mostly like arrive at a conclusion against our null conclusion 5% of the time and this is not good for the the study because it is a type 1 error and has to be eliminated and it cannot be eliminated totally .

hence One would expect about 5% of tests to be significant just by chance if the null hypothesis is true, and for 60 tests, this is 0.05(60) = 3 tests. This could explain why these tests are statistically significant.

jmonterrozar jmonterrozar · Answer 2 · 2020-06-20T04:09:19+02:00

We are told that the marketing study conducts 60 tests and that 3 tests are statistically significant at the 0.05 level, in addition to this the study's final report emphasizes only the tests with significant results without mentioning the other 57 tests.

We can interpret it as follows:

By the 0.05 level, we mean that 5% of the tests are significant by chance when the null hypothesis is true.

Here 5% of the 60.

60 * 0.05 = 3

Therefore it only mentions 3 tests and does not mention the other 57 tests, which in itself is misleading because all 60 tests should be fully mentioned in the final report.