Statistical tests

SOSstat

Introduction

[Statistical tests] (sosstat_fct_statintro.md) play a key role in the practical implementation of a decision-making process. There are a great many of them. This page presents the main tests available in SOSstat.

Comparison tests

When we talk about comparison tests, we often refer to simple hypothesis tests that aim to accept or reject a hypothesis. Among these tests, we can distinguish two main families [1]:

  • The parametric tests, which as their name suggests, are built on the calculation of samples parameters. These tests assume that the samples come from a normal population.
  • The non-parametric tests, which unlike their predecessors make no assumptions about the distribution of the population.

(1): On this page, we also describe equivalence tests, which follow a completely different logic.

To choose between these two familly of tests, it is advisable to make a Normality test.

Normality tests

There are many tests of normality and everyone can have preferences. SOSstat offers the three most commonly used tests in Quality applications:

  • The Kolmogorov-Smirnov test, developed from the sample empirical distribution function
  • The Anderson-Darling test, which is based on fairly similar principles , but that is more sensitive with small samples
  • The Shapiro-Wilk test, that can process large samples, thanks to relatively recent algorithms

SOSstat provides a clear interpretation of the test decision, in addition to the traditional p-value (not always obvious to interpret).

Kolmogorov-Smirnov normality test with SOSstat

To complete the use of these tests, SOSstat represents the data on graphs such as the histogram or the Normal Quantile plot.

Parametric tests

Parametric tests are, without a doubt, the most used tests in industrial activities. They are both powerful and simple to interpret. Indeed, their denomination comes from the fact that they use estimations of parameters to compare certain aspects of the studied populations: one generally uses the average to treat problems of centering and the variance or standard-deviation to compare variability or dispersion.

SOSstat offers comparison tests of means or standard deviations in three practical situations:

  • Cases where a sample is compared to a theoretical value
  • The case where comparing two samples
  • Cases where more than two samples are compared (k samples)
Number of samples Average Standard deviation
1 Student test [2] Khi-square test
2 Welch test [3] Fisher test
k Analyse of variance Bartlett test
* (2): Known or estimated standard deviation
* (3): Known or estimated standard deviation, paired or matched samples

These tests are widely used for industrial data analysis for problem solving :

  • Performance comparison of two processes or two measurement instruments
  • Product batches comparison
  • Comparing before and after maintaining a machine ...

1 sample Student test with SOSstat

Fisher test with SOSstat

Anova with SOSstat

The non-parametric tests

Unlike previous tests, non-parametric tests make no assumption about the variable distribution. It's by using the ranks, not the measures, that these tests do not make any assumption on data distribution. It is also said that these tests are distribution free .

Non-parametric tests, therefore have a wide potential for applications: They can be applied to continuous variables (measurements for example) when the characteristics do not follow normal distributions, but also on discrete variables (counting) or rank variables (especially in the case of sensory analysis).

As with the parametric tests, the tests can be classified according to whether they focus on the centering or dispersion of variables. On the other hand, we find the three sampling situations described above 3 usual families of tests: tests with one variable, 2 variables and more than 2 variables (k).

Number of samples Centering Dispersion
1 Wilcoxon sign test
2 Mann Whitney U test Ansari-Bradley test
k Kruskal Wallis test and Friedman test Levene test

Mann-Whitney U test with SOSstat

Test Wizard

To guide the user in choosing the most suitable test, SOSstat offers a test assistant. Using a series of questions organized in a decision tree, the user is directed to the test, that best fits his needs. It is no longer necessary to resume his "old courses of statistics".

Comparison test selection wizard - SOSstat

The test wizard is responsible for opening the dialog box, with the appropriate settings, in order to carry out the test directly.

Equivalence tests

Comparison tests, such as the Student test for example, are designed to detect discrepancies. In this type of test, we assume ($H_0$) that there is no difference. If the analysis reveals a significant difference between the samples, the null hypothesis ($H_0$) is rejected: we therefore show a difference. On the other hand, if the test does not show a significant difference, we can not conclude anything (and especially not an equivalence!)

In equivalence tests, the assumption is made so that the test proves equivalence. In contrast, the alternative hypothesis ($H_1$) is that the difference between samples is below an acceptable gap $\Theta$. Thus, if one rejects the null hypothesis, we can state that the samples are equivalent. It proves that the gap was less than the acceptable gap $\Theta$.

In the practice of hypothesis testing, there is an implicit logical asymmetry between the roles of the null hypothesis ($H_0$) and the alternative hypothesis ($H_1$). In equivalence tests, it is proposed to test the hypothesis of non-equivalence ($H_0$) with a controlled risk to conclude wrongly equivalence.

  • $H_0$ : There is no equivalence
  • $H_1$ : Equivalence is demonstrated

SOSstat offers two types of equivalence tests :

  • The mean equivalences, which aim to demonstrate that a difference between the means is less than an acceptable difference $\Theta$
  • The population equivalent , which compares both sample mean and variance. This technique is recommended by the FDA for bioequivalence.

Population equivalence

Equivalence tests have a wide potential for application in industry. They are used in the pharmaceutical industry to demonstrate the equivalence between a reference medical device and a generic device. They can be used to validate the equivalence between two measuring equipment (to compare new and historical equipment), they can also be used to qualify industrial processes in case of production transfer or duplication of equipment.

Bibliography

DROESBEKE, J. - Éléments de Statistique , Éditions Ellipses, 2015, ISBN-13: 978-2340009080 GoogleBooks

SAPORTA, G. - Probabilités, analyse des données et statistique , Technip, 2011- 622 pages, ISBN-13: 978-2710809807 GoogleBooks

Chow SC, Liu JP. - Design and Analysis of Bioavailability and Bioequivalence Studies. 2nd edn. Marcel Dekker: New York, 1999. GoogleBook

Gopal K. Kanji - 100 Statistical tests , SAGE Publications Ltd, 2006 , ISBN-13 : 978 14129 2375 0 Amazon