Normality test

How it works

Each method approaches normality detection differently:

Choosing a method:

• Use the Anderson-Darling test when your data may have heavy or skewed tails and you need a sensitive statistical test.

• Use the Kolmogorov-Smirnov test for a standard non-parametric comparison against a normal reference distribution.

• Use the Q-Q plot test alongside either statistical test to visually confirm the result.

• Enable all three to cross-validate conclusions. If results disagree, the Q-Q plot often clarifies which test to trust.

Configure the component

PAI -name normality_test
    -project algo_public
    -DinputTableName=test
    -DoutputTableName=test_out
    -DselectedColNames=col1,col2
    -Dlifecycle=1;

Example

This example creates a sample input table with 10 rows and runs the Anderson-Darling and Kolmogorov-Smirnov tests on column x.

Step 1: Create the input table

Add a SQL Script component. Deselect Use Script Mode and Whether the system adds a create table statement, then enter the following SQL:

drop table if exists normality_test_input;
create table normality_test_input as
select * from
(
  select 1 as x union all
  select 2 as x union all
  select 3 as x union all
  select 4 as x union all
  select 5 as x union all
  select 6 as x union all
  select 7 as x union all
  select 8 as x union all
  select 9 as x union all
  select 10 as x
) tmp;

Step 2: Run the normality test

Add another SQL Script component. Deselect Use Script Mode and Whether the system adds a create table statement, then enter the following PAI command. Connect the output of Step 1 to this component.

drop table if exists ${o1};
PAI -name normality_test
    -project algo_public
    -DinputTableName=normality_test_input
    -DoutputTableName=${o1}
    -DselectedColNames=x
    -Dlifecycle=1;

Step 3: Run the pipeline

Click the icon in the upper left corner to run the pipeline.

Step 4: View the results

Right-click the SQL Script component from Step 2 and choose View Data > SQL Script Output.

The output table has four columns: colname, testname, testvalue, and pvalue:

In this example, both tests return high p-values (Anderson-Darling: 0.957, Kolmogorov-Smirnov: 0.9999888...), indicating no significant departure from normality for column x.