- Source: Sensitivity index
- Letusan Samalas 1257
- Bluetooth
- Perubahan iklim dan gender
- Metil benzoat
- Penglihatan binokular
- Nilai pajanan
- Ekstraversi dan introversi
- Hati
- Helium
- Wolfram
- Sensitivity index
- Sensitivity and specificity
- Prothrombin time
- Variance-based sensitivity analysis
- Sensitivity analysis
- Anxiety sensitivity
- Oil spill
- Disposition index
- Thyrotroph Thyroid Hormone Sensitivity Index
- Detection theory
The sensitivity index or discriminability index or detectability index is a dimensionless statistic used in signal detection theory. A higher index indicates that the signal can be more readily detected.
Definition
The discriminability index is the separation between the means of two distributions (typically the signal and the noise distributions), in units of the standard deviation.
= Equal variances/covariances
=For two univariate distributions
a
{\displaystyle a}
and
b
{\displaystyle b}
with the same standard deviation, it is denoted by
d
′
{\displaystyle d'}
('dee-prime'):
d
′
=
|
μ
a
−
μ
b
|
σ
{\displaystyle d'={\frac {\left\vert \mu _{a}-\mu _{b}\right\vert }{\sigma }}}
.
In higher dimensions, i.e. with two multivariate distributions with the same variance-covariance matrix
Σ
{\displaystyle \mathbf {\Sigma } }
, (whose symmetric square-root, the standard deviation matrix, is
S
{\displaystyle \mathbf {S} }
), this generalizes to the Mahalanobis distance between the two distributions:
d
′
=
(
μ
a
−
μ
b
)
′
Σ
−
1
(
μ
a
−
μ
b
)
=
‖
S
−
1
(
μ
a
−
μ
b
)
‖
=
‖
μ
a
−
μ
b
‖
/
σ
μ
{\displaystyle d'={\sqrt {({\boldsymbol {\mu }}_{a}-{\boldsymbol {\mu }}_{b})'\mathbf {\Sigma } ^{-1}({\boldsymbol {\mu }}_{a}-{\boldsymbol {\mu }}_{b})}}=\lVert \mathbf {S} ^{-1}({\boldsymbol {\mu }}_{a}-{\boldsymbol {\mu }}_{b})\rVert =\lVert {\boldsymbol {\mu }}_{a}-{\boldsymbol {\mu }}_{b}\rVert /\sigma _{\boldsymbol {\mu }}}
,
where
σ
μ
=
1
/
‖
S
−
1
μ
‖
{\displaystyle \sigma _{\boldsymbol {\mu }}=1/\lVert \mathbf {S} ^{-1}{\boldsymbol {\mu }}\rVert }
is the 1d slice of the sd along the unit vector
μ
{\displaystyle {\boldsymbol {\mu }}}
through the means, i.e. the
d
′
{\displaystyle d'}
equals the
d
′
{\displaystyle d'}
along the 1d slice through the means.
For two bivariate distributions with equal variance-covariance, this is given by:
d
′
2
=
1
1
−
ρ
2
(
d
′
x
2
+
d
′
y
2
−
2
ρ
d
′
x
d
′
y
)
{\displaystyle {d'}^{2}={\frac {1}{1-\rho ^{2}}}\left({d'}_{x}^{2}+{d'}_{y}^{2}-2\rho {d'}_{x}{d'}_{y}\right)}
,
where
ρ
{\displaystyle \rho }
is the correlation coefficient, and here
d
x
′
=
μ
b
x
−
μ
a
x
σ
x
{\displaystyle d'_{x}={\frac {{\mu _{b}}_{x}-{\mu _{a}}_{x}}{\sigma _{x}}}}
and
d
y
′
=
μ
b
y
−
μ
a
y
σ
y
{\displaystyle d'_{y}={\frac {{\mu _{b}}_{y}-{\mu _{a}}_{y}}{\sigma _{y}}}}
, i.e. including the signs of the mean differences instead of the absolute.
d
′
{\displaystyle d'}
is also estimated as
Z
(
hit rate
)
−
Z
(
false alarm rate
)
{\displaystyle Z({\text{hit rate}})-Z({\text{false alarm rate}})}
.: 8
= Unequal variances/covariances
=When the two distributions have different standard deviations (or in general dimensions, different covariance matrices), there exist several contending indices, all of which reduce to
d
′
{\displaystyle d'}
for equal variance/covariance.
Bayes discriminability index
This is the maximum (Bayes-optimal) discriminability index for two distributions, based on the amount of their overlap, i.e. the optimal (Bayes) error of classification
e
b
{\displaystyle e_{b}}
by an ideal observer, or its complement, the optimal accuracy
a
b
{\displaystyle a_{b}}
:
d
b
′
=
−
2
Z
(
Bayes error rate
e
b
)
=
2
Z
(
best accuracy rate
a
b
)
{\displaystyle d'_{b}=-2Z\left({\text{Bayes error rate }}e_{b}\right)=2Z\left({\text{best accuracy rate }}a_{b}\right)}
,
where
Z
{\displaystyle Z}
is the inverse cumulative distribution function of the standard normal. The Bayes discriminability between univariate or multivariate normal distributions can be numerically computed (Matlab code), and may also be used as an approximation when the distributions are close to normal.
d
b
′
{\displaystyle d'_{b}}
is a positive-definite statistical distance measure that is free of assumptions about the distributions, like the Kullback-Leibler divergence
D
KL
{\displaystyle D_{\text{KL}}}
.
D
KL
(
a
,
b
)
{\displaystyle D_{\text{KL}}(a,b)}
is asymmetric, whereas
d
b
′
(
a
,
b
)
{\displaystyle d'_{b}(a,b)}
is symmetric for the two distributions. However,
d
b
′
{\displaystyle d'_{b}}
does not satisfy the triangle inequality, so it is not a full metric.
In particular, for a yes/no task between two univariate normal distributions with means
μ
a
,
μ
b
{\displaystyle \mu _{a},\mu _{b}}
and variances
v
a
>
v
b
{\displaystyle v_{a}>v_{b}}
, the Bayes-optimal classification accuracies are:
p
(
A
|
a
)
=
p
(
χ
′
1
,
v
a
λ
2
>
v
b
c
)
,
p
(
B
|
b
)
=
p
(
χ
′
1
,
v
b
λ
2
<
v
a
c
)
{\displaystyle p(A|a)=p({\chi '}_{1,v_{a}\lambda }^{2}>v_{b}c),\;\;p(B|b)=p({\chi '}_{1,v_{b}\lambda }^{2}
,
where
χ
′
2
{\displaystyle \chi '^{2}}
denotes the non-central chi-squared distribution,
λ
=
(
μ
a
−
μ
b
v
a
−
v
b
)
2
{\displaystyle \lambda =\left({\frac {\mu _{a}-\mu _{b}}{v_{a}-v_{b}}}\right)^{2}}
, and
c
=
λ
+
ln
v
a
−
ln
v
b
v
a
−
v
b
{\displaystyle c=\lambda +{\frac {\ln v_{a}-\ln v_{b}}{v_{a}-v_{b}}}}
. The Bayes discriminability
d
b
′
=
2
Z
(
p
(
A
|
a
)
+
p
(
B
|
b
)
2
)
.
{\displaystyle d'_{b}=2Z\left({\frac {p\left(A|a\right)+p\left(B|b\right)}{2}}\right).}
d
b
′
{\displaystyle d'_{b}}
can also be computed from the ROC curve of a yes/no task between two univariate normal distributions with a single shifting criterion. It can also be computed from the ROC curve of any two distributions (in any number of variables) with a shifting likelihood-ratio, by locating the point on the ROC curve that is farthest from the diagonal.
For a two-interval task between these distributions, the optimal accuracy is
a
b
=
p
(
χ
~
w
,
k
,
λ
,
0
,
0
2
>
0
)
{\displaystyle a_{b}=p\left({\tilde {\chi }}_{{\boldsymbol {w}},{\boldsymbol {k}},{\boldsymbol {\lambda }},0,0}^{2}>0\right)}
(
χ
~
2
{\displaystyle {\tilde {\chi }}^{2}}
denotes the generalized chi-squared distribution), where
w
=
[
σ
s
2
−
σ
n
2
]
,
k
=
[
1
1
]
,
λ
=
μ
s
−
μ
n
σ
s
2
−
σ
n
2
[
σ
s
2
σ
n
2
]
{\displaystyle {\boldsymbol {w}}={\begin{bmatrix}\sigma _{s}^{2}&-\sigma _{n}^{2}\end{bmatrix}},\;{\boldsymbol {k}}={\begin{bmatrix}1&1\end{bmatrix}},\;{\boldsymbol {\lambda }}={\frac {\mu _{s}-\mu _{n}}{\sigma _{s}^{2}-\sigma _{n}^{2}}}{\begin{bmatrix}\sigma _{s}^{2}&\sigma _{n}^{2}\end{bmatrix}}}
. The Bayes discriminability
d
b
′
=
2
Z
(
a
b
)
{\displaystyle d'_{b}=2Z\left(a_{b}\right)}
.
RMS sd discriminability index
A common approximate (i.e. sub-optimal) discriminability index that has a closed-form is to take the average of the variances, i.e. the rms of the two standard deviations:
d
a
′
=
|
μ
a
−
μ
b
|
/
σ
rms
{\displaystyle d'_{a}=\left\vert \mu _{a}-\mu _{b}\right\vert /\sigma _{\text{rms}}}
(also denoted by
d
a
{\displaystyle d_{a}}
). It is
2
{\displaystyle {\sqrt {2}}}
times the
z
{\displaystyle z}
-score of the area under the receiver operating characteristic curve (AUC) of a single-criterion observer. This index is extended to general dimensions as the Mahalanobis distance using the pooled covariance, i.e. with
S
rms
=
[
(
Σ
a
+
Σ
b
)
/
2
]
1
2
{\displaystyle \mathbf {S} _{\text{rms}}=\left[\left(\mathbf {\Sigma } _{a}+\mathbf {\Sigma } _{b}\right)/2\right]^{\frac {1}{2}}}
as the common sd matrix.
Average sd discriminability index
Another index is
d
e
′
=
|
μ
a
−
μ
b
|
/
σ
avg
{\displaystyle d'_{e}=\left\vert \mu _{a}-\mu _{b}\right\vert /\sigma _{\text{avg}}}
, extended to general dimensions using
S
avg
=
(
S
a
+
S
b
)
/
2
{\displaystyle \mathbf {S} _{\text{avg}}=\left(\mathbf {S} _{a}+\mathbf {S} _{b}\right)/2}
as the common sd matrix.
Comparison of the indices
It has been shown that for two univariate normal distributions,
d
a
′
≤
d
e
′
≤
d
b
′
{\displaystyle d'_{a}\leq d'_{e}\leq d'_{b}}
, and for multivariate normal distributions,
d
a
′
≤
d
e
′
{\displaystyle d'_{a}\leq d'_{e}}
still.
Thus,
d
a
′
{\displaystyle d'_{a}}
and
d
e
′
{\displaystyle d'_{e}}
underestimate the maximum discriminability
d
b
′
{\displaystyle d'_{b}}
of univariate normal distributions.
d
a
′
{\displaystyle d'_{a}}
can underestimate
d
b
′
{\displaystyle d'_{b}}
by a maximum of approximately 30%. At the limit of high discriminability for univariate normal distributions,
d
e
′
{\displaystyle d'_{e}}
converges to
d
b
′
{\displaystyle d'_{b}}
. These results often hold true in higher dimensions, but not always. Simpson and Fitter promoted
d
a
′
{\displaystyle d'_{a}}
as the best index, particularly for two-interval tasks, but Das and Geisler have shown that
d
b
′
{\displaystyle d'_{b}}
is the optimal discriminability in all cases, and
d
e
′
{\displaystyle d'_{e}}
is often a better closed-form approximation than
d
a
′
{\displaystyle d'_{a}}
, even for two-interval tasks.
The approximate index
d
g
m
′
{\displaystyle d'_{gm}}
, which uses the geometric mean of the sd's, is less than
d
b
′
{\displaystyle d'_{b}}
at small discriminability, but greater at large discriminability.
Contribution to discriminability by each dimension
In general, the contribution to the total discriminability by each dimension or feature may be measured using the amount by which the discriminability drops when that dimension is removed. If the total Bayes discriminability is
d
′
{\displaystyle d'}
and the Bayes discriminability with dimension
i
{\displaystyle i}
removed is
d
−
i
′
{\displaystyle d'_{-i}}
, we can define the contribution of dimension
i
{\displaystyle i}
as
d
′
2
−
d
−
i
′
2
{\displaystyle {\sqrt {d'^{2}-{d'_{-i}}^{2}}}}
. This is the same as the individual discriminability of dimension
i
{\displaystyle i}
when the covariance matrices are equal and diagonal, but in the other cases, this measure more accurately reflects the contribution of a dimension than its individual discriminability.
Scaling the discriminability of two distributions
We may sometimes want to scale the discriminability of two data distributions by moving them closer or farther apart. One such case is when we are modeling a detection or classification task, and the model performance exceeds that of the subject or observed data. In that case, we can move the model variable distributions closer together so that it matches the observed performance, while also predicting which specific data points should start overlapping and be misclassified.
There are several ways of doing this. One is to compute the mean vector and covariance matrix of the two distributions, then effect a linear transformation to interpolate the mean and sd matrix (square root of the covariance matrix) of one of the distributions towards the other.
Another way that is by computing the decision variables of the data points (log likelihood ratio that a point belongs to one distribution vs another) under a multinormal model, then moving these decision variables closer together or farther apart.
See also
Receiver operating characteristic (ROC)
Summary statistics
Effect size
References
Wickens, Thomas D. (2001). Elementary Signal Detection Theory. OUP USA. ch. 2, p. 20. ISBN 0-19-509250-3.
External links
Interactive signal detection theory tutorial including calculation of d′.