Sajun.org
In
probability theory, to say that two
events are '''independent''' intuitively means that knowing whether or not one of them occurs makes it neither more probable nor less probable that the other occurs. For example, the event of getting a "1" when a die is thrown and the event of getting a "1" the second time it is thrown are independent.
Similarly, when we assert that two
random variables are independent, we intuitively mean that knowing something about the value of one of them does not yield any information about the value of the other. For example, the number appearing on the upward face of a die the first time it is thrown and that appearing the second time are independent.
== Independent events ==
The standard definition says:
:Two events ''A'' and ''B'' are '''independent'''
iff P(''A'' ∩ ''B'')=P(''A'')P(''B'').
Here ''A'' ∩ ''B'' is the
intersection of ''A'' and ''B'', that is, it is the event that both events ''A'' and ''B'' occur.
More generally, and collection of events -- possibly more than just two of them -- are '''mutually independent''' iff for any finite subset ''A''
1, ..., ''A''
''n'' of the collection we have
:<math>P(A_1 \cap \cdots \cap A_n)=P(A_1)\,\cdots\,P(A_n).</math>
This is called the ''multiplication rule'' for independent events.
If two events ''A'' and ''B'' are independent, then the
conditional probability of ''A'' given ''B'' is the same as the "unconditional" (or "marginal") probability of ''A'', that is,
:<math>P(A\mid B)=P(A).</math>
There are at least two reasons why this statement is not taken to be the definition of independence: (1) the two events ''A'' and ''B'' do not play symmetrical roles in this statement, and (2) problems arise with this statement when events of probability 0 are involved.
When one recalls that the conditional probability P(''A'' | ''B'') is given by
:<math>P(A\mid B)={P(A \cap B) \over P(B)},</math>
one sees that the statement above is equivalent to
:<math>P(A \cap B)=P(A)P(B)</math>
which is the standard definition given above.
== Independent random variables ==
Two random variables ''X'' and ''Y'' are independent iff for any numbers ''a'' and ''b'' the events [''X'' ≤ ''a''] and [''Y'' ≤ ''b''] are independent events as defined above. Similarly an arbitrary collection of random variables -- possible more than just two of them -- is independent precisely if for any finite collection ''X''
1, ..., ''X''
''n'' and any finite set of numbers ''a''
1, ..., ''a''
''n'', the events [''X''
1 ≤ ''a''
1], ..., [''X''
''n'' ≤ ''a''
''n''] are independent events as defined above.
The measure-theoretically inclined may prefer to substitute events [''X'' ∈ ''A''] for events [''X'' ≤ ''a''] in the above definition, where ''A'' is any
Borel set. That definition is exactly equivalant to the one above when the values of the random variables are
real numbers. It has the advantage of working also for complex-valued random variables or for random variables taking values in any
topological space.
If any two of a collection of random variables are independent, they may nonetheless fail to be mutually independent; this is called
pairwise independence.
If ''X'' and ''Y'' are independent, then the
expectation operator ''E'' has the nice property
:E[''X''· ''Y''] = E[''X''] · E[''Y'']
and for the
variance we have
:var(''X'' + ''Y'') = var(''X'') + var(''Y'').
If ''X'' and ''Y'' are independent, the
covariance cov(''X'',''Y'') is zero; otherwise we would have
:var(''X'' + ''Y'') = var(''X'') + var(''Y'') + 2 cov(''X'', ''Y'').
(The converse of this, the proposition that if two random variables have a covariance of 0 they are independent, is not true. See
uncorrelated.)
Furthermore, if ''X'' and ''Y'' are independent and have
probability densities ''f''
''X''(''x'') and ''f''
''Y''(''y''), then the combined random variable (''X'',''Y'') has a joint density
:''f''
''XY''(''x'',''y'') d''x'' d''y'' = ''f''
''X''(''x'') ''f''
''Y''(''y'') d''x'' d''y''.
== Conditionally independent random variables ==
Intuitively, two random variables ''X'' and ''Y'' are conditionally independent given ''Z'' if, once ''Z'' is known, the value of ''Y'' does not add any additional information about ''X''. For instance, two measurements ''X'' and ''Y'' of the same underlying quantity ''Z'' are not independent, but they are conditionally independent given ''Z'' (unless the errors in the two measurements are somehow connected).
The formal definition of conditional independence is based on the idea of
conditional distributions. If ''X'', ''Y'', and ''Z'' are
discrete random variables, then we define ''X'' and ''Y'' to be ''conditionally independent given'' ''Z'' if
: P(''X'' = ''x'', ''Y'' = ''y'' | ''Z'' = ''z'') = P(''X'' = ''x'' | ''Z'' = ''z'') · P(''Y'' = ''y'' | ''Z'' = ''z'')
for all ''x'', ''y'' and ''z'' such that P(''Z'' = ''z'') > 0. On the other hand, if the random variables are
continuous and have a joint
probability density function ''p'', then ''X'' and ''Y'' are ''conditionally independent given'' ''Z'' if
: ''p''
''XY''|''Z''(''x'', ''y'' | ''z'') = ''p''
''X''|''Z''(''x'' | ''z'') · ''p''
''Y''|''Z''(''y'' | ''z'')
for all real numbers ''x'', ''y'' and ''z'' such that ''p''
''Z''(''z'') > 0.
If ''X'' and ''Y'' are conditionally independent given ''Z'', then
: P(''X'' = ''x'' | ''Y'' = ''y'', ''Z'' = ''z'') = P(''X'' = ''x'' | ''Z'' = ''z'')
for any ''x'', ''y'' and ''z'' with P(''Z'' = ''z'') > 0. That is, the conditional distribution for ''X'' given ''Y'' and ''Z'' is the same as that given ''Z'' alone. A similar equation holds for the conditional probability density functions in the continuous case.
Independence can be seen as a special kind of conditional independence, since probability can be seen as a kind of conditional probability given no events.
it:Indipendenza stocastica
