Sajun.org
In
mathematics, the '''tensor product''', denoted by <math>\otimes</math>, may be applied in different contexts to
vectors,
matrices,
tensors and
vector spaces. In each case the significance of the symbol is the same: the most general
bilinear operation.
A representative case is the
Kronecker multiplication of any two rectangular arrays, considered as matrices.
'''Example:'''
<math>\begin{bmatrix}a_1 & a_2 & a_3\end{bmatrix} \otimes \begin{bmatrix}b_1 & b_2 & b_3 & b_4\end{bmatrix} = \begin{bmatrix}a_1b_1 & a_2b_1 & a_3b_1 \\ a_1b_2 & a_2b_2 & a_3b_2 \\ a_1b_3 & a_2b_3 & a_3b_3 \\ a_1b_4 & a_2b_4 & a_3b_4\end{bmatrix}</math>
Resultant rank = 2, resultant dimension = 12.
Here rank denotes the number of requisite indices, while dimension counts the number of degrees of freedom in the resulting array.
== Tensor product of two tensors ==
There is a general formula for the product of two (or more)
tensors
:<math>V \otimes U = V_{\left[ i_1,i_2,i_3,...i_n\right] }U_{\left[ j_1,j_2,j_3,...j_m\right] }</math>.
We are assuming here ''
orthogonal'' tensors, with no distinction of covariant and contravariant indices, for simplicity.
The parameters introduced above work out like this:
:<math>rank( U \otimes V )=rank(U)+rank(V)</math>
:<math>dim( U \otimes V )=dim(U) \cdot dim(V)</math>
See also:
Tensor-classical
==Kronecker product of two matrices==
With matrices this is usually called the Kronecker product, which is used to make clear that the result has a particular block structure imposed upon it, in which each element of the first matrix is replaced by the second matrix, scaled by that element. For matrices <math>U</math> and <math>V</math> this is:
:<math>U \otimes V
= \begin{bmatrix} u_{11}V & u_{12}V & \cdots \\
u_{21}V & u_{22}V \\
\vdots & & \ddots
\end{bmatrix}
= \begin{bmatrix}
u_{11}v_{11} & u_{11}v_{12} & \cdots & u_{12}v_{11} & u_{12}v_{12} & \cdots \\
u_{11}v_{21} & u_{11}v_{22} & & u_{12}v_{21} & u_{12}v_{22} \\
\vdots & & \ddots \\
u_{21}v_{11} & u_{21}v_{12} \\
u_{21}v_{21} & u_{21}v_{22} \\
\vdots
\end{bmatrix}</math>.
==Tensor product of multilinear maps==
Given
multilinear maps <math>f(x_1,...x_k)</math> and <math>g(x_1,... x_m)</math>
their tensor product
is the multilinear function <math> (f \otimes g) (x_1,...x_{k+m})=f(x_1,...x_k)g(x_{k+1},... x_{k+m})</math>
==Tensor product of vector spaces==
The tensor product <math>V \otimes W</math> of two
vector spaces ''V'' and ''W'' has a formal definition by the method of ''generators and relations''. The equivalence class under these relations (given below) of <math>(v,w)</math> is called a ''tensor'' and is denoted by <math>v \otimes w</math>. By construction, one can prove several identities between tensors and form an algebra of tensors.
To construct <math>V \otimes W</math>, take the vector space generated by <math>V \times W</math> and apply (factor out the subspace generated by) the following multilinear relations:
* <math>(v_1+v_2)\otimes w=v_1\otimes w</math><math>+v_2\otimes w</math>
* <math>v\otimes (w_1+w_2)=v\otimes w_1+v\otimes w_2</math>
* <math>cv\otimes w=v\otimes cw=c(v\otimes w)</math>
where <math>v,v_i,w,w_i</math> are vectors from the appropriate spaces, and <math>c</math> is from the underlying field.
We can then derive the identity <math>0v\otimes w=v\otimes 0w=0(v\otimes w)=0</math>, the zero in <math>V \otimes W</math>.
The resulting tensor product <math>V \otimes W</math> is itself a vector space, which can be verified by directly checking the vector space axioms.
Given bases <math>\{v_i\}</math> and <math>\{w_i\}</math> for ''V'' and ''W'' respectively, the tensors of the form <math>v_i \otimes w_j</math>
forms a basis for <math>V \otimes W</math>. The dimension of the tensor product therefore is the product of dimensions of the original spaces; for instance <math>\mathbb{R}^m \otimes \mathbb{R}^n</math> will have dimension <math>mn</math>.
==Tensor product for computer programmers==
If ''a'', ''b'', and ''c'', are rank-one tensors (i.e. one-dimensional arrays), with indices ''i'',''j'',''k'', respectively, then the tensor product of them is a rank-three tensor(i.e. three-dimensional array):
for( int i = 0; i < i_dim; i++)
for( int j = 0; j < j_dim; j++)
for( int k = 0; k < k_dim; k++)
result[i][j][k] = a[i]*b[j]*c[k];
If ''a'' is a rank-two tensor and ''b'' is a rank-one tensor, with indices ''i'' & ''j'', and ''k'', respectively, then the tensor product of them is a rank-three tensor:
for( int i = 0; i < i_dim; i++)
for( int j = 0; j < j_dim; j++)
for( int k = 0; k < k_dim; k++)
result[i][j][k] = a[i][j]*b[k];
==Universal property of tensor product==
The space of all bilinear maps from <math>V\times W</math> to <math>\mathbb R</math> is naturally isomorphic to the space of all linear maps from <math>V \otimes W</math> to <math>\mathbb R</math>. This is built into the construction; <math>V\otimes W</math> has only the relations that are necessary to ensure that a homomorphism from <math>V\otimes W</math> to <math>\mathbb R</math> will be bilinear.
The tensor product in fact satifies the
universal property of being a
fibered coproduct.
==Tensor product of Hilbert spaces==
The tensor product of two
Hilbert spaces is another Hilbert space, which is defined as described below.
===Definition===
Let ''H''
1 and ''H''
2 be two Hilbert spaces with inner products <·,·>
1 and <·,·>
2, respectively. Construct the tensor product of ''H''
1 and ''H''
2 as vector spaces as explained above. We can turn this vector space tensor product into an
inner product space by defining
:<math> \langle\phi_1\otimes\phi_2,\psi_1\otimes\psi_2\rangle = \langle\phi_1,\psi_1\rangle_1 \, \langle\phi_2,\psi_2\rangle_2 \quad \mbox{for all } \phi_1,\psi_1 \in H_1 \mbox{ and } \phi_2,\psi_2 \in H_2 </math>
and extending by linearity. Finally, take the
completion under this inner product. The result is the tensor product of ''H''
1 and ''H''
2 as Hilbert spaces.
===Properties===
If ''H''
1 and ''H''
2 have
orthonormal bases {φ
''k''} and {ψ
''l''}, respectively, then {φ
''k'' ⊗ ψ
''l''} is an orthonormal basis for ''H''
1 ⊗ ''H''
2.
===Examples and applications===
The following examples show how tensor products arise naturally.
Given two
measure spaces ''X'' and ''Y'', with measures μ and ν respectively, one may look at
L2(''X'' × ''Y''), the space of functions on ''X'' × ''Y'' that are square integrable with respect to the product measure μ × ν. If ''f'' is a square integrable function on ''X'', and ''g'' is a square integrable function on ''Y'', then we can define a function ''h'' on ''X'' × ''Y'' by ''h''(''x'',''y'') = ''f''(''x'') ''g''(''y''). The definition of the product measure ensures that all functions of this form are square integrable, so this defines a
bilinear mapping L
2(''X'') × L
2(''Y'') → L
2(''X'' × ''Y'').
Linear combinations of functions of the form ''f''(''x'') ''g''(''y'') are also in L
2(''X'' × ''Y''). It turns out that the set of linear combinations is in fact dense in L
2(''X'' × ''Y''), if L
2(''X'') and L
2(''Y'') are separable. This shows that L
2(''X'') ⊗ L
2(''Y'') is
isomorphic to L
2(''X'' × ''Y''), and it also explains why we need to take the completion in the construction of the Hilbert space tensor product.
Similarly, we can show that L
2(''X''; ''H''), denoting the space of square integrable functions ''X'' → ''H'', is isomorphic to L
2(''X'') ⊗ ''H'' if this space is separable. The isomorphism maps ''f''(''x'') ⊗ φ ∈ L
2(''X'') ⊗ ''H'' to ''f''(''x'')φ ∈ L
2(''X''; ''H''). We can combine this with the previous example and conclude that L
2(''X'') ⊗ L
2(''Y'') and L
2(''X'' × ''Y'') are both isomorphic to L
2(''X''; L
2(''Y'')).
Tensor products of Hilbert spaces arise often in
quantum mechanics. If some particle is described by the Hilbert space ''H''
1, and another particle is described by ''H''
2, then the system consisting of both particles is described by the tensor product of ''H''
1 and ''H''
2. For example, the state space of a
quantum harmonic oscillator is L
2('''R'''), so the state space of two oscillators is L
2('''R''') ⊗ L
2('''R'''), which is isomorphic to L
2('''R'''
2). Therefore, the two-particle system is described by wave functions of the form φ(''x''
1, ''x''
2). A more intricate example is provided by the
Fock spaces, which describe a variable number of particles.
== Intrinsic Description -- (please make more accessible)==
In
abstract algebra, the subject of
linear algebra is upgraded to
multilinear algebra by introducing the '''tensor product''' of two
vector spaces.
It is introduced to reduce the study of bilinear operators to that of
linear operators. This is sufficient to do the same to all multilinear maps.
The general construction of tensor products over an arbitrary
ring comes readily using the language of
category theory. Let ''R'' be a ring (not necessarily
commutative). The
product of two ''R''-
modules, ''M'' and ''N'', ''viewed as sets'', will be denoted ''M'' × ''N''. For any ''R''
op-module, ''M'', any ''R''-module, ''N'', and any
abelian group, ''Z'', we set
:Bilin
''R''(''M'',''N'';''Z'') = the set of φ : ''M'' × ''N'' → ''Z'' such that
# for all ''m'', ''m′'' in ''M'', for all ''n'' in ''N'', φ(''m'' + ''m′'',''n'') = φ(''m'',''n'') + φ(''m′'',''n''),
# for all ''m'' in ''M'', for all ''n'', ''n′'' in ''N'', φ(''m'',''n'' + ''n′'') = φ(''m'',''n'') + φ(''m'',''n′''),
# for all ''m'' in ''M'', for all ''n'' in ''N'', for all ''r'' in ''R'', φ(''m'' ''r'',''n'') = φ(''m'',''r'' ''n'')}
Observe that
* The set Bilin
''R''(''M'',''N'';''Z'') is an abelian group under
addition; i.e., if φ, ψ are in Bilin
''R''(''M'',''N'';''Z''), then φ + ψ is in Bilin
''R''(''M'',''N'';''Z'').
* The map ''Z'' → Bilin
''R''(''M'',''N'';''Z'') is a
functor from the category of abelian groups to the category of sets. This functor is
representable and the representing abelian group is the tensor product. From this comes, for example, the following
universal construction of tensor product.
Consider two vector spaces ''V'' and ''W'' over the same base
field ''F''. Their tensor product satisfies the following
universal property: it is a vector space ''T'' over ''F'', together with a
bilinear operator <math> \otimes : V \times W \rarr T</math>, such that for every bilinear operator
<math>B : V \times W \rarr X</math>, where <math>X</math> is a vector space over the
field ''F'', there exists a unique
linear operator
''L: T → X'' with <math>B = L \circ \otimes</math>, i.e. <math> B(x,y) = L(x \otimes y)</math> for all ''x'' in ''V'' and ''y'' in ''W''.
The tensor product is
up to a unique
isomorphism uniquely specified by this requirement, and we may therefore write <math>V \otimes W</math> instead of ''T''. By direct construction, as suggested in the previous section, one can show that the tensor product for any two vector spaces exists.
The space <math>V \otimes W</math> is generated by the image of <math> \otimes </math>, and even more: if ''S'' is a basis of ''V'' and ''T'' is a basis of ''W'', then <math> \{ s \otimes t: s \in S \mbox{ and } t \in T \} </math> is a basis for <math>V \otimes W</math>. The
dimension of the vector space <math>V \otimes W</math> is therefore given by the product of the dimensions of ''V'' and ''W''.
It is possible
to generalize the definition to a tensor product of any number of spaces. For example, the universal property
of <math>V \otimes W \otimes X</math> is that every tri-linear operator on
<math>V \times W \times X</math> corresponds to a unique linear operator on
<math>V \otimes W \otimes X</math>. The binary tensor product is associative: <math> (V \otimes W) \otimes Z</math> is naturally isomorphic to <math>V \otimes (W \otimes Z)</math>. The tensor product of all three may therefore be identified with either of those: the binary <math> \otimes </math> will suffice. Tensor spaces allow us to use the theory of linear operators to study multi-linear operators, and this says the bilinear case is the main hurdle.
== Relation with the
dual space ==
Note that the space <math> (V \otimes W)^\star</math> (the
dual space of <math>V \otimes W</math> containing all linear functionals on that space)
corresponds naturally to the space of all
bilinear functionals on <math>V \times W</math>. In other words, every bilinear functional is a functional
on the tensor product, and vice versa.
There is a natural
isomorphism between
<math> V^\star \otimes W^\star </math> and <math>(V \otimes W)^\star</math>.
So, the tensors of the linear functionals are bilinear functionals. This
gives us a new way to look at the space of bilinear functionals, as a tensor
product itself.
== Types of tensors, e.g. alternating ==
Linear subspaces of the bilinear
operators (or in general, multilinear operators) determine natural quotient
spaces of the tensor space, which are frequently useful. See
wedge product for the first major example. Another would be the treatment of
algebraic forms as symmetric tensors.
== Over more general rings ==
It is also possible to generalize the definition to tensor products
of
modules over the same
ring. If the
ring is non-commutative, we'll need to be careful about distinguishing right
modules and left modules. We will write ''
RM'' for a left module,
and ''M
R'' for a right module. If a module ''M'' has both a left module
structure over a ring ''R'' and a right module structure over a ring ''S'',
and in addition for every ''m'' in ''M'', ''r'' in ''R'' and ''s'' in ''S'' we have ''r(ms) = (rm)s'', then we
will say ''M'' is a
bimodule, and will denote it by
''
RM
S''. Note that every left-module is a bimodule with '''Z''' acting by ''mn'' = ''m + m + ... + m'' as the right ring, and vice versa.
When defining the tensor product, we need to be careful about the ring: most modules can be considered as modules over several different rings or over the same ring with a different actions of the ring on the module elements.
The most general form of the tensor product definition
is as follows: let ''M
R'' and
''
RN'' be a right and a left module, respectively. Their tensor product over ''R'' is an
abelian group ''P'' together with an
''R''-bilinear operator ''T'': ''M'' × ''N'' → ''P'' such that for every
''R''-bilinear operator ''B'': ''M'' × ''N'' → ''O'' there is a unique
group homomorphism ''L'': ''P'' → ''O'' such that ''L'' o ''T'' = ''B''.
''P'' need not be a module over ''R''. However, if ''
S1M
R'' is an ''S
1-R''-bimodule, then there is a unique left ''S
1''-module structure on ''P'' which is compatible with ''T''. Similarly, if ''
RM
S2'' is an ''R-S
2''-bimodule, then there is a unique right ''S
2''-module structure on ''P'' which is compatible with ''T''.
If ''M'' and ''N'' are both bimodules, then ''P'' is also a bimodule, again in a unique way. (''P'', ''T'') are unique up to a unique isomorphism, and are
called the "tensor product" of ''M'' and ''N''.
If ''R'' is a ring, ''
RM'' is a left ''R''-module, and the
commutator ''rs-sr'' of any two elements ''r'' and ''s'' of ''R'' is in the
annihilator of ''M'', then we can make ''M'' into a right ''R'' module by setting ''mr'' = ''rm''. Note that in this situation the action of ''R'' on M factors through an action of the commutative ring ''R/Z(R)'', ''R'' modulo its
center. In this case the tensor product of ''M'' with itself over ''R'' is again an ''R''-module. If ''M'' and ''N'' are both ''R''-modules satisfying this condition, then their tensor product is again an ''R''-module. This is a very common technique in commutative algebra.
Example:
Consider the
rational numbers '''Q''' and the
integers modulo n '''Z'''
''n''. Both can be considered as modules over the
integers, '''Z'''.
Let ''B'': '''Q''' × '''Z'''
''n'' → ''M'' be a '''Z'''-bilinear operator. Then
''B(q,i) = B(q/n, n*i) = B(q/p, 0) = 0'', so every bilinear operator
is identically zero. Therefore, if we define ''P'' to be the trivial
module, and ''T'' to be the zero bilinear function, then we see that
the properties for the tensor product are satisfied. Therefore, the
tensor product of '''Q''' and '''Z'''
''n'' is {0}.
