EECS 16B Lectures - Shared screen with speaker view
was it A^T A
don't both forms work?
Yea but the u vectors are eigenvectors for a different matrix than the v vectors
an m x n matrix, with the first rxr as S, and 0's everywhere else
S but extended to the right
S in the left top
zero everywhere else
with zero filling in
R x r
should say rx(n-r) in the upper right
yeah isn't the top right r(n-r)
we haven't proven that U^TU and V^TV equal I right?
i think he sort of proved it above
i thought UTU was 0?
He gave the basic idea of it as just doing inner products
We did earlier since orthonormal so just dot of unit vectors has magnitude 1
Its cuz vectors in U and V are orthonormal
but we haven't proven that yet right
we're just accepting it as true for now?
yeah have we proven why all the evectors are orthonormal?
We proved it already
Not super formal but he showed us it
Proven that they’re orthonormal or that if they’re orthonormal then VTV is I
So would the remaining vectors in U1 or ViT be the null space of AAT or ATA?
why are they all orthonormal tho
Does the order of evectors in U2 or V2 matters?
they're eigenvectors (so they are orthogonal to each other) and we have created them sth they are orthonormal
eigenvectors aren't orthogonal by definition
they are just linearly independent
oh yeah huh
yeah counter example [1,2] and [2, 1] are e.vectors but not orth
I think that’s something we proved from construction last lecture
I think it’s bc ATA and AAT are symmetric matrices actually
But no I don’t think that’s been proven yet in lecture
can we ask if the top right corner should be r x (n-r)
how can a wide matrix have linearly independent columns?
how can all the columns be linearly independent?
r of them
As many as possible. So all the rows are
For the number of rows *
svd is a religion
definitely not a cult**
why is this preferred over the other method
Where we us S instead of sigma
cuz everything is square
(except for sigma itself lol)
so in which case is S the upper left quadrant
the general case
if m = r or n = r, then m-r = 0 or n-r = 0 respectively, giving us our two special cases he just drew out
I think when it does not have independent columns or rows
but if we already looked at m < n and m > n, then is the general case where m = n?
and square root?
technically yeah, but that doesn't help us in this case
eigenvalue of 1
plus that only works for vectors I think, not matrix-vector products?
A rotation matrix
scales and changes the size of the vector?
sigma matrix would change the length of VTx right?
the rotation matrices don't (u, v) but sigma does
Why doesn’t V transpose change the length?
Does sigma change length and direction tho
Like basically acting as a mapping
Cause its orthonormal @steven
Because its an orthonormal matrix
length becomes corresponding sigma
scaled by rho?
or sigma, rather*
So is VT and U just change of bases?
Sorry I missed it what are the v1 and v2 vectors
eigenvectors of A^TA
the vectors in V
all matrices are change of basis technically
What is x here?
some arbitrary input
will the angle stay the same?
for VT*x -> sigma VT*x
Vt rotates the vector space so that the columns of v are on the axes
and U scales back
where would Vtx land
When multiplied by sigm
@ryan don’t think so
how does he know that those two are AV1 and AV2?
its between sigma1 and sigma2
how do we know all the vectors are length 1 initially? Do we always set it so?
because they are orthonormal
Isn’t sigma1 just stretching one component? How does this bound apply to the entire vector?
Is x also orthonormal to v1 and v2?
x is an arbitrary length 1 vector
b/c of the way we ordered the sigmas, Vincent
x is a linear combination of v1, v2, ...
sigma 1 was the greatest value
Oh ok thanks
are relative angles between vectors preserved when multiplying by orthonormal vectors like U and V^T?
(1) so to clarify, the goal of this section is to get from x to Ax using SVD? (2) how do we know the length of Ax? And how do we know it is less than sigma1 * the length of x? (3) you showed a rotation matrix earlier. Is this the usual behavior when SVD is applied, or is this just a particular example? Thanks
This just provides a geometric interpretation of what Ax does
^for us Visual-types
Oh ok—but what is Ax?
I don’t think we know the length of Ax, but we can find an upper bound given ||x|| = 1
||V^T x|| = 1
oh ok thanks
and then we know that sigma scales each component of V^T x by the small sigma
I don't think we preserve angle because angle b/w x and V1 changes after multiplying by Vt
so could A be any linear transformation and then U, sigma, and VT perform that same linear transformation through rotation/reflection and scaling?
@Sakshi (1) more or less, yes; (2) we know the length of A*x = U*Sigma*VT*x by analyzing each individual multiplication (note VT*anyvector and U*anyvector preserves magnitude since VT and U are orthonormal, so the only place where the length is scaled is in the multiplication by sigma) Since sigma1 is the largest Singular value, the largest u can be scaled by is if your entire length is along that axis and scaled by sigma1 (3) we observed that the orthonormal matrices didnt change magnitude, and we know that rotational matrices have this property so we basically are saying that orthonormal matrices rotate vectors
why is knowing the upper bound so useful
Oh ok that makes sense, thanks so much
what is this used for?
why do we not need to take conjugate of Q again? Is it by constuction?
Q is all real
it's real so the conjugate is Q
Sorry when/why did we say Q is real? Is that in the definition of symmetric matrix?
the claim is for symmetric real matrices
Oh wait yeah I see that highlighted now thanks!!
Wait how are we getting this step where we go from lambda conjugate * xTx means that lambda conjugate is lambda
so this is to show that ATA and AAT have real eigenvectors and eigenvalues so we can do SVD, right?
technically this should’ve been proven first
why the specification of "can be chosen to be orthonormal" rather than "are orthonormal"
Well because whether or not vectors are orthonormal depends on magnitude
And evector magnitude is arbitrary
ye by linearity the scalar multiplication of an eigenvector by any scalar maintains that it's still an eigevector
Sorry I missed it, why is X2TQX1 a scalar
can look at the dimensions
Also, if there are repeated eigenvalues, then you might have to choose the direction too. For example, with the identity matrix, every nonzero vector is an eigenvector, but you can still choose orthogonal ones.
wait, why can you do that again?
Regarding the question above
x2Tx1 = 0?
eigenvalues are 1 or 0
lambda1 = lambda 2
wait @Bijan with the identity matrix you still always have orthogonality with the eigebasis
no matter what magnitudes you choose? or am i missing something
why is x2Tx1 = x1Tx2
is same regardless of order
Well, you could choose an eigenbasis where they aren't orthogonal as well.
oh right yeah
Thank you professors
Thank you! :)