Logo

EECS 16B Lectures - Shared screen with speaker view
Vanshaj Singhania
35:49
maybe zoom broke something when they removed the facebook trackers lol
Subham Dikhit
36:13
Zoom had an update for me like 20 minutes ago, and it doesn't work no more
Stephen Wang
36:20
will today's lecture be on mt2?
Sal Husain
36:29
titan said no yesterday
Oliver Puffer
36:35
Zoom recently automatically signed me out even though I had saved my login and stuff, and it was kinda tricky to get signed in again. Idk if that's the trouble other people are having, but just make sure everyone checks that they are still signed in. Zoom didn't necessarily tell me I wasnt
Vanshaj Singhania
36:49
welp my zoom asked me to update and I said later so I guess that's the move
Ayush Sharma
37:15
So I was just having the Zoom issue where signing in with Berkeley ID/Google wasn't working, but I clicked "Sign in with SSO" and entered in "Berkeley" for the domain name, it took me to the Calnet 2-step where I was able to log in just fine.
Ayush Sharma
37:21
Maybe that'll do something?
Alberto Checcone
43:35
V2?
Sal Husain
43:41
null of a
Yimo Xu
43:41
A^T
Gaurav Rohit Ghosal
43:41
Nullspace
Yimo Xu
43:54
nullspace*
dylanbrater
48:51
||||||||\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\
fy
51:23
my notes from tuesday have the third square matrix as VTranspose
Bijan Fard
51:32
It should be.
Nisha Prabhakar
52:26
How was the U in front of sigma cancelled?
Stephen Wang
52:35
factored out
Nisha Prabhakar
52:37
oh it was pulled out in front?
Vanshaj Singhania
52:37
it was factored out
Nisha Prabhakar
52:38
Ok thanks
Hamza Kamran Khawaja
52:41
rotates?
Jennifer Zhou
53:24
Why doesn’t U affect norm?
Michael Jayasuriya
53:30
yeah why was that I forget now
Jake Whinnery
53:34
Unit vectors
Edward Im
53:44
U is orthonormal @jennifer
Hamza Kamran Khawaja
53:47
Cause its orthonormal vectors
Atharv Vanarase
53:52
Its orthonormal I think. So it magnitude 1 for all vectors
Hamza Kamran Khawaja
53:56
pay attention to the normal part
Terrance Li
56:39
nothing
Alberto Checcone
57:31
probably can make the first term 0
Alberto Checcone
57:45
yeah
Sal Husain
57:47
probably
Terrance Li
57:54
yes
Gilbert
57:56
yes, its diagonal
Nisha Prabhakar
57:57
yes
Terrance Li
58:19
yes because its composed of eigenvectors
Brandon
58:22
yes because of full column rank of A
Sal Husain
58:35
V?
Bryan Ngo
58:37
V
Bijan Fard
58:46
Can the singular values not be 0?
Edward Im
59:04
They’re all greater than 0
Sal Husain
59:06
We defined S to be all sv that aren't 0
Sal Husain
59:10
sigma can
Bijan Fard
59:23
Ok, thanks.
Atharv Vanarase
01:00:41
Why was V transponse V equal to identity again?
Bryan Ngo
01:00:49
orthonormal
Atharv Vanarase
01:01:06
Ok, thanks
Hamza Kamran Khawaja
01:01:24
when you multiply them out, diagonal entries become 1, while the rest are 0
Hamza Kamran Khawaja
01:02:55
Is multiplication by U a type of rotation then?
Alberto Checcone
01:03:03
yup
Jennifer Zhou
01:03:18
is there an intuitive explanation?
Brandon
01:03:26
U can also be reflection I think
Hamza Kamran Khawaja
01:06:04
How do you know the first is projection of y onto columnspace of A?
Ashwin Rammohan
01:07:20
why isn't U1*U1^T*y = y? Aren't all of the vectors in U1 orthonormal?
Hamza Kamran Khawaja
01:08:00
can we have an example of this?
Ashwin Rammohan
01:10:57
Yes, thanks
Heidi Hu
01:11:34
So only if the matrix is square AND has orthonormal basis can UU^T be I?
Xiaoyi Zhu
01:11:45
Do we need a condition that rank(U1) = rank(A)
Stephen Wang
01:11:49
I believe so
Xiaoyi Zhu
01:12:05
hence U1 lies in the col space of A
Ajay Singh
01:12:06
So the point of this is that we have a new formula now for least squares involving V S-1 and U1 transpose?
Stephen Wang
01:12:22
a summary of Professor Arcak's explanation is that since U1 is only a portion of U, it is not square, so it is not Identity
Seongwook Jang
01:12:24
why is (V transpose)^-1 = V?
Sarina Sabouri
01:12:28
Can you explain why U1U1Ty is the projection of y onto the column space of A?
Stephen Wang
01:12:36
VTV=I
Sal Husain
01:12:39
Why would we do this over regular least squares?
Stephen Wang
01:12:44
So V = VT inverse
Ashwin Rammohan
01:14:27
how much y is in the same direction as u1
Bryan Ngo
01:14:29
how parallel y is to u_1
Xiaoyi Zhu
01:15:38
Are we using the fundamental theorem of linear algebra to show U = U1 + U2 ,where U1 lies in range(A), U2 lies in N(A)
Brandon
01:15:53
(V S^-1 U_1^T) is called the pseudoinverse and can be applied to other things as well, not just regular least squares
Sarina Sabouri
01:16:04
Thank you!
Ashwin Rammohan
01:16:12
why is that projection onto column space of a though
Stephen Wang
01:16:56
is (U1TY) the projection of Y onto U?
Vanshaj Singhania
01:17:23
omp incoming
Nisha Prabhakar
01:22:05
is it supposed to be v transpose
Nisha Prabhakar
01:22:08
As the last matrix
Vanshaj Singhania
01:22:36
it is isn't it?
Nisha Prabhakar
01:22:54
I sent it before he changed it lol its fine
Vanshaj Singhania
01:23:11
oh lol I didn't notice he changed it
Ryan Zhao
01:26:36
can you visualize the n-r degrees of freedom for V1^T * x?
Ayush Sharma
01:27:29
Is there a reason for why we're not left-multiplying by V to clean things up and leave the right-hand-side of S^-1 * U^T * y = V1^T * x (currently drawn in a box)?
Aidan Higginbotham
01:27:42
did prof cut out?
Ayush Sharma
01:27:43
Uh, did anyone's audio drop out?
Nisha Prabhakar
01:27:43
am I the only one who can’t hear him anymore?
vincentwaits
01:27:43
Is the lecture frozen?
Ayush Sharma
01:27:44
Yeah...
CharlieWu
01:27:45
uhh i can’t hear anything?
Eugenia Chien
01:27:49
I can't here ;-;
Michael Jayasuriya
01:27:50
Can anyone hear anything?
Micah Feras
01:27:51
cant hear
David McAllister
01:27:52
yeah
Stephen Wang
01:27:52
yeah
Aidan Higginbotham
01:27:52
yes
Edward Im
01:27:52
rip
Alberto Checcone
01:27:53
yes
Seongwook Jang
01:27:53
yeah
Gaurav Rohit Ghosal
01:27:53
yes
nathan
01:27:53
yes
Michael Jayasuriya
01:27:54
yeah
Sal Husain
01:27:54
oski got him
Hamza Kamran Khawaja
01:27:54
yes
Heidi Hu
01:27:55
yea
Micah Feras
01:27:55
yrah
CharlieWu
01:27:57
yes
Vanshaj Singhania
01:27:58
audio gone video frozen
Edward Im
01:28:00
F
Jet Situ
01:28:02
arcak's end is frozen
Nisha Prabhakar
01:28:07
i can here professor sanders
Nisha Prabhakar
01:28:14
*hear omg
Yimo Xu
01:28:19
+1
Bryan Ngo
01:28:20
F
dylanbrater
01:28:21
f
Vanshaj Singhania
01:28:21
screen's gone now too
Franklin Huang
01:28:22
F
Sal Husain
01:28:23
F
Edward Im
01:28:24
F
Yimo Xu
01:28:24
F
Jasper Zhou
01:28:24
f
Anton Zabreyko
01:28:25
F
Vanshaj Singhania
01:28:25
F
Nicholas Berberi
01:28:30
F
Ruslana
01:28:30
F
Michael Jayasuriya
01:28:30
rip
Alberto Checcone
01:28:32
f
fy
01:28:33
F
Heidi Hu
01:28:38
What does F mean
Stephen Wang
01:28:39
can someone explain real quick the U1*U1transpose*Y part? why is it the projection?
Hamza Kamran Khawaja
01:28:39
what is this chat lol
Sal Husain
01:28:46
twitch but it's now zoom
Bryan Ngo
01:28:51
we twitch now
Franklin Huang
01:28:56
*zoom tries to fix problem* *breaks the whole system*
Michael Jayasuriya
01:28:58
pog
Michael Jayasuriya
01:29:00
kappa
Oliver Puffer
01:29:05
f
Hamza Kamran Khawaja
01:29:10
its facebook 2.0
Vanshaj Singhania
01:29:14
rip
Cooper
01:29:24
there goes our lecture
Vanshaj Singhania
01:29:24
hey the facebook chat isn't that bad
Alberto Checcone
01:29:40
there's a fb chat?
Micah Feras
01:29:45
google hangouts
Michael Jayasuriya
01:29:45
wats the fb chat?
Micah Feras
01:29:46
lol
Hamza Kamran Khawaja
01:29:48
its horrendous
Alex Yang
01:29:55
twitch emotes only now
Michael Jayasuriya
01:30:02
failfish
Vanshaj Singhania
01:30:03
i try my best to moderate it LOL
Cooper
01:30:08
Murat died of coronavirus
Eugenia Chien
01:30:08
monkers
CharlieWu
01:30:10
nani
jeff
01:30:15
Prof said aight ima head out
Hamza Kamran Khawaja
01:30:16
^^WTF
Micah Feras
01:30:24
lol
Michael Jayasuriya
01:30:26
Wait is there a fb chat for this class
Vanshaj Singhania
01:30:29
@Cooper let's not
Michael Jayasuriya
01:30:30
can I be invited lol
Vanshaj Singhania
01:30:36
no the FB chat is for (ee)cs
Bryan Ngo
01:30:38
^
Vanshaj Singhania
01:30:39
its full
Michael Jayasuriya
01:30:42
lame
Bryan Ngo
01:30:45
oh yea
Jennifer Zhou
01:30:47
Wait yeah can someone explain the diagram he drew for U1U1Ty again? I thot I understood it but then I confused myself
Vanshaj Singhania
01:31:09
HE'S BACK
Hamza Kamran Khawaja
01:31:20
yes
Sal Husain
01:31:20
we have been SAVED
Oscar Bjorkman
01:31:20
yes
Micah Feras
01:31:21
yeah
dylanbrater
01:31:21
The return
Aidan Higginbotham
01:31:21
yes
Eugenia Chien
01:31:25
we can hear you
Stephen Wang
01:31:32
so Utranspose * U is the normal, but why does multiplying that by Y make it a projection?
Atharv Vanarase
01:31:43
Hes muted right now, I think
Oscar Bjorkman
01:31:57
yay
Vanshaj Singhania
01:31:58
yea
dylanbrater
01:31:58
ya
Aidan Higginbotham
01:31:59
yes
Eugenia Chien
01:32:02
poggies
Will Panitch
01:32:03
hallelujah
Michael Jayasuriya
01:32:44
poggg
Bryan Ngo
01:32:55
0
Sal Husain
01:33:04
zoom
Ethan Wu
01:33:06
0
Vanshaj Singhania
01:33:09
zulu
Hamza Kamran Khawaja
01:33:17
lmao
Alex Yang
01:33:18
monkaS
Hamza Kamran Khawaja
01:34:09
howo do we get the norm squared from the norm?
Alex Yang
01:34:22
square both sides ^
Jake Whinnery
01:34:45
Why was x = VT X
Nisha Prabhakar
01:35:09
What happened to the 0 at the bottom
Jake Whinnery
01:35:10
Oh nvm
Jake Whinnery
01:35:12
magnitude
Nisha Prabhakar
01:35:17
oh ok
Jake Whinnery
01:35:39
Why was V2TX = 0 tho
Stephen Wang
01:35:49
we want to set it as 0
Stephen Wang
01:35:55
we choose the X that would make it 0
Stephen Wang
01:36:07
because if that condition is true, we get the minimum norm
Hamza Kamran Khawaja
01:36:07
to minimize X
Jake Whinnery
01:36:25
Why did we choose to minimus V2TX instead of V1TX
Sarina Sabouri
01:36:48
How did we prove that this term multiplied by V2T is equal to 0?
Natalie Kalyuzhny
01:36:58
I think the minimum norm solution satisfies V2TX = 0 because when you put V2 at the left of the solution, you get V2V1 at the left which is 0?
Natalie Kalyuzhny
01:37:11
Because the vectors in V2 and V1 are orthonormal
Bryan Ngo
01:37:52
wait given the minimum norm solution does that imply V_1 V_1^T = Identity
Bryan Ngo
01:38:10
since we can just left multiply our equivalence by V_1
Ayush Sharma
01:38:22
Yeah, I think that's a fact!
Ayush Sharma
01:38:37
Since V^T * V = V * V^T = 1.
Ayush Sharma
01:38:45
*Identity
Seongwook Jang
01:39:19
how did we go from V1Tx to S-1UTy
Ryan Zhao
01:40:07
might be too basic of a question, but why can you still do row by column matrix multiplication for [V1 V2] * [S^-1 U^T y, 0]^T when V1, V2, etc. are matrices?
Ayush Sharma
01:40:31
@Seongwook, that's the equation that we got from some manipulations earlier of y = Ax.
Seongwook Jang
01:40:46
oh i see thanks
Ayush Sharma
01:42:02
Yup! :)
Jamie
01:42:35
For anyone still wondering why U1 U1^T y is the projection, you can multiply it out: U1^T y gives us [u1^T y, u2^T y, …, ur^T y], which = the lengths of y along each u vector in U1. To get the direction, we then left multiply by U1 => [ u1(u1^T y) … ur(ur^T y) ] which is really the projection of y along u1, along u2, … , along ur, where u1 … ur form an orthonorm. basis for col(A). Thus, U1 U1^T y is the projection of y on col(A).
Hamza Kamran Khawaja
01:42:54
what are the dimensions of V2?
Bijan Fard
01:43:42
n x (n - r)
hello
01:45:04
* left multiplying by U1 gives you u1(u1^T y) + u2(u2^T y) + … + ur(ur^T y)
Stephen Wang
01:50:47
wait, PCA and SCD try to uncover the correlation?
Alberto Checcone
01:51:05
line
Hamza Kamran Khawaja
01:52:37
like a type of standard deviation?
Vanshaj Singhania
01:52:53
yeah
Subham Dikhit
01:53:00
Why do we subtract the avg again?
Aidan Higginbotham
01:53:15
the top one
Hamza Kamran Khawaja
01:53:16
to have it centered around 0
Vanshaj Singhania
01:53:29
to center the data around 0 -- it makes it simpler to distinguish relative trends i think
Subham Dikhit
01:53:40
Thanks!
Hamza Kamran Khawaja
01:54:17
what is the subspace he is talking about again?
Zhiping Gu
01:54:25
2*2
Vanshaj Singhania
01:54:27
mxm?
Vanshaj Singhania
01:54:36
oh wait
Vanshaj Singhania
01:54:40
2x2 and mxm
Hamza Kamran Khawaja
01:56:39
A^TA is square, but why do we say it is symmetric? Is it a property of our method (subtract the mean)?
Edward Im
01:57:01
because if you take the transpose it’s the same matrix
Jake Whinnery
01:57:17
Look up spectral theorem
Hamza Kamran Khawaja
01:57:19
how do we know that?
Edward Im
01:57:36
(A^TA)^T = A^TA
Hamza Kamran Khawaja
01:57:58
oh thanks
Stephen Wang
01:58:45
so by subtracting average, would you create the graph that is like a line or the one that is super centered around the origin where sigma 1 approximately = sigma2
Atharv Vanarase
01:59:05
This is the A that has already been subtracting the average of the columns right? In the covariance matrix?
Vanshaj Singhania
01:59:23
@stephen, the latter is correct i think
Vanshaj Singhania
01:59:49
the average height is different from the average weight, so the resulting shape won't necessarily be linear
Jake Whinnery
02:00:08
Wait from discussion yesterday did anyone else have m it just as (1/M)ATA instead of (1/m-1)?
Bijan Fard
02:00:26
@Stephen, subtracting the mean in each dimension will just translate the data so that it's centered around the origin, it won't change the shape.
Sal Husain
02:00:31
The average distance from the mean?
Calvin Yan
02:00:52
Yeah @Jake I noticed that
Alex Yang
02:01:33
O.o
Ajay Singh
02:01:36
Is today’s lecture in scope
dylanbrater
02:01:40
no
dylanbrater
02:01:44
Not according to piazza
Hamza Kamran Khawaja
02:01:47
that's what Netflix is for. Covariance matrixes
Atharv Vanarase
02:01:50
What was the last one that was in scope
Jennifer Zhou
02:02:02
whats the difference between htw and wth?
dylanbrater
02:02:06
Tuesday I think
Ashwin Rammohan
02:02:18
Thanks
Atharv Vanarase
02:02:20
Ok thanks
Ayush Sharma
02:02:22
Thank you!
Bijan Fard
02:02:29
@Jennifer, they're equal
Jennifer Zhou
02:02:59
@bijan thanks!!
Bijan Fard
02:03:06
No problem
Jiachen Yuan
02:05:01
Professor could you show us the proof of why columns of V2 form an basis for the null space of A again. I was a little bit confused there.
Hamza Kamran Khawaja
02:06:19
did he cut out?
Bijan Fard
02:06:30
I can still hear him.