Vanshaj Singhania

35:49

maybe zoom broke something when they removed the facebook trackers lol

Subham Dikhit

36:13

Zoom had an update for me like 20 minutes ago, and it doesn't work no more

Stephen Wang

36:20

will today's lecture be on mt2?

Sal Husain

36:29

titan said no yesterday

Oliver Puffer

36:35

Zoom recently automatically signed me out even though I had saved my login and stuff, and it was kinda tricky to get signed in again. Idk if that's the trouble other people are having, but just make sure everyone checks that they are still signed in. Zoom didn't necessarily tell me I wasnt

Vanshaj Singhania

36:49

welp my zoom asked me to update and I said later so I guess that's the move

Ayush Sharma

37:15

So I was just having the Zoom issue where signing in with Berkeley ID/Google wasn't working, but I clicked "Sign in with SSO" and entered in "Berkeley" for the domain name, it took me to the Calnet 2-step where I was able to log in just fine.

Ayush Sharma

37:21

Maybe that'll do something?

Alberto Checcone

43:35

V2?

Sal Husain

43:41

null of a

Yimo Xu

43:41

A^T

Gaurav Rohit Ghosal

43:41

Nullspace

Yimo Xu

43:54

nullspace*

dylanbrater

48:51

||||||||\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\

fy

51:23

my notes from tuesday have the third square matrix as VTranspose

Bijan Fard

51:32

It should be.

Nisha Prabhakar

52:26

How was the U in front of sigma cancelled?

Stephen Wang

52:35

factored out

Nisha Prabhakar

52:37

oh it was pulled out in front?

Vanshaj Singhania

52:37

it was factored out

Nisha Prabhakar

52:38

Ok thanks

Hamza Kamran Khawaja

52:41

rotates?

Jennifer Zhou

53:24

Why doesn’t U affect norm?

Michael Jayasuriya

53:30

yeah why was that I forget now

Jake Whinnery

53:34

Unit vectors

Edward Im

53:44

U is orthonormal @jennifer

Hamza Kamran Khawaja

53:47

Cause its orthonormal vectors

Atharv Vanarase

53:52

Its orthonormal I think. So it magnitude 1 for all vectors

Hamza Kamran Khawaja

53:56

pay attention to the normal part

Terrance Li

56:39

nothing

Alberto Checcone

57:31

probably can make the first term 0

Alberto Checcone

57:45

yeah

Sal Husain

57:47

probably

Terrance Li

57:54

yes

Gilbert

57:56

yes, its diagonal

Nisha Prabhakar

57:57

yes

Terrance Li

58:19

yes because its composed of eigenvectors

Brandon

58:22

yes because of full column rank of A

Sal Husain

58:35

V?

Bryan Ngo

58:37

V

Bijan Fard

58:46

Can the singular values not be 0?

Edward Im

59:04

They’re all greater than 0

Sal Husain

59:06

We defined S to be all sv that aren't 0

Sal Husain

59:10

sigma can

Bijan Fard

59:23

Ok, thanks.

Atharv Vanarase

01:00:41

Why was V transponse V equal to identity again?

Bryan Ngo

01:00:49

orthonormal

Atharv Vanarase

01:01:06

Ok, thanks

Hamza Kamran Khawaja

01:01:24

when you multiply them out, diagonal entries become 1, while the rest are 0

Hamza Kamran Khawaja

01:02:55

Is multiplication by U a type of rotation then?

Alberto Checcone

01:03:03

yup

Jennifer Zhou

01:03:18

is there an intuitive explanation?

Brandon

01:03:26

U can also be reflection I think

Hamza Kamran Khawaja

01:06:04

How do you know the first is projection of y onto columnspace of A?

Ashwin Rammohan

01:07:20

why isn't U1*U1^T*y = y? Aren't all of the vectors in U1 orthonormal?

Hamza Kamran Khawaja

01:08:00

can we have an example of this?

Ashwin Rammohan

01:10:57

Yes, thanks

Heidi Hu

01:11:34

So only if the matrix is square AND has orthonormal basis can UU^T be I?

Xiaoyi Zhu

01:11:45

Do we need a condition that rank(U1) = rank(A)

Stephen Wang

01:11:49

I believe so

Xiaoyi Zhu

01:12:05

hence U1 lies in the col space of A

Ajay Singh

01:12:06

So the point of this is that we have a new formula now for least squares involving V S-1 and U1 transpose?

Stephen Wang

01:12:22

a summary of Professor Arcak's explanation is that since U1 is only a portion of U, it is not square, so it is not Identity

Seongwook Jang

01:12:24

why is (V transpose)^-1 = V?

Sarina Sabouri

01:12:28

Can you explain why U1U1Ty is the projection of y onto the column space of A?

Stephen Wang

01:12:36

VTV=I

Sal Husain

01:12:39

Why would we do this over regular least squares?

Stephen Wang

01:12:44

So V = VT inverse

Ashwin Rammohan

01:14:27

how much y is in the same direction as u1

Bryan Ngo

01:14:29

how parallel y is to u_1

Xiaoyi Zhu

01:15:38

Are we using the fundamental theorem of linear algebra to show U = U1 + U2 ,where U1 lies in range(A), U2 lies in N(A)

Brandon

01:15:53

(V S^-1 U_1^T) is called the pseudoinverse and can be applied to other things as well, not just regular least squares

Sarina Sabouri

01:16:04

Thank you!

Ashwin Rammohan

01:16:12

why is that projection onto column space of a though

Stephen Wang

01:16:56

is (U1TY) the projection of Y onto U?

Vanshaj Singhania

01:17:23

omp incoming

Nisha Prabhakar

01:22:05

is it supposed to be v transpose

Nisha Prabhakar

01:22:08

As the last matrix

Vanshaj Singhania

01:22:36

it is isn't it?

Nisha Prabhakar

01:22:54

I sent it before he changed it lol its fine

Vanshaj Singhania

01:23:11

oh lol I didn't notice he changed it

Ryan Zhao

01:26:36

can you visualize the n-r degrees of freedom for V1^T * x?

Ayush Sharma

01:27:29

Is there a reason for why we're not left-multiplying by V to clean things up and leave the right-hand-side of S^-1 * U^T * y = V1^T * x (currently drawn in a box)?

Aidan Higginbotham

01:27:42

did prof cut out?

Ayush Sharma

01:27:43

Uh, did anyone's audio drop out?

Nisha Prabhakar

01:27:43

am I the only one who can’t hear him anymore?

vincentwaits

01:27:43

Is the lecture frozen?

Ayush Sharma

01:27:44

Yeah...

CharlieWu

01:27:45

uhh i can’t hear anything?

Eugenia Chien

01:27:49

I can't here ;-;

Michael Jayasuriya

01:27:50

Can anyone hear anything?

Micah Feras

01:27:51

cant hear

David McAllister

01:27:52

yeah

Stephen Wang

01:27:52

yeah

Aidan Higginbotham

01:27:52

yes

Edward Im

01:27:52

rip

Alberto Checcone

01:27:53

yes

Seongwook Jang

01:27:53

yeah

Gaurav Rohit Ghosal

01:27:53

yes

nathan

01:27:53

yes

Michael Jayasuriya

01:27:54

yeah

Sal Husain

01:27:54

oski got him

Hamza Kamran Khawaja

01:27:54

yes

Heidi Hu

01:27:55

yea

Micah Feras

01:27:55

yrah

CharlieWu

01:27:57

yes

Vanshaj Singhania

01:27:58

audio gone video frozen

Edward Im

01:28:00

F

Jet Situ

01:28:02

arcak's end is frozen

Nisha Prabhakar

01:28:07

i can here professor sanders

Nisha Prabhakar

01:28:14

*hear omg

Yimo Xu

01:28:19

+1

Bryan Ngo

01:28:20

F

dylanbrater

01:28:21

f

Vanshaj Singhania

01:28:21

screen's gone now too

Franklin Huang

01:28:22

F

Sal Husain

01:28:23

F

Edward Im

01:28:24

F

Yimo Xu

01:28:24

F

Jasper Zhou

01:28:24

f

Anton Zabreyko

01:28:25

F

Vanshaj Singhania

01:28:25

F

Nicholas Berberi

01:28:30

F

Ruslana

01:28:30

F

Michael Jayasuriya

01:28:30

rip

Alberto Checcone

01:28:32

f

fy

01:28:33

F

Heidi Hu

01:28:38

What does F mean

Stephen Wang

01:28:39

can someone explain real quick the U1*U1transpose*Y part? why is it the projection?

Hamza Kamran Khawaja

01:28:39

what is this chat lol

Sal Husain

01:28:46

twitch but it's now zoom

Bryan Ngo

01:28:51

we twitch now

Franklin Huang

01:28:56

*zoom tries to fix problem* *breaks the whole system*

Michael Jayasuriya

01:28:58

pog

Michael Jayasuriya

01:29:00

kappa

Oliver Puffer

01:29:05

f

Hamza Kamran Khawaja

01:29:10

its facebook 2.0

Vanshaj Singhania

01:29:14

rip

Cooper

01:29:24

there goes our lecture

Vanshaj Singhania

01:29:24

hey the facebook chat isn't that bad

Alberto Checcone

01:29:40

there's a fb chat?

Micah Feras

01:29:45

google hangouts

Michael Jayasuriya

01:29:45

wats the fb chat?

Micah Feras

01:29:46

lol

Hamza Kamran Khawaja

01:29:48

its horrendous

Alex Yang

01:29:55

twitch emotes only now

Michael Jayasuriya

01:30:02

failfish

Vanshaj Singhania

01:30:03

i try my best to moderate it LOL

Cooper

01:30:08

Murat died of coronavirus

Eugenia Chien

01:30:08

monkers

CharlieWu

01:30:10

nani

jeff

01:30:15

Prof said aight ima head out

Hamza Kamran Khawaja

01:30:16

^^WTF

Micah Feras

01:30:24

lol

Michael Jayasuriya

01:30:26

Wait is there a fb chat for this class

Vanshaj Singhania

01:30:29

@Cooper let's not

Michael Jayasuriya

01:30:30

can I be invited lol

Vanshaj Singhania

01:30:36

no the FB chat is for (ee)cs

Bryan Ngo

01:30:38

^

Vanshaj Singhania

01:30:39

its full

Michael Jayasuriya

01:30:42

lame

Bryan Ngo

01:30:45

oh yea

Jennifer Zhou

01:30:47

Wait yeah can someone explain the diagram he drew for U1U1Ty again? I thot I understood it but then I confused myself

Vanshaj Singhania

01:31:09

HE'S BACK

Hamza Kamran Khawaja

01:31:20

yes

Sal Husain

01:31:20

we have been SAVED

Oscar Bjorkman

01:31:20

yes

Micah Feras

01:31:21

yeah

dylanbrater

01:31:21

The return

Aidan Higginbotham

01:31:21

yes

Eugenia Chien

01:31:25

we can hear you

Stephen Wang

01:31:32

so Utranspose * U is the normal, but why does multiplying that by Y make it a projection?

Atharv Vanarase

01:31:43

Hes muted right now, I think

Oscar Bjorkman

01:31:57

yay

Vanshaj Singhania

01:31:58

yea

dylanbrater

01:31:58

ya

Aidan Higginbotham

01:31:59

yes

Eugenia Chien

01:32:02

poggies

Will Panitch

01:32:03

hallelujah

Michael Jayasuriya

01:32:44

poggg

Bryan Ngo

01:32:55

0

Sal Husain

01:33:04

zoom

Ethan Wu

01:33:06

0

Vanshaj Singhania

01:33:09

zulu

Hamza Kamran Khawaja

01:33:17

lmao

Alex Yang

01:33:18

monkaS

Hamza Kamran Khawaja

01:34:09

howo do we get the norm squared from the norm?

Alex Yang

01:34:22

square both sides ^

Jake Whinnery

01:34:45

Why was x = VT X

Nisha Prabhakar

01:35:09

What happened to the 0 at the bottom

Jake Whinnery

01:35:10

Oh nvm

Jake Whinnery

01:35:12

magnitude

Nisha Prabhakar

01:35:17

oh ok

Jake Whinnery

01:35:39

Why was V2TX = 0 tho

Stephen Wang

01:35:49

we want to set it as 0

Stephen Wang

01:35:55

we choose the X that would make it 0

Stephen Wang

01:36:07

because if that condition is true, we get the minimum norm

Hamza Kamran Khawaja

01:36:07

to minimize X

Jake Whinnery

01:36:25

Why did we choose to minimus V2TX instead of V1TX

Sarina Sabouri

01:36:48

How did we prove that this term multiplied by V2T is equal to 0?

Natalie Kalyuzhny

01:36:58

I think the minimum norm solution satisfies V2TX = 0 because when you put V2 at the left of the solution, you get V2V1 at the left which is 0?

Natalie Kalyuzhny

01:37:11

Because the vectors in V2 and V1 are orthonormal

Bryan Ngo

01:37:52

wait given the minimum norm solution does that imply V_1 V_1^T = Identity

Bryan Ngo

01:38:10

since we can just left multiply our equivalence by V_1

Ayush Sharma

01:38:22

Yeah, I think that's a fact!

Ayush Sharma

01:38:37

Since V^T * V = V * V^T = 1.

Ayush Sharma

01:38:45

*Identity

Seongwook Jang

01:39:19

how did we go from V1Tx to S-1UTy

Ryan Zhao

01:40:07

might be too basic of a question, but why can you still do row by column matrix multiplication for [V1 V2] * [S^-1 U^T y, 0]^T when V1, V2, etc. are matrices?

Ayush Sharma

01:40:31

@Seongwook, that's the equation that we got from some manipulations earlier of y = Ax.

Seongwook Jang

01:40:46

oh i see thanks

Ayush Sharma

01:42:02

Yup! :)

Jamie

01:42:35

For anyone still wondering why U1 U1^T y is the projection, you can multiply it out: U1^T y gives us [u1^T y, u2^T y, …, ur^T y], which = the lengths of y along each u vector in U1. To get the direction, we then left multiply by U1 => [ u1(u1^T y) … ur(ur^T y) ] which is really the projection of y along u1, along u2, … , along ur, where u1 … ur form an orthonorm. basis for col(A). Thus, U1 U1^T y is the projection of y on col(A).

Hamza Kamran Khawaja

01:42:54

what are the dimensions of V2?

Bijan Fard

01:43:42

n x (n - r)

hello

01:45:04

* left multiplying by U1 gives you u1(u1^T y) + u2(u2^T y) + … + ur(ur^T y)

Stephen Wang

01:50:47

wait, PCA and SCD try to uncover the correlation?

Alberto Checcone

01:51:05

line

Hamza Kamran Khawaja

01:52:37

like a type of standard deviation?

Vanshaj Singhania

01:52:53

yeah

Subham Dikhit

01:53:00

Why do we subtract the avg again?

Aidan Higginbotham

01:53:15

the top one

Hamza Kamran Khawaja

01:53:16

to have it centered around 0

Vanshaj Singhania

01:53:29

to center the data around 0 -- it makes it simpler to distinguish relative trends i think

Subham Dikhit

01:53:40

Thanks!

Hamza Kamran Khawaja

01:54:17

what is the subspace he is talking about again?

Zhiping Gu

01:54:25

2*2

Vanshaj Singhania

01:54:27

mxm?

Vanshaj Singhania

01:54:36

oh wait

Vanshaj Singhania

01:54:40

2x2 and mxm

Hamza Kamran Khawaja

01:56:39

A^TA is square, but why do we say it is symmetric? Is it a property of our method (subtract the mean)?

Edward Im

01:57:01

because if you take the transpose it’s the same matrix

Jake Whinnery

01:57:17

Look up spectral theorem

Hamza Kamran Khawaja

01:57:19

how do we know that?

Edward Im

01:57:36

(A^TA)^T = A^TA

Hamza Kamran Khawaja

01:57:58

oh thanks

Stephen Wang

01:58:45

so by subtracting average, would you create the graph that is like a line or the one that is super centered around the origin where sigma 1 approximately = sigma2

Atharv Vanarase

01:59:05

This is the A that has already been subtracting the average of the columns right? In the covariance matrix?

Vanshaj Singhania

01:59:23

@stephen, the latter is correct i think

Vanshaj Singhania

01:59:49

the average height is different from the average weight, so the resulting shape won't necessarily be linear

Jake Whinnery

02:00:08

Wait from discussion yesterday did anyone else have m it just as (1/M)ATA instead of (1/m-1)?

Bijan Fard

02:00:26

@Stephen, subtracting the mean in each dimension will just translate the data so that it's centered around the origin, it won't change the shape.

Sal Husain

02:00:31

The average distance from the mean?

Calvin Yan

02:00:52

Yeah @Jake I noticed that

Alex Yang

02:01:33

O.o

Ajay Singh

02:01:36

Is today’s lecture in scope

dylanbrater

02:01:40

no

dylanbrater

02:01:44

Not according to piazza

Hamza Kamran Khawaja

02:01:47

that's what Netflix is for. Covariance matrixes

Atharv Vanarase

02:01:50

What was the last one that was in scope

Jennifer Zhou

02:02:02

whats the difference between htw and wth?

dylanbrater

02:02:06

Tuesday I think

Ashwin Rammohan

02:02:18

Thanks

Atharv Vanarase

02:02:20

Ok thanks

Ayush Sharma

02:02:22

Thank you!

Bijan Fard

02:02:29

@Jennifer, they're equal

Jennifer Zhou

02:02:59

@bijan thanks!!

Bijan Fard

02:03:06

No problem

Jiachen Yuan

02:05:01

Professor could you show us the proof of why columns of V2 form an basis for the null space of A again. I was a little bit confused there.

Hamza Kamran Khawaja

02:06:19

did he cut out?

Bijan Fard

02:06:30

I can still hear him.