Math is useful: Eigenvectors

I remember sitting in the theater for Avengers: Endgame, my ears perking up when I heard Tony Stark mention eigenvalues while trying to use quantum physics to achieve time travel. It was around the same time I was taking linear algebra, where I was first introduced to them. Eigenvectors and eigenvalues have followed me ever since because they're important properties of data. As with other concepts in linear algebra, they tend to be hard to grasp because there are not many visual representations. Here, I want to explain the importance of Eigenvectors/values.

Let's start at the very beginning. A collection of information is referred to as a dataset. Let's say you want to make a simple budget for yourself, so you begin collecting data based on bills from the previous two months that looks like this:

	House	Food
Month 1	$2300	$450
Month 2	$2000	$600

We can see that the rows represent each month and the columns represent the category of expense. When we do math, we can generalize this because it is the same regardless of what the rows and columns are named. Instead, we create a matrix where rows are samples and columns are features:

2300	450
2000	600

This is a matrix with a size 2x2 (2 rows by 2 columns). Now let's take a very simple matrix to work with:

And we often name our matrices 'A' for simplicity. Now remember, each column is a 'feature' or 'category'. Another name for this is dimension. Yes, that kind of dimension. More formally, the number of dimensions represents the number of vectors that are linearly independent and span the entire space. This means there are no redundant pieces of information in the set. Let's take our earlier example of the budget matrix. If I were to add a column that represented the amount of money spent on groceries, this would not be a new dimension because it is already represented in the 'food' vector (it is not linearly independent).

Since we have 2 dimensions, we can plot this on a 2D graph like this:

Now, you understand vectors and matrices!

Next, we have to bring in Eigenvectors. The German prefix "eigen" means "own" or "characteristic". Sometimes, we want to transform matrices to bring out certain characteristics. For example, if we have a matrix where each row represents a person and each row represents a facial feature ("eyes", "nose", "mouth", "ears"), the features will be slightly different from each other. There is a matrix that tells us how every pixel changes in relation to every other pixel to make these characteristics more prominent (called a covariance matrix). Like, the leftmost side of your nose will contain darker pixels than the center because the nose casts a shadow on the face.

This matrix will transform each image in a different way because each face is unique! People have different colored eyes, spacing between their nose and mouth, different-sized noses, etc. Now, here's something interesting: when we transform the data, some vectors do not change, only shrink or stretch. An example here might be the skull being roughly the same shape, or the eyes always located above the nose, which is always above the mouth. This is extremely valuable when we need a big picture from millions of data points. Think of it like this: if I drew two ovals above an upturned curve like this:

You would recognize this as a smiley face! Why? Because people have two eyes and a mouth in a circular-shaped skull. This is basically an eigen-face! And the eigenvalue tells us how much each eigenvector shrinks or stretches when it's transformed. For example, these are all smiley faces, but they all look slightly different:

That's why eigenvectors are so interesting to us! They take millions of data points and tell us the most important features! This is exactly what is used in facial recognition data. Wonder why your phone still unlocks while you have sunglasses on? Eigenvectors, baby!