What do you not fully understand?

[Khoa Tran]

I personally have a lot. There are many things that I understand only the technical definition (or statement) but not its intuition, application, or simply its power. I'm sure I'm not the only one.

A common example is eigenvalues/eigenvectors which are "more important than they look" and not well presented in many Linear Algebra courses/textbooks. I still remember that I started learning eigenvalues by its definition, by solving , and by the theorem , etc. It is very useless learned that way, I think, and I feel that I had not fully understood why we care so much about it until a few months ago.

Another example is the Hahn-Banach theorem, which my teacher repeatedly said "it is more important than it looks" but I don't quite catch its spirit yet. Though the proof of almost every theorem in Functional Analysis ultimately relies on the Hahn-Banach theorem, I still don't feel like I master this theorem.

In Quant Finance, Risk-neutral Pricing is also a major concept that I understand very superficially. I've always taken the PDE-approach (via replications) and avoided the Risk-neutral Measure thinking.


Anyway, those are some of my own problems. Do you have any insights to better describe those? I think it would be particularly useful also if we collectively share what we know/are told more important than they look.

[pde]

Why don't you start first, Khoa? What is the usage of eigenvalues?

[YADD]

Eigenvalues are good for, well... energy levels of a hydrogen atom. Just kidding.

I have a conceptual question here: would hard sciences (and the world at large) be the same if someday complex numbers disappeared from mankind's knowledge? Having acquired complex analysis in undergrad, how fully do we understand its usage/meaning/importance?

Related to Khoa's post of algorithms, if complex numbers vanished from the Earth's surface, MRI scanners would perhaps still be functioning, so would their CPUs be running with the top 10 or 20 revolutionary algorithms that help calculate nuclear excitation levels in magnetic field (whatever). The world seems oblivious.

Or does it?




.

[Khoa Tran]

Originally Posted by pde

Why don't you start first, Khoa? What is the usage of eigenvalues?

I knew that I would be asked to start first, as usual. What a dilemma. Due to my time constraint, I cannot describe much but I've briefly mentioned the most essential aspect of eigenvalues here.

các eigenvectors tạo thành một orthogonal basis (not always) của 1 vector space, nghĩa là bất kì vector nào cũng biểu diễn được dưới dạng linear combination of eigenvectors. Basis này “quan trọng” vì: * Làm việc trực tiếp với 1 hệ tuyến tính (linears system) thông thường rất messy. Dùng biểu diễn linear combination of eigenvectors thì các tính toán trên ma trận trở thành scalar computations => rất dễ dàng
* Hệ quả nói chung là nhiều: từ convergence analysis trong iterative methods cho đến việc estimate that usually come up in solving systems of ODEs hoặc estimate that come up a lot in applications when we apply a linear transformation repeatedly, etc.

Tổng quát lên linear operator: spectral representation of a linear operator, nếu có là một điều tuyệt vời, nếu tìm được thì tuyệt vời hơn nữa. Cái này chắc để dành cho dân Quantum Mechanics phát biểu.

Originally Posted by YADD

I have a conceptual question here: would hard sciences (and the world at large) be the same if someday complex numbers disappeared from mankind's knowledge? Having acquired complex analysis in undergrad, how fully do we understand its usage/meaning/importance?

This is an excellent question. I've used complex numbers pretty often (as they come up in Fourier transform, wave scattering, or dynamical systems) but do I really understand complex numbers? I wish.

[neomikeo]

Originally Posted by Khoa Tran

A common example is eigenvalues/eigenvectors which are "more important than they look" and not well presented in many Linear Algebra courses/textbooks. I still remember that I started learning eigenvalues by its definition, by solving , and by the theorem , etc. It is very useless learned that way, I think, and I feel that I had not fully understood why we care so much about it until a few months ago.

In Quant Finance, Risk-neutral Pricing is also a major concept that I understand very superficially. I've always taken the PDE-approach (via replications) and avoided the Risk-neutral Measure thinking.

I like the use of eigen value/vectors in Principle Component Analysis (PCA) to analyze time series of a quite homogeneous system like term structure of interest rates. It is basically about decomposing a space into different orthogonal vectors. So instead of working with a lot of time series data, you only work with two or three factors that explains a lot of variations => easier to model and implement. An example of PCA that I like is Carol Alexder's paper on implied volatility (an improvement of Derman's implied tree models).

IMHO, the phrase risk-neutral pricing or state prices (in continuous time) or stochastic discount factor (in discrete time) are quite interchangeable. They also come from marginal utility of consumption of a representative agent, i.e. the agent would be indifferent between consuming now or investing. Then, as every one knows about the mechanics of risk-neutral pricing to price future payoff ....

However, I am not sure whether the existence of state-prices/stochastic discount factor leads to No Arbitrage or No-Arbitrage could exist without state prices. Any ideas?

[hanguyen]

Eigenvalues and eigenvectors are obsivously among the most important engineering tools. This is because 1) many engineering applications tend to base on linearity and 2) eigenvalues and eigenvectors are inherent properties of linear operators.

Results concerning eigenvalues and eigenvectors, and beyond such as the singular value decompositions-or SVD, can be best interpreted using the language of operator theory. This view frees us from any particular basis, hence we can work directly in the basis formed by eigenvectors. The operator is nothing but a diagonal matrix with eigenvalues in the diagonal. Many other properties will be intuitive.

[Hung.Q.Ngo]

I've posted two common usages/intuition on eigenvalues/eigenvectors here:

http://www.procul.org/blog/2007/10/23/eigen/

[ngtridung]

I think Hung. Q. Ngo’s article provides a good overview of the motivations and usage of eigenvalues and eigenvectors. I just want to add a few more words on their application to quantitative finance (somewhat elaborate the PCA application that Neomikeo and H.Q. Ngo have mentioned).

In modern portfolio theory, covariance matrix is the common parameter used for modeling risk. Specifically, in mean-variance portfolio optimization, risk is usually modeled as x’*Cov*x where x is the portfolio weights (the percentages of wealth allocated to assets) and Cov is the covariance matrix. Since the Covariance matrix is unknown, it needs to be estimated from historical data. However, the estimation of the entire Covariance matrix is very hard as it involves the estimation of n^2/2 parameters for a matrix of size n. (in finance, n is typically around 20 but it could be up to a few hundreds of assets). Thus, some dimension reduction technique need to be used.

The PCA approach: The eigenvectors that corresponds to a few largest eigenvalues are used as the main factors (besides the common market factors) to explains the risks in each asset. From this factor model, we just need to work with a much smaller dimension.

Random matrix approach: In stead of trying to estimate the covariance matrix, we could assume the truth eigenvectors are closed to the empirical eigenvectors and try to estimate the eigenvalues. Random matrix theory provides us the distribution of the eigenvalues when the covariance matrix is pure noise. Filtering (usually a low pass filter) out this distribution from the empirical distribution of the eigenvalues would provides us “cleaned” eigenvalues.

I am not sure if it has been mentioned in H. Q. Ngo article or not but the eigenvalues are also used in system control where an absolute eigenvalue greater than 1 can blow up the system.

To summarize, eigenvalues are used because they provide the convenient representation of the entire matrix. This is especially true when the matrix is SPD symmetric and (semi)positive definite such as the covariance matrix in quantitative finance.

[Khoa Tran]

Originally Posted by ngtridung

I am not sure if it has been mentioned in H. Q. Ngo article or not but the eigenvalues are also used in system control where an absolute eigenvalue greater than 1 can blow up the system.

He indirectly mentioned this characteristics in his example on difference equation. But it's worth to mention this qualitative behavior explicitly.

To summarize, eigenvalues are used because they provide the convenient representation of the entire matrix.

This is the key, in my opinion. That's what makes Spectral Theorem the single most important theorem in Spectral Theory.


How's about complex numbers? How do you see this super-beautiful invention? (Anh Hoang raised this question but for some reason, he deleted his own post???)

[hanguyen]

Anh Hoang's question is excellent. What if complex number is not invented, could we still reach to where we are now?

IMHO, the answer is NO. The invention of complex number, extending the real numbers to an algebraically closed field, helps to understand and easily manupulate the field of real numbers. Without this understanding and ease of use, can we assume that Control Theory, Harmonic Analysis, Quantum Theory, and so on will be within our reach?

To draw an analogy, although in theory the invention of bike is "not necessary" to the invention of motorbike, one would never have invented a motorbike without the experience of using a bike .