Computer Intensive Methods in Control and Signal Processing: The Curse of Dimensionality (1997)

Due to the rapid increase in readily available computing power, a corre- sponding increase in the complexity of problems being tackled has occurred in the field of systems as a whole. A plethora of new methods which can be used on the problems has also arisen with a constant desire to deal with more and more difficult applications. Unfortunately by increasing the ac- curacy in models employed along with the use of appropriate algorithms with related features, the resultant necessary computations can often be of very high dimension. This brings with it a whole new breed of problem which has come to be known as "The Curse of Dimensionality" . The expression "Curse of Dimensionality" can be in fact traced back to Richard Bellman in the 1960's. However, it is only in the last few years that it has taken on a widespread practical significance although the term di- mensionality does not have a unique precise meaning and is being used in a slightly different way in the context of algorithmic and stochastic complex- ity theory or in every day engineering. In principle the dimensionality of a problem depends on three factors: on the engineering system (subject), on the concrete task to be solved and on the available resources. A system is of high dimension if it contains a lot of elements/variables and/or the rela- tionship/connection between the elements/variables is complicated.