ica and svd

been poking around some more with svds, and came across this presentation that refers to independent component analysis. the wikipedia article is decent, with good refs. one way to view the ica is as a nonlinear generalization of pca; the components are not necessarily orthogonal but they minimize the mutual information and all the higher-order cross cumulants, rather than just the second like pca. the conditions of uncorrelated and independent are equivalent only for gaussion random vars, but ica has determinacy problems of more than one of the components are gaussian. this ebook has good info on it in chapter 6, if you can get access.

time-varying svd

recently i've been trying to see if anyone else has worked on time-varying singular value decompositions. there's surprisingly little out there on it, at least that i can find. i've tried thinking of the svd eigenmodes in terms of rotating coordinate systems. i remembered from dynamics classes once upon a time that some ways of representing rotation have a gimbal lock problem, but quaternions do not. and (i think) they also have linear propagation equations. the wikipedia article on it looks pretty good. i should probably read that and the 'see also' articles at the bottom.

random thought fragments on multiresolution analysis

someday i'd like to write a textbook on multiresolution techniques in surrogate modeling. in the intro i would present a unified approach to orthogonal decompositions like eigensystems, svd, and ffts with wavelets, linear model sensitivity analysis, and mdl. i would go on to answer questions like, 'how much information is gained in terms of the modes and singular values if i update an svd representation of a surrogate model?' and how do i do an svd surrogate model in the first place? will it capture features on different scales, or can i force it to? how do i use that predicted information gain to guide adaptive sampling?