cape -- cyclically adjusted price/earnings

i saw an interesting quote from page 3 of the money pull-out section of the january 8, 2011 issue of the financial times. 'eleven reasons to worry - but two reasons to invest' by merryn somerset webb. i thought the identification of the 'only really reliable long-term indicator' was interesting. i wonder if the assumption that predicted returns can come from either price adjustments as well as earnings depends on a dividend-paying stock. 'as societe generale's albert edwards points out, on a cyclically adjusted price/earnings (cape) ratio, the us market remains seriously overvalued. so, unless this time really is different (and i'd bet a good deal that it is not), we can expect to revert to mean at some point. let's not forget, as edwards put it, that while the history of the last 130 years or so has been both remarkable and appalling -- "the deaths of empires, the birth of nations, periods of deregulation, periods of re-regulation, world wars, revolutions, plagues and huge technological and medical advances -- "none of these events mattered from the perspective of value" the long run average cape remained much the same. ... i'm not worried about valuation in the short term. the cape is -- so far -- just about the only really reliable long-term indicator of stock market returns we have. but it is rarely much use to anyone in the shorter term. som, while we need to watch it, we don't need to panic every time it flashes at us.'

rotation, quaternions, etc.

i always have to look this stuff up when i need it, so i found a great ref here. from the abstract: We present the three main mathematical constructs used to represent the attitude of a rigid body in three- dimensional space. These are (1) the rotation matrix, (2) a triple of Euler angles, and (3) the unit quaternion. To these we add a fourth, the rotation vector, which has many of the bene¯ts of both Euler angles and quaternions, but neither the singularities of the former, nor the quadratic constraint of the latter. There are several other subsidiary representations, such as Cayley-Klein parameters and the axis-angle representation, whose relations to the three main representations are also described. Our exposition is catered to those who seek a thorough and uni¯ed reference on the whole subject; detailed derivations of some results are not presented. Keywords{Euler angles, quaternion, Euler-Rodrigues parameters, Cayley-Klein parameters, rotation matrix, di- rection cosine matrix, transformation matrix, Cardan angles, Tait-Bryan angles, nautical angles, rotation vector, orientation, attitude, roll, pitch, yaw, bank, heading, spin, nutation, precession, Slerp 1

fifty years in wall street

a highly recommended book on the history of us finance and some of the big names and players during the latter half of the 19th century is out of copyright and available for free online. it's a heft tome (~1000 pages) but you can get plain text, pdf with scanned images, or djvu files here or here. the plain text looks like it was ocr and is pretty good quality, though might cause some problems with tts.

windows python in cygwin

finally solved a problem (or found a workaround, at least) for something that had bothered me for a while: when i tried to use windows python (not cygwin python, which worked fine) in a xterm, it seemed not to be connected to stdout, stderr, and stdin. neither the interpreter nor the debugger prompt would show up, and nothing happened when i used print or sys.stdout.write. the mysterious thing was it would work from a non-x cygwin shell. but i needed mouse action on the desktop and screen (which uses a text-based x windows server) remotely. turns out the problem is how cygwin interfaces a non-cygwin console app from the terminal. it talks to it through pipes rather than with a real pty, and the issues there are deep and woolly. so all these windows programs are buffering in the pipe, not realizing how impatient i'm getting on the other end. fortunately, python has an easy workaround. the -i option makes it assume interactivity, skipping the tty check. i can use it on the cli or #! shebang, and now it's working. only problem is it drops me into an interpreter when the script finishes, so i have to type quit() (c-d, c-z, c-c are all ignored). ref here

parallel, numpy, shared memory,...

trying to figure out how to do parallel processing efficiently with python, and numpy in particular. i want something simple, closely related to the original serial code (sorry, mpi, you're not welcome here). parallelpython holds some promise, dodging the gil by starting separate interpreters and piping pickles back and forth. similar to pyro, and it looks pretty seemless between smp vs. cluster. unfortunately, pp does not provide for any shared mem so big data (even read only) must be copied (and pickled!) on smp. multiprocessing is now built in to 2.6 and backported as far as 2.4 or 2.3. doesn't handle remote processes, though the pp/pyro-type pickle server (manager) interfaces with inet ports. i think it basically forks the process to make the worker processes, so you get less overhead (os service vs. cranking up a new python). and there's no need to feed it modules or any other globals; these get copied on the fork. it has some capability to share memory, though i think these are only kinda raw ctype buffers. (i think all of this is similar to the approach posh used, though more generally for user-defined types -- high quality hackery but unmaintained since 2003.) apparently some people have coaxed numpy into using these ctype arrays to make np arrays sit in shared memory land, with views available to the children. (maybe using this sort of thing.) the approach got an attaboy from the big man himself, travis oliphant, but (in the same dir) sturla has a sharedmem module written later (cleaned up and posted here) that looks like it makes lower level sys calls to create shared memory space manually. does that mean the multiprocessing shm is unsatisfactory? the paper does warn that it's a moving target, and the scipy cookbook indicates the same thing: 'this page was obsolete as multiprocessing's internals have changed.' epd has a webinar coming up promising to demo multiprocessing with large arrays, so maybe i should see what they do. anyway, if i do use this for parallel stuff, this blog post might be useful. here's another page that looks very useful for multiprocessing. fabric looks interesting, too, though more geared toward sysadmin stuff. maybe similar to posh in some ways.

financial times rss

FT offers a dizzying array of separate rss feeds for its articles, but i don't think all of them end up in the print newspaper. i'd like to get the articles that the editors have deemed most important, without having to pay the £2/paper to get them. so... i bought a copy and tried to find where the articles in the paper show up in rss. and here they are, in roughly the order of importance given at the end of 'all you need to know about the city': lex column lex main 2nd (companies and markets) section companies: companies main, uk (companies|uk, though there's a lot that's not in the paper here), uk smaller companies (companies|uk) markets: markets main (markets section, last page) 1st section management: management main (business life) comment: comment main, opinion, analysis (comment & analysis) world: world main, europe, asia-pacific, africa, us, uk business, uk economy looks like google reader might be a convenient way to combine all the streams into one and maybe even keep track of which have been read already. and i was hoping the mobile version (http://www.google.com/reader/m) would simplify the scraping to whittle it down around the body text. but it doesn't always work; sometimes it only takes the first paragraph or two and scraps the rest, probably because it catches a break before a table or image. so i think i'll have to navigate through to the original page and scrape from there. EDIT: i guess the people at ft are smart enough to make it easy for me. they post links for their print edition (and us, europe, middle east, and asia editions). only thing i didn't find on that page was the 'money' special pullout from the weekend edition. i think most of those articles were in the 'personal finance' section of the website. and i discovered that each html page for a section has an rss icon link in the upper right, so it's easy to snag stuff once i know where they are on the website. going through the sections, i found they went roughly in order with pretty close, though not exact, correspondence to the articles in print. here are the sections on the website and the page numbers of articles listed under them, to give you an idea of the density: front page: 1,1 must read national news: 2,2,2,3,3,4,4,4,4,4,4 skip 1/2 to 2/3 world: 5,5,5,5,6,6,6,6,6,6,6,6,7,7,7,7,8,8,8 good read comment & analysis: ,9,10,10,10,10,11,11,11,11 skip some of these, though the latter ones are really good letters: 10,10,10,10,10 skip all of these! life & arts: (pull-out) 1,2,2,2,2,3,4,4,5,5,5,6,7,6,6,7,8,9,19,10,10,11,11,11,12,12,13,13,14,14,20,20,17,17,17,17,17,17,17,17,17,17,17 a lot of things skipped between 14 ad 17, but i would skip this whole section. ft magazine: (pull-out) 15,54,12,7,10,8,44,44,47,43,43,46,46,52,53,51,50,49,48 i would skip almost all of this house & home: (pull-out) 1,2,2,3,6,7,7,8 i would skip practically all of this section lex: 24,24,24,24 must read companies: 12,12,?,13,13,12,14,14,14,14,14,15,15,15,14,15,16,16,16,16,?,17,17,?,17 good read, especially toward the end markets: 22,22,?,23,23,23,23,24 must read