Duhctaep's Lab

Funding beyond discounting: collateralagreements and derivatives pricing

2012-01-31T06:20:00.000-08:00

apparently ground-breaking article by vladimir piterbarg, the head of barcap quantitative research and author of a well-known 3 volume series on interest rate modeling.
http://www.scribd.com/doc/34328165/Risk-Magazine-Piterbarg-Funding-Beyond-Discounting-Collateral-Agreements-and-Derivatives-Pricing

important for correctly valuing derivatives, and what the 'risk-free' rate really is.

debugging c++ extensions to python

2012-01-11T04:37:00.001-08:00

trying to debug a python extension module written in c++ (wrapped with swig). i think this would be so much easier if i were using gcc, but python is built with msvc... setup.py wants the debug versions of python libs, but i don't have them and don't really want to try to build python from scratch right now.

these refs seem relevant:
http://www.velocityreviews.com/forums/t677466-please-include-python26_d-lib-in-the-installer.html
http://vtk.org/gitweb?p=VTK.git;a=blob;f=Wrapping/Python/vtkPython.h;h=9d01ac21bafae0a24252398f268b6b3563df62cd

design patterns

2012-01-10T08:14:00.000-08:00

comment from duffy on most useful from gof (gang of four) book of object-oriented design patterns: "GOF is useful but should not become an objective in itself. The most useful ones in general (dependent on the domain of course) are Visitor, Strategy, Facade and Template Method pattern. Singleton and Observer can best be avoided."

most relaxing music

2012-01-10T07:30:00.000-08:00

The top 10 most relaxing tunes were: 1. Marconi Union - Weightless 2. Airstream - Electra 3. DJ Shah - Mellomaniac (Chill Out Mix) 4. Enya - Watermark 5. Coldplay - Strawberry Swing 6. Barcelona - Please Don't Go 7. All Saints - Pure Shores 8. AdelevSomeone Like You 9. Mozart - Canzonetta Sull'aria 10. Cafe Del Mar - We Can Fly

Stream it from SoundCloud right here

Malliavin calculus

2012-01-10T02:14:00.000-08:00

saw a reference to Malliavin calculus. used in financial math to take derivatives of stochastic processes. looks interesting, might be useful to learn some day.

trading strategies

2011-10-14T01:38:00.001-07:00

here's an idea: delta hedge triple-leveraged etfs. want to have (price of underlying)/(price of hedge) as high as possible, with high vol on underlying. take advantage of the fact that leveraged etfs rebal their derivative guts daily, so their delta changes with the underlying price. hold the hedge, buy/sell the underlying to match delta, make $$ every time the underlying price bounces up, with very low risk as long as you adjust the hedge daily. if you have too much cash, buy more hedge when the price goes up instead of selling the underlying. look for 3x etfs at http://3xetf.com/ this is very similar to the way people delta hedge options, but very simple to manage and with no trader-type restrictions. another way to do it could be to hold the underlying and buy/sell the hedge. i'll need to check into liquidity/trading cost issues there.

for example:
long term treasuries: hedge tlt (or maybe vglt?) with sbnd.
financials: hedge sef with fas.
real estate: hedge schh with drv.
russia: hedge ? with rusl
s&p500: hedge spy, ivv with spxu

or maybe the factorshares spread etfs?

python/c++ with microsoft visual c++

2011-09-14T03:08:00.000-07:00

finally got a 64 bit pyd python extension working, compiled with ms visual c++ and visual studio for epd on windows. figured out that the version string in python (MSC v.1500 64 bit (AMD64)) was for visual c++ 2008 == v9.0, and the express edition only builds 32 bit. so i had to get the sdk (version 7 works with 2008) and make sure the amd64 stuff got installed with it.

some notes say to install the service pack before the 64 bit stuff, but i didn't find this necessary.

i ran the 'Windows SDK Configuration Tool' from the start menu, since it sounded logical, and ticked the box to link the sdk with VC 2008. not sure if that was necessary or not.

one change i had to kludge manually was changing the references in C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\vcvarsall.bat under the amd64 label. originally they were pointing to "%~dp0bin\amd64\vcvarsamd64.bat"; they need to be "%~dp0bin\vcvars64.bat". at that point, running vcvarsall.bat amd64 should work and a simple use of weave passes:

import scipy.weave as w
c = w.inline(r'printf("hi.");',verbose=2)

now that the compiler, etc., are set up i can use swig and distutils to build bigger extensions (like in the swig docs), and it Just Works!

thinking in c++

2011-09-01T03:02:00.000-07:00

lots of versions out there, but this one seems to be the best one of volume 1 i can find. (get the non-landscape one.) don't get that version of volume 2. it is old and incomplete.

the 27 dec 2003 version of volume 2 is the print version; i can't find any electronic version other than the html. i think the quality and depth is bettern than vol 1, but it's too long for me to go through right now. c++ is definitely a language that has a vast landscape to explore. i think i would need to read the 'design and evolution' book to understand the rationale and really get at how things work, and i will eventually need to read the templates book and the gang of four design patterns book to know how to use it properly. maybe the 'effective' and 'exceptional' books, too. for now i will prioritize the parts of vol 2 that i haven't read yet.

1: Exceptions: p. 53-75
9: Multiple inheritance
10: Design patterns
6: Generic algorithms
7: Generic containers
8: Runtime type identification
2: Defensive Programming
5: Templates: template programming idioms - end (p. 252 in html imported to word doc)
11: Concurrency (might skip, focuses on specific library)

a few tips to remember:

-- return value optimization: return a constructor call; it saves a copy-ctor and dtor
-- it's ok to throw in a ctor, as long as you clean up resource allocations first
-- never throw in a dtor, since the dtor could have been called due to a previous exception and c++ requires 2 unhandled exceptions to crash the program
-- throw by value, catch by reference: allow the exception handling to deal with the memory management
-- deal with errors and catch exceptions at the nearest handler that has enough information to deal with them

wireless on suspend/resume

2011-07-20T07:38:00.000-07:00

had a problem with some ubuntu upgrades hosing my ath5k wifi after resuming from a sleep. something was messed up with the power getting turned back on, i think, and it seemed to be something that would get fixed and rebroken judging from the dmesg errors and bugposts on kernel.org. downgrading my kernel didn't fix the problem.

fortunately, i found that turning it off and on with rfkill avoided the problems, so i put it into a script to do it automatically.

/etc/pm/sleep.d/50_wireless:

#!/bin/sh
case "${1}" in
suspend)
#date >> /tmp/pm.log
#echo ' suspend' >> /tmp/pm.log
/usr/sbin/rfkill block wlan
;;
resume)
#date >> /tmp/pm.log
#echo ' resume' >> /tmp/pm.log
/usr/sbin/rfkill unblock wlan
;;
hibernate)
# nothing
;;
thaw)
# nothing
;;
esac

cygwin-ports

2011-07-07T07:08:00.000-07:00

cygwinports seems to be just what i was looking for to get packages that don't come with the standard set. they have rtmpdump and mplayer, and i'll check back there before trying to compile stuff again.

android sdk, avd, !*&$!$%^

2011-07-07T07:02:00.000-07:00

tried to get android emulator working on my admittedly underpowered netbook. it works... sort of. the 3.1 platform was _really_ slow. i tried the snapshot method to make it at least boot faster; no dice. maybe 2.2 would be better, but i think what i'll really try (if i do again) is the android-x86 iso on a flash drive. or just wait -- google and/or intel seems to be coming out with x86 native android later this year.

kindle on linux

2011-07-07T07:00:00.001-07:00

followed these instructions to get kindle for pc running on my ubuntu netbook with wine:

sudo add-apt-repository ppa:ubuntu-wine/ppa

sudo apt-get update

sudo apt-get install wine1.3

after installation, just run wine on the installed Kindle.exe in ~/.wine/drive_c/ etc.

karnickel: macros in python

2011-05-07T03:37:00.001-07:00

not sure when exactly a macro would be useful in python. i remember seeing such a thing in some cython code, to deal with a c++ template, though karnickel deals with the python ast so probably not useful for that. but, there it is, if the need arises.

python embedded in gdb

2011-05-05T07:55:00.000-07:00

version >=7 of gdb has an embedded python interpreter. here's a tutorial on it. very handy if i need to debug c or c++. i'm guessing a million new debugger guis will be built on top of this.

compressive sampling

2011-04-26T02:32:00.000-07:00

just came across a new take on an old problem: compressed sensing. here's a gentle introduction to the subject. looks like youtube has some interesting lectures on this, too, especially linking compressive sensing or sampling to information theory.

one thing that occurred to me is that the sparseness object expressed by the '0-norm' requires the part of the domain where the function goes to zero to do so faster than the norm parameter. otherwise, the 0^0 would be 1, just like x^0==1 for x!=0. so, something like lim x->0 exp(-1/x)^x. maybe i can think of it as an epsilon-norm, where epsilon is infinitessimal for all non-zero quantities but not exactly zero for the norm of zero.

trading optimization

2011-03-29T02:39:00.000-07:00

i've been thinking about how to optimize trades for some algorithmic trading. i need to define objective functions, obviously, but how exactly? risk vs. return, but risk (for a liquid asset) depends on a portfolio state between transactions, and returns are realized upon the transaction.

it just occurred to me that i can reduce it to simple shift and scaling operations on a (somewhat unknown) pdf. if i have a certain unhedged amount, x, of a risky asset, and i buy or sell to end up with a*x, then that scales the pdf f(x) to f(a*x)/a. the trade should give me a return, r, so the pdf of my total return after the trade would be something like f(a*x-r)/a. so i know what the shift and the scale are, even though i don't exactly know what f(x) is. now it's a question of whether f(a*x-r)/a is better than f(x), and i trade if it is (i.e., maximize whatever metrics to find the optimal r,a).

does this answer the question of how to place limit orders away from the market prices? i'm not sure yet.

factor shares

2011-03-17T08:00:00.000-07:00

ib is offering its customers commision-free trades on factorshares spread etfs. this looks really interesting: diy hedge fund with asset classes. trying to figure out how the funds operate, based on the holdings info from their website...

each of the 5 has basically $3m in treasuries, $2m in cash, and a thousand in a treasury fund. each one then takes a position in nearest future contract (long and short) for the two spread assets, for 2x the nav/share * 100k shares/unit * 2 units (currently). the treasuries are probably to offset the time discount on the futures contracts, and that and the cash are for margin, obviously. the fund is probably for a little bit of liquidity during the daily rebalance. i'm guessing the nav comes from the $5m risk free + net market value of the futures. the nav/share doesn't exactly match the price, probably given the constraints of the contract sizes and maybe the lower liquidity of the etf at the moment. the futures holdings are updated daily, to restore dollar neutrality (same forward contract dollar amounts). for the current number of shares and risk-free holdings, the funds basically started at $25/share. the s&p e minis are apparently for lots of 50. the 30yr tbill futures are for lots of 100.

the 5 etfs are s&p/tbill, tbill/s&p, s&p/usd, oil/s&p, gold/s&p, where each is bull/bear. 2x leverage on each leg, for a 4x total leverage (but still just 2x on the spread).

right now the volume is pretty light, around 10k/day, since they just lauched a few weeks ago. i wonder if there is any arbitrage opportunity for these, knowing their methodology. especially for s&p/tbill vs. tbill/s&p.

does the rebalance accumulate anything based on independent underlying price movements? no, because all the money gained from selling an advancing future is put into the one that declined, and the same thing will happen at the end of the next day. but if i rebalance between these, it will. i'm just not sure if it's any better than just rebalancing among 2x single asset funds, unless you think they're anticorrelated instead of just uncorrelated. maybe it has the advantage of not needing to rebalance with cash, like single funds would.

one thing maybe i can try is to arb the s&p/tbill vs. tbill/s&p like this: right now fse has 139 emini and -71 tbill, and fsa has -177 emini and 91 tbill. closing for fse was $22.57 and fsa was 28.18. if i could buy 177/139 fse it would cost 28.74, or 91/71 would cost 28.92. so if i had some of each, i could have sold fse and bought fsa near the end of the day, eg sell 9 fse for 203.13 and buy 7 fsa for 197.26. that would be selling a share of 1251 emini, -639 tbill and buying a share of -1239, 637. so the net would be 42, -2.

nav calculation time is based on the first of the futures contracts to settle:
s&b/tbond 3pm (ET)
tbond/s&p 3pm
s&p/usd 3pm
oil/s&p 2:30pm
gold/s&p 1:30pm
the nav must depend on the price of the futures and what exposure they can be rebalanced to.

futures prices might not be as easy to come by, so maybe i could compare to other leveraged etfs like sso, sds, dgl (or iau/gld (not leveraged) or dgp/dzz (monthly, not daily)), dbo (or dig?, uso, oil), tlt (not leveraged), tbt (or pst? no), udn (not leveraged) (or uup, not leveraged and bull instead of bear but much higher volume) (these only use dx contracts, not front month). this wouldn't necessarily be perfect since nav can deviate from price, but it should be close for the heavily traded ones.

here's a good ref list for leveraged etfs

aptitude tests

2011-03-12T07:30:00.000-08:00

seems like a lot of companies are using tests by their hr department to screen applicants. shl is one that is mentioned a lot. this one was referred to as having good practice tests.

audio + video

2011-03-11T06:07:00.000-08:00

figured out how to get an alternate audio into a video file:

# take mp3 out of audio flv
ffmpeg -i altAudio.flv -acodec copy altAudio.mp3
# take the video only out of the video
mencoder -ni -ovc copy -nosound -o noAudio.mp4 origVideo.mp4
# put the audio and video together
mencoder -ovc copy -oac copy -audiofile altAudio.mp3 -o out.avi noAudio.mp4

might not need both the ffmpeg step and -ni on the video extraction, but at least one of them is necessary. i tried without both and the video played at double speed while the audio was normal speed in the out.avi. might also be able to use a different format out with -of. not sure if avi transcodes it.
EDIT: tried without -ni, still works fine.

might need to adjust delay with - and +.

also, i found that rtmpdump has very handy -e and -o options. i dropped the -A option with -e and it seems to work (kind of), not sure if that is absolutely necessary.

now i'm trying to get mencoder on a file to let me playback faster. tried -vf filmdint and filmdint=io=2:1, and it doesn't hurt but doesn't seem to be essential. so far, best results are with just -fps 30000/1001 -ofps 20000/1001 and speed it up with ']'. -oac copy doesn't work; have to use something else like pcm (uncompressed) or mp3lame. the original audio is 48000Hz aac, which i can encode with -oac lavc -lavcopts acodec=aac, but that makes encoding really slow and messes up a/v sync, even when i try to compensate by changing -srate.

so far this version looks promising, but the avi has no idx:
mencoder -ovc copy -oac mp3lame -fps 30000/1001 -ofps 20000/1001 -o out.avi infile.mp4

could try -of mpeg or lavf, with a lavf format=mp4 for example, since the original is mp4 and plays pretty well (except when i speed it up too much).

ok, this one seems to work pretty well:
mencoder -ovc copy -oac mp3lame -fps 30000/1001 -ofps 20000/1001 -of lavf -lavfopts format=mp4 -o out.mp4 infile.mp4

national review online audio

2011-03-02T04:31:00.000-08:00

outloudopinion.com has apparently been posting audio files of select nro articles for at least a year now, much to my surprise. even have a convenient rss feed.

REMINISCENCES OF A STOCK OPERATOR

2011-02-23T01:12:00.000-08:00

by Edwin LeFevre. recommended by more than one trader, to give an accurate feel of what being a trader is really like. one trader i talked to described trading as a spiritual experience, that you have to do a gut check every day when you have that much on the line.

amazon has a nice annotated copy available that gives good historical background information and explanation of some of the things that are harder to understand now. but gutenberg has the original up.

pybtex

2011-02-16T06:34:00.000-08:00

if i ever need to mess around with beastly bst (bibtex style) files, i won't. i'll switch to pybtex, which looks like it's a lot more fun to use.

quant topics i should learn

2011-02-01T06:12:00.000-08:00

looked through a big glossy from wilmott's certificate in quantitative finance. saw some terms that i wasn't sure i knew about. so here's a list, so i can make sure i learn about them and i don't need to take any of their classes.

fokker-planck and kolmogorov
the radon-nikodym derivative
girsanov's theorem
yield, duration, and convexity
stochastic and spot-rate models
affine stochastic models
heath, jarrow and morton
reduced-form model and the hazard rate
structural default models
cds pricing, market approach
synthetic cdo pricing
risk of default, structural and reduced form
copulas
brownian bridge (monte carlo?)
sobol' (quasi monte carlo?)
crank-nicolson
black-litterman (portfolio management)
levy copulas (cdo pricing)
fixed income: bgm, black 76
variance gamma
vg le'vy
stochastic monetary policy models for interest rate derivatives
gram-schmitdt process

how i became a quant

2011-02-01T06:08:00.001-08:00

very good book to get a feel for the kind of personalities in the quant world and peeks into the various places that quants work. also small glimpses into the types of problems quants work on.

neil chriss has some interesting refs for using binomial and trinomial tree to match price/volatility of all available options simultaneously for european and american options. also, some other work he did on optimization of portfolio transition/liquidation combining transaction cost/liquidity with risk of holding a position too long. probably worth a look.

peter j\"{a}ckel makes some interesting comments about (not surprisingly) the monte carlo method with low-discrepancy sequences in finance, in particular about getting good results with sobol' numbers in high dimensions.

andrew weisman, big shot at merrill lynch, gives some warnings about the silicon ceiling: don't appear too nerdy or academic or people will think you can't make decisions. other aspects of the culture are more like a dog pack than a meritocracy. also, information-free performance enhancements: smoothing, selling volatility, and doubling up. use cvar to help with nonsymmetric hedge fund returns, resampled optimization to deal with difficult error estimation. risk metrics are ordinal, not cardinal, quantities.

chapter 23 has an interesting reference to market microstructure -- using supply and demand curves on the level of bid/ask spread and probabilities of tick movements. also, a ref to a publisher that prints classic biographicals on turn-of-the-century market operators.

cape -- cyclically adjusted price/earnings

2011-01-29T03:21:00.000-08:00

i saw an interesting quote from page 3 of the money pull-out section of the january 8, 2011 issue of the financial times. 'eleven reasons to worry - but two reasons to invest' by merryn somerset webb. i thought the identification of the 'only really reliable long-term indicator' was interesting. i wonder if the assumption that predicted returns can come from either price adjustments as well as earnings depends on a dividend-paying stock.

'as societe generale's albert edwards points out, on a cyclically adjusted price/earnings (cape) ratio, the us market remains seriously overvalued. so, unless this time really is different (and i'd bet a good deal that it is not), we can expect to revert to mean at some point.

let's not forget, as edwards put it, that while the history of the last 130 years or so has been both remarkable and appalling -- "the deaths of empires, the birth of nations, periods of deregulation, periods of re-regulation, world wars, revolutions, plagues and huge technological and medical advances -- "none of these events mattered from the perspective of value" the long run average cape remained much the same.
...
i'm not worried about valuation in the short term. the cape is -- so far -- just about the only really reliable long-term indicator of stock market returns we have. but it is rarely much use to anyone in the shorter term. som, while we need to watch it, we don't need to panic every time it flashes at us.'

rotation, quaternions, etc.

2011-01-25T03:11:00.000-08:00

i always have to look this stuff up when i need it, so i found a great ref here. from the abstract:

We present the three main mathematical constructs used to represent the attitude of a rigid body in three-
dimensional space. These are (1) the rotation matrix, (2) a triple of Euler angles, and (3) the unit quaternion. To
these we add a fourth, the rotation vector, which has many of the bene¯ts of both Euler angles and quaternions, but
neither the singularities of the former, nor the quadratic constraint of the latter. There are several other subsidiary
representations, such as Cayley-Klein parameters and the axis-angle representation, whose relations to the three main
representations are also described. Our exposition is catered to those who seek a thorough and uni¯ed reference on
the whole subject; detailed derivations of some results are not presented.
Keywords{Euler angles, quaternion, Euler-Rodrigues parameters, Cayley-Klein parameters, rotation matrix, di-
rection cosine matrix, transformation matrix, Cardan angles, Tait-Bryan angles, nautical angles, rotation vector,
orientation, attitude, roll, pitch, yaw, bank, heading, spin, nutation, precession, Slerp
1

fifty years in wall street

2011-01-19T05:38:00.000-08:00

a highly recommended book on the history of us finance and some of the big names and players during the latter half of the 19th century is out of copyright and available for free online. it's a heft tome (~1000 pages) but you can get plain text, pdf with scanned images, or djvu files here or here. the plain text looks like it was ocr and is pretty good quality, though might cause some problems with tts.

windows python in cygwin

2011-01-06T07:54:00.000-08:00

finally solved a problem (or found a workaround, at least) for something that had bothered me for a while: when i tried to use windows python (not cygwin python, which worked fine) in a xterm, it seemed not to be connected to stdout, stderr, and stdin. neither the interpreter nor the debugger prompt would show up, and nothing happened when i used print or sys.stdout.write. the mysterious thing was it would work from a non-x cygwin shell. but i needed mouse action on the desktop and screen (which uses a text-based x windows server) remotely.

turns out the problem is how cygwin interfaces a non-cygwin console app from the terminal. it talks to it through pipes rather than with a real pty, and the issues there are deep and woolly. so all these windows programs are buffering in the pipe, not realizing how impatient i'm getting on the other end.

fortunately, python has an easy workaround. the -i option makes it assume interactivity, skipping the tty check. i can use it on the cli or #! shebang, and now it's working. only problem is it drops me into an interpreter when the script finishes, so i have to type quit() (c-d, c-z, c-c are all ignored).

ref here

parallel, numpy, shared memory,...

2011-01-06T03:50:00.000-08:00

trying to figure out how to do parallel processing efficiently with python, and numpy in particular. i want something simple, closely related to the original serial code (sorry, mpi, you're not welcome here).

parallelpython holds some promise, dodging the gil by starting separate interpreters and piping pickles back and forth. similar to pyro, and it looks pretty seemless between smp vs. cluster. unfortunately, pp does not provide for any shared mem so big data (even read only) must be copied (and pickled!) on smp.

multiprocessing is now built in to 2.6 and backported as far as 2.4 or 2.3. doesn't handle remote processes, though the pp/pyro-type pickle server (manager) interfaces with inet ports. i think it basically forks the process to make the worker processes, so you get less overhead (os service vs. cranking up a new python). and there's no need to feed it modules or any other globals; these get copied on the fork. it has some capability to share memory, though i think these are only kinda raw ctype buffers. (i think all of this is similar to the approach posh used, though more generally for user-defined types -- high quality hackery but unmaintained since 2003.) apparently some people have coaxed numpy into using these ctype arrays to make np arrays sit in shared memory land, with views available to the children. (maybe using this sort of thing.) the approach got an attaboy from the big man himself, travis oliphant, but (in the same dir) sturla has a sharedmem module written later (cleaned up and posted here) that looks like it makes lower level sys calls to create shared memory space manually. does that mean the multiprocessing shm is unsatisfactory? the paper does warn that it's a moving target, and the scipy cookbook indicates the same thing: 'this page was obsolete as multiprocessing's internals have changed.' epd has a webinar coming up promising to demo multiprocessing with large arrays, so maybe i should see what they do. anyway, if i do use this for parallel stuff, this blog post might be useful.

here's another page that looks very useful for multiprocessing.

fabric looks interesting, too, though more geared toward sysadmin stuff. maybe similar to posh in some ways.

financial times rss

2011-01-06T03:13:00.000-08:00

FT offers a dizzying array of separate rss feeds for its articles, but i don't think all of them end up in the print newspaper. i'd like to get the articles that the editors have deemed most important, without having to pay the £2/paper to get them. so... i bought a copy and tried to find where the articles in the paper show up in rss. and here they are, in roughly the order of importance given at the end of 'all you need to know about the city':

lex column
lex main

2nd (companies and markets) section
companies: companies main, uk (companies|uk, though there's a lot that's not in the paper here), uk smaller companies (companies|uk)
markets: markets main (markets section, last page)

1st section
management: management main (business life)
comment: comment main, opinion, analysis (comment & analysis)
world: world main, europe, asia-pacific, africa, us, uk business, uk economy

looks like google reader might be a convenient way to combine all the streams into one and maybe even keep track of which have been read already. and i was hoping the mobile version (http://www.google.com/reader/m) would simplify the scraping to whittle it down around the body text. but it doesn't always work; sometimes it only takes the first paragraph or two and scraps the rest, probably because it catches a break before a table or image. so i think i'll have to navigate through to the original page and scrape from there.

EDIT: i guess the people at ft are smart enough to make it easy for me. they post links for their print edition (and us, europe, middle east, and asia editions). only thing i didn't find on that page was the 'money' special pullout from the weekend edition. i think most of those articles were in the 'personal finance' section of the website. and i discovered that each html page for a section has an rss icon link in the upper right, so it's easy to snag stuff once i know where they are on the website.

going through the sections, i found they went roughly in order with pretty close, though not exact, correspondence to the articles in print. here are the sections on the website and the page numbers of articles listed under them, to give you an idea of the density:
front page: 1,1 must read
national news: 2,2,2,3,3,4,4,4,4,4,4 skip 1/2 to 2/3
world: 5,5,5,5,6,6,6,6,6,6,6,6,7,7,7,7,8,8,8 good read
comment & analysis: ,9,10,10,10,10,11,11,11,11 skip some of these, though the latter ones are really good
letters: 10,10,10,10,10 skip all of these!
life & arts: (pull-out) 1,2,2,2,2,3,4,4,5,5,5,6,7,6,6,7,8,9,19,10,10,11,11,11,12,12,13,13,14,14,20,20,17,17,17,17,17,17,17,17,17,17,17
a lot of things skipped between 14 ad 17, but i would skip this whole section.
ft magazine: (pull-out) 15,54,12,7,10,8,44,44,47,43,43,46,46,52,53,51,50,49,48 i would skip almost all of this
house & home: (pull-out) 1,2,2,3,6,7,7,8 i would skip practically all of this section
lex: 24,24,24,24 must read
companies: 12,12,?,13,13,12,14,14,14,14,14,15,15,15,14,15,16,16,16,16,?,17,17,?,17 good read, especially toward the end
markets: 22,22,?,23,23,23,23,24 must read

dashboard and screen scraping

2010-12-23T03:14:00.000-08:00

one thing that's been on my low-priority radar is a way to scrape through the complex flaming hoops that banks, credit cards, and investment brokerages put up so i can have an auto dashboard, showing me account balances and net worth at a glance.

mechanize looks like a nice package for performing many browser functions, including form interaction; probably the best of its kind i've found (and nice faq). however, it does mean writing a browsing session from scratch (read: lots of online debugging) and i'm not sure how well it can handle javascript, frames/windows, and all the other eye candy screen junk these sites like to throw at you.

someone out there recommended pyxpcom (combined with pydom in pythonext) as a way to do anything mozilla can. i think that must be true, since it seems to be just the pieces that mozilla-esque browers are made of. as powerful and difficult to use as a build-your-own-ferrari kit.

i think the most promising option seems to be selenium, which is apparently merging with webdriver for version 2.0. basically drives a real browser, but can record and play back scripts in a variety of languages (including python). the webdriver type of interface seems to be the future of selenium, and it has the advantages of better navigation and less to install. written in java, but i think it can do python (though the docs are behind if so). so i'm not sure if i should just wait for an official release of 2.0, but it does look like selenium is what i'm after. here's the doc on using ide.

EDIT: did some more looking around with selenium, and wow! i love the ide/rc combo. i think i need to look at this blog post to get the most out of locators (css vs. xpath). some of the extra plugins for selenium-ide are worth getting, and the selenium.py module can apparently just be copied into the python path to use selenium-rc. 1.0.11 has firefox 4 support in the ide, but it's very recent (2011-04-12).

they have put out a number of rcs for v2; apparently the v2 release is coming summer 2011. no remote control javascript server is necessary for version 2 since it's integrated with webdriver. i need to know if the ide and python export will still work. right now i think python will work, but no ide yet (though 2.0 is probably backwards compatible so might run the code generated by the version 1 ide).

more selenium links: command locators, xpath/css/dom rosetta, css locators are faster than xpath, good info, stay up to date,good example,

managed to get selenium python bindings installed on a windows machine (not surprisingly, a bit more involved than on linux) with my epd python. had to manually download tar ball, python setup.py install, and manually create the test dir structure that it would then complain about. maybe there's an option to make it skip tests, but the kludge was faster than looking that up. now i have selenium 2 with the webdriver interface, much better than rc! and btw, my experiments confirm what others have said about locators: css is much faster than xpath, even on firefox. i've also found that, while the selenium ide is really good for getting started with the locators, it's often possible to find shorter, more informative, and likely more stable tags and ids by poking around in the html just a little rather than using the first thing that pops up in the ide table. so i'm not going to try to keep a drop-in interface to call into the ide-generated scripts; cut-n-paste of one-liners will be good enough for both dev and maintenance. still, there is tremendous value in starting with something that works, and that alone makes the ide worth the install.

some other things i've learned: the 'andWait' stuff is only relevant from the java interface. in python, there's no way to keep running asynch while stuff is still loading. click, get, etc. only return to the python script once it's fully loaded, so that can be a latency bottleneck. i did poke around and find a possible place to change that, but i'll see if i really need to.

scraping off microsoft's look and feel

2010-12-13T07:51:00.000-08:00

visually impaired people have at least one thing going for them: the high contrast windows theme is much better than windows standard. and the opticwhite theme for google chrome helps as well. now i don't feel quite so out of place and my eyeballs won't melt.

virtualbox

2010-12-11T05:51:00.000-08:00

holy grail: windows 7 and ubuntu dual boot, with either also running under a vm in the other. i've been looking around for a good virtual machine; some of the old ones seem kinda dead or just emulate (read: way too slow) (bochs, plex86) or are too slow except on linux host or maybe forked off somewhere else (qemu/kqemu or whatever). xen looks pretty good, especially with its capability to run a guest os off a partition. but it only runs on a linux host, so that's only half the answer, and i'm not sure if anyone has gotten it to use an existing windows install.

virtual box can run on windows or linux host, and it can run either guest at virtualized native speed. and maybe i can get it to run an existing windows or ubuntu from the other. one problem with booting the oem windows partition is that setting up virtualbox from linux will require bootrec.exe from an install (not recovery) dvd.

(colinux might be able to boot an existing install, but colinux is limited to one processor atm.)

getting spyder and python to work on windows 7

2010-12-10T10:24:00.000-08:00

had to struggle to get my python install working with numpy and spyder, probably because i copied them over from another install.

with spyder, i had to move the 2.0.0beta5 egg and give the --prefix option to setup.py in order to install and use 2.0.3.

with numpy, i was getting 'ImportError: DLL load failed: The specified module could not be found.' when i tried to import numpy, _unless_ i was in the Python26_64\Scripts dir. i think it's because of the mkl lapack dlls in there. but it all worked once i added that to my PATH (not PYTHONPATH) with the help of cygwin -- just using export in bash allows me to peek at what it would be in windowsese with os.environ['PATH'] so i could put it into the env editor in spyder. viola!

sshd on cygwin

2010-12-10T05:23:00.000-08:00

got a new machine with windows 7, and i think i'll have to actually use the windows partition. ugh.

fortunately, cygwin comes with a kajillion unix packages that make microsoft bearable. and i just got my openssh server up and running, so i can still log in from home or elsewhere. here's how:

first, i followed the steps here to get rid of any old failed-attempt kruft. then, even though i did 'run as administrator' on the cygwin bash startup bat, it still gave me warnings when i ran ssh-host-config and tried to use my windows user for running sshd. so i went back and did all the editrights lines. rerunning ssh-host-config (probably unnecessary) gave no warnings, so i started it up with 'cygrunsrv -S sshd' as suggested here (also used the 'tty ntsec' for CYGWIN, as he suggests). and it Just Works.

the ascent of money

2010-12-06T06:51:00.000-08:00

just watched a documentary on pbs called 'the ascent of money' (google it, easy to find). nice 4-part series on some of the history of finance and how it ties in with the history of the world. should be required viewing. i hear the book is good, too, but the video is easier for lazy people like me.

potato-powered web server

2010-11-27T04:11:00.001-08:00

when i see pages like this, it scares me a little to think how similar i am to this person.

15.433

2010-11-23T01:30:00.000-08:00

from ocw

3
harry markowitz, 'portfolio selection', journal of finance, march 1952, u. of chicago (grad student)
'the process of selecting a portfolio may be divided into two stages. the first stage starts with observation and experience and ends with beliefs about the future performances of available securities. the second stage starts with the relevant beliefs about future performances and ends with the choice of portfolio'
my code concerns second part for computation, but could also guide quantitative, rational, unemotional thought and validate assumptions made in the first part.
s&p 500 returns look nearly bimodal, with a larger positive mode and smaller negative mode
10 year treasury bills returns have small tails

9
fama french three-factor model
connections to other ratio patterns
estimates for the factors 1963-2000
momentum (short-term positive correlation, long-term reversals) most studied anomaly in finance (2000)

10
equity option valuation
risk neutral pricing
binomial trees
put/call parity
black scholes formula
implied volatility
survey: why do institutions use options?

at page 200, printed landscape (should be portrait)

thinking in c++

2010-11-22T06:28:00.000-08:00

both volumes of the 2nd edition are freely available online, as is a draft version of 'thinking in python'. they're all a bit old, though they look like worthwhile reads.

the office

2010-11-22T06:04:00.000-08:00

dwight cracks me up:
http://www.noob.us/humor/the-office-fire-drill/
http://www.youtube.com/watch?v=zthtgZNJ5sc&feature=related
http://www.youtube.com/watch?v=oLsOS362hkM
http://www.youtube.com/watch?v=cbm5EoAm1bI

c++ design patterns and derivatives pricing (2nd ed)

2010-11-17T06:15:00.000-08:00

just finished a nice quant book by mark joshi. still need to go back through the exercises, but it was a nice taste of some basic but real quantitative finance data structures and their c++ implementation, as well as more general c++ and oo issues. here are a few points that i think i should remember:

open-closed principle
open for extension, closed for modification (don't require editing files to extend their functionality)
const
good for discipline, safety, and optimization
rule of three
with no declared copy constructor, the compiler will do a shallow copy
if destructor, assignment operator, or copy constructor is defined, define the other two, too
rule of almost zero
almost never declare any of destructor, assignment, or copy
use smart pointers instead: shared_ptr, scoped_ptr, or Wrapper
to avoid memory leaks
'almost': class with abstract methods is likely to be deleted by pointer to the base class, so it needs a virtual destructor
stlport.org provides an stl with range checking, useful if you're not using visual c++ 8.0 (it already has debug mode)
boost.org is intended to become part of c++ standard
quantlib and xlw
commands never to use
malloc
free
new []
delete []
throw exception: ok in constructor, never in a destructor
new is slow
floating point errors do not cause exceptions by default, but they can be made to
pimpl
private implementation: one class is visible to clients with no data except a pointer to another class, which is defined in the source file in an unnamed namespace
difference between encapsulation and insulation: private changes do not affect classes protected by encapsulation, but fail to insulate because all clients have to recompile
lots of good refs like 7, 11, and 18

ath5k woes

2010-11-02T03:16:00.000-07:00

okay, is everyone else out there as bugged as i am about the ath5k driver? it seems like every ubuntu maintenance update i've done over the last couple of months on my acer aspire one has made it worse. i just downgraded back to 2.6.32-21-generic, which works pretty well (though i did get a bunch of 'ath5k phy0: failed to wake up the MAC Chip' and 'ath5k phy0: can't reset hardware (-5)' after i let it run for a while).

i've been have other hardware problems with the ssd drive and the web cam, but i think this one is a driver issue given the chatter i've found on mail lists and bug reports. i know it's oss, so i can't really complain, but i'm really looking forward to this being resolved. i wonder if i should lag my kernel updates and/or switch to madwifi or something....

EDIT: don't think madwifi is the answer, since it looks like their dev effort got folded back into the ath5k. at least, i couldn't find a madwifi version that would both compile with my kernel and work with my pci-e wifi. but i seem to recall looking at the L0s, L1 ASPM stuff before with the unsupported jumbo problems, and i noticed that now all aspm support is turned off by default on my card. (dmesg even says the pci driver explicitly decides to do it.) this change was apparently made because junky old pcie wireless cards, like i have in my acer aspire, get the unsupported jumbos in L0s. but L1 is supposed to be a mandatory part of the standard (L0s is optional). so maybe something is trying to shove it into L1 without checking that that's enabled.

i tried turning L1 back on with the enable-aspm script (with root complex 00:1c.2 and endpoint 03:00.0) but it didn't seem to do anything. so i put the pcie_aspm=force kernel option into /boot/grub/grub.cfg, and that worked. in fact, it only enabled aspm for the atheros and its pci-e port, and only L1 for that. i guess the ath5k devs or somebody actually put code in to check if it can do it. so that's good; shouldn't get any of the jumbo problems, even with the =force.

so now we'll just see if this helps with the 'ath5k phy0: failed to wakeup the MAC Chip' and 'ath5k phy0:can't reset hardware (-5)' problems.

booting from sdhc on acer aspire one

2010-10-30T07:23:00.000-07:00

finally got my aa1 to boot with root on the sdhc card. these instructions are really good, and this reference is good, too. (though he is of the opinion that the nand flash write cycle limit is nothing to worry about, and i used to think that, too, but.... i wonder if my swap and ext4 fs had anything to do with the built-in ssd drive crashing so completely and painfully. this time i'll be going ext2 retro, just to be safe.)

had to boot from a usb drive (yeah, ssd is that hosed... can't even spare a few mb for a boot partition. heck, i can't even alter the partition table.) but it seems to be booting. now i wonder if i can hack this case open and put the usb drive in the unused bay...

flashed my aa1 bios

2010-10-28T01:05:00.000-07:00

my acer aspire one seemed to be having hardware issues, so i flashed the bios from 0.3310 10/06/2008, InsydeH2O Rev. 3.5, vga bios IntelV1585 to... the same thing. it's the latest version on acer's website, but i reflashed it anyway in case it had been munged somehow. i used the FLASHIT.EXE and zg5ia32.fs method that doesn't require any freedos junk. but it doesn't seem to make any difference.

mutual information for linearly dependent rvs

2010-10-27T07:09:00.000-07:00

i can't remember if i had recorded this anywhere, but i want to make sure i have it because it took me a little while to derive.

if x1 and x2 are both uniform random variables on [0,1] and y = a*x1 +(1-a)*x2, then the mutual information between x1 and y is ln(a/(1-a)) + (1-a)/(2*a) nats, where ln is the natural log. this is a useful result for testing mutual information estimators because it's on a bounded domain so might converge more quickly than functions of exponents.

the key to deriving this result is to realise that the marginal pdf for y is a trapezoid, and be careful about the log base when using the chain rule/integration by parts.

imperial websites

2010-10-27T01:42:00.000-07:00

www.imperialbaseball.co.uk
www.bright-futures.org.uk
www.icfinancesociety.com

cheap hotels in london

2010-10-13T02:26:00.000-07:00

http://golondon.about.com/od/londonhotels/tp/cheaphotels.htm

tune hotel is supposed to have really cheap rooms available, but i can find them on the online reservations.

PlayerPiano

2010-10-09T02:39:00.000-07:00

cool tool for showing 'interactive' presentations of python code. you can type random keys and it looks like you're typing in a fake python shell while it just runs your doctests (like an old-fashioned player piano with the punchcard scrolls). fast, easy, and frees you up to think about what you're saying instead of what you're typing.

just don't let anyone ask questions -- they might want you to change something!

impressive -- pdf slide presenter

2010-10-01T07:55:00.000-07:00

cool little presentation slide viewer, does transitions, overview screens, highlight boxes, and cursor-following spotlights. written in python with xpdf as the renderer. runs on windows, mac, linux. also allows for hyperlinks, so my page and overview links from beamer should still work. not sure about embedded movies, audio, or 3d objects, but... still cool.

optimization of conditional value-at-risk

2010-09-24T01:59:00.000-07:00

by rockafellar and uryasev

good description of cvar and why it's superior to var (value at risk) for portfolio optimization. they use a math technique sort of like a lagrange multiplier (not really, but works in a similar way) to transform the otherwise 2-step procedure for computing and optimizing conditional value at risk to a single stochastic optimization, convex (piecewise linear). it can then be solved with either linear programming or a nonsmooth optimizer.

interesting side note: they use a sobol sequence to get superior performance over straight-up mc. (the wikipedia article on sobol quasi-random sequences is quite dense and hard to understand, but here's a nice article that shows a monte-carlo integration example with a finance application, refs to niederreiter, sobol, and faure qrs. bottom line: niederreiter is (maybe) best.) also shows examples of portfolio optimization and optimal hedging with a butterfly spread.

certainly this is a better read than the highly mathematical paper that introduced cvar (convex measures of risk and trading constraints by f\"ollmer and schied). all i can really remember about that one is the point about cvar being convex (while var is not) and why that's important: diversification (mathematically, linearly interpolating between two portfolios) should not increase risk.

still, i think it's interesting that an industry-standard book on portfolio optimization like 'active portfolio management' by grinold and khan would brush off all risk metrics other than variance so lightly. maybe return distributions really are close enough to gaussian (with exceptions for derivatives, etc.) that it doesn't matter in practice, as they claim. at least anything that can compute those quantities should also give the variance for comparison.

call number:336.767 GRI ID:2406743083
Active portfolio management : a quantitative approach for providing
superior returns and controlling risk / Richard C. Grinold, Ronald N.
Kahn.
good book to have in the personal library, although there will be an updated version out later this year or next.

nice fashionista blog

2010-09-12T15:36:00.000-07:00

here's a blog that goes into some detail about why some types of clothing are higher quality than others, and how to take care of them. i don't normally go in for the gq stuff, and i currently can't afford anything in the blog, but someday i might have to. and i like the take permanentstyle has on fashion: permanent style, not semiannual closet churn.

pycxx vs. cython for python + c++

2010-09-03T04:20:00.000-07:00

cython apparently has some capability for wrapping c++, with some workarounds and remaining issues as of v0.13. pycxx (or cxx) takes on the interface question from the c++ side, giving python a more c++ friendly api that can call/access either from the other (though i think python is still intended to be in the driver's seat).

i guess the question is whether i want to use more c++ (with pycxx) or a pythonesque third language (cython pyx).

i've seen some references to using c++ with weave.accelerate, but not sure if that has any advantage over either of the other two.

quant job interview questions and answers

2010-09-01T09:21:00.000-07:00

nice book by mark joshi et al. recent, relevant, $35 print on demand from lulu. lots of advice and questions to prep for that interview and gauge how ready you are to apply. good refs, including the big one by author 1 ('the concepts and practice of mathematical finance'); xlw, the c++ plug in maker for excel api; and 'thinking in c++' (available free online from various mirrors).

beyond deterministic optimization

2010-08-27T03:17:00.000-07:00

i've come to realize that my multiobjective optimization problem is far from simple, with a lot of functions with noise-induced multimodality (FNIM) (numerical noise creating local minima). so i've been looking for previous work on how to deal with it. the fnim literature i've found seems to be very navel-gazing in that they look at how global minima bifurcate at the noise level is increased. but not as much on how to overcome the problem.

the term 'robust optimization' seems to be dominated by some guys from stanford, mit/singapore, and israel who have a particular framework that is not so useful to me right now. they assume the data going into the objective function are from a distribution that is an unknown member of a set of possible distros, and they want to protect against the worst-case from that set while strictly obeying the constraints. i think it's more of a mini-max problem, and what i need is to account for the uncertainty without needing to be so conservative. also, most, though not all, of the applications have a linear programming bend.

i'm not sure if stochastic optimization is what i need, but i'll check it out.

proposition 8 decision

2010-08-25T07:24:00.000-07:00

the emergency motion to stay judge walker's ruling is a well-written review and critique of the district court's prop 8 decision, especially the first 3 pages (even though it's a summary, it's a thorough summary). actually, the whole thing is pretty good although there are a couple of pages in the middle that are more dense.

trading hocus pocus

2010-08-20T07:50:00.000-07:00

i think i found the source for that reference to the use of fibonacci numbers in trading. there's a charlatan in peddling books and software based on numerological mysticism. i can't believe people are paying $500 a pop for his stupid software and >$150 for the book. really makes me wonder why i can't put together some of my code and sell it, if anyone is willing to buy that bovine byproduct. check out this gem from his website:

What's in the book: Fibonacci ratios as market tools.

For centuries, scientists, mathematicians, and artisans have discovered that the Fibonacci sequence of numbers is replicated throughout nature. This amazing number series defines the appearance of physical structures, as well as the PROGRESS OF CHANGE AND GROWTH governing dynamic structures and systems.

The Fibonacci sequence is found in the structures of trees, plants, and flowers. A regular sequence of Fibonacci numbers occurs in the relationship between a particular branch and the next branch. Fibonacci relationships also exist between leaves and stems. Radio signals generated by pulsars conform to Fibonacci numbers. Research reveals that many natural crystals contain the Fibonacci golden ratio, 1.618. There is even speculation by Dr. John Penrose of the Institute of Mathematics at Oxford, that the golden ratio may provide the crucial link between the sub-atomic and the supra-atomic world!

Human behavior has a dualistic nature. We think and act both as individuals and as part of a crowd. A crowd has its own energy and collective mind. It can be viewed as a dynamic system and, as such, it is governed by the same laws that exist throughout nature. Since a crowd is a dynamic system, and since financial and agricultural markets exhibit crowd behavior, it follows logically that Fibonacci relationships should be intrinsic to all liquid markets. These all important numbers and ratios indicate areas of contraction and expansion in price-wave movements. The concepts of Fibonacci support, resistance, and expansion in price function due to the principle inherent in the unfolding of all natural phenomena, including the behavior of crowds and the collective mind. The real difficulty is in the PROPER AND PRACTICAL APPLICATION of this natural phenomena to price movements. It is precisely that difficulty which DiNapoli Levels is designed to address. The text puts it all together in a unified trading approach you can act on!

This book also dispels myths about Fibonacci analysis and only covers those techniques that are useful and practical to employ in real life trading situations.

oh, good. it dispels all the myths and only covers the practical parts of this nonsense.

forecasting commodity markets: using technical, fundamental, and econometric analysis

2010-08-19T06:48:00.000-07:00

1995 book by julian roche is a bit disappointing and maybe just dated; it mentions that less work has been done with commodity price prediction than in other markets, and the fact that i have had such a hard time finding refs on it bears that out. refers briefly, though apparently seriously, to the idea of using numerical patterns from astrological sources and the fibonacci series... yikes! it also briefly mentions the possibility of using futures for prediction, but then cops out with some easily overcome excuses about how hard it might be. the types of technical analysis that it cites are really dirt dumb simple, though it does justify that by stating that those predictions are only good in the short term anyway, and you're not going to have enough data to fit a complex model reliably.

one very worthwhile part of the book is the table of forecasting approaches on pages 175-179 (heavily adapted from strategic business forecasting, pp 159-164) and their characteristics, including short-, medium-, and long-term accuracies. looks like trend analysis, seasonal adjustments, and box-jenkins type arima models are easy ways to the top of the game.

chaco, traits callbacks, and garbage collection

2010-08-14T10:01:00.001-07:00

i just learned something about traits (from enthought) the hard way.... if you allow the object that has a trait callback defined on it to drop all references to it (other than adding it to a container), the callback won't work. in my case, i had a lasso selection turned on for a chaco plot that was put into a HPlotContainer. it plotted just fine, interaction tools were fine.... but no reaction to the selection.

all i had to do was hang onto a reference to that traits object, and the selection worked. tough, tough bug to find. i feel cheated somehow.

notchup

2010-07-16T05:32:00.000-07:00

here's a recruiting/headhunter marketplace jobs site that i should get on my radar (or, rather, on theirs). they use independent talent scouts that scour sites like linkedin and personal contacts to find candidates that might already be employed ('passive' candidates) under the assumption that really good people are very rarely out of a job or trolling the job websites, looking for work. hang out my shingle and let them race to me with offers... now that's the way to get a job.

using googlecl with blogger from bash

2010-07-14T05:26:00.001-07:00

i think i finally found a good way to post from the cli.

google blogger post --title 'blog post title' --tags 'tag1,tag2' $'multiline content...'

the $ in front of the hard (single) quote lets bash escape single quotes inside that last string, but nothing else. so i can put all kinds of crazy chars in my multiline post, including a backslash-escaped single quote, with impunity. only problem is editing a previous line.... well, now i have an excuse for leaving the typos in.

stock index futures and price forecasting

2010-07-14T05:18:00.001-07:00

trying to figure out exactly what numbers financial commentators use to make headlines like, "futures point to higher open..." often they point to specific futures contracts, like SPc1, NDc1, or DJc1. the cme group equity futures quotes seem to be what people are referring to (or at least have equivalent price percent changes). unfortunately, the javascript in those pages that handle the quote updates prevents convenient scraping. options pages like these, otoh, are exceedingly easy to scrape as whitespace-delimited plain text. and with the eom (end of month, european-style) options, the price/probability is fairly simple.

if i really need to get at the futures contract prices, i could use bloomberg, which just has static html tables (with major us and world indices on one page).

back on the cme group site, i couldn't find quotes for options for oil, and even the time and price transactions list is another inaccessible js job. however, if i only care about end-of-day data, there are a lot in the ftp server (see the settle/ dir, for example) and other links from the volume:volume & open interest tab of one of the futures pages. daily settlement prices (near the bottom) has links for interest rates, equities, (agricultural) commodities, precious and industrial metals (in comex), oil and energies (in nymex and cme clearport clearing), and fx.

the volume by price data on that page might be particularly useful for building distributions. it combines total volume for the day at a each price, and the historical data available on the ftp server goes back a year and a half. the format is a bit obfuscated, but there are links to one-page format descriptions next to the data links.

one other thought about using derivatives for price forecasting/asset valuation: the price of the derivative depends not only on what people believe about the future price, but also on what everyone knows about the current price. so, if i had intraday data for the derivative price, i should subtract out the effect of the underlying price to get the future price. i wonder if ica on the log derivatives would achieve this... obvious maybe, but worth noting so i don't forget.

more python libs

2010-07-14T02:02:00.001-07:00

rthread exposes a threading interface on remote processes, if you are into that kind of thing. \
\
struct (standard library) interprets a string as a c structure.\
\
timeit (standard library) for quick and dirty timing of a command string, like the ipython magic and tic,toc in matlab.

python from excel

2010-07-13T04:54:00.001-07:00

xlrd, xlwt, and xlutils are fine for accessing excel files from python, but what about accessing python from excel (like calling out to vba hooks to define user functions)? pyxll seems to do what i would want (even claims numpy, ctypes, and cython support!), but the license seems very restrictive. if there is nothing better, it is probably the way to go. i might also check out the ironpython hook (http://www.ironpython.info/index.php/Interacting_with_Excel) and discoveryscript (http://www.xefion.com/discoveryscript.html). discoveryscript is free, but i think its license will also restrict redistribution (and the other products on the website are in the $500/licence range), so that might be problematic.

clyther

2010-07-05T04:22:00.001-07:00

tool for generating opencl code directly from python, as an alternative to cuda and pycuda. might be a little alpha at the moment, but i should keep tabs....

ubuntu one

2010-07-05T03:47:00.001-07:00

can't believe i haven't signed up for this yet. is there any downside? i guess i could use cryptoloop on all my machines that link to it if i'm paranoid about privacy. but that would prevent me from accessing files on the web.

googlecl

2010-07-05T03:42:00.001-07:00

google released command-line access. now i can post (like this one) from my tilda window.

tripadvisor for cheap hotels

2010-07-02T07:23:00.001-07:00

tripadvisor.co.uk has a nice feature -- it searches a number of other sites for cheap rates on the same hotel. helps prevent non-refundable clicker's remorse.

coroutines in python

2010-07-02T06:52:00.001-07:00

wow, am i behind the times or what? i didn't realize python 2.5 added real coroutine support by making yield an expression. now i can call the send method on a generator and it will pop in as the evaluated result of the yield.

also the webbrowser module can pop up a browser window (or tab). handy for showing docs.

i'd only just started using with. i need to stay up to speed on these things.

test coverage in python

2010-07-02T06:22:00.001-07:00

i think it's about time for me to abandon the half-home-baked test coverage tool i have been using in favor of coverage.py by ned batchelder. nice html reports (easy to lauch view, now that i have google-chrome) and i'm already using html reports generated by cython -a.

another possibility is canopy and instrumental. uses ast tools to analyse code; maybe useful for other things, too.

tahoe-lafs

2010-07-02T03:44:00.000-07:00

decentralized secure data store. if i ever need cloud storage, this might be a good alternative to cryptoloop.

3d pdf animation

2010-07-01T07:42:00.000-07:00

was looking into adding animation to my 3d pdf objects again. it's really hard to find any useful info on the newer prc format (the best i could find was a low level adobe tech support guy whining about the iso application taking a long time). u3d has at least some skin and bone based animation, beyond the rigid body + scale part-level animation that is useless to me (every reference to key frame animation, afaik, refers to this). but i'm not sure if acrobat or the adobe reader support that part of the u3d ecma standard. this page (google cache) from right hemisphere seems to say that it doesn't, but that might refer just to their deep exploration code. at any rate, i think they were one of the founding parties to the standard, so it's not promising if they never bothered. somewhere i found an unreferenced statement that everything supports ecma 1st edition, but acroread 7 doesn't support ecma 3rd edition.

however, the JavaScript for Acrobat 3D Annotations API Reference doc on the adobe website describes a Bone object and specifically says it 'is usually moved over time to create animated characters.' Mesh objects contain geometry, but nothing in the api doc looks like it could be actual vertex positions, unless they're lurking in the Node.metadataString. i can find no additional info on Bone.

the U3D Supported Elements doc from the 3d dev center states that skeleton descriptions are 'parsed and unused (that is, no skeletal animation, skinning, or bones)'

also, opacity and material color animation are not supported, and i'm pretty sure that's a basic u3d limitation.

it's hard to find much info about the newer prc format. i don't see anything in the adobe spec docs that obviously provides for it (though not much anything obvious in the specs... quite sparse). i think the best i can do right now is look at the asymptote code (there was a passing reference to animating the prc in one of their journal articles as a todo item; alas, the only animation they show in that article is an ordinary movie). rumor has it that adobe turned the prc implementation over to a third party, bailing again on a drive into the 3d market, so i'm not sure prc has a future any brighter than u3d, which they have said (fwiw) they will always support.

so the only way out i can see right now is to create multiple meshes, one for each time step, and use javascript to cycle the visibility toggles. i guess the only real wasted space comes from the redundant connectivity, and that might be manageable if i continue to use only the exterior surfaces. and it would allow me to circumvent the u3d material color animation limitation. a rough estimate is 6.5 bytes/exterior node for u3d file size. actually, maybe that's not too bad... a 5 second, 20 fps animation with 8000 exterior nodes would fit in under 5MB. not too big for a short report or presentation. (maybe ~10 fps is a more realistic limit on what acroread can achieve.)

a little playing around with the quality options: diffuse color quality moved down from 300 to 100, -dcq 100, make practically no difference in file size. specular color quality, -spq 100, and animation quality 1000 -> 100, -aq 100, made zero difference since i'm not using either. geometry default quality down from 300 to 100, -gq 100, makes no difference, as does texture quality 75 -> 50, -tq 50. texture coordinate quality 300 to 100, -tcq 100, and normal quality 300 to 100, -nq 100, make zero difference. the one thing that does change the file size significantly is position quality. i had been using -pq 500, and -pq 100 decreases the size by about 7-8%. and maybe decreasing the palette from 256 to, say, 16 colors would help, though it would surely be less than a 10% difference. so i don't think there's any way i could squeeze it down more than 20-25%.

pypy 1.3

2010-07-01T01:30:00.000-07:00

pypy now has stackless, jit, sandbox, and low-memory versions out in 1.3. and support for c extension modules is coming along. and, as part of one of this year's gsoc projects, they are working on adding numpy to pypy. cool!

another python debugger

2010-06-29T09:22:00.000-07:00

pudb. text-based but looks easy to use.

a bit less temporary /tmp

2010-06-29T08:54:00.000-07:00

hat-tip to pycuda, i got my ubuntu machine to relax about wiping my temp dir on every reboot:

On Debian (and possibly Ubuntu?), edit the file /etc/default/rcS and change

TMPTIME=0

to the number of days that you'd like to keep files in /tmp around. "30" works for me.

now i won't be so paranoid about installing kernel updates right away.

gpu programming with python

2010-06-29T08:10:00.000-07:00

theano seems to have an advanced api, but it might be too advanced since it uses its own optimization and other magic goodies, and it seems so focused on its own objects that it's like a metaprogramming language. still, dangerously interesting. uses cuda, so nvidia only. here's some free advice from their tutorial:

Only computations with float32 data-type can be accelerated. Better support for float64 is expected in upcoming hardware but float64 computations are still relatively slow (Jan 2010).

Matrix multiplication, convolution, and large element-wise operations can be accelerated a lot (5-50x) when arguments are large enough to keep 30 processors busy.

Indexing, dimension-shuffling and constant-time reshaping will be equally fast on GPU as on CPU.

Summation over rows/columns of tensors can be a little slower on the GPU than on the CPU

Copying of large quantities of data to and from a device is relatively slow, and often cancels most of the advantage of one or two accelerated functions on that data. Getting GPU performance largely hinges on making data transfer to the device pay off.

the same website has a link to cudamat, which might be a more cooperative if lower-level way to go. target seems to be basic matrix and element-wise ops. actively developed.

pystream was developed as another cuda wrapper until about mid 2008, then abandoned when the company went off to develop gpulib, a cuda api for idl and (?) matlab.

pycuda handles the background stuff, but you still have to feed it c code (though there are tools for run-time code generation). looks like it's actively developed, though, with an impressive list of users, and like the others it does play nicely with numpy arrays. i think this is the place to start.

pygpu uses pycg and pyglew to generate cg code directly from python. so you write python and it will run on nvidia or ati hardware, under both linux and windows. unfortunately, neither the homepage nor the google code page show any signs of activity in the last few years. pycg (developed by the same guy) seems to have trickled off in late 2007, though ubuntu packages were uploaded to launchpad just a year ago. too bad, this looked like it might have been a good one.

the gpu stuff on scikits oddly seems intended for actual graphics stuff.

pyopencl ? here's a faq page contrasting cuda and opencl.

pycublas maybe just does matrix mult.

bond spread data

2010-06-29T01:54:00.000-07:00

fred (research.stlouisfed.org) has a lot of economic indicator historical data, including moody's bond yields, but only for aaa and baa. i'd like to find something for more junky bonds....

moodys.com certainly would have these data, along with others i'd like to see. but registration is required and i don't know if that stuff is available for free.

Best Practices in Estimating the Cost of Capital: Survey and Synthesis

2010-06-28T09:13:00.000-07:00

Robert F. Bruner, Kenneth M. Eades, Robert S. Harris, and Robert C. Higgins

nice peek at the popularity of various financial analysis techniques. just a bit dated now, as it came out in 1998, but still worth a look.

tea party != conservative populism

2010-06-28T02:29:00.000-07:00

jeffrey friedman makes some good points about why the tea party leaders should not assume (or maybe even aim for) a conservative populism. here's the most sobering stat for me:

An April Rasmussen survey found that only 60 percent of Americans now believe that capitalism is better than socialism. Among those under 30, socialism and capitalism are nearly tied at 33 percent and 37 percent.

oil spill -> green pork

2010-06-28T02:24:00.000-07:00

charles krauthammer lets fly a critique of obama's face-the-nation. seems a bit harsh for ck, but he does an effective job of deconstructing the basic argument.

obama vs. science

2010-06-28T02:19:00.000-07:00

jonah goldberg makes an interesting connection between the anti-science claims against bush and the drilling moratorium. a minor point, perhaps, but one to put on the record.

portfolioscience

2010-06-28T02:05:00.000-07:00

here's a company that sells software for portfolio optimization and risk analysis. maybe i should fill out their form and check out the demo some time, just to see how they do it and which metrics seem to be featured. in particular i'm curious about the riskapi efficient frontier optimizer.

aorda is another one, started circa 2006 by a ufl prof who was one of the early proponents of cvar. free download for crippleware, but have to register first. pay versions are very highly priced: commercial license is $10k/year.

why defend bp?

2010-06-28T01:55:00.000-07:00

why are some on the right defending bp? rich lowry has a great editorial up on nro making the point that we don't have a dog is this fight while some republicans are determined to get bitten.

15.535

2010-06-23T02:50:00.000-07:00

looking at the assignments and exams, i think i understand the concepts just fine, although the intricacies of interpreting how management might be manipulating their accounting numbers are not easy.

http://mit.edu/wysockip/www has useful stuff but doesn't have all the stuff from class anymore

peg ratios, often cited

cash flows over firm's life cycle

trend analysis: cfo vs ebx

red flags: growing discrepancy between net income and cash flows

undervalued liabilities, overcapitalization

investment activity

key: proceeds from exercise of stock options. good?

firm type: growth options vs assests in place

tech, growth: not much depreciation, financing primarily related to equity

airlines: cfo large compared to net income, even in loss years; large depreciation, investing; debt financing

retailers: walmart has large difference between cfo and net income

problems with residual income valuation

p/e or m/b with real options?

abnormal earnings with dcf (discrete cash flows)

what do analysts use? refs asquith et al., 2001

earnings multiple 99%

p/e 97

relative p/e 35

revenue multiple 15

price-to-book 25

cf multiple 13

dcf 13

eva 2

'model' 4

estimate price multiples for comparable firms avg/median/etc. why not use distro?

if current earnings are not good prediction for future: forward p/e or pro forma earnings (remove non-recurring) or price to operating cash flow

other p/e: peg, p/cf, levered, (debt+equity)/ebitda

m/b market to book

stock screener links

profitability: roa (return on assets)

roa decomposed into profit margin and asset turnover

roe (return on common equity)

roe decomposed into profit margin, turnover, leverage

short term liquidity

current ratio = current assets/current liabilities: short-term debt paying ability

quick ratio = (current assets-inventory)/current liabilities: acid test ratio

long-term solvency

long term debt ratio = long term debt/(long term debt+shareholder's equity)

d/e = long term debt/shareholders' equity

total liabilities/total assets

forecast eps goes down the last 6 months before release due to expectations management

detecting earnings management

ratio of volatility (stddev/mean) of accrual income measures to underlying volatility of sales and cfo

risk assessment

turnover: accounts receivable turnover, inventory turnover, fixed asset turnover, accounts payable turnover, days payable outstanding

short-term liquidity: current ratio, quick ratio (acid test), operating cash flow to current liabilities

long-term solvency (maybe a good way to value bonds?): debt/equity, long-term debt ratio (simple function of d/e), liabilities/assets

interest coverage ratio, in terms of both income and expenses or cash flow

refs modigliani-miller theorem without explaining: debt and equity financing are equivalent

absolute metrics: interest coverage, current ratio

cost of capital

equity cost of capital (discount rate)

capm: estimate beta (key issue) period typically 5 years; bloomberg, analysts, yahoo finance, etc

http://research.stlouisfed.org/fred/data/irates.html for risk-free rate and other data

fama-french 3-factor model extends capm with size, b/m (higher b/m->higher returns)

http://mba.tuck.dartmouth.edu/pages/faculty/ken.french/data_library.html for rates, other data

long run averages: r_m-r_f (market-riskfree) 7.95% per year, r_smb (size premium) 3.32%, r_hml 5.05%

international

segmented/integrated capm: bekaert and harvey 1995

world capm holds if country stock market is integrated: http://www.msci.com/equity/index.html

ow, use r_country

'institutional investor' magazine ranks country credit risk 0-100

impressive fit to data: r_country = alpha + beta*rank

expected return depends on systematic risk

alpha = abnormal return = actual return - capm, for example

multiples valuation key assumption: earnings and book equity are comparable

drift strategies

returns over last 6-12 months predict next 6-12 months

post earnings announcement drift from under-reaction to news

red flag: again, gap between reported income and cfo

quality of earnings ratio: (earnings-cfo)/avg total assets

'widely accepted' evidence on fundamental trading strategies

e/p, b/m, cf/p: high->high future abnormal stock returns

var(cf)/p: high->low future abnormal stock returns

v/p (firm value from abnormal earnings model/price): high->low returns

short term reversal: high return this month->low next month

medium term momentum: high return past 6-12 months->high return next 6-12

accrual anomaly: high accounting accruals this quarter->low returns next quarter and beyond

bankruptcy detection

http://www.ibbotson.com/content/cc_1v11.asp cost of capital:$15/beta

altman z-score fit from manufacturing firm data

linear function of ratios

moody's, s&p use similar models to z-score to rate corp bonds

http://riskcalc.moodysrms.com/us/research/crm/45768.pdf

http://riskcalc.moodysrms.com/us/research/defrate.asp

mergers and acquisitions

'old' purchase method: goodwill asset created and amortized over 40 years

pooling of interests no longer permitted for valuing

employee stock options

off balance sheet activities

enron background

pension plans

defined benefit plans cause accounting problems

international financial analysis

insider (code law) codified system

close interplay gov, banks, unions, big firms

continental euro, japan

less public disclosure

outsider (common law)

us, uk, english-speaking

us vs uk differences

sarbanes-oxley and review

sarbox 2002

identify comparable firms

multex (?) via yahoo for quick industry benchmarks

will change: accounting rules, tech, market integration, contracting methods

won't change: thought process, economics

guild wars

2010-06-23T02:46:00.000-07:00

skills at the end of nightfall:

n: meekness, well of dark

p: harrier's toss, never surrender, stand your ground

memory profiling with python

2010-06-16T07:01:00.000-07:00

tough to find good memory profiling for python. heapy-pe and the other (didn't bother to remember the name; pysizer?) turned out to be no good to me with numpy arrays which (surprise!) tend to be the biggest data structures i deal with. here a couple of others to try some time:

meliae is new and more cli-oriented, but looks easy enough to try (and script).

dowser spawned off of cherrypy, but i think it works for any python code with the web server as sort of a gui (i think unlike dozer, which targets wsgi apps. or maybe dozer is just a wsgi version of the 'gui'?).

here's an example of objgraph to analyse memory usage.

i think these are more garbage collector approaches, rather than hook-and-trace, so maybe more likely to work with libs like numpy.

r-cran-fimport

2010-06-15T08:10:00.000-07:00

ubuntu has a package for downloading free econometrics data: provides import function to access (free) data from Economagic, the US Federal Reserve, Forecasts.Org, Yahoo and other web sources. worth a look to see some sources that would be good to look at. the group of people who wrote this also have a link to a brief discussion of portfolio risk surfaces over the convex hull of achievable sets. interesting... i was thinking something along the same lines, and it's a little gratifying that working pros seem already to be doing something similar.

get method for fun and profit

2010-06-15T06:21:00.001-07:00

just learned (or maybe relearned) something cool about python: the __get__ special method gets called when an instance is accessed as an attribute of another instance. not only are there potential uses for this, it also holds the key to understanding the 'self' and 'class' special arg in methods. this is something that confused me a couple of times before, such as passing references to instance vs. class methods from outside the class to be used inside the instance.

so, for example, i could allow instances of one of my classes to know how and where it's getting passed around, and something about the context when something is asked of it. maybe a quick and dirty memory leak tracker, when i know beforehand which objects are the big boys but i don't know who's pointing at them.

or maybe a little internal usage auditor, when i'm considering the impact of a refactor.

valuation books

2010-06-14T03:14:00.000-07:00

couple of books recommended by people in the valuation business. one comment about duffie is that he was kind of a disappointment as a consultant, since he likes to stay more in the theoretical than the practical. not sure if this is the best book from him, but it's fairly recent. (search for 'dynamic')

Investment valuation : tools and techniques for determining the value of any asset

Damodaran, Aswath.

interesting that he says most analysis/justification for valuation is on discounted cash flows (as it seems to be in the book), but most valuation in practice is with ratios in relative valuation. contingent claim valuation is a more recent perspective, looking at opportunities available to a firm and pricing them like options. i was disappointed at how little there is on bond and commodity valuation, especially given the promise in the subtitle. interesting chapter on evidence of market efficiency.

Credit risk : pricing, management and measurement

Duffie, Darrell.

financial statement analysis and security valuation, 3rd ed

stephen h. penman

658.15

more on valuation, including slightly less than simple forecasting and detecting financial statement manipulation

dynamics of markets: econophysics and finance

joseph l. mccauley

658.15:519.217

empirical refutation of common modeling assumptions

value at risk: the new benchmark for managing financial risk, 3rd ed

philippe jorion

658.155

different types of risk, some 'industry-standard' real-life practical-experience rules of thumb

mplayer dump

2010-06-08T05:44:00.000-07:00

tried to record a lecture in realmedia format with mplayer, but it keeps dying in the middle. cache seems to help it get farther, but it still chokes. maybe i just need to give it a _big_ cache, or use the -cache-min option:

mplayer -audiofile-cache 8192 -cache 8192 -dumpstream -dumpfile out.rm rtsp://etc

encrypted pdf files

2010-06-08T04:06:00.000-07:00

managed to get a pdf file with restricted permissions using pdftk (pdf toolkit). the man page was a bit unclear on this, since if you just put in an owner password it will not actually restrict the file and no passwd is requested (or needed) to open it. this will do the trick, with all restrictions:

pdftk in.pdf output out.pdf user_pw foo

or, to make 2 levels of access:

pdftk in.pdf output out.pdf owner_pw baz user_pw foo

now the restrictions will be in place if you put in foo as the passwd, but they will not if you put in baz.

redirecting stdin, stdout, stderr from python

2010-06-08T03:33:00.000-07:00

saw a nice comment in a post on python daemon forks that explains how to redirect std* pipes, even when they are accessed from c. i've been frustrated trying to figure that out before.

More reliable i/o stream redirection. Just reassigning to the sys streams is not 100% effective if you are importing modules that write to stdin and stdout from C code. Perhaps the modules shouldn't do that, but this code will make sure that all stdin and stdout will go where you expect it to.

import os, sys

out_log = file('/out/log/file/name', 'a+')
err_log = file('/err/log/file/name', 'a+', 0)
dev_null = file('/dev/null', 'r')
sys.stdout.flush()

sys.stderr.flush()

os.dup2(out_log.fileno(), sys.stdout.fileno())

os.dup2(err_log.fileno(), sys.stderr.fileno())
os.dup2(dev_null.fileno(), sys.stdin.fileno())

(and another poster suggests closing sys.std* before duping.) cool. i need to remember this next time i wrap somebody's code that thinks it's a good idea to barf on the terminal without a --quiet option.

also, the demon implementation looks pretty clean, with extra tidbits sprinkled into the comments, and the author explains the reasons for doing things. i don't think i will need the double fork for my udev script, but the first fork will be necessary.

printing presentations from evince

2010-06-08T02:40:00.001-07:00

sometimes i need to print presentation slides with just big text, and i want to do it n-up so it doesn't waste paper. i've found that the best way to do it with evince is to set 'layout' to two-sided: long edge (standard), pages per side: 6, page ordering: left to right, top to bottom, and 'paper' orientation: landscape. 12 slides per sheet and still very readable.

also, it often helps to concatenate pdfs together so the nup has multiple presentations without a page break:

pdftk in1.pdf in2.pdf in3.pdf cat output out.pdf

ocw

2010-06-07T08:57:00.000-07:00

mit's opencourseware has quite a few classes up from course 15 (sloan school of management). worth a look.

15.010, maybe 15.060, 15.062, maybe 15.063, maybe 15.223/15.224, 15.351, 15.352, 15.356, 15.358, 15.369, maybe 15.391, 15.394, 15.402, 15.414, 15.431, 15.433, 15.616, 15.617, 15.628, 15.963, and 15.997 look interesting.

15.501, 15.511, 15.514, 15.515, 15.516, 15.518, 15.521, and 15.535 look like particularly interesting classes on finance.

i should also peek at 15.070, 15.075, 15.081, 15.084J, 15.085J, 15.093, 15.094J, 15.098,15.099, just to make sure i haven't missed any of the math there.

udev rules

2010-06-04T02:09:00.000-07:00

finally got a udev rule in /etc/udev/rules.d/ so it will run my script on a hotplug event. first i had to run udevadm info --attribute-walk --name /dev/sdc1 to dump out a list of potential attributes to search. since this spits out info on the whole device chain, i found out that the section to look at was /devices/pci0000:00/0000:00:1d.7/usb1/1-7/1-7:1.0 (i tried matching on a lower level, but my script was executed twice with different env vars.) then i put a rule line into /etc/udev/rules.d/91-myhotplug.rules (91 because i want it to run after everything else) like this:

ACTION=="add", SUBSYSTEM=="usb", ATTRS{modalias}=="usb:v0...50", RUN+="/home/user/svn/bin/hotplugscript"

had to put in a sleep 6 before doing anything in the newly mounted fs because it seems to take about 5 s to settle down. (probably should loop-and-test in production-level code.) oh, and one other handy thing i googled up: use blockdev --flushbufs /dev/sdc1, for example, to flush one block device. sync just flushes them all. i have the whole bash script in brackets with a trailing & to make it fork off and return immediately; that's important because otherwise i found it hangs or dies at the sleep and nothing after that gets executed. and i did find that a umount after business is done in the udev script causes no problems.

financial accounting info on google finance

2010-06-03T07:53:00.001-07:00

google finance has easy access to the balance sheet, cash flow statement, and income statement of publicly traded companies for the last 5 quarters and 4 years, ready to scrape from an html table in a standard format, all in one page. for example, all this info for cisco is at

http://www.google.com/finance?q=NYSE:C&fstype=ii

probably would be a bit more reliable if there were one place i could grab xbrl files, but right now i can only find links to individual companies' websites. not very scriptable.

also, the summary page (eg, http://www.google.com/finance?q=NYSE:C) has a 'key stats and ratios' sidebar. i wonder if any of these are really metrics that the pros use. yahoo has insider and institutional trades at http://finance.yahoo.com/q/it?s=C although dailyfinance shows total holdings for insider trading at http://www.dailyfinance.com/company/citigroup-incorporated/c/nys/insider-transactions

yahoo has convenient pages for analyst estimates, showing average, high, and low earnings estimates from a number of analysts for current and next quarters and years; estimate and actual eps histories; eps estimate recent revisions; and estimates for revenue and growth for the company, industry, sector, and s&p 500 for comparison. for example, dell's page is at http://finance.yahoo.com/q/ae?s=dell pretty good start for fundamentals valuation. the analyst opinion page at http://finance.yahoo.com/q/ao?s=DELL also has price targets and quantitative recommendation poll numbers (in the chart near the bottom; i think the 'mean recommendation' listed above is is basically the dot product of the votes in each category with the arange(0,5) divided by the total votes). not sure i would completely base my decision making on free advice, but it could be worthwhile for a sanity check right before clicking the 'trade' button. looks like it only has these analyst opinions for individual companies; no etfs, etc. but the etfs have holdings info, so i could go from there to get some projections on stocks.

now i need good ways to project near/medium term trends in the bonds, precious metals, commodities, and fx markets.

guild wars on linux

2010-06-03T05:44:00.000-07:00

cool. got guild wars running with wine on linux. the install and running the game was no problem, but i did need a couple of tweaks to get performance up to a playable level.

this page on the wiki was important, as it pointed out things like the emulated hardware acceleration for the alsa audio. (winecfg) i found that the pixel shader was fine to leave on, even though my hardware ain't that great. the wiki page says to use the pbuffer for osr on low-grade hardware, which i guess means to use a pixel buffer for off-screen rendering. i couldn't see how to do that, but fortunately it wasn't needed. also, i had to check the 'allow the window manager to control the windows' or else my keyboard wouldn't work and i couldn't even log in. i had previously changed the windows version to 2000 and installed a few winetricks, but i don't think those are too important.

some of the other changes i'm not sure if they are essential in isolation, but they're part of the final solution: (regedit) UseGLSL disabled, Multisampling enabled, anti-aliasing 2x in game. OffscreenRenderingMode to backbuffer seemed to help a lot with freezing graphics when moving (loading new shaders), even though the wiki page says to do that when the screen turns black. also, setting the game to fullscreen with the same resolution as my linux xwindows (1280x1024, like i said my hardware ain't that great) seemed to get better performance than any other resolution, even than lower ones. (it also avoids the annoying problem of panning around the virtual screen when i move the mouse to the bottom or right.) i also used WINEDEBUG=-all to turn off the errors and warnings, but i doubt that matters much. and i didn't bother to use the -windowed flag.

EDIT: based on the wine db, looks like pbuffer is a registry setting for OffscreenRenderingMode. a few other suggested settings, but i don't think i'll need them.

giffgaff

2010-06-02T03:31:00.000-07:00

mobile phone company on the 02 network. maybe i need to look into this...

arizona's immigration law

2010-06-02T03:13:00.000-07:00

william bennett has a good article up with some hard stats on the effects of illegal immigration crime in arizona. worth a read.

global warming

2010-06-01T08:47:00.000-07:00

boy, do i get tired of hearing this: you get ten hot days in a row, and when the eleventh day is cold someone says, 'pfth! global warming....' then when there are ten cold days in a row, other people shake their heads on the next hot day and curse the knuckle-draggers who could doubt the truth about climate change in the face of such incontrovertible proof. if all those people realized how much they have in common, maybe they would just hang out with each other and leave me alone.

day to day weather patterns don't prove _anything_, about _anything_.

(disclaimer: i'm not trained in climate or geo sciences. i do have a background in engineering, and in particular i have experience with large-scale multiphysics modeling and computer simulation.)

atmospheric carbon dioxide levels have shot up dramatically in the last couple hundred years, to levels significantly higher than anything the earth has seen for at least hundreds of thousands to millions of years. a very large fraction of that c02 is almost certainly due to the fact that humans have been digging carbon deep out of the ground and burning it. co2, like other translucent media, scatters radiation which causes a greenhouse effect. the average global temperature of the earth has increased over the last couple of hundred years by about .5 to 1 kelvin. i don't think there are any (informed) people who seriously dispute any of these things.

the problem is how to connect and interpret them in the context of other physical processes. how much of that ~.7 k warming is anthropogenic? what additional changes would be caused by various human activity levels in the future? what effects will a higher average global temperature have? if you can come close to answering any of these questions, you deserve a nobel prize. heck, you deserve obama's, gore's, carter's, and arafat's, and probably a few others.

i think a lot of the confusion comes from a basic misunderstanding about what science is, what it's for, and how it works. for example, sometimes people hurl the label of 'theory', as if it somehow discredits or diminishes an idea. (you hear that one a lot in the evolution worm can, too.) yeah, guess what? general relativity is just a theory, too, but most of us walk around without worrying about suddenly flying off into space. or sometimes you hear talk of a 'consensus' among scientists, as if that proved diddly squat about the science itself. maybe i'll post more on this some time, but i wonder if the root of these problems is that, for much of the population, the last bit of science education they get is early in high school. and the people teaching those classes also had their last bit of science education early in high school. the connection to the research frontier, where you have to have a grounding in the basic epistemology, is very tenuous.

john derbyshire wrote an article last year in response to the climategate blip on the news cycle, in characteristically disgruntled style. i don't always agree with the derb, but i thought this one was well-written, even-handed, and insightful.

maybe a lot of people feel a need to have one answer, yes or no, delivered to their doorstep before morning coffee so they can just move on in the comfort of knowing that the matter has been settled. sorry folks, ain't gonna happen. this is a complex subject, with mountains of uncertainty in the model form, the inputs climate scientists feed into their simulations, and the experimental data collected to validate them. it might be that we will not know these answers with high confidence for a long time. we might never know. it might not be knowable. even if we did know, it might already be too late to do anything about it. i see people on both sides of the politics desperate to claim science as a bludgeon to use on the other side. well, maybe science just won't cooperate with either of you. feel better now?

i don't think this will be close to settled for a long time, but here's a clue to scientific status: you'll never hear a real scientist argue that his or her current research is a settled question. (duh! why would they be working on it?) but there are some basics that we can be fairly confident about.

problem is, we have to decide what we're doing tomorrow, today. so here's my prescription: keep studying it. meanwhile, make the cheap changes that reduce co2, especially in favor of energy technology that concentrates pollution rather than dilutes it (nuclear, anyone?). don't use hysteria to justify marxist central planning or costly, widespread upheaval. don't think you can shoot down a scientific hypothesis without proposing a better alternative. forget about the carbon-credit trees and cow farts. that's just carbon swishing around the surface of the earth, and it geologic time it doesn't stay in one place for very long unless it's well buried in the crust.

and don't talk to me about the weather.

bullet physics engine

2010-06-01T03:52:00.000-07:00

bullet is a foss physics engine (mostly rigid body?) that comes in third against nvidia physx and havok among game designers (open dynamics engine is a more distant fourth). also used for movie effects on major releases. plugin for blender, as well as some other closed-source modellers. just saw some chatter on cython that a couple of different people are wrapping it for a python api.