https://travis-ci.org/
very cool public build server for open source projects. you can set it up to run the test suite on commits, and it takes its config out of the repo itself.
there's a travis pro for continuous integration on private repos, but it looks like it's not live yet. (?)
Sunday, April 21, 2013
Saturday, April 20, 2013
pythonanywhere
https://www.pythonanywhere.com/
hosted python environments, for a webapp for your site, an on-the-go interpreter session, or... whatever else you can think of. i think they also give you bash and latex, maybe other goodies. it does have numpy, scipy, sympy, matplotlib, cython. it doen't have enthought, numba. the (very impressive) list of packages it has is here:
https://www.pythonanywhere.com/batteries_included/
free for basic account, $5/mo for 'hacker' account, and 12$/mo for more serious hosting. sounds very reasonable, considering you get access to their hardware and no wasting time on sysadmin maintenance.
demo console here:
http://blog.pythonanywhere.com/4/
run selenium test example here:
http://blog.pythonanywhere.com/3/
hosted python environments, for a webapp for your site, an on-the-go interpreter session, or... whatever else you can think of. i think they also give you bash and latex, maybe other goodies. it does have numpy, scipy, sympy, matplotlib, cython. it doen't have enthought, numba. the (very impressive) list of packages it has is here:
https://www.pythonanywhere.com/batteries_included/
free for basic account, $5/mo for 'hacker' account, and 12$/mo for more serious hosting. sounds very reasonable, considering you get access to their hardware and no wasting time on sysadmin maintenance.
demo console here:
http://blog.pythonanywhere.com/4/
run selenium test example here:
http://blog.pythonanywhere.com/3/
logging
http://vimeo.com/couchmode/pydata/videos/sort:date/63377197
lots of good advice on writing reliable, maintainable infrastructure code from a guy who now works at disney supporting social media games. in particular, some points on logging:
appropriate levels:
LOG for start and end of each major section of work
WARN for input data errors and environment errors
ERROR for code problems
DIE for unrecoverable errors, after a short delay
say what happened, what was expected, and what to do/whom to contact
lots of good advice on writing reliable, maintainable infrastructure code from a guy who now works at disney supporting social media games. in particular, some points on logging:
appropriate levels:
LOG for start and end of each major section of work
WARN for input data errors and environment errors
ERROR for code problems
DIE for unrecoverable errors, after a short delay
say what happened, what was expected, and what to do/whom to contact
vagrant
http://www.vagrantup.com/
vagrant vm looks like very interesting open source (MIT) tool for setting up clean, uniform development environments. looks like it's written in ruby.
vagrant vm looks like very interesting open source (MIT) tool for setting up clean, uniform development environments. looks like it's written in ruby.
evernote
http://evernote.com/
here's something i need to check out when i have more time: evernote. looks like you can put notes, copies of web pages, scanned documents, audio and photos, etc. in one place and have it synced across your devices. recommended by one of the developers of google glass.
lots of video tutorials with more info:
https://evernote.com/video/
here's something i need to check out when i have more time: evernote. looks like you can put notes, copies of web pages, scanned documents, audio and photos, etc. in one place and have it synced across your devices. recommended by one of the developers of google glass.
lots of video tutorials with more info:
https://evernote.com/video/
Wednesday, April 17, 2013
rapidquant
http://www.rapidquant.com/
new startup by some former quants and developers of pandas. looks like it might have some overlap with quantopian, but i'm not sure what their business model is. part open source, part proprietary.
new startup by some former quants and developers of pandas. looks like it might have some overlap with quantopian, but i'm not sure what their business model is. part open source, part proprietary.
jake vanderplas blog
http://jakevdp.github.io/
blog with lots of interesting python examples and demos. not just code, though; the guy seems plugged in to big-picture developments.
blog with lots of interesting python examples and demos. not just code, though; the guy seems plugged in to big-picture developments.
Saturday, April 13, 2013
python presentation videos
http://vimeo.com/pydata/videos/page:1/sort:date
http://pyvideo.org/category
no need to attend pycon or pydata, or no need to take notes if you do.
http://pyvideo.org/category
no need to attend pycon or pydata, or no need to take notes if you do.
time management quotes
don't lose interest when it's easy; don't lose hope when it's hard. just the right amount of stress and difficulty is what makes us improve.
http://www.youtube.com/watch?v=oTugjssqOT0
randy pausch
most important point: in 7 habits 4 task quadrants, do 1. important/urgent, 2. important/not urgent, 3. unimportant/urgent, 4. unimportant/not urgent. that way, 2s don't become 1s and you don't get stressed.
many people do unimportant/urgent second, but that's wrong.
being successful doesn't make you manage your time well.
managing your time well makes you successful.
(me) time are money are related; so are time and stress.
don't do the wrong things beautifully; do the right things adequately.
why am i doing this? what will happen if i don't?
good judgement comes from experience.
experience comes from bad judgement.
there are no shortcuts to good judgement.
you can't change the plan unless you have one.
do the ugliest thing first.
if you have to eat a frog, don't look at it first.
if you have to eat 3 frogs, start with the big one.
only have 1 document on your desk at a time; that way you fight them one-on-one
touch each piece of paper (or email) only once.
use a speaker phone when you're, and tell them politely how long you've been on hold.
stand while talking on the phone.
start with giving an agenda for the call.
have something on your desk that you'd like to get back to.
'i'd love to keep talking but i have ___ waiting.'
group your calls and make them right before lunch and quitting time. then they will have something to get back to.
use a phone headset.
keep kleenex and thank you notes on your desk.
people will remember you if you send them a thank you note.
keep a paper recycle bin near your desk that is emptied infrequently. then you will not hesitate to throw it out since you have a grace period to get it back.
make your office comfortable for you, and optionally comfortable for others.
keep folding chairs to the side.
you don't 'find' time for important things. you 'make' time by neglecting something else. consider opportunity cost and learn to say gentle 'no's.
gentle no:
'i'm strapped, but i don't want you to be in a bind. i'll be your fallback if you can't find someone else.' then you will see how that person treats you, based on whether or not they keep looking.
find your creative time. defend it ruthlessly and spend it alone.
find your dead time. schedule meetings, phone calls, and exercise (when you don't need to be at your mental best.)
an interruption of 6-9 minutes takes 4-5 minutes to recover from.
when people interrupt you, say 'i'm in the middle of something' or 'i only have 5 minutes'. then they will want to handle it quickly and you can leave gracefully later.
if someone won't leave your office, go to the door, complement and thank them, and shake their hand. if they still won't leave, leave.
time journal
monitor yourself in 15min intervals, update every 30min, for 3days to 2weeks.
what am i doing that doesn't need to be done, what can i delegate to someone else, where am i wasting time, and where am i wasting other people's time?
time management is collaborative, not selfish; help other people be more efficient
remember the point is not to do more, it's to go home on time.
effective vs. efficient
do what will work better, even if it takes more time.
we don't procrastinate from laziness. identify why you are not enthusiastic. are you afraid of failure? are you avoiding the decision to prioritize?
doing things at the last minute is much more expensive than just before the last minute.
make up a fake deadline and pretend it's real.
sometimes you just have to ask and wonderful things happen.
delegation
don't treat it like dumping.
give responsibility and authority.
keep the ugliest part of the job yourself, so you and they know you are not dumping.
treat your people well.
you cannot be vague if you want something to get done. specific thing to do, specific date _and_ time. specific penalty or reward (for them).
challenge people; they usually yearn for an opportunity.
communication has to be clear. get it in writing, even if you've spoken.
give objectives, not procedures. let them surprise you with their solutions. you want to get people smarter than you, so let them prove it.
let them know which tasks are most important.
beware upward delegation; don't take it back.
praise and thank people. it's the best reward.
don't email more than one person to get something done, unless you are ccing their boss.
it's okay to nag if they haven't responded within 48 hours.
meetings
book: one minute manager
don't go to a meeting without an agenda.
don't let people be half-there. it's either worth the time or not.
only rarely have meetings > 1 hour.
1 minute for the minutes: who is responsible for what by when and email to everybody.
use technology that changes the workflow.
manage from beneath
when do we next meet?
what should i have done by then?
where else can i get help?
your boss wants results, not an excuse.
time off
2 options: alternate contact, or contact me when i get back. that way you don't have work piling up when you're gone.
kill your television.
turn money into time, especially with young children.
you always have time to sleep.
never break a promise, but renegotiate if necessary.
if you don't have time to do it right, you don't have time to do it wrong.
most things are pass/fail.
get feedback sleuths, and ask in confidence. the most valuable thing in the world is someone will tell you the truth about what you're doing right and wrong.
http://www.youtube.com/watch?v=oTugjssqOT0
randy pausch
most important point: in 7 habits 4 task quadrants, do 1. important/urgent, 2. important/not urgent, 3. unimportant/urgent, 4. unimportant/not urgent. that way, 2s don't become 1s and you don't get stressed.
many people do unimportant/urgent second, but that's wrong.
being successful doesn't make you manage your time well.
managing your time well makes you successful.
(me) time are money are related; so are time and stress.
don't do the wrong things beautifully; do the right things adequately.
why am i doing this? what will happen if i don't?
good judgement comes from experience.
experience comes from bad judgement.
there are no shortcuts to good judgement.
you can't change the plan unless you have one.
do the ugliest thing first.
if you have to eat a frog, don't look at it first.
if you have to eat 3 frogs, start with the big one.
only have 1 document on your desk at a time; that way you fight them one-on-one
touch each piece of paper (or email) only once.
use a speaker phone when you're, and tell them politely how long you've been on hold.
stand while talking on the phone.
start with giving an agenda for the call.
have something on your desk that you'd like to get back to.
'i'd love to keep talking but i have ___ waiting.'
group your calls and make them right before lunch and quitting time. then they will have something to get back to.
use a phone headset.
keep kleenex and thank you notes on your desk.
people will remember you if you send them a thank you note.
keep a paper recycle bin near your desk that is emptied infrequently. then you will not hesitate to throw it out since you have a grace period to get it back.
make your office comfortable for you, and optionally comfortable for others.
keep folding chairs to the side.
you don't 'find' time for important things. you 'make' time by neglecting something else. consider opportunity cost and learn to say gentle 'no's.
gentle no:
'i'm strapped, but i don't want you to be in a bind. i'll be your fallback if you can't find someone else.' then you will see how that person treats you, based on whether or not they keep looking.
find your creative time. defend it ruthlessly and spend it alone.
find your dead time. schedule meetings, phone calls, and exercise (when you don't need to be at your mental best.)
an interruption of 6-9 minutes takes 4-5 minutes to recover from.
when people interrupt you, say 'i'm in the middle of something' or 'i only have 5 minutes'. then they will want to handle it quickly and you can leave gracefully later.
if someone won't leave your office, go to the door, complement and thank them, and shake their hand. if they still won't leave, leave.
time journal
monitor yourself in 15min intervals, update every 30min, for 3days to 2weeks.
what am i doing that doesn't need to be done, what can i delegate to someone else, where am i wasting time, and where am i wasting other people's time?
time management is collaborative, not selfish; help other people be more efficient
remember the point is not to do more, it's to go home on time.
effective vs. efficient
do what will work better, even if it takes more time.
we don't procrastinate from laziness. identify why you are not enthusiastic. are you afraid of failure? are you avoiding the decision to prioritize?
doing things at the last minute is much more expensive than just before the last minute.
make up a fake deadline and pretend it's real.
sometimes you just have to ask and wonderful things happen.
delegation
don't treat it like dumping.
give responsibility and authority.
keep the ugliest part of the job yourself, so you and they know you are not dumping.
treat your people well.
you cannot be vague if you want something to get done. specific thing to do, specific date _and_ time. specific penalty or reward (for them).
challenge people; they usually yearn for an opportunity.
communication has to be clear. get it in writing, even if you've spoken.
give objectives, not procedures. let them surprise you with their solutions. you want to get people smarter than you, so let them prove it.
let them know which tasks are most important.
beware upward delegation; don't take it back.
praise and thank people. it's the best reward.
don't email more than one person to get something done, unless you are ccing their boss.
it's okay to nag if they haven't responded within 48 hours.
meetings
book: one minute manager
don't go to a meeting without an agenda.
don't let people be half-there. it's either worth the time or not.
only rarely have meetings > 1 hour.
1 minute for the minutes: who is responsible for what by when and email to everybody.
use technology that changes the workflow.
manage from beneath
when do we next meet?
what should i have done by then?
where else can i get help?
your boss wants results, not an excuse.
time off
2 options: alternate contact, or contact me when i get back. that way you don't have work piling up when you're gone.
kill your television.
turn money into time, especially with young children.
you always have time to sleep.
never break a promise, but renegotiate if necessary.
if you don't have time to do it right, you don't have time to do it wrong.
most things are pass/fail.
get feedback sleuths, and ask in confidence. the most valuable thing in the world is someone will tell you the truth about what you're doing right and wrong.
Wednesday, April 10, 2013
mobiunpack.py
http://wiki.mobileread.com/wiki/Mobi_unpack
nice little script for yanking the epub out of an amazon mobi files: azw, tpz, azw4, or kf8.
nice little script for yanking the epub out of an amazon mobi files: azw, tpz, azw4, or kf8.
exporting word doc to html
http://www.mobileread.com/forums/showthread.php?t=142530
microsoft word puts a lot of junk into html documents when you export them from word. this macro might help clean it up.
microsoft word puts a lot of junk into html documents when you export them from word. this macro might help clean it up.
aveotsd
i did some looking around for a tongue-retaining device to reduce the incidence of sleep apnea events. looks like the best one is from aveosleep. clinical trial info is here:
http://www.aveosleep.co.uk/clinical-trials.php
journal article here:
http://www.aveosleep.co.uk/documents/Tongue%20Stabilizing%20Device%20-%20Pilot%20Study.pdf
http://www.aveosleep.co.uk/clinical-trials.php
journal article here:
http://www.aveosleep.co.uk/documents/Tongue%20Stabilizing%20Device%20-%20Pilot%20Study.pdf
quandl
http://www.quandl.com/
amazing amount of economic indicators, financial, and social data for free, in convenient formats. if i need any time series in the future, i'll check to make sure it's not there already.
quantopian already has hooks to grab data from there.
amazing amount of economic indicators, financial, and social data for free, in convenient formats. if i need any time series in the future, i'll check to make sure it's not there already.
quantopian already has hooks to grab data from there.
Friday, April 5, 2013
quantopian and zipline
quantopian allows you to write algorithmic trading (black box trading algorithms) in python and then backtest them easily with their data. really cool. i need to try this out.
i think i heard them say they have 1 min data for 10 years for all equities. once they have their features in place, they will add futures and then options.
zipline is their backtest system, which they have opensourced. eventually they will support real trading with their algo system.
https://www.quantopian.com/
i think i heard them say they have 1 min data for 10 years for all equities. once they have their features in place, they will add futures and then options.
zipline is their backtest system, which they have opensourced. eventually they will support real trading with their algo system.
https://www.quantopian.com/
When cointegration of a pair breaks down
interesting article with more info on multivariate cointagration and pairs trading (or other multiples trading). comments contain lots of other links.
http://epchan.blogspot.ca/2011/06/when-cointegration-of-pair-breaks-down.html
http://epchan.blogspot.ca/2011/06/when-cointegration-of-pair-breaks-down.html
get important things done
completion of an unimportant task is not a success. prevent or punish it as the failure that it is.
there is a world of difference between staying busy and getting things done.
why is working on a task so much more comfortable than defining and prioritizing tasks?
when writing or presenting research: "the truth,the whole truth, and nothing but the truth." the truth, only the important parts that guide the audience through the present narrative, and including simplifications that facilitate comprehension.
"you can waste a lot of time trying to do things quickly." i've seen this happen, but why does it happen? maybe because i try to build complex processes that will run very quickly once they finally start. but the facts on the ground change before my system is in place, and i can't adapt my process or the goal looses relevance and importance by the time i'm ready to use it. better to do something less efficient but starting sooner.
your home is not your head. remember to interact.
there is a world of difference between staying busy and getting things done.
why is working on a task so much more comfortable than defining and prioritizing tasks?
when writing or presenting research: "the truth,
"you can waste a lot of time trying to do things quickly." i've seen this happen, but why does it happen? maybe because i try to build complex processes that will run very quickly once they finally start. but the facts on the ground change before my system is in place, and i can't adapt my process or the goal looses relevance and importance by the time i'm ready to use it. better to do something less efficient but starting sooner.
your home is not your head. remember to interact.
ipython parallel and acyclic graphs
heard an interesting tidbit from a talk by brian granger. apparently the ipython parallel kernel has the ability to take acyclic graph dependencies and intelligently distribute the computation. i need to look into that.
python garbage collection
read a couple of interesting articles on how the garbage collector works in python. this post points out performance issues when you create lots of new objects without deleting any:
http://dsvensson.wordpress.com/2010/07/23/the-garbage-garbage-collector-of-python/
pymotw is very good, very informative:
http://pymotw.com/2/gc/
http://dsvensson.wordpress.com/2010/07/23/the-garbage-garbage-collector-of-python/
pymotw is very good, very informative:
http://pymotw.com/2/gc/
thomson reuters qa direct
continuum.io is doing some interesting things these days. wakari is something i don't really understand yet, but things like blaze, numba, and anaconda look really neat. i need to check them out for my next project.
an interesting example of wakari usage is in a data service from thomson reuters. they have some demo data available if you sign up.
http://continuum.io/blog/wakari-and-big-finance
an interesting example of wakari usage is in a data service from thomson reuters. they have some demo data available if you sign up.
http://continuum.io/blog/wakari-and-big-finance
uncertainty in global warming data
here's an interesting paper on some of the uncertainty present in temperature data and in results that use it. similar issues of measurement error and computational uncertainty come up everywhere you look.
https://www.ncdc.noaa.gov/oa/climate/research/ersst/papers/SEA.temps08.pdf
https://www.ncdc.noaa.gov/oa/climate/research/ersst/papers/SEA.temps08.pdf
Subscribe to:
Posts (Atom)
Posts
