using mlbviewer for gameday audio

got my account working with mlbviewer (make sure to get the svn version for 2010 changes!) and the latest rtmpdump. only change i had to make was to change

DEFAULT_F_RECORD = 'rtmpdump -f \"LNX 10,0,22,87\" -o %f -r %s'

to

DEFAULT_F_RECORD = os.path.join(os.environ['HOME'],'checkinstall','rtmpdump','rtmpdump-2.2b','rtmpdump -f \"LNX 10,0,22,87\" -o %f -r %s')

in MLBviewer/mlbtv.py since i haven't installed rtmpdump in root. my ~/.mlb/config is

use_nexdef=False

user = <>

pass = <>

#debug = True

show_player_command = True

speed = 800

audio_player = mplayer -dumpstream %s -dumpfile %f

use_xml = False

audio_follow = 'bos'

favorite = 'bos'

i'm not totally sure about needing that use_xml = False, but i had some connection problems (the fact that the mlb seems to look me out for some time after i fail and/or try to reconnect soon doesn't help) before messing around with stuff and this Just Works. now i just run mlbviewer.py, navigate to the game, and press 'a' to download.

even though mlbviewer names the file .mp3, it's actually a flv with a single (mp3) stream. these commands are handy for demuxing and chopping the mp3 to more manageable sizes for my mp3 player.

ffmpeg -i 2010-03-19-bosmlb-pitmlb-1-MLB.com.mp3 -acodec copy tmp.mp3

mpgsplit -18 tmp.mp3 -b 2010-03-19-bos-pit

now i just need to rig up mlblistings, test/gamedayaudio.py, and the above commands to do this automatically and put stuff on my mp3 player. also need to put some tags on the files so they are conveniently grouped.

plotting numpy data with mayavi

just getting started trying to use mlab in mayavi to visualize unstructured 3d data in numpy arrays. found some links from this examples page:

unstructured grid example

surface from irregular data

scipy cookbook has an example of structured data (one of the last examples on the page). most of the non-mlab mayavi examples use data loaded from a file, so unstructured numpy examples are a little harder to find. here is (i think) an updated version for mayavi2. also, here is an example of how to pick a point on a surface with the mouse, which should come in handy at some point.

also, i could use the auto script generator to help me figure out which functions and object i need to use.

also, make sure to use -wthread with ipython, and this from the docs:

You can also use show() to decorate a function, and have it run in the event-loop, which gives you more flexibility:

from enthought.mayavi 

import mlab 

from numpy import random  

@mlab.show 

def image(): 

   mlab.imshow(random.random((10, 10))) 

With this decorator, each time the image function is called, mlab makes sure an interactive environment is running before executing the image function. If an interactive environment is not running, mlab will start one and the image function will not return until it is closed.

mayavi examples

interesting code examples to look at for using mayavi.

lorentz_ui - traits + mayavi for updating visualization

vtk_commodities - vtk/mayavi + traits + chaco for 2d plot connected to and communicating with 3d plot

also, i should look at uses of mlab from mayavi. apparently it lets you just plot something in a figure, with a button that pops up the full gui.

rhythmbox with local python

fixed a mysterious problem with rhythmbox. it was giving me segfault immediately on startup, along with some other (apparently unrelated) clutter.

** (rhythmbox:5239): CRITICAL **: atk_object_set_name: assertion `name != NULL' failed

** (rhythmbox:5239): CRITICAL **: atk_object_set_name: assertion `name != NULL' failed

WARNING: Unhandled message: interface=org.freedesktop.DBus.Introspectable, path=/, member=Introspect

and complaints about pygtk. i thought i could fix it by putting pygtk from the system python in my PYTHONPATH, but no joy. only by taking my local python bin out of the PATH would it work.

m$ access mdb files in linux

looks like the only game in town for reading an mdb database file from microsoft access is mdbtools. several other tools are built off of it, which can help to give a quick gui look at what a file contains. gmdb2 is easy enough to use, but i couldn't get oobase (from openoffice) set up before my attention span expired. here's a little python snippet that dumps out the data:

import sys, os

mdb = sys.argv[-1]

out = os.popen('mdb-tables -S -1 '+mdb)

tables = out.read().split('\n')[:-1]

s = ''

for table in tables:

#print table

s += table+'\n'

out = os.popen('mdb-export %s "%s"'%(mdb,table))

s += out.read()+'\n'

print s

it seems quite clear that the mdb jet 4 format is grossly inefficient for small databases. this snippet spits out some csv text which is about 13kB, compared to the original mdb that weighs in at 430kB. grokking around the mdb confirms that almost all of it is 0s (fortunately easy to see, since i understand the jet standard does not require allocated file space to be wiped clean).

scp remote backup

i know, i know.... i should really just use some kind of rsync to backup one computer to another. but sometimes you just need to get the job done without installing anything. here's a pretty good way to do it. it uses the handy extglob option for bash, which i'm sure i'll use for other things now that i know about it.

shopt -s extglob

cd /

scp -r -p -C !(dev|proc) user@remote.machine:/Backup/

shopt -u extglob

fixing munged avi

recently i had to patch up an avi that was cut off in the middle of recording my desktop (with camstudio for windoze). the movie file it produced had data blocks, but there was no file or stream header and the first 2048 bytes were 0.

i managed to get it limping by recording a 10-second avi with the same software and transplanting that file's header onto my 2.7gb waste of time, with dd:

dd if=good.avi of=header count=4

dd if=noheader.avi of=tmp skip=4

cat header tmp > withheader.avi

probably a faster and more disk-efficient way, but i'm lazy like a good engineer should be. my mplayer doesn't like any of camstudio's avis, but vlc will play this frankenstein as long as i select 'do not repair'.

Reproducing kernel Hilbert space

technique used in machine learning. might need to look into this...

aa1 wifi, vpn, wpa2, etc

this blog points out some fixes to update the default NetworkManager on the acer aspire one. i needed to use this stuff on my aa1 to get it to work with a wpa2 enterprise network.

i still need to get a working pptp plugin for NetworkManager-pptp-0.7.0. maybe NetworkManager-pptp-0.7.0-0.12.svn4326.fc8?

a comment on the post lists something else i need to try:

I also found that passwords stopped being remembered after installing these new NetworkManager versions. After a bit of googling I found the solution to be to add a file /usr/share/dbus-1/servicI also found that passwords stopped being remembered after installing these new NetworkManager versions. After a bit of googling I found the solution to be to add a file

/usr/share/dbus-1/services/gnome-keyring-daemon.service with the contents [D-BUS Service] Name=org.gnome.keyring Exec=/usr/bin/gnome-keyring-daemon Hope this works for you.

hp scanner card reader

finally got the sd card reader on a networked hp scanner/printer mounted on my ubuntu machine. this allows me to scan to the card and then access it remotely.

here's the relevant line to put into fstab:

//hp.scanner.ip/memory_card /media/scanner cifs users,uid=localusername,gid=localgroup,user=guest,password=hp 0 0

previously, i was getting mysterious samba/cifs errors in dmesg. now that i'm using guest and hp as the username and password (and maybe thanks to a card reformat) it's magically working.

link to local files with pybliographic

haven't quite made this work yet, but...

in pybliographic, go to settings->preferences->resource. select another field from the pull-down at the bottom of 'viewable-fields', like 'location', and add it. also had to move the 'acroread' association in viewers below evince (and put it google-chrome above mozilla while i was at it) since that's what's on ubuntu by default.

problem is that when i have anything in the location field, nothing shows up in the resource right-click pop-out (not even 'url'). i'm also having problems with spaces in paths in the url field.

file readers

i've spent way too much time already writing a file reader for slurping up binary data from an experimental setup. i've been burned in so many different ways by incorrect or nonsensical data in these files, i've decided on a rule i should follow any time i need to do this again: read each atomic unit of data with as few context assumptions as possible; i.e., loop over units in the stream, do not loop over any assumed structure for the units. both missing and duplicate data have wreaked havoc on my pretty little reader, each requiring a new refactoring. next time i will just start out iterating on the stream and plan on dealing with nonsensical structure, even when (!!) there is metadata that could turn out to be wrong. for now i'll just build the most convenient structure in memory for all the data that makes sense and throw junk into an extra array that i can check later if i need to. i won't be sure if that's the best way until i get a chance to use it.

maybe i should use something like python-hachoir for this.

ompc

wow, wow, wow... a prophylactic barrier between me and matlab. sorry for the imagery, but i'm just so tired of being forced to deal with matlab by legacy code/coders. i need to get up to speed on this.

there were a couple of simple bugs i had to fix to get it to compile, and the distutils install choked looking for sandbox/ompc.pth. not sure what should be in there, so i just touched it to keep things moving. also, i had to fix a startling bug near the end in the ompc/ompcply.py file from the examples.zip linked from the examples page (startling because it surely could never have run before).

if not os.path.exists(sys.path[1]): should be if not os.path.exists(sys.argv[1]):

also, ompcply.py needs to be executable with the appropriate shebang at the top.

so, most examples work (lots of warnings about how the bytecode stuff doesn't work with > python 2.5) but a quick test of some matlab code i had lying around didn't. surprisingly, the krufty matlab did work out of the box on octave (even the plotting). so i'm not sure ompc is quite ready for primetime yet, though to be fair i should have used 2.5.

also there is a libermate package. haven't tried it.

Python for Information Theoretic Analysis of Neural Data

an excellent article comparing mi bias correction methods also discusses reasons for switching from matlab to python. one of the interesting ideas mentioned is the shuffled information estimator, which can be used along with any of the other techniques. i just read through it quickly, but i think there is some kind of connection there to the bootstrap or other resampling methods to estimate, and thereby reduce, conditional entropy bias. more on that in 'tight data-robust bounds to mutual information combining shuffling and model selection techniques', which claims robust upper and lower bounds and precise estimates even for high correlation.

the python code used in the paper for estimation of entropies and mi is on google code. i installed it from the svn downloaded source by going into ~/checkinstall/pyentropy/pyentropy-read-only/pyentropy and

checkinstall python setup.py install

the section, 'A python library for information theoretic estimates' gives a description of the library and how to use it. only problem for me is that it assumes a finite alphabet. i think some smart histogram binning should allow me to work with it, though.

reasons for migrating their code from matlab to python are interspersed throughout the article (for example, they say cython is easier to use than mex), and they mention using mlabwrap to facilitate.