kaolin fire presents :: software tips

Software tips

c/c++

I'm getting linker errors for variables that shouldn't be externalized...? Why is the linker looking for them?

Who knows? But if you make them static, then it'll know better regardless!

vim

Even crazier goldmine of vim tips

Crazy goldmine of vim tips

"David Rayner (zzapper) 15 Years of Vi + 3 years of Vim and still learning" ((at the time of this link -- I'm only on about 1/2 year of vi, 6 years of Vim))

Admin tips

OpenGL

See what your system supports

GLview ~ http://www.realtech-vr.com/glview/

Windows administration

create a boot disk

Download the HP Flash Utility hpflash1.zip and also download the Windows 98 MS-DOS System Files win98boot.zip. links at sevenforums

Windows host file locations

Windows XP = C:\WINDOWS\SYSTEM32\DRIVERS\ETC\
Windows 2000 = C:\WINNT\SYSTEM32\DRIVERS\ETC\
Win 98/ME = C:\WINDOWS\

open ports

netstat -ano to see what process id is listening to what ports. Then cross-ref with the task manager (you can view PIDs: view->select columns, sort by PID).

Yahoo listservs

getting off of yahoo, getting the list archives back

http://www.tt-solutions.com/en/products/yahoo2mbox/

Fedora

adding a service to fedora (chkconfig/service)

quick answer: chkconfig --add; much better answer, see http://www.thelinuxblog.com/adding-a-service-on-fedora/

Linux

disk rescue

Start with:

ddrescue -f -n /dev/in /dev/out logfile

Pull the drive and reinsert as necessary—presuming you can hot-swap—restarting with: (note that your /dev/in may change)

ddrescue -f -b -T /dev/in /dev/out logfile

And then grind on the remaining bits:

ddrescue -d -f -r3 /dev/in /dev/out logfile

BSD

ports maintenance

portmaster -s, portmaster -e, ... http://lists.freebsd.org/pipermail/freebsd-ports/2009-January/052206.html

see what processes have the most files open

lsof | cut -d ' ' -f 1 | sort | uniq -c | sort -rn | head ~ the bit I was missing in trying to figure out how to do this was the -c flag to uniq!

watch the output of a command run over and over

watch: it's like "top" for any command you want to watch

remove ^M characters from a DOS file

col -bx < dosfile > newfile [[from FreeBSD fortune file, via Dru <genesis@istar.ca>]]

I'm using mrtg and can tell "something" is soaking up my bandwidth. But I can't tell what it is. Can you help me?

trafshow -f -i {interface} "not(host {myip})" -- the "not(host {myip}) is important to keep your ssh/whatnot connection running trafshow to top out the charts. Don't know what interfaces you have? /sbin/ifconfig!

I want to verify two directories (recursively) on two machines have the exact same files. What's the quickest/easiest way to check?

What I find works well: make sure ls is NOT aliased on either box (unalias ls) [[or make sure they're aliased to the same thing... but if ls displays differently, the following will give you a lot of noise]]. Then, in the directory you want to compare, ls -lR > box1. On the other machine, ls -lR > box2. Then get those two files together, for the following:

    cat box1 | tr -s ' ' ' ' | cut -d ' ' -f 5,9 > box1.clean
    cat box2 | tr -s ' ' ' ' | cut -d ' ' -f 5,9 > box2.clean
    diff box1.clean box2.clean
  

tr substitutes characters of one sort for another... here, we're using a trick to compress (that's the -s) multiple spaces into a single space. Then we chop out just the important fields of the ls (5 == size, 9 == filename), and get the difference. Now, this won't catch if the files are different but have the same number of bytes. You could pass an md5 in, but that would probably take a find -exec or some such. this is just quick and relatively easy. :)

If you only want to see if the file names and directory structure are the same (don't care about file sizes) then you can leave off the "5" in the clean step, or you can run through a second clean step of cat boxn.clean | cut -d ' ' -f 2 > boxn.clean2.

Bourne shell tips

R125: UNIX Bourne Shell. case, if, test, do, while, for, [, variables...

also...

• just redirect error: >&
• tie stdout into stdin for later filtering: 2>&1 (i.e. runmydbupdate.sh 2>&1 | grep ERROR | head...)

sysctl

If you made changes to /etc/sysctl.conf and want them to take without rebooting, 'sysctl -p'

Wondering if a port is in use, or what's using it if it is?

lsof -i shows you exactly that! You can match the pids to running processes with ps.

Want to edit all files with a given word in them?

vim `grep -m 1 -r [searchterm] . | cut -d ':' -f 1` -- there's probably an easier way to do this, but whatever. :) "-m 1" tells grep to stop searching a file after the first match, -r is recursive, and the cut grabs just the filename from the grep results. (Piping the results through xargs) instead of (using the backticks and passing to vim directly) can cause strange artifacts, like losing the ability to pass a carriage return. Another way to do the grep that comes to mind a little easier but is the slightest bit more wasteful is grep -r [searchterm] . | cut -d ':' -f 1 | sort | uniq. Just a thought. :) NOTE: If you're wondering why grep is "taking soooooo long", make sure you put the period after the search term. That's the path it's supposed to recurse from; else it will be parsing your stdin, instead.

Want to just search and replace? perl is your friend! recursively, even.

find . -type f -exec /usr/bin/perl -pi -e 's/find/replace/g' {} \; -- what I've used this for a couple of times is moving an svn repository, or rather, cleaning up after doing so: find . -type f -name entries -exec /usr/bin/perl -pi -e 's#http://oldrepos/location#http://newrepos/newlocation#g' {} \;

Count all "lines of code" in the .php and .html files under the current directory that aren't under the "includes" or "forum" directories. wc runs wordcount on each file that matches. "tr" through "bc" takes those numbers and adds them up. Your particular "wc" may have slightly different results, so if you get errors run without "wc" et. al. to see how else you might need to parse...

find . -not \( -path ./includes -prune \) -not \( -path ./forum -prune \) -regex '.*/.*\(php\|html\)' -exec wc -l \{\} \; | tr -s ' ' ' ' | cut -d ' ' -f 2 | paste -sd+ - | bc

syslog synchrony

put a minus "-" in front of the file-path in your syslog.conf and it won't sync to disk after every entry. [[Richard Huxton, from pgadmin list]]

remove " from a fileOld and save to fileNew:

cat fileOld | sed 's/\"//g' > fileNew

setting up ssh-agent

• cd ~/.ssh (create .ssh directory)
• rm identity
• rm identity.pub
• ssh-keygen -t rsa
• scp id_rsa.pub [remotehost]:.ssh/authorized_keys2

pretty diffs

-u: the 'unified output format', seems also to be the most readable

making a non-definitive backup

Say you want to backup your user partition, but you don't want to waste time on their excessive mp3's and whatnot. tar has an excellent 'exclude' parameter, and mixed with {}'s replacing magic, you can tighten an exclude down to the following:

tar cspzf backup.tgz "--exclude \"*.{mp3,mpg,avi,tgz}\" *

create an iso image; extract an iso from a cd

dd if=/dev/cdrom bs=2048 count=326239 conv=notrunc,noerror > file-name.iso
you might have to be root, and you might have to use whatever file /dev/cdrom is pointing to (ls -al /dev/cdrom). Grabbed from Yo Linux, which has a bunch of useful information about burning cds.

CMake

see what those generated makefiles are actually doing (debug output, gcc and g++ and such)

cmake
make -VERBOSE=1
or just
cmake -DCMAKE_VERBOSE_MAKEFILE=1

MySQL

optimize and analyze all tables in all dbs on a system

mysqlcheck -Aao -uroot -p or, more verbosely? mysqlcheck -u root -p --auto-repair --check --optimize --all-databases

Postgresql

current running postgresql queries

SELECT datname,procpid,current_query FROM pg_stat_activity

Forget what your plpgsql function does?

select prosrc, pronargs, prorettype, proargtypes from pg_proc where proname = 'yourfuncname';

What is the SQL syntax for to do what the psql client can do?

If you run psql with the -E option and then do a \d tablename, the sql used to calculate the response will be displayed. That sql would be the right way to do what you want.

How do I tell what database files (/var/lib/pgsql/data/base) are associated with which database?

select datname, datid from pg_stat_database

How can I tell what encoding my database is using?

in psql, '\l', which (-E will tell you) looks a bit like:

SELECT d.datname as "Name", u.usename as "Owner", pg_catalog.pg_encoding_to_char(d.encoding) as "Encoding" FROM pg_catalog.pg_database d LEFT JOIN pg_catalog.pg_user u ON d.datdba = u.usesysid ORDER BY 1;

What should I use for money?

postgresql docs say NUMERIC. Though many would argue for whole numbers, shifted the appropriate fractional amount.

What users exist?

select * from pg_user. pg_group for groups.

Having to work on a different system?

http://www.oreilly.com/news/sqlnut_1200.html: common functions on mysql, oracle, postgresql, and MS SQL.

watch your sql connections from the process-side

watch --interval=1 "ps ax | grep post | grep -v watch | grep -v grep | sed 's/^ .\*?//' | tr -s ' ' ' ' | cut -d ' ' -f 5-"

backing up and restoring...

pg_dump -b -f filename -F c -U postgres dbname

or pg_dumpall --globals -U postgres

then pg_restore filename -C -d template1 -U pgsql -W

SQL Server

How can I find all columns with a given name?

SELECT obj.name,col.name from databasename.dbo.syscolumns col join databasename.dbo.sysobjects obj on col.id = obj.id where obj.xtype = 'U' AND col.name like '%yourcolumn%'

website design

color selection

select a base color and automatically get a bunch of related colors for a color scheme

video cruft

record a realplayer stream to disk

1. mplayer -dumpfile [dumpfile] -dumpstream [dumpstream]
2. mplayer -vo null -vc null -ao pcm -aofile [wav output] [dumpfile]
3. lame [wav output] [mp3 output]

resize a video, then reencode it H264

There really has to be a better way, but this is quick-and-dirty (after hours of compiling and installing ffmpeg/imagemagick)

• convert -resize 320x240 10000.mov 10000_resized.mov
• ffmpeg -i 10000_resized.mov -vcodec libx264 -threads 0 -b 320k -f mov -y 10000_resized_webok.mov

Voila! HTML5-ready video! sized small, and lowish-bitrate. :)

And it looks like -vf scale=320:240:-1 to ffmpeg would do the resize there (with -1 "trying to keep aspect ratio"?)

a note on ImageMagick's delegates

locate delegates.xml—there's stuff in there. :) That you can change (or at least see to know what defaults are being used). you might also try "convert -list configure" and look at the line starting with DELEGATES. :)