tail -f findings.out

Personal Settings in SVN Increases Happiness by 125%!

This may sound a little silly, but when I first started using aliases in Linux (and loving their potential utility), I was put off by the realization that once I got used to using them, I would end up being on another machine without them. This annoyance made me resistant to using them for some time. Eventually, I started using them so much, that I copied my .bashrc (and .vimrc and other such files) to all the remote servers I would need to interact with regularly.

This helped quite a bit, but it introduced another level of annoyance. Whenever I made a change, improved a command, added a new alias, I would have to add that to all the copies I had floating around. Dumb. Dumb and annoying.

So, I thought, why not set up a repo? While I use SVN regularly, I had never setup one on my own before, so it seemed like a win/win situation. In sum, the following is what I did to get it all to work.

The first thing to decide is how the repository will be accessed, either via SSH or Apache. The use of Apache is very common, and there are many tutorials on how to do it (like this one). I chose not to use Apache for 2 reasons: I did not want to have to be concerned with securing Apache on the server in question unless necessary, and secondly any computer on which I would want to access the repositories would have SSH. If you want to be able to see your repo in a browser, use Apache (or Trac).

Initial steps:

sudo apt-get install subversion # As of now on Feisty, this installs 1.4.3sudo mkdir /home/svn/configfiles #For the repo, easy place to remember

You can replace “configfiles” with whatever you want to call the repo. For this case, it does not matter a lot, since I will be checking out the files in many various places, not all in one directory.

Now for permissions and such:

sudo addgroup subversionsudo usermod -a -G subversion YOURUSER

And the real magic:

sudo svnadmin create /home/svn/configfiles/sudo chown -R subversion configfiles # make sure perms are right after repo creationsudo chmod g+rws configfiles # Dittosudo svn import /your/initial/files file:///home/svn/configfiles/repo -m "initial import"sudo apt-get install xinetd

Now run

sudo vim /etc/xinetd.conf

and add:

defaults{}

service svn{port = 3690socket_type = streamprotocol = tcpwait = nouser = www-dataserver = /usr/bin/svnserveserver_args = -i -r /home/svn}

Now go into the conf dir in your repo and

sudo vim svnserve.conf

and edit what you like. I uncommented “anon-access = read” and “password-db = passwd”.

You are done! The test: “svn co svn+ssh://SERVERWITHREPO/home/svn/configfiles”. You will be prompted for as password 3 times, do not worry. If all is well, you should see “Checked out revision 1.”, and the file(s) in the repo will not be in the dir where you ran the above command.

For more info, see the SVN Book. Thanks to this guide for getting me through the main steps.

[NOTE:] After some experimenting, I ended up creating the repo like this:

sudo svnadmin create /home/svn/configfiles/trunksudo svn import ~/.bashrc file:///home/svn/configfiles/trunk/bashrc -m "Initial import of .bashrc"

Then I checked it out via:

svn co svn+ssh://SERVERWITHREPO/home/svn/configfiles/trunk ~/DIR/TO/CHECKOUT/IN

Also, the way to delete a repo completely (as I did during testing) is simply to remove the entire dir (e.g. “sudo rm -rf /home/svn”).

June 30, 2007 - 10:17 PM No Comments

Exporting a MySQL DB to .csv

The CSV file format is extremely useful as well as common for dumping, parsing, and loading data. Being able to export the data of a MySQL database into a .csv format file is a handy ability to have. For one-off uses of exports in this format the MySQL Query Browser provides an option to export a resultset to CSV. But if you want to generate the dump programmatically (or have greater output flexibility) you’ll need to use command line calls (or an API, like mysqldb). In a recent case I did not need the actual SQL for my database tables, just their data. The command I tried first was:

1
2
mysqldump -u USER -pPASSWORD --fields-terminated-by=, \
--fields-enclosed-by=\" --tab=DUMPDIR DATABASE

This dumps the data and the create table statements for tables in DATABASE to DUMPDIR. The “–fieldsX” options here dump the data in CSV format. If you just want the schema you can add “–no-data”, but I haven’t found a way to just dump the data and not the schema as well. Anyway, the problem I ran into was that I first tried DUMPDIR as my home directory. This threw an error:

1
mysqldump: Got error: 1: Can't create/write to file 'dumpdir/tablename.txt' (Errcode: 13) when executing 'SELECT INTO OUTFILE'

A post on the MySQL forums clued me in: It’s because the mysql user doesn’t have the right permissions to write to that dir (my home dir). One solution would be making a new dir to hold the dump, chowning it to mysql or allowing world write (chmod a+w DIR). Or you can write it to /tmp, which is always world-writable. I’d recommend creating a directory for the dump (in /tmp or elsewhere) in any case since the files all get dropped into the DB specified at the same level. So if you have lots of tables and already have files in /tmp or wherever you dump to it could get confusing. One other caution: Some distributions set a size limit on /tmp (since it’s world-writable a rogue process could otherwise fill your filesystem). So be careful if you are dumping a lot of data.

The final version:

1
2
3
4
5
6
7
mkdir /tmp/db_dump
# Free tip: Press alt + . to insert the final arg from the previous line!
chmod 777 /tmp/db_dump
mysqldump -u USER -pPASSWORD --fields-terminated-by=, \
--fields-enclosed-by=\" --tab=DUMPDIR DATABASE
rm /tmp/db_dump/*.sql
rename 's/\.txt$/\.csv/' /tmp/db_dump/*.txt

This produces a .txt file per table in the specified DB containing all the data and a .sql file per table containing the drop/create table statements. It then drops the unneeded .sql files and corrects the extension on the data files.

June 29, 2007 - 3:21 PM No Comments

New and Helpful Aliases

Some aliases I have added of late and found to be useful:

Displays a calendar and then your locale’s 12-hour clock time:

1
alias now='cal;date +%r'

For those who often like to edit X display config:

1
alias editxorg='sudo vim /etc/X11/xorg.conf'

For those use window managers without a shutdown button:

1
alias turnoff='sudo shutdown -h now'

Does all you need to do for updating:

1
2
alias updateme="sudo aptitude update && sudo aptitude safe-upgrade && \
sudo aptitude dist-upgrade && sudo aptitude autoclean"

I am also working on something that I hope to works as follows. “help CMD” will run “CMD -h” or “–help”, checking the results. If that fails, it runs “info CMD”, then “man CMD”. The idea being that you can use one command to display information on a command, and it will get you to the first that has something to show. This would be helpful when one runs into programs that do not handle -h, or only have a man page, etc.

June 29, 2007 - 1:45 PM No Comments

Summary Execution

I find myself coming across strange files on occasion. Crowded folders, shady thrice nested system dirs and the like are filled with them. Usually system files, files whose nature and purpose I might not know. To get to know such individuals, there are several built-in tools I might employ. I could run file on them, cat them, tail them, head them, run ls -lh in the folder, et al. But this takes too many steps. I want one command. And if I can get what I want in 5 steps, I can get it in one. It is the Linux way.

Thus I created the “summarize” function. It’s a simple script that basically just grabs data using the methods I listed above, and formats them into an easy-to-read summary. Right now, it consists of:

#!/bin/bash
# A command to provide lots of
# info about a file at glance!
FILE=`file $1`
echo "$FILE has `wc -l $1 | cut -d" " -f1` lines." &&
echo -e "\e[1;34mOwned by: \e[0m`ls -l $1 | cut -d" " -f3` | \e[1;34mGroup: \e[0m`ls -l $1 | cut -d" " -f4`" &&
echo -e "\e[1;34mSize: \e[0m`ls -lh $1 | cut -d" " -f5` | \e[1;34mLast updated: \e[0m`ls -lh $1 | cut -d" " -f6`" &&
echo "**********************************"
echo -e "\e[1;34mTop bits: \n \e[0m" &&
head -n 5 $1 | nl -b a &&
echo "" &&
echo -e "\e[1;34mBottom bits: \n \e[0m" &&
nl -b a $1 | tail -n 5
exit 0

One handy thing I learned while writing this was how to display text in chosen colors in bash, using the

1
echo -e

syntax. I find this makes it easier for my eye to quickly move to different fields.

You can also add something like this to your .bashrc:

alias summarize='/location/of/script/summarizer.sh'

Example output on a test .txt file (screenshot to show colors. Your base text color will be different):

Summarizer run on a poem on vi.
[EDIT, 07/03/07]: It is also useful to add contextual lines to the head and tail output. This can be done via:

head -n 5 $1 | nl -b a

This prints the first and last five lines, numbered according to the number of lines in the entire file. I find this makes it much easier to immediately see how long the file is. (Additional note: without the "-b a" option, nl will print lines for each textual line, skipping blank ones. I myself would rather the blank lines also be included, as this is standard practice.) Screenshot above has been updated to reflect this new functionality.

June 29, 2007 - 12:30 PM No Comments

Twitter links powered by Tweet This v1.6.1, a WordPress plugin for Twitter.