tail -f findings.out

Catching warnings from the MySQLdb module

The MySQLdb Python module implements the Python DB API for MySQL. I’ve written about its use before. MySQL issues warning messages in a number of circumstances and PEP 249 (which specifies the Python DB API) describes a Warning error message to be included.

One issue I ran into recently was how to catch warnings thrown by this module when running queries. Oftentimes tutorials or forum discussions that cover warnings in the context of MySQLdb describe how to filter them (they can clog up script output). But in a recent case I wanted to grab and check the warning, logging a dependent result. I had hoped this clean implementation would work in a method used for all calls to the MySQL DB:

1
2
3
4
5
6
7
8
9
10
11
try:
    cursor.execute(query)
except MySQLdb.Error, e:
    raise e
except MySQLdb.Warning, e:
    raise e
finally:
    data = cursor.fetchall()
    rows_returned = cursor.rowcount
    cursor.close()
    db.close()

But the warnings just went right through. Instead I needed the warnings module’s assistance:

1
2
3
4
5
6
7
8
9
10
11
12
import warnings
with warnings.catch_warnings():
    warnings.simplefilter('error', MySQLdb.Warning)
    try:
        cursor.execute(query)
    except MySQLdb.Error, e:
        raise e
    finally:
        data = cursor.fetchall()
        rows_returned = cursor.rowcount
        cursor.close()
        db.close()

This catches the warnings and raises them as errors, although their class is still correct, allowing a clean implementation to call the above code (wrapped into a method called do_query):

1
2
3
4
5
try:
    self.do_query(make_cool_table)
    logger.info("Created cool_table table.")
except MySQLdb.Warning:
    logger.info("cool_table already exists.")
Tags: , ,
October 25, 2009 - 10:53 PM No Comments

Viewing all users on a Linux system

There are a number of widely-used and stable utilities on Linux systems that allow you to view information related to users. You can see who’s logged in with who, get info on a particular user with finger, see who you are with whoami and see who logged in last with last and lastlog. And there are commands to create, remove, and change users and their groups.

But what if you just want to see all the users defined on the entire system? Looking in /home won’t show you users that don’t have a home dir or one not located there. All users are indeed listed in /etc/passwd, but having to look through this file every time you just want a list of users is tedious. I haven’t found a utility that performs this role.

This was somewhat surprising, as it’s not an uncommon need. Perhaps I’ve missed some essential program along my Linux journey. If so, feel free to enlighten me in a comment :-) In the meantime, let’s fill this gap:

1
2
3
4
5
6
7
8
9
10
11
12
# Prints all users, divided by login ability and homedir:
function userinfo {
    echo "----- Users that can login -----"
    awk -F":" '!/bin\/false/ { print "username: " $1 ", uid: " $3 ", homedir: " $6 }' /etc/passwd

    echo -e "\n----- And have /home dir -----"
    awk -F":" '!/bin\/false/ && /\/home/ { print "username: " $1 ", uid: " $3 ", homedir: " $6 }' /etc/passwd

    echo -e "\n----- Users that can't login -----"
    awk -F":" '/\/bin\/false/ { print "username: " $1 ", uid: " $3 ", homedir: " $6 }' /etc/passwd
    echo ""
}

This function should work on any *nix system. Place it inside your ~/.bashrc, reload that (. ~/.bashrc), and try out “userinfo”:

userinfo

Three sections are displayed showing usernames, user IDs, and homedirs for users in /etc/passwd who:

  • Don’t have /bin/false for a shell (i.e. users that can login)
  • Also have a homedir under /home (i.e. the users that aren’t for system processes)
  • Do have /bin/false for a shell (i.e. users that can’t login)

Note that the users that can login category only shows users to which one could switch with “sudo su – USERNAME” from a shell. This doesn’t mean anything about whether they can login via SSH or other access methods.

Tags: , , ,
October 4, 2009 - 3:35 PM Comments (7)

Great open source virtualization with VirtualBox

My first forays into virtualization were at work and utilized VMware’s myriad offerings (Player, Workstation, Server, ESX, VI). I was impressed by the capabilities of VMware’s products, and of course excited by the possibilities of virtualization. But for home use I didn’t push past getting VMware Player installed. I was annoyed at not being about to create my own virtual machines with the free products, and the product not being open source didn’t generate much excitement for me either.

So when I finally decided to set up some VMs for my own use, my criteria were straightforward: I wanted a broadly supported, performant, free virtualization solution that was open source. There is quite a range of options out there, e.g. Xen and OpenVZ. I decided to give Sun’s VirtualBox a try and have been nothing but impressed so far:

virtualbox_small

I’ve found VirtualBox easy to install, quite fast, attractive and easy to use.

Installation

Installation on Ubuntu 9.04 was fairly simple following this guide. You add the VirtualBox repository, install the package, and follow the configuration wizard. I didn’t have to do anything special kernel-wise that the installation guide mentions. One thing they leave out: you need to add your user to the vboxusers group. Easy enough:

1
sudo usermod -a -G vboxusers YOURUSERNAME

The manual describes installing on a wide range of other platforms, and there are plenty of tutorials out there as well.

First steps

After VirtualBox is installed, you need to create a VM and then install an OS on it. Click “New” in the top left of the VirtualBox window and follow the wizard. You might run into an issue in this process (only annoyance I’ve encountered so far) when selecting the OS Type and Version. For me the OS options in the drop down show up as very light grey on white until you hover over them:

Nearly invisible OS options

Not a critical issue, you can see the options by hovering down the list.

After you’ve followed the wizard to create the VM and provided it with storage space and other resources, you’ll end up at the VirtualBox main screen. At this point, select the VM you just made and click settings. Then go to CD/DVD-ROM on the left and check “Mount CD/DVD Drive”. If you have an actual install CD/DVD you want to use, select Host CD/DVD Drive. You can also just download the ISO for the OS you want to install and select ISO Image File:

vm_settings_small

After selecting your drive or ISO for installation, save the settings and start the VM. A new window will open displaying the output of the running VM. If all goes well, you should be shown the installation menu for the CD or ISO you’ve mounted.

Miscellaneous

  • You can take snapshots of your VMs in case you need to roll back to a known safe state. When the VM is turned off, go to the Snapshots tab to take new snapshots, review current ones, and roll back if needed.
  • Once you click on the VM window your output is sent to it alone. To get back out and interact with your own desktop, press the right CTRL key (configurable). Or install VirtualBox Guest Additions (see below) and not worry about this at all!
  • Your VMs (and assorted configuration info) are stored in ~/.VirtualBox:
    virtualbox_home_tree
  • Installing VirtualBox Guest Additions is quite helpful. Aside from graphical improvements, after it’s installed you don’t have to click inside a VM to get focus. You just move your pointer into the VM and it has focus. Move out and the host GUI has control again.

Resources

Tags: ,
October 4, 2009 - 12:49 PM Comment (1)

Editing Trac comments at the DB layer

Today I needed to alter the content of a comment made on a Trac ticket. I didn’t anticipate this would be a difficult task, but I wasn’t too familiar with Trac’s DB schema. So to save you a few consternated queries, here’s the spoiler. Comments on tickets aren’t kept within their own table (as you might expect) but are instead stored in the ticket_change table.

In my particular case the Trac DB was MySQL, so the examples I’ll provide will be MySQL queries. If you have an SQLite-backed Trac the operations should be the same, you’ll just need to use SQLite’s syntax.

Say the ticket with the naughty comment is #1234. To see all comments on this ticket:

1
2
3
SELECT * FROM trac.ticket_change
WHERE FIELD = "comment"
AND ticket = 1234;

And to update a comment:

1
2
3
4
5
UPDATE trac.ticket_change
SET newvalue = "New hotness"
WHERE FIELD = "comment"
AND ticket = 1234
AND oldvalue = "Old and busted";

Happy revisionizing.

Tags: ,
October 2, 2009 - 7:58 PM No Comments

Twitter links powered by Tweet This v1.6.1, a WordPress plugin for Twitter.