tail -f findings.out

Feed now updating again

If you follow this blog’s feed, you might have thought I hadn’t written a post for a while. As it turns out, the time it was taking my server to respond when Feedburner asked for recent items was greater than Feedburner’s timeout period (over 10 seconds). I’m not sure why this was the case, my feed URL simply started serving at a much lower transfer rate one day. My temporary solution has been to reduce the number of items shown in the feed from ten to five. This has allowed Feedburner to update, so the last three posts that were missing should now be shown.

I apologize for the lacuna, and hope to perform some optimization to get the site running faster soon. Thanks for reading!

May 21, 2009 - 6:53 AM Comments (2)

Rails on Ubuntu

[EDIT, 2009-12-20]: For some time I steered away from installing Rails via APT on Ubuntu systems. I ran into a number of annoyances early on trying to get various bits of Ruby, Rails, libraries, and others installed and working together properly. And along the way I just never tested the following on a fresh Ubuntu system:

1
sudo apt-get install rails sqlite3

That’s it. On a freshly baked Ubuntu 9.10 Desktop or Server, this will install (as of today):

  • ruby: 1.8.7
  • rails: 2.2.3
  • irb: 0.9.5
  • rake: 0.8.4
  • rdoc: 1.0.1
  • sqlite: 3.6.16
  • rubygems: 1.3.5

And a host of dependencies of course. Perhaps I had simply FUBAR’ed my system by the time I tried this in the past, but it is working swimmingly now. Most of the other tutorials I’ve come across for Rails on Ubuntu take the more manual approach as well, but for now this looks like the way to go. For comparison, when I went through the more stepwise setup process using the script mentioned in the original content of this post below all the same versions were installed except for Rails itself, which was 2.3.5. So a slightly newer Rails version via gem than the Ubuntu repo, otherwise identical.

What follows is the original, more manual approach to getting everything installed. It might be useful to try if you run into problems with the repo package.


As I tried to get Ruby on Rails up and running on multiple (Jaunty Jackelope) Ubuntu 9.04 boxes, I came across a few… gotchas. While the steps described in this post will get Ruby, Rails and friends in place and working, there are a few places that have other potentially perilous options resulting in much gnashing of teeth.

What follows are the commands needed to get Rails working on a fresh Ubuntu 9.04 (server) box. To save some copying and pasting you can use this script which sets everything up and provides logging.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
sudo apt-get install ruby ruby-dev libopenssl-ruby rdoc irb build-essential
cd /tmp
# Check here to see if this is the latest:
# http://rubyforge.org/frs/?group_id=126
wget http://rubyforge.org/frs/download.php/60718/rubygems-1.3.5.tgz
tar xzvf rubygems-1.3.5.tgz
cd rubygems-1.3.5/
sudo ruby setup.rb
# Without this 'gem' will either not work or will be something else:
sudo ln -s /usr/bin/gem1.8 /usr/bin/gem
# Test:
gem --version
# Should be: 1.3.5
sudo gem install rails
# Test:
rails --version
# Should be: Rails 2.3.2 (or whatever is latest)
# To get sqlite setup (default Rails DB backend):
sudo apt-get install sqlite3
sudo apt-get libsqlite3-dev
sudo gem install sqlite3-ruby

Warnings:

  • Don’t install gem with apt-get. This is not RubyGems. And don’t even install rubygems via apt-get. Install it from RubyForge.
  • Don’t forget to make the symlink from gem1.8. This can cause no end of pain if you didn’t listen to the last warning.
  • After RubyGems is in place, install Rails and other related items you need with it. Don’t use apt-get for these.

For more information on setup, covering DB and web server options on Ubuntu, check out the Community Guide.

[EDIT, 2009-10-04]: I just tried starting up Rails installed through the process above on a fresh Ubuntu 9.04 desktop. Turned out I also had to install libopenssl-ruby via apt-get. This is apparently due to some splitting of packages in the Ubuntu repo.

Tags: , ,
May 17, 2009 - 7:35 PM Comment (1)

Send events to Zenoss from scripts

ZenossWorking at Zenoss the company, our IT group uses Zenoss the application to monitor our infrastructure. Initially, we setup the needed monitoring for various resources, added thresholds and received email alerts if there were problems. That was all well and good, but in scripts written to perform various maintenance tasks on our systems, problems encountered were instead logged and emailed.

One of my clever co-workers noted it would make more sense to send such problems to our Zenoss instance, in order to incorporate them into our existing alerting schemes, as well as quickly see if the problem encountered had been caused by other problems on the same machine, etc. As it turns out, it’s actually quite easy to do this! This howto explains the process fairly well, giving basic examples in several languages. I’ll provide a more robust Python example here, and explain how we used a non-documented feature to great effect.

I found it best to create a function to try to send the event to Zenoss, falling back to sending an email if problems were encountered. This would then be called whenever an event needs to be sent:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
# For sending events to Zenoss:
from xmlrpclib import ServerProxy

def send_zenoss_event(server, component, message, eventclass):
    """
    Send an INFO event to defined instance of Zenoss.
    """

    # Setup connection:
    serv = ServerProxy('http://USER:PASSWORD@ZENOSS_SERVER:8080/zport/dmd/ZenEventManager')
    # Define the event:
    evt = {'device':server,
            'component':component,
            'summary':message,
            'severity':2,
            'eventClass':eventclass}
    logger.info("Sending %s event for %s to Zenoss..." % \
        (component, server))
    try:
        serv.sendEvent(evt)
    except:
        send_alert("Couldn't send event to Zenoss for %s." % server)
    logger.info("%s event for %s sent." % (component, server))

xmlrpclib is part of the standard library, so you don’t need to install anything. send_alert() is a function I commonly include in my sysadmin scripts. It logs the message passed (on how to setup logging, see this post), then emails it to the appropriate party. This way you can ensure that even if something is up with the Zenoss instance, someone still finds out there’s a problem. You could also use this same function as a way to exit your problem cleanly if problems are encountered that can’t be resolved, or connect it to another that handles this.

On the event attributes:

  • The value for ‘device’ should match a device name in Zenoss.
  • Severity is a numeric value mapped to what sort of event it is. The following options are available:
    • Severity: Description
    • 5: Critical
    • 4: Error
    • 3: Warning
    • 2: Info
    • 1: Debug
    • 0: Clear

    It might be useful to change severity in the above function to a parameter, so the full range of event types would be available.

  • Component is an object contained by a device, such as CPU or hard drive space. It’s basically free form, although there are a number setup in Zenoss by default. You might decide to utilize and augment this structure, or you might create your own custom one.
  • Event classes exist to categorize events. A number of these are available in Zenoss by default, and you can also add your own.

There is an additional attribute that turned out to be quite useful: timeout. When a timeout is passed, it is added to the events.heartbeat table in the Zenoss database. Once the specified time has passed, an alert will be raised for that device. This is why the device name passed needs to match the device name in Zenoss, otherwise nothing happens after the timeout. To send it, simply add these two lines to the evt dictionary defined in the above function:

1
2
'eventClass':'/Heartbeat',
'timeout':timeout # An integer

We took advantage of timeout to keep track of our backup processes. When the backup process for a device started, it sent an event to Zenoss, the timeout being the time recorded for the last backup, plus some buffer. Then if the backup didn’t complete in the proper period of time, an alert was raised for that device, with the component of “/Backup”. In addition, when the backup completed, another event was sent to Zenoss, with a new timeout of the time between backups of the current server minus the time taken for the current backup, again plus some buffer. Then if the next backup didn’t start (and thus reset the timer) in time, an alert was also sent. This system combined with email fall back means that we always know if our backups don’t start and/or don’t complete.

Tags: , , ,
May 16, 2009 - 3:11 PM Comments (3)

Ruby, Rails, and Gems: Getting Started

I recently started learning Ruby (mostly to flesh out my programming knowledge) and since one of the main self-detected shortcomings in my programming knowledge and skillset is web frameworks and web programming, I decided to give Ruby on Rails a shot. This post will describe the first steps needed to begin using Ruby, Ruby on Rails, and their entourage. I’m running Ubuntu 9.04 Jaunty Jackelope, but there are many guides out there to get you up and running. It will also cover some basic tips for using RubyGems and other related topics to get you going faster. Thanks to friend and co-worker Trevor Rosen for ample (cough, truculent, cough) encouragement, information, and assistance in this bauble-strewn journey.

First install Ruby and a few related items:

1
sudo apt-get install ruby rdoc irb

RDoc allows you to generate HTML API presentations from Ruby code. irb is the interactive Ruby prompt, great for trying out quick ideas.

Next you need to install RubyGems. RubyGems is the Ruby packaging system, as easy_install is to Python and apt-get is to Ubuntu. It’s a lovely package handling system. While I use and love setup_tools, it does have its shortcomings. RubyGems actually allows you to uninstall packages! Sounds simple enough, but easy_install users know this is a continual pain. RubyGems also allows much better searching and provides more information about potential packages.

I don’t recommend installing this from the Ubuntu repos. I ran into some odd problems, while the latest from RubyForge worked fine. If you do get it from the repo, be sure to install “rubygems”, not “gem” (Graphics Environment for Multimedia – PureData library)! To install from source, get the latest stable version from here on RubyForge, then:

1
2
3
4
tar xzvf rubygems-VERSION.tgz
cd rubygems-VERSION/
sudo ruby setup.rb
sudo ln -s /usr/bin/gem1.8 /usr/bin/gem

That last step is needed since the setup script doesn’t make its own symlink. This can make for wretched confusion… Here are some basic RubyGems commands and example output:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# View available commands
> gem help commands
GEM commands are:
...All the commands and descriptions thereof.
# Search for a gem by name:
> gem query --remote --name-matches flickr
*** REMOTE GEMS ***
flickr (1.0.2)
...Many more...
# Install a gem:
> sudo gem install --remote flickr
Successfully installed flickr-1.0.2
1 gem installed
# Get information on an installed gem:
> sudo gem specification flickr
--- !ruby/object:Gem::Specification
name: flickr
version: !ruby/object:Gem::Version
version: 1.0.2
...Lots more...
# Remove a gem:
> sudo gem uninstall flickr
Successfully uninstalled flickr-1.0.2

Consider adding a few Bash aliases for faster access to the basic commands:

1
2
3
4
5
6
7
8
9
10
# Look for gems with "gquery name":
alias gquery="gem query --remote --name-matches"
# Install new gems with "ginst name":
alias ginst="sudo gem install"
# Get information on gems with "ginfo name":
alias ginfo="sudo gem specification"
# Remove gems with "gdel name":
alias gdel="sudo gem uninstall"
# Update gems:
alias gupdate="sudo gem update"

Next install Rails and its merry band:

1
sudo gem install rails

The Rails Logo was created by Kevin Milden and is distrubuted under the BY-ND Creative Commons Licence.

This will install Rails, its library dependencies, rake, etc. I first installed Rails via apt-get, but ran into a small issue.

Next, to get the general sense of creating a Rails app, I recommend starting off with The Original Ruby on Rails Screencasts by Sam Stephenson and David Hansson on ShowMeDo. These are by no means comprehensive guides, but they illustrate doing some basic things with Rails, and end up with some pretty nice apps in a few minutes. You’ll be hard pressed to follow along yourself, depending on how many hours of Doom you played as a child.

To really start to grasp things, check out the Getting Started guide on the RailsGuides site. This goes through each step from nothing to a working Rails app. As you proceed, you might want additional information on various technical details. Look no further than the Rails Searchable API site. You can browse this online or download it locally. It provides a decent looking (finally for an HTML API: readable) and useable API reference for Rails. The search is amazing, and they’ve done a great job compressing a lot of information into a relatively small amount of space.

Once you begin looking for information on how to accomplish various discrete tasks, check out Railscasts. These assume some Rails knowledge, but are decent simple guides to how to add features like reliable authentication, avoiding SQL injection, generating PDFs, and a lot more (160 so far!).

I hope to post more on Ruby and Ruby on Rails as I learn and play around with them. If you are new to Ruby, check out this page. I came across it after I started going through a few tutorials, and find it amply summarizes most of the features I started to love at first blush.

Tags: , , ,
May 10, 2009 - 3:28 PM Comments (3)

More efficient HTML editing in vim

When I’m writing HTML and CSS, being forced to type out repetitive and fixed syntax, as well as remembering the particularities of the syntax and its formatting, makes me rather unhappy. Not to mention less productive. Let’s fix that.

HTML highlighting and snippets

Place html.vim in ~/.vim/syntax. Then in a file ending in .html, start a tag, e.g. <script, and press <Tab>:

Now the skeleton of the tag will be added for you, and your cursor placed in the first place you will likely need to edit. You navigate through these editable areas with <Tab>, adding your particulars. When at the last one, simply tab again to be placed after the tag. This makes properly adding tags much easier, especially for less familiar tags that are only occasionally needed. Here’s another example with a tag that is often needed and quite tedious:

The <{}> indicate places you can navigate to with tab. Once you are in one and tab past, those characters are removed. You can also add javascript.vim in ~/.vim/syntax, and the previous script will load this in appropriately.

This script also provides color highlighting for matching filetypes. When in a file ending in .html, type “:help html.vim” to see more information.

Closing tags

It’s also useful to be able to trigger close tags, if you started typing them manually. I find this most troublesome when editing HTML and XML. To add this capability, download the latest version of this script in ~/.vim/ftplugin/html.vim.

Then when you have started a tag and wish to close it (in files ending in .html), press <Ctrl> + <_> (that’s an underscore, so Shift + -). This will insert the close tag matching the closest previous open tag under the cursor:

Formatting

The following directives in your ~/.vimrc will make proper formatting automatic for CSS and HTML files:

1
2
3
4
5
6
7
8
" for CSS, also have things in braces indented:
autocmd FileType css set smartindent
" for HTML, generally format text, but if a long line has been created
" leave it alone when editing:
autocmd FileType html set formatoptions+=tl
" for both CSS and HTML, use genuine tab characters for
" indentation, to make files a few bytes smaller:
autocmd FileType html,css set noexpandtab tabstop=2

Watch your sppellinng

While writing HTML code involves plenty of code, you also end up writing text that others might just see. Don’t let mistakes in spelling slip by any more than you would let syntax errors. Add this to your ~/.vimrc to be able to toggle display of spelling errors with <F7>:

1
2
" F7 to toggle spell-checking
map <silent> <F7> :set nospell!<CR>:set nospell?<CR>

This post explains more advanced commands, such as bringing up suggestions, as well as customizing the formatting.

Tidy up

Being able to clean up your HTML code with a single command, making it more standards compliant and easier to read, is a great boon. To get this working for vim, first install tidy, an HTML syntax checker and reformatted. On Ubuntu, it’s available in the repository as “tidy”.

After that’s installed, add the following lines to your ~/.vimrc:

1
2
setlocal makeprg=tidy\ -quiet\ -errors\ %
setlocal errorformat=line\ %l\ column\ %v\ -\ %m

Now when in files ending in .html, type “:make”. All the errors Tidy found will be shown a list. Then press Enter to be taken back into the file, with the cursor on the first error, which is described at the bottom of the window:

Press “:cn” to move to the next error found, and “:cp” for the previous. As you encounter each error, you can fix it and move on. Once done, save the file, and run “:make” again to verify everything is fixed.

Tags: , , , , ,
May 9, 2009 - 4:15 PM No Comments

« Older Entries

Twitter links powered by Tweet This v1.6.1, a WordPress plugin for Twitter.