tail -f findings.out

100th blog post!

Filed under Meta  |  by Samuel Huckins  |  Date Posted: January 20, 2009 - 2:32 AM

This is my 100th post to this blog! And there’s a lot more where that came from!

Token graphage:

A domain change (formerly: assistedsilicon.blogspot.com), a platform change (Blogger to Wordpress), 805.5 themes, and coming on two years later, the topics are mostly the same:

  • Linux: vim config, bash aliases, useful scripts, hidden options, and lots more
  • MySQL: How to do things you will likely need to do, and do them better
  • Python: Resources, recipes, and tricks for using this incredible language
  • Vista: How to live with it, partially
  • Occasional other tech/geek items

On the graph:

  • The nine month lacuna in 2007-08 is to be blamed on World of Warcraft.
  • The spike near the end of 2008 corresponded to when I started using Windows Vista (at work, under duress). Discoveries do seem to increase in wartime…

For some reason I couldn’t wrangle my data into a graph I liked in OO Spreadsheet. So, of course, I made the graph in Python! To get the post data, I ran this against my Wordpress MySQL DB:

1
2
3
4
mysql -u root -p -e "use WORDPRESS_DB; \
select date(post_date) as date, post_title as title from wp_posts \
where post_type='post' and post_status='publish' \
group by post_date order by date(post_date) asc;" > posts.csv

This grabs the dates and titles for all actual published posts, grouping and ordering them on the date. Then I put that output file into a short ugly script for graphing:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
#!/usr/bin/env python
try:
    from pylab import *
except:
    graphing = False
import pprint
import datetime

# Read in the file of titles and dates:
f = open('posts.csv')
flines = []
for line in f: flines.append(line.split('\t'))
# Create dict of year-month and counts therein:
date_counts = {}
for line in flines:
    if date_counts.has_key(line[0][0:7]):
        date_counts[line[0][0:7]] += 1
    else:
        date_counts[line[0][0:7]] = 1
# Make a list of the counts, dates and sort them:
counts = date_counts.items()
counts.sort()
# Turn the partial date strings into date objects for graphing:
dates = [key for key, value in counts]
dated_days = []
for day in dates:
    year = int(day[0:4])
    month = int(day[5:7])
    day = 1
    date = datetime.date(year, month, day)
    dated_days.append(date)
counts = [ value for key, value in counts]
# Set labels
x = xlabel('Date')
setp(x, fontweight='bold')
y = ylabel('Posts')
setp(y, fontweight='bold')
# Subplot to handle date positioning
ax = subplot(111)
labels = ax.get_xticklabels()
setp(labels, fontweight='bold', rotation=30, fontsize=10)
# Plot as dates
plotted = plot(dated_days, counts, '--')
setp(plotted, marker='s')
title('Posts over time')
grid(True)

Latest version in my Code Trac.

Tags: , ,
January 20, 2009 - 2:32 AM No Comments

Easy and informative: Call graphs in Python

Filed under Programming  |  by Samuel Huckins  |  Date Posted: January 20, 2009 - 12:28 AM

While I was looking for different graphing modules in Python, I came across pycallgraph. This module allows you to create a graph of all the different calls that occur between the time you initiate it and when you tell it to make a graph. Not only that, but it shows you the number of times each was called, the total time the calls took, and how they all connected!

Running this on an entire script, much less a large app, would create a huge unreadable mess that would have value only in making an awesome poster. However, showing the calls and time taken for a single function, or part of a function, may serve well in pointing out where to look in speeding up your application. Now you can make a diagram of all the connections in your bottleneck code, colored to emphasize what took the longest (hotter colors took longest).

To use pycallgraph, you need to:

  • 1
    sudo easy_install pycallgraph
  • 1
    sudo apt-get install python-pygraphviz

This will get all the dependencies. Then in your code, right above where you want to start tracking, add:

import pycallgraph
pycallgraph.start_trace()

Then right after where you want to stop tracking, add:

pycallgraph.make_dot_graph('test.png')

You can pass various options to these, but just these three lines will produce an image file, test.png, in the folder where you ran the script. I added these lines around the main function of the follow-largest-directories script I blogged about recently, right after all the option parsing stuff. Here was the result:

Tags: , , ,
January 20, 2009 - 12:28 AM No Comments

Twitter links powered by Tweet This v1.6.1, a WordPress plugin for Twitter.