I have been using the quite capable logging module for all of my logging in Python programs for some time. After trying a number of different options and formats, I have stabilized my setup into what I will describe below. I have found it effective in providing helpful, concise logging for a wide range of tasks.
First, up in the import section of your script, add:
1 | import logging |
This module has been part of the standard library since Python 2.3, so it should be found. If not, upgrade Python! Now you need to setup for your particular logging needs. I like to place the logging setup section after the imports, before the function definitions:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 | # Setup logging logger = logging.getLogger(sys.argv[0]) logger.setLevel(logging.DEBUG) # Use file output for production logging: logfilename = "/var/log/myapp/my_super_app.log" filelog = logging.FileHandler(logfilename, 'a') filelog.setLevel(logging.INFO) # Use console for development logging: conlog = logging.StreamHandler() conlog.setLevel(logging.DEBUG) # Specify log formatting: formatter = logging.Formatter("%(asctime)s - %(name)s - %(lineno)s - \ %(levelname)s - %(message)s") conlog.setFormatter(formatter) filelog.setFormatter(formatter) # Add console log to logger logger.addHandler(conlog) logger.addHandler(filelog) |
That’s all the setup you’ll likely need. From the top, this first initializes a Logger object, providing it the name of the current script as the name of the log. This makes it easy to differentiate logs from similar scripts with different names. Next we define different types of handlers that will log various types of messages. The logging module provides a wide array of log handlers, listed here under “Logging Levels”. You can log to a file (which I use for production programs), to console (for debugging), or to more exotic places if you so desire, such as directly to a socket, syslog, and more.
So after we define a file log, and what handler we want to use for it, we set the level of log messages it will record, in this case of INFO level and up. (These various levels are not entirely standard. See the section entitled “Levels” in PEP 282 for more information, in case you are not familiar with the logging module’s particular levels). For lower level messages, we add a console handler (goes to STDOUT).
Then we define the format for our log messages, and assign that format to the console and file loggers. There is quite a range of variables we have access to in the format of the logs. They are listed and described here. I have found this format to be the most informative without overloading the display:
1 | 2009-03-15 02:00:12,077 - ./test.py - 24 - DEBUG - I am a test message! |
The scale of the timestamp is small enough to differentiate between quick changes, we immediately know what file and what line in that file is causing the message to appear, what level it is, and finally its contents. The same format is applied to the file handler, which will be written to at the same time the console log displays, providing a permanent record.
Using the logging module removes the need for test print statements, providing an informative framework that is easily shifted into a production mode. You can add as much information as you want to see in development by handling DEBUG level messages, and sending them to console, then raise the level of message shown to INFO once ready to deploy. For additional information, see this O’Reilly article.
No related posts.
Related posts brought to you by Yet Another Related Posts Plugin.
Pingback: Running MySQL queries in Python | tail -f findings.out
Pingback: Script to find longest running events in log files | tail -f findings.out
Pingback: Send events to Zenoss from scripts | tail -f findings.out
This is great! Thanks for the article. I am just starting python and this is a big help.
Thank you very much for putting me on python logging track in no time.
Bests
Thank you for an informative post. The logging module would be much simpler to use if it had
better constructors. The syntax ought to look like this:
format = “%(asctime)s – %(name)s – %(lineno)s – %(levelname)s – %(message)s”
logfilename = “/var/log/myapp/my_super_app.log”
filelog = logging.FileHandler(logfilename, ‘a’, level= INFO, format=format)
conlog = logging.StreamHandler(level=DEBUG, format=format)
logger = logging.getLogger(sys.argv[0], level = DEBUG, handlers=[filelog,conlog])
Thank you , finally simple and working example.