Friday, August 19, 2005

Managing DNS zone files with dnspython

I've been using dnspython lately for transferring some DNS zone files from one name server to another. I found the package extremely useful, but poorly documented, so I decided to write this post as a mini-tutorial on using dnspython.

Running DNS queries

This is one of the things that's clearly spelled out on the Examples page. Here's how to run a DNS query to get the mail servers (MX records) for dnspython.org:

import dns.resolver

answers = dns.resolver.query('dnspython.org', 'MX')
for rdata in answers:
print 'Host', rdata.exchange, 'has preference', rdata.preference
To run other types of queries, for example for IP addresses (A records) or name servers (NS records), replace MX with the desired record type (A, NS, etc.)

Reading a DNS zone from a file

In dnspython, a DNS zone is available as a Zone object. Assume you have the following DNS zone file called db.example.com:

$TTL 36000
example.com. IN SOA ns1.example.com. hostmaster.example.com. (
2005081201 ; serial
28800 ; refresh (8 hours)
1800 ; retry (30 mins)
2592000 ; expire (30 days)
86400 ) ; minimum (1 day)

example.com. 86400 NS ns1.example.com.
example.com. 86400 NS ns2.example.com.
example.com. 86400 MX 10 mail.example.com.
example.com. 86400 MX 20 mail2.example.com.
example.com. 86400 A 192.168.10.10
ns1.example.com. 86400 A 192.168.1.10
ns2.example.com. 86400 A 192.168.1.20
mail.example.com. 86400 A 192.168.2.10
mail2.example.com. 86400 A 192.168.2.20
www2.example.com. 86400 A 192.168.10.20
www.example.com. 86400 CNAME example.com.
ftp.example.com. 86400 CNAME example.com.
webmail.example.com. 86400 CNAME example.com.

To have dnspython read this file into a Zone object, you can use this code:

import dns.zone
from dns.exception import DNSException

domain = "example.com"
print "Getting zone object for domain", domain
zone_file = "db.%s" % domain

try:
zone = dns.zone.from_file(zone_file, domain)
print "Zone origin:", zone.origin
except DNSException, e:
print e.__class__, e
A zone can be viewed as a dictionary mapping names to nodes; dnspython uses by default name representations which are relative to the 'origin' of the zone. In our zone file, 'example.com' is the origin of the zone, and it gets the special name '@'. A name such as www.example.com is exposed by default as 'www'.

A name corresponds to a node, and a node contains a collection of record dataset, or rdatasets. A record dataset contains all the records of a given type. In our example, the '@' node corresponding to the zone origin contains 4 rdatasets, one for each record type that we have: SOA, NS, MX and A. The NS rdataset contains a set of rdatas, which are the individual records of type NS. The rdata class has subclasses for all the possible record types, and each subclass contains information specific to that record type.

Enough talking, here is some code that will hopefully make the previous discussion a bit clearer:

import dns.zone
from dns.exception import DNSException
from dns.rdataclass import *
from dns.rdatatype import *

domain = "example.com"
print "Getting zone object for domain", domain
zone_file = "db.%s" % domain

try:
zone = dns.zone.from_file(zone_file, domain)
print "Zone origin:", zone.origin
for name, node in zone.nodes.items():
rdatasets = node.rdatasets
print "\n**** BEGIN NODE ****"
print "node name:", name
for rdataset in rdatasets:
print "--- BEGIN RDATASET ---"
print "rdataset string representation:", rdataset
print "rdataset rdclass:", rdataset.rdclass
print "rdataset rdtype:", rdataset.rdtype
print "rdataset ttl:", rdataset.ttl
print "rdataset has following rdata:"
for rdata in rdataset:
print "-- BEGIN RDATA --"
print "rdata string representation:", rdata
if rdataset.rdtype == SOA:
print "** SOA-specific rdata **"
print "expire:", rdata.expire
print "minimum:", rdata.minimum
print "mname:", rdata.mname
print "refresh:", rdata.refresh
print "retry:", rdata.retry
print "rname:", rdata.rname
print "serial:", rdata.serial
if rdataset.rdtype == MX:
print "** MX-specific rdata **"
print "exchange:", rdata.exchange
print "preference:", rdata.preference
if rdataset.rdtype == NS:
print "** NS-specific rdata **"
print "target:", rdata.target
if rdataset.rdtype == CNAME:
print "** CNAME-specific rdata **"
print "target:", rdata.target
if rdataset.rdtype == A:
print "** A-specific rdata **"
print "address:", rdata.address
except DNSException, e:
print e.__class__, e

When run against db.example.com, the code above produces this output.

Modifying a DNS zone file

Let's see how to add, delete and change records in our example.com zone file. dnspython offers several different ways to get to a record if you know its name or its type.

Here's how to modify the SOA record and increase its serial number, a very common operation for anybody who maintains DNS zones. I use the iterate_rdatas method of the Zone class, which is handy in this case, since we know that the rdataset actually contains one rdata of type SOA:
   
for (name, ttl, rdata) in zone.iterate_rdatas(SOA):
serial = rdata.serial
new_serial = serial + 1
print "Changing SOA serial from %d to %d" %(serial, new_serial)
rdata.serial = new_serial


Here's how to delete a record by its name. I use the delete_node method of the Zone class:

node_delete = "www2"
print "Deleting node", node_delete
zone.delete_node(node_delete)
Here's how to change attributes of existing records. I use the find_rdataset method of the Zone class, which returns a rdataset containing the records I want to change. In the first section of the following code, I'm changing the IP address of 'mail', and in the second section I'm changing the TTL for all the NS records corresponding to the zone origin '@':

A_change = "mail"
new_IP = "192.168.2.100"
print "Changing A record for", A_change, "to", new_IP
rdataset = zone.find_rdataset(A_change, rdtype=A)
for rdata in rdataset:
rdata.address = new_IP

rdataset = zone.find_rdataset("@", rdtype=NS)
new_ttl = rdataset.ttl / 2
print "Changing TTL for NS records to", new_ttl
rdataset.ttl = new_ttl

Here's how to add records to the zone file. The find_rdataset method can be used in this case too, with the create parameter set to True, in which case it creates a new rdataset if it doesn't already exist. Individual rdata objects are then created by instantiating their corresponding classes with the correct parameters -- such as rdata = dns.rdtypes.IN.A.A(IN, A, address="192.168.10.30").

I show here how to add records of type A, CNAME, NS and MX:
  A_add = "www3"
print "Adding record of type A:", A_add
rdataset = zone.find_rdataset(A_add, rdtype=A, create=True)
rdata = dns.rdtypes.IN.A.A(IN, A, address="192.168.10.30")
rdataset.add(rdata, ttl=86400)

CNAME_add = "www3_alias"
target = dns.name.Name(("www3",))
print "Adding record of type CNAME:", CNAME_add
rdataset = zone.find_rdataset(CNAME_add, rdtype=CNAME, create=True)
rdata = dns.rdtypes.ANY.CNAME.CNAME(IN, CNAME, target)
rdataset.add(rdata, ttl=86400)

A_add = "ns3"
print "Adding record of type A:", A_add
rdataset = zone.find_rdataset(A_add, rdtype=A, create=True)
rdata = dns.rdtypes.IN.A.A(IN, A, address="192.168.1.30")
rdataset.add(rdata, ttl=86400)

NS_add = "@"
target = dns.name.Name(("ns3",))
print "Adding record of type NS:", NS_add
rdataset = zone.find_rdataset(NS_add, rdtype=NS, create=True)
rdata = dns.rdtypes.ANY.NS.NS(IN, NS, target)
rdataset.add(rdata, ttl=86400)

A_add = "mail3"
print "Adding record of type A:", A_add
rdataset = zone.find_rdataset(A_add, rdtype=A, create=True)
rdata = dns.rdtypes.IN.A.A(IN, A, address="192.168.2.30")
rdataset.add(rdata, ttl=86400)

MX_add = "@"
exchange = dns.name.Name(("mail3",))
preference = 30
print "Adding record of type MX:", MX_add
rdataset = zone.find_rdataset(MX_add, rdtype=MX, create=True)
rdata = dns.rdtypes.ANY.MX.MX(IN, MX, preference, exchange)
rdataset.add(rdata, ttl=86400)

Finally, after modifying the zone file via the zone object, it's time to write it back to disk. This is easily accomplished with dnspython via the to_file method. I chose to write the modified zone to a new file, so that I have my original zone available for other tests:

new_zone_file = "new.db.%s" % domain
print "Writing modified zone to file %s" % new_zone_file
zone.to_file(new_zone_file)

The new zone file looks something like this (note that all names have been relativized from the origin):

@ 36000 IN SOA ns1 hostmaster 2005081202 28800 1800 2592000 86400
@ 43200 IN NS ns1
@ 43200 IN NS ns2
@ 43200 IN NS ns3
@ 86400 IN MX 10 mail
@ 86400 IN MX 20 mail2
@ 86400 IN MX 30 mail3
@ 86400 IN A 192.168.10.10
ftp 86400 IN CNAME @
mail 86400 IN A 192.168.2.100
mail2 86400 IN A 192.168.2.20
mail3 86400 IN A 192.168.2.30
ns1 86400 IN A 192.168.1.10
ns2 86400 IN A 192.168.1.20
ns3 86400 IN A 192.168.1.30
webmail 86400 IN CNAME @
www 86400 IN CNAME @
www3 86400 IN A 192.168.10.30
www3_alias 86400 IN CNAME www3

Although it looks much different from the original db.example.com file, this file is also a valid DNS zone -- I tested it by having my DNS server load it.

Obtaining a DNS zone via a zone transfer

This is also easily done in dnspython via the from_xfr function of the zone module. Here's how to do a zone transfer for dnspython.org, trying all the name servers for that domain one by one:

import dns.resolver
import dns.query
import dns.zone
from dns.exception import DNSException
from dns.rdataclass import *
from dns.rdatatype import *

domain = "dnspython.org"
print "Getting NS records for", domain
answers = dns.resolver.query(domain, 'NS')
ns = []
for rdata in answers:
n = str(rdata)
print "Found name server:", n
ns.append(n)

for n in ns:
print "\nTrying a zone transfer for %s from name server %s" % (domain, n)
try:
zone = dns.zone.from_xfr(dns.query.xfr(n, domain))
except DNSException, e:
print e.__class__, e


Once we obtain the zone object, we can then manipulate it in exactly the same way as when we obtained it from a file.

Various ways to iterate through DNS records

Here are some other snippets of code that show how to iterate through records of different types assuming we retrieved a zone object from a file or via a zone transfer:

print "\nALL 'IN' RECORDS EXCEPT 'SOA' and 'TXT':"
for name, node in zone.nodes.items():
rdatasets = node.rdatasets
for rdataset in rdatasets:
if rdataset.rdclass != IN or rdataset.rdtype in [SOA, TXT]:
continue
print name, rdataset

print "\nGET_RDATASET('A'):"
for name, node in zone.nodes.items():
rdataset = node.get_rdataset(rdclass=IN, rdtype=A)
if not rdataset:
continue
for rdataset in rdataset:
print name, rdataset

print "\nITERATE_RDATAS('A'):"
for (name, ttl, rdata) in zone.iterate_rdatas('A'):
print name, ttl, rdata

print "\nITERATE_RDATAS('MX'):"
for (name, ttl, rdata) in zone.iterate_rdatas('MX'):
print name, ttl, rdata

print "\nITERATE_RDATAS('CNAME'):"
for (name, ttl, rdata) in zone.iterate_rdatas('CNAME'):
print name, ttl, rdata
You can find the code referenced in this post in these 2 modules: zonemgmt.py and zone_transfer.py.

Monday, August 08, 2005

Agile documentation in the Django project

A while ago I wrote a post called "Agile documentation with doctest and epydoc". The main idea was to use unit tests as "executable documentation"; I showed in particular how combining doctest-based unit tests with a documentation system such as epydoc can result in up-to-date documentation that is synchronized with the code. This type of documentation not only shows the various modules, classes, methods, function, variables exposed by the code, but -- more importantly -- it also provides examples of how the code API gets used in "real life" via the unit tests.

I'm happy to see the Django team take a similar approach in their project. They announced on the project blog that API usage examples for Django models are available and are automatically generated from the doctest-based unit tests written for the model functionality. For example, a test module such as tests/testapp/models/basic.py gets automatically rendered into the 'Bare-bones model' API usage page. The basic.py file contains almost exclusively doctests in the form of a string called API_TESTS. The rest of the file contains some simple markers that are interpreted into HTML headers and such. Nothing fancy, but the result is striking.

I wish more projects would adopt this style of automatically generating documentation for their APIs from their unit test code. It can only help speed up their adoption. As an example, I wish the dnspython project had more examples of how to use the API it offers. That project does have epydoc-generated documentation, but if it also showed how the API actually gets used (via unit tests preferably), it would help its users avoid a lot of hair-pulling. Don't get me wrong, I think dnspython offers an incredibly useful API and I intend to post about some of my experiences using it, but it does require you to dig and sweat in order to uncover all its intricacies.

Anyway, kudos to the Django team for getting stuff right.

Monday, August 01, 2005

White-box vs. black-box testing

As I mentioned in my previous post, there's an ongoing discussion on the agile-testing mailing list on the merits of white-box vs. black-box testing. I had a lively exchange of opinions on this theme with Ron Jeffries. If you read my "Quick black-box testing example" post, you'll see the example of an application under test posted by Ron, as well as a list of back-box test activities and scenarios that I posted in reply. Ron questioned most of these black-box test scenarios, on the grounds that they provide little value to the overall testing process. In fact, I came away with the conclusion that Ron values black-box testing very little. He is of the opinion that white-box testing in the form of TDD is pretty much sufficient for the application to be rock-solid and as much bug-free as any piece of software can hope to be.

I never had the chance to work on an agile team, so I can't really tell if Ron's assertion is true or not. But my personal opinion is that there is no way developers doing TDD can catch several classes of bugs that are outside of their code-only realm. I'm thinking most of all about the various quality criteria categories, also known as 'ilities', popularized by James Bach. Here are some of them: usability, installability, compatibility, supportability, maintainability, portability, localizability. All these are qualities that are very hard to test in a white-box way. They all involve interactions with the operating system, with the hardware, with the other applications running on the machine hosting the AUT. To this list I would add performance/stress/load testing, security testing, error recoverability testing. I don't see how you can properly test all these things if you don't do black-box testing in addition to white-box type testing.

In fact, there's an important psychological distinction between developers doing TDD and 'traditional' testers doing mostly black-box testing. A developer thinks "This is my code. It works fine. In fact, I'm really proud of it.", while a tester is more likely to think "This code has some really nasty bugs. Let me discover them before our customer does." These two approaches are complementary. You can't perform just one at the expense of the other, or else your overall code quality will suffer. You need to build code with pride before you try to break it in various devious ways.

Here's one more argument from Ron as to why white-box testing is more valuable than black-box testing:

To try to simplify: the search method in question has been augmented with an integer "hint" that is used to say where in the large table we should start our search. The idea is that by giving a hint, it might speed up the search, but the search must always work even if the hint is bad.

The question I was asking was how we would test the hinting aspect.

I expect questions to arise such as those Michael Bolton would suggest, including perhaps:

What if the hint is negative?
What if the hint is after the match?
What if the hint is bigger than the size of the table?
What if integers are actually made of cheese?
What if there are more records in the table than a 32-bit int?

Then, I propose to display the code, which will include, at the front, some lines like this:

if (hint < 1) hint = 0;
if (hint > table.size) hint = 0;

Then, I propose to point out that if we know that code is there, there are a couple of tests we can save. Therefore white box testing can help make testing more efficient, QED.

My counter-argument was this: what if you mistakenly build a new release of your software out of some old revision of the source code, a revision which doesn't contain the first 2 lines of the search method? Presumably the old version of the code was TDD-ed, but since the 2 lines weren't there, we didn't have unit tests for them either. So if you didn't have black-box tests exercising those values of the hint argument, you'd let an important bug escape out in the wild. I don't think it's that expensive to create automated tests that verify the behavior of the search method with various well-chosen values of the hint argument. Having such a test harness in place goes a long way in protecting against admittedly weird situations such as the 'old code revision' I described.

In fact, as Amir Kolsky remarked on the agile-testing list, TDD itself can be seen as black-box testing, since when we unit test some functionality, we usually test the behavior of that piece of code and not its implementation, thus we're not really doing white-box testing. To this, Ron Jeffries and Ilja Preuss replied that in TDD, you write the next test with an eye on the existing code. In fact, you write the next test so that the next piece of functionality for the existing code fails. Then you make it pass, and so on. So you're really looking at both the internal implementation of the code and at its interfaces, as exposed in your unit tests. At this point, it seems to me that we're splitting hairs. Maybe we should talk about developer/code producer testing vs. non-developer/code consumer testing. In fact, I just read this morning a very nice blog post from Jonathan Kohl on a similar topic: "Testing an application in layers". Jonathan talks about manual vs. automated testing (another hotly debated topic on the agile-testing mailing list), but many of the ideas in his post can be applied to the white-box vs. black-box discussion.

Thursday, July 28, 2005

Quick black box testing example

There's an ongoing debate on the agile-testing mailing list on whether it's better to have a 'black box' or a 'white box' view into the system under test. Some are of the opinion that black boxes are easier to test, while others (Ron Jeffries in particular) say that one would like to 'open up' one's boxes, especially in an agile environment. I suspect that the answer, as always, is somewhere in the middle -- both white-box and black-box testing are critical and valuable in their own right.

I think that it's in combining both types of tests that developers and testers will find the confidence that the software under test is stable and relatively free of bugs. Developers do white-box testing via unit tests, while testers do mostly black-box testing (or maybe gray-box, since they usually do have some insight into the inner workings of the application) via functional, integration and system testing. Let's not forget load/performance/stress testing too...They too can be viewed as white-box (mostly in the case of performance testing) vs. black-box (load/stress testing), as I wrote in a previous post.

I want to include in this post my answer to a little example posted by Ron Jeffries. Here's what he wrote:

Let's explore a simple example. Suppose we have an application that includes an interface (method) whose purpose is to find a "matching" record in a collection, if one exists. If none exists, the method is to return null.

The collection is large. Some users of this method have partial knowledge of the collection's order, so that they know that the record they want, if it is in there at all, occurs at or after some integer index in the collection.

So the method accepts a value, let's say a string /find/, to match the record on, and an integer /hint/, to be used as a hint to start the search. The first record in the table is numbered zero. The largest meaningful /hint/ value is therefore N-1, where N is the number of records in the table.

We want the search to always find a record if one exists, so that if /hint/ is wrong, but /find/ is in some record, we must still return a matching record, not null.

Now then. Assuming a black box, what questions do we want to ask, what tests do we want to write, against our method

public record search(string find, int hint)?

And here's my answer:

I'll take a quick stab at it. Here's what I'd start by doing (emphasis on start):

1. Generate various data sets to run the 'search' method against.

1a. Vary the number of items in the collection: create collections with 0, 1, 10, 100, 1000, 10000, 100000, 1 million items for starters; it may be the case that we hit an operating system limit at some point, for example if the items are files in the same directory (ever done an ls only to get back a message like "too many arguments"?)

1b. For each collection in 1a., generate several orderings: increasing order, decreasing order, random, maybe some other statistical distributions.

1c. Vary the length of the names of the items in the collection: create collections with 0, 1, 10, 100, 1000 items, where the names of the items are generated randomly with lengths between 1 and 1000 (arbitrary limit, which may change as we progress testing).

1d. Generate item names with 'weird' characters (especially /, \, :, ; -- since they tend to be used as separators by the OS).

1e. Generate item names that are Unicode strings.

2. Run (and time) the 'search' method against the various collections generated in 1. Make sure you cover cases such as:

2a. The item we search for is not in the collection: verify that the search method returns Null.

2b. The item we search for is in position p, where p can be 0, N/2, N-1, N.

2c. For each case in 2b, specify a hint of 0, p-1, p, p+1, N-1: verify that in all combinations of 2b and 2c, the search method returns the item in position p.

2d. Investigate the effect of item naming on the search. Does the search method work correctly when item names keep getting longer? When the item names contain 'weird' or Unicode characters?

2e. Graph the running time of the search method against collection size, when the item is or is not in the collection (so you generate 2 graphs). See if there is any anomaly.

2f. Run the tests in 2a-2d in a loop, to see if the search method produces a memory leak.

2g. Monitor various OS parameters (via top, vmstat, Windows PerfMon) to see how well-behaved the search functionality is in regards to the resources on that machine.

2h. See how the search method behaves when other resource-intensive processes are running on that machine (CPU-, disk-, memory-, network- intensive).

If the collection of records is kept in a database, then I can imagine a host of other stuff to test that is database-related. Same if the collection is retrieved over the network.

As I said, this is just an initial stab at testing the search method. I'm sure people can come up with many more things to test. But I think this provides a pretty solid base and a pretty good automated test suite for the AUT.

I can think of many more tests that should be run if the search application talks to a database, or if it retrieves the search results via a Web service for example. I guess this all shows that a tester's life is not easy :-) -- but this is all exciting stuff at the same time!

Sunday, July 24, 2005

Django cheat sheet

Courtesy of James: Django cheat sheet. I went trough the first 2 parts of the Django tutorial and I have to say I'm very impressed. Can't wait to give it a try on a real Web application.

Friday, July 22, 2005

Slides from 'py library overview' presentation

I presented an overview of the py library last night at our SoCal Piggies meeting. Although I didn't cover all the tools in the py library, I hope I managed to heighten the interest in this very useful collection of modules. You can find the slides here. Kudos again to Holger Krekel and Armin Rigo, the main guys behind the py lib.

And while we're on this subject, let's make py.test the official unit test framework for Django!!! (see the open ticket on this topic)

Friday, July 15, 2005

Installing Python 2.4.1 and cx_Oracle on AIX

I just went through the pain of getting the cx_Oracle module to work on an AIX 5.1 server running Oracle 9i, so I thought I'd jot down what I did, for future reference.

First of all, I had ORACLE_HOME set to /oracle/OraHome1.

1. Downloaded the rpm.rte package from the AIX Toolbox Download site.
2. Installed rpm.rte via smit.
3. Downloaded (from the same AIX Toolbox Download site) and installed the following RPM packages, in this order:
rpm -hi gcc-3.3.2-5.aix5.1.ppc.rpm
rpm -hi libgcc-3.3.2-5.aix5.1.ppc.rpm
rpm -hi libstdcplusplus-3.3.2-5.aix5.1.ppc.rpm
rpm -hi libstdcplusplus-devel-3.3.2-5.aix5.1.ppc.rpm
rpm -hi gcc-cplusplus-3.3.2-5.aix5.1.ppc.rpm
4. Made a symlink from gcc to cc_r, since many configuration scripts find cc_r as the compiler of choice on AIX, and I did not have it on my server.
ln -s /usr/bin/gcc /usr/bin/cc_r
5. Downloaded Python-2.4.1 from python.org.
6. Installed Python-2.4.1 (note that the vanilla ./configure failed, so I needed to run it with --disable-ipv6):
gunzip Python-2.4.1.tgz
tar xvf Python-2.4.1.tar
cd Python-2.4.1
./configure --disable-ipv6
make
make install
7. Downloaded cx_Oracle-4.1 from sourceforge.net.
8. Installed cx_Oracle-4.1 (note that I indicated the full path to python, since there was another older python version on that AIX server):
bash-2.05a# /usr/local/bin/python setup.py install
running install
running build
running build_ext
building 'cx_Oracle' extension
creating build
creating build/temp.aix-5.1-2.4
cc_r -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I/oracle/OraHome1/rdbms/demo -I/oracle/OraHome1/rdbms/public -I/oracle/OraHome1/network/public -I/usr/local/include/python2.4 -c cx_Oracle.c -o build/temp.aix-5.1-2.4/cx_Oracle.o -DBUILD_TIME="July 15, 2005 14:49:28"
In file included from /oracle/OraHome1/rdbms/demo/oci.h:2138,
from cx_Oracle.c:9:
/oracle/OraHome1/rdbms/demo/oci1.h:148: warning: function declaration isn't a prototype
In file included from /oracle/OraHome1/rdbms/demo/ociap.h:190,
from /oracle/OraHome1/rdbms/demo/oci.h:2163,
from cx_Oracle.c:9:
/oracle/OraHome1/rdbms/public/nzt.h:667: warning: function declaration isn't a prototype
/oracle/OraHome1/rdbms/public/nzt.h:2655: warning: function declaration isn't a prototype
/oracle/OraHome1/rdbms/public/nzt.h:2664: warning: function declaration isn't a prototype
/oracle/OraHome1/rdbms/public/nzt.h:2674: warning: function declaration isn't a prototype
/oracle/OraHome1/rdbms/public/nzt.h:2683: warning: function declaration isn't a prototype
/oracle/OraHome1/rdbms/public/nzt.h:2692: warning: function declaration isn't a prototype
/oracle/OraHome1/rdbms/public/nzt.h:2701: warning: function declaration isn't a prototype
/oracle/OraHome1/rdbms/public/nzt.h:2709: warning: function declaration isn't a prototype
/oracle/OraHome1/rdbms/public/nzt.h:2719: warning: function declaration isn't a prototype
In file included from /oracle/OraHome1/rdbms/demo/oci.h:2163,
from cx_Oracle.c:9:
/oracle/OraHome1/rdbms/demo/ociap.h:6888: warning: function declaration isn't a prototype
/oracle/OraHome1/rdbms/demo/ociap.h:9790: warning: function declaration isn't a prototype
/oracle/OraHome1/rdbms/demo/ociap.h:9796: warning: function declaration isn't a prototype
In file included from Variable.c:93,
from Cursor.c:211,
from Connection.c:303,
from SessionPool.c:132,
from cx_Oracle.c:73:
DateTimeVar.c: In function `DateTimeVar_SetValue':
DateTimeVar.c:81: warning: unused variable `status'
creating build/lib.aix-5.1-2.4
/usr/local/lib/python2.4/config/ld_so_aix cc_r -pthread -bI:/usr/local/lib/python2.4/config/python.exp build/temp.aix-5.1-2.4/cx_Oracle.o -L/oracle/OraHome1/lib -lclntsh -o build/lib.aix-5.1-2.4/cx_Oracle.so -s
ld: 0711-317 ERROR: Undefined symbol: .OCINumberFromInt
ld: 0711-317 ERROR: Undefined symbol: .OCINumberFromReal
ld: 0711-317 ERROR: Undefined symbol: .OCINumberFromText
ld: 0711-317 ERROR: Undefined symbol: .OCINumberToReal
ld: 0711-317 ERROR: Undefined symbol: .OCINumberToText
ld: 0711-317 ERROR: Undefined symbol: .OCINumberToInt
ld: 0711-317 ERROR: Undefined symbol: .OCIParamGet
ld: 0711-317 ERROR: Undefined symbol: .OCIDescriptorFree
ld: 0711-317 ERROR: Undefined symbol: .OCIAttrGet
ld: 0711-317 ERROR: Undefined symbol: .OCIStmtExecute
ld: 0711-317 ERROR: Undefined symbol: .OCISessionGet
ld: 0711-317 ERROR: Undefined symbol: .OCIServerDetach
ld: 0711-317 ERROR: Undefined symbol: .OCITransRollback
ld: 0711-317 ERROR: Undefined symbol: .OCISessionEnd
ld: 0711-317 ERROR: Undefined symbol: .OCISessionRelease
ld: 0711-317 ERROR: Undefined symbol: .OCIHandleFree
ld: 0711-317 ERROR: Undefined symbol: .OCIHandleAlloc
ld: 0711-317 ERROR: Undefined symbol: .OCIAttrSet
ld: 0711-317 ERROR: Undefined symbol: .OCITransStart
ld: 0711-317 ERROR: Undefined symbol: .OCISessionPoolCreate
ld: 0711-317 ERROR: Undefined symbol: .OCIErrorGet
ld: 0711-317 ERROR: Undefined symbol: .OCIEnvCreate
ld: 0711-317 ERROR: Undefined symbol: .OCINlsNumericInfoGet
ld: 0711-317 ERROR: Undefined symbol: .OCISessionPoolDestroy
ld: 0711-317 ERROR: Undefined symbol: .OCITransCommit
ld: 0711-317 ERROR: Undefined symbol: .OCITransPrepare
ld: 0711-317 ERROR: Undefined symbol: .OCIBreak
ld: 0711-317 ERROR: Undefined symbol: .OCIUserCallbackRegister
ld: 0711-317 ERROR: Undefined symbol: .OCIUserCallbackGet
ld: 0711-317 ERROR: Undefined symbol: .OCIServerAttach
ld: 0711-317 ERROR: Undefined symbol: .OCISessionBegin
ld: 0711-317 ERROR: Undefined symbol: .OCIStmtRelease
ld: 0711-317 ERROR: Undefined symbol: .OCIDescriptorAlloc
ld: 0711-317 ERROR: Undefined symbol: .OCIDateTimeConstruct
ld: 0711-317 ERROR: Undefined symbol: .OCIDateTimeCheck
ld: 0711-317 ERROR: Undefined symbol: .OCIDateTimeGetDate
ld: 0711-317 ERROR: Undefined symbol: .OCIDateTimeGetTime
ld: 0711-317 ERROR: Undefined symbol: .OCILobGetLength
ld: 0711-317 ERROR: Undefined symbol: .OCILobWrite
ld: 0711-317 ERROR: Undefined symbol: .OCILobTrim
ld: 0711-317 ERROR: Undefined symbol: .OCILobRead
ld: 0711-317 ERROR: Undefined symbol: .OCILobFreeTemporary
ld: 0711-317 ERROR: Undefined symbol: .OCILobCreateTemporary
ld: 0711-317 ERROR: Undefined symbol: .OCIDefineByPos
ld: 0711-317 ERROR: Undefined symbol: .OCIStmtGetBindInfo
ld: 0711-317 ERROR: Undefined symbol: .OCIStmtPrepare2
ld: 0711-317 ERROR: Undefined symbol: .OCIStmtFetch
ld: 0711-317 ERROR: Undefined symbol: .OCIBindByName
ld: 0711-317 ERROR: Undefined symbol: .OCIBindByPos
ld: 0711-345 Use the -bloadmap or -bnoquiet option to obtain more information.
collect2: ld returned 8 exit status
running install_lib
At this point, I did a lot of Google searches to find out why the loader emits these errors. I finally found the solution: Oracle 9i installs the 64-bit libraries in $ORACLE_HOME/lib and the 32-bit libraries in $ORACLE_HOME/lib32. Since setup.py is looking by default in $ORACLE_HOME/lib (via -L/oracle/OraHome1/lib), it finds the 64-bit libraries and it fails with the above errors. The quick hack I found was to manually re-run the last command that failed and specify -L/oracle/OraHome1/lib32 instead of -L/oracle/OraHome1/lib (I think the same effect can be achieved via environment variables such as LIBPATH).
bash-2.05a# /usr/local/lib/python2.4/config/ld_so_aix cc_r -pthread -bI:/usr/local/lib/python2.4/config/python.exp build/temp.aix-5.1-2.4/cx_Oracle.o -L/oracle/OraHome1/lib32 -lclntsh -o build/lib.aix-5.1-2.4/cx_Oracle.so -s
Then I re-ran setup.py in order to copy the shared library to the Ptyhon site-packages directory:

bash-2.05a# /usr/local/bin/python setup.py install
running install
running build
running build_ext
running install_lib
copying build/lib.aix-5.1-2.4/cx_Oracle.so -> /usr/local/lib/python2.4/site-packages


At this point I was able to import cx_Oracle at the Python prompt:

bash-2.05a# /usr/local/bin/python
Python 2.4.1 (#1, Jul 15 2005, 14:44:07)
[GCC 3.3.2] on aix5
Type "help", "copyright", "credits" or "license" for more information.
>>> import cx_Oracle
>>> dir(cx_Oracle)
['BINARY', 'BLOB', 'CLOB', 'CURSOR', 'Connection', 'Cursor', 'DATETIME', 'DataError', 'DatabaseError', 'Date', 'DateFromTicks', 'Error', 'FIXED_CHAR', 'FNCODE_BINDBYNAME', 'FNCODE_BINDBYPOS', 'FNCODE_DEFINEBYPOS', 'FNCODE_STMTEXECUTE', 'FNCODE_STMTFETCH', 'FNCODE_STMTPREPARE', 'IntegrityError', 'InterfaceError', 'InternalError', 'LOB', 'LONG_BINARY', 'LONG_STRING', 'NUMBER', 'NotSupportedError', 'OperationalError', 'ProgrammingError', 'ROWID', 'STRING', 'SYSDBA', 'SYSOPER', 'SessionPool', 'TIMESTAMP', 'Time', 'TimeFromTicks', 'Timestamp', 'TimestampFromTicks', 'UCBTYPE_ENTRY', 'UCBTYPE_EXIT', 'UCBTYPE_REPLACE', 'Warning', '__doc__', '__file__', '__name__', 'apilevel', 'buildtime', 'connect', 'makedsn', 'paramstyle', 'threadsafety', 'version']

Thursday, July 14, 2005

py lib gems: greenlets and py.xml

I've been experimenting with various tools in the py library lately, in preparation for a presentation I'll give to the SoCal Piggies group meeting this month. The py lib is choke-full of gems that are waiting to be discovered. In this post, I'll talk a little about greenlets, the creation of Armin Rigo. I'll also briefly mention py.xml.

Greenlets implement coroutines in Python. Coroutines can be seen as a generalization of generators, and it looks like the standard Python libray will support them in the future via 'enhanced generators' (see PEP 342). Coroutines allow you to exit a function by 'yielding' a value and switching to another function. The original function can then be re-entered, and it will continue execution from exactly where it left off.

The greenlet documentation offers some really eye-opening examples of how they can be used to implement generators for example. Another typical use case for greenlets/coroutines is turning asynchronous or event-based code into normal sequential control flow code -- the Python Desktop Server project has a good example of exactly such a transformation.

I've also been reading and looking at the code from Armin's EuroPython talk on greenlets. The talk itself must have been highly entertaining, since it is presented as a PyGame-based game. In one of the code examples I downloaded, I noticed yet another application of the asynchronous-to-sequential transformation, this time related to parsing XML data. In a few lines of code, Armin showed how to turn an asynchronous, Expat-based parsing mechanism into a generator that yields the XML elements one at a time. This approach combines the advantages of 1) using a stream oriented parser (and thus being able to process large amounts of XML data via handlers) with 2) using a generator to expose the XML parsing code in the shape of an iterator.

Here is Armin's code which I saved in a module called iterxml.py (I made a few minor modifications to make the code more general-purpose):

from py.magic import greenlet
import xml.parsers.expat

def send(arg):
greenlet.getcurrent().parent.switch(arg)

# 3 handler functions
def start_element(name, attrs):
send(('START', name, attrs))
def end_element(name):
send(('END', name))
def char_data(data):
data = data.strip()
if data:
send(('DATA', data))

def greenparse(xmldata):
p = xml.parsers.expat.ParserCreate()
p.StartElementHandler = start_element
p.EndElementHandler = end_element
p.CharacterDataHandler = char_data
p.Parse(xmldata, 1)

def iterxml(xmldata):
g = greenlet(greenparse)
data = g.switch(xmldata)
while data is not None:
yield data
data = g.switch()

Consumers of this code can pass a string containing an XML document to the iterxml function and then use a for loop to iterate through the elements yielded by the function, like this:

for data in iterxml(xmldata):
print data

When iterxml first executes, it instantiates a greenlet object and associates it with the greenparse function. Then it 'switches' into the greenlet and thus is calls that function with the given xmldata argument. There is nothing out of the ordinary in the greenparse function, which simply assigns the 3 handler functions to the xpat parser object, then calls its Parse method. However, the 3 handler functions all use greenlets via the send method, which sends the parsed data to the parent of the current greenlet. The parent in this case is the iterxml function, which yields the data at that point, then switches back into the greenparse function. The handler functions then get called again whenever a new XML element is encountered, and the switching back and forth continues until there is no more data to be parsed.

I've wanted for a while to check out the REST API offered by upcoming.org (which is a free alternative to meetup.com), so I used it in conjunction with the XML parsing stuff via greenlets.

Here's some code that uses the iterxml module to parse the response returned by upcoming.org when a request for searching the events in the L.A. metro area is sent to their server:

import sys, urllib
from iterxml import iterxml
import py

baseurl = "http://www.upcoming.org/services/rest/"
api_key = "YOUR_API_KEY_HERE"
metro_id = "1" # L.A. metro area
log = py.log.Producer("")

def get_venue_info(venue_id):
method = "venue.getInfo"
request = "%s?api_key=%s&method=%s&venue_id=%s" %
(baseurl, api_key, method, venue_id)
response = urllib.urlopen(request).read()
for data in iterxml(response):
if data[0] == 'START' and data[1] == 'venue':
attr = data[2]
venue_info = "%(name)s in %(city)s" % attr
break
return venue_info

def search_events(keywords):
method = "event.search"
request = "%s?api_key=%s&method=%s&metro_id=%s&search_text=%s" %
(baseurl, api_key, method, metro_id, keywords)
response = urllib.urlopen(request).read()
for data in iterxml(response):
if data[0] == 'START' and data[1] == 'event':
attr = data[2]
log("\n" + "-" * 80)
log.EVENT("%(name)s" % attr)
log.WHAT("%(description)s" % attr)
log.WHERE(get_venue_info(attr['venue_id']))
log.WHEN("%(start_date)s @ %(start_time)s" % attr)

if __name__ == "__main__":
if len(sys.argv) < 2:
print "Usage: %s " % sys.argv[0]
sys.exit(1)

keywords = "%%20".join(sys.argv[1:])
search_events(keywords)

An upcoming.org API key is automatically generated for you when you click here.

I tested the script by searching for Python-related events in L.A.:

./upcoming_search.py python

--------------------------------------------------------------------------------
[EVENT] SoCal Piggies July Meeting
[WHAT] Monthly meeting of the Southern California Python Interest Group.
[WHERE] USC in Los Angeles
[WHEN] 2005-07-26 @ 19:00:00

Note that I'm also using the py.log facilities I mentioned in a previous post. The only thing I needed to do was to instantiate a log object via log = py.log.Producer("") and then use it via keywords such as EVENT, WHAT, WHERE and WHEN. Since I didn't declare any log consumer, the default consumer is used, which prints its messages to stdout. Each message string is nicely prefixed by the corresponding keyword.

I'm still experimenting with greenlets, and I'm sure I'll use them in the future especially for event-based GUI code.

I want to also briefly touch on py.xml, a tool that allows you to generate XML and HTML documents almost painlessly from your Python code.

Here's the XML returned by the event.search method of upcoming.org when called with a search text of 'python':

<rsp stat="ok" version="1.0">
<event id="24868" name="SoCal Piggies July Meeting" description="Monthly meeting of the Southern California Python Interest Group." start_date="2005-07-26" end_date="0000-00-00" start_time="19:00:00" end_time="21:00:00" personal="0" selfpromotion="0" metro_id="1" venue_id="7425" user_id="14959" category_id="4" date_posted="2005-07-13" />
</rsp>


And here's how to generate the same XML output with py.xml:

import py

class ns(py.xml.Namespace):
"my custom xml namespace"

doc = ns.rsp(
ns.event(
id="24868",
name="SoCal Piggies July Meeting",
description="Monthly meeting of the Southern California Python Interest Group.",
start_date="2005-07-26",
end_date="0000-00-00",
start_time="19:00:00",
end_time="21:00:00",
personal="0",
selfpromotion="0",
metro_id="1",
venue_id="7425",
user_id="14959",
category_id="4",
date_posted="2005-07-13"),
stat="OK",
version="1.0",
)

print doc.unicode(indent=2).encode('utf8')


The code above prints:

<rsp stat="OK" version="1.0">
<event category_id="4" date_posted="2005-07-13" description="Monthly meeting of the Southern California Python Interest Group." end_date="0000-00-00" end_time="21:00:00" id="24868" metro_id="1" name="SoCal Piggies July Meeting" personal="0" selfpromotion="0" start_date="2005-07-26" start_time="19:00:00" user_id="14959" venue_id="7425"/></rsp>


As the py.xml documentation succintly puts it, positional arguments are child-tags and keyword-arguments are attributes. In addition, indentation is also available via the argument to the unicode method.

I intend to cover other tools from the py library in future posts. Stay tuned for discussions on py.execnet, little-known aspects of py.test and more!

Friday, July 01, 2005

Recommended reading: Jason Huggins's blog

I recently stumbled on Jason's blog via the Thoughtworks RSS feed aggregator. Jason is the creator of Selenium and a true Pythonista. His latest post on using CherryPy, SQLObject and Cheetah for creating a 'Ruby on Rails'-like application is very interesting and entertaining. Highly recommended! Hopefully the Subway guys will heed Jason's advice of focusing more on "ease of installation and fancy earth-shatteringly beautiful 10 minute setup movies" -- this is one area in which it's hard to beat the RoR guys, but let's at least try it!

Tuesday, June 21, 2005

Keyword-based logging with the py library

I've been experimenting lately with the logging functionality recently introduced in the py library. Most of the functionality is still in early experimental stage, but the main ideas are pretty well defined. I want to review some of these ideas here and hopefully get some feedback.

Holger Krekel, the main py library developer, introduced the idea of keyword-based logging and of logging producers and consumers. A log producer is an object used by the application code to send messages to various log consumers. The consumers can be STDOUT, STDERR, log files, syslog, the Windows Event Log, etc. When you create a log producer, you define a set of keywords that are then used to both route the logging messages to consumers, and to prefix those messages.

Here's a quick example of code that works with the current svn version of the py library (which you can get from http://codespeak.net/svn/py/dist). I saved the following code in a file called logmodule.py:
# logmodule.py

import py

log = py.log.Producer("logmodule")

def runcmd(cmd):
log.cmd(cmd)
# real code follows

def post2db(data):
log.post2db(data)
# real code follows

if __name__ == "__main__":
runcmd("Running command CMD1")
post2db("PASS")
I started by instantiating a log object of class py.log.Producer with the keyword "logmodule". Since I didn't associate any consumer with the log producer, a default consumer will be used, which prints all messages to STDOUT.

The log producer object does not have any pre-defined methods associated with it. Instead, you can invoke arbitrarily-named methods. When accessing a brand new method (such as log.cmd or log.post2db), another producer object is returned on the fly, with the name of the method added as a keyword to the already-existent set of keywords. When calling the returned object as a function with the log message as the argument, the keywords are looked up to see if there is any log consumer associated with them. If no consumer is found, the default one is used and the message is pri nted to STDOUT.

Running the code above produces:
[logmodule:cmd] Running command CMD1
[logmodule:post2db] PASS
What happened behind the scenes when invoking log.cmd("Running command CMD1") was that a log producer object called log.cmd was created with ('logmodule', 'cmd') as the set of keywords. When calling the object as a function with the message as the argument, the message was by default printed to STDOUT, since no consumer was found to be associated with those keywords.
The message was also prefixed with the keywords, joined by ":".

Now let's assume we don't want any messages to be logged when using the logmodule module directly. To achieve this, we associate a null consumer with the "logmodule" keyword by adding the following line somewhere at the top of our module :
py.log.setconsumer("logmodule", None)
(NOTE: the name of the setconsumer function might change, but the idea will be the same, namely that any keyword or set of keywords can be associated with log consumers).

Now, when we invoke log.cmd("Running command CMD1"), the keyword matching logic will first look for a consumer associated with ('logmodule', 'cmd'), since this is the complete set of keywords. If it doesn't find it, it looks for a consumer associated with 'logmodule'. In our case, it will find the consumer we designated (None), so it will use that consumer and not log the message anywhere.

The simple mechanism of associating log consumers with keywords offers a tremendous amount of flexibility in configuring logging for an application. The association keywords-consumers can be done at run time, via a configuration file or via command-line arguments, and the application code doesn't need to know where logging gets routed. As an example, let's assume that we want to use the 2 top-level functions runcmd and post2db from the logmodule module, and at the same time we want to log messages to other consumers. Here is some code I saved in a file called use_logmodule.py:
# use_logmodule.py

import py
from logmodule import runcmd, post2db

py.log.setconsumer("logmodule", py.log.STDERR)
py.log.setconsumer("logmodule post2db", None)

runcmd("Running command CMD2")
post2db("FAIL")
My goal was to print all messages to STDERR by default, and not print any messages associated with the "post2db" keyword (in a real life situation, those messages could be associated with some database-management code for example).

To achieve my goal, I associated a consumer that uses STDERR with the keyword "logmodule" and a "/dev/null"-type consumer (i.e. None) with the keywords "logmodule post2db". Now when I run the above code, I get the following message printed to STDERR:
[logmodule:cmd] Running command CMD2
I could just as easily have routed the logging messages to a log file by associating a producer of type py.log.Path (currently there are 3 types of pre-defined consumers offered by the py.log package: py.log.STDOUT, py.log.STDERR and py.log.Path; in the future, they will be augmented with other types of consumers such as syslog, Windows Event Logs, etc.). Here's how to log all "post2db"-related messages to a file:
py.log.setconsumer("logmodule post2db", py.log.Path("/tmp/logmodule.out"))
When running the code in use_logmodule.py, the following line will be printed to /tmp/logmodule.out:
[logmodule:post2db] FAIL
A question you might have at this point is how does all this relate to a more traditional use of logging, based on severity levels such as debug, info, warn, error, critical. Let's assume we want our application code to issue calls to log.debug, log.info and log.error, while being able at the same time to direct these various levels of severity to different logging mechanisms. We could for example log debug messages to a log file, informational messages to the console (STDOUT or STDERR), and error messages to both a log file and the console. Here's one way to achieve this via the py.log keyword-based approach. I saved the following code in a file called config.py:
# config.py

import sys
import py

log = py.log.Producer("")
log.debug = py.log.Producer("logfile")
log.info = py.log.Producer("console")
log.error = py.log.Producer("console logfile")

logfile = "/tmp/myapp.out"
py.log.setconsumer("logfile", py.log.Path(logfile))
py.log.setconsumer("console", py.log.STDOUT)

l = open(logfile, 'a', 1)

def console_logfile_logger(msg):
print >>sys.stdout, str(msg)
print >>l, str(msg)

py.log.setconsumer("console logfile", console_logfile_logger)
Most of this stuff should look familiar by now. I instantiated an object called log of class Producer, with no keywords associated with it. This object will serve as the default producer object, so the application will be able to call log(some_msg) and have some_msg printed to STDOUT by default. Next, I created the debug, info and error attributes for the log object. This will enable me to call log.debug(msg), log.info(msg) and log.error(msg) in my application code, without necessarily caring where those messages get routed. As it stands right now in the configuration file, the log.debug object is declared as a log.Producer object associated with the keyword "logfile". Similarly, the log.info object is associated with the keyword "console" and the log.error object is associated with both keywords "console logfile".

The next step is defining consumers for the various keywords. For "logfile" I set a consumer of type Path, pointing to the file /tmp/myapp.out. For "console" I set the STDOUT consumer, and for "console logfile" I set a custom consumer that prints its argument (the message to be logged) to STDOUT and also appends it to the log file. As an aside, I want to note that consumers are simply functions that take a message as an argument and manage the message as they see fit (if you look in the testing code for the py.log module in the py library, you'll see consumers that simply append the message to a list for example).

Here is an example of application code that uses the above config.py file:
# appcode.py

import config

log = config.log

log.debug("DEBUG MESSAGE")
log.info("INFO MESSAGE")
log.error("ERROR MESSAGE")
log.warn("WARNING MESSAGE")
Running appcode.py produces:
[console] INFO MESSAGE
[console:logfile] ERROR MESSAGE
[warn] WARNING MESSAGE
At the same time, the file /tmp/myapp.out contains:
[logfile] DEBUG MESSAGE
[console:logfile] ERROR MESSAGE
Note that we didn't declare log.warn anywhere. It was dynamically transformed by py.log into a producer object with keyword "warn" and printed by default to STDOUT, since the "warn" keyword didn't have any consumer associated with it.

I didn't want to complicate things too much in the code above, but in my opinion it would be better to have 'debug', 'info' and 'error' as part of the keywords that also prefix each message, so that you can see at a glance what kind of severity level the message has. To achieve that, you would change the declarations for log.debug, log.info and log.error in config.py to:
log.debug = py.log.Producer("logfile").debug
log.info = py.log.Producer("console").info
log.error = py.log.Producer("console logfile").error
or alternatively to:
log.debug = py.log.Producer("logfile debug")
log.info = py.log.Producer("console info")
log.error = py.log.Producer("console logfile error")
Now if you run appcode.py, you'll see this printed on the screen:
[console:info] INFO MESSAGE
[console:logfile:error] ERROR MESSAGE
[warn] WARNING MESSAGE
This also makes it easier to turn off debugging for example. All you need to do is to associate the null consumer with the keywords "logfile debug":
py.log.setconsumer("logfile debug", None)
Note that the consumer associated with the single keyword "logfile" is still the Path consumer that prints to /tmp/myapp.out, so non-debug messages would still get routed to that file.

In conclusion, I think that the keyword-based approach works really well for the logging needs of any application. I emphasize once more that the py.log code is still in its infancy and as such is prone to changes (including naming changes), so I wouldn't rely on py.log just yet for production code. But the main idea of associating log producers with log consumers via keywords should be the same, and this is I think the novelty of this approach.

One enhancement that I've discussed with Holger is to have a way to specify that you want multiple keywords to be routed each to a different consumer. So for example, if you declared
log.error = py.log.Producer("console logfile")
a call to log.error(some_msg) would try to match the "console" keyword individually to a consumer, and the "logfile" keyword to another consumer.

Right now, the keyword matching logic only looks for a single consumer that matches either ("console", "logfile") or just ("console",), so you need to declare a custom consumer such as the console_logfile_logger function I described above.

In the multiple-keyword approach, you wouldn't need to do that, instead you would just associate "logfile" with a consumer of type Path, "console" with a consumer of type STDOUT/STDERR, and both consumers would then receive the message when log.error(some_msg) was called.

If you think you can use the keyword-based approach to logging in your own application code, or if you have a use case that you would like to see implemented via py.log, or if you have any suggestions at all, please leave a comment here of send email to py-dev at codespeak dot net.

Wednesday, June 08, 2005

Handling the 'Path' Windows registry value correctly

A recent recipe posted to the Python Cookbook site shows how to manipulate environment variables in Windows by modifying the registry. I've been using something similar in my code, mainly for adding directories on the fly to the 'Path' environment variable by modifying the \HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Environment\Path registry value.

Here are two simple functions for getting and setting a registry value of type string (REG_SZ):
def get_registry_value(reg, key, value_name):
k = _winreg.OpenKey(reg, key)
value = _winreg.QueryValueEx(k, value_name)[0]
_winreg.CloseKey(k)
return value

def set_registry_value(reg, key, value_name, value):
k = _winreg.OpenKey(reg, key, 0, _winreg.KEY_WRITE)
value_type = _winreg.REG_SZ
_winreg.SetValueEx(k, value_name, 0, value_type, value)
_winreg.CloseKey(k)
To add a directory dir to the Path registry value, you would do:
reg = _winreg.ConnectRegistry(None, _winreg.HKEY_LOCAL_MACHINE)
key = r"SYSTEM\CurrentControlSet\Control\Session Manager\Environment"
path = get_registry_value(reg, key, "Path")
path += ";" + dir
set_registry_value(reg, key, "Path", path)
However, there was a problem with this approach. If you look at a typical Path registry value (using for example regedit), you'll see that it contains directories such as %SystemRoot% and %SystemRoot%\system32, where %SystemRoot% is a variable which has the Windows system directory as its value. When I modified the Path value, I mistakenly set its type to REG_SZ, and in consequence the command prompt interpreter did not replace the %SystemRoot% variable with its value. The unfortunate side effect of all this was that typical Windows binaries residing in the system32 directory, such as ping, ipconfig, etc., were not found anymore by the command prompt.

As a workaround, I replaced %SystemRoot% with os.environ['SYSTEMROOT'] in the path variable, like this:

dirs = path.split(';')
try:
systemroot = os.environ['SYSTEMROOT']
except KeyError:
pass
else:
dirs = [re.sub('%SystemRoot%', systemroot, dir)
for dir in dirs]
path = ';'.join(dirs)


A much better approach, suggested by Ori Berger in a comment to the recipe I mentioned, is to set the value type for the Path registry value to REG_EXPAND_SZ, and not to REG_SZ. This makes variables like %SystemRoot% be automatically expanded by the command prompt interpreter. So the modified version of my set_registry_value function is now:

def set_registry_value(reg, key, value_name, value, value_type=_winreg.REG_SZ):
k = _winreg.OpenKey(reg, key, 0, _winreg.KEY_WRITE)
_winreg.SetValueEx(k, value_name, 0, value_type, value)
_winreg.CloseKey(k)
For modifying the 'Path' value, you would now do:
reg = _winreg.ConnectRegistry(None, _winreg.HKEY_LOCAL_MACHINE)
key = r"SYSTEM\CurrentControlSet\Control\Session Manager\Environment"
path = get_registry_value(reg, key, "Path")
path += ";" + dir
set_registry_value(reg, key, "Path", path, _winreg.REG_EXPAND_SZ)
Of course, the whole registry value manipulation code can be neatly packaged in a class, like here.

Friday, May 20, 2005

Installing the Firebird database on a 64-bit RHEL Linux server

I just finished the painful exercise of installing Firebird on an Intel E64MT server running RHEL 3. I first tried installing the RPM versions of Firebird from here -- tried both the 1.0 version and the 1.5 version of the database, in both Classic and SuperServer incarnations. In all cases, the RPMs installed without error, but when I tried to install the KInterbasDB Python module (and also the DBD::Interbase Perl module), the loader complained about the Firebird shared libraries, which were obviously not 64-bit. So I ended up installing Firebird from source. Here's what I did:

1. Downloaded and bunzip-ed firebird-1.5.2.4731.tar.bz2.
2. Ran ./configure, then make and make install in the firebird-1.5.2.4731 directory.

By default, this installs the Classic version of Firebird in /usr/local/firebird (the Classic version allows processes to connect directly to database files; the SuperServer version requires processes to open a socket to a server process, which then accesses the database files).

The 'make install' command also asks for a password for the SYSDBA user. If you happen to forget the password, you can always retrieve it by looking at the file /usr/local/firebird/SYSDBA.password as root. You change the password by running /usr/local/firebird/bin/changeDBAPassword.sh.

3. Downloaded the kinterbasdb-3.1.src.tar.gz Python module and installed it via python setup.py install.
4. Downloaded and installed the mxDateTime Python module from here (this module is required by kinterbasdb.)
5. Tested kinterbasdb against the example employee database shipped with Firebird:
>>> import kinterbasdb
>>> conn = kinterbasdb.connect(database='/usr/local/firebird/examples/employee.fdb')
>>> cursor = conn.cursor()
>>> cursor.execute('select * from country')
>>> print cursor.fetchall()
[('USA', 'Dollar'), ('England', 'Pound'), ('Canada', 'CdnDlr'), ('Switzerland','SFranc'), ('Japan', 'Yen'), ('Italy', 'Lira'), ('France', 'FFranc'), ('Germany', 'D-Mark'), ('Australia', 'ADollar'), ('Hong Kong', 'HKDollar'), ('Netherlands', 'Guilder'), ('Belgium', 'BFranc'), ('Austria', 'Schilling'), ('Fiji', 'FDollar')]

At this point, I created a Firebird database file called test.gdb, owned by user root and group root. When I connected to this database as root via the isql utility in /usr/local/firebird, everything looked good:

[root@localhost]# /usr/local/firebird/bin/isql
Use CONNECT or CREATE DATABASE to specify a database
SQL> connect test.gdb;
Database: test.gdb
SQL>quit;


However, when I tried to connect to the database as a non-root user, I got this error:

$ /usr/local/firebird/bin/isql
Use CONNECT or CREATE DATABASE to specify a database
SQL> connect test.gdb;
Statement failed, SQLCODE = -551

no permission for read-write access to database test.gdb

What I needed to do was to change ownership and permissions for the test.gdb file so that it was owned by user firebird and group firebird, and it was group-writeable:

chown firebird.firebird test.gdb
chmod 664 test.gdb


My goal was to have the Firebird database accessible from a CGI script running from Apache.
The httpd process was running as user apache, so I had to add this user to the firebird group in /etc/group. I then restarted httpd to take into account the new group membership.

At this point, from my CGI script, I was able to open a connection to test.gdb via:
conn = kinterbasdb.connect(database='test.gdb', user='SYSDBA',
password='password')

I found the firebird-devel mailing list very helpful in solving some of the issues I faced. In particular, this message allowed me to get past the non-root user problems.

Wednesday, May 18, 2005

Third SoCal Piggies meeting

The SoCal Piggies had their third meeting at USC on May 17th. A record number of 7 Piggies attended (Daniel, Howard, Mark, Grig, Brian, Titus, Diane), joined by a guest from Caltech. It was nice to meet Howard, Brian and Mark and to talk about Python while munching on a fairly decent Pizza Hut nourishment.

Titus Brown presented part 2 of his Quixote 2.0 tutorial. He showed a few introductory examples, then delved into more advanced stuff, showing how to manage sessions with Quixote. One thing I took away from it is that it takes remarkably little code. Generally speaking, Quixote seems very crisp and clean, and Titus' tutorials are very good in smoothing the learning curve, which can otherwise be steep.

Titus then spent some time talking about his Web app. testing tool twill. We couldn't get a live demo unfortunately, but it was an interesting discussion nevertheless, covering things such as the cmd module, pyparsing, and especially a great use of metaclasses. Titus ended his presentation talking about his Cucumber module, another magical metaclass-heavy module that provides a mapping between Python code and SQL statements, hiding the complexity of SQL away from the programmer.

Daniel Arbuckle was next, and he gave us an overview of iterators, generators and continuations. He highly recommended the little-used itertools module, which provides functions designed to work with iterators. The functions are similar to standard Python built-ins like map and range and allow you to write code that is faster memory-efficient. Daniel then covered generators, which can be seen as a way to express iterators using functions as opposed to classes (if you see a yield statement anywhere in a function, that function is a generator). However, there are many other and sometimes slightly brain-melting uses of generators, such as cooperative multitasking and state machine control.

Daniel ended with a mind-boggling discussion of continuations, which (luckily or unluckily?) are not and will not be offered as a feature of Python, due to GvR's dislike of them. Especially interesting was the parallel Daniel made between continuations and the HTTP GET mechanism.

Many thanks to Titus and Daniel for their presentations.

The next meeting will be at USC again, on a date to be announced. Topics for next meeting:

  • CherryPy tutorial (if Greg McClure will be able to attend the meeting)
  • SimpleTAL and Pyrex tutorials (Diane)
  • Others?

Friday, May 13, 2005

Installing and using cx_Oracle on Unix

Here's a mini-HOWTO on installing the cx_Oracle Python module on Unix systems (tested on Linux and Solaris.) I always forget one of these steps every time I install this module on one of my systems, so I thought I'd write them down. Other people might find this useful, so here goes:

1. Download cx_Oracle-4.1.tar.gz from here.
2. Become root on the target system.
3. Make sure you have the ORACLE_HOME environment variable defined for user root and pointing to the correct Oracle installation directory on your server. This step is necessary because the cx_Oracle installation process looks for headers and libraries under ORACLE_HOME.
4. Install cx_Oracle by running 'python setup.py install'.
5. Become user oracle, since this is the user you'll most likely want to run python scripts as when interacting with Oracle databases. Make sure you have the following line in oracle user's .bash_profile (or similar for other shells):
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ORACLE_HOME/lib
If you don't add ORACLE_HOME/lib to your LD_LIBRARY_PATH, you'll get an error such as:

>>> import cx_Oracle
Traceback (most recent call last):
File "", line 1, in ?
ImportError: ld.so.1: python: fatal: libclntsh.so.9.0: open failed: No such file or directory
Now you're ready to use the cx_Oracle module. Here's a bit of code that shows how to connect to an Oracle database and run a SELECT query:
import cx_Oracle
from pprint import pprint

connection = cx_Oracle.Connection("%s/%s@%s" % (dbuser, dbpasswd, oracle_sid))
cursor = cx_Oracle.Cursor(connection)
sql = "SELECT * FROM your_table"
cursor.execute(sql)
data = cursor.fetchall()
print "(name, type_code, display_size, internal_size, precision, scale, null_ok)"
pprint(cursor.description)
pprint(data)
cursor.close()
connection.close()

Friday, May 06, 2005

New features in Selenium 0.3

A new version of the Selenium Web app test tool has just been released by Jason Huggins and the rest of the team at ThoughtWorks. I'll highlight some of the new and updated features.

New TestRunner look and feel

The most striking feature in version 0.3 is a new and improved look for the TestRunner. Click here for a screenshot.


Simplified directory structure

Another important improvement is a much simplified directory structure for the Selenium package. The main Selenium package contains the 'BrowserBot' JavaScript engine and nothing else. All the other language-specific drivers/bindings are available for download separately.

Here's all you need to do to get started with Selenium 0.3:

1. Download selenium-0.3.0.zip from the download page
2. Unzip the archive. This will create a directory called selenium-0.3.0, with 2 sub-directories: doc and selenium
3. Copy the selenium directory somewhere where it can be served by the Web server hosting your Web app
4. Open http://www.yourwebserver.com/selenium/TestRunner.html -- you will see a screen similar to the screenshot above
5. Start tinkering with the default test suite shipped with Selenium.

To add your own test suite, create an HTML file containing a table with links to your tests:

Custom Test Suite

Test1

Save this file as CustomTestSuite.html in the tests subdirectory of the selenium directory.

In the table above, Test1 is a link to the actual test file, which is another HTML file (called Test1.html) containing your custom test table. To have Selenium run your custom test suite, pass a ‘test’ parameter to TestRunner.html by opening an URL such as this:

http://www.yourwebserver.com/selenium/
TestRunner.html?test=./tests/CustomTestSuite.html

(note: this is a single line)

Improved browser support

In addition to Firefox/Mozilla on Windows/Linux/Mac OS X and IE on Windows, Selenium now supports Konqueror on Linux. There is also experimental support for Safari on Mac OS X.

Standalone server for win32 platforms

A standalone server is available for win32 platforms. This used to be known as the 'Twisted Server' implementation of Selenium, but now the fact that it is based on Twisted is hidden behind the scenes, so that new users don't have to jump through the various hoops of installing Python/Twisted/etc.

The main advantage of using the standalone version of Selenium is that it provides a workaround for the JavaScript 'same origin policy' security limitation. The same origin policy aims to prevent Cross-Site-Scripting attacks by preventing documents or scripts loaded from one origin from getting or setting properties of a document from a different origin. The net effect of all this is that in order to be able to use Selenium to test your Web application, you need to deploy the Selenium engine/framework on the server hosting the AUT (application under test). With the standalone Selenium server, you don't have this limitation and you can start doing what all new users to Selenium seem to want to do: test Google searches :-)

For users on non-Windows platforms, the Twisted Server version is still available via Subversion. See my previous blog entry for a tutorial on using the Twisted Server version. Many concepts discussed in there still apply to the standalone server version, in particular how to compose special URLs for testing 3rd party Web sites.

Plone Product version available for download

While Selenium has been available as a Plone Product version for quite some time, it was never part of the 'official' download page. Now you can find it among the Selenium 0.3 downloads as selenium-0.3.0-python-plone.zip.

For details on using Selenium with Plone, see 2 of my tutorials: part 1 and part 2.

Finally, here is a screen recording movie in AVI format that shows Selenium in action (note: choose 'Full screen' in your media player for better resolution.) The test run goes in 'Walk' mode through a new user registration test in a default, out-of-the-box installation of Plone. Enjoy!

Friday, April 29, 2005

New version of sparkplot on sparkplot.org

Due to positive feedback to my sparkplot post, I registered sparkplot.org and made it into a home for the sparkplot module.

I also added another option to sparkplot: -t or --transparency. Setting this option produces a transparent background for the generated PNG image (thanks to usagi for the suggestion).

I was glad to see somebody actually using the sparkplot module at the T&J blog. It looks like visualizing trading data is a natural application of sparklines. I also liked the term used by T to describe the act of embedding sparklines into blog entries: datablogging.

Saturday, April 23, 2005

sparkplot: creating sparklines with matplotlib

Edward Tufte introduced sparklines in a sample chapter of his upcoming book "Beautiful Evidence". In his words, sparklines are "small, high-resolution graphics embedded in a context of words, numbers, images. Sparklines are data-intense, design-simple, word-sized graphics." Various implementations of sparklines that use different graphics packages have already been published, and probably the best known one is a PHP implementation that uses the GD backend. In what follows, I'll show some examples of sparkline graphics created via matplotlib by the sparkplot Python module I wrote.

Example 1

Since a picture is worth a thousand words, here is the Los Angeles Lakers' road to their NBA title in 2002. Wins are pictured with blue bars and losses with red bars. Note how easy it is to see the streaks for wins and losses.

The Lakers' 2004 season was their last with Shaq, when they reached the NBA finals and lost to Detroit (note the last 3 losses which sealed their fate in the finals).

Compare those days of glory with their abysmal 2005 performance, with only 2 wins in the last 21 games. Also note how the width of the last graphic is less than the previous 2, a consequence of the Lakers not making the playoffs this year.

Example 2

The southern oscillation is defined as the barometric pressure difference between Tahiti and the Darwin Islands at sea level. The southern oscillation is a predictor of El Nino which in turn is thought to be a driver of world-wide weather. Specifically, repeated southern oscillation values less than -1 typically defines an El Nino.

Here is a sparkline for the southern oscillation from
1955to 1992 (456 sample data points obtained from NIST). The sparkline is plotted with a horizontal span drawn along the x axis covering data values between -1 and 0, so that values less than -1 can be more clearly seen.

Example 3
Here is the per capita income in California from 1959 to 2003.
And here is the "real" per capita income (adjusted for inflation) in California, from 1959 to 2003.

Example 4

Here is the monthly distribution of messages sent to comp.lang.py from 1994 to 2004, plotted per year. Minimum and maximum values are shown with blue dots and labeled in the graphics.

Year

Total
1994
clpy 1994
3,018
1995
clpy 1995
4,026
1996
clpy 1996
8,378
1997
clpy 1997
12,910
1998
clpy 1998
19,533
1999
clpy 1999
24,725
2000
clpy 2000
42,961
2001
clpy 2001
55,271
2002
clpy 2002
56,750
2003
clpy 2003
64,548
2004
clpy 2004
56,184


There was an almost constant increase in the number of messages per year, from 1994 to 2004, the only exception being 2004, when there were fewer message than in 2002 and 2003.

Details on using sparkplot

1) Install the Numeric Python module (required by matplotlib)
2) Install matplotlib
3) Prepare data files: sparkplot simplistically assumes that its input data file contains just 1 column of numbers
4) Run sparkplot.py. Here are some command-line examples to get you going:

- given only the input file and no other option, sparkplot.py will generate a gray sparkline with the first and last data points plotted in red:

sparkplot.py -i CA_real_percapita_income.txt

produces:

The name of the output file is by default .png. It can be changed with the -o option.

The plotting of the first and last data points can be disabled with the --noplot_first and --noplot_last options.

- given the input file and the label_first, label_last, format=currency options, sparkplot.py will generate a gray sparkline with the first and last data points plotted in red and with the first and last data values displayed in a currency format:

sparkplot.py -i CA_real_percapita_income.txt --label_first --label_last --format=currency

produces:

The currency symbol is $ by default, but it can be changed with the --currency option.

- given the input file and the plot_min, plot_max, label_min, label_max, format=comma options, sparkplot.py will generate a gray sparkline with the first and last data points plotted in red, with the min. and max. data points plotted in blue, and with the min. and max. data values displayed in a 'comma' format (e.g. 23,456,789):

sparkplot.py -i clpy_1997.txt --plot_min --plot_max --label_min --label_max --format=comma

produces:

- given the input file and the type=bars option, sparkplot.py will draw blue bars for the positive data values and red bars for the negative data values:

sparkplot.py -i lakers2005.txt --type=bars

produces:

As a side note, I think bar plots look better when the data file contains a relatively large number of data points, and the variation of the data is relatively small. This type of plots works especially well for sports-related graphics, where wins are represented as +1 and losses as -1.

- for other options, run sparkplot.py -h

I hope the sparkplot module will prove to be useful when you need to include sparkline graphics in your Web pages. All the caveats associated with alpha-level software apply :-) Let me know if you find it useful. I'm very much a beginner at using matplotlib, and as I become more acquainted with it I'll add more functionality to sparkplot.

Finally, kudos to John Hunter, the creator of matplotlib. I found this module extremely powerful and versatile. For a nice introduction to matplotlib, see also John's talk at PyCon05.

Note: the Blogger template system might have something to do with the fact that the graphics are shown with a border; when included in a "normal", white-background HTML page, there is no border and they integrate more seamlessly into the text.

Update 5/2/05: Thanks to Kragen Sitaker for pointing out a really simple solution to the "borders around images" problem -- just comment out the CSS definition for .post img in the Blogger template.

Thursday, April 14, 2005

More on performance vs. load testing

I recently got some comments/questions related to my previous blog entry on performance vs. load vs. stress testing. Many people are still confused as to exactly what the difference is between performance and load testing. I've been thinking more about it and I'd like to propose the following question as a litmus test to distinguish between these two types of testing: are you actively profiling your application code and/or monitoring the server(s) running your application? If the answer is yes, then you're engaged in performance testing. If the answer is no, then what you're doing is load testing.

Another way to look at it is to see whether you're doing more of a white-box type testing as opposed to black-box testing. In the white-box approach, testers, developers, system administrators and DBAs work together in order to instrument the application code and the database queries (via specialized profilers for example), and the hardware/operating system of the server(s) running the application and the database (via monitoring tools such as vmstat, iostat, top or Windows PerfMon). All these activities belong to performance testing.

The black box approach is to run client load tools against the application in order to measure its responsiveness. Such tools range from lightweight, command-line driven tools such as httperf, openload, siege, Apache Flood, to more heavy duty tools such as OpenSTA, The Grinder, JMeter. This type of testing doesn't look at the internal behavior of the application, nor does it monitor the hardware/OS resources on the server(s) hosting the application. If this sounds like the type of testing you're doing, then I call it load testing.

In practice though the 2 terms are often used interchangeably, and I am as guilty as anyone else of doing this, since I called one of my recent blog entries "HTTP performance testing with httperf, autobench and openload" instead of calling it more precisely "HTTP load testing". I didn't have access to the application code or the servers hosting the applications I tested, so I wasn't really doing performance testing, only load testing.

I think part of the confusion is that no matter how you look at these two types of testing, they have one common element: the load testing part. Even when you're profiling the application and monitoring the servers (hence doing performance testing), you still need to run load tools against the application, so from that perspective you're doing load testing.

As far as I'm concerned, these definitions don't have much value in and of themselves. What matters most is to have a well-established procedure for tuning the application and the servers so that you can meet your users' or your business customers' requirements. This procedure will use elements of all the types of testing mentioned here and in my previous entry: load, performance and stress testing.

Here's one example of such a procedure. Let's say you're developing a Web application with a database back-end that needs to support 100 concurrent users, with a response time of less than 3 seconds. How would you go about testing your application in order to make sure these requirements are met?

1. Start with 1 Web/Application server connected to 1 Database server. If you can, put both servers behind a firewall, and if you're thinking about doing load balancing down the road, put the Web server behind the load balancer. This way you'll have one each of different devices that you'll use in a real production environment.

2. Run a load test against the Web server, starting with 10 concurrent users, each user sending a total of 1000 requests to the server. Step up the number of users in increments of 10, until you reach 100 users.

3. While you're blasting the Web server, profile your application and your database to see if there are any hot spots in your code/SQL queries/stored procedures that you need to optimize. I realize I'm glossing over important details here, but this step is obviously highly dependent on your particular application.

Also monitor both servers (Web/App and Database) via command line utilities mentioned before (top, vmstat, iostat, netstat, Windows PerfMon). These utilities will let you know what's going on with the servers in terms of hardware resources. Also monitor the firewall and the load balancer (many times you can do this via SNMP) -- but these devices are not likely to be a bottleneck at this level, since they usualy can deal with thousands of connections before they hit a limit, assuming they're hardware-based and not software-based.

This is one of the most important steps in the whole procedure. It's not easy to make sense of the output of these monitoring tools, you need somebody who has a lot of experience in system/network architecture and administration. On Sun/Solaris platforms, there is a tool called the SE Performance Toolkit that tries to alleviate this task via built-in heuristics that kick in when certain thresholds are reached and tell you exactly what resource is being taxed.

4. Let's say your Web server's reply rate starts to level off around 50 users. Now you have a repeatable condition that you know causes problems. All the profiling and monitoring you've done in step 3, should have already given you a good idea about hot spots in your applicationm about SQL queries that are not optimized properly, about resource status at the hardware/OS level.

At this point, the developers need to take back the profiling measurements and tune the code and the database queries. The system administrators can also increase server performance simply by throwing more hardware at the servers -- especially more RAM at the Web/App server in my experience, the more so if it's Java-based.

5. Let's say the application/database code, as well as the hardware/OS environment have been tuned to the best of everybody's abilities. You re-run the load test from step 2 and now you're at 75 concurrent users before performance starts to degrade.

At this point, there's not much you can do with the existing setup. It's time to think about scaling the system horizontally, by adding other Web servers in a load-balanced Web server farm, or adding other database servers. Or maybe do content caching, for example with Apache mod_cache. Or maybe adding an external caching server such as Squid.

One very important product of this whole procedure is that you now have a baseline number for your application for this given "staging" hardware environment. You can use the staging setup for nightly peformance testing runs that will tell you whether changes in your application/database code caused an increase or a decrease in performance.

6. Repeat above steps in a "real" production environment before you actually launch your application.

All this discussion assumed you want to get performance/benchmarking numbers for your application. If you want to actually discover bugs and to see if your application fails and recovers gracefully, you need to do stress testing. Blast your Web server with double the number of users for example. Unplug network cables randomly (or shut down/restart switch ports via SNMP). Take out a disk from a RAID array. That kind of thing.

The conclusion? At the end of the day, it doesn't really matter what you call your testing, as long as you help your team deliver what it promised in terms of application functionality and performance. Performance testing in particular is more art than science, and many times the only way to make progress in optimizing and tuning the application and its environment is by trial-and-error and perseverance. Having lots of excellent open source tools also helps a lot.

Modifying EC2 security groups via AWS Lambda functions

One task that comes up again and again is adding, removing or updating source CIDR blocks in various security groups in an EC2 infrastructur...