The Power of Introspection in Python

Pardon the Interruption

This blog post was written under a previous major version of willcarh.art (e.g. v1.x vs. v2.x). This may seem trivial, but because some of my blog posts reference the website's code, links and code snippets may no longer be valid. Thank you for understanding and please enjoy the post!

What is Introspection?

Introspection is one of those programming buzzwords that gets thrown around, but what does it actually mean? A quick Google search of introspection returns the examination or observation of one's own mental and emotional processes. For a human, introspection is essentially thinking about thinking, such as reconsidering why we acted a certain way or made a decision in the past.

Introspection with Python is conceptually the same as with humans. We are essentially asking Python to give us some information about itself, whether it be about an instance of a class, object, etc. On the surface, this can sound complex, but in practice it's quite simple. Let's dive right in.

Introducing getattr()

Suppose we have the following Python class:

class Car():
    def __init__(self):
        self.miles = 0
    def drive(self, miles):
        self.miles += miles

Now, let's create a new Car and have it drive a little bit.

>>> car = Car()
>>> car.drive(10)
>>> car.miles
10

We can also use the Python builtin function getattr to accomplish this.

>>> car = Car()
>>> getattr(car, 'drive')(10)
>>> getattr(car, 'miles')
10

What just happened? We used the getattr function to get named attributes from our Car class. Even cooler, not only can we use getattr to get the value of class attributes, we can also use it to call functions!

Why is this powerful?

On the surface, the introspective power of getattr may not be immediately apparent. After all, it took us the same number of steps to drive our Car with introspection as without. However, consider the case where you want to call a function via a variable, like a string. Let's rewrite our Car class to be a bit more generic:

class Car:
    def __init__(self):
        self.miles = 0
        self.velocity = 0
    def drive(self, miles):
        self.miles += miles
    def accelerate(self, velocity):
        self.velocity += velocity

    def do_action(name, value):
        getattr(self, name)(value)

Now we can drive our car by calling drive() or speed up by calling accelerate(). However, we can also use the new do_action() function:

>>> car = Car()
>>> car.do_action('drive', 10)
>>> car.miles
10
>>> car.do_action('accelerate', 30)
>>> car.velocity
30

This makes automating things in Python much easier!

Why does this matter?

I'm definitely one to learn by example. Why does introspection matter? Where can I actually use it in my Python? Here's a simple example.

Consider the Python builtin dunder __dict__. This returns a dictionary of the class attributes for a class. If we had our Car class from above, we could use __dict__ to get its values:

>>> car = Car()
>>> car.miles = 10
>>> car.velocity = 30
>>> car.__dict__
{'miles': 10, 'velocity': 30}

Do you think we can recreate some of __dict__'s functionality using introspection? You bet we can! Let's use getattr to write a function that will JSONize a class, or take its attributes and turn them into a JSON string.

def jsonize(self):
    variables = [var for var in dir(self) if not var.startswith(('_', '__')) and not callable(getattr(self, var))]
    return "{" + ",".join([f"\"{var}\": \"{getattr(self, var)}\"" for var in variables]) + "}"

This might seem like some Python mumbo-jumbo, so let's break it down! The first line of jsonize gets all of the variables in the class (self). Then, the second line calls getattr for each variable, and arranges them nicely into JSON format. See, not so bad! Check out the code here.

A real life example

getattr is actually used in willcarh.art! All of the content for the site's database is read from a JSON file. Rather than hard coding this content in a Python file, I wrote a simple script called the Scribe to read from the JSON file and upload to willcarh.art's database. Now, there are multiple different models, or classes, in the database, so Scribe needs to be able to dynamically create Python objects. Here's how I used getattr to accomplish this...

My JSON schema is defined as such:

[
  {
    "class": "...",
    "contents": "..."
  }
]

After some data validation, I attempt to make an instance of the class, load in its content from the contents field in the JSON, and save it to the database. Note that in this call to getattr, the first argument is the module and the second is the class, whereas in our earlier usage the first argument was the class and the second argument was the attribute. getattr is very flexible!

def parse_data(entity):
    Class = getattr(models, entity['class'])
    instance = Class(**entity['content'])
    instance.save()

And that's it! These few lines of code save me the hassle of micromanaging my database. This is a watered down version of Scribe for demonstration purposes. If you'd like to see the full source code, check it out here.

If you're interested in learning more about getattr, here's a great introspection article. You could also consider looking into setattr, hasattr, and delattr.

🦉