Python __dunders__

What they are, why they're used and why they're great.

code
Meme
The British are wrong.
This post is a work in progress.

What’s a __dunder__?

If you’ve used python for a while, you’ll probably have heard the term dunder thrown around a lot. dunder init, dunder main and dunder name are some of the more common dunders. You might, at one point have seen, or even been told to do:

1
2
3
4
5
def main():
    ...
    
if __name__ == "__main__": #hey look, dunders!
    main()

The name “dunder”, as you might have figured out by now, is a short-form of double underscore, the characters surrounding these names. However, they’re more than just just a naming convention. They’re also known as magic methods, because as you’ll see in a while, they’re magic.

Dunder methods

__init__

This is likely one of the first dunder methods you’ll come across. It can be thought of as analogous to a constructor in languages like C++ and Java, though pedantically it’s not a constructor (that would be __name__), rather it’s an initializer. The British are wrong.

While I won’t go into the details of a python object lifecycle, the gist of its:

graph LR; new(__new__) --> init(__init__) --> do(do whatever) --> del(__del__);

__new__ creates the object and passes it along to __init__. At this stage, the object has no attributes. __init__‘s job is to fix that. __init__ gives this new object properties and attributes. After this, your code does whatever it wants with the object. At the end of it all, we say goodbye. __del__ is run right before the object is deleted.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
class Value:
	def __new__(cls, *args):
		print("Hello,", *args)
		return object.__new__(cls)
    
	def __init__(self, value):
		self.val = value
		print("I have a value of", value)
        
	def __del__(self):
		print("Goodbye,", self.val)
        
# v only exists within the runtime of the function
def limited_scope():
	v = Value(123)
	print(v, "exists!")

limited_scope()
# Hello, 123
# I have a value of 123
# <__main__.Value object at 0x038E56D0> exists!
# Goodbye, 123

We’ll talk about __main__ later :wink:

Notice that __init__ is the method that gives v the attribute val. It initialized an empty object with the attributes for its type. However, this object was not created here, rather it was created in __new__.

A shorthand

Let’s take the example of string comparison in two languages: Java & Python

In Python, we’d do:

1
2
3
a = "hello!"
b = "hel" + "lo!"
print(a == b)

In Java, we’d do:

1
2
3
String a = "hello!";
String b = "hel" + "lo!";
System.out.println(a.equals(b))

Focus on the equality statement. In python, we did a short == to compare the two objects. In Java, we did .equals. Here’s the thing: we also did .equals in python, just indirectly. What’s happened is that python sees the statement a == b, and converts it into the equivalent statement a.__eq__(b)

graph LR eq(a == b) --> dunder("a.__eq__(b)") dunder --> self("str.__eq__(a, b)")
Read The elegance of self for the third box

In a similar fashion, the following dunders are called during the following operations (non-exhaustive):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
#arithmetic
self +  other # => self.__add__(other) 
              # => type(self).__add__(self, other)
    
self -  other # __sub__
self *  other # __mul__
self /  other # __truediv__
self // other # __floordiv__
self ** other # __pow__
self %  other # __mod__
- self        # self.__neg__()
+ self        # __pos__

#logical 
self == other # __eq__
self != other # __ne__
self >= other # __ge__
self >  other # __gt__
self <= other # __le__
self <  other # __lt__

Implementing the appropriate dunder in self‘s class enables us to use these neater shorthands. For example,

1
2
3
4
5
6
7
8
9
10
11
12
13
class Value:
    def __init__(self, val): 
        self.val = val
    
    def __eq__(self, other):
        return self.val == other
    
val = Value(1)
num = 2

print(val == my_num) # => val.__eq__(num) 
                     # => Value.__eq__(val, num) 
                     # => False

Neat! But what about…

1
print(num == val) # num.__eq__(val) => ???

I’ll let you think about that. If you want, keep reading, or skip to this section for the answer.

Builtin Support

Apart from special notation, dunder methods also enable us to use builtin functions with little thought. We’ve already seen the

More than a shorthand

Try this out:

1
2
print(num == val)
print(num.__eq__(val))

It prints False, then NotImplemented. Interesting. There’s more going on here.

Well, dunders are more than just shorthands. In this case, Python evaluates num.__eq__(val) into NotImplemented. After all, int is a builtin, and Value is our own class. How is Python supposed to know how to compare them? Note that it didn’t raise the error, it returned the error type. Now what? Well, it inferred that a == b should be the same as b == a. It then re-runs the function call, but this time on val.__eq__(num). This time, as expected, it returns False, hence the expression num == val is False.

graph TD; eq(a == b) --> dunder{{"a.__eq__(b)"}} dunder -- success --> s1(return value) dunder -- NotImplemented --> t1{{"b.__eq__(a)"}} t1 -- success --> s2(return value) t1 -- NotImplemented --> s3("return False") style s3 stroke-width:2px,fill:none,stroke:red; style s1 stroke-width:2px,fill:none,stroke:green; style s2 stroke-width:2px,fill:none,stroke:green;

See? Magic! This behavior can also be seen with a >= b becoming b <= a, a > b to b < a and vice versa. Note that a >= b does not get inferred to be equal to not (b < a), as there are use cases where both are true, or both false. Case in point, nan. In general, this behavior will be seen when operations can be “flipped”.

Dunder variables

There are a few ubiquitous dunder variables. These follow the familiar double underscore syntax but aren’t special methods of a class. Rather, they refer to special attributes. Again, this list is non-exhaustive and only serves as a taste of what dunder variables are out there.

__name__

Depending on the context, __name__ refers to slightly different things, but always refer to some sort of name.

Functions

1
2
3
4
5
6
def my_function():
    pass
print(my_function.__name__) # my_function

f = lambda x: x
print(f.__name__) # <lambda>

Modules

Finally, we get back to __name__ == "__main__". 

1
2
3
4
5
6
# A.py
print("A: My __name__ is", __name__)

# B.py
import A
print("B: My __name__ is", __name__)

Now, when we run python B.py, what we get is:

1
2
A: My __name__ is A
B: My __name__ is __main__

__name__ is __main__ if it was run as the main file! Makes sense. When a file is imported, it’s __name__ is simply the file name.

By checking __name__ == "__main__", we can run code selectively based on if the file is being imported, or ran directly.