Combining Python assignment expressions and destructuring: Having your variable and eating it too

Mikołaj Koziarkiewicz

Intro

One of Python’s probably most "wow, this is great!" features for newcomers is destructuring in assignments, whether of tuples, lists, or – indeed – any iterables:

destructuring_intro.py
a, b = ("a", "b")
[zero, one, two] = range(3)

For the longest time, this destructuring, and for comprehensions, were, sadly, all that Python had to offer in terms "structurally-oriented" syntactic sugar. In contrast, Scala’s pattern matching, while arriving a bit later in the game, has always been more feature-packed and functional.

Fortunately for Python, it has been eventually extended with structural pattern matching, which brings it way closer to the expression power of Scala’s pattern matching. "Almost" for several reason, the largest being that structural pattern matches are statements, not expressions. Why this was done was actually addressed in the PEP here; personally, the argument, possibly paraphrased as "it wouldn’t be Pythonic" is as lost on me as the opportunity that was available here.

In any case…​ this post is not about structural patching matching, but closing another type of expression power gap between Python’s and Scala’s destructuring semantics, also becoming a reality relatively recently: with Python 3.8.

By the way: this is definitely a shorter entry, written simply because I stumbled upon the relevant problem (and solution) while writing a considerably longer series of blogs. Hopefully, they’ll start appearing this quarter – but until then, let me show you something…​

Schrödinger’s destructuring

We’ll start with an example in Scala[1]. Let’s say we have a three-element list, and we want to assign said elements into separate variables. We end up with something like this:

pattern_matching_simple.scala
val List(one, two, three) = List(1,2,3)

The code above maps easily to Python, with the addition of the val keyword (for final, i.e. non-reassignable variables) and the syntax of defining a list being the only two differences. Indeed, the corresponding Python code looks like this:

[one, two, three] = [1, 2, 3]

OK, let’s say we know want to store both the individual elements and the entire list. In Scala, this is done easily enough with the addition of an @ assignment:

pattern_matching_whole.scala
val result@List(one, two, three) = List(1,2,3)

// In the console, this prints out:
//
// result: List[Int] = List(1, 2, 3)
// one: Int = 1
// two: Int = 2
// three: Int = 3

So what about Python? Until a couple of years ago, all one could do is something like this:

result = [1, 2, 3]
[one, two, three] = result

Not exactly the end of the world, but a bit annoying.

Things have changed with the introduction of assignment expressions, which you may be familiar with, by way of :=, i.e. the walrus operator[2]. As the name suggests, these are simply assignments (=) that are also expressions – meaning they evaluate to some value, unlike the standard assignment statement.

Let’s have a first go at trying to use an assignment expression in our test case:

result = ([one, two, three] := [1, 2, 3])

The snippet emulates Scala’s ordering of assignments, and looks like it could work, except the interpreter prints out:

  File "<stdin>", line 1
    result = ([one, two, three] := [1, 2, 3])
              ^^^^^^^^^^^^^^^
SyntaxError: cannot use assignment expressions with list

The same (or similar) gets printed out for tuples, sets, and so on. Using destructuring in an assignment expression is simply not supported. Not all is lost, however. All we have to do is reverse the sequence of assignments:

pattern_matching_whole.py
[one, two, three] = (result := [1, 2, 3])

And this works! Here’s the corresponding, verifying console output:

>>> one
1
>>> two
2
>>> three
3
>>> result
[1, 2, 3]

Collapsing the wave function

To summarize – for a given value, what we wanted is to assign:

  • that full value, and

  • a destructured version of the value,

  • in one step.

Our solution boils down to using an assignment expression together with good-old destructuring, represented by the general pattern below:

destructure_assign_pattern.py
<destructuring_vars> = (<whole_value_vars> := <value_to_destructure>)

Again, this is possible in any version of Python starting from 3.8, inclusive.

One important remark is that, like in the "old" two-liner version, we actually reference the original value within the assignment expression. Said identity has consequences when the value is mutable, such as the list in the example below:

destructure_assign_gotcha.py
org_value = [1, 2, 3]
[one, two, three] = (result := org_value)
result.append(4)
print(org_value)
# prints:
# [1, 2, 3, 4]

This might be a disadvantage when rewriting code where, originally, the "whole result" is reconstructed from the destructured, constituent parts, like so:

destructure_assign_reconstruct.py
[one, two, three] = [1, 2, 3]
result = [one, two, three]

creating a brand-new list. In other words, be wary of the quirk when rewriting code to this pattern.

Now, to get to a final talking point, one that’s probably on a number of the readers' heads: should this kind of "trick"/pattern be used?

For starters, I am definitely not a "pure" Python programmer (nor, TBH, do I aim to be one), and so no authority of whether something is "Pythonic" or not.

Otherwise, it can be argued that this pattern sacrifices readability for the sake of terseness – but the same could be said about any use of the assignment expression. The syntax in question is indeed a very sharp tool in the developer’s drawer, able to muddle up the codebase if abused. However, it still exists for a reason, and that reason is that sometimes these "shortcuts" do improve readability by increasing proximity of the assignment to other, relevant parts of the code.

For me specifically, the need arose when I was transforming OpenCV’s bounding boxes. The BBs are encoded as a 4-tuple[3]; I had a scenario where I needed to process both meaningfully-named, individual components of the BB, and the entire BB tuple.

The function in question contained less than 10 LoCs, so the call, in my mind, was simple. And this is probably the answer to the question posited above: when following general software engineering practices (including considering your team’s style and abilities), it’s OK to use the pattern whenever necessary and practical.

Hope this window into the possibilities offered by Python’s expanding syntax will prove useful to at least some readers. Happy coding!

PS. To be completely clear: I categorically do not claim "discovery" of this pattern, and I frankly strongly doubt I was the first to describe it. I simply haven’t seen – or don’t remember seeing – it being discussed anywhere, hence this blog entry :).
1. Don’t worry, the examples in Scala will be simple and completely explained.
2. To finalize my rant from the intro, note how this PEP – introducing a brand-new way of "converting" what was always a statement in Python to an expression – was defined and implemented much earlier than the structural pattern matching one.
3. X, Y, width, height