PEP 750 – Template Strings

Looking at the various conversations involving string interpolation, this characterization is extremely unkind. They've clearly spent a lot more than 5 minutes thinking about this, including writing their own mini-proposals[1].

Are they wrong about this issue? I think they are. There is a big difference in ergonomics between String interpolation and something like fmt.Sprintf, and the performance cost of fmt.Sprintf is non-trivial as well. But I can't say they didn't put any thought into this.

As we've seen multiple times with Go generics and error handling before, their slow progress on correcting serious usability issues with the language stem from the same basic reasons we see with recent Java features: they are just being quite perfectionist about it. And unlike Java, the Go team would not even release an experimental feature unless they feel quite good about it.

[1] https://github.com/golang/go/issues/57616

No, the format function doesn't "arbitrarily execute code."

An f/t string is syntax not runtime.

Instead of

    "Hello " + subject + "!"

    f"Hello {subject}!"

And instead of

    query(["SELECT * FROM account WHERE id = ", " AND active"], [id])

    query(t"SELECT * FROM account WHERE id = {id} AND active")

In many languages, f-strings (or f-string like constructs) are only supported for string literals, not user-supplied strings.

When compiling, those can be lowered to simple string concatenation, just like any for loop can be lowered to and represented as a while.

> A format function that arbitrarily executes code from within a format string

So, a template? I certainly ain't gonna be using go for its mustache support.

No, it's exactly the opposite--f-strings are, roughly, eval (that is, unsanitary string concatenation that is presumptively an error in any nontrivial use) to t-strings which are just an alternative expression syntax, and do not even dereference their arguments.

That tracks. Ruby followed in the footsteps of Perl, which had string manipulation as a main priority for the language.

This led to the OpenD language fork (https://opendlang.org/index.html) which is led by some contributors who had other more general gripes with D. The fork is trying to merge in useful stuff from main D, while advancing the language. They have a Discord which unfortunately is the main source of info.

Sure, here are the two Go/C-style formatting options:

    fmt.Sprintf("This house is %s tall", measurements(2.5))

    fmt.Sprint("This house is ", measurements(2.5), " tall")

    f"This house is {measurements(2.5)} tall"

Sprint is better in that regard, we can read from left to right without interruptions, but is a pain to write due to all the punctuation, nevermind refactor. For example, try adding a new variable between "This" and "house". With the f-string you just type {var} before "house" and you're done. With Sprint, you're now juggling quotation marks and commas. And that's just a simple addition of a new variable. Moving variables or substrings around is even worse.

Summing up, f-strings are substantially more ergonomic to use and since string formatting is so commonly done, this adds up quickly.

    _log(f”My variable is {x + y}”)

    _log(“My variable is {}”.format(x+y)) 

    _log(“My variable is {z}”.format(z=x+y))

There are many long-term users of Python who participate in PEP discussion who argue for beginners[0], often because they professionally are teaching Python.

There are also loads of people basically defaulting to "no" on new features, because they understand that there is a cost of supporting things. I will often disagree about the evaluation of that cost, but it's hard to say there is no cost.

Nobody wants a system that is unusable, slow, hard to implement for, or hard to understand. People sometimes just have different weights on each of these properties. And some people are in a very awkward position of overestimating costs due to overestimating implementation effort. So you end up in discussions like "this is hard to understand!" "No it isn't!"

Hard to move beyond, but the existence of these kinds of conversations serve, in a way, as proof that people aren't jumping on every new feature. Python is still a language that is conservative in what it adds.

This should actually inspire more confidence in people that features added to Python are _useful_, because there are many people who are defaulting to not adding new features. Recent additions to Python speeding up is more an indicator of the process improving and identifying the good stuff rather than a lowering of the bar.

[0]: I often think that these discussions often get fairly intense. Understandability is definitely a core Python value, but I Think sometimes discussions confuse "understandability" with "amount of things in the system". You don't have to fully understand pervasive hashing to understand Python's pervasive value equality semantics! A complex system is needed to support a simple one!

All discussion on PEP's happens in public forums where anyone can opine on things before they are accepted. I agree that the experts are more likely to participate in this exchange. And while this is wish-washy, I feel like the process is really intended to benefit the experts more than the novices anyways.

There have been processes put into place in recent years to try to curb the difficulty of things. One of those is that all new PEPs have to include a "how can you teach this to beginers" section, as seen here on this pep: https://peps.python.org/pep-0750/#how-to-teach-this

You might find the Python discussion forums ([0] and [1]) interesting; conversation that guides the evolution of PEPs happens there.

As Nick mentioned, PEP 750 had a long and winding road to its final acceptance; as the process wore on, and the complexities of the earliest cuts of the PEPs were reconsidered, the two converged.

[0] The very first announcement: https://discuss.python.org/t/pep-750-tag-strings-for-writing...

[1] Much later in the PEP process: https://discuss.python.org/t/pep750-template-strings-new-upd...

Building off this question, it's not clear to me why Python should have both t-strings and f-strings. The difference between the two seems like a stumbling block to new programmers, and my "ideal python" would have only one of these mechanisms.

PEP 501 when originally written (not by me) was intended to be the competing standard against f-strings, and to have been more inline with ES6's template strings. There was debate between the more simple f-string PEP (PEP 498) and PEP 501. Ultimately, it was decided to go with f-strings as a less confusing, more approachable version (and also easier to implement) and to "defer" PEP 501 to "see what happens". Since then, the python internal have also changed, allowing t-strings to be even easier to implement (See PEP 701). We have seen what happens, and now its introduced. f-strings and t-strings are not competing systems. They are different. Similar to ES6 templates and namedTaggedTemplates, they are used for different things while API feels similar intentionally. f-strings are not inferior to t-strings, they are better for most use cases of string templating where what you really want, is just a string.

I'm not familiar with ES6 template strings, but why are they better than f-strings? F-strings just work, and work well, in my experience so I'm wondering what I'm missing out on. Especially since the language I use the most is c++... So I guess I don't expect much out of string manipulation lol.

One solution that existed for a while was using the `!` operator for deferred. `t!'my defered {str}'`

The main reason for non having deferred evaluation was that it over-complicated the feature quite a bit and introduces a rune. Deferred evaluation also has the potential to dramatically increase complexity for beginners in the language, as it can be confusing to follow if you dont know what is going on. Which means "deferred by default" wasnt going to be accepted.

As for LLM's, it was not the main consideration, as the PEP process here started before LLM's were popular.

> were prompt templates for LLM prompt chaining a use case that influenced the design in any way

Maybe not directly, but the Python community is full of LLM users and so I think there's a general awareness of the issues.

The expression (x) is eagerly evaluated in both cases, cuz that's how Python works. You can defer the format call but Python fundamentally doesn't have an equivalent of lazy/compile time flag argument evaluation and this doesn't change that.

For a logger t-strings are mostly just a more pleasant and less bug-prone syntax for #2

T-strings, like f-strings, are eagerly evaluated -- so in this sense, no, there's no benefit.

My hope is to write some new documentation as 3.14 nears release that explains the (growing) constellation of string formatting mechanisms in Python and describes when they might each be useful. They overlap to some degree, but each has a unique twist that makes them useful in different situations. PEP 292 is going nowhere and is used, for instance, in really powerful libraries like `flufl.i18n`

yes, it does actually need syntax support. In order for it to work, you need to preserve which parts of the string are static (hard coded) and which parts are dynamic (likely user input). Which you can only do at a syntax level. You could potentially do it by hand, using placeholders, like with `%`, but as we now live in f-string world, we have something better. The syntax highlighting and ergonomics of f-strings are so good, devs prefer it in most cases. The idea is to make the most ergonomic thing, also the safest thing. By decreasing ergonomics, you reduce the adoption of safer symantics.

A library can't capture interpolated variables

> There is an observation that you can use `lambda` inside to delay evaluation of an interpolation, but I think this lambda captures any variables it uses from the context.

Actually lambda works fine here

    >>> name = 'Sue'
    >>> template = lambda name: f'Hello {name}'
    >>> template('Bob')
    'Hello Bob'

> I'm not sure if t-strings help here?

That's correct, they don't. Evaluation of t-string expressions is immediate, just like with f-strings.

Since we have the full generality of Python at our disposal, a typical solution is to simply wrap your t-string in a function or a lambda.

(An early version of the PEP had tools for deferred evaluation but these were dropped for being too complex, particularly for a first cut.)

> An early version of this PEP proposed that interpolations should be lazily evaluated. [...] This was rejected for several reasons [...]

Bummer. This could have been so useful:

    statement_endpoint: Final = "/api/v2/accounts/{iban}/statement"

You could do something like t”Hello, {“name”}” (or wrap “name” in a class to make it slightly less hacky).

Lambda only captures variables which haven't been passed in as argument.

I tend to agree. I think it's easy enough to use a lambda in this case

    >>> template = lambda name: f'Hello {name}'
    >>> template('Bob')

Then it worked.

Added example to parent comment.

This example still works, the entire f-string is sanitised (including whatever the value of name was). Assuming sqlstring is the sanitisation function.

The “template” would be a separate function that returns an f-string bound from function arguments.

Does it make it easier? The “escape” for both is to just use unsafe version of the Template -> string function or explicitly mark an unsafe string as sanitised. Both seem similar in (un)safety

How do you leave arguments unbound?

I’m not sure you understood my example. The f-string is within a function. The function argument only accepts sanitised input type.

If you create a subclass of str which has an init function that sanitises, then you can’t create a Sanitised type by casting right?

And even if you could, there is also nothing stopping you from using a different function to “html” that just returns the string without sanitising. They are on the same relative level of safety.

I don't think that's the reason.

https://peps.python.org/pep-0750/#arbitrary-string-literal-p...

Python already has built-in data structure literals that allow you to express lispy DSLs:

    html(['ul', {'class': 'foo'}, *(['li', item] for item in items)])

The benefit of lisp-like representation is you have the entire structure of the data, not just a sequence of already-serialized and not-yet-serialized pieces.

Quoting operators are something I actually miss in Python whereas t-strings are something I have never wanted in 17 years of writing Python.

In the motivation for the t-string types, their gripe is that f-strings are not templates.

My issue with them is that you have to write your syntax in the string complex expressions dictionary access and such become awkward.

But, this whole thing is bike-shedding in my opinion, and I don't really care about the color of the bike shed.

I believe that jab was that PHP has a bunch of ways to do similar things and Python, in their view, is turning out that way, too.

Thank you for the historical context and details!

Your example is interesting - `join`, a generator expression and a ternary, and Python requires us to write all of them inside-out. It's a shame we can't write that in a more natural order, something like:

    def f(template):
        return (for item in template:
            isinstance(item, str) then item else
            format(convert(item.value, item.conversion), item.format_spec)
        ).join('')

> This is how Python has always been.

Nah, idiomatic Python always used to prefer comprehensions over explicit loops. This is just the `match` statement making code 3x longer than it needs to be.

I think for someone with a basic knowledge of both languages, the Ruby version is more understandable than the Python. It's a combination of basic Ruby features, whereas Python's `match` statement is much more obscure - it isn't really Python at all, it's "a DSL contrived to look like Python [...] but with very different semantics" [0].

I don't particularly love the Ruby code either, though - I think the ideal implementation would be something like:

    fn stringify(item) =>
        item.is_a(Interpolation) then
            item.value.convert(item.conversion).format(item.format_spec)
        else item.to_string()

    fn f(template) => template.map(stringify).join()

> Readability counts

Almost all Python programmers should be familiar with list comprehensions - this should be easy to understand:

    parts = [... if isinstance(item, Interpolation) else ... for item in template]

    parts = []
    for item in template:
        match item:
            case str() as s:
                parts.append(...)
            case Interpolation(value, _, conversion, format_spec):
                parts.append(...)

I think for someone with a basic knowledge of Ruby, it's more understandable than the Python. It's a combination of basic Ruby features, nothing advanced.

I don't particularly love Ruby's syntax either, though - I think the ideal implementation would be something like:

    fn stringify(item) =>
        item.is_a(Interpolation) then
            item.value.convert(item.conversion).format(item.format_spec)
        else item.to_string()

    fn f(template) => template.map(stringify).join()

IMO it's much closer to the ideal way to write the function, which would be something like:

    fn stringify(item) =>
        item.is_a(String) then item else
        item.value.convert(item.conversion).format(item.format_spec)

    fn f(template) => template.map(stringify).join()