Skip to content(if available)orjump to list(if available)

Hidden Messages in Emojis and Hacking the US Treasury

cogman10

> Now, all of this might have been fine had Beyond Trust not written a feature which allowed users to directly, programmatically interact with psql (the postgres command line interface).

That's the buried lede.

Yes, there was a vulnerability in psql... but that's so much less a problem than the huge gaping hole of allowing users to directly interact with psql.

No DB can be safe if you are turning untrusted user commands into psql executions. It'd be like giving untrusted users ssh access and then complaining when they find a privilege elevation exploit.

0xbadcafebee

The only reason BeyondTrust implemented that was it wasn't untrusted user commands. They sanitized the data, so it should have been fine. The unfortunate problem was that the sanitizer didn't sanitize.

Systems are built on a set of expectations. Undermine the expectations and you undermine the system.

cogman10

> They sanitized the data, so it should have been fine.

This is a 101 rookie level approach to SQL or injection defense.

It's dumb for exactly the same reason why this is dumb

    "SELECT * FROM foo WHERE bar=" + sanitize(userInput)
The correct way to do something like this will always be parameterized input which looks something like this

    "SELECT * FROM foo WHERE bar=?"
    bindParameter(1, userInput);
Why? Because that the postgres protocol splits out the command and the data for the command in a way that can't be injected. Something that should be viewed as impossible to do when data and command are merged into 1 String.

IF this company wanted to build dynamic queries, then the only correct way to do that is to limit input to only valid variables. IE "isValidColumnName(userInput)" before sending the request. And even then, you'd not use psql to do that.

You simply can't use a generalized sanitizer and expect good results.

Nijikokun

His argument is that you are building on the assumption that this is safe:

    "SELECT * FROM foo WHERE bar=?"
    bindParameter(1, userInput);
But what happens when it turns out that isn't safe?

gowld

Let's be clear: Beyond Trust is not a company that wrote a database-backed web app and made the all-too-common mistake of writing insecure code that tickled a bug in the database that allowed privilege escalation. Beyond Trust's is a company whose entire contribution is adding a security layer to prevent privilege escalation, and their solution here was to bypass Postgres's standard functionality and use this weird `psql` hack instead.

They had one job, and they failed at it. This amateur-level mistake should sink the entire company.

singpolyma3

Giving untrusted users ssh access... You mean like every shared hosting company or shell provider?

zoeysmithe

Just a guess but this looks like politically powerful dev culture overwriting cybersecurity culture, demanding, thus getting an exception from management for 'productivity' and 'being agile.'

I dont think we appreciate how much of a wild west things are with the incredible mix of hugely complex and powerful tools available trivially to developers and the concept of "move fast, break things."

Especially as corporate sees devs like they see salesmen (big moneymakers who deserve perks, exceptions) and top-down security culture as a cost center.

The other buried ledes are that postgres allows emojis (not sure if that's intended but it works) and that you can just run system commands and scripts directly from postgres cli. I imagine a lot of eyes are going to be on new hardening guidelines for postgres now.

I also imagine the first high performance enterprise friendly drop-in db written in something like rust is going to one day be a big deal.

randmeerkat

> Especially as corporate sees devs like they see salesmen…

You’re onto something here. People perceive the world through the lens of their education and environment. Sales, legal, finance, are all easy constructs for a business leader to view the rest of the world through. The secret of the game isn’t to have the best tech or to code the most, it’s to “outsell” your competing business unit.

Terr_

Hey now, a large portion of developers are seen as cost-centers too! Not everybody has the skill of flattering managers into approving greenfield projects, and then transferring away before they break horribly. :p

mtrovo

So many questions:

- why a no side-effects function on a database can be used to get lateral access to the whole database instance

- why do you need to validate strings on the database itself and not on the client anyway, heck why are there no type safe way of doing it

- why would you want to execute shell commands from the database itself

- Even if there's a real use case for executing commands like that why is it enabled by default on a regular connection to the database without you specifying a THIS_IS_REALLY_DANGEROUS_BUT_I_PINKY_PROMISE_I_KNOW_WHAT_IM_DOING flag to the connection handshake.

It's not always PHP but there are some kirks that are shrugged off on PHP that makes me really concerned about the reliability of projects coded with it.

benmmurphy

They mentioned PAM module so maybe the sql injection just allowed bypassing the authorization of a system that was using the PAM module. Like it’s in the realm of possibility that a PAM module that wanted to validate a user against credentials stored in a pg database might shell out to the psql command to do this. Though, the whole thing is very questionable.

rapind

Yeah we’re missing some info.

What account were they authenticating with when attaching to psql?

If you have the connection string why does psql even matter, couldn’t you use any client? Or is this a case of your input being forwarded to a running, already authenticated, psql instance?

And finally, why do we need unicode support for schema? I assume it’s because the schema is itself data?

jarebear6expepj

Your questions are programming language agnostic-- where did your PHP angst come in? And are there specific things in PHP that are problematic and avoidable by using a different Turing complete language?

marsovo

PHP has grown up but in its wild youth was notorious for such gems as mysql_escape_string vs mysql_real_escape_string, rather than proper parameterization

It's not so much about Turing as it is libraries and patterns

After all, as I understand it this very issue was caused by escaping SQL rather than parameterizing it

xmprt

I learned about the breach a few days ago but I didn't know that you could "adopt" an emoji: https://aac.unicode.org/sponsors

That's a neat tidbit.

jfengel

Does "adopting an emoji" mean something other than "appearing on that page"?

From the adoption page: Each adoption comes with a digital badge and certificate that you can proudly display.

nkrisc

It’s a fun way to donate.

Terr_

> In order to to this, PQescapeStringInternal must call pg_utf_mblen.

For a moment I had a dev-flashback to the problem of utf8mb4 versus (broken) utf8 in mySQL. Easy now, this is Postgres, everything is safe(er)...

dalemhurley

I have been playing a lot with emoji smuggling with https://toolnames.com/utilities/emoji-smuggler and https://chat.full.cx inspired by https://emoji.paulbutler.org/

I had not considered it as an injection attack.

hnlmorg

As someone who’s done a fair amount with parsing of Unicode strings lately, I’m not at all surprised by this bug.

Unicode is a surprisingly elegant system but also an open invitation for all kinds of abuse.

rapind

> Unicode is a surprisingly elegant system…

s/elegant/clever

What could go wrong? I bet unicode is how AGI escapes and enslaves humanity.

giancarlostoro

I havent deep dived unicode enough to know, and probably someone somewhere already made it, but I often wonder if we can do compression more efficiently by abusing unicode somehow, especially regarding plain text.

pavel_lishin

I swear qntm has written something about this.

netsharc

Geez, any summary of this article that tells it like the reader isn't five years old?

defen

UTF-8 encodes a unicode codepoint into 1, 2, 3, or 4 bytes. Assuming that you have a valid UTF-8 encoding of a codepoint, then the first byte tells you how many bytes are in the encoding. 0-127 inclusive means one byte, 192-223 means 2, 224-239 means 3, and 240-247 means 4. If the first byte is 0xC0 (192), then the sequence is two bytes long. However, not every 2-byte sequence that starts with 0xC0 is valid UTF-8. The uppermost bits of the second byte must be `10` in a valid 2-byte UTF-8 sequence. 0x27 does not meet that criteria, so `0xC0 0x27` is not valid UTF-8. If your escape function operates at the level of unicode codepoints but doesn't actually verify that they're valid, you end up copying a single quote into your "escaped" buffer that downstream parts of the code will hit.

0xbadcafebee

A PHP app called a Postgres library function to "escape strings" for use in Postgres, and that called a function to get a utf8 string length, but the function was bullshit:

> The PQescapeStringInternal method doesn’t actually validate that the string it is parsing with pg_utf_mblen is valid Unicode. So, instead, it just takes the length of 2, and grabs the next byte.

So the bug was a shitty function in a generic open source library which was probably never properly tested or fuzzed, which ended up letting attackers move laterally through the database. And this is one reason you want full test coverage; tiny stupid functions matter.

(Another fix for this is to enforce at the boundaries of every function that the input data has been "blessed" or sanitized by some other function whose purpose is just to validate that the data is what it's supposed to be. That would have to happen before escaping, and every function that uses that data would need to confirm that it got blessed. Basically you want a home-rolled strong-typing system with types (or data classes?) for all your data. But that's a lot of work, I don't expect many would do that for most apps)

gowld

* Custom Unicode parsing code in the middle of an "escape" function

* No static type-checking for Unicode data

charcircuit

>Beyond Trust did their due diligence by properly calling a sanitization method on the user’s string input using it in a PostgreSQL query.

This is not due diligence. In band messaging of user controlled data has been proven to be bad for security and this is not the first time "escaping" user controlled data for SQL has been done incorrectly.

Spivak

Yep, it's prepared statements or bust. But the long tail of legacy code, examples, documentation, that uses escaping is gonna take a while to get through.

One of the nice things about modern ORMs like SQLAlchemy 2 is that it forces you to use prepared statements even when when calling raw queries.

kevinsync

Fun fact: MySQL (and I'm sure many other databases?) lets you pass in values directly as hexadecimal strings.

I've avoided SQL injection since ancient times not by escaping strings, but by transforming any given input via "0x" + bin2hex(value) and plopping that into the query.

No quotes needed, no code buried deep in included libraries needed, handles any kind of data possible, and also no funny business sneaking in based on how you may have mangled the input.

vessenes

Cool idea. Genuinely. But it does not in any way guarantee that an injection attack like used here won’t work - unless it’s maintained as hex through the whole pipeline. In this case the (malformed) Unicode was sent to a command line call - if your hex text needed to be parsed to be understood on the command line, then your security plan would have failed.