Fintech founder charged with fraud; AI app found to be humans in the Philippines

> 5) Uh-oh, the model is underperforming, and the human worker pipeline is now some significant part of the full workflow.

AI stands for "Actually, Indians."

> Reputation is everything at this level, so they hire some human workers to catch and fix the edge cases that end up badly.

The most important part of your reputation is admitting fault. Sometimes your product isn't perfect. Lying to your investors about automation rates is far worse for your reputation than just taking the L.

I'd think ambiguous statements about the scope of your AI would make it hard to prove fraud, if you were being careful at all. "Involving AI" could mean 1% AI.

So it's doubly surprising to me the government chose (criminal) wire fraud, not (civil) securities fraud, which would have a lower burden of proof.

Government lawyers almost never try to make their job harder than it has to be.

To be perfectly honest, I am more amazed that it was a valid business model and people were willing not just invest in it, but offer their rather personal information to an unaffiliated third party.

In this case it's a little bit worse; the "nate" app had a literally "0% automation rate," despite representations to investors of an "AI" automation rate of "93-97%" powered by "LSTMs, NLP, and RL." No ML model ever existed! [1]

See:

> As SANIGER knew, at the time nate was claiming to use AI to automate online purchases, the app’s actual automation rate was effectively 0%. SANIGER concealed that reality from investors and most nate employees: he told employees to keep nate’s automation rate secret; he restricted access to nate’s “automation rate dashboard,” which displayed automation metrics; and he provided false explanations for his secrecy, such as the automation data was a “trade secret.”

> SANIGER claimed that nate's "deep learning models" were "custom built" and use a "mix of long short-term memory, natural language processing, and reinforcement learning."

> When, on the eve of making an investment, an employee of Investment Firm-1 asked SANIGER about nate's automation rate, that is, the percentage of transactions successfully completed with nate's AI technology, SANIGER claimed that internal testing showed that "success ranges from 93% to 97%."

(from [1])

[1]: https://www.justice.gov/usao-sdny/media/1396131/dl?inline

> Turns out their ML/AI method doesn't generalize so well.

I'd argue the opposite. AI typically generalizes very well. What it can't do well is specifics. It can't do the same thing over and over and follow every detail.

That's what's surprised me about so many of these startups. They're looking at it from the bottom-up, something ai is uniquely bad at.

I think you're being excessively generous. According to the linked article,

> But despite Nate acquiring some AI technology and hiring data scientists, its app’s actual automation rate was effectively 0%, the DOJ claims.

Sometimes people are just dishonest. And when those people use their dishonestly to fleece real people, they belong in prison.

This is what we did internally. Someone said we could use LLMs for helping engineering teams solve production issues. Turned out it was just a useless tar pit. End game is we outsourced it.

Neither of these solved the problem that our stack is a pile of cat shit and needs some maintenance from people who know what the hell they are doing. It’s not solving a problem. It’s adding another layer of cat shit.

It is likely a real machine vision system if it was the same system our former company evaluated.

It worked by camera tracking the shelves contents, and would adjust the inventory level for a specific customers actions. And finally, tracked the incremental mass change during the checkout process to cross reference label swap scams etc.

Thus, people get flagged if their appearance changes while in the store, mass of goods is inconsistent with scanned labels, or the cameras don't see the inventory re-stocked.

You would be surprised how much irrational effort some board members put into the self-checkout systems. Personally, I found the whole project incredibly boring.... so found a more entertaining project elsewhere... =3

Percepta was a company that was doing a lot of CV/ML in this space looking for shoplifting traits. They had a few paying customers before they were completely acquired by ADT Business. A lot of shoplifters use the PLU for bananas when tag swapping higher-ticket items at the self checkout, so, more than likely, they wanted to check that you were actually purchasing bananas.

At the Circle K they have the option of doing self checkout by putting all your items under a camera and the register will automagically count 'em up and assess your total. I keep wondering if it's done by AI -- All Indians. Same with the OCR ATMs do on cheques.

I've been flagged as a potential shoplifter by the self-checkout at the grocery store based on some video analysis of CCTV footage of my hand motions.

Shopping in 2025 must be a frustrating experience for magicians.

I mean there is precedent: https://www.theguardian.com/commentisfree/2024/apr/10/amazon...

Sorry to hear.

Why would it matter to you if it’s a real human or AI? Wrong in any case.

Amazon didn't raise money from credulous investors. Alphabet's Waymo was also having humans take over for some of the driving as well.

And everyone knows that ChatGPT Pro is exclusively powered by capuchin monkeys.

I had no idea. There was an Amazon Go right in my workplace in 2019 (Brookfield Place) and I got lunches there almost daily. I loved it -- felt like magic, and it was crazy fast. I guess it was just an illusion (as all magic is).

> I was wondering why there wasn't a DOJ concern when Amazon Go did the same thing:

"Mostly AI, but they failed at getting close enough to 100%" and "effectively 0% AI" are not the same thing.

Elon has also made a lot of claims over the years. Where is FSD or whatever they call it now? The whole solar roof tiles presentation was a lie at the time. P2P Starship travel is impossible but is being "sold" to the public as possible and many other things.

Exactly. In this case it's pretty clear how Nate was defrauding investors with the claims. Amazon Go made fraudulent claims, but not only had the legal savvy to hedge those claims, they didn't directly raise fund from investors based on those claims.

IANAL, of course.

AI standards for "actually Indians."

It's the same tech used at Intuit Dome for the food stalls.

Yeah, how quickly we forget.

[dead]

It’s fraud when they lie to investors, or allow them to assume the wrong thing.

Doesn’t matter what consumers believe, it’s more or less legal to lie to consumers about how a product works, as long as investors know how the sausage is made. (Though, in reality it’s near impossible to lie to customers without also misleading investors, especially for publicly listed companies)

In this case, investors were under the impression that the AI worked, completing 99% of transactions without any human intervention. In reality, it was essentially 0%

When you claim "without human intervention... except for edge cases" and the truth is it's all "edge cases" ie 0% AI.

> Saniger raised millions in venture funding by claiming that Nate was able to transact online “without human intervention,” except for edge cases where the AI failed to complete a transaction. But despite Nate acquiring some AI technology and hiring data scientists, its app’s actual automation rate was effectively 0%, the DOJ claims.

> I'm curious when it crossed the line into "fraud" here.

Fraud is often defined as gaining something (or depriving someone else from something, or both) via false pretences. Here the something is money (this is most commonly the case) and the gaining/depriving is gaining money and depriving investors of it. It is more complicated than that, with many things that fit this simple description not legally being considered fraud (though perhaps being considered another crime), and can vary a fair bit between legal jurisdictions.

A cynical thought is that the key line being crossed here is that the victims are well-off investors, if you or I were conned similarly the law might give less of a stuff because we can't afford the legal team that these investors have. This is why cases like this one are successful, but companies feel safe conning their customers (i.e. selling an “unlimited” service that has, or developers five minutes after signing up, significant limits). Most investors wouldn't agree to the forced arbitration clauses and other crap that we routinely agree to by not reading and subsequently not accepting the Ts & Cs, etc, and anyway can afford large, capable, legal resources where our only hope would be a class-action from which only the lawyers really benefit.

Another cynical thought is that the line crosses was the act of not being successful. I'm sure the investors wouldn't have cared about the fraud if the returns had been very good.

Crossing the line into fraud is how you pitch it.

I would imagine it turns into fraud when you don't tell investors about the human fall backs.

The funny thing is you could probably make money on Amazon Mechanical Turk by hooking it up to an LLM. We’re at this weird limbo point in history where the fraud could go either way, depending on what you think you’re paying for…

This should be the headline link.

It contains the details people are asking about, including (to me) what made this actionable fraud: the solicitation of $40MM from investors based on the completely false representation that his company used AI.

Exactly the approach I’ve taken for my startup and is baked into the business plan. I have some pretty unique flows leveraging current-gen LLMs. Then there is an AI agent marketplace. It’s there because shoot, maybe AI agents will be super potent and I want a place for them to be integrated. At the same time, the product works perfectly fine with just humans on the platform. It’s a hedge.

The world is run by people with no ability beyond being able to 'talk a big game.' Business promotes people with no ability beyond being able to 'talk a big game.' Investors fund people with no ability beyond being able to 'talk a big game.' It's all talk and bullshit, all the way up the totem pole.

remember Gates just write a post about how he "lied" to CEO regarding their non-existing Altair BASIC software