Grafana: Why observability needs FinOps, and vice versa
40 comments
·February 6, 2025brunoarueira
sda2
ha, we use NewRelic for our application but the company is so cheap, they won’t even buy the infra team a license!
brunoarueira
Haha appears some managers change only the address but the behavior is the same, I already worked on a place where we had to use free heroku addons besides the paid PostgreSQL and the dynos
jsiepkes
One of the advantages of self-hosting is that you don't need this level of FinOps. You also don't have to live in fear of bill-mageddon.
danpalmer
And one of the disadvantages is that you can't solve problems by just spending more. It's a real trade-off, and too often is simplified to one option being obviously better.
m1keil
You can't spend more fast, but you can always spend more.
danpalmer
Why would you spend more if it's not solving problems?
You can certainly spend more, but on-prem/self-hosted, time is usually the limiting factor, either directly or through opportunity cost. Contrary to popular belief, time does not always equal money, if you need more storage and the blocker is that you need to build out an S3 equivalent (rather than just paying for S3), then you'll be blocked by hiring, by hardware lead times, etc.
jterrys
lol.
We recently switched from Grafana to Prometheus. Reason being that a license refresh took longer to process on their end. What happens when a license expires on Grafana? They fucking shut down all your shit cold turkey. Don't care if you're in prod or have a dedicated guy on their end for support or whatever. So you're happily churning along and then suddenly you're blind. Nice FinOps. With Prometheus there's a grace period where they'll happily overcharge you. But we've never had a product absurdly blow up on us like this before. It's truly mind boggling that they're out here talking about 'FinOps' now.
kozikow
I am big fan of "cost monitoring".
In my previous company I had a good setup for costs monitoring - including release to release comparisons, drill downs, statistics, etc.
After each release I looked at this data. It saved a lot of $, by simple fixes like "why we are calling this API twice?".
It also quite some issues that weren't strictly customer related, but weren't apparent from other type of data (you will always have some "unknown unknowns" in your monitoring, and costs data seem to be pretty wide net to catch some of those)
m1keil
What levels of observability did you have for costs of data transfer and how did you do it?
m1keil
So next time someone says "in cloud you don't need anyone to manage it for you" you can link them this article.
floating-io
Seriously? Not everything needs to be xxxOps. And if you need a "FinOps" team to manage your cloud cost, I would argue that there's something wrong with your whole damn paradigm.
StratusBen
I'm the author of the blog post and the CEO of https://vantage.sh/
I kind of agree that I don't love the term. That being said it has become the de-facto way that people refer to the space and practice. The FinOps Foundation hyped up the term and space quite a bit which they deserve credit for but do wish there was a better name :)
sebazzz
> And if you need a "FinOps" team to manage your cloud cost, I would argue that there's something wrong with your whole damn paradigm.
FinOps isn't a dedicated job but something a cloud engineer can do as part of its job. In the same way that DevOps doesn't need to be a dedicated function itself.
And as for the cloud... Yes that turned out to be a whole lot expensive for companies than predicted - and you share compute with anyone so that dualcore CPU isn't always that fast. But cloud is also flexible and that is where FinOps comes in.
j0rdans
I think for most smaller orgs you can get away with an off the shelf product to surface some more basic cost stuff. In relatively large engineering orgs, you're looking at optimising on stuff like cross-region calls to save millions a year so yeah there's a good reason to invest in cloud cost management.
moandcompany
The _Ops team needs an _Ops' team.
The game here is defining parts of the job away to be someone else's job or responsibility.
arccy
we invent new functions to use to decorate our CVs with
svilen_dobrev
> And if you need a "FinOps" team to manage your cloud cost,
somehow i misread that as "massage your cloud costs", and it.. sticks..
m1keil
FinOps isn't even new (my definition of new is "once you have an O'Reilly book on the topic it is not new anymore").
aledalgrande
I had the same thought. Why do we need to keep coming up with weird names/acronyms/portmanteau
MortyWaves
Sadly it’s one of the only ways of getting through the thick skulls of dumb middle management that seem to always be leading software projects despite having no technical background.
If it’s not got a familiar marketing thing going on, they’ll refuse to acknowledge it even if their devs are practically begging for it.
aledalgrande
> dumb middle management that seem to always be leading software projects despite having no technical background
Or you change to a company with capable & technical middle management :)
It always blows my mind when people say "management does not have to be technical".
senko
Wait until you hear about "RevOps".
At this point I'm waiting for someone to start flogging "CodeOps".
NotGMan
You know you have a problem when you're afraid to add a few more metrics because the bill might get too high.
danpalmer
This was the reason that we ended up choosing Datadog over Grafana Cloud at my previous place. Most metrics came from "integrations", and Datadog doesn't charge extra for any of those (they just curate them so that the cardinality can't get too high), whereas Grafana charges (charged?) for each metric and didn't provide anything to reduce cardinality. Their solution was to suggest we did more engineering and ran more infrastructure to aggregate before sending to them, not something we wanted to invest in given that the whole point was to not self-host Grafana anymore.
Datadog is expensive, but at least we were only making these decisions for the ~hundreds of custom business metrics, and not the ~tens of thousands of metrics from our infrastructure.
EfficientDude
What's Grafana? I see it mentioned here a lot, nowhere else though. Is it a YC property?
YZF
Very popular open source dashboard/monitoring software: https://grafana.com/
Nothing to do with YC AFAIK.
brunoarueira
Grafana is mostly knows for the most used interface to query Prometheus and create dashboards for collected metrics
MortyWaves
And more recently inventing and reinventing Prometheus alternatives. It’s a bit much trying to keep up.
rajin112
Observability tool
On my last job, the company was using NewRelic (for two environments we was using at the time) which had an ok cost and "suddenly" we'd been forced to use Datadog which costs way over for our budget and after the person responsible for the change and integration see the estimated high costs, started to cut everything possible to keep it low. So, our tools degraded and we wasn't able to test things on staging and collect metrics like we was when using NewRelic. FinOps is certainly a good approach, but we need it from the start!