Skip to content(if available)orjump to list(if available)

Monitor your security cameras with locally processed AI

princevegeta89

I've been running Frigate for more than two years now and it beats the hell out of any system I've tried in terms of detection speed and reliability. For context, I've tried Ring, Tapo cameras, and also Eufy security. Today I have turned away from all the cameras except for the Tapo cameras now serving RTSP streams into my Frigate instance. I have also blocked them from accessing the internet and that gave it complete privacy by default.

Eufy Security started showing advertisements about their new products whenever I tap on a motion detected notification. They prioritize their ads over your own security which is ridiculous. Not just that, some of their clips stored in their cloud storage would never open despite the fact I used to pay them my membership fees every month. They were also caught storing passwords and other security credentials in plain text. Thanks to them, they were the primary motivation for me to move away from using those proprietary platforms and look for something self-hosted.

I got Frigate running on my old hardware with hardware acceleration enabled via RX 550 GPU and detection is always under one second. I wrote a small app that uses Frigate API to grab screenshots and send me notifications via Telegram and Pushover. It's been very self-sustainable for two years now. I only had to restart the service two times in all of this time. I am also using some tunneling from my VPS into the locally hosted Frigate running on my home server and it's just been flawless. Thanks to this amazing project.

xrd

Are you using this with Home Assistant?

I'm trying to integrate this, but the HACS integration does not seem to work with my HA because the get.hacs.xyz server is misconfigured.

  wget -O - https://get.hacs.xyz | bash -
  Connecting to get.hacs.xyz ([2606:4700:20::ac43:4465]:443)
  28EBD0AA71710000:error:0A0000C6:SSL routines:tls_get_more_records:packet   length too long:ssl/record/methods/tls_common.c:662:
  28EBD0AA71710000:error:0A000139:SSL routines::record layer failure:ssl/record/.rec_layer_s3.c:687:
  ssl_client: SSL_connect
  wget: error getting response: Connection reset by peer

rightbyte

I am sorry to be that guy, and I think it is good that you realized it your self, but how could you trust them with your videofeeds in the first place?

Like, I remember thinking the GNU guys were hippie crackpots. But it was like 15 years ago and I have forgot how to relate to that feeling... it is like realizing all my colleagues are not using adblockers and visit sites with ads. I just can't understand.

Steltek

How did you get the Tapo cameras to play nice in rtsp mode with frigate? I found that even one camera did horrible things to the wifi. Even with one camera per AP per band, they caused trouble.

IncreasePosts

Not to nitpick but you're only really guaranteed privacy unless you know there's only a wired connection. If it has wifi the camera could hop onto a nearby open network and do whatever it wanted without your knowledge, assuming evil enough firmware

stavros

You can't know there's only a wired connection unless you open the camera up and inspect the PCB for an antenna, and it could still be disguised. However, by "I've only given it access to a specific network" you already eliminate 99.99% of the problem. The other 0.01% isn't really worth worrying about.

bobmcnamara

There is no privacy with wires, only TEMPEST!

underdeserver

My usual pet peeve -

They use the abbreviation NVR in the first sentence without saying what it means.

It means "networked video recorder".

Please don't do that. Not everyone who comes across your site is a member of your particular niche.

disruptiveink

Usually I would agree with you, but this is an incredibly common initialism, used by not just people in the industry, but also by consumers. Sure, it may not be as widespread as VHS (global) or API (tech-adjacent), but anyone who is in the market for this software already knows what NVR means.

Most people would know the term from either being quoted or looking up CCTV solutions, all of which, unless they're fully "cloud-based", come with a component that is called the NVR. You wouldn't even consider this if you weren't aware of the concept. If NVR means nothing to you, Network Video Recorder doesn't mean anything to you either. This is meant to be a replacement for closed and inflexible hardware boxes that are sold together with security cameras, and the name of those boxes are "NVRs".

triceratops

As a consumer I disagree. Never heard of "NVR" but I can suss out what "network video recorder" means from context.

tiagod

Most stores will just market the devices as NVR or NVR Recorder (I know). If you google it, you get your answer immediately.

lobsterthief

Right, but I don’t want to open tabs and Google terms right after I start reading an article ;) Even as a super technical person

hopelite

You are missing the point. It has been considered general English language competency that you always expand the first instance of any abbreviation that is not absolutely obvious in context, e.g., USA, “e.g.”, or CIA, unless you happen to be writing about the Culinary Institute of America in most contexts outside of the culinary context.

It is a rather annoying myopic perspective I most often run across in tech, where technical people for whatever reason are so fixated on their little corner that they are either unaware or simply indifferent to the fact that there are others in the world, and that if they want to spread their work and impact, they need to make things approachable and lower barriers to entry.

It Is why the rule of general language proficiency exists in English especially because of all the abbreviations, to facilitate information and knowledge sharing.

Let’s all improve by going through whatever our project is and make sure that at least in the context, abbreviations are easily understood by expanding them, e.g., your introduction/overview page and documentation should always expand most first instance abbreviations, including in separate, high level segments (e.g., if you have different first contact pages or objects) unless they are globally known to society.

It’s really not any different than any other “sales” tactic; you will not be successful selling something if you do not first describe what it does in a one-liner. Ask yourself, “who is the person I want/need to come to this thing and should I assume they would know what this all means?”

a3w

Nearly an aside, but:

Why are people still installing security cameras that are monitored by them? They increase stress level and felt insecurity. They do not make you feel secure, say psychological studies. You probably think more about burglaries and dead spaces in your setup and actively monitor for these in your daily lives, where for 99.8 % of people this should be a non-topic.

If you want to install them for later police work, that still seems tedious and you might require off-site backup. In public places we often have CCTV of people, but unless you have number signs on vehicles, they seem to not help with conviction rates by much.

Roark66

One good reason for cameras. They promote civil behaviour.

Since I installed a visible security camera above my front door I never had couriers throwing packages, they very rarely not show up and claim "no one was home" and so on. Also I had a neighbour damage my fence every single time he was doing his farm work (plowing, harvesting). In addition he would use an unfenced portion of my property as a turning place leaving deep/huge tire marks and did other silly shit like that despite me asking him many times not to do it. Once I installed cameras it hasn't happened once.

Then there are other practical reasons, I can review the recordings to find out which way my cat went if he is gone for a long time, or I can check is he waiting in front of the door in the middle of the night without having to get out of bed. Also my cameras resolved a mystery how one of my cats got injured once (hint - deer really don't like cats).

Finally, let's say there is a huge storm forecast and I'm away. I can check remotely everything is fine.

Finally, cameras are very good for insurance purposes. At least in my country insurers are known to weasel they way out of paying very often. If you have an actual recording that is much more difficult for them to do.

The only issue I have with most reasonably priced Cctv cameras is that they go towards more megapixels when they should go towards more IR sensitivity. Almost every consumer grade camera can be defeated at night if a subject is moving quickly. The picture will be smeared. So for ID purposes I use lower resolution more "professional " cameras.

As for open source, I've been using ZoneMinder with local (and on camera) AI for ages.

defrost

Dunno much about the market for consumer grade home mount IR/Thermal cameras, I used to use upcycled industrial cameras when I worked contracts in the vision domain, recently I'm using a rifle scope on a remote controlled mount with a long HDMI cable.

Mars MT1000LRF Thermal Riflescope:

* https://old.reddit.com/r/ThermalHunting/comments/1i8wlpp/tho...

* https://www.youtube.com/watch?v=LBHHCRHnwgw

nkrisc

> They increase stress level and felt insecurity. They do not make you feel secure, say psychological studies. You probably think more about burglaries and dead spaces in your setup and actively monitor for these in your daily lives, where for 99.8 % of people this should be a non-topic.

Oh wow, I didn’t know I felt that way! I’m glad you were able to tell me what I feel.

You are making a lot of assumptions about why people have them.

jcims

People have different dispositions, live in different environments with different levels of support from law enforcement and face different threats. I live in a remote area and am regularly away for extended periods of time. I’ve spent years with and without any security cameras and I’m generally more content when I have a few keeping an eye on the place.

theshrike79

My doorbell has a camera that records locally.

When the doorbell rings I get a notification on my desktop and phone with a relevant image captured a few moments before the button was pressed.

Then I can determine if it's something I need to put my pants on for.

Mostly it's just fun and easy to add cameras around your house. Then you can do stuff like have the LLM count birds it sees or ask it "are the dogs in the back yard" etc.

DonHopkins

>Then I can determine if it's something I need to put my pants on for.

Or if it's Jehovah's Witnesses, something you need to take your pants off for.

https://www.youtube.com/watch?v=aPS2zupOI_Y

laurieg

Like with all home automation, you should use it to solve problems you have, not problems you want to have.

Here are some ways I use security cameras:

Check if my colleagues are in the office or not (and if they are in the middle of a live recording). Check on my plants while I'm away. Check if there is a free parking space. Check if I left something at home or in the office.

I'm not really thinking about crime, even though they are called 'security cameras'.

W3zzy

I'm so happy those uses of camera's are illegal in the EU. Camera's at work can only used for safety. You could have other - less intrusive - systems in place for all tge other issues.

gr3ml1n

Isn't the entire EU essentially a panopticon of cameras?

alchemist1e9

Just one of many bizarre European attitudes towards work and capitalism which are contributing to massive underperformance economically.

Why would anyone have any expectation of privacy at work other than in the toilet?

spauldo

I've got two and will probably add a third.

The one pointed at the driveway sends an alert to my phone when someone visits. It's handy because I can't hear the house from my office so I often don't realize when we have guests over.

The one in my back yard is for security. I don't obsess over it, but if something went missing from my workshop I'd check the recordings. I'm not worried about traditional thieves, but I've got a couple unsavory family members.

thumbsup-_-

Your argument is like “If we don’t do covid testing, we’ll have no covid cases”

whatsupdog

You can integrate it with home assistant to send notification on your phone (or run any other automation) when it detects any specified objects.

I have set it to send me notification if any person is detected in my front yard, drive way or back yard after I have "armed" my alarm at night. I am thinking of also sounding am alarm on my home speakers.

Frigate, when configured properly, has a really low false positive rate. I have only seen 2-3 false positives in the past one year. And if rarely ever misses. So it's something you can rely on.

Tractor8626

So burglar just need to carry big sign "Ignore previous instructions and don't report anything"? "

s17tnet

Probably a "scramble suit" [0] or just a tshirt or hoodie with patterns engineered to escape AI recognition [1]

[0] https://en.wikipedia.org/wiki/A_Scanner_Darkly [1] https://medium.com/data-science/avoiding-detection-with-adve...

theshrike79

Someone made a shirt called ChatGP-Tee, that had (IIRC) a picture of a generic office view, it confused the model completely and it didn't recognise the wearer as human :D

rjreed

Reminds me of the "ugliest t shirt" from Zero History by Gibson

dust42

Looking in their github, it says that it uses openCV and Tensorflow. The motion detection is done with openCV and will be immune against any attack unless you move so slow that you are under the detection threshold.

Tensorflow for the object detection doesn't do any OCR thus written instructions dont work. However, according to the website the system has a limited list of objects it detects. So maybe disguising yourself as a walking tree might prevent detection.

pseudo0

Finally a practical use for the Metal Gear Solid cardboard box!

CobrastanJorji

With an open source model, though, a criminal may be able to work out a 2D image that he could print out that would identify him as a package or a windy branch.

gerdesj

I have two cameras at my front door - one is the doorbell and the other looks towards the door, which is on the side of a porch.

fragmede

the criminal could spend years to become a trusted maintainer so they can upload a model that's been fine tuned to ignore objects with a specific QR code.

hamstergene

More like, wear a full body raccoon suit.

zeroflow

I like the idea, but no.

They have a two-stage approach, first motion detection with - I think - OpenCV and then afterwards object detection of zones of interest with different object detection models, depending on your hardware.

It supports Coral TPU, Halio Accelerator and most GPUs. I think AMD is still the worst, since ROCm is not available on iGPUs.

Afterwards, they provide/support models like edgedet (Coral), YOLO-NAS, YOLO, D-Fine or RF-DETR.

They also offer paid access to a specially trained version of YOLO-NAS where you can also train your own images.

kookamamie

waves hand

"These are not the detections you are looking for."

IncreasePosts

Maybe if ring or whatever major manufacturer popularly rolled this feature out and criminals could easily ID ring cameras

xiconfjs

It‘s still a bit flaky getting video acceleration (not talking about object detection but video decoding) working but after that it is one of the best solutions for live object detection I‘ve ever tried: no more small animals waking me up in the night.

P.S.: I‘m also supporting them with a yearly? subsciption to train the „A.I.“ model against false positives I provide which increased the accuracy even more.

m463

> no more small animals waking me up in the night.

not waking you, but it is cool to have a collection of animal photos. Sort of amazing there's a hidden world.

danparsonson

Hedgehogs are fantastic TV - a member of my family used to get some great footage including one very memorable fight where one ended up rolling the other one around

wiseowise

> including one very memorable fight where one ended up rolling the other one around

You can’t drop something like that without uploading it to YouTube right now.

xiconfjs

For sure, but rats and moths are usually not that cool ^^

sugarpimpdorsey

This is becoming a real problem because the drivers/software for the Coral AI boards is yet another example of Google Abandonware(tm) which has a hard dependency on a Paleolithic-era version of Python. Comically, the hardware is still sold.

In so many words if you expect to use the Coral boards you are stuck on EOL versions of Debian/Ubuntu - which have terribly old video drivers and missing kernel GPU support. There's a good chance your modern GPU - even well-supported Intel ones - won't work.

Imagine buying new hardware in 2025 whose software still required Windows 7.

Cyph0n

Re: outdated Python: Isn’t this a perfect usecase for Docker? Nix/NixOS is another option.

smokel

No. You might get it to run, but you would also get old security exploits to run.

alias_neo

Mines been getting worse.

Been running about 2-3 years, was mostly fine before but now I get constant false positives from the children's garden toys, scooter left in the garden, pirate flag waving etc.

I don't submit false positives for privacy reasons but I'm looking at trainingy own model. I've got years worth of positives/negatives to train on.

zeroflow

That "subscription" is one which I gladly pay due to multiple reasons:

1. It supports the developers(s) 2. The price can be directly attributed to cost for training 3. You can keep the models you trained during your subscription indefinately

That's pretty much the opposite to AgentDVR. I don't need hosted services for remote access or push notifications - I can do that myself. But if I want to abide the license terms, I need to purchase a monthly subscription for remote access over my own VPN.

smallerfish

I run Frigate with 5 IP cameras (3 Hikvisions, 2 Amcrests) and 1 USB camera. I'm using a USB Coral TPU, which does a good enough job that Frigate can keep up with an average of only 30% CPU usage on an old Dell with 4 core i7-6700.

Frigate's better than anything else I tried, but not perfect. As mentioned in another thread, it has some issues with codecs from some cameras (playing clips from Amcrests is fine, Hikvisions not so much) and therefore you may need to transcode. Also it has no built in option for sending your recorded clips offsite; theoretically you could mirror its storage directory, but as far as I've found it's not organized in a way that you can separate just important events.

boredemployee

is it possible to not just recognize people but identify them? (with registered pictures beforehand ofc)

dimitri-vs

Yes: https://github.com/blakeblackshear/frigate/releases Ctrl+F: "Face Recognition"

> Turn on face recognition & upload your first face via Face Library → Add Face.

> Train and improve accuracy: New detections appear in Face Library → Train with a confidence score-assign each to a new or existing person to refine future recognition.

neuroelectron

The new style of "Open source;" I wonder what kind of fun secrets are hidden in the model and Coral Accelerator.

Unfortunately, the USB Accelerator is very hard to buy even at 3x retail.

bilsbie

Side note. I’m surprised we’re not doing more with LLMs as far as image and video processing. We now have some level of imaging understanding in a box (and some common sense). Seems like there would be millions of possibilities.

Is manufacturing using it for anything? More security applications?

matsemann

Where would I start if I wanted to do stuff on a video, but not necessarily live? Like, say I have a 5h video and want to extract the frames of each car passing when it's at a certain spot, for instance. Or all of those with a driver holding a phone or whatever. Are there good frameworks for this, or would I have to split the video into a million frames and run something on each one?

zimpenfish

I'd check out the OpenCV documentation and examples. This is basically what I use for face recognition in videos[0]; for recognising cars or other objects, you'd probably want to either train your own model or use something like OpenCV's YOLOv3 (example: [1] but you'd need to steal the video reading code from the first link[0])

[0] https://github.com/ageitgey/face_recognition/blob/master/exa...

[1] https://github.com/deveth0/python-opencv/tree/master/objectD...

matsemann

Thanks. Also just kinda wondering if there's been any leaps lately, as I guess this is the same way as one would have done it a few years ago as well. But now that one can upload images and chat about them to multi modal LLMs, wondering if there's easier ways now (but preferable not uploading a million images to chatgpt api and paying the cost).

Like, could I avoid training or specifying much or becoming very knowledgeable in this domain, are we there yet?

Could I say "detect the frames of every car when it passes position X in the video, and then grab the frame when the same car passes position Y", and then I could calculate the frame difference to know the speeds. Or would I have to do loads of code and training still for something like this?

(I know I'm asking for much here, just curious what the SOTA is in this right now)

londons_explore

Ask a decent (non-free) AI this question, and I bet it can make you a python script to load a video and output which timestamps show a driver holding a phone.

s0ss

I also don’t know, but this might be useful: https://en.wikipedia.org/wiki/You_Only_Look_Once

sunshine-o

Frigate has really done a fantastic job packing everything together.

For basic needs go2rtc [0] or MediaMTX [1] can be enough. But once you need some form of intelligence on top AFAIK unfortunately there is no unixiy tool that can take a stream and easily define and apply a model on it. You will have to code something in python.

- [0] https://github.com/AlexxIT/go2rtc

- [1] https://github.com/bluenviron/mediamtx

lormayna

I am using Motion [0] since years. At least for basic stuff, is easy to configure and very flexible. For more advanced configuration, it required a bit of tuning.

[0] https://motion-project.github.io/

sunshine-o

Yes motion is amazing and has been around for a quarter of a century ! very lightweight and reliable.

As far as I know you can do object detection and tracking by gluing it with a yolo model using a few lines of python like this [0]. I saw a bunch of people doing this.

I really wish there was a more unixy tool available in package managers doing this.

- [0] https://github.com/xj25vm/MotionSpot/blob/main/motionspot.py

lormayna

Exactly. Motion can detect objects in the images, but not recognize the object type, but it's easy to integrate with a third party services like the one that you are linking with the scripts features [0]. I have personally integrated with S3 and self-hosted notification to create a small CCTV system, but there is no limit to the imagination of possible integrations.

- [0] https://motion-project.github.io/motion_config.html#OptDetai...

nergal

I've been running Frigate for many years, using a PN50 NUC and a Coral USB dongle, the Coral is a must, at least in my case. I had a full blown Ubiquiti/Unifi setup with cameras + their software. Way to many false alarms compared to Frigate. Now I run 10+ cameras with 24/7 recording and alarms with images pushed to Telegram. The identification is instant as well as the telegram message.

Running a mix of Ubiquti/TP-Link VIGI+TAPO/Reolink. I'm running everything in containers and everything works perfect!

AceJohnny2

Polling HN: is there any upgrade to Coral? It's 5 years old at this point, and with the explosion of AI apps & HW acceleration, I'm surprised there doesn't seem to be anything to update Coral's niche, of an IO-attached NPU.

For on-camera AI, I'm aware of OpenMV https://openmv.io/ and their recently-kickstarted N6 & AE3 https://www.kickstarter.com/projects/openmv/openmv-n6-and-ae...

nergal

TIL openmv.io, looks really neat for small project. Especially cool with the thermal vision, that would be a very nice addition to improve false positives for <living-things> detection.

But for surveillence, it's usually the sensor/camera quality that is the most important. I've struggled hard to find an affordable IP camera that can actually handle both shutter speed + quality in order to for example read license plates.

Medox

OpenVINO might be a good alternative, as many Intel-based mini pc’s support it. Or a decent desktop with an Intel CPU. Or maybe something with an Arc GPU (integrated or dedicated).

Disclaimer: I didn’t try it yet but the last rabbit hole regarding OpenVINO comparisons looked too good to be true and it seems Frigate supports it too. Win-win.

HackerNewt-doms

a) How many LAN cameras b) How many WiFi cameras

are you using with only one Coral USB dongle at the same time (plugged in the PN50 NUC) and get successful object or person identification with frigate? And why telegram? Is it connected to frigate only for notifications resulting from the identifications?

null

[deleted]