YOLO-World: Real-Time Open-Vocabulary Object Detection

If this is military related, im terrified of the future. Sci-fi movies with crazy drones from back when are no longer that cute.

Is this OSS?

YOLO is way faster. We used to run both, with YOLO finding candidate bounding boxes and SAM segmenting just those.

For what it's worth, YOLO has been a standard in image processing for ages at this point, with dozens of variations on the algorithm (yolov3, yolov5, yolov6, etc) and this is yet another new one. Looks great tho

SAM wouldn't run under 1000ms per frame for most reasonable image sizes

SAM doesn't do open vocabulary i.e. it segments things without knowing the name of the object, so you can't ask it to do "highlight the grapes", you have to give it an example of a grape first.

You can build a pipeline where you use: GroundingDino (description to object detection) -> SAM (segmenting) -> Stable Diffusion model (inpainting, I do mainly real photo so I like to start with realisticVisionV60B1_v51HyperVAE-inpainting and then swap if I have some special use case)

For higher quality at a higher cost of VRAM, you can also use Flux.1 Fill to do inpainting.

Lastly, Flux.1 Kontext [dev] is going to be released soon and it promises to replace the entire flow (and with better prompt understanding). HN thread here: https://news.ycombinator.com/item?id=44128322

There are plenty of Stable Diffusion based models that are capable of inpainting, of course they are heavier to run than LaMa.

Same here but for urban foxes.

We had motion triggered sprinklers that worked great, but they did not differentiate between foxes and 4 year old children if I forgot to turn them off haha.

We have more or less 360 degrees CCTV coverage of the garden via 7 or 8 CCTV cameras so rough plan is to have basic motion pixel detection to detect frames with something happening then fire those frames off for inference (rather than trying to stream all video feeds through the algorithm 24/7) and turn the sprinklers on. Hope to get to about 500ms end-to-end latency from detection to sprinklers/tap activated to cement the "causality" of stepping into the garden and then ~immediately getting soaked and scared in the foxes brains. Most latency will be for the water to physically move and make the sprinklers start, but that is another issue really.

Probably will use a RPi 5 + AI Hat as the main local inference provider, and ZigBee controlled valve solenoid on the hose tap.

Funnily enough, that was my computer science capstone project back in 2010!

I don’t know if our project sponsor ever got the company off the ground, but the basic idea was an automated system to scare geese off of golf courses without also activating in the middle of someone’s backswing.

Old stuff is often reposted here to attract attention. It mostly goes unnoticed.

The setback of YOLO architectures is that they use predefined object categories that are a part of the training process. If you want to adapt YOLO to a new domain you need to retrain it with your new category label.

This work presents a version of YOLO that can work on new categories without needing to retrain the algorithm, but instead having a real-time "dictionary" of examples that you can seemlessly update. Seems like a very useful algorithm to me.

Edit: apologies i misread your comment I thought it was asking why this is different that regular YOLO

It was new to me, serf. And judging by the number of upvotes, it was new to a few other people too.