Map Features in OpenStreetMap with Computer Vision
92 comments
·March 22, 2025stereo
daavoo
> But we still need human intervention to make sure the detected objects are drawn correctly.
Hi, I am the author.
The demo app and any provided code example includes a step asking a human to verify the detected features. You can't upload them automatically unless you modify the source code.
I reiterate the human verification across the docs, linked post, and any code samples.
I haven't ever uploaded features automatically. In fact I manually edited and labeled hundreds of swimming pool samples myself before even training the first version.
Happy to hear and implement any ideas on how to improve the process to prevent automated features to be uploaded.
I know some people might say: just don't publish the tool, I think we can do better at embracing AI and having an open discussion.
boredpudding
I don't understand the 'human verification' aspect.
Your docs show a simple image where the user can choose to keep a new object or not. [0] Afterwards it says: "The ones you chose to keep will be uploaded to OpenStreetMap using upload_osm.". This is uploading features automatically. The fact that it asks 'are you sure' is just silly. We all know if humans have to click yes 90% of the time, and no 10% of the time, they'll miss a lot of no's.
The image also proofs that:
- You don't see any polygons properly. You just see a an image of where the pool is. Already on the image I can see that if the polygons align to that image, it will be a total mess.
- You don't see any polygons further away from the object.
Both these points are in stereo's reply that the resulted data was a mess.
Please consider pulling the project. This will generate a lot of data that volunteers will have to check and revert.
[0] https://github.com/mozilla-ai/osm-ai-helper/blob/main/docs/s...
daavoo
> This will generate a lot of data that volunteers will have to check and revert.
This is just not true. The data can be easily identified with the `created_by` tag. And I have been reviewing myself any data uploaded with the demo (with a clear different criteria on what is good enough)
yorwba
> Happy to hear and implement any ideas on how to improve the process to prevent automated features to be uploaded.
Idea: do not automatically create features that a human can simply approve, instead require them to draw the polygon themselves.
K2h
long time editor of osm here. what you describe is what the rapid [1] editor from meta does where user is forced to manually select objects overlayed sat imagery. is limited to 50 objects before user must push. a great method i think
daavoo
That is what I will implement before bringing back the demo https://news.ycombinator.com/item?id=43448649
IshKebab
That kind of defeats the point surely?
halosghost
I am afraid your “human in the loop” isn't [0].
You acknowledge the problem of AI slop up-front, but seem to have chosen to plow forward anyway. Please do consider your actions and their effects carefully [1]. You are in a position of potential influence; try not to squander it.
All the best,
-HG
hgomersall
Tracing satellite data is really boring to do but easy to check. I would describe AI acting as a centaur here, or perhaps a pairing of equals.
Vinnl
"plow forward" and implying they didn't consider their actions and their effects seem ungenerous, given the list of precautions in the comment you're replying to.
You can disagree on whether the measures are effective, of course, but they're clearly not thoughtless.
necovek
I believe your critique would be more valuable if you've actually uncovered cases where the human in the loop missed these issues and bad or "wobbly" data ended up in the OSM database: did this happen? (the other comment from another person confirms it has)
Otherwise, you are discounting their effort based on a prejudice — others might be unable to supervise an AI, but someone who's actually developed it might have a better chance of success.
null
banana_dick_2
[dead]
cratermoon
[flagged]
daavoo
> straight or rectangular objects as wobbly, as shown in the second-to-last screenshot.
This is a because the polygon is drawn as a mask in order to overlay it on the image. The actual polygon being uploaded doesn't have the wobbly features.
It is True there are cases were the predicted polygon is wobbly and I encourage people to discard them. However I didn't publish this demo until I got a first version of the model that reached some minimum quality.
There is logic in the code to simplify the shape of the predicted polygon in order to avoid having too many nodes.
stereo
Hi! The Data Working Group had a look at the data, and decided to revert the two pool changesets. The polygons the algorithm had drawn were consistently of poor quality with stray nodes and nodes far outside the pool boundaries, and the imports hadn't been discussed with local communities.
daavoo
Hi, thanks for the feedback.
I have disabled the hosted demo for now, and will remove the uploading part from the code in favor of showing an URL that will open the editor at the location.
If its of any help, you can find any contributed polygon with the tag `created_by=https://github.com/mozilla-ai/osm-ai-helper`. Feel free to remove all of them (or I can do it myself once I access a PC).
I will be happy to continue the discussion on what is a good prediction or not. I have mapped a lot of swimming pools myself and edited and removed a lot of (presumably) human contributed polygons that looked worse (too my eyes) than the predictions I approved to be uploaded.
boredpudding
Can you link the (now reverted) changesets? I can't seem to find them.
Aachen
I tried this out like a week ago and I was wondering the same so I tried to upload and... it's definitely uploading crap. I don't know what to tell you but all the clearly square ones I saw have bends on the straight lines
It's useful for finding ones that haven't been mapped but not for drawing them. It can get the 4 corners pretty accurate for pools that are square, many are half round at the ends though
daavoo
Hi, sorry if the project or narrative gave the wrong impression but my idea was to show the potential, not providing a polished solution.
As disclaimed in the demo and code, the example model was trained only with data from Galicia on a Google Colab. A robust enough models would require more data and compute.
> it's definitely uploading crap.
What was uploaded was what a human approved.
> It's useful for finding ones that haven't been mapped but not for drawing them. It can get the 4 corners pretty accurate for pools that are square, many are half round at the ends though
I couldn't dedicate enough time on the best way to refine the predictions, but happy to hear and discuss any ideas.
Ideas I have are:
- Try an oriented bounding box model instead of detection + segmentation. It will not be useful for not square shapes but will definitely generate more accurate predictions. - Build some sort of https://es.wikipedia.org/wiki/RANSAC that tries to fits rectangles and/or other shapes as an step to postprocess the predicted mask.
banana_dick_1
[flagged]
AyyEye
Replied to daavoo, can I suggest adding a tag for ML originated features? As other comments have stated, it is likely that these tools are already being used (potentially semi-automatically) and this could help prevent them from polluting the DB wholesale.
banana_dick_5
[dead]
banana_dick_3
[dead]
banana_dick_1
[flagged]
throwaway346434
Oh, great re swimming pools - solar detection is another one on my list to have a go at.
I feel like a lot of the pushback here is an idea that OSM can grow from hand mapping; but as someone with 60k changesets over a decade... no amount of human volunteer enthusiasm is to the point that it can "solve" mapping at a global scale to the standards that make the map data overwhelmingly useful.
I feel we need a scalable framework for importing and maintaining data: ways to annotate the quality, sources, where to report bugs in the data source, and guidance to consumers. Ie if I want to query "businesses of type X" "mapped by humans within the last year", I can sort of do that with "check date".
But who knows how many of those attributes are accurate, or if the mapper who checked only checked one aspect (name/location)? Would it be better to ingest alltheplaces opening hours to maintain this data automatically, every month?
Would it be better as a data consumer if I could filter to only certain sources I trust? Or I could use data - even if the polygons aren't perfect or similar, even with known limitations like "poi inferred by AI".
matkoniecz
> Would it be better to ingest alltheplaces opening hours to maintain this data automatically, every month?
I am working on such project
See https://community.openstreetmap.org/t/what-you-think-about-i...
https://www.openstreetmap.org/user/Mateusz%20Konieczny%20-%2...
https://codeberg.org/matkoniecz/list_how_openstreetmap_can_b...
Freak_NL
> Would it be better to ingest alltheplaces opening hours to maintain this data automatically, every month?
Alltheplaces plays dangerously loose with (also) using resources clearly marked as copyrighted and protected with an API-key. As it is that project can serve as inspiration, but it is incompatible with OpenStreetMap.
matkoniecz
Can you give any specific example?
I am currently working on project that would use ATP and I am vetting its spiders. So far I have not found any bad ones.
If you found one then knowing which one you mean would be highly useful!
(BTW, just marking something as copyrighted does not make it copyrighted)
Freak_NL
Why ask? An example was pointed out in this thread you started:
https://community.openstreetmap.org/t/what-you-think-about-i...
There are probably more spiders configured to do this.
> (BTW, just marking something as copyrighted does not make it copyrighted)
For OpenStreetMap, it means that at the very least the Licensing Working Group should have a look. When you combine a copyright claim with directly using a third-party API with an API-key without clearance from the owner, doubly so. This was already pointed out to you in that thread.
Currently, only spiders which directly use the websites and domains of the shop chain (or its owner) are cleared for use.
Take heed of what Andy Townsend from OpenStreetMap's Data Working Group wrote:
> OSM has traditionally avoided situations where it could be legally challenged by people with more money to pay lawyers than we have, even if, in a fair and balanced process OSM might actually be in the right; for the simple reason being that any legal cost could far outweigh other costs of runnng the project.
You are a senior mapper in our project. You know this.
cinntaile
Solar might be problematic. How will you discern between a solar panel and a solar thermal collector? They look practically the same but their function is very different.
orbital-decay
Experiencing automated mapping first-hand makes me extremely wary of it. I've travelled across South America on a motorcycle, and OSM has a large amount of edits there that look automated (particularly in Brazil), making it barely usable in certain places. I'm not even talking about rural roads but also fairly large cities.
pastage
Armchair mapping will always result in bad maps. I usually try to use mapwithme and leave photo notes describing issues when I travel, I take photos of fences and play grounds, other people take scenic pictures.
So it might very well be automatic mapping, just saying that I know my armchair mapping can be pretty bad when I gonand check it OTG.
Aachen
By bad, do you mean incomplete or that it's often actually wrong? Because my experience is that it's very complementary and I see things on aerial that I didn't irl and of course also vice versa. Mapping from the air seems to work well to me
North Korea should be an interesting example, I guess we have no chance of anyone commenting on it but there's people who've been mapping it from actual satellite imagery, so very poor quality compared to the airplane-based georeferenced photographs we're used to in most countries. If armchair mapping makes bad maps, that will be the best example to have someone check out, but alas. (The visitor-accessible parts may not count as much because a mapper could have visited those)
null
banana_dick_5
[dead]
banana_dick_4
[dead]
ks2048
I did some work in this field, although years ago. There is a huge amount of existing models, datasets, tools, etc.
nativeit
This is an amazing set of resources, thanks for sharing. I have been tinkering with QGIS, and signed up for a slew of public/private satellite imagery APIs to bring in data to play with. The EU’s space agency has a lot of really good data sources with fully open access (no user accounts necessary). I am looking forward to working with this new collection of ML-specific tools.
daavoo
> really good data sources with fully open access (no user accounts necessary)
Ola!
I am the author of the repo, worked in satellite projects for the Galician goverment some years ago.
You don't need an account to download the data from OSM (you do need to contribute back, which makes sense IMO). You don't need an account to download tiles from some publicly available sources (i.e. https://pnoa.ign.es/ in Spain) but I prefer to made the code work with MapBox and let them pay the infra (until they stop free offering). Happy to share a simple snippet to use a different tile provider.
Any public dataset (that I am aware) is not really meant to be frequently updated, at best you get a second version release a year later. A lot of public money (I know because have been payed a small portion of these budgets) is spent on building datasets that are used for a couple of research papers, uploaded to the web and then become outdated.
If I want to help updating them or correcting label mistakes, in most (all?) cases, there is no practical way for me to do it.
I believe that OpenStreetMap has the potential to be the best publicly available spatial "dataset" for (some specific) CV use cases.
If we focus on creating ways to contribute with quality data (the idea behind this small project), it will just keep getting a better dataset, that anyone can contribute to be up-to-date.
banana_dick_4
[dead]
ySteeK
Wait, we are not mapping things we see in sattelite images, we are mapping things that have ground truth.
Please do not contribute anything ai-fantasized
moffkalast
Satelite images are the ground truth OSM is traced on. And the quality of those tracings varies wildly at times, I've had to fix weirdly offset shores that had roads on them placed on the sea on more than one occasion.
If this can be somewhat consistent then it'll probably do better than the average OSM contributor. Something like segmenting houses, roads, bodies of water, comparing against current data and highlighting inconsistencies for correction would be a good start though.
mistrial9
> it'll probably do better than the average OSM contributor
let us reconsider this statement, please. An unexpected and powerful effect of the Openstreetmap project is iterated convergence on ground truth. No person is perfect in contribution, and few people are consistently terrible. Revision and updates, common vision towards accuracy, an appreciation of cooperative contributions.. have astounded the public and humbled critics repeatedly. Not because every key stroke and mouse click is perfect, but because iteration and plural sources have converged in a usable system of software and data.
AI inputs to Openstreetmap are not new, as noted in other comments. The path forward is bright, useful to humans and participatory in the Openstreetmap project.
qwertox
Google would not allow this, but Mapbox seems to be OK with this, if it is used for non-commercial purposes or OSM, and only if their satellite data is used (not their vector data):
1.6. No Tracing, Deriving, or Extracting. Customer shall not trace or otherwise derive or extract content, data
and/or information from the Service Offerings except that Customer may use Studio or third-party
software to trace Mapbox Maps solely comprised of satellite imagery to produce derivative vector
datasets (i) for non-commercial purposes or (ii) for OpenStreetMap.
mtmail
Bing also allows OpenStreetMap mappers to use their aerial imagery for tracing. https://wiki.openstreetmap.org/wiki/Bing_Maps#Aerial_imagery
banana_dick_4
[dead]
banana_dick_3
[dead]
banana_dick_4
[dead]
pierotofy
Worked on something similar a few months ago (albeit for smaller scale geographic data): https://github.com/uav4geo/GeoDeep
daavoo
Hi there! That is some cool work, happy to chat and discuss ideas for collaboration :)
banana_dick_2
[dead]
anakaine
Id love to see a bit more detail on fine tuning SAM/2 to do things like detect pools or solar arrays. Both these are fantastic things to have mapped for community resilience projects, but I've not been able to follow along with SAM2 fine tuning at all.
I've got a Yolov8 model which does quite a good job of finding and segmenting out solar, but the edges are absolutely horrible and require an insane amount t of work to clean up. I've seen results from SAM2 that has been trained, and the results look massively better.
Wouldn't put these in OSM due to stereo6 comment about accuracy, but I could sure use them elsewhere.
gpvos
Can Mozilla please focus on making a good browser?
curtisszmania
[dead]
banana_dick_7
[dead]
Hi from the OpenStreetMap Foundation. Please don't add AI-detected features directly to the database.
The algorithms have problems with false positives, and with mapping straight or rectangular objects as wobbly, as shown in the second-to-last screenshot.
As a helper to detect missing features, this is a precious tool. But we still need human intervention to make sure the detected objects are drawn correctly.
See also: https://wiki.openstreetmap.org/wiki/Import/Guidelines and https://wiki.openstreetmap.org/wiki/Automated_Edits_code_of_...