41. looking-for-inaturalist-observation-map-visualisation-suggestions

https://forum.inaturalist.org/t/looking-for-inaturalist-observation-map-visualisation-suggestions/7322/11

the api.inat.org and api.gbif.org URLs above won’t actually take you anywhere. but they do provide a GIS tool – in this case, ArcGIS Online – the instructions needed to get map tiles from iNat and GBIF.

you may have noticed that when you load many online maps, they get rendered a section or a tile at a time, not all at once. as you pan beyond the original extent of the map, new sections or tiles get loaded and added to the map. as you zoom in and out, new tiles get loaded to provide a better representation of the map at the given zoom level.

you’ll notice in the tile URLs i provided above, they all contain a {level}, a {col}, and a {row}. ArcGIS will dynamically replace these with a zoom level, an x value (or column index), and a y value (or row index). that tells the API to return a map tile that represents a particular section of the Earth at a given zoom level. when the API provides ArcGIS that tile (usually a picture file – in these cases, all *.png files), ArcGIS puts that tile on the appropriate place in the map, along with all the other tiles needed to fully display the map.

here’s how you can create a map with one of these tilesets in ArcGIS Online:

go to https://www.arcgis.com 1
at the top of the screen, find Maps, and click on it. (you don’t have to sign in.)
that should get you to a screen with a blank map. at the top right of that screen, click Modify Map.
that wll add some editing options to the toolbar below. find Add in that tool bar (on the left side of the screen), and click on that, then select “Add Layer from Web” from the drop list.
that will give you a pop up. in that pop-up, select “A Tile Layer”, then fill in the URL (using one of the URLs i provided before), a title for it (which is what show up in the left Contents pane in the screenshots), and credits (which is what shows up in the bottom right corner of the map in the screenshots). then click Add Layer.
finally, you may want to use a basemap that is less busy than the default topo basemap. you can either select one by clicking the Basemap button in the map toolbar, or searching for one via Add > Search for Layer. (ArcGIS has lots of layers available.)
if you don’t like ArcGIS, you can also do something similar in other GIS tools. the specific steps will differ, of course, but the concept is the same. (the URLs may differ in the other tools, too. for example, QGIS and Leaflet.js expect {z}, {x}, and {y} in the URL instead of {level}, {col}, and {row}.)

hope that helps. if you have other questions, feel free to ask.

also note there are a few more advanced things that can be done. for example, you can use different tilesets for different zoom level ranges. so for example, the hexagons that GBIF provides may look okay at that size in my example at a continent level, but they might look too much like points at a higher (more granular zoom level). so you might choose to use relatively bigger hexagons at higher zoom levels by changing the hexPerTile value in the URL (smaller values result in bigger hexagons in this case). you can also modify the transparency of the tiles within your particular GIS tool.

2

Reply

pisum
1
8d
it occurred to me last night / this morning that i could probably take the results from the UTFGrid tiles that the iNaturalist API serves up and use them to create my own custom observation maps. so when i got some time this evening, i tried to create my own custom map based on UTFGrid data, and it actually worked out better than i expected. i think UTFGrids might be a little less precise, but they are much more customizable than either iNat’s standard observation tiles or GBIF’s tiles, and they offer all the query options that iNat provides. below is an example of all observations from iNat. i compared it against another map i created from GBIF (https://forum.inaturalist.org/t/observation-density/6305/6), and it looks similar (which is good from a data quality perspective). i’ll probably work on it a little bit more when i get time just for fun, but if anyone is interested in it, i’ll clean it up a little bit more and can share the code when it’s more functional.

tbd%20utfgrid%20alternative%20implementation
tbd utfgrid alternative implementation.jpg
1534×1330 133 KB
UPDATE: i was doing some more sanity checks on the data before i got too far down the road of coding something that turned out later to be flawed – so far, things look ok – and i just wanted to share some more maps that i thought were interesting.

here’s a map that shows observations over the last 4 years (r=2019, y=2018, g=2017, b=2016, brightest dots represent at least 20k observations). you can sort of see how iNat usage is spreading across the world:
tbd%20obs%20by%20year%20r2019%20y2018%20g2017%20b2016%20max20000
tbd obs by year r2019 y2018 g2017 b2016 max20000.jpg
1535×790 102 KB
this map shows observations over the last 4 months (r=Oct, y=Sept, g=Aug, b=July, brightest dots represent at least 2500 observations):
tbd%20obs%20by%202019%20month%20r10%20y9%20g8%20b7%20max2500
tbd obs by 2019 month r10 y9 g8 b7 max2500.jpg
1536×785 120 KB
this map shows observations Monarchs over the last 4 months (r=Oct, y=Sept, g=Aug, b=July, brightest dots represent at least 25 observations). you can sort of see a shift in observations over these months related to migration:

https://forum.inaturalist.org/t/looking-for-inaturalist-observation-map-visualisation-suggestions/7322/11

k… i’ve been coding a bit, and i’m at a point of diminishing returns for further coding, i think. i didn’t get to the point of producing a mapping interface, but the code is here (https://github.com/jumear/stirfry/blob/gh-pages/iNat_UTFgrid_based_custom_density_map.html 2), and hopefully it’s relatively easy to understand and tinker with. you’re welcome to adapt it as you please.

here are some examples of different custom maps i created using the UTFgrids:
https://jumear.github.io/stirfry/iNat_UTFgrid_based_custom_density_map.html?example=1 2
https://jumear.github.io/stirfry/iNat_UTFgrid_based_custom_density_map.html?example=2 1
https://jumear.github.io/stirfry/iNat_UTFgrid_based_custom_density_map.html?example=3 1
https://jumear.github.io/stirfry/iNat_UTFgrid_based_custom_density_map.html?example=4 1
https://jumear.github.io/stirfry/iNat_UTFgrid_based_custom_density_map.html?example=5 1
https://jumear.github.io/stirfry/iNat_UTFgrid_based_custom_density_map.html?example=6 1
https://jumear.github.io/stirfry/iNat_UTFgrid_based_custom_density_map.html?example=7 1
https://jumear.github.io/stirfry/iNat_UTFgrid_based_custom_density_map.html?example=8 1
https://jumear.github.io/stirfry/iNat_UTFgrid_based_custom_density_map.html?example=9 1

UPDATE:

one last contribution here: https://jumear.github.io/stirfry/iNat_UTFgrid_based_density_map_for_Leaflet.html
(code: https://github.com/jumear/stirfry/blob/gh-pages/iNat_UTFgrid_based_density_map_for_Leaflet.html)

the previous examples were built in a static map viewer that i cobbled together. but this new example is built as an extension of Leaflet.js. so it might be easier to tinker with for those who are familiar with Leaflet.js, and this example is also easier for exploration since you can pan and zoom. the markers are also created a little differently here, in a way that is less resource intensive but is a little less flexible (tradeoffs).

that’s it for now.

hmm… if you’re trying to do heatmaps, then you should try the tilesets served up by iNat 1 and GBIF 1, if you haven’t already.

iNaturalist’s tiles are a little bit uglier in most cases (in my opinion), but they can be set to any color using the color parameter. GBIF’s tiles look a little better (in my opinion), and the polygon tilesets in particular offer lots of options for formatting, though there are fewer color options available.

here are 4 quick examples of observation tilesets for Diloma concameratum (Wavy Top) that i pulled up in ArcGIS Online over a dark basemap (each snapshot is preceded by the tileset URL i used for that layer):

https://api.gbif.org/v2/map/occurrence/density/{level}/{col}/{row}@1x.png?srs=EPSG:3857&style=orange.marker&taxonKey=5797922 2
tbd%20heatmap%20gbif%20marker
tbd heatmap gbif marker.jpg
1236×957 117 KB
https://api.gbif.org/v2/map/occurrence/density/{level}/{col}/{row}@1x.png?srs=EPSG:3857&style=iNaturalist.poly&taxonKey=5797922&bin=hex&hexPerTile=16 2
tbd%20heat%20gbif%20hex%2016
tbd heat gbif hex 16.jpg
1236×957 120 KB
https://api.inaturalist.org/v1/heatmap/{level}/{col}/{row}.png?color=orange&taxon_id=861925 1
tbd%20heatmap%20inat%20heat
tbd heatmap inat heat.jpg
1236×957 111 KB
https://api.inaturalist.org/v1/colored_heatmap/{level}/{col}/{row}.png?color=purple&taxon_id=861925

?interpolate_coordinates=true

photos/?advanced=on

Here's the link for creating a traditional project: https://www.inaturalist.org/projects/new_traditional
https://forum.inaturalist.org/t/looking-for-inaturalist-observation-map-visualisation-suggestions/7322/11
https://www.inaturalist.org/comments?mine=true
https://github.com/glmory/iNaturalist-Uploads
https://www.inaturalist.org/journal/glmory/21539-updated-python-upload-script
https://www.inaturalist.org/journal/ahospers/26917-7a-analysing-inat-data-pynaturalist
https://forum.inaturalist.org/t/downloading-a-csv-of-all-observations-of-a-species-with-python/30655/20

Posted on October 30, 2019 09:24 PM by ahospers ahospers

Comments

https://forum.inaturalist.org/t/looking-for-inaturalist-observation-map-visualisation-suggestions/7322/14
because maps can be so variable, it would take a lot more code than i’m willing to do to create a proper front-end to allow people to really configure their own maps.

that said, the Leaflet-integrated version of the map does allow you to specify parameters in the URL that will filter down the results (just like you can in the iNaturalist Explore and Identify pages), and i just added the ability to pass in custom scale factors, too (which will allow you to customize scaling ranges).

for example, this gives you Rudbeckia amplexicaulis observations scaled from 0-5 observations per cell at 0 zoom, down to just 1 observation per cell at the highest zoom levels:
https://jumear.github.io/stirfry/iNat_UTFgrid_based_density_map_for_Leaflet.html?taxon_id=200073&scale_factor=5,4,3,2,2,1

or this will apply scaling for 0-5 observations per cell across all zoom levels:
https://jumear.github.io/stirfry/iNat_UTFgrid_based_density_map_for_Leaflet.html?taxon_id=200073&scale_factor=5

if you need advice on how to tinker with either of my examples, let me know what you’re trying to achieve.

they’re both just HTML files. so you can begin tinkering by doing the following:

click on the Github code link of the example that you’re interested in
just above the code on that page, there’s a Raw button. click that.
select and copy all the text.
open a code or plain text editor. (i just use Notepad in Windows.)
paste the code into the code/text editor, and then save the file as an .html file.
now find the newly saved file, and open it in your favorite internet browser just to make sure it’s working.
if the browser opened up the file okay, you can go back to your code/text editor and start editing. when you’re ready to see changes. just save the changes, and then reload the page in your browser.
i’ve been thinking about a 3rd version of this that could take filter parameters in a URL and generate a static map that might scale automatically, automatically set the map extent, and provide the option to generate a proper heatmap, based on parameters entered, but i probably won’t get to that any time soon. https://forum.inaturalist.org/t/looking-for-inaturalist-observation-map-visualisation-suggestions/7322/14

Posted by ahospers over 4 years ago

https://forum.inaturalist.org/t/using-the-field-similar-observation-set-for-linking-observations-of-lepidoptera-when-raising-on/1018 E.g., “Similar Observation Set 5,” or “Linked Observation” work well for this. And yes, you’ll definitely want to make new observations for this.

Posted by ahospers over 4 years ago

https://www.inaturalist.org/journal/ahospers/26917-7a-analysing-inat-data-pynaturalist

While I haven’t done it with iNaturalist data, if you are trying to start doing data analysis on large datasets I suggest learning some python. Packages in python such as Numpy, Pandas, Scipy, Scikit-learn and Matplotlib go awfully far for data analysis.

I suspect it is possible to use pyinaturalist 3 and the API reference 1 for getting data out of iNaturalist as well, but it seems like you were already able to extract to excel already. https://pypi.org/project/pyinaturalist/ https://www.inaturalist.org/pages/api+reference Data analysis and visualization is really a field of its own. There are many different software applications, like R, python, etc, but you really need to have a working knowledge of these programming languages. If you’re only familiar with Excel, that might be the best place to start. There are not any programs specifically for easily working with iNat data.

https://forum.inaturalist.org/t/analysing-inat-data/6001/3

https://pypi.org/project/pyinaturalist/

https://www.inaturalist.org/pages/api+reference

?interpolate_coordinates=true

Find observations missing a location
https://www.inaturalist.org/observations?quality_grade=any&place_id=any&not_in_place=91708,97391&user_id=ahospers&verifiable=any works!

But there is still a huge need for classical projects: i.e. projects to which observations are manually added, or added by virtue of observation fields that are required. To my mind these are "tagged" projects. (in iNat logic one traditionally "added observations to projects", but for similar projects on iSpot we added tags and created projects to filter on the tags - the net effect is the same, a lookup table of project-tag and observation, that can be used as a filter - either alone or with other filters).
The addition of a project-tag filter to the new projects will make the "traditional" projects into "collection type" projects too, while preserving the Project Icons on the observations.
((the additional functionality of defining observation fields from within a project is an extremely useful feature.))

A nitpick: is the central filter a Location filter (using google places) or a Place filter (using user-created iNat polygons) or both? It would help if the terminology was standardized with the observations pages and its filters.

?interpolate_coordinates=true

photos/?advanced=on

Here's the link for creating a traditional project: https://www.inaturalist.org/projects/new_traditional

https://www.inaturalist.org/journal/ahospers/26917-7a-analysing-inat-data-pynaturalist

Posted by ahospers about 3 years ago

Add a Comment

Sign In or Sign Up to add comments