21. iNaturalist workflow Flickr Remove GeoTags Upload

How this fits into my iNaturalist workflow

My photo workflow is as follows.

Import photos to Darktable (http://www.darktable.org/)

Geotag DSLR photos without locations using GPX files (I track where I go using the Cyclemeter app on my iPhone)

Add lots of tags to my photos (dragged onto photos from my big tag library of taxonomy, locations, and other things I've imported from various sources over the years).

This includes the place name, entered as a hierarchical tag in Darktable starting with "Places|", e.g., "Places|New Zealand|South Island|Canterbury|Banks Peninsula|Akaroa harbour|Hinewai Reserve". iNat just needs "Hinewai Reserve".
It also contains heirarchical tags for the notable species in the photo, tagged starting with "Species|" and including EOL machine tags for the higher taxonomy, e.g., "Species|taxonomy:kingdom=Animalia|taxonomy:phylum=Arthropoda|taxonomy:class=Insecta|taxonomy:order=Lepidoptera|taxonomy:family=Erebidae|taxonomy:genus=Nyctemera|taxonomy:binomial=Nyctemera annulata|taxonomy:common=Magpie moth|Magpie moth|Nyctemera annulata". iNat just needs "Nyctemera annulata".
Any tags I want shared as tags in iNaturalist get prefaced with "iNaturalist tag|" (e.g. "iNaturalist tag|Garden").

Any tags I want to become observation field values in iNaturalist (http://inaturalist.org/observation_fields) start with "iNaturalist field|", e.g., "iNaturalist field|Insect life stage=adult" (an observation field needs to already exist on iNat for that to work).
None of my other tags, like the people in the photo, or the event name, or my tags tracking where I've shared a photo, match those Place, Species, or iNaturalist formats.
Export the photos I want to send to iNaturalist to one folder on my computer

Run my bash script 'iNatTagCleaner.sh' in the Terminal app on my Mac. It automatically strips out all the tags except those starting with "iNaturalist field|", "iNaturalist tag|", "Species|", and "Places|" from all the photos in the folder, using the awesome power of exiftool (http://www.sno.phy.queensu.ca/~phil/exiftool/).

All those photos are then dragged onto the iNaturalist Add Observation page (https://www.inaturalist.org/observations/upload), which automatically assigns each observation a species name from my tags, gets any observation field values from my tags, and it adds all my tags as observation tags. The only manual thing to do, if necessary, is to combine multiple photos into single observations. Submit, delete the exported images from my iNaturalist folder, and I'm done.

Useful for other things?

My bash script could be modified for other tag-stripping purposes, as long as the tags you want to keep are consistently labelled.

I'm no programmer

Note that I'm a newbie to bash scripting so I'm sure my code could be written more efficiently. It works though. :-)

Requirements and Disclaimer:

You'll need to have exiftool installed on your computer for the bash script to work (it's free, http://www.sno.phy.queensu.ca/~phil/exiftool/).

The current build assumes that there are no spaces in photo file names

I built this for my own use.
It is not an official product of iNaturalist.
Use at your own risk.

===================0000000000000000000
Like Charlie said we strip EXIF to remove potentially sensitive
location data. To do so, we use the imagemagick -strip command, which
sadly just removes ALL metadata. If you or anyone else knows of a way
to selectively strip tags using imagemagick, please let us know. We
could probably do this with exiftool, but I'm loath to add yet another
dependency to iNat if we can achieve this with the tools we have

exiftool \
-Subject="Dermestes lardarius" \
-Subject="Hanmer" \
-Subject="Indoors" \
-Subject="Insect life stage=adult" \
-Subject="Mt Isobel Place" \
-o ~/Downloads/output.jpg \
~/Downloads/initial.jpg

Posted on January 30, 2019 06:39 PM by ahospers ahospers

Comments

But you're welcome to write your own script that posts through our API
https://www.inaturalist.org/pages/developers

Here's a snippet of posting users and obs over the API written in Ruby

require 'rubygems'
require 'rest_client'
require 'json'

First, enter your app_id, app_secret, and redirect_uri from

http://gorilla.inaturalist.org/oauth/applications/206

site = "http://gorilla.inaturalist.org"
app_id = '308714d38eaf78ed57c11c0790f639d7d05e86cb7564f641629116e5b3bea024'
app_secret = '8be0ee61e1b7858a14c050b982eb3e6447b2d674075f3d8c58c8759ed2ee02a6'
redirect_uri = 'http://www.bd.dix/utils/migratelanding.cfm'

Next, visit this link on your browser while logged in as 'tegenligger'

http://gorilla.inaturalist.org/oauth/authorize?client_id=308714d38eaf78ed57c11c0790f639d7d05e86cb7564f641629116e5b3bea024&redirect_uri=http%3A%2F%2Fwww.bd.dix%2Futils%2Fmigratelanding.cfm&response_type=code

and get your auth code

auth_code = "d9c5335b17c0ec05f7444b1673c675b16a9a4d77f2d499b852778888503760a3"

Next, get a token for tegenligger

payload = {
:client_id => app_id,
:client_secret => app_secret,
:code => auth_code,
:redirect_uri => redirect_uri,
:grant_type => "authorization_code"
}
response = RestClient.post("#{site}/oauth/token", payload)
token = JSON.parse(response)["access_token"]
headers = {"Authorization" => "Bearer #{token}"}

Now make a user using tegenligger's token

username = 'testuser1'
email = 'testuser1@bar.net'
password = 'testuser1password'

results = RestClient.post("#{site}/users.json", {"user[login]" =>
username, "user[email]" => email, "user[password]" => password,
"user[password_confirmation]" => password}, headers)
puts "created http://gorilla.inaturalist.org/users/#{JSON.parse(results)["id"]}"

Now get a token for testuser1

payload = {
:client_id => app_id,
:client_secret => app_secret,
:grant_type => "password",
:username => username,
:password => password
}
puts "POST #{site}/oauth/token, payload: #{payload.inspect}"
response_for_user1 = RestClient.post("#{site}/oauth/token", payload)
token_for_user1 = JSON.parse(response)["access_token"]
headers_for_user1 = {"Authorization" => "Bearer #{token}"}

Now make a observation on behalf of testuser1

results = RestClient.post("#{site}/observations.json",{
"observation[species_guess]" => "Northern Cardinal",
"observation[taxon_id]" => 9083,
"observation[observed_on_string]" => "2013-01-03",
"observation[time_zone]" => "Eastern Time (US %26 Canada)",
"observation[description]" => "what a cardinal",
"observation[tag_list]" => "foo,bar",
"observation[place_guess]" => "clinton, ct",
"observation[latitude]" => 41.27872259999999,
"observation[longitude]" => -72.5276073,
"observation[map_scale]" => 11,
"observation[location_is_exact]" => false,
"observation[positional_accuracy]" => 7798,
"observation[geoprivacy]" => "obscured"
}, headers_for_user1)

puts "created http://gorilla.inaturalist.org/observations/#{JSON.parse(results)[0]["id"]}"

Now make a another user using tegenligger's token

username = 'testuser2'
email = 'testuser2@bar.net'
password = 'testuser2password'

results = RestClient.post("#{site}/users.json", {"user[login]" =>
username, "user[email]" => email, "user[password]" => password,
"user[password_confirmation]" => password}, headers)
puts "created http://gorilla.inaturalist.org/users/#{JSON.parse(results)["id"]}"

Now get a token for testuser2

payload = {
:client_id => app_id,
:client_secret => app_secret,
:grant_type => "password",
:username => username,
:password => password
}
puts "POST #{site}/oauth/token, payload: #{payload.inspect}"
response_for_user2 = RestClient.post("#{site}/oauth/token", payload)
token_for_user2 = JSON.parse(response)["access_token"]
headers_for_user2 = {"Authorization" => "Bearer #{token}"}

Now make a observation on behalf of testuser2

results = RestClient.post("#{site}/observations.json",{
"observation[species_guess]" => "Northern Cardinal",
"observation[taxon_id]" => 9083,
"observation[observed_on_string]" => "2013-01-03",
"observation[time_zone]" => "Eastern Time (US %26 Canada)",
"observation[description]" => "what a cardinal",
"observation[tag_list]" => "foo,bar",
"observation[place_guess]" => "clinton, ct",
"observation[latitude]" => 41.27872259999999,
"observation[longitude]" => -72.5276073,
"observation[map_scale]" => 11,
"observation[location_is_exact]" => false,
"observation[positional_accuracy]" => 7798,
"observation[geoprivacy]" => "obscured"
}, headers_for_user2)

puts "created http://gorilla.inaturalist.org/observations/#{JSON.parse(results)[0]["id"]}"

Posted by ahospers almost 5 years ago

e I’m not querying the API directly is the rate limit any higher? Here is the script I’m currently using to download 1 photo/sec for 10k/day:

import os
import pickle
import requests
import shutil
import sys
import time

def download(url, name):
print(name)
with requests.get(url, stream=True) as r:
if r.status_code != 200:
print('{} error: {}'.format(r.status_code, url), file=sys.stderr)
return
r.raw.decode_content = True
with open('./ducks/{}.jpg'.format(name), 'wb') as f:
shutil.copyfileobj(r.raw, f)

def main():
os.makedirs('ducks', exist_ok=True)
present = set([x.replace('.jpg', '') for x in os.listdir('ducks')])

with open('duck_urls', 'rb') as f:
urls = pickle.load(f)

ctr = 0
for url in urls:
if ctr > 10000:
return
name = os.path.normpath(url).split(os.path.sep)[-2]
if name in present:
continue
download(url, name)
ctr += 1
time.sleep(1)

if name == 'main':
main()

Posted by ahospers over 3 years ago

Add a Comment

Sign In or Sign Up to add comments