ID'ing observations for other... Paper on species ID's by experts and non-experts...

So, I was glancing at a few papers today, and I spotted this one:

Species identification by experts and non-experts: comparing images from field guides. G. E. Austen, M. Bindemann, R. A. Griffiths & D. L. Roberts -- Scientific Reports 6, Article number: 33634 (2016)

Abstract: Accurate species identification is fundamental when recording ecological data. However, the ability to correctly identify organisms visually is rarely questioned. We investigated how experts and non-experts compared in the identification of bumblebees, a group of insects of considerable conservation concern. Experts and non-experts were asked whether two concurrent bumblebee images depicted the same or two different species. Overall accuracy was below 60% and comparable for experts and non-experts. However, experts were more consistent in their answers when the same images were repeated, and more cautious in committing to a definitive answer. Our findings demonstrate the difficulty of correctly identifying bumblebees using images from field guides. Such error rates need to be accounted for when interpreting species data, whether or not they have been collected by experts. We suggest that investigation of how experts and non-experts make observations should be incorporated into study design, and could be used to improve training in species identification.

Anyways, I was thinking about the ID system here on iNat. I really enjoy ID'ing observations for folks -- I try to focus on the stuff from TX, but every now and then I try to tackle some observations from outside of TX.

I've been wrong hundreds/thousands of times, probably. My ego's not too too massive -- it's ok that I've been wrong! I do learn a lot from mistakes and from being corrected. And as any natural history collection (digital or physical) goes, each mistake doesn't diminish the integrity of the database -- the mistakes are eventually caught... "Eventually" can mean a long time though. That's the nature of collections! :)

The more folks that ID, the stronger the database gets too. Even if it's verifying an observation that's already been verified/research grade, that's a valuable annotation, in my mind. It's a safe-guard against some of the new folks to iNat that give a more general ID and change the taxon as well.

Tagging some of my favorite ID'er to see if you'd like to look at that paper too.

@greglasley @carrieseltzer @gcwarbler @d_kluza @mako252 @maractwin @aguilita @kueda @kevinhintsa @john8 @loarie @lisa_bennett @nlblock @susanhewitt @cosmiccat @charlie @silversea_starsong @glmory @robberfly @borisb @nathantaylor7583 @muir

Posted by sambiology sambiology, February 03, 2017 03:45

Comments

Thumb

That's a rough subject -- Bombus are very difficult to identify, even at the best of times. On the other hand, good for raising awareness that we all need to improve. Being a naturalist is a constantly evolving process...

Posted by silversea_starsong over 3 years ago (Flag)
Thumb

Good piece. It furthers my belief that when I try to on board someone to iNaturalist that I talk of not only getting your stuff ID'd and contributing to the global database, but that its important to be involved in identifying stuff you know and slowly grow into stuff you don't know as it helps to hone one's skills when you can't be in the field and allows you to pick up key traits on organisms that you might not see on a regular basis. I was not a good contributor to IDs when I first started, but the past year I've been trying to be better about it and it definitely has helped me become more aware of what to look for in the field. It also makes the platform feel more like a community circle rather than one way streets when your involved on both sides of the equation.

Posted by damontighe over 3 years ago (Flag)
Thumb

interesting - thanks for sharing

Posted by loarie over 3 years ago (Flag)
Thumb

Very interesting, and I agree with you on many points! I find my (frequent) errors help me learn too, and I often go back and try and see how I made the error. I'm not an 'expert' of many taxa, but I find the more exposure you get to photos/ real life organisms the more skilled you get at ID'ing them. Some groups will always be hard/impossible to ID from photographs, but I still think there is value in the observations even if they can't be confirmed at species level.
Also, I'm quite chuffed to be included in your favourite IDers :)

Posted by lisa_bennett over 3 years ago (Flag)
Thumb

Excellent work. I am a retired pathologist. We have done many studies on to what degree pathologists offer the same diagnosis from an identical set of slides. The ID is improved with more participants.

P.S. Always get a second opinion on any very important medical diagnosis.

Posted by denniswross over 3 years ago (Flag)
Thumb

Awesome! with plants: some you can tell with any photo from a half mile away (if you know the habitat, etc), some you can tell only with good photos, and some you can't tell with photos at all. One of the hard parts is knowing that. Also figuring out what to do when you hvae two cryptic species and 99.9% of the tie it is the same one. What margin of error do you accept? On the one hand there is possible false data but on the other hand most of it is good solid stuff.

Overall, other than a few student projects that teachers don't adequately curate, the research grade observations for plants on iNat seems to have an accuracy similar to that of a field crew of trained or semi-trained botanists making a photoless plant list. Mistakes happen but overall it's really good. And the photos let you fix old mistakes which you can't do on a plant list. I've seen many old plots and thought really? no way that plant was there... but you never know without a photo.

Posted by charlie over 3 years ago (Flag)
Thumb

Interesting but not too surprising. I'm going to be plunging into my bumblebee backlog soon for iNat and a couple other projects. Now that I've taken a look at the identification guide I've gotten from the library I'm certain many if not most will remain at the genus level.

Posted by driftlessroots over 3 years ago (Flag)
Thumb

Just in general, not about bumblebees in particular, I think one thing that a lot of people don't realize is the following.

When an observation has no ID, or only an extremely general ID, if you can ID the organism to family level with confidence, please go ahead and put the family in as an ID!

I think a lot of people (especially new people) feel they should only ID to the species level or not at all. But when you put a family level ID into an observation which previously only had phylum or class, the local person may be able to look that family up in a local felid guide and then work out which species they saw, or maybe at least which genus it was.

If you know a group well, you may be able to ID to family level for observations in parts of the world that you have never experienced yourself.

Posted by susanhewitt over 3 years ago (Flag)
Thumb

That should be "field guide" not "felid guide", unless it is cats you are trying to ID!

Posted by susanhewitt over 3 years ago (Flag)
Thumb

Great article but as @charlie and others have mentioned above, there is great variability in the ID potential of many different groups of plants, animals, etc. charlie's comment about plants you can ID from half a mile in a photo, or others that cannot be identified from a photo at all is also true for the groups I have a little knowledge of, birds, dragonflies and butterflies. As far as bees....I can tell a bee from a hawk but I'd probably not push my ID skills of bees much beyond that. I often find it amusing/actually a little frustrating, that folks who are really very new to any sort of nature observations will make an incorrect ID of something and post it on iNat and a number of folks who have a broad level of knowledge and expertise on the subject will come along and correct it, yet the initial observer sometimes simply sticks by their original ID (even though it is totally wrong) and refuses to take advantage of the information they have been offered by others. But, I guess that is the way a community forum such as this has to work. We all make mistakes, and I do it all the time, and as @silversea_starsong commented, being a naturalist is a constantly evolving (and learning) process. As was stated in the article you cite, the longer I have been at this, the more I have learned to be cautious with many IDs. (as @sambiology knows, I am reluctant to ID most winter meadowlarks, most Empidonax, and others). Some folks open their first field guide and suddenly are experts. Oh well. But I guess I was that way when I started as well.
There are lots of bird folks on iNat and a fair number of dragonfly and butterfly folks, so much of what I can contribute to IDs is really redundant. I will occasionally find a bird record with two or three "agrees" on an initial ID that is, in fact, incorrect and I can offer reasons why, and usually folks appreciate that. But if not, that is fine too. I really wish iNat could attract more knowledgeable folks on moths (the few that are on iNat get overwhelmed with the volume of moth records and they just can't keep up). Other groups, also are under-represented with knowledgeable people, but hopefully as iNat continues to grow, we will continue to attract more people with expertise and knowledge in many different areas. But I have to say that iNat has offered me a great learning experience, more than I ever would have anticipated when I first came on board 4 years ago. I gain knowledge all the time about different species of mammals (and other groups), distribution records, seasonal variations, etc., that I was unaware of before, so even to an old fart who has been at this game a long time, I learn from iNat every single day.

Posted by greglasley over 3 years ago (Flag)
Thumb

I've seen this paper before and agree with it completely. I remember how many Euphorbia specimens Barton Warnock misidentified and that is example enough for me. Marshall Johnson corrected some of these but also had a few extra misidentifications in there place. Billy Turner corrected most of what was left, but left plenty of misidentifications for me to find and correct. I really hope I haven't misIDed too many specimens, but I find it inevitable. In addition to this, I remember seeing a paper studying the phylogeny of the Hawaiian members of sect. Anisophyllum. The researcher was unable to get specimens (because the plants are listed) and so she relied on the "field botanists" to obtain DNA. Many individuals within the same species ended up on opposite ends of the phylogeny. Also, without the vouchers to reference back to, the validity of the paper cannot be confirmed. However, I can't help but feel that many of these were misidentified. This problem is important to take into account and why it is important to voucher. If there is a specimen to refer back to, at least your work can be corrected. If there isn't, it's very difficult to determine where the mistakes were made and if there are any others that will be made.

Posted by nathantaylor over 3 years ago (Flag)
Thumb

@susanhewitt I completely agree. Most observations that ID to family from the more general plants are identified to genus or species the following day! It really does speed up the identification process. However, I have noticed that the way iNaturalist "perceives" a broader ID is as a conflicting ID, so I usually take mine down if I don't know the species after two people have IDed the observation to species. If I don't, the community ID may stay at family level for a considerable amount of time and not get to research grade as quickly as it should.
I am curious to know what other people's opinions are on this, though.

Posted by nathantaylor over 3 years ago (Flag)
Thumb

I'll add a little to the comments by @nathantaylor7583 from my own experiences. I spent two years helping to reorganize the odonate collection at the Univ. of Texas at Austin when Dr. John Abbott was the curator there. (He is now at the Univ. of Alabama). Anyway, I was adding bar graph cards to each specimen envelope and thus I handled every specimen as I was doing this. I'm not sure of the total number of odonates in the collection, but somewhere near 20,000 is a fair guess. Over two years I found at least 150 misidentified specimens and with John's approval, corrected the IDs. Not a big percentage of errors, but it showed me that mistakes can show up anywhere.

Posted by greglasley over 3 years ago (Flag)
Thumb

@greglasley i think a lot of times when people don't fix an incorrect ID it is because they aren't on the site any more. A lot of time it is a case where students were assigned to use iNat under 'duress' but one the assignment is over they leave. They don't curate their data and often the teacher doesn't either. And sometimes it's marked as rejecting community ID so a bad observation that is obviously wrong is stuck there forever. A little off topic but... something to consider moving forward

Posted by charlie over 3 years ago (Flag)
Thumb

Thanks @charlie, and very true.

Posted by greglasley over 3 years ago (Flag)
Thumb

Thanks all for the comments and stuff to chew on.

This actually came out of an internal conversation I had with myself -- I want to defend the importance of field guides (physical or digital) and the importance of knowing what to call something... Or even to know that something has a name!

@gcwarbler and @maractwin talked about their field guide book shelves. I too have a nice little collection of field guides that I really enjoy. Hell, when I went to my wife's house for the first time when we were still dating, I noticed all of the field guides and nature books on her shelves and said, "yep, this is the one." :) I love field guides and think they're important. They helped me "develop a passion for natural history" like this page mentions: https://arthropodecology.com/2012/04/30/field-guides-and-natural-history/

But are they (field guides) important? When I tag a name onto something, is it more important than it was before? Did it matter more?

iNat engages that same sort of passion I have for finding different critters and plants and for ID'ing what others find. The more that I flip through the pages of my field guide and through the pages on iNat of observations, the more I learn -- it's wonderful to hear that's what others do too.

I worked in a herbarium of around 1 million plant specimens at BRIT -- errors were abundant. Errors among the credentialed were abundant as well. It's ok! We find errors and temporarily correct them -- only to be recognized as something else in the future, perhaps. Photos and angles of photos can be deceptive, and it's ok to add a temporary ID (in my mind).

This organism exists, it's sharing the planet with us in time and space, and we document its existence. Now, what kind of bumblebee is it?!? ;) @greglasley , what species of female Argia or which meadowlark is it? @charlie, what name should we give this cryptic species in this species complex? @susanhewitt , what kind of slug is that again? @nathantaylor7583 , look at that super blurry pic of that spurge and see if you can name it. @damontighe , could you look at this mushroom that lacks all of the appropriate pics for its ID? Even if we can't put the exact right name on every single observation, each observation shows that organism that existed -- I think that's pretty important. :)

Appreciate the conversation here. Thanks!

Posted by sambiology over 3 years ago (Flag)
Thumb

Wow, very nice Sam. Thanks for the inspiration. :)

Posted by susanhewitt over 3 years ago (Flag)
Thumb

I'm pretty cautious about ID'ing my observations. A lot of mine still haven't been properly ID'ed yet because I haven't gathered enough data (like seeds) or it's like a similar species.

Posted by james52 over 3 years ago (Flag)
Thumb

What an interesting paper, Sam! Thank you for sharing. I agree with @susanhewitt that it's important to orient people to the function of adding those levels in between kingdom or phylum and species. I have also found that it's important to explain the basics of the tree of life and how that relates to naming and community IDs. If you haven't had biology since high school, that may not be a way you're thinking about naming organisms. I (unsuccessfully) tried to get people to add better initial IDs (especially before the smarter suggestions for taxon names) by saying something along the lines of, "...so even if you don't know exactly what it is, enter what you know! What do you know about this organism?" and I'd get something like "blue flower" (doh!). So I haven't nailed that explanation yet. There are some helpful images on the getting started page that I'd recommend if you're presenting to newbies (https://www.inaturalist.org/pages/getting+started#organisms).

As a kid I remember being frustrated with my Audubon insect guide when I couldn't find exactly whatever I was looking at. Of course, I didn't appreciate just how impossibly big such a volume would be! I wish that I had learned earlier in life to be ok with a family or genus level ID. I think if I had embraced the uncertainty sooner I might have actually been empowered to dig deeper, because at least I'd have a better starting place.

Posted by carrieseltzer over 3 years ago (Flag)
Thumb

You are absolutely right Carrie. I also wish that someone had explained to me when I was young that being able to ID things to the family level or genus level is, in some ways, much MORE useful than knowing exactly which species something is.

But generally people start out in life learning the common names of a bunch of local organisms, and they may have no idea that those creatures fit into families of related creatures, which all kind-of look similar. And they might be astonished to learn that most families are worldwide in distribution.

And most field guides are not that good as a teaching tool because they don't introduce families to people to all.

The notable exception in my field is Tucker Abbott's "Seashells of North America, a Golden Guide", which puts a lot of helpful emphasis on families, and that is why I love that little book so much, and recommend it so often.

I personally believe that taxonomy is best learned hands-on, and learned from the bottom up, rather than from the top down.

Posted by susanhewitt over 3 years ago (Flag)
Thumb

Some thoughts:
Specialists can "subscribe" to higher level taxa, so there is merit to placing a higher level taxa in order to bring it within the subscription scope of those that specialise in that area. I think this taxa subscription doesn't happen much, but I also think it will happen more in the future, as the volume of obsevations increases and the "experts/specialist" need to reduce their workload.
It has already been mentioned, but these identifications are a dynamic thing. In the past an ID was likely given and it would "stick" from that moment on, a part of totals and statistics immutably. I know that when I find myself in a discussion/debate about an ID, I go rolling back through past observations, looking at different angles of view, comparing colours, building a greater understanding of the characters involved, and those past observations are sometimes changed or dragged into the debate. Their position within the statistics can change respectively!
Taxonomy is a peg and hole thing. not all pegs are round or square, Each level of taxanomic identity defines further characters that group or differentiate organisms. We can talk about all the spiders, or specifically just the Latrodectus katipo, but we can also talk about specific individuals, like the first one I have ever seen which I have named Katy, she has 2 eggsacs now... something that differentiates her from Bobby, who has none and just bobs up and down in her snare when I peek inside! Names are just a way of communicating about groups and individuals, so that we (the participants in that communication) understand what is being referred to.
It's a level of focus thing. Kind of like when you get a camera for the first time. You point, you shoot, and you get a somewhat decent photo, bit blurry, not framed the best, lighting a bit off. The more you use it, and the more you talk with other photographers, you learn what the knobs do, you pick up the tricks, you gain insight to the aesthetics that make a great photo. It's the same with ID'ing. You have to make the blurry IDs in order to discover the wonderful resources out there that help refine the art!
I think of all the wrong IDs that I have made, and I think of the (very few) incorrect IDs that I have corrected. At least 90% of those resulted in dialogue and a sharing of knowledge about the characters that are relevant. I think back to what I knew when I started in iNaturalist and where I am at now, and I project that out into the future to where I am likely to be in 10 years time, and I am excited! I urge everyone to push their boundaries, if you feel the need you can qualify your guesses with "I think" in the comments field of the identification. An archer shoots their first arrow at the target based on what they know at that time, and the RESULT of that attempt tells them what the wind effect is, and they compensate in the next flight. So it's the same for us. If we miss, why did we miss...? what can we learn from it that will help us next time?

Posted by kiwifergus over 3 years ago (Flag)
Thumb

Here's another thought:
If I need to fill a hole with dirt, I measure the sides of the hole and can calculate the volume of dirt needed. It the tape measure is 100mm short (fibreglass tapes can stretch!), then I will end up short in what I order. But if I am using the same inaccurate tape measure to determine the length of a dwang/noggin and then use that same tape measure to measure and cut the dwang, it will fit perfectly! My point is, it is less important that the identifications be 100% accurate, and more important that those USING the data understand its limitations :)
Another example: I was monitoring citrus for pests, the results of which were used to determine spray applications. One of the beneficials we recorded were Tasmanian lacewings. I soon discovered that one of the critters that were being counted as lacewings by the other monitors was in fact a winged stage of psocid. If we change at that point and stop identifying psocids as lacewings, then the ability to compare data from year to year and assess the optimum spray times is compromised. Or at least further complicated. If we changed how we identified/counted and those using the data were unaware of the change, then it is even worse! The point here is accuracy of identification is not always what breaks or makes the value of the data!

Posted by kiwifergus over 3 years ago (Flag)
Thumb

@kiwifergus As support for one of your arguments, I subscribe to Euphorbiaceae and Euphorbia.

Posted by nathantaylor over 3 years ago (Flag)
Thumb

@sambiology Your point regarding whether something having a name makes it more important is an interesting one. I think when we put a name on something, then we have something on which to hang the information about it that we discover. We can create reports that describe what damage it does, or profiles of how it can be sustainably managed, or guides on how to bring it back to full health. Once you know it's name, you can find a huge amount of information relating to it, some useful and some not. For instance, Latrodectus katipo has a medically significant bite, where as Steatoda capensis (which is easily confused for the other) does not. I can easily find information about L. katipo that tells me the black spider I have found in my compost bin will not be L. katipo because it is completely the wrong kind of habitat for it. This is something that I am able to know without ever having seen a katipo, made available to me via the name that has been used to hang all that information on. The individual organism isn't made any more important by tagging a name to it, but what it does do is facilitate our ability to share information about it!

Posted by kiwifergus over 3 years ago (Flag)
Thumb

I'm a map nerd myself, and the names show you which map to stick the pins on. That's why the taxonomic database is so crucial, so we can group these things into boxes so we can find them to put on maps.

Posted by charlie over 3 years ago (Flag)
Thumb

This may be heresy, but here goes: In general iNat data can not really be used for distribution or density analysis of a species because observations are not randomly taken, but rather dependant upon where humans choose to observe, and biased towards unusual sighting because they are more interesting. So in a lot of ways it doesn't affect the quality of the data if you can't reliably distinguish between two similar species from a photo, if they are both known to be in the area*. The problem is when something is mis-identified (to research grade) as something that isn't otherwise known to be in the area - so extra caution is needed, extra evidence is needed, for out-of-range observations. (*Of course if you have collected the photographed specimen, or it is something like a tree that has a relatively specific location the exact ID of that individual is more crucial).

Posted by tony_wills over 3 years ago (Flag)
Thumb

well, i wouldn't say it can't be used to tell a story about a diestribution of a species (including one that influences management or ecology)... but it is true that there is no absence data and there are lots of types of analysis you can't do. But, I think it's a case of 'the enemy of the perfect is the good' and it's free data :)

I'm not using iNat to publish papers about species distribution but to increase our knowledge about species range, absolutely. but yeah, for big out of range observations, very good evidence is needed, and ideally eventually someone goes and gets a voucher for the important ones!

Basically the world is so big and focused surveys haven't been conducted most areas... so there is still so much to discover!

Agreed that when picking apart species that both occur in an area, or especially subspecies, it's not providing that much value to obsess over. Though I don't want wrong IDs either

Posted by charlie over 3 years ago (Flag)
Thumb

I had noticed early on that kereru seem to be more prevalent around peoples houses, parks and along motorways! Joking aside though, it is part and parcel of my point made earlier, that the important thing is that those that use the data are aware of it's qualities. Even as biased data, it is still useful... even if it is just used as planning data for designing more comprehensive/accurate distribution studies. To carry on my katipo example, we know that up north there is black katipo and down south there is red katipo. I live somewhat in the middle of the cross-over (as far as current distribution knowledge would have us believe). My observations of occasional katipo here and there might show no red katipo at all, and any parties interested in the distribution of katipo forms might notice that, and plan a (more scientific) distribution study to ascertain what is happening regarding the respective ranges of the two. But the greatest benefit I see is simply that my observations of katipo on the dunes of the beaches in Gisborne, will raise awareness of the fragile ecosystem and could stimulate interest amongst the local community in learning about and maintaining that ecosystem. Indeed, until I saw my first katipo there, I was not aware they were there at all...! I did not know that there was such a thing as giraffe weevils, until I saw an observation by someone else... seeing what others see is making me look closer at what is around me.
@charlie What constitutes a "wrong ID"? taxonomy itself is an evolving dynamic thing, with new discoveries moving things around and re-classifying and so forth. That is the truly great thing about iNaturalist: IDs are dynamic, and they can be "corrected" and improved upon and reviewed at any time.

Posted by kiwifergus over 3 years ago (Flag)
Thumb

right, well i just mean an ID that doesn't match the current taxonomy one way or another.

Posted by charlie over 3 years ago (Flag)

Add a Comment

Sign In or Sign Up to add comments

Is this inappropriate, spam, or offensive? Add a Flag