Clarification

• by George_Sims

  I discovered this site this afternoon, and have been diligently staring at worm butts for well over an hour. I'd estimate that I observe egg laying in about 5% of the worms viewed. It's gotten to the point where I can almost predict whether or not a worm will lay an egg within the 30 seconds. In ALL cases, I've been able to see the distinctive swelling, and usually can actually see the eggs inside the worm's body. I have NOT counted those eggs unless they actually exit the body within the time period. Initially, I was confused by dark streaks on the background, made sometimes when the worm "turned sharply". I don't count those.

  In MANY, if not most, of the cases where someone else is also observing the same worm, the other person counts at least one egg, when I see none visible. Either they, or I, am doing something wrong. Which is it? I must actually see the egg exit the body, and it must be clearly discernible as a dark round spot on the background, whether or not I can distinguish any eggs inside the body.

  Posted March 8, 2014 11:03 PM

• by pbc

  I agree with you. There are very very few worms actually laying and everyone else 'in competition with me' thinks that they are laying eggs when they are not, as far as I can see. Hence, is there any value in this 'citizen science' project when most of the results are inaccurate?

  Posted March 10, 2014 9:54 AM

• by aexbrown scientist

  First or all, thanks for your help! You're right that most of the videos will not have egg-laying in them and you should only count the new eggs that pop out of the side of the worm. We're still in the early days of the project and so we don't yet have an idea of how accurate people are over all. I've been going through videos myself to provide a baseline that will help us figure that out.

  There is definitely still value in your work. All citizen science projects have the problem of inaccurate classifications. That's why we need to have multiple people looking at each video to be confident in the result. It depends on the exact pattern of the inaccurate classifications of course, but we should be able to filter them out.

  Posted March 10, 2014 2:55 PM

• by pbc

  Thanks for your response. I think I can now understand that when we get the message 'no other timings yet!' (please remove some of the exclamation marks) it means that other people have seen these worms and not seen them laying eggs, and Anonymous is anyone who hasn't logged in. It would have helped me to understand what was going on if this had been expalained somewhere at the start.

  Posted March 13, 2014 8:35 PM

• by Der_Das

  I feel that giving this a touch of 'competition' (score/opponent) results in many false positives.

  Initially I thought: what am I missing here when the other user is seeing eggs? I went over and over some videos to spot what I could have missed and got rather frustrated. I almost gave up on this project, had I not come up with my conclusion above.

  Posted July 2, 2014 4:42 PM

• by aexbrown scientist

  Thanks for your diligence and for sticking with it. People should only be getting points either for being first to see a video or for eggs that are validated by other users so that there's no incentive for labelling non-existent eggs.

  Posted July 2, 2014 9:49 PM

• by massivedonkey

  It would be a very good idea, I think, to provide some examples of "stuff that you might mistake for eggs but which are not actually eggs" in the tutorial. It seems that a lot of people are mistaking video artefacts, and bulges apparently picked up by the image recognition software when a worm strays into a path already carved into the medium, as actual egg-laying incidents.

  The tutorial could be a little more comprehensive too - we're not all C. elegans experts!

  Posted July 11, 2014 2:57 PM

• by aexbrown scientist

  I totally agree. I'm hoping it will be possible to update the tutorial or possibly include a short video with someone talking through a couple of examples.

  Posted July 15, 2014 11:22 AM

• by massivedonkey

  There's also no indication of how accurate you're being, so it's not really possible to learn and improve as you go, and no incentive against mashing Z whenever anything remotely eggy shows up. By the nature of the experiment, it's understandable that ascertaining the accuracy of the participants might be tricky, but I can think of a few ways of giving participants an idea of how well they're doing. The tutorial could include a couple of "test yourself" clips that have been checked by actual researchers, in which you're afterwards marked 'correct'/'incorrect' and therefore get a concrete idea of how good you are at IDing. It might even be a good idea to show such expert-vetted clips once in a while as you're going through the clips to make sure you're not getting rusty

  Posted July 21, 2014 5:45 PM

• by massivedonkey

  ... or indeed forgotten what egg-laying looks like! (it's not hugely frequent)

  Posted July 21, 2014 5:48 PM

• by aexbrown scientist

  All good ideas. We already do the expert vetting on a selection of clips and that will be helpful in determining accuracy afterwards, but it's true that we're not giving feedback on it in real time.

  Posted July 25, 2014 4:04 PM

• by shocko61 in response to aexbrown's comment.

  Hi aexbrown , dah I should have come here first . By reading this I have worked out the old and new eggs problem if the eggs are seen to actually come out of the worm then they are new eggs . Cheers Ian

  Posted November 18, 2014 7:04 AM

• by DZM admin in response to shocko61's comment.

  That is correct. 😃

  Happy worm-watching!

  Posted November 18, 2014 2:36 PM

• by shocko61

  thank you DZM

  Posted November 21, 2014 12:14 AM

• by PsyHam

  I came to this thread because I too noticed a confusing trend of players marking eggs where I didn't see any. At first I started wondering if I wasn't doing my job right. But once I saw my first video of actual egg laying it became a lot simpler to spot, since I realized how obvious an egg laying actually is.

  Perhaps as part of the tutorial you could include an actual video and not just an animation, to show that the eggs really are quite noticeable on screen, and easy to spot when they are laid? That might help cut down on the number of false positives from users who perhaps are just not sure what they should be looking for. And as others have said, putting even more emphasis on the fact that you're not supposed to mark eggs that already exist at the start of the video.

  Other than that, this project is quite fun! I'm enjoying watching the little worms slithering around 😄

  Posted March 12, 2015 9:18 PM

• by Kelvets

  Every user should have to watch at least one video of an actual egg-laying taking place. Other Zooniverse projects have a test with several samples of known result, to determine if that user got the idea and is accurate. You should implement that here as well.
  Also, the warning that you're not supposed to mark eggs that are already there should be displayed MULTIPLE times.

  Posted March 12, 2015 11:21 PM

• by p1r0kun

  Unless I'm mistaken, PsyHam, it seems that clicking the "Help" button toward the bottom of the screen (next to the light-colored "Guide" button) takes you to a video of an exemplary egg-laying event. I was thinking similarly to what you posted, and the video helped me a lot. Hope this is helpful!

  I also agree with Kelvets about the multiple warnings - marking pre-existing eggs seems to happen frequently.

  I've come upon a video that seems to show no eggs being laid, but another user has marked about 740 events! I wonder if there's a way to screen a user for abuse? Hope that kind of thing isn't common; it may have been a stuck key or something. I wish I could remember the video specifics, but I could probably come up with an approximation of the username if requested.

  Posted March 13, 2015 3:20 AM

• by squishycakes

  I'm amazed and frustrated at the number of videos I watch where there are either no eggs laid but multiples marked by the other player, or existing eggs (but none further laid) that are marked by them.

  Posted March 13, 2015 5:46 PM

• by ElisabethB

  It is a nuisance, but don't worry, these clips will be classified by a lot of other people and these malicious or uninformed classifications will not harm the end classification.

  Posted March 13, 2015 6:12 PM

• by PsyHam

  Ah, that's perfect, p1r0kun! Exactly what I was looking for during the first tutorial. Maybe if the tutorial was updated to automatically run the "Help" activity before starting classification, that would be great 😄

  Posted March 13, 2015 9:45 PM