-
Notifications
You must be signed in to change notification settings - Fork 24
Moderation
Experts in killer whale bioacoustics collaborate to ensure that predictions from the OrcaHello real-time inference system are validated as efficiently as possible. The goal it to provide notifications Southern Resident Killer Whales are present near a monitored hydrophone as rapidly as possible, but also with high reliability (i.e. a false positive rate near zero).
This is the recommended procedure (to maximize reliability and minimize latency):
- As soon as possible after receiving a notification as a moderator, navigate to the moderator UI and authenticate.
- Review the most recent candidate(s), or -- if many false positives occurred recently -- sort to review the highest-confidence recent candidate.
- If you find a true positive, check "Yes"
- Add labels (using the procedure and standards described below)
- Add comments (a brief synopsis or notes beyond what the labels provide)
- Hit submit (which will issue a notification email to subscribers)
- At a minimum for true positives, add pod(s) and any individuals known to be present (e.g. from visual IDs). Ideally, also label any call types that you recognize within the candidate.
- For false positives, or for "unknown" candidates (for which you cannot confidently assign a sound source), add labels as you see fit. Ideally, any label you add will be consistent with existing cloud of OrcaHello tags and any new labels that you create will aspire to be consistent with the annotation efforts of the broader Orcasound and bioacoustic communities.
If you are unsure if a label exists yet for a particular sound, you can browse existing OrcaHello tags and playback candidates of interest.
To efficiently improve the OrcaHello system, as a moderator you can create Github issues in the aifororcas-livesystem repo for bugs or other problems you experience. For each issue, please use a label to characterize it for the dev team, e.g. "bug" (something's not working) and/or "moderatorportal."
Simple rules when deciding whether to file an bug/issue:
- If there was ever an "event" with whales of any kind but you did not receive an email - please file an issue.
- Create separate issues for separate hydrophones. For example, if on 11/2/21 there were 2 independent events on Bush Point and Port Townsend and you did not receive an email for either, please file 2 different issues.
- Create separate issues for different events happening on the same day. For example, if there were 2 different events -- one at 9 AM and one at 9 PM -- on the same hydrophone on the same day, file 2 separate issues.
- When in doubt, file separate issues!
It is not useful to file new issues under existing issues unless we are sure that we have root-caused it correctly. This adds to confusion.
What is useful when filing an issue is a full description of the bug with facts and pictures, minus any conjecture.
We'll work on creating an issue template later, but for now consider including, at least:
- Which hydrophone?
- When did the KW event start?
- When did the KW event end?
- What was the nature of the event? Faint calls, killer whales/ humpback?
- Was the container running? (If you have this info?)
You can also create issues with ideas for feature requests or enhancements to the existing moderator portal UI. Please use the "enhancement" label for these issues.
If you're an expert at listening for orcas, we'd welcome your help as a moderator. We're particularly interested in moderators who live well beyond the Pacific time zone (so that you may respond when local moderators are asleep). At a minimum, you should be able to differentiate between Southern Resident and Bigg's killer whales, humpbacks, and other common Salish Sea sounds. Ideally, you should be able to infer which SRKW pod is the source of calls, and be familiar with 5-10 of the most common calls in the SRKW repertoire.