Can human error on ImageNet dataset be non-zero if the same person who created the dataset in the first place took the test?
Somehow I can't digest the reason given in the here. Which is binary classification is done when creating the dataset; However, multi-class classification on the test would give some error. It might be if a different person took the test, but what if the same person who created the test took the test? Can we see a few errors?
One thing that is bugging me is that if someone thinks that an object in the image is a dog with 51% in binary classification. Then that also means that he thinks it is everything else with 49%. Now in multi-class classification, the same image--I think so--would be classified as a dog with 51%, and the rest top-4 guesses combined would be 49%. Anyway, it would still be in the top-5; Hence, no error.
I want to know if the results of binary classification and top-1 of multi-class classification can be different.
Background knowledge- Introductory level in ML
Topic machine-learning
Category Data Science