Request: Stop using polls, stop using personal experience as arguments for proposals

[Xananax]

I write this in answer to #5561. It's just an example, but I could write this same request following innumerable proposals over the years.

"I personally don't use it" or "I asked N people" shouldn't be a valid reason for axing a feature or implementing one.

1. The amount of people you polled is not a statistically significant number.
2. Were it a statistically significant number, it still wouldn't be statistically significant, because there is no proper sampling. There could be 4 billion people who use the feature, but you wouldn't know if you ask around you and it happens that none of those people are around. This is compounded by the fact that you often ask your followers/friends, which all but precludes any faint attempt at sampling.
3. The number of people using the feature is anyway not, on its own, a valid reason to axe a feature. If out of 10 people, 2 use a feature, but not using the feature has no negative impact on the other 8, then removing the feature is a net negative.
4. Polling people who have no stake in an outcome produces wildly wrong results, as any statistics primer course will teach. Asking people how much they would pay for a product yield 90% of people saying $5, but then only 2% of those do pay $5 when the product is out. Asking 1000 people who don't use a feature if it is useful is exponentially worse, it's likely to yield a result that is strictly worse than useless.

Such polls are, at best, tests of compliance of followers, but mostly only bias-reinforcing exercises. They have negative value in regard to proposals.

Once you polled people, the 500 people who answered, who don't use the feature, have no stake, are unconcerned, did an in-passing vote, still expect some action to result from their vote, specially if the outcome looked significant.

These proposals are therefore four times absurd. It is demonstrated by the conversation in the linked issue, which is summed up by:

• no one uses the feature
• I think people use the feature
• no, I think no one uses the feature
• I think people use it

I wish proposals stopped being opened on a completely biased basis.

Sometimes I agree with the premise, and sometimes I don't, but it is in every case intensely frustrating to see.

So such polls aren't just useless, they are actively harmful, to a degree, no other action of the community can be.

You cannot possibly act upon a worse incentive. These reasons are strictly the absolute worse.

This is made worse by arguments such as "the feature isn't complete/perfect". Something not covering 100% of a specific use case, or even of each use case, is not an argument against it. No feature can cover all use cases, and covering some is often better than none.

Unless that argument is qualified with further explanations as to why an incomplete feature is less desirable than nothing, then it is a null argument and only serves to muddy the waters further.

For example, if a feature's incompleteness might mislead users, then there may be mitigations other than axing the feature (eg, a popup warning users). Saying the feature is incomplete is posing a problem in Y instead of X terms. It isn't by itself consequential or worth mentioning.

I implore all collaborators to apply the four behaviors below:

1. never resort to polling again without proper training

Never. Under no circumstances. It's a pure, net negative energy. Do not ask people for votes or opinions. Do not play popularity contests.

Do say "I don't think..." or "I think...". This has the exact same argumentative value but it is honest.

"I asked 100 people" is dishonest and muddies the waters.

2. If you insist on polling, try to do it right

If you do feel like your casual question to your friends/followers leads to an insight worth following, then follow the scientific process and attempt as much as possible to disqualify your hypothesis first, by finding the people outside your circle. Do a best effort to guess where the people who might have a different usage than yours are, and ask them.

If you don't know how to find the people outside your circle, then you do not have a valid hypothesis. Do not open a proposal.

3. Don't use the poll in your proposal

You didn't apply 1 or 2, you polled your circle, you didn't try to disqualify your hypothesis, you didn't ask professionals their opinions, and you still insist on opening a proposal.

Fine.

Then at least do not invoke "me and my friends" as an argument! If you aren't able to support your proposal without resorting to a popularity contest, then you don't have a valid proposal.

4. never invoke "incomplete feature" as an argument

Saying the feature is incomplete is not an argument. It's not a "bad argument", it simply isn't one. It's an emotional appeal that has no basis in any logic.

"a knife isn't a screwdriver", sure, but if I don't have a screwdriver, I can use a knife somewhat for 40% of screws. It's not ideal, but it's better than nothing.

If you want to make that argument, you need to qualify it with why being incomplete matters. Is it because people wouldn't expect that? Then that can fixed by warning them. Is it because the missing parts aren't replaceable with an external tool? Then that can be fixed by making the feature compatible with an external tool. And so on.

Thank you for reading, and I hope we can have more productive conversations in issues.

[Xananax]

I'm going to add an addendum, following the conversation in the linked thread:

Negative votes should be counted much less than positive ones. If out of 20 people who use stairs, one is too weak and requires the elevator, then that person's vote should supersede all negative votes.

(I'm not trying to make an analogy for professionalism or knowledge despite the image).

I'd also say:

Lack of opposition should be seen as a big problem.

If in a community of hundred of thousands of users, you don't get a very strong opposition to the removal of any feature, that definitely implies you're not asking the correct people, and the actual users are not aware of your imminent removal.

Any axing should be strongly opposed. As long as it isn't, you didn't ask the right people.

[golddotasksquestions]

Negative votes should be counted much less than positive ones.

This might be a frame of reference thing, but I still feel like I have to point out how in a literal sense, negative votes (thumb downs) on proposals should definitely NOT be counted less. If anything they need to be counted more than positive (thumb up, hurray).

It's difficult to oppose. It's easy to say yes and cheer. This is true even more so in a community like Godot which is so dependent on the good will and motivation of it's community members to work for free.

People will not downvote a respected Contributor or community member unless there is some really severe issue. It's much easier to give an upvote as you respect and appreciate their work. Contributors also typically have more knowledge about the inner workings. In case of doubt, or if I don't know much about the topic at hand, I will trust the contributors/maintainer/core dev judgement. Openly declaring opposition to these people is difficult, and as a result people do it less often then upvoting.