I've been noticing for a couple months that the 10k flag queue seems to have NAA and VLQ flags for things that I think wouldn't be flagged that way on other sites I'm active on. These are generally poor answers (usually pure opinion) deserving of downvotes, but my understanding of those flags is that there's a higher bar. Are we using these flags correctly?

My understanding from things that CMs have posted on MSO and per-site metas (no links handy right now, sorry) is that these mean:

  • NAA: does not even attempt to answer the question (a wrong or unsupported answer isn't NAA)

  • VLQ: Shog once evocatively described this kind of post as "a steaming cowpile". That is, VLQ means "unredeemable", not just "anecdote" or "wrong" or "opinion".

If those aren't what these flags mean on this site, could we have some better local definitions? I've been finding in recent weeks that I feel deterred from acting on that flag queue at all, while especially post-graduation I should be taking a more active role with it. Please help me understand how I should be viewing these flag types.

share
3  
Short answer: VLQ means the post can't possibly be edited into shape and should just be deleted. Not an answer means it doesn't attempt to answer the question. Next step should be to determine if NAA should apply to answers that don't meet back it up rule guidelines or if we should flag those as "needs citation". See chat.stackexchange.com/transcript/message/13870646#13870646 for context. I'll write something more substantial later on. :) –  jmort253 Feb 26 at 21:12
    
Can you point out an example to two of each? –  Joe Strazzere Feb 26 at 21:51
1  
Is it kosher to reveal what's been flagged how? If so I'm happy to do so; just not sure if that's allowed. –  Monica Cellio Feb 26 at 22:14
    
@JoeStrazzere workplace.stackexchange.com/tools/flagged –  enderland Feb 26 at 22:42
2  
No we aren't, but it is the only tool available to us (short of personal appeal to our mod-overlords) to identify and deal with poor quality posts, especially on hot questions where first-time visitors upvote everything. I think we need to try a new approach instead. –  jmac Feb 27 at 5:01
2  
@jmac yes, I understand why people are reaching for the only apparent tool. But I'm starting to think that (while waiting for the community-annotation feature we'd like), it would be better to use custom flags to request that mods annotate. As it is, as a 10k user all I can do when presented with one of these flags is to make that request myself -- but the original flagger could have, too. The flags just pile up in the queue and deter reviewers (or at least some reviewers); it's harder to act on the real NAAs and VLQs because of all the other flags. –  Monica Cellio Feb 27 at 15:01
    
@MonicaCellio - Our current mods prefer to take a sit and wait to see what the rest of the community thinks before we act tact. Which is fine, but the only way we can start to rally the community is using flags that the rest of the 10k users can see. We dont see the custom flags –  ReallyTiredOfThisGame Feb 27 at 16:45
    
compared to SO standards (I have 1K+ helpful flags there, for the record), I would say we're using NAA/VLQ flags very incorrectly, but in the absence of adequate System for Dealing with Poor Answers this "incorrect" usage feels surprisingly fair –  gnat Feb 27 at 18:59
    

2 Answers 2

Here is an example of a post where I would expect to see a very low quality flag. If I were to leave a comment here, I'm 99.9% certain this post will not be improved by the poster and there is no way anyone else could begin to improve it simply because there's nothing there:

Very Low Quality

As for not an answer, I've seen these flags on the following types of posts:

  • Questions as answers
  • Comments as answers
  • Answers that answer the question, but they're missing key elements from Good Subjective, Bad Subjective like explanations of why or how, objectivity, or citations/references.

If we had a "primarily opinion based" flag, I'd say we'd then only use not an answer flags for the first two cases and not the latter. But for now, not an answer is pretty standard for answers that don't meet guidelines of a Workplace answer.

share
1  
I think of answers that say "Yes / No that is (not) ok / normal / someothersubjectivejudgement" but have no other valuable context as VLQ. That is an answer to the question but it is not really salvageable with out writing a completely new answer, which is not salvaging it is writing a new answer. –  ReallyTiredOfThisGame Feb 27 at 16:41
    
I NEED to know.. What does that backwards R do? –  RyanS Mar 4 at 15:02
2  
@RyanS, it's just a shortcut to the post revision history, which is handy if no one has actually edited a post but there's history on the post, like closed, reopened, notice added, etc. I think it's a userscript I installed from Stack Apps. –  jmort253 Mar 4 at 15:07

Look ma, it scales

For what it's worth, below is what I learned by studying how Low Quality review queue works at Stack Overflow. Per my understanding of recent feature changes, NAA and VLQ flags push post into this queue unless it has been already reviewed there.

Side note it's not the first time that I am active in this queue but things feel really different compared to my past experience.

Oblivious of meta

Review in this queue is supposed to run without any meta guidance.

This is a matter of simple math: there are about 30,000 eligible reviewers (2Kers), while top viewed flags question at MSE has only about 18K views - including views from lower rep users, as well as views from users of other SE sites and from all the internet at all.

No diamonds

Review in this queue is supposed to run without diamond moderator involvement.

This is a matter of simple timing: in straightforward cases, deletion takes only a few minutes. If it was expected that moderators control such a fast paced activity, they would have to be on a 24x7 permanent watch, which is certainly not the case.

Mob rules, balanced

At first glance, it seems that just six-seven 2K users can in theory create a death ring and remove 20 any answers a day through flagging and consequent voting in LQ queue (20 is review limit when queue has less than 1000 items).

But system is indeed designed to prevent anything like this from happening.

To start with, if post author ever notices deletion they can simply undelete it solely at their discretion. And this is not all, system appears to be designed to either dissipate into disputed flag or alert moderators any time when things deviate from happy path ("happy path" here being unambiguous decision of reviewers and voters that post isn't wort keeping, followed by total negligence of its author).

For the sake of completeness, moderators have means to block repetitive abusive flagging (flag suspensions). But if you think of it, system is designed so that most bad flags are expected to be blocked before they even bother a moderator.

When there are 30,000 mostly reasonable reviewers in the queue, chances for a substantially mis-flagged post to pass through without being stopped by regular reviewers are pretty slim.

First round: light as the breeze

Overall, it looks like no risk and no pain are involved for all the parties in the initial phase of dealing with VLQ/NAA flags. Worst case for flagger is they get painless "disputed" flag. Worst case for author who noticed deletion of their answer is they click undelete.

Bad answers are easy and quick to delete, and when deletion mistakes happen, it's easy to revert for post author.

  • Interesting that per my observations, SO hot questions recently look like carrying less of trash answers than it was before. Not having 10k over there, I can only guess, but it's quite possible that garbage simply is quickly cleaned up via queue (and if the posters undelete, moderators get notified and can intervene).

How does it feel like for a (responsible) flagger?

In a word, great.

Best thing is, one doesn't need to study cumbersome rules of the game anymore. You just flag answers you feel better be deleted and regularly check amount of your disputed flags.

When you feel like it's higher that you would want it to be, you visit posts with your disputed flags and try to figure what could make your peers - regular 2K users - disagree with your flag.

The system feels simple, transparent and fair.

What does this mean for Workplace?

Hard to tell, because amount of posts, flags, reviewers and moderators differs so dramatically. Taking into account site specific quality requirements like Back It Up / Don't Repeat Others adds to the complication.

Above approach works at Stack Overflow but they simply don't have an alternative due to large scale.

At smaller sites, it is technically possible to have more mod control over review queues, for example like in Programmers close queue (where moderators tend to intervene when it looks like there's too much stuff in there) or even a really tight control like in Low Quality queues at Programmers and Workplace.

Related reading:

share

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .