3

I noticed a trend in a couple recent questions.

Both questions were closed as "too broad". I imagine this is because they touch on a big subject: Natural Language Processing. The thing is, they're not asking for a full explanation of NLP. Both questions describe a fairly discrete scenario (especially joke plagiarism). So why "too broad"?

My hunch is that at first glance it seems impossible to write a rich answer in the space SO provides. After all, it would require the equivalent of several academic papers. But is that true? When we think about expertise, it's easy to forget about the layers and layers of abstraction. Experts don't talk to each other in gory detail. They communicate using convenient abstractions. When I provide an answer about straight-up programming, I leave out many, many details. My answer uses abstract concepts that I assume you know or will look up in your own time. For example, I don't describe Dependency Injection from first principles, when I say "functional programming" I assume there's a shared history between us, I casually mention a Facade without explaining exactly what I mean, etc.

I imagine that NLP (and many other specialties) have their own abstractions. I can't say for sure, but I imagine an expert might be able to competently answer the questions above in a few paragraphs. The answer wouldn't be step-by-step instructions from first principles; it would be a bunch of abstraction that I don't (initially) understand. If I'm right, then we're losing interesting questions because reviewers are uncomfortable with topics in which they lack expertise. Is this something we can improve?


Hopefully, it's not too obnoxious to anticipate questions/concerns:

  • The OP isn't going to understand the answer anyway. -- That may be true initially, but remember back to your huge leaps in knowledge. They always start with confusion and misunderstanding. Selfishly, I'd love to learn something new regardless of what happens to the OP. Others may enjoy these answers as well.
  • Specialties don't belong on Programmers. -- I agree that we don't want to become a Q&A for academic papers. Still, it seems there are opportunities to explore interesting algorithms and practices. I may not be able to answer the specialty questions, but exposure to the topic makes me a better developer. If a specialty is big enough, maybe it will break off to its own site and we can direct people there. Until then, let's tackle the on-topic questions here.
  • No one is going to answer these questions. -- This one is tricky. As of today, it seems that the majority of up-voted answers come from a small handful of users. These users may not be equipped to answer specialty questions. On the other hand, if we exclude specialty questions we're not likely to attract new and interesting contributors. It's a chicken and egg problem.

    The trend has been growing restrictions on the types of questions that can be asked. This is good for keeping out the riff-raff. But there's also a cost. As the pool of questions shrink, there are fewer opportunities to attract participants. Seems there could be a win-win compromise. Toss the homework questions because they don't add lasting value for anyone involved, but open up some of the specialty questions in an attempt to lure a few fascinating experts with fascinating answers.

  • We'll receive too many answers from unqualified respondents. -- Sure. That's always the case. Too often, we kill off whole categories of questions because they attract bad answers, even if there's a possibility for an excellent one. I prefer an approach that allows for greatness while acknowledging you may have to sift through some garbage versus an approach that prevents all garbage and greatness as collateral damage.
  • 3
    Personally, when I use "too broad", I usually mean either "it'd take a several-hundred-page book to answer this" or "there are so many possible solutions we'd have to play twenty questions with OP to figure out which one applies". I agree that your four bolded reasons are not valid justifications for a "too broad" vote. However, the two questions you linked are imo no narrower than "How do I tell if two sentences have the same meaning?", which for me is definitely "several-hundred-page book" territory. – Ixrec Dec 16 '15 at 21:06
  • @lxrec Alright, so you're asking me to defend the weaker of the two questions. ;) Off the top of my head as a non-NLP expert, I imagine you could identify parts-of-speech, recognize lunch(noun) == lunch(noun), apple(noun) == apple(noun), and had(verb) closely related to consumed(verb). Whereas the second pair doesn't share any of those things. It seems very answerable, especially from someone who knows what they're talking about. – Corbin March Dec 16 '15 at 21:32
  • @gnat Don't know that we should close questions based on our guess of what the OP will understand or not. Robert's "what's a mouse?" is funny, but I remember being the noob. I learned a lot from people giving me more information than I could handle. Who's to say what they will understand? Also, the answers could be fun for the rest of us. (We're all noobs at something.) – Corbin March Dec 16 '15 at 21:36
  • what's a mouse is not really funny, see How to Answer. "Read the question carefully. What, specifically, is the question asking for? Make sure your answer provides that..." Answers telling asker "you need a mouse... a keyboard... text editor... computer..." would be just as legitimate as your attempt at giving conceptual explanation - that's what makes it too broad. Probably the way out is to edit the question and remove that part that poisons it – gnat Dec 16 '15 at 21:49
5

I was involved in both of these questions... so here we go.

Both of these questions delve quite far into natural language programming. I had a college class that touched on this for a week (as part of a general survey of AI) and this is the type of thing that really is the subject of semester long classes, heavy books, and research papers.

This brings us to the first point you have:

The OP isn't going to understand the answer anyway.

Elsewhere I wrote on the importance of including what you know. Neither of the questions included shows that if we were to go to the NLP level of trying to answer the question - word taxonomies, part of speech identification, we'd find ourselves writing a tutorial for one of the libraries out there.

Writing one of these from scratch, you would need to show that you have at least a handle of the scope of the problem.

When the question with joke similarity said that edit distance wasn't a valid solution (though it did work for the material provided in the question) it leaves the door wide open for "how do you solve two jokes being similar?"

The similarity between two sentences question doesn't even show that the OP is familiar with the domain, and again, is either asking someone to write a tutorial for a python NLP library or is looking to write this from scratch.

Putting the entirety of Natural language processing: an introduction into an answer isn't helpful to people (and it doesn't fit) - this is something that can be found by searching.

The answers that we give will either be too high level to be useful to the OP, way too technical (until the OP clarifies the question to show that they are capable of handling that technical level), repeating tutorials that can be trivially found by searching, or don't even fit in the space allotted for an answer.

Consider that the following answer would 'answer' both questions:

You will need to:
* do sentence boundary detection which also identifies abbreviations and titles (so that "I watched Dr. Who last night." isn't two sentences).
* do tokenization of words and ideas (so that "5 miles/day" is one token as is "2,6-diaminohexanoic")
* Identify the part of speech of the word including homographs and gerunds
* decompose compound words, possibly doing lemmatization to reduce a word to its root by removing its suffix
* chunk the text into phrases

Once this is complete, you will need to identify if there are any grammatical or spelling errors. For jokes this is especially problematic as they may depend on puns and the error recovery will either mangle the sentence or render it unable to be parsed.

Following that, you will need to do named entity recognition which identifies the various people, places, locations, and other categories of nouns.

Unless you are familiar with linguistics and some computation, thats really not a useful answer for anyone.


The key decider for me when looking at something that is too broad it has the following two points:

  • The high level answer isn't useful
  • The low level answer isn't specified

Its clear what the general problem is, but the focus of the problem needs to be narrowed. Are you having trouble with the sentence boundary recognition? With the part of speech tagging? Chunking phrases? Identifying the name in the syntax tree?

These questions, given that the OP knows what they are asking about, are really easy to fix. The OP goes in and modifies the question to say "I am having a problem recognizing if this noun phrase is similar to that noun phrase." But the question of "explain NLP to me" which is at this time, the current written core question to both posts is something that falls into the "high level not useful" area.

Without the specification, these questions morph from one form to another as the OP gets a slightly better idea of what they are asking about and then refines the question again, and again.

And I hope I am never asked to try to figure out if one telling of The Aristocrats is the same as another.


I'm also going to point out that the Jokes question is a also literature request:

Can anyone point me in the direction of useful literature?

A google scholar search for plagiarism detection comes back with 26,000 results.

And while this is easily edited out, it again points to the unfocused and too broad nature of the question. I have no idea if any of those are what the OP wants, or are at the level that the OP is able to understand. Providing that material for me to be able to properly answer the question is something that the OP needs to do in each case - I can't guess about that.

  • 1
    Sorry you were on both examples. I didn't mean to focus only on NLP; the questions were just conveniently recent. Obviously, I don't agree that we should access the OP's skill level. I imagine that's error prone and it's condescending. I believe in simply answering the question asked. If the answer is over the OP's head, they have a little research to do. That certainly won't hurt them. – Corbin March Dec 17 '15 at 15:36
  • 2
    @CorbinMarch No need to apologize, its more a "I know what guided my decisions and that I didn't have to guess about others." The problem I've had in the past with answering the question over the OP's head is that the question starts morphing quickly as it gets "but what does that mean, I've modified my question" comments until it becomes a rather high level glossed over answer. This runs the risk of invalidating other answers in the process and causing other confusion. There is also the bit I wrote about this in the past on MSE: meta.stackexchange.com/a/234663 – user40980 Dec 17 '15 at 15:42
  • With the question about jokes, this should have come into play as the OP had already discounted the edit distance but hadn't explained that in the question originally posted. This returns to the key point - The full answer that gets into the technical details is too broad (a semester long class) and probably misses the actual point of the question if forced to fit within the constraints of the question (and then morphs to another question). The high level answer glosses over everything and gets a 'yes, I know that, but the problem I am having is..' and morphs to a new question. – user40980 Dec 17 '15 at 16:09
  • Both of these attempts at answering and morphing the question can properly be served within the current workflow of how close and reopen proceeds if the question is closed as too broad, the too broad nature is explained in comments, and the OP then refocuses the question to the actual problem being had rather than looking for a high level summary of the material which they can get by reading Wikipedia or searching google. – user40980 Dec 17 '15 at 16:10
  • One other point that I'd like to make (as I continue this comment monolog) is that if the answer above is valid in both questions as an answer, and if posted in one the other question wouldn't be a duplicate of the first, then there is something else wrong with the question and answer combo posted - the question asked is likely too broad and the answer unhelpful for being too high level. And there's the related "avoid spurious dups to too broad questions" discussion also going on. – user40980 Dec 17 '15 at 16:29

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .