Are our judgments of "too broad" too broad?

Question

I noticed a trend in a couple recent questions.

• Identifying plagiarized jokes? [closed] (Two jokes with different proper nouns or slightly different word order may actually be the same joke.)
• How to calculate/determine similarity between two English Sentences? [on hold] (Determine common meaning between two sentences.)

Both questions were closed as "too broad". I imagine this is because they touch on a big subject: Natural Language Processing. The thing is, they're not asking for a full explanation of NLP. Both questions describe a fairly discrete scenario (especially joke plagiarism). So why "too broad"?

My hunch is that at first glance it seems impossible to write a rich answer in the space SO provides. After all, it would require the equivalent of several academic papers. But is that true? When we think about expertise, it's easy to forget about the layers and layers of abstraction. Experts don't talk to each other in gory detail. They communicate using convenient abstractions. When I provide an answer about straight-up programming, I leave out many, many details. My answer uses abstract concepts that I assume you know or will look up in your own time. For example, I don't describe Dependency Injection from first principles, when I say "functional programming" I assume there's a shared history between us, I casually mention a Facade without explaining exactly what I mean, etc.

I imagine that NLP (and many other specialties) have their own abstractions. I can't say for sure, but I imagine an expert might be able to competently answer the questions above in a few paragraphs. The answer wouldn't be step-by-step instructions from first principles; it would be a bunch of abstraction that I don't (initially) understand. If I'm right, then we're losing interesting questions because reviewers are uncomfortable with topics in which they lack expertise. Is this something we can improve?

---

Hopefully, it's not too obnoxious to anticipate questions/concerns:

• The OP isn't going to understand the answer anyway. -- That may be true initially, but remember back to your huge leaps in knowledge. They always start with confusion and misunderstanding. Selfishly, I'd love to learn something new regardless of what happens to the OP. Others may enjoy these answers as well.

• Specialties don't belong on Programmers. -- I agree that we don't want to become a Q&A for academic papers. Still, it seems there are opportunities to explore interesting algorithms and practices. I may not be able to answer the specialty questions, but exposure to the topic makes me a better developer. If a specialty is big enough, maybe it will break off to its own site and we can direct people there. Until then, let's tackle the on-topic questions here.

• No one is going to answer these questions. -- This one is tricky. As of today, it seems that the majority of up-voted answers come from a small handful of users. These users may not be equipped to answer specialty questions. On the other hand, if we exclude specialty questions we're not likely to attract new and interesting contributors. It's a chicken and egg problem.

  The trend has been growing restrictions on the types of questions that can be asked. This is good for keeping out the riff-raff. But there's also a cost. As the pool of questions shrink, there are fewer opportunities to attract participants. Seems there could be a win-win compromise. Toss the homework questions because they don't add lasting value for anyone involved, but open up some of the specialty questions in an attempt to lure a few fascinating experts with fascinating answers.

• We'll receive too many answers from unqualified respondents. -- Sure. That's always the case. Too often, we kill off whole categories of questions because they attract bad answers, even if there's a possibility for an excellent one. I prefer an approach that allows for greatness while acknowledging you may have to sift through some garbage versus an approach that prevents all garbage and greatness as collateral damage.

Answer 1

I was involved in both of these questions... so here we go.

Both of these questions delve quite far into natural language programming. I had a college class that touched on this for a week (as part of a general survey of AI) and this is the type of thing that really is the subject of semester long classes, heavy books, and research papers.

This brings us to the first point you have:

The OP isn't going to understand the answer anyway.

Elsewhere I wrote on the importance of including what you know. Neither of the questions included shows that if we were to go to the NLP level of trying to answer the question - word taxonomies, part of speech identification, we'd find ourselves writing a tutorial for one of the libraries out there.

Writing one of these from scratch, you would need to show that you have at least a handle of the scope of the problem.

When the question with joke similarity said that edit distance wasn't a valid solution (though it did work for the material provided in the question) it leaves the door wide open for "how do you solve two jokes being similar?"

The similarity between two sentences question doesn't even show that the OP is familiar with the domain, and again, is either asking someone to write a tutorial for a python NLP library or is looking to write this from scratch.

Putting the entirety of Natural language processing: an introduction into an answer isn't helpful to people (and it doesn't fit) - this is something that can be found by searching.

The answers that we give will either be too high level to be useful to the OP, way too technical (until the OP clarifies the question to show that they are capable of handling that technical level), repeating tutorials that can be trivially found by searching, or don't even fit in the space allotted for an answer.

Consider that the following answer would 'answer' both questions:

You will need to:
* do sentence boundary detection which also identifies abbreviations and titles (so that "I watched Dr. Who last night." isn't two sentences).
* do tokenization of words and ideas (so that "5 miles/day" is one token as is "2,6-diaminohexanoic")
* Identify the part of speech of the word including homographs and gerunds
* decompose compound words, possibly doing lemmatization to reduce a word to its root by removing its suffix
* chunk the text into phrases

Once this is complete, you will need to identify if there are any grammatical or spelling errors. For jokes this is especially problematic as they may depend on puns and the error recovery will either mangle the sentence or render it unable to be parsed.

Following that, you will need to do named entity recognition which identifies the various people, places, locations, and other categories of nouns.

Unless you are familiar with linguistics and some computation, thats really not a useful answer for anyone.

---

The key decider for me when looking at something that is too broad it has the following two points:

• The high level answer isn't useful
• The low level answer isn't specified

Its clear what the general problem is, but the focus of the problem needs to be narrowed. Are you having trouble with the sentence boundary recognition? With the part of speech tagging? Chunking phrases? Identifying the name in the syntax tree?

These questions, given that the OP knows what they are asking about, are really easy to fix. The OP goes in and modifies the question to say "I am having a problem recognizing if this noun phrase is similar to that noun phrase." But the question of "explain NLP to me" which is at this time, the current written core question to both posts is something that falls into the "high level not useful" area.

Without the specification, these questions morph from one form to another as the OP gets a slightly better idea of what they are asking about and then refines the question again, and again.

And I hope I am never asked to try to figure out if one telling of The Aristocrats is the same as another.

---

I'm also going to point out that the Jokes question is a also literature request:

Can anyone point me in the direction of useful literature?

A google scholar search for plagiarism detection comes back with 26,000 results.

And while this is easily edited out, it again points to the unfocused and too broad nature of the question. I have no idea if any of those are what the OP wants, or are at the level that the OP is able to understand. Providing that material for me to be able to properly answer the question is something that the OP needs to do in each case - I can't guess about that.