8

Today we stumbled across an old question:

Should UTF-16 be considered harmful?

I'm going to ask what is probably quite a controversial question: "Should one of the most popular encodings, UTF-16, be considered harmful?"

The long and short of the question is that UTF-8 is unambiguous and compact, while UTF-32 is unambiguous and fast. UTF-16 is ambiguous (at times) and wins neither on speed nor on size. Since it is non-optimal in every case and there are known defects in operating systems and popular language implementations, why use it?

The answers and all eleventy bajillion comments are excessive, at times inaccurate or opinion-based, but nonetheless do contain a lot of useful information. They make compelling arguments for and against using UTF-16 in various situations. As someone who has been programming for many years and using Unicode for most of them, I learned a few things myself.

We discussed it in chat for a bit, with the following options:

  • Delete the whole question.

  • Clean up the comments and answers, then lock it.

  • Just lock it.

Currently, the question is locked. Should we do anything further with this question? Delete it? Unlock, clean it up, lock it again?

Keep in mind this question does have over 70k views and four questions link to it.

  • Definitely needs answer cleanup, no matter what else happens. Well, I guess not if it gets deleted. – durron597 Aug 13 '15 at 17:05
  • @durron597 sounds like you should write up an answer "clean it up then lock it again" – user22815 Aug 13 '15 at 17:08
  • 3
    i.stack.imgur.com/xVKoS.jpg – durron597 Aug 13 '15 at 17:13
  • 2
    Personally I found that information useful in the past, and I greatly dislike deleting useful information. – Rachel Aug 13 '15 at 18:33
  • 1
    @Rachel that is why I asked here. I found it useful now as I am working on a program that I want to be Unicode-aware and work in the minefield that is "C++ on Windows". I did learn a few things, and I already knew a lot about Unicode. – user22815 Aug 13 '15 at 18:35
12

I think the contents of the question, answers, and comments has some problems. If this were asked today it would be aggressively curated, if not closed as "primarily opinion-based."

However, this is a historical question asked near the beginning of the site when the focus and on-topicness was very different. The key question is this: does the value provided outweigh the cruft that collected over the years?

I believe it does, and I am in favor of keeping it locked. Reading through it I found a lot of information and specific details about various implementations and how they deal with UTF-whatever. There are many things that can go wrong when dealing with Unicode, and this question covered quite a few of them.

This particular question is trickier than many other historical questions. Specifically, this one has a ton of comments and the comments add substantially to the knowledge in each answer. Simply deleting all of the comments would lose quite a bit of useful information while also clearing out a lot of useless junk. Given the large amount of work required to curate this question after the fact with little benefit, it is not worth it to spend the time. Just keep it locked.

  • I checked in logged out mode, to see how it looks for outside visitors. Looks sufficiently harmless to let it stay that way. Most troublesome stuff was luckily deleted prior to lock and default sorting (by votes) helps to deemphasize lower quality content – gnat Aug 13 '15 at 21:20

You must log in to answer this question.