Bad news for idn developers

[Miguel]

Bad news for idn developers

I've decided to do little idn development on my own: Chinese & Russian. Just simple perl/php/etc. - search & data manipulation. That kind of stuff. Usually my partner deals with this stuff.
I'm your average or below avg developer. And boy, what I found isn't pretty. Turns out that none of the technologies on the market can easily deal with the non-ASCII character set and non-ANSI stuff. I checked Baidu & Yandex: both of them are very carefully separating the languages. Their internal searches are cranked using ascii only.
I'm sure there are ways around the tech issues involved. But not everybody is like Jose.

[phio]

I think that's why we see so many blogger generation type sites originated from the native countries. I'm not a developer per say, but even in simple ways there is much to be desired. Non-english speaking natives have probably had to deal with this stuff for the last 15 years and have found work-arounds -- they are the people to find to help with development because obviously they have already been there.

[Drewbert]

Yep. amazon is utf-8 for japan but the rest are latin1 BUT internally some of their stuff is utf-8 and you end up with them doing double decoding errors and all sorts of shit.

These people need to take the plunge and convert their entire systems to unicode internally NOW, toavoid lots of pain later.

[mdw]

The problem is Baidu and Yandex dragging their feet, year after year, not making the effort to bring their bread and butter code up to speed.

They should have been written right in the first place, or at least should have been fixed long ago. Makes you wonder.... There is some amazing work going on in the world of searching, but none of it is taking place in any of the companies you've likely to have ever heard of. These guys are anything BUT innovators.

[Rubber Duck]

I'm pretty sure the Chinese have their own development packages that function in Chinese.