What can I do to prevent
slugify filter from stripping out non-ASCII alphanumeric characters? (I'm using Django 1.0.2)
cnprog.com has Chinese characters in question URLs, so I looked in their code. They are not using
slugify in templates, instead they're calling this method in
Question model to get permalinks
def get_absolute_url(self): return '%s%s' % (reverse('question', args=[self.id]), self.title)
Are they slugifying the URLs or not?
There is a python package called unidecode that I've adopted for the askbot Q&A forum, it works well for the latin-based alphabets and even looks reasonable for greek:
>>> import unidecode >>> from unidecode import unidecode >>> unidecode(u'διακριτικός') 'diakritikos'
It does something weird with asian languages:
>>> unidecode(u'影師嗎') 'Ying Shi Ma ' >>>
Does this make sense?
In askbot we compute slugs like so:
from unidecode import unidecode from django.template import defaultfilters slug = defaultfilters.slugify(unidecode(input_text))
The Mozilla website team has been working on an implementation : https://github.com/mozilla/unicode-slugify sample code at http://davedash.com/2011/03/24/how-we-slug-at-mozilla/