Tuesday, May 04 2004

Today I stumbled across an issue with filename encodings on Mac OS X. We have a server that has been around since the good 'ol days of rhapsody, and since many files on this server differ only in the case of their names we've stayed on UFS. With the introduction of Case Sensitive HFS+ in Mac OS X version 10.3 I thought the time was right to migrate the old UFS filesystem.

There's a problem though. It seems UFS is quite forgiving in respect to the encoding used in filenames. A lot of files on our UFS disk has been FTP:ed from Windows machines and have swedish characters in them, encoded in Windows Latin. These files cannot be copied to an HFS+ partition as HFS+ seems to require correct UTF-8 encoding (or, as discussed on this page, HFS+ actually uses an Apple-modified version of Unicode's Normalisation Form D).

I'm not sure how to handle this. We could convert all filenames on the UFS disk to UTF-8 (which HFS+ seems to handle) before moving them to an HFS+ partition. I don't think this solves the problem where someone uploads a file with international characters in its name from a Windows machine using FTP, or when a cgi script like UseModWiki tries to create a page with international characters in its name. Perhaps there are nice workarounds to these problems, but for now I feel it's safer to stay with UFS.