Home → Bug Reports →

language characters in 6.10.1

• Edit

Eric L. Epps

Aug 31, 2011 @ 05:59 PM

Since upgrading to 6.10.1, characters such as ó (o-acute) and ñ (n-tilde) are being mangled in index blocks. These displayed correctly prior to the 6.10.1 upgrade.

Only characters in metadata fields are affected, and the encoding error appears in the index XML, as the attached screen shots show.

• 2011-08-31_1254.png 57.2 KB
• 2011-08-31_1253.png 53.2 KB
• 2011-08-31_1254_001.png 22.1 KB

1. 1 Posted by Joel on Aug 31, 2011 @ 06:02 PM

   

   Hi Eric,

   I'm thinking your database's collation could be off, could you verify this? Every table needs to be set to UTF-8 encoding, and as well the engine type to InnoDB. Please let me know what you find.

   Thanks!

   • Edit

2. 2 Posted by Eric L. Epps on Aug 31, 2011 @ 06:59 PM

   

   All tables are UTF-8 and InnoDB.

   • Edit

3. 3 Posted by Joel on Aug 31, 2011 @ 09:34 PM

   

   Eric,

   Could you attach your cascade.sh/cascade.bat (the former is *nix, the latter is if your environment is Windows) as well as your context.xml file? The cascade.sh/.bat file can be found within the root directory of Cascade (/CascadeServer/) and the context file can be found within /CascadeServer/tomcat/conf/.

   Thanks!

   • Edit

4. 4 Posted by Eric L. Epps on Aug 31, 2011 @ 09:49 PM

   

   Attached. Also, if it helps, we're running as a Windows service

   • cascade.bat 818 Bytes
   • context.xml 2.24 KB
   • Edit

5. 5 Posted by Joel on Sep 01, 2011 @ 02:04 PM

   

   Eric,

   I believe your instance of Cascade Server is fine, but perhaps your browser's encoding setting is set to Western instead of UTF-8. Please look into this and let me know what you find.

   Thanks!

   • Edit

6. 6 Posted by Eric L. Epps on Sep 01, 2011 @ 03:32 PM

   

   OK, I found that was indeed the case, but that leaves out a couple factors:

   1. This only changed upon upgrading to 6.10.1 from 6.8. My browsers have always been set to Western, I haven't changed that setting (until today).
   2. These characters, when entered in a WYSIWYG field, display correctly. When entered in metadata fields, the characters don't display correctly.

   I changed my browser character encoding in Chrome and Firefox, and the pages displayed the same. I even edited a page, submitted and published the change with the same result.

   Also, one piece of information I left out to start with is that certain other special characters (like special left- and right-quotes) did the same thing. I don't have a screen cap or anything of that, though, because I just replaced them with straight quotes.

   It seems that something that does a find-and-replace for special characters on metadata fields for index blocks changed in 6.10.

   • Edit

7. 7 Posted by Eric L. Epps on Sep 01, 2011 @ 04:44 PM

   

   Oh, sorry, found out one more piece of information. I have a text File asset with the ñ character that is displayed as a ? on the web page. When I edit the file, it is always converted to an unknown character symbol after submitting (doesn't matter whether "Advanced Editor" features are on or off).

   So I guess it's not just metadata fields.

   • Edit

8. 8 Posted by Joel on Sep 01, 2011 @ 06:56 PM

   

   Eric,

   Your browser should likely always be set to use Unicode UTF-8 encoding. As for reproducing this in a text file, what happens when you use the following XML entity instead of the plain "ñ" character?

   ñ
   

   Please let me know.

   Thanks!

   • Edit

9. 9 Posted by Eric L. Epps on Sep 01, 2011 @ 07:10 PM

   

   ñ displays correctly in the text file, it is rewritten as ˜ in metadata fields.

   • Edit

10. 10 Posted by Joel on Sep 01, 2011 @ 07:14 PM

    

    Eric,

    Metadata fields are converted to their XML entities upon submission, so you won't have to worry about using XML entities there. You will however have to use XML entities within file assets as we do not do any conversion upon submission.

    Thanks!

    • Edit

11. 11 Posted by Eric L. Epps on Sep 01, 2011 @ 07:26 PM

    

    Right, I expected it to be rewritten in the metadata field. I just tried it "just in case."

    I'm fine with using ñ in the text files, but it's still a change from 6.8. Those same files displayed correctly under 6.8; after upgrading to 6.10, they stopped displaying correctly. (both the text file and the metadata fields)

    It can't be a browser issue, because the characters are mangled in the index block--before it's even published. Has to be either Cascade or database.

    • Edit

12. 12 Posted by Joel on Sep 01, 2011 @ 08:25 PM

    

    Eric,

    Could you paste a screen shot of your Windows Service for Cascade Server? Specifically the startup parameters for it? Also, please attach your service.bat file found within /CascadeServer/tomcat/bin.

    Thanks!

    • Edit

13. 13 Posted by Eric L. Epps on Sep 01, 2011 @ 10:02 PM

    

    Attached.

    • 2011-09-01_1701.png 13.2 KB
    • startup.bat 2 KB
    • Edit

14. 14 Posted by Joel on Sep 02, 2011 @ 02:05 PM

    

    Eric,

    It seems we may have found the underlying issue here. In past versions of Cascade Server there was a parameter (-Dfile.encoding=UTF-8) set when the Service was created, however in 6.10.x this has been left out. To resolve this, please follow the steps below.

    • Shutdown Cascade Server
    • Navigate to the /CascadeServer/tomcat/bin directory
    • Locate tomcat6w.exe and make a copy named cascadew.exe if there is not already such a file
    • Open cascadew.exe and click on the Java tab
    • Find the Java Options box and insert the following line at the bottom: -Dfile.encoding=UTF-8
    • Enter your memory settings below the Java Options box for Minimum and Maximum values
    • Hit Apply
    • Start your Cascade Service back up

    This should resolve the collation issues you're experiencing, but let us know if you continue to experience issues. Our apologies for the inconvenience!

    Thanks!

    • Edit

15. 15 Posted by Eric L. Epps on Sep 02, 2011 @ 02:13 PM

    

    That did it, thanks!

    • Edit

16. Eric L. Epps closed this discussion on Sep 02, 2011 @ 02:13 PM.