Is it time to consider what changes must be planned for the coming inclusion of utf-8 strings in domain names?

Reference what is coming:
http://www.cnet.com/news/from-coffee-to-email-to-xyz-the-webs-crazy-new-reality-is-here/?tag=nl.e404&s_cid=e404&ttag=e404&ftag=CAD1acfa04

These exist now:
http://newgtlds.icann.org/en/program-status/delegated-strings

Much that pertains to a user's profile is contained in the file "prefs.js". According to my Notepad++ editor, that file is in ANSI.

Also the "Manual domain completion" in preferences stores data there.

user_pref("kmeleon.plugins.macros.domComplete1.suffix", ".сайт");
user_pref("kmeleon.plugins.macros.domComplete2.suffix", ".paris");

Paris works but the Russian сайт for "site" does not.

Does any change here come from K-Meleon or is it from Mozilla? It may not be possible to make changes until something comes from Mozilla.

That's dangerous as hell, evil people can fake domain names with similar looking characters from other languages :-( At least there's this punycode thing, that makes it more recognizable.

Anyway, my KM1.6 prefs.js is in "UTF-8 without BOM" acc. to Notepad++
Have tons of sessions stored inside, strongly suspect it's because some page titles have special characters.
Perhaps test if converting your prefs.js to utf-8 and writing manual entries in it will stick? Wonder if there's only a prob with adding settings through xul or something, no idea.
Hmm, or another little test, change this post title to contain сайт, save the session and check prefs.js again?



Edited 1 time(s). Last edit at 10/13/2014 08:52PM by siria.

AFAIK comes from Mozilla. And is security sensitive (see siria) since crooks can fake URLs for their money-making with letters that look similar or identical to the human eye. But they are not.

And no: The future is not ANSI. Many (if not the majority) are only catered by UTF8 or higher. Most nations already use the metric system. ANSI reads as American National Standards Institute (ANSI). And like Europe's share the US's one will be decreasing.

Letters and measure values, the first for certain - possibly the latter too.



Edited 1 time(s). Last edit at 10/13/2014 09:32PM by guenter.

Quote
siria
Perhaps test if converting your prefs.js to utf-8 and writing manual entries in it will stick? Wonder if there's only a prob with adding settings through xul or something, no idea.
Hmm, or another little test, change this post title to contain сайт, save the session and check prefs.js again?

I did not have to change prefs.js file. I went to file with Notepad++ to make the change and it was already UTF-8. The pref was correct.

user_pref("kmeleon.plugins.macros.domComplete1.suffix", ".сайт");

I guess that at shutdown, when KM rewrote the file some check was made and the encoding was adjusted by KM.

I will do some more testing, but maybe it is a problem already solved. As usual, Dorian is way ahead of me.

The conversion does not seem to happen in real time. I tried putting a value in prefs.js and reading it in the same session. That did not work. After a restart the value was correct, but by then I had already used it and it did not work to make a Favorite's name.

By the same token, if you are keeping a list in a pref then the list may have bad data until a restart. I am saying the data is bad, but that means is only that it looks different. I have very little understanding of UTF-8.

While km runs the current prefs.js is a copy in memory. So it is not worth while to add Your changes then. It is written to file when km is shut down. km rearanges prefs.js. So You can add Your changes anywhere bottom or top.

Lore from K-Meleon's past when about: did not work.

The news for me is that the km/GRE can check whether UTF8 data are sound & corrects them now. K-Meleon has had a recent bug with encoding in search.xml. Where the DATA were UTF8 and written down without marking the file as UTF8 but ANSI. When You had to open the file for work with an editor this corrupetd the encoding.

UTF8 was designed backward compatible so it starts with the ANSI letters.
That is all You will normally need in the US.

http://www.w3schools.com/charsets/ref_html_utf8.asp



Edited 1 time(s). Last edit at 10/14/2014 05:31AM by guenter.

Quote
guenter
While km runs the current prefs.js is a copy in memory. So it is not worth while to add Your changes then. It is written to file when km is shut down.

Sure, but it's also written to disk during the session all the time, and those edits are all that remains after a crash. But in my version (with old session titles) that temp-file is still in UTF-8...

The system that I use to create IE Favorites lets me rename the favorite and place it into a folder other than Favorites root. The macro language writes two prefs to hold the name and url and executes a js file to read the prefs and create the favorite. Obviously the js reads the temp file which has the data before conversion. This works fine when the data is ASCII, but not so good when UTF-8.

Note that I use the term ASCII and not ANSI. I don't know if that just means that I am from the USA or just old school.

Quote
JamesD
Note that I use the term ASCII and not ANSI. I don't know if that just means that I am from the USA or just old school.

ASCII (American Standard Code for Information Interchange) and ANSI are US terms. Old DOS used ASCII. ANSI was the default character set in Windows up to Windows 95. The later term supposedly comes from the fact that it was a draft intended for ANSI (US ISO board).

So old school mainly. Much like me. I still remember when they changed the spelling of my name to guenter because they tried to get along with ASCII and punched holes in cartboard.

I do not understand why any of Your codes can work "not so good when UTF-8".
Officially MS New Technology products use UNICODE internally.

Maybe "many applications and APIs continue to use the default encoding of the computer's locale when reading and writing text data to files or standard output"? (Wikipedia)

But that should IMHO not matter on a machine that uses English?

Quote
w3schools.com
"The first 128 characters of Unicode (which correspond one-to-one with ASCII) are encoded using a single octet with the same binary value as ASCII, making valid ASCII text valid UTF-8-encoded Unicode as well."

Edited 1 time(s). Last edit at 10/14/2014 01:30PM by guenter.

From what I remember, ASCII means basically english characters only, while ANSI contains most european languages, incl. german umlauts äöü and french ê etc., and Unicode has worldwide languages incl. chinese font etc. Should probably look it up again, but not enough time ;-P I do have the impression that most people on the net commonly think that ASCII=ANSI (wrong)

But am surprised about temp. storing of new favorite name+URL in a pref - why not simply variables? Or are they...?

I am storing information in prefs because I am too lazy to rewrite FavRenAdd.kmm and FavRenAdd.js to pass variables directly. Years ago the limit was very small for what could be on an 'exec' statement in the Macro Language. Maybe I should re-visit that macro/js system to see if I can pass the location of the js, the name for the favorite and the URL for the favorite in the exec statement.

There is also a version which writes to the registry. That was supposed to work for UTF-8.

Like I said, I am old and from the USA. I grew up using the term ASCII.

Oh... there is a limit to exec-lines? Oops, never heard, now am a bit worried. Should probably do some tests, but well, like you too... not now

JamesD, why rewrite if it mostly works?

Quote
siria
From what I remember, ASCII means basically english characters only, while ANSI contains most european languages, incl. german umlauts äöü and french ê etc., and Unicode has worldwide languages

Yes English is ASCII only - and ANSI is Windows system language specific.
Contains another 128 characters - last addition was €. 1996?

ANSI can be European (with Umlauts etc.) but also be non European pre- Windows NT characters.

Unicode is Web and NT based systems. Your 98 is in between ANSI/UNCIDODE AFAIR.

@ siria

When I wrote the macro back in 2007-2008, there was a limit to the length of the command line for the exec statement. Since then Dorian has increased many short limits. I just don't remember if this one got an increase.

Below is an excerpt from FavRenAdd.kmm. You can see my comment and the exception that I put in the code. Since I moved the name and URL to prefs, I have only the filespec for the js in the command line. It never even comes close to the limit now. Prior to using prefs, I did have some attempts which failed. Some sites had URLs which were very long.

@ guenter

I probably won't do anything unless I begin to see URLs that are not ANSI.

$_FavRenAdd_s1 = "k-meleon.FavRenAdd.s1" ;
$_FavRenAdd_s2 = "k-meleon.FavRenAdd.s2" ;
delpref($_FavRenAdd_s1);
delpref($_FavRenAdd_s1);
setpref(STRING,$_FavRenAdd_s1,$_FavRenAdd_PathName );
setpref(STRING,$_FavRenAdd_s2,$_FavRenAdd_CurURL );
$_FavRenAdd_Profile_Path = getfolder( ProfileFolder );
# $_fmax: The length of the command line must not exceed 258 characters!
$_fmax=258; $_fcmd="(wscript.exe \"" .$_FavRenAdd_Path. "\" \"".$_FavRenAdd_Profile_Path.")" ;
length($_fcmd)>$_fmax ? &_FavRenAdd_Error : exec("wscript.exe \"".$_FavRenAdd_Path."\" \"".$_FavRenAdd_Profile_Path."\"") ;
}

_FavRenAdd_Error{
setclipboard($_FavRenAdd_Path. "\n" .$_FavRenAdd_Profile_Path. "\n" .length($_fcmd) );
alert(_("Unable to create favorite - data too long!\n\nThe data was copied to the clipboard."),_("Failure to complete"),EXCLAIM);
}

Quote
guenter
Old DOS used ASCII. ANSI was the default character set in Windows up to Windows 95.

Do you remember ANSI.SYS? Absolutely necessary if you wanted to do spiffy things like placement of characters, colours, and whatever in DOS. Not sure if DOS2 had it, but it was certainly there in DOS3, both PC- and IBM-. I don't think DR-DOS had it... I used ANSI.SYS to build a "GUI" for a DB3 members register, so the secretary didn't have to "browse" all the records. That was fun.

But

Quote

ANSI is Windows system language specific.
Contains another 128 characters - last addition was €. 1996?

You are sort of right. DOS used ANSI for its code-pages, which ASCII could not supply. Then MS dropped ASCII from its Windows shell because it was simply not needed.

Gordon.

____________________
Understanding the scope of the problem is the first step on the path to true panic. [Florence Ambrose, "Freefall" 01372 January 22, 2007 http://freefall.purrsia.com/ff1400/fv01372.htm]

Don't melt ansi.sys which allows to use ansi sequences (I used it there is a very very long time ago) and ansi codepage for characters ( I used there is a few years ago)...

For exemple under Linux we use utf-8, and ansi sequences works correctly in utf-8 console ...

I use it, and it's really nice ...



A+

---

Mozilla/5.0 (x11; U; Linux x86_64; fr-FR; rv:38.0) Gecko/20100101 Ubuntu/12.04 K-Meleon/76.0

---

Web: http://jujuland.pagesperso-orange.fr/
Mail : alain [dot] aupeix [at] wanadoo [dot] fr

---

Ubuntu 12.04 - Gramps 3.4.9 - Harbour 3.2.0 - Hwgui 2.20-3 - K-Meleon 76.0 rc

---

Edited 5 time(s). Last edit at 10/15/2014 03:06PM by JujuLand.

Quote
JamesD
Years ago the limit was very small for what could be on an 'exec' statement in the Macro Language.

Finally did a test with my own macro using kko's old scripts. Relieved to find that a 400+ signs URL for testing got transfered without any probs into the shortcut via exec(wscript...$_JSpath...$_favUrl...$_favTitle...$_favFolder)
Looking at the contents with a text editor, the URL is just plain text inside, so it could surely be much longer yet. Uff ;-)

Still a bit curious about possible alternatives to passing a string from JS back into a macro in general. So writing it into the registry might be a way (probably again with that stupid alert-box workaround, as long as there's no wait-a-split-sec macrocommand or wait-until-injectJS-is-finished command).
But writing into a km pref I'd much prefer. Is there a JS command to write a km pref?

Quote
siria
But writing into a km pref I'd much prefer. Is there a JS command to write a km pref?

Any deeper reason why don't You use the kmm to set it?

p.s. http://kmeleonbrowser.org/forum/read.php?9,128836,129085#msg-129085

AFAIK contains a code sample that sets a pref via kmm.

Quote
gordon451
Do you remember ANSI.SYS? Absolutely necessary if you wanted to do spiffy things like placement of characters, colours, and whatever in DOS. Not sure if DOS2 had it, but it was certainly there in DOS3, both PC- and IBM-. I don't think DR-DOS had it... I used ANSI.SYS to build a "GUI" for a DB3 members register, so the secretary didn't have to "browse" all the records. That was fun.

Yes I remember. You needed to load it specifically via config.sys if You wanted to use it. Yes, it was in DR-DOS but I cheated and checked it out now. There were third party replacements so You could have it with any DOS. But I only used MS, Novel and PTS DOS which AFAIR shipped it.

p.s. And was much annoyed when the two later did not work anymore under Win95. I saw DOS the first time on the first CEBIT in Hanover. It was a brand new version maybe 3? I never build anything for it - I played Larry.



Edited 1 time(s). Last edit at 10/16/2014 04:03AM by guenter.

Quote
guenter

Quote
siria
But writing into a km pref I'd much prefer. Is there a JS command to write a km pref?

Any deeper reason why don't You use the kmm to set it?

What I keep saying all the time: because of the line-order-bug since KM1.6
Macro commands do not wait anymore until a script command in the previous macro line is finished, but execute FIRST. For example in line-1 you set a variable or a pref, then in line-2 run a script that needs that setting, and in line-3 change that var or pref again by macro command. Result: The script in line-2 uses the values set in line-3!
The only workaround to avoid this so far is to insert an annyoing alert-box between line2+3, which a user must first confirm.

Sorry I did not understand the problem initially. Alert box is a shoddy workaround for sure.

p.s. I have no knowledge to code any macros or JS.

But: Did You ever look here: https://developer.mozilla.org/en-US/Add-ons/Code_snippets/Preferences

Maybe that can help You to change this kmeleon.js that was used by Dorian to create a menu or to create something similar?

Components.utils.import('resource://gre/modules/XPCOMUtils.jsm');
Components.utils.import('resource://gre/modules/Services.jsm');

function startup() {
var KMeleon = Components.classes["@kmeleon/jsbridge;1"].getService(Components.interfaces["nsIJSBridge"]);
KMeleon.SetMenuCallback("MailNews", "NewsFo&x JS", function(mode) {
KMeleon.Open('chrome://newsfox/content/newsfox.xul', KMeleon.OPEN_NEWTA;
});
}

Also maybe look into the way kko set and changed prefs with the code in his prefs panel.