Addendum to fifth installment: prescience?; more on xmodmap

Prescience?

Ok, grappling with this xmodmap issue has caused me to revisit the issue of inputting non-latin characters under Linux, something I dealt with a few years ago. Finally, with helpful hints gotten in on-line forums and reading various README files, I had some success. I observe now that I displayed a bit of prescience back then as well.

For posterity's sake, I'm pasting here the resolution I came up with--the prescient bit (written almost 8 years ago) highlighted in bold:

. . . I'm taking the other approach I mentioned earlier (editing XF86Config-4 to allow for keyboard map/font switching for X apps).  He's also helped me out with this. . . . there is an additional line possible in the keyboard stanza of XF86Config-4 (this refers to 4.3.0something and may not apply to earlier releases): the XkbVariant" line.  In order to get the phonetic Russian and polytonic layouts (or at least a semblance of polytonic) I want, I've added the "XkbVariant" line with ",phonetic,polytonic" following.  That has gotten me most of the way to where I need to be in entering unicode input into Xapps.  It also works in a console if you have the right fonts installed.  As an additional detail, I should mention that I think you must have a unicode locale specified in your system for this to work.  In my case, I've set locale to en_US.UTF-8 (how to set that varies by distribution: dpkg-reconfigure locales is how Debian does it).  I'll provide the relevant section of my XF86Config-4 for those interested:

Section "InputDevice"
Identifier      "Generic Keyboard"
Driver          "keyboard"
Option          "CoreKeyboard"
Option          "XkbRules"      "xfree86"
Option          "XkbModel"      "pc104"
Option          "XkbLayout" "us,ru,el"
Option          "XkbVariant" ",phonetic,polytonic"
Option          "XKbOptions" "grp:alt_shift_toggle,grp_led:scroll"
EndSection

Note the "grp_led:scroll" segment too, which causes the scroll lock LED on the keyboard to light up when I'm in one of the alternate keyboard/font mappings.  In summary, I've written this in the hope it may help someone else wanting to do this under Linux. . . . my little research/how to project here could likely be itself obsoleted pretty shortly.

Now, as those of you who've been working with GNU/Linux this long will know, we no longer edit GUI configuration files by hand: these are to be auto-generated by the system. But sometimes, for whatever crazy reason, you'll need to do some tricky thing like I was doing then so that you can switch keyboard layouts on the fly.

In current circumstances, for example, my work computer uses a very minimalist desktop that includes no keyboard-layout switching utility/applet--something Gnome, KDE, XFCE4, and the like, do include. So, if I want to implement keyboard-layout switching there, I'm going to need a solution like the one I implemented several years ago.

I suppose it's still possible to hand-edit the GUI configuration file--now called xorg.conf, as we may all know--but that, I believe, is not the canonical method. Instead, what's now expected is that modifications such as the one above be entered into a different configuration file, from whence they get loaded into the xorg.conf file the system automatically generates.

That file is located (on Debian-based systems) in /etc/default and is called, appropriately enough, "keyboard." The content needed to implement the modifications listed above--along with further context from that file--will look as follows:

# If you change any of the following variables and X is configured to
# use this file, then the changes will become visible to X only if udev
# is restarted. You may need to reboot the system.

# The following variables describe your keyboard and can have the same
# values as the XkbModel, XkbLayout, XkbVariant and XkbOptions options
# in /etc/X11/xorg.conf.

XKBMODEL="pc105"
XKBLAYOUT="us,ru,el"
XKBVARIANT=",phonetic,polytonic"
XKBOPTIONS=""ctrl:nocaps,grp:alt_shift_toggle,grp_led:scroll,terminate:ctrl_alt_bksp""

# If you don't want to use the XKB layout on the console, you can
# specify an alternative keymap. Make sure it will be accessible
# before /usr is mounted.
# KMAP=/etc/console-setup/defkeymap.kmap.gz

There's my demonstration of prescience for the year (I hope that doesn't change again within the year). Now, on to a revisitation of the xmodmap issue.

More on xmodmap
Turns out the xmodmap issue is in no way unique to XFCE4: rather, it seems to affect many, and perhaps all, graphical environments. As a moderator from the Arch Linux forums put it almost exactly two years ago, "This is an unsolved problem for quite some time" (see this thread for a bit of further detail).

The symptom on the machine I'm trying this with is that it xmodmap will not initiate when the GUI comes up, despite my attempts at placing commands intended to invoke it in relevant start-up files: xmodmap will only take effect when run manually after the graphical environment is running. Even then, it causes 100% CPU usage for several seconds while it is initiating (this is a pretty low-resource system, at only 800 Mhz, which partly explains the delay).

What is the nature of this problem? Why can't this re-mapping or re-binding be done in connection with the user's log-in and graphical environment's initiation? Does it have something to do with evdev which, as I understand it, is what implements the system's auto-configuration of peripherals like keyboard and mouse? I simply do not understand, unfortunately.

But this matter seems to have foiled greater minds than my own. As a case in point, while trying to resolve this and similar problems, I bumped into the following example in the Xsession man page--which even seemed calculated to address specifically the problem I was trying to resolve:

Here is an example of how one might write a script, named 40custom_load-xmodmap, to invoke xmodmap(1):

SYSMODMAP="/etc/X11/Xmodmap"
USRMODMAP="$HOME/.Xmodmap"

   if [ -x /usr/bin/X11/xmodmap ]; then
      if [ -f "$SYSMODMAP" ]; then
           xmodmap "$SYSMODMAP"
      fi
     fi

    if [ -x /usr/bin/X11/xmodmap ]; then
        if [ -f "$USRMODMAP" ]; then
            xmodmap "$USRMODMAP"
        fi
     fi

Well, I tried that. I created a file with that content, put it in the recommended directory (/etc/X11/Xsession.d) and gave it the same permissions as the other files in that directory. But the xmodmap modifications I'd stipulated were not in effect once the graphical environment came up: I still had to run xmodmap manually.

The only thing approaching a resolution I've found that's calimed to work, is a really ugly looking hack I read about on the Archbang forums (you can read about it here). I haven't tried it yet and so cannot confirm that it works: and I hope to avoid trying it. I really hope there's something a bit less kludgy that'll resolve the issue.

As for things I might attempt, I've thought of entering each of my eight modifications on separate lines in the .xsession or .xinitrc file to see whether that'll cause the modifications to take effect when the user logs in--though that solution isn't much prettier than the one outlined on the Archbang forum. If I did do that, I assume the lines would take the form xmodmap -e “keycode 12 = 3 numbersign 3 numerosign” &. If I try that and it's any more successful, I'll report back about it.

Meantime, I'm still looking for more informed helpers with this issue. Got any insights?

Fifth installment: "fun" with xmodmap

In case it's not evident I'll point out at the outset that this entry is facetiously entitled (as the scare quotes might indicate). What I did with xmodmap wasn't exactly fun. Maybe mildly rewarding, finally. But also a bit perturbing.

Why perturbing? Because, after finally discovering a means of accomplishing the goal at which I was aiming, I bumped into what appears to be a long-standing bug that thwarts the resolution about half of the time. I'm still looking for a better resolution but, at least for the time being, I have a mostly workable solution.

Anyway, back to describing the task at hand, which I record here in case it may benefit anyone else and so that the lesson might stick with me longer.

So, I needed to help someone get all needed keys working on a keyboard that would, for the most part, be used for creating non-latin content. For what it's worth, the user will be switching back and forth between English and the non-latin language occasionally.

The target machine has XFCE4 installed as graphical environment. And it happens to be a "new-world Mac" that runs Linux--Debian, to be precise--a factoid which may prove fodder for a future entry. And yes, it's the powerpc architecture, and yes, it has a Mac keyboard (but not a Mac mouse).

XFCE4 has facilities for keyboard switching, which I hoped would make my job a little easier. Its graphical utilities did allow for selection of the needed encoding and layout, so I got a good part of the way using that. The user brought along stickers for the keyboard's keys which could be placed on the appropriate latin-lettered keys so that, being a non-touch-typist, it would be clear which letters should be typed in.

Almost all the letter keys all worked fine under the initial configuration. The problem was that the numerical keys across the top of the keyboard, when shifted, were supposed to be printing certain signs, and the stickers and the actual signs were not matching up. Something like 8 keys were affected. That needed to be fixed.

On looking into what could be done about this state of affairs, xmodmap began to look like it could provide an appropriate solution. So, I dove in. I did get it to work for the most part, though it seems XFCE4 cannot play nicely with xmodmap, so I had to introduce a kludge to be used on certain occasions (see here and here for some dated descriptions of the issues).

It was a bit mind-numbing trying to figure out exactly how to deploy xmodmap, given that I'd never used it before and that I was trying to make it handle two different encodings. The command xmodmap -pke >.Xmodmap-pke proved very helpful, since it created a text file of the current keyboard mapping.

Looking over that output I was able to identify the target keys. Some further reading and experimentation enabled me to find the right syntax to be used for modifying the current mapping.

It seems the canonical way of causing xmodmap to modify the current keyboard mapping is to create a file named .Xmodmap in the user's home directory. The window manager or graphical environment is supposed to read that file on starting and modify the mapping that will be used for those keys within the graphical environment.

For information on how the entries in that file should look, you should, of course, consult the man page--which, in this case, actually contains examples (many man pages, for reasons I as non-programmer can neither fathom nor appreciate, do not).

I did, of course, find helpful information in my web searches as well. The initial red herrings that needed to be sifted out were related to the fact that virtually all pages I found dealing with the matter of re-mapping macintosh keyboards were focused on re-mapping system keys like control, alt, or the "Apple key." I did not find any information dealing with the specific issue I was researching, namely, on re-mapping number keys.

But, as I observed, using xmodmap -pke gave me the data that, once I'd figured out the relevant portions, enabled me to do the required mapping. I needed, for example, the 3 key, when shifted in the non-latin mapping, to produce not the hash or number sign (i.e., #), but something that looks like the following: №. That, as I discovered, is called the numerosign.

Likewise the 7 key, when shifted in the non-latin mapping, needed to produce the question mark. The question-mark/slash key, on the other hand, needed to produce a period and a comma in the non-latin mapping, the latter when shifted.

The output of xmodmap -pke indicated that the 3 key corresponded to what to xmodmap is keycode 12 (xmodmap uses the terms keycodes and keysims to designate the various keys found on the keyboard but I do not claim to understand what is the technical difference between those terms/concepts). The Xmodmap-pke file I had created showed the following line for keycode 12:

12 0x0033 (3) 0x0023 (numbersign) 0x0033 (3) 0x0023 (numbersign)

From looking at other entries in Xmodmap-pke I was able to discern that the second half of the various lines which make up that list referred to the non-latin mappings for this keyboard, while the first half referred to the default, latin, mapping. So 0x0033 (3) 0x0023 (numbersign) is the output the key generates when the latin mapping is in effect, 3 being output when the keyboard is unshifted, and the hash sign being output when it's shifted.

The second half of the line is where my problem lay, then. In its present configuration, the keyboard was mapped to output the same 3 and hash character in the non-latin mapping as it was configured to output in the latin one. So I set about changing that.

I found some helpful samples for testing this at http://mac.linux.be/content/tuning-ubuntu-macintosh-key-mapping-and-3-mouse-button-emulation-0 . Specifically, I discovered that keyboard mapping can be changed sort of on-the-fly from the command line by issuing a command like xmodmap -e “keycode 12 = 3 numbersign 3 numerosign”. Once I did that, the 3 key, when shifted under the non-latin mapping, did, in fact, cause the desired № to appear.

That provides a good way to test the re-mapping to make sure it's working as desired. But if you want the change to be permanent, as I did, you have to construct an .Xmodmap file in your home directory that has all the relevant re-mapping lines in it and which should be read and processed each time the graphical environment starts. Entries in the .Xmodmap,file, by the way, look as follows: keycode 12 = 3 numbersign 3 numerosign. My .Xmodmap file consists in 8 such lines.

Once I had all relevant entries in that file I expected that the mapping modifications would be set and working once the graphical environment had fully loaded. In practice, however, I found that only some of the time was that the case: in many instances I had to open a terminal and type xmodmap .Xmodmap for the re-mapping to go into effect. Further internet searching revealed to me that there is some sort of incompatibility between xmodmap and XFCE4 that causes them to not work together very well. So I created a graphical shortcut that would run xmodmap .Xmodmap when the user clicked on it so as to get the desired mapping to take effect. Not a very pretty solution, but one that works for the time being.

There may be other solutions to this re-mapping issue. The xmodmap solution is one of the first I came across and pursued, so I can't speak to any of the other possible solutions. If you know of some other, possibly more effective one, I'd appreciate hearing about it.

NOTES TO SELF: setxkbmap provides some interesting key re-mapping options as well, a list of which can be found in /usr/share/X11/xkb/rules/base.lst. Another possibility I ran across suggests editing the relevant mapping file in /usr/share/X11/xkb/symbols/, but those files make little sense to me and I'm hesitant to go fiddling with them.