xclip to the rescue? How I wish . . .

I find the blogging platform I've been using for this blog to be horrendous. First, I don't want to have to log into a website to write articles: I'd rather type them on my computer, usually using my favored command-line text editor nano.

I was iniatially enthused to discover that blogspot/blogger (the host for this blog) offered the possibility of e-mailing posts. Esssentially, you add a custom word to an e-mail address they provide, and you can use that to send posts directly to your blog (subject line becomes the blog posting's title). Great, I thought, I can type posts on my computer and send them to my blog.

But my enthusiasm was shortlived. Initial tests indicated that the blogging platform was inserting a bunch of extraneous html tags into postings I'd send like that. No problem, I thought, I'll just write my posts with html, tags pre-inserted. Tests using that method indicated that the platform was continuing to insert those extraneous tags. See the graphic below for an indication of how the file looks in html editing view on blogger's site (all the &gt &lt &#39 and <br> were added by their system, not by me):

So I was left with a dilemma: how can I more conveniently, according to my preferred practices, continue blogging here? Researching the matter revealed the xclip utility, which appeared to be a good workaround for helping deal with the constraints involved in continuing to blog on this platform.

I discovered that, after I save the posting I've written under nano, I can simply cat the file into the system's clipboard by running a command like cat myfile.txt | xclip (see the LATER EDIT note added at the very end of this post for the corrected invocation for the blogger site) and from there paste it into the platform's editor interface by placing my cursor there and clicking the middle mouse button. Granted, it's pretty much the same as using a graphical text editor like leafpad and hitting ctrl-a to highlight the whole of the file, ctrl-c to copy the highlighted text to the clipboard, then ctrl-v after placing the cursor into the platform's editor interface. Not ideal, since I would much prefer if the e-mailing method would work as I had hoped by just importing the text as written in the file, but it's an acceptable (at least for now) workaround.

Perhaps more to come later, if I manage to make improvements or to figure out some heretofore overlooked factor.

Afterthought: I tried to paste this post into the blogger editing interface using xclip and it didn't work. Nothing would paste. So I had to paste the text into leafpad, then do the ctrl-a, ctrl-c, ctrl-v trick to be able to paste it here. So I changed the exclamation sign I had initially placed at the end of the title to a question mark, since it may not provide the solution after all. Sigh.

LATER EDIT: IMPORTANT! I've discovered that the way I can paste text into blogger using xclip is by running it with a couple of switches, as follows: cat myfile.txt | xclip -selection clipboard: having done that, I can then place the cursor in blogger's html editor and hit ctrl-v and the text from the file will paste into the editor as intended.

TeX/LaTeX: tex to txt

Since a .tex file is already essentially compatible with all text editors, you may be wondering why anyone would need to do any sort of conversion to .txt format. I would also have wondered at why such a task would need to be done--that is, until I ran into the need myself.

But I did run into such a need. The issue was that I had a nicely formatted .tex document--an article, actually, that I had translated with the aid of my computer, into a foreign language. I had one fairly competent translator check the machine translation over and do some corrections and thought I was ready to go. Then, another translator had a look and offered to do further improvements, to which I gladly assented. This is where I ran into a problem with the nicely-formatted file.

This translator, although quite well-qualified and fairly capable in matters technical, was nonetheless not at all familiar with Tex/LaTeX formatting. So I couldn't really give him the document in the most optimum format for me (.tex) for correcting. And at the same time getting from him the corrected text in a format like .pdf or .doc would further complicate my task of getting it back to its nicely-formatted .tex state. Thus, I decided that .txt would be the most neutral format to use for providing the translator with the computer-translated text for further revision. But how to do that?

Well, I had already created a pdf of the document, so I had that to work from. Using sed or awk to strip out all the TeX formatting codes would be an option for someone far better versed in those utilities that I am. But even that might prove a fairly involved and time-consuming task.

Some on-line searching revealed another possible solution: it involved using the utility pdftotext. It seemed worth a try.

Sure enough, running pdftotext file.pdf file.txt actually gave quite good results. There were a few anomalies I needed to clean up, but they were actually fairly few in number. I'd say the whole process took about 15 minutes total, after which, I had a .txt version of this 5k-word file that I could submit to the translator.

So, in the unlikely event that you may need to convert your .tex file to .txt, I can recommend the routine of first converting it to .pdf, then the resulting file to .txt. I should probably mention that this file didn't contain graphics, a table, or any sort of chart. So I can't vouch for how it would work on files containing comparatively more complex elements such as that. So, probably the less complex the document is, the more successfully it will convert using this method. So, there you have it, a method for converting .tex to .pdf to .txt

A motd project

Long time no post. Well, life precludes various tech experimentation and hoped-for successes sometimes.

But I do have a couple of projects to describe. One involves a back-up routine I developed a few years ago. I have doubts that one will be of much interest to most in that it was developed specifically for the non-uefi machines I tend to use and target the paltry-sized hard drives (120 gb and less) and comparativeley meager installations I favor. But it's fairly straightforward and, to my somewhat unrefined tech sensibilities, clever. The other is yet simpler and is the one I'll be describing in this post.

I refer to it as a motd (message of the day) project/routine because I want the system to provide me with certain information when I log in. My implementation differs a bit in that it also even provides this information whenever I start a new bash session.

The information I want the system to provide me is twofold: 1) reminder of when I last backed up the system, and 2) on some of my machines, a reminder of when I last updated the operating system. In most cases what I'll need reminding about is only number 1), since I'm regularly using these systems and do operating system updates on these rolling-release (Arch and Void) systems every 1-2 weeks.

On other systems, however, such as my laptop, I'll need reminding about number 2) as well. That machine, and the VPS I use, are systems I am only occassionally logged into. So knowing when was the last operating system update is quite important on those.

So my solution involves creating the files motd-backup.txt and motd-update.txt in my home directory and writing target information to them. Since I haven't scripted the back-up routine, that has to be done manually by running something like echo "++last system back up done on $(date)++" >/home/user/motd-backup.txt after I've finished running my back-up routine. That's something I'll be adding to the notes I keep as a reminder about how to use that routine.

As for the operating system updates, I've semi-automated those. I run a script at certain intervals that opens a bash session and queries whether I'd like to do a system update: a response in the positive triggers the update. I've added to that a command that runs after a successful update and writes information to ~/motd-update.txt. It looks as follows: echo "++last update $(cat /var/log/pacman.log | tail -1 | cut -c2-11)++" >/home/user/motd-update.txt

Arch'ers will understand that what's happening is that the pacman log file is being queried and information from the final entry is being excerpted and cat'ed (is this a misuse of cat?) into the motd-update.txt file. It has worked well in my testing and should suffice for the purpose.

The final piece of the puzzle is getting the information to display. I want it to be available during my day-to-day computer usage and not in a file stored somewhere on the system. Experience has indicated that I am likely to forget about such a file and so become remiss in my system administration responsibilites.

Since I'm regularly using a terminal, I decided a good place to cause that information to be auto-displayed with regularity would be make it part of the process of starting terminal sessions. So I put the following line at the end of my .bashrc: cat ~/motd-update.txt && cat ~/motd-backup.txt

And that addressesd my issue. Every time I log into the system or start a new terminal, that information appears.

As I write this, I begin to think of alternatives. For example, I could use something like Xmessage and cause a pop-up to appear at regular intervals--say once weekly--that would contain such information. I'm sure other options like zenity could be fairly easily configured to do the same. Or perhaps making a .png out of the information as I wrote about a few years ago (https://audaciousamateur.blogspot.com/2020/04/another-imagemagick-trick-txt-to-png.html) and using cron to pop up feh or some image-displaying utility that shows the information in bright text on a dark background. That could be more disruptive but might be more effective.

In any case, what are your ideas for displaying important information like that? Auto-generated e-mails, for example? There are lots of possibilities. The ways to skin this cat (pun intended) are numerous.

Another imagemagick trick: txt to png

I ran across an article some time ago that discussed exporting text to an image format using imagemagick's convert option. Although it looked interesting, I wasn't sure at the time whether it would actually be useful to me in any real-world way. Since I'm now considering various schemes for making paper back-ups of some sensitive data, I've begun investigating again this means of making an image from text. So I thought I'd provide a bit of documentation here and just make a comment or two on some recent tests.

So the current iteration of the command I've been testing is

convert -size 800x -pointsize 6 caption:@my-data.txt my-data.png

The -size switch performs the obvious function of setting the image size, or at least its width. The -pointsize switch defines the size of font that will be used in the text that will appear in the image--in the case of this example, a very small font. I'm sure font face can be specified as well, though I am not experimenting with that at this time.

In the example given here, the name of a text file is specified. Long lines are split by the program according to the width specified, but if no width is specified the width of the image will correspond to the longest line length. The output of a command can also be directed to an image. Slightly different syntax from what is seen in this example would need to be used in that case, of course.

Another convert option that works similarly to caption is label. It seemed to me the caption option was more relevant to the task I was experimenting with since a larger amount of text could be involved.

The experiments I've been conducting are for the possible purpose of making a paper back-up of the sensitive data. An image file of the data could be created, printed, then the image file erased.

Finally, I recently discovered that there is a fork of imagemagick called graphicsmagick. I have not looked into that very deeply or used it so far. But I will be investigating further.

For reference I got my original introduction to this neat feature from an article at https://www.ostechnix.com/save-linux-command-output-image-file/

More can be found at http://www.imagemagick.org/Usage/text/

view youtube videos while browsing with elinks

I recently wanted to view a youtube video using a computer that had no graphical browser installed. Long story short, the computer runs Gentoo and is used almost exclusively for recording and playback of television content: so for almost all use case scenarios, having a graphical browser is not needed, the excessive compile times they would entail (over 24 hours on this somewhat low-resource machine) being unjustifiable.

I decided there must be some way, using the non-graphical browser I did have on this machine--elinks,to view youtube videos. A bit of online research revealed how I could accomplish this task. Though there is undoubtedly more than one way to skin this cat, I used the one that seemed most straightforward to me, as described below.

Since I already had installed the mpv utility, all I had to do was some minor tweaks to elinks. First, I went into Setup > Option manager > Document > URI passing and added a new entry. I simply named it youtube-handle_mpv. Of course the final task for this step is to save that option.

I then edited that entry, using information found at the Arch wiki entry for elinks, and added the line mpv %c (this allows elinks to feed the designated URI to mpv). Having done that, I next needed to assign a key which, when pressed, would trigger the URI passing.

I went to Setup > Keybinding manager > Main mapping > Pass URI of current frame to external command and there designated the grave or backtick key as the one that would trigger the URI passing. Again I selected "save" and exited the Setup menu.

After having done that, I navigated elinks to youtube's site, searched for the video I wanted to view and, having highlighted the desired link using the arrow keys, pressed the grave/backtick key. After a brief pause (for downloading and caching some of the data, I presume), mpv opened and the video began to play.

NOTE: the pause between pressing the designated key and the actual playback of the video under mpv could vary based, I believe, on the length/quality of the video.

Stress-testing hard drives

This entry will consist mostly in someone else's content. The back story is that, about 3 years ago on the mythtv-users list-serv, one of the list members offered such a concise, straightforward, and apparently sound description of how she tests out new hard drives, that it remained in my memory.

Well, now that it has come time for me to replace an aging hard drive in my MythTV machine, it's time to dig out those directives and actually use them. And while I'm doing that, I may as well post them on this blog for future reference. Credit for the material goes to faginbagin/Helen, the user who posted it in the list-serv.

Without further ado, here is the content:

I look for drives with the longest warranty. This Christmas I bought 4 3TB HGST drives with 3 year warranties. Got to look close, because some only have 2 year warranties. Before I put any new drive into service, I use the following procedure to increase the chances it's a good drive
that will last. 

Capture SMART data via:
smartctl -a /dev/sdx > smart0.out 

Write semi random data on the whole drive with:
nohup shred -n1 -v /dev/sdx > shred.out & 

Go away for a couple of hours. Check shred.out and figure out how long
it will take to finish. Come back when it should be done. 

Read the whole drive and compute a checksum:
nohup md5sum /dev/sdx > md5sum.out & 

Go away for roughly the same time it took shred to write to the drive.
Read the whole drive again and make sure the checksum matches:
nohup md5sum -c md5sum.out > md5sum.chk & 

Go away for roughly the same time it took the first md5sum to read the
drive. 

Write zeros to the drive:
nohup dd if=/dev/zero of=/dev/sdx bs=1M > dd.out & 

Capture SMART data via:
smartctl -a /dev/sdx > smart1.out 

Compare the two smart runs:
diff smart0.out smart1.out 

Make sure there are no complaints about sectors. 

Make sure the kernel didn't report any errors:
dmesg| tail 

If no SMART or kernel reported errors, create partition table, create
partitions, mount, etc... 

If any errors, return immediately. 

 Original post located at http://lists.mythtv.org/pipermail/mythtv-users/2016-April/386438.html

dhcpcd hooks: what are they and why should you care?

I know I didn't know what dhcpcd hooks were and why I should care about them. That is, until I set up a file server that runs headless and which I want to make sounds when certain events, such as the network coming up, occur. This, as may be evident, will help me ascertain whether, in absence of a monitor displaying graphical indicators, the machine booted and came online successfully--a pretty important status for a machine like a file server.

dhcpcd (the final "d" stands for "daemon") is, of course, the utility that runs on a lot of Linux computers in order to get them connected to the network. It polls for an IP from whatever dhcp (dynamic host configuration protocol) server is running on the network, and the IP received gets assigned to the computer's designated network interface. So the "hooks" under discussion essentially latch onto that process and cause some other process(es) to be triggered once dhcpcd accomplishes its task. This, then, could allow me to implement the desired behavior on my headless file server

I had part of the solution in place already, namely the beep program, a utility that allows for the playing of various tones through the pc speaker. Yes, I'm aware that most computer users seem to want only to disable the pc speaker: I, on the other hand, have found it quite useful on my systems.

Having done some on-line research on this matter, I was able to succeed at the task of getting the pc speaker to play a tone once the computer had booted and gotten an IP by using the following steps (geared toward the Void Linux distribution installed on that target system).

I first created a file owned by root and in the root group in /usr/share/dhcpcd/hooks/--(call it, say) 10-beeponIP--with the following content:

if ([ $reason = "BOUND" ] || [ $reason = "RENEW" ]) then
# your script commands here (see below for the command I used)
/usr/bin/beep -f 1000 -r 5 -n -r 5 -l 10
fi

I can't go into many specifics of the bash syntax seen in this file since I understand it rather poorly myself (it was simply lifted from the askubuntu link listed below). But some testing of its claimed efficacy revealed that it would, in fact, result in the behavior I was aiming to enable.

I had to set proper permissions on that file, then symlink it. Doing the former was straightforward (permissions needed are, like the rest of the files in that directory, 444). I ran the following command to create the needed symlink: sudo ln -s /usr/share/dhcpcd/hooks/10-beeponIP /usr/libexec/dhcpcd-hooks.

Having done that, on rebooting the computer, the designated tone plays though the pc speaker, letting me know that the system booted normally and is now on-line. Mission accomplished!

Some links that helped me to better understand and accomplish this task are given below:

https://askubuntu.com/questions/1005653/how-do-i-execute-a-script-after-dhcp-assigns-an-ip-address-on-start-up
https://wiki.voidlinux.eu/Network_Configuration#Starting_wpa_supplicant_through_dhcpcd_hooks
https://man.voidlinux.eu/dhcpcd-run-hooks.8