handyfloss

Maybe the constant increases in hard disk capacity provide us with more space we can waste with our files, but there is always a situation in which we would like to squeeze as much data in as little space as possible. Besides, it is always a good practice to keep disk usage as low as possible, just for tidiness.

The first and most important advice for saving space: for $GOD’s sake, delete the stuff you don’t need!

Now, assuming you want to keep all you presently have, the second tool is [[data compression]]. Linux users have long time friends in the [[gzip]] and [[bzip2]] commands. One would use the former for fast (and reasonably good) compression, and the latter for when saving space is really vital (although bzip2 is really slow). A more recent entry in the “perfect compression tool” contest would be [[Lempel-Ziv-Markov chain algorithm]] (LZMA). This one can compress even more than bzip2, being usually faster (although never as fast as gzip).

One problem with compression is that it is a good way of storing files, but they usually have to be uncompressed to modify, and then re-compressed, and this is very slow. However, we have some tools to interact with the compressed files directly (internally decompressing “on the fly” only the part that we need to edit). I would like to just mention them here:

Shell commands

We can use zcat, zgrep and zdiff as replacements for cat, grep and diff, but for gzipped files. These account for a huge fraction of all the interaction I do with text files from the command line. If you are like me, they can save you tons of time.

Vim

[[Vim (text editor)|Vim]] can be instructed to open some files making use of some decompression tool, to show the contents of the file and work on them transparently. Once we :wq out of the file, we will get the original compressed file. The speed to do this cycle is incredibly fast: almost as fast as opening the uncompressed file, and nowhere near as slow as gunzipping, viming and gzipping sequentially.

You can add the following to your .vimrc config file for the above:

" Only do this part when compiled with support for autocommands.
if has("autocmd")

 augroup gzip
  " Remove all gzip autocommands
  au!

  " Enable editing of gzipped files
  " set binary mode before reading the file
  autocmd BufReadPre,FileReadPre	*.gz,*.bz2,*.lz set bin

  autocmd BufReadPost,FileReadPost	*.gz call GZIP_read("gunzip")
  autocmd BufReadPost,FileReadPost	*.bz2 call GZIP_read("bunzip2")
  autocmd BufReadPost,FileReadPost	*.lz call GZIP_read("unlzma -S .lz")

  autocmd BufWritePost,FileWritePost	*.gz call GZIP_write("gzip")
  autocmd BufWritePost,FileWritePost	*.bz2 call GZIP_write("bzip2")
  autocmd BufWritePost,FileWritePost	*.lz call GZIP_write("lzma -S .lz")

  autocmd FileAppendPre			*.gz call GZIP_appre("gunzip")
  autocmd FileAppendPre			*.bz2 call GZIP_appre("bunzip2")
  autocmd FileAppendPre			*.lz call GZIP_appre("unlzma -S .lz")

  autocmd FileAppendPost		*.gz call GZIP_write("gzip")
  autocmd FileAppendPost		*.bz2 call GZIP_write("bzip2")
  autocmd FileAppendPost		*.lz call GZIP_write("lzma -S .lz")

  " After reading compressed file: Uncompress text in buffer with "cmd"
  fun! GZIP_read(cmd)
    let ch_save = &ch
    set ch=2
    execute "'[,']!" . a:cmd
    set nobin
    let &ch = ch_save
    execute ":doautocmd BufReadPost " . expand("%:r")
  endfun

  " After writing compressed file: Compress written file with "cmd"
  fun! GZIP_write(cmd)
    if rename(expand(""), expand(":r")) == 0
      execute "!" . a:cmd . " :r"
    endif
  endfun

  " Before appending to compressed file: Uncompress file with "cmd"
  fun! GZIP_appre(cmd)
    execute "!" . a:cmd . " "
    call rename(expand(":r"), expand(""))
  endfun

 augroup END
endif " has("autocmd")

I first found the above in my (default) .vimrc file, allowing gzipped and bzipped files to be edited. I added the “support” for LZMAed files quite trivially, as can be seen in the lines containign “lz” in the code above (I use .lz as termination for LZMAed files, instead of the default .lzma. See man lzma for more info).

Non-plaintext files

Other files that I have been able to successfully use in compressed form are [[PostScript]] and [[Portable Document Format|PDF]]. Granted, PDFs are already quite compact, but sometimes gzipping them saves space. In general, PS and EPS files save a lot of space by gzipping.

As far as I have tried, the [[Evince]] document viewer can read gzipped PS, EPS and PDF files with no problem (probably [[Device_independent_file_format|DVI]] files as well).

The default [[MediaWiki]] installation will leave you with [[URL]]s of the type:

This is ugly! Following instructions at the MediaWiki.org site, you can make it simpler and nicer:

To achieve that, add the following to /etc/apache2/httpd.conf:

Then add/modify the following at /var/lib/mediawiki/LocalSettings.php (again, Debian default path):

Recall that you must have two “directories”, which in the example above are /w and /wiki. The former is “real” and the latter is “virtual”.

The real dir (the one used as value for $wgScriptPath) must contain the MediaWiki files, thus it must point to the /usr/share/mediawiki dir. To this end, it must either exist in the [[Apache HTTP Server|Apache]] root (usually /var/www/), or be an alias. If you follow the first route, you can make a link, like in the following example:

The second route would imply adding this line to /etc/apache2/httpd.conf:

The latter requires restarting the Apache daemon, but I personally prefer it.

The virtual dir (the one used as value for $wgArticlePath) will be our path to get rid of the URL ugliness, and point directly to an article’s title. As such, it must be aliased in /etc/apache2/httpd.conf adding the following line to it, as mentioned above:

Finally, you shold enable the rewrite PHP module, if it’s not enable already, and reload Apache:

After that, pointing to website/wiki/somearticle should lead you to the wiki page for somearticle. For more information, refer to the MediaWiki.org site.

No matter what they say, [[Hibernate (OS feature)|hibernating]] Linux laptops has always been a problem. I managed to get my MacBook to [[sleep mode|suspend]] to RAM quite reliably without much of a hassle. It suspends when I close the lid, and it resumes when I open the lid back. I even configured it to suspend when battery is critically low. This way, and due to the really low power consumption while suspended, I can safely forget my laptop on and unplugged for extended periods of time, and the worst that can happen is that I will have to resume it. A huge difference from the nasty surprise of finding it off and losing all the information not saved to disk.

However hibernating to disk is a whole different business. I never managed it to work, and that was an itch I wanted to scratch. Finally I managed, with the following recipe.

HowTo

First, make sure that you have enough [[paging|swap space]] available in disk. In Linux you generally create a swap [[disk partitioning|partition]] when installing the OS. The old adage states that one should make the swap partition twice as big as the RAM memory of the computer. With modern computers this is both unnecessary (because the big RAM makes sure you’ll never run out of it, and if you do, you are screwed anyway) and wasteful (if you have a 4GB RAM, it means that you dump 8GB of disk space). However, if you intend to hibernate your computer, all the information in the RAM memory has to be copied to the hard disk, so you sure need at least as much swap as RAM (but not twice).

Second, you need to use the correct tool. I use [[Xfce]] as desktop environment under Ubuntu, and the Exit menu presents me with six options: “Switch user”, “Log out”, “Restart”, “Shut down”, “Suspend” and “Hibernate”. I think that the latter two make use of the tools in the acpi-support package. The suspend action seems to work OK, but the hibernate one doesn’t (for me). It runs the command /etc/acpi/hibernate.sh, and it gives me problems. Thankfully I found some utilities that work reliably, namely pm-utils.

The pm-suspend command seems to work as correctly as the “Suspend” button in the Exit menu of Xfce. The pm-hibernate, on the other hand, works perfectly, unlike the “Hibernate” button. The drawback is that only root can run it. My solution is to put a launcher button in the Xfce task bar, that will run “gksudo pm-hibernate”. This way I am asked for my password and, if sudo is correctly set up, pm-hibernate will run.

More info

Sometimes it is very interesting to run some commands at suspension/hibernation moment, or at resuming/thawing. One such command is [[hdparm]], with which you can fix the long known load/unload cycle problem (you can google about it). Another one is one to fix a problem that apparently appears on MacBooks: the [[touchpad]] is lost when the computer wakes up back. The keyboard works, and USB mice work, but the touchpad doesn’t. This problem can be fixed by reloading the appletouch [[Loadable kernel module|kernel module]]:

You can fix both issues above by creating a file named, e.g., 99-macbook_fix, in /etc/pm/sleep.d/, and making it executable. Then write in it the following:

Last friday I wrote about how to install a Huawei E220 modem under MacOSX. Today I will write the corresponding HowTo for Linux.

Usually installation of hardware with non-free drivers is a bit more difficult in Linux than in MacOS and Windows, because the drivers are only made for the latter two. However the E220 is well supported by the Linux kernel (starting at 2.6.20, apparently), so we only need to tweak some configuration files.

1 – Make the system see it properly

The Huawei E220 is a dual machine: apart from being a modem, it is also an USB flash device, with some space to save the Mac/Windows drivers, so that it will “autoinstall” when plugging it under those OSs.

This adds a small level of difficulty, because we have to make sure that the OS sees it as a modem, not as a storage device. In principle the command dmesg (or the file /var/log/messages) will tell us about it. However, I have had it work when dmesg would say that it was a storage device!

The short story is that some [[Kernel (computer science)|kernel modules]] must be loaded, and some others unloaded, when you plug the device. Needed modules: option, usbserial, ppp_async. Must not be present: airprime. In my case usb_storage made no harm, some people say you should unload it. For airprime not to be automatically loaded, put it in some [[Modprobe#Blacklist|blacklist]] file in /etc/modprobe.d/. I decided to add the following line to /etc/modprobe.d/blacklist-modem:

You can ensure the required modules are loaded by taking advantage of [[udev]], but it is not really necessary (in my case it wasn’t). udev can also give you a consistent name for the modem. For me the relevant device was always /dev/ttyUSB0, but you can make it /dev/huawei if you will. For that, you can put the following optional rules in a file in /etc/udev/rules.d/ (for example create 55-huawei.rules):

Two notes: the strings in idProduct and idVendor are obtained running the command lsusb when the modem is plugged. It will show something like:

This is a very neat trick for any USB device we want to manage with udev. The second note is that [[kppp]] (see later) only allows to choose a modem device from a list. If you make the modem be /dev/huawei, you will not be able to use kppp, since that device won’t appear in the list.

2 – Configure wvdial / kppp

You can make use of programs such as [[wvdial]] or [[kppp]] to make the actual connection. I use kppp myself, but that’s up to you (wvdial is apparently more flexible).

wvdial

To use it you have to create a /etc/wvdial.conf file. You can achieve this by running wvdialconf as root, or editing the file by hand, if you are brave.

For me, the output of wvdialconf yielded:

And my current /etc/wvdial.conf looks as follows:

In bold, the relevant user-provided settings. In italics, some items in which you can put whatever, because it doesn’t seem to make a difference.

To connect, run wvdial simyo (or whatever you put in the “[Dialer xxx]” setting above), in the command line. To terminate, Ctrl+C.

kppp

This is the one I use. To open the config/run dialog, run kppp (you can do this as user). There you will have to configure two things: the account and the modem. By pressing “Configure” you will be presented with a window with four tabs. In the first one you will create a new account, in which the relevant data is:

• Phone number: *99#
• Authentication: PAP/CHAP
• Callback type: none

In the second tab you will configure the modem:

• Modem device: /dev/ttyUSB0
• Flow control: Hardware
• Line termination: CR/LF
• Connection speed: 921600

Please note that those are parameters that work for me. I can not assure that they are the “correct” ones. I have player around with different values, and many times the modem would work all the same with different settings. If you find some error in my setup, please tell me :^)

It is common to hear that recent advances in the Linux desktop, such as [[Compiz Fusion]], are more of a fancy but useless aesthetic contribution to the desktop. While it may be true for many of the CF features, it is no less true that you never know when a given effect will turn out to be useful.

In this post I want to praise the Enhanced Zoom Desktop plugin. It turned out to be of great use for me in the following situation. I wanted to run [[Diablo II]] in my laptop (yes, it runs in Linux, under Wine). The native resolution of the program (640×480 or 800×600) is lower than that of my screen (1280×800), so I have two options: to execute it in windowed mode, or fullscreen. In windowed mode the window occupies less than 2/3 of the 13.3″ screen, wasting space and making it unnecessarily small. Fullscreen mode seems to be better, but it isn’t. Since the width/height ratio is smaller for Diablo than for the screen, the former will be stretched horizontally, distorting the images (everything looks more squat). Fullscreen mode also gave me other problems, like crashing more easily when alt-tabbing.

Here is where the zooming of Compiz Fusion comes in handy. Apart from an arbitrary zoom (using the mouse wheel while pressing the Super key, a.k.a. windows key), there is a handy shortcut (Super+r) that zooms up to the point of the screen under the cursor occupying the whole screen. When zooming, the movement of the mouse makes the zooming “window” to move around, showing different parts of the desktop. To avoid it (clearly unwanted if we want to stay inside the Diablo window), we have another shortcut: Super+l. This shortcut toggles on and off the “zooming lens follows the mouse” movement.

So now, if I want to play Diablo I open it in windowed mode, then put the cursor inside the window, then hit Super+r, then Super+l, and I have a Diablo window as big as possible to fit in my screen, preserving height/width ratio, and keeping the mouse inside the window.

I am a heavy user of the [[Grace (plotting tool)|Xmgrace]] plotting program, and I love it. An operation very ofter used is to scale the X and Y axes to our liking, to show different parts of our data in the resulting plot. You can do that from the command line by setting the “world” of the graph, providing four numbers as X,Y boundaries:

% xmgrace -world xmin ymin xmax ymax file.dat

Apart from setting the maximum and minimum values for X and Y, we can make use of the autoscale option to selectively show some ranges. The four options to autoscale are:

• none – show the X,Y ranges defined by the “world” variable (if not set, the default is “0 0 1 1”).
• xy – forget about “world” data, make plot range in X and Y enough to plot all data in input.
• x – autoscale X to show all data, but respect Y given by “world”. This means that if a point is not shown because it lies outside the Y range, then it doesn’t count to force X autoscale. This is a wee bit trickier than it sounds.
• y – see previous point, with X and Y swapped.

But Xmgrace is not only about [[command-line interface|command line]], or even [[Graphical user interface|GUI]]. You can write a .agr file (for example by saving a plot from the Xmgrace GUI), and manipulate it so that the following command:

% xmgrace file.agr

will bring up a plot with all the data and formatting we have put into the .agr file. It’s really handy to save a file as-is.

Now, the syntax for inputting the world in the .agr is well known:

@ world xmin, ymin, xmax, ymax

where xmin etc. are floating point numbers.

The problem is how to hardcode the autoscale feature into the .agr. I had always been forced to do:

% xmgrace -autoscale none file.agr

from the command line, because I couldn’t find out how to include it in the .agr. Finally I did find it, and that’s the main reason of this post. The syntax is explained in the manual at the Xmrace site, but I found it after googling for agr files containing “autoscale” in them. The line to include seems to be:

@ autoscale onread none

A .agr containing the above line will produce, when called as follows:

% xmgrace file.agr

the same output as a file not containing it, when called as follows:

% xmgrace -autoscale none file.agr

A request from a friend made me face the problem of converting a color [[Portable Document Format|PDF]] into a [[grayscale]] one. Searching the web provided some ways of doing so with [[Adobe Acrobat]], via some obscure menu item somewhere.

However, the very same operation could be undertaken with free tools, such as [[ghostscript]]. I found a way to do it in the YANUB blog, and I will copy-paste it here, with a small modification.

Assuming we have a file called color.pdf, and we want to convert it into grayscale.pdf, we could run the following command (all in a single line, and omitting the “\” line continuation marks):

I prefer the above to YANUB’s version below (in red what he lacks, in blue what I lack), because a shell operation is substituted by some option(s) of the command we are running:

A sample [[Perl]] script to alleviate the tedious writing above:

Assuming we call the Perl script “togray.pl”, and that we have a color file “input.pdf”, we could just issue the command:

and we would get a grayscale version of it, named “input_gray.pdf”.

I recently subscribed to [[simyo]]’s mobile internet service. I was considering also [[Orange (brand)|Orange]], as explained in a previous post (es), but simyo’s offer is better.

I am writing how to make the modem simyo provides (the commonplace [[Huawei E220]]) on MacOSX first, because apparently the [[Personal identification number|PIN]] has to be deactivated for the modem to work in Linux. I have to admit that in MacOSX installation was a breeze.

Software installation

Start MacOS, then plug the USB modem. A window will open automatically, with two objects inside: “MobileConnect” and “User Manual”. The former is the installer binary, and the latter is a folder with the manuals in PDF format (for me, they were in English and Spanish).

Clicking on the “MobileConnect” icon the installer will start, and after being asked to accept an [[Software license agreement|EULA]], then introduce the admin password, then choosing a location for placing the files (actually just a hard disk, not a concrete dir), the installer does its thing.

Profile setting

After that, we only need to configure a connection in the “Mobile Connect” window that opens automatically after installation. For that, click on “Setting…” and create a new profile. If you read the manual (see above), it is easy to fill in the blanks. In short:

• Profile name: whatever you want
• Access Point Name: this is the APN value that simyo tells you in some paper (gprs-service.com)
• Telephone number: *99#
• Account name: irrelevant
• Password: irrelevant

Save the above, then choose the profile you just created in the drop-down list in “Profile name”, then hit the “Connect” button. If after saying “Dialing up, please wait”, it tells you “Connection succesfull!”, then everything is fine!

PIN deactivation

Apparently using the modem under Linux requires that the PIN is deactivated. Doing that under MacOSX is easy: when the “Mobile Connect” window is active, go to the “Manage PIN” drop-down menu in the top bar. There you can find “Activate”, “Deactivate” and “Modify”. Self-explanatory, ain’t it?

I recently updated the [[MediaWiki]] installation in one of my [[DreamHost]] domains from 1.12 to 1.13, and I started to see the following error messages when trying to edit/save pages (the capital letter triplets used for privacy):

Database error

A database query syntax error has occurred. This may indicate a bug in the software. The last attempted database query was:

(SQL query hidden)

from within function “Article:getHiddenCategories”. MySQL returned error “1146: Table XXX.YYY_page_props’ doesn’t exist (mysql.ZZZ.AAA)”.

After a Google search that yielded only two results, I checked a mediawiki.org page talking about the subject. The (maybe obvious) reason for my error was that I hadn’t run the MediaWiki update script, as one should after any upgrade.

The procedures is outlined in this other mediawiki.org page. However there is a little catch: the [[PHP]] version (at least in my case) accessible in the shell of the server where my wiki is is 4.4.8, but the MediaWiki update script needs PHP5. No problem, I checked the DreamHost wiki, and found out that for PHP5 I could use the following executable: /usr/local/dh/cgi-system/php5.cgi.

Running that executable on the corresponding update.php script (after setting up AdminSettings.php as told to), everything was OK again.

I use [[Inkscape]] to do many of the drawings for my articles and talks, and have come across an irritating problem: I could not include [[LaTeX]] formulas on it. I have googled a bit about it, and the first match already led me to a bug report, where a comment by Kees Cook gives a fix that I quote below:

% cd /usr/share/inkscape/extensions
% curl -s 'http://launchpadlibrarian.net/12978623/eqtexsvg.py.patch' | sudo patch -p0

The bug affects (and the patch fixes) Inkscape 0.46 on [[Ubuntu]] Hardy Heron and [[Debian]] Lenny (that I know of).