handyfloss

After another boring month, I have little to write about, besides my LWD project data update. You can read this May 2008 post for an intro.

As usual D2D means “days to domination” (the expected time for Windows/Linux shares to cross, counting from Feb 3, 2008), and DD2D means difference (increase/decrease) in D2D, with respect to last report (around a month ago).

Except for SETI@home, all the D2Ds have gone up, showing that maybe the predictions so far were too optimistic. On the bright side (for FLOSS), SETI is, by far, the project with most users, so its results are the most reliable.

Recall that confidence percents are below 10%, except in two cases, which means logged data extent is small, compared to prediction time. This periodic report, thus, will get more and more accurate as months pass by.

No Comments yet »

This is a (in principle, monthly) update to my “Linux World Domination” project. You can read the intro in this May 2008 post.

The data presented is different from the one in the aforementioned post:

• Mac is dropped from it
• Predictor@home is also dropped
• Two projects have been added: POEM and Spinhenge
• D2D means “days to domination”. The expected time for Windows/Linux shares to cross, counting from Feb 3, 2008.
• DD2D means difference (increase/decrease) in D2D, with respect to last report (a month ago)

Except for QMC@home, all the projects have reduced the D2D. Rosetta and Einstein were expected to never lead to LWD, and now they are.

See you next month!

No Comments yet »

Pedazo de noticia que he leído vía EmpresaDigitala.net. Según esta, Microsoft apuesta por la seguridad para diferenciarse del software libre.

¡Oh, sorpresa! MS, la empresa que produce el ubícuo sistema operativo Windows, tan conocido por su invulnerabilidad ante ataques de software malicioso y su práctica ausencia de bugs y errores, ha decidido seguir por el camino de la perfección que la caracteriza, para diferenciarse del software libre, que tantos agujeros de seguridad sabemos que tiene, y cuyos bugs sabemos que se tardan tanto en solucionar. ¡Menos mal! Estoy ya harto de los virus y troyanos en mi Debian. Es que cada vez que me mandan un e-mail tengo miedo de abrirlo, no sea que se me infecte el ordenador. Además el antivirus que tengo todo el tiempo corriendo en segundo plano me come muchos recursos, y no puedo trabajar a gusto. ¡Ojalá pueda pasarme a Windows, y olvidarme de todo este rollo del malware!

¡Venga ya!

3 Comments »

Remember Project BHS? It is an effort I am making to log the evolution of Windows/Linux/Mac/Other market share, via the respective contributions to BOINC projects.

I have taken a further (and very crude) step towards the estimation of when will the Beast from Redmond fall, by extrapolating the “Number of hosts vs. time” curves to the points of crossing. For that I have fitted the data so far to (very crude, I know) second order polynomials (with Xmgrace), and calculated the crossing points (with GNU Octave).

The results can be:

1. Windows seems to go upwards and Linux/Mac downwards (will never cross)
2. The crossing point is above 100% or below 0% market share: the extrapolation is unfit (will never cross)
3. There is a crossing point and lies within 0-100% market share: that’s the World Domintion date!!

I will be posting data for different projects, along with a “confidence” percent. This value corresponds to the fraction of the total time required for Linux/Mac to overcome Windows (according to the present tendency) that is represented in the collected data. If 10-day data suggests that Linux will overcome Windows in 1000 days, then the result is not really very trustable. OTOH, 999-day data suggesting the same is compelling.

An important notice: expected times are not measured from “now”, but from the moment I started collecting data, on Feb 3, 2008 (3 months ago).

The following table illustrates the aforementioned data for some selected projects, with time in days and confidence percent in parenthesis.

As an example, the curve fits and corresponding crossing points are given in the following figure, for the case of SETI@Home. You can infer the limited trustability of the predictions from the tiny time extent of the data points used to extrapolate the curves. As time goes by, curves will be more and more trustable, so expect updates to this “project”.

SETI@Home data (click to enlarge)

The software used to process the data is BHS, and can be found at my home page.

2 Comments »

I regularly check the visit stats at this blog, and today is the first time that the browser with most accumulated visits is Firefox and not Internet Explorer. IE began ahead because some Chinese hacker(s) used the Windows XP/IE6 duo to try some nasty things at the site, and generated a lot of visits. After I banned these IPs, the visits from Firefox users have gone up steadily, and now the total visit stats are:

1. Firefox 2 (33.5%)
2. Internet Explorer 6 (32.6%)
3. Firefox 1.0 (10.4%)
4. IE 7 (4%)

Regarding OSes, Windows XP is still the most prominent one, with 53% of the visits, followed by generic Linux at 12.9%, then other versions of Windows and specific Linux distros.

No Comments yet »

As outlined in some previous posts[1,2,3,4], I have been playing around with a piece of Python code to process some log files. The log files to process were actually host.gz files from some BOINC projects, and the data I want to extract from them is quite simple: the Windows, Linux and Mac shares in the number of computers contributing to them (and the work they do). By logging this processed data myself, I can see the time evolution of this share, and hopefully show the slow but steady rise of GNU/Linux :^)

I figured out that the contribution to distributed computing projects could be a reasonable indicator of the Windows predominance status. There are many other indicators (for example the number of visits to a web site, e.g. this very one), and I don’t claim that this one is “better”. I just want to add it to the reference list for the reader.

There is a problem with “Windows vs. Linux” figures, and it is that they are not really “competing” products. When cars or soft drinks are the subject, one can figure out the market share, looking at the number of items sold. Linux being free software, one can hardly measure the amount of “sold copies”, and with Windows being pre-installed in most new computers, one can not really trust the “number of computers sold = number of Windows copies sold”, because some users even remove the Windows partition and install Linux on top of it.

Counting the visits to some sites is not without problems, either. Any web site will have a particular audience, and the result will be biased by that fact. When my blog was in WordPress.com, I had roughly as many visits from Windows users as from Linux users, and almost all of them used Firefox as a browser. Obviously this data is not an accurate reflection of the world at large. It so happened that free software users are more likely to surf to sites like mine, hence the bias.

So, without further ado, let me introduce the “BOINC Host Statistics” program (BHS). Here you are a link to its home page. You can find results I have harvested so far in the Screenshots section. For example, the SETI@home credit generation rate statistics follows:

What the plot tells us is that (at the time of writing this) 500 million cobblestones are being granted to contributors each day. Of them, around 82% are being given to Windows computers, 9-10% to Mac, 8% to GNU/Linux, and the rest to computers running other OSs.

No Comments yet »

You would think that, as a researcher in a serious center like the DIPC, one would hardly ever encounter a MS product, at least in the server/cluster section (more than one fellow here has Windows in his/her computer, but don’t tell anyone, it’s a secret).

However, we do have some server running Windows, and its presence is almost transparent for the user (which is good). And I say “almost”, because it stumbled upon one of its “features”, and the sysadmin ended up confessing :^)

The thing is that I happened to try to create a directory with “CO” (carbon monoxide) in its name (it was a dir for a SIESTA calculation), when a dir with the same name already existed, except it had “Co” (cobalt) in the name. Well, the filesystem complained that a dir by the same name already existed! I could not believe my eyes!!

Basically the filesystem would not make any difference at all for different capitalizations. For example:

<pre>
% mkdir testdir
% mkdir TESTDIR
mkdir: cannot create directory `TESTDIR': File exists

% touch testfile
% rm TESTfile
rm: remove regular empty file `TESTfile'?
</pre>

The explanation? The directory I was in was exported from a NAS running… ta-da: Windows 2000 Server!

How incredibly stupid and annoying is it to have a filesystem that ignores character case altogether? And how error prone? Because if you are not aware of that, you might delete a file you didn’t intend to!! Someone could try to excuse MS by saying that, OK, that was in 2000. But, look, Linux could tell upper case from lower case since its inception in 1991, and Unix since the seventies! The root of the problem is the filesystem used by the OS, of course. But it so happens that the filesystems used by Linux since 1991 (beginning in ext and then many others) had this capability (and many more), and are free. All that MS had to do was to use them, instead of FAT or NTFS. But instead they chose to develop those (inferior) filesystems in parallel for almost 20 years now. I call that stupidity.

No need to say that the sysadmin of the DIPC absolutely regrets having been naive enough to ever buy that MS crap.

No Comments yet »

Intro

Any GNU/Linux user ends up hearing, sooner or later, the Ultimate Argument(tm) from a Windows fanboy:

If Linux is so good, and is given away for free, how can Windows still be so prominent?

Ironically, many Windows users answer the question themselves, when explaining why they can’t make the switch to Linux:

• There are no games for Linux
• Photoshop or AutoCAD are vital for me, and they only work under Windows
• I fear some pieces of hardware won’t work under Linux
• The web page of my bank only displays correctly under IE
• My friends/colleagues/business partners share documents in MS Office formats, and I need to be compatible

The concept than embraces all the preceding points, and answers the rhetorical question above, is vendor lock-in. I will try to explain the concept with a humble tale I have used twice so far in comments to entries in Enrique Dans’s blog.

Tales of bicycles and cars

Imagine a country with no bicycles.

One day a guy comes up with the idea of making them, and starts to produce, and sell, bicycles that we shall call of type A. Being an empty market, the A-type bicycles quickly triumph, and the maker makes a lot of money.

But some time later, a second guy devises a better bike design (type B), and decides to produce and sell it. The price and the quality are better, so when people buy a new bike or replace an old one, they tend to buy bikes of type B. Soon enough, the market is dominated by the new, better, bicycle.

Now imagine a country with no cars.

One day a guy comes up with the idea of making cars, and starts to produce and sell cars of type A. As cars need petrol to run, A-type gas stations develop in parallel to car sales. Building gas stations is expensive, but sales are guaranteed, as everyone has or will have A-type cars, and they need A-type petrol: their growth is synergistic.

But some time later, a second guy devises a better car design (type B) and decides to produce and sell it. The price and the quality are better, BUT drivers can not buy B-type cars, because there is no B-type gas station. The problem is that, since noone has a B-type car, making B-type gas stations is doomed to bankrupt. So, no B-type cars are sold, because there are no B-type gas stations, and B-type gas stations will not be made until B-type cars are popular!

The result is a vendor lock-in.

Consequences

When a market (such as the one in the tale above) is dominated by vendor lock-ins, the producers benefited by the lock-in have little, if any, incentive to make better products. Their sales are guaranteed by the lock-in situation, not by their superior product in a fair market.

On the other hand, other producers will have an extremely hard time for competing, as their products will be almost unusable for the buyers.

The moral is that the lock-in situation is bad for either the potential users of the product locked-out (the B-type car above) and the locked-in one (the A-type car). Even if a consumer would never choose the locked-out product, the lack of competition will adversely affect the evolution of the product they do choose. The lock-in is bad for everyone: all consumers and all producers but the locker ones.

And this relates to Windows vs. Linux in what way?

In a really straightforward way. Microsoft, cunning as they are, have tried their best to get as many lock-ins on the software market as they can. Ironically, instead of abhorring this practice, most Windows users happily continue not only using, but even defending the product. I shudder at the simpleton comment that “freedom is not using Linux, but using Windows and Linux whenever you feel like it”. Literally taken, it is a very wise argument. But unfortunately the reality is not so simple: using Windows helps enforce a lock-in that keeps Linux out (while in this case the contrary is not true). You can not use Windows/Linux 50/50, because Windows asks you for monogamy.

The many lock-ins that MS has forced down the throats of the users, while the latter still claim them to be benefits of Windows include (as mentioned at the beginning):

• Proprietary communication protocols that will not work with any other OS. This includes modifications on the IE web browser, so that web pages had to be done for it, and then be incompatible with other browsers. Or the MSN protocol, that is kept as closed as possible, to make free clones of the MSN client as little compatible as possible.
• Proprietary file formats that will not be possible to modify with tools other than the “official” MS ones: WMV for video, DOC, XLS, PPT for office documents.
• As much “Windows-only” software as possible, including games. Making games for platforms other than Windows ensues the wrath of MS, something that game makers can not take lightly, since their sales depend on the game actually running under Windows.
• As much “Windows-only” hardware as possible. The first idea that someone gets about an OS is that it is the piece of software that interacts with the hardware. If so, it is astonishing why it is not MS the one incorporating the drivers in the OS, instead of the hardware makers (hardware != software) providing them. We have all grown accustomed to buying printers, mice, external CD/DVD/HDs… with a CD with the “Drivers for Windows 98″ or some such. Why? Windows can not make the drivers out of the blue, true. But the hardware makers can just make the necessary data public, so anyone will be able to make drivers. If the maker keeps these specifications secret, they will simply not sell anything.

2 Comments »

One argument I tend to hear from Windows users is that in Windows you can do as much as you can with Linux, and that the technical advantages of Linux only show up if you are really an utter geek. This is one of (I hope) a series of entries in my blog, illustrating some cases where this doesn’t hold: I took advantage of tools provided by Linux in a way that anyone could have, not just geeks.

The moral of it all is that Windows encourages a lack of choice and flexibility that makes users tend not to be creative, and think the cage Windows keeps them in is actually a shelter from the storm, when it’s not. They think that what can’t be done with Windows, needs not be done. I think otherwise…

Today I will try to provide an example in which hard links can be useful. Under Windows XP hard links can be created, using the fsutil utility, but only for NTFS file systems, and only by the Administrator account (and only from the command line). If you want to learn more about links and specially Windows links, read this interesting sell-shocked.org article.

The problem

I download a lot of music from Jamendo, using the BitTorrent p2p protocol. After having downloaded a given album, I tend to leave the torrent open, so that people can continue uploading from my computer.

However, I also want to have my music collection tidy and ordered, so I immediately organize the newly-dowloaded songs moving them to a neat directory tree I have, will all my music.

So, there is a conflict between keeping the files in the bittorrent download/upload dir, and properly organizing them. I don’t want to have to wait until I decide to stop sharing a file to organize it, and I don’t want to risk deleting the files if I remove them from the bittorrent client before saving them elsewhere. I could get over all this by simply making a copy of the files… but then I would be filling twice as much disk space, and with GBs of shared files, this is not neat at all.

The solution

What I do is hardlink all the downloaded files to their final location. If I download all torrents to /scratch/ktorrent/, a downloaded album will look like that:

% ls /scratch/ktorrent/album1/
song1.ogg song2.ogg song3.ogg [...]

If I want to save the album under my artist1 directory, I do the following:

% mkdir /scratch/music/artist1/album1
% ln /scratch/ktorrent/album1/* /scratch/music/artist1/album1/

This way all the “song*.ogg” files will appear to be in both /scratch/music/artist1/album1/ and /scratch/ktorrent/album1/ at the same time.

Benefits:

1 - I can keep sharing the files in /scratch/ktorrent/album1/, while listening to and/or manipulating the /scratch/music/artist1/album1/ files as if I had 2 copies of each.

2 - The total size is not affected. The hard links do not “occupy” space (only a few bytes each).

3 - I can delete the files in the shared directory without any fear. Only the “copy” in /scratch/ktorrent/ disappears, while the other “copy” in /scratch/music/artist1/album1/ becomes the only copy (just as if it had always been a “normal” file, and the only one).

Recall that all files are hard links. Normally a given file is the only hard link to a given piece of data in the hard disk, but there can be more “links” pointing to that data. When we remove files, we only remove the “link” pointing to the data.

No Comments yet »

The setup

A colleague wanted to edit a video (actually, three) for a presentation she intended to make in a laptop with Windows Vista and MS Office 2007. The video was a WMV, and the required edition included removing fragments, changing the speed of one of the fragments (and join it back with the others), and adding a soundtrack.

The problem

She could not, for the life of her, edit the damned thing on a Windows computer.

The solution

Why, Free Software, of course.

The motivation of this post

To help dispell two ideas: that “Windows is easy”, and that “With Linux, you waste your time finding out how to do things”.

The story

Part I - Linux

OK, so I proposed her to use some Free Software called Avidemux. Our first problem was that apparently Avidemux was unable to cut the video in pieces (it crashed at the attempt). After much perusing, and using the humble file command, I found out the reason: the WMV had no playing FPS set. Players, like MPlayer would reproduce it by guessing 25 or 30 frames per second, but editors need a precise value to count on. I readily fixed it by reencoding the video to 25fps with MEncoder:

% mencoder in.wmv -ovc lavc -nosound -fps 25 -lavcopts vcodec=wmv1 -o out.avi

Once a proper FPS given, I used Avidemux to split the file. However, I encountered a second problem: I couldn’t split the file anywhere. I could only cut it at points 10 seconds appart. I had to sweat a bit more to fix that, but I also learned something more in the way. Most (all?) compressed video formats use at least two kinds of frames: normal frames and keyframes. The latter are the frames where any player can seek to in the video. According to the man page of MEncoder:

keyint
maximum interval between keyframes in frames (default: 250 or one keyframe every ten seconds in a 25fps movie. [...] Keyframes are also needed for seeking, as seeking is only possible to a keyframe - but keyframes need more space than other frames, so larger numbers here mean slightly smaller files but less precise seeking. 0 is equivalent to 1, which makes every frame a keyframe. [...]

So here you are: the problem was the default value of some variable called keyint. To make the video seekable to any frame (so it could be cut at any point), I set keyint to 1:

% mencoder in.wmv -ovc lavc -nosound -fps 25 -lavcopts vcodec=wmv1:keyint=1 -o out.avi

Once the movie was split into parts with Avidemux, and the unwanted parts were removed, the next step consisted on playing one fragment faster. The problem here is that I don’t know how to make a variable FPS video, so we had to make it so all the video played at the same FPS, but a part was faster. How? Removing frames, of course. I used MPlayer to deconstruct the relevant fragment into individual frames (in PNG format):

% mplayer -vo png:z=2 fragment

The command above generates a whole lot of 0000xxxxx.png files, with frames ordered by the number in the filename. Next, I deleted every second frame. How? With a stupid GUI I don’t know, but from the command line it is trivial:

% rm -f 00*[13579].png

Now, I just re-constructed the video with half the frames, to get an effectively double-speed video, with same FPS as original:

% mencoder “mf://*.png” -mf fps=25 -o output.avi -ovc lavc -lavcopts vcodec=wmv1

If I am allowed to say it, the effect is really great. You wouldn’t tell the sped-up video from the original, except from the increased play speed.

Using Avidemux for joining the video fragments was a breeze, and it could even be done from the command line:

% avidemux fragment1.avi –append fragment2.avi –append fragment3.avi –save total.avi –quit

The last (Linux) part consisted on adding a soundtrack, which Avidemux can do, from a MP3, WAV, or another video. This was easy.

Part II - Windows Vista

OK, the last Linux step consisted on reencoding the video in some format that Vista could read. This was no immediate task, but after some tests, we made it. Windows Media Player could reproduce the movie with no problem.

Finally, we opened the wonderful Office 2007 in the shiny and new Vista laptop, and created a PowerPoint slide to insert the video (the rest of the presentation was already done). Everything seemed to work, but when we played the presentation, we discovered that either the video or the sound could be played (depending on how we had encoded the video in Linux), but not both simultaneously. WMP would play the videos just fine, but the embedded player in PowerPoint would not… go figure why. After at least 3 crashes of Office (yes, Office crashes), some bitching because we could not make any sense of the new Office interface (we are experienced pre-Vista and Linux users, and Windows is for idiots, right? We must be idioter than average) having to stand the fact that the semitransparent border of a window refused to disappear when we closed it (so we kept working with a blue-greenish stripe across a part of the desktop), and one Windows reboot (yes, Vista still hangs from time to time), we managed to insert and play the darned video. How? We just inserted two videos: one for which only the audio was playing, and another one for which only the image was showing. We then make these two objects to kick off at the same time, et voila!. Not the cleanest of solutions, but with Windows “everything just works”, right?

The moral

The moral of the first step (the FPS not being set) was that I had to play around for a while with my Linux tools, but the culprit was MS, and their lousy WMV. I have never produced a video with no FPS (and all other necessary metadata) set, because my FLOSS tools do it automatically. Secondly, I didn’t waste my time. Thanks to the usefulness of the FLOSS tools, I ended up learning something about movies, FPSs, and that they are required. I also learned about key frames, and seeking and cuting video streams.

On the other hand, for a much simpler job, we spent relatively (and maybe absolutely) longer with Windows, and we did lose our time with it. The problems we encountered with Linux were difficulties of the situation itself: the original WMV was flawed, the AVI we created had too high an inter-keyframe interval… and the FLOSS tools we used helped us fix them and learn in the process. In the case of Windows, the task was so simple, and all the problems we met were created by Vista. We didn’t learn anything from all of our struggle, because we only struggled against Windows (the GUI, the crashes, how to encode the video in Linux so that Vista could read it, why the darned Office would not play the audio or the video), not our problem (editing and embedding the file). All the time was devoted to learning how to overcome the limitations and errors of our tool, not to how to use our tool to perform some task, learning about the task itself in the process. Thus, it was wasted time.

No Comments yet »