After several unsuccessful attempts, I thought of using the locale Python module and see if it could help me get a list of locale names that Windows supports. I managed to get a list of locales, but it turned out that those would not be accepted by initdb either.

As a last resort, I decided to dig into MSDN for an answer. My experience with that place is minimal, so it took me a while before familiarizing myself and making progress with my research. But finally, I managed to locate the needed information. So, Windows uses the same pattern for locale names:

language_territory.codeset

But the names for the language and territory parts are different. To get a proper name it is required to combine information from the following three pages:

1. language: Language Strings
2. territory: Country/Region Strings
3. codeset: Code Pages Supported by Windows

So, for instance, if you want to set the “en_US.UTF-8” locale in Windows, you have to use “american_usa“. It is a bit frustrating that locale names are not common among the various operating systems. I needed to do a very simple thing and this has added a lot of overhead. I am very disappointed.

As an example, I post the command line options I finally used with initdb in order to initialize the PostgreSQL database cluster:

initdb.exe --auth=md5 --locale=american_usa --encoding=UTF8 --pgdata=G:\PostgreSQL\data --xlogdir=G:\PostgreSQL\xlogs  -U postgres -W

As a general conclusion, I would say it would be a lot better if we spent our time doing more creative things than trying to match the locale names between various operating systems. I am quite sure that both Linux and Windows use names based on some kind of standard, but why can’t they be based on a single standard?

Locale in Windows, unless otherwise expressly stated, is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.

Started using rdiff-backup on Windows, unless otherwise expressly stated, is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.

Windows 7 Beta is already installed in my triple-boot (CentOS, OpenSolaris, Windows) system, but I hadn’t had enough free time to spend on it during the last weeks. Anyhow, I’ve downloaded the RC version and intend to install it soon. Just a quick tip: although you can upgrade a Windows Vista installation to the Windows 7 RC version, upgrading a Windows 7 Beta system to the RC version is not supported.

Microsoft Releases Windows 7 RC to the Public, unless otherwise expressly stated, is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.

I also installed OpenSolaris 2008.11. I had to make some room for a primary partition as OpenSolaris does not recognize/manage extended or logical partitions (even mentioning LVM is unnecessary). I used the GParted LiveCD for this purpose and all was perfectly done. Due to the fact that OpenSolaris cannot read logical partitions, my CentOS installation was inaccessible from the GRUB menu, but that was something I fixed easily later using CentOS in rescue mode, so that triple booting would be flawless well, not exactly flawless at the moment, but will work on that. Before performing the install, I did some research about the different installation methods (local or across the network), but I was a bit disappointed by the fact that the OpenSolaris installer is not very flexible or user-friendly. It would be a joke to even compare it to the RHEL/CentOS/Fedora installer; anaconda that is.

As usual, my onboard gigabit ethernet card (Attansic L1) was not recognized. I downloaded a 3rd party driver, but it was only after a few hours before I started enjoying network connectivity. At first, I had not realized that the driver distribution also included binaries. I spent most of the time trying to install the gcc compiler in order to compile the driver. What puzzled me the most was the fact that, not only the compiler was not available in the installation medium, but also the fact that there was no way to download the relevant package (SUNWgcc and its dependencies) and manually install it using pkgadd. SUN’s Image Packaging System (IPS) has reached a point that the entity of a package, as we know it, does not exist! This was unbelievable at first… Anyway, some hours later I was told that the driver’s distribution (atge) included binaries that I could use to get my network adapter up. So, that was what I did. I had a few more problems with NWAM, Network Auto Magic (the Opensolaris equivalent of Network Manager), but soon I managed to disable the automatic network configuration and bring the interface up manually.

Believe me, after all the trouble with the network connectivity, I did not want to like OpenSolaris. But the truth is I really did like it. There are some bugs here and there, some parts of the operating system are not user-friendly at all, and also some graphical system administration utilities are not really useful at the moment. But, the more someone uses OpenSolaris the more he gets the impression that there is some serious professionalism behind this project. It is being built in order to be useful and do things right. Also a quick tour around OpenSolaris.org and its relative web-sites reveals that the effort, both SUN’s and the effort of the community, behind this operating system is much more coordinated compared to the development of Linux distributions. So, it was not a surprise for me the fact that Firefox with Adobe’s Flash player performed way better (IMHO) than in Linux. Also, despite the fact that a few months ago there was no Solaris compatible Adobe Reader, now Solaris/OpenSolaris users enjoy the latest version of Adobe Reader (9.1 at the moment of writing), while Linux users still use the 8.X version — note: just checked and Adobe Reader 9.1 is available for Linux too. Well, if Adobe has to create an application that has to run on all the major Linux distributions which use different versions of libraries, the above seems reasonable and we, Linux users, should not complain at all about it. Well, Skype has not been ported yet. Finally, it should be noted that there is plenty of software available for OpenSolaris users from official and third party package repositories.

Hmm, when I was starting writing this document I wouldn’t expect it to be that long. Anyway, time permitting, I’ll test the above operating systems some more in the upcoming weeks. I had tried OpenSolaris on Virtualbox, but it had not performed well in virtualized mode on my computer. Now, that I have installed it on a physical partition things are very different.

Windows 7, OpenSolaris – put to the test, unless otherwise expressly stated, is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.

The Linux desktop’s progress in terms of stability, reliability and performance has been extremely slow. This is not the developers’ fault or the users-testers’ fault. I believe the problem lies in the huge number of linux distributions which result in a tremendous waste of time and effort. Moreover, I believe that people spend way too much time and effort in organising the communities that surround the linux distributions instead of spending time to improve the software, which is the actual goal after all.

So far, I am impressed with windows Vista and with the other software I am checking out. Time will show what I will finally use on my primary desktop.

Checking on Windows Vista, unless otherwise expressly stated, is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.

My primary desktop has been running GNU/Linux exclusively for 4+ years. During that time, I had the opportunity to use every FOSS graphical CD/DVD burning frontend that has ever been released, but I always ended to the good old command-line tools (mkisofs – now genisoimage-, dvd+rw-tools) to do the job.

Making an exception for K3B, I wouldn’t rate all other FOSS graphical CD/DVD burning frontends higher than “crapware“.

Of course, things would have been better if all those FOSS devs joined forces, had a plan and assembled a team of volunteer beta testers (I would have loved to participate). I am sure that the result would have been an awesome burning application. But, unfortunately, each of those selfish geniuses decided to torture the world by delivering his own version of crap to our desktop, dammit!

Burning a DVD still an adventure on GNU/Linux, unless otherwise expressly stated, is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.

VMWare is an incredibly useful piece of software. Problem is, it feels like it’s perched on the tip of an eggshell. Any change to its environment tends to cause it to fall over, isolating all your data in the VM until you can find the time to get it working again. It’s not fun to trust important data to such a brittle environment.

Apart from the concern about the stability of the environment that hosts the guest operating system I can think of 1 or 2 more reasons why someone would want to run VMWare on a pre-existing installation of Windows on a physical volume. The first would be “convenience” and the other would be just “fun“.

Read the entire article »

Howto: Run VMWare on a Physical Windows Partition, unless otherwise expressly stated, is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.

Why I decided to do this is quite simple:

• Refreshing the knowledge of MS Windows and MS Office is the primary goal. Don’t forget that Windows is still the most popular desktop OS worldwide and, sincerely, I do not see how this situation could possibly change in the near future. This is not a flamebait, but rather a strong belief of mine after thorough and objective evaluation of the Linux Desktop and many of the companies/organizations/people surrounding it during the last 30 months. Obviously, this statement does not involve Linux Servers, whose popularity, security and quality are undisputed.
• Before using Linux as a desktop OS, I was using lots of open source software under Windows. The secondary goal is to check how much the quality of that software has increased and how it can be used in order to secure the Windows environment more effectively than in the past.

So, my OEM version of Windows XP Home is currently the second desktop operating system in my PC. [editor's note: Do not underestimate home editions. They become quite powerful when combined with software like cygwin] I have also downloaded a 60-day trial version of MS Office 2007 to check out and a bunch of software from Sourceforge and other sources.

I hope I find enough time to test all these things. Also, time-permitting in the following weeks, I will try to publish 2-3 articles about how one can effectively secure Windows with open-source software. This, of course, is entirely possible and I hope this info will be useful to all those who have to use Microsoft Windows.

Dual-booting again after 30 months, unless otherwise expressly stated, is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.

Theory

Rapidshare uses cookie-based authentication. This means that every time you log into the service, a cookie containing information which identifies you as a registered user is stored in your browser’s cookie cache. Both wget and curl support saving and loading cookies, so before using them to download any files, you should save such a cookie. Having done this, then the only required action in order download from RapidShare is to load the cookie, so that wget or curl can use it to authenticate you on the RapidShare server. This is pretty much the same you would do with a graphical download manager. The difference now is that you do it on the command line.

Below you will find examples about how to perform these actions using both wget and curl.

IMPORTANT: Please note that in order to use these command-line utilities or any other download managers with RapidShare, you will have to check the Direct Downloads option in your account’s options page.

Save your RapidShare Premium Account Cookie

Saving your RapidShare cookie is a procedure that needs to be done once.

The login page is located at:

https://ssl.rapidshare.com/cgi-bin/premiumzone.cgi

The login form requires two fields: login and password. These are pretty self-explanatory.

In the following examples, the RapidShare username is shown as USERNAME and the password as PASSWORD.

Using wget

In order to save your cookie using wget, run the following:

wget \
    --save-cookies ~/.cookies/rapidshare \
    --post-data "login=USERNAME&password=PASSWORD" \
    -O - \
    https://ssl.rapidshare.com/cgi-bin/premiumzone.cgi \
    > /dev/null

–save-cookies : Saves the cookie to a file called rapidshare under the ~/.cookies directory (let’s assume that you store your cookies there)
–post-data : is the POST payload of the request. In other words it contains the data you would enter in the login form.
-O - : downloads the HTML data to the standard output. Since the above command is run only in order to obtain the cookie, this option prints the HTML data to stdout (Standard Output) and then discards it by redirecting stdout to /dev/null. If you don’t do this, wget will save the HTML data in a file called premiumzone.cgi in the current directory. This is just the Rapidshare HTML page, which is absolutely not needed.

Using curl

In order to save your cookie using curl, run the following:

curl \
    --cookie-jar ~/.cookies/rapidshare \
    --data "login=USERNAME&password=PASSWORD" \
    https://ssl.rapidshare.com/cgi-bin/premiumzone.cgi \
    > /dev/null

–cookie-jar : Saves the cookie to a file called rapidshare under the ~/.cookies directory (it has been assumed previously that cookies are stored there)
–data : contains the data you would enter in the login form.
Curl prints the downloaded page data to stdout by default. This is discarded by sending it to /dev/null.

Download files using your RapidShare Premium Account Cookie

Having saved your cookie, downloading files from RapidShare is as easy as telling wget/curl to load the cookie everytime you use them to download a file.

Downloading with wget

In order to download a file with wget, run the following:

wget -c --load-cookies ~/.cookies/rapidshare <URL>

-c : this is used in order to resume downloading of the file if it already exists in the current directory and is incomplete.
–load-cookies : loads your cookie.

Downloading with curl

In the same manner, in order to download a file with curl, run the following:

curl -L -O --cookie ~/.cookies/rapidshare <URL>

-L : Follows all redirections until the final destination page is found. This switch is almost always required as curl won’t follow redirects by default (read about how to check the server http headers with curl).
-O : By using this switch you instruct curl to save the downloaded data to a file in the current directory. The filename of the remote file is used. This switch is also required or else curl will print the data to stdout, which is something you won’t probably like.
–cookie : loads your Rapidshare account’s cookie.

Setting up a Download Server

Although most users would be satisfied with the above, I wouldn’t be surprised if you would want to go a bit further and try to setup a little service for your downloading pleasure. Here is a very primitive implementation of such a service. All you will need is standard command line tools.

This primitive server consists of the following:

1. A named pipe, called “dlbasket“. You will feed the server with URLs through this pipe. Another approach would be to use a listening TCP socket with NetCat.
2. A script, which, among others, contains the main server loop. This loop reads one URL at a time from dlbasket and starts a wget/curl process in order to download the file. If dlbasket is empty, the server should just stay there waiting.

So, in short, the service would be the following:

cat <> dlbasket | ( while ... done )

All credit for the “cat <> dlbasket |” magic goes to Zart, who kindly helped me out at the #fedora IRC channel.

So, let’s create that service. The following assume that a user named “downloader” exists in the system and the home directory is /var/lib/downloader/. Of course you can set this up as you like, but make sure you adjust the following commands and the script’s configuration options accordingly.

First, create the named pipe:

mkfifo -m 0700 /var/lib/downloader/dlbasket

If it does not exist, create a bin directory in the user’s home:

mkdir -p /var/lib/downloader/bin

Also, create a directory where the downloaded files will be saved:

mkdir -p /var/lib/downloader/downloads

The following is a quick and dirty script I wrote which actually implements the service. Save it as rsgetd.sh inside the user’s bin directory:

#! /usr/bin/env bash  
 
#  rsgetd.sh - Download Service
 
#  Version 0.3
 
#  Copyright (C) 2007 George Notaras (http://www.g-loaded.eu/)
#
#  This program is free software; you can redistribute it and/or modify
#  it under the terms of the GNU General Public License version 2 as
#  published by the Free Software Foundation.
#
#  This program is distributed in the hope that it will be useful,
#  but WITHOUT ANY WARRANTY; without even the implied warranty of
#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#  GNU General Public License for more details.
 
# Special thanks to 'Zart' from the #fedora channel on FreeNode
 
# v0.3 - Oct 31 2009:
# - corrected error: "./bin/rsgetd.sh: line 37: [: argument expected"
 
# CONFIG START 
HOMEDIR="/var/lib/downloader"
DLBASKET="$HOMEDIR/dlbasket"
DLDIR="$HOMEDIR/downloads/"
LOGFILE="$HOMEDIR/.downloads_log"
CACHEFILE="$HOMEDIR/.downloads_cache"
LIMIT="25k"
WGETBIN="/usr/bin/wget"
# Rapidshare Login Cookie  
RSCOOKIE="$HOMEDIR/cookies/.rapidshare"
# CONFIG END
 
DATETIME="`date '+%Y-%m-%d %H:%M:%S'`"
 
cat <> $DLBASKET | (
        while read url ; do
                # First, check the cache if the file has been already downloaded
                if [ -f "$CACHEFILE" -a -n "$(grep -i $(basename $url) $CACHEFILE)" ] ; then
                       echo "$DATETIME File exists in cache. Already downloaded - Skipping: $url" >> $LOGFILE
                else
                        echo "$DATETIME Starting with rate $LIMIT/s: $url" >> $LOGFILE
                        if [ $(expr match "$url" '[rapidshare.com]') = 1 ] ; then
                                # If it is a Rapidshare.com link, load the RS cookie 
                                echo "RAPIDSHARE LINK"
                                $WGETBIN -c --limit-rate=$LIMIT --directory-prefix=$DLDIR --load-cookies $RSCOOKIE $url
                        else
                                $WGETBIN -c --limit-rate=$LIMIT --directory-prefix=$DLDIR $url
                        fi
                        echo "$DATETIME Finished: $url" >> $LOGFILE
                        echo $url >> $CACHEFILE
                fi
        done )
 
exit 0

As you might have already noticed, two extra files are created inside the home directory: .downloads_cache and .downloads_log. The first contains a list of all the urls that have been downloaded. Each new download is checked against this list, so that the particular URL is not processed if the file has already been downloaded. The latter file is a usual logfile stating the start and end times of each download. Feel free to adjust the script to your needs.

Here is some info about how you should start the service:

-1- You can simply start the script as a background process and then feed URLs to it. For example:

rsgetd.sh &
echo "<URL>" > /var/lib/downloader/dlbasket

-2- Use screen in order to run the script in the background but still be able to see its output by connecting to a screen session. Although this is not a screen howto, here is an example:

Create a new screen session and attach to it:

screen -S rs_downloads

While being in the session, run rsgetd.sh

rsgetd.sh

From another terminal feed the download basket (dlbasket) with urls:

echo "<URL>" > /var/lib/downloader/dlbasket
cat url_list.txt > /var/lib/downloader/dlbasket 

Watch the files in the screen window as they are being downloaded.

Detach from the screen session by hitting the following:

Ctrl-a   d

Re-attach to the session by running:

screen -r

Note that you do not need to be attached to the screen session in order to add URLs.

Feeding the basket with URLs remotely

Assuming that a SSH server is running on the machine that runs rsgetd.sh, you can feed URLs to it by running the following from a remote machine:

ssh downloader@server.example.org cat \> /var/lib/downloader/dlbasket

Note that the > needs to be escaped so that it is considered as part of the command that will be executed on the remote server.

Now, feel free to add as many URLs as you like. After you hit the [Enter] key the url will be added to the download queue. When you are finished, just press Ctrl-D to end the URL submission.

Conclusion

This article provides all the information you need in order to use wget or curl to download files from your RapidShare Premium account. Also, information on how to set up a service that will assist you in order to commence downloads on your home server from a remote location has been covered.

The same information applies in all cases that wget and curl need to be used with websites that use cookie-based authentication.

Use wget or curl to download from RapidShare Premium, unless otherwise expressly stated, is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.

The Windows port of mkisofs and, generally, the Win32 binaries of all the utilities included in the cdrtools package can be found in the Cdrtools: Win32 Binaries page. Please note that the page also includes a Win32 binary version of the latest (unstable) version of cdrtools. It is highly recommended to use the latest stable version.

So, change to the directory that contains the Windows installation files and issue the following command:

mkisofs \
    -b cdboot/msboot.img -no-emul-boot -boot-load-seg 1984 -boot-load-size 4 \
    -iso-level 2 -J -l -D -N -joliet-long -relaxed-filenames \
    -V "WINSP" \
    -o ../winsp.iso .

The above command ends with a dot (.).

All of the above flags seem to be necessary in order to relax many of the ISO restrictions. For more information about each of them please refer to the mkisofs (or genisoimage) manual page (man 8 mkisofs).

This tip assumes that the boot image which is used in the Microsoft bootable CDs is available and that it has been placed in a cdboot/ subdirectory under the windows installation files root dir:

.
    cdboot/msboot.img
    other_dir/some_other_file
    file1
    file2

I admit that this is a nasty workaround, but I didn’t have enough time to investigate why the -b option caused that much trouble. The path to the boot image which is supplied with the -b option is supposed to be a relative path to the path of the root directory of the windows files, which is our current directory, aka (.). But, everytime I placed the msboot.img outside the win files directory and used -b ../msboot.img, mkisofs complained with the following pesky error message:

mkisofs: Uh oh, I cant find the boot image '../msboot.img' !

It didn’t make any sense, so I did not bother any more with it. If you have any info on this or if I miss anything, which is rather possible since I have spent very little time with this, your feedback is appreciated.

The above command will produce an image of a bootable Windows CD, which will also contain the boot image. But, this should not be a problem in any case.

Note that the ISO image will be saved in the parent directory of the one that holds the Windows files.

Note: How to integrate a service pack into a Windows OS is beyond the scope of this document. There are numerous guides out there that cover this topic in great detail.

How to create a Windows bootable CD with mkisofs, unless otherwise expressly stated, is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.