I think everyone who is interested in collaborative content should be happy about Creole. This is for the benefit of both the projects and their contributors because a standard wiki markup language promotes productivity. I think the language still needs more official additions, but I guess this is just a matter of time.

I got the impression that the Creole Project deserves more popularity, so I wrote this small post for all of you who might have never heard of it. It would be nice if you did the same!

Related posts:

1. Zim – a Desktop Wiki
2. Issue addressed: Author feeds deliver full content
3. Fast-Static-Feed WordPress Plugin

From that point on, I tried several utilities. The only program that could still read the exact size of the NTFS partition was gparted (through ntfsprogs). After several hours, having tried Testdisk, ntfsprogs and the Windows Recovery Console tools and being extremely careful so that I did not touch the contents of the partition (my data!!!) in any way, just before trying PhotoRec as a last resort for raw file recovery, a deep partition scan with Testdisk revealed the NTFS partition. The size was wrong, but all files were listed properly. I can tell you that I did not miss the opportunity; I mounted a SAMBA share and copied everything there.

Fortunately, data is safe now. I spent a week checking it and cleaning it up and everything is there.

There are a couple of things I’d like to say to everyone:

1. Never use utilities that can destroy critical parts of your partitions by instinct. RTFM!
2. Never mess with the partition table or the partition’s boot sector etc early in the morning. I am serious! There are numerous great things you can do in the morning! Stick to those great things, unless it is your daily routine to recover data or recover boot sectors and unless you always know what you are doing.
3. Backup data regularly. Your desktop computers and servers are equally important. Schedule daily backups of critical data. There are many great open source backup tools, like rdiff-backup, rsync, unison, BackupPC to name a few. Also Windows includes a backup tool that is very convenient to setup and use. There is no excuse for not doing frequent data backups.

They say that it is the idiots who learn from their own mistakes. Smart people learn from the mistakes of others. Stick to that principal.

Related posts:

1. Choosing a format for data backups – tar vs cpio
2. More Data Recovery Tools
3. Effective data wiping with a single complete overwrite

[Update: It has been resolved. Update you SA rules]

To inactivate the FH_DATE_PAST_20XX test system-wide add the following line to the Spamassassin configuration file, usually located at /etc/mail/spamassassin/local.cf:

score FH_DATE_PAST_20XX 0

If you use the spamassassin server (spamd), don’t forget to restart it for the changes to take effect.

As I have mentioned in the update above, the issue has been resolved (Kudos). The Spamassassin developers have released updated rules for the FH_DATE_PAST_20XX test, so make sure you run sa-update:

/usr/bin/sa-update

If you compile your rules, make sure you also run /usr/bin/sa-compile and of course restart spamd.

Related posts:

1. Windows 7, OpenSolaris – put to the test
2. Fedora Core 5 Test 2 Review
3. A real-time log viewer

No related posts.

For instance, my virtual CentOS server GRUB configuration has the following entry:

title CentOS (2.6.18-164.6.1.el5)
    root (hd0,0)
    kernel /vmlinuz-2.6.18-164.6.1.el5 ro root=/dev/VGSERVER/LVROOT divider=10 clocksource=acpi_pm
    initrd /initrd-2.6.18-164.6.1.el5.img

The divider option adjusts the kernel’s internal clock by dividing its default frequency by 10. So, my kernel is now clocked at: 1000Hz / 10 = 100Hz. Well, that’s not optimal, but suits my current needs. Until CentOS/RHEL 5.3, this option also served as a workaround for time synchronization between the host and the guest OS. For CentOS 5.4 and later, we just need the “divider” in order to underclock the kernel without actually recompiling it. Also, the “clocksource” option is not required any more.

You can find information about whether the divider option is supported by your distribution by checking the Timekeeping best practices for Linux guests article on the VMware knowledge base.

But, make no mistake. All the above are just a workaround. The kernel configuration includes the option to configure it as “tickless” (CONFIG_TICK_ONESHOT & CONFIG_NO_HZ):

Processor type and features ---> [*] Tickless System (Dynamic Ticks)

From the kernel docs:

This option enables a tickless system: timer interrupts only trigger on an as-needed basis both when the system is busy and when the system is idle.

This sounds like a Real Solution™. I tried to build a minimal kernel (version 2.6.32), using the “tickless” feature, specifically for Virtualbox, but, after spending many hours with it, I could not make it recognize my root LVM volumes. So, being extremely frustrated, I abandoned the whole project and used the divider option as I mentioned above.

If anyone has built a minimal custom kernel specifically for Virtualbox or VMware, which supports LVM/RAID and can actually boot a root LVM system, please let me know.

Related posts:

1. Howto: Run VMWare on a Physical Windows Partition
2. Install OpenSolaris in VirtualBox – Screencast
3. CentOS, Debian, FreeBSD, OpenSolaris

This is the Corporate World. You cannot just rely on good faith and logical assumptions. Commitment is necessary. During all these years, many people have sacrificed part of their lives in order to contribute to the development of MySQL and make it the great open source project that it is today. Despite the fact that software licenses could not practically provide full protection against the rules of the Market, I am very glad that there are still other mechanisms that can fill this gap and prevent the outcome of so much collaborative effort from going up into smoke.

Related posts:

1. Modifying Your Name In The WordPress Comments
2. How to fix the WordPress database’s character-set issue
3. Redmine

1. security
2. reliability
3. backwards compatibility
4. extensibility

A few years ago, I had published an approach on how to organize mailing list messages using Evolution. The same tips and tricks apply to Thunderbird, so those of you who deal with many mailing lists might find that old article interesting.

The highlights of the new release include:

• Tabbed Email: Provides the ability view individual emails and folders in tabs.
• Filtered Search: The new search interface contains filtering and timeline tools
• Message Archive: When an email message is archived, it is moved from the inbox into the new archive folder system, de-cluttering the inbox, while, at the same time, enabling users to find email months or years from now.
• One-click Address Book: A very quick and easy way to add people to an address book, by simply clicking on the star icon in the messages received from new correspondents.
• New Mail Account Setup Wizard: Getting started with Thunderbird 3 is faster than ever with the new account set-up wizard that requires simple information, like email addresses and passwords to get going instead of a user’s IMAP, SMTP, SSL/TLS settings.
• Smart Folders: Combines individual mailboxes to help manage multiple email accounts in one spot. For example, by selecting the Inbox, users can view all the incoming emails from all their different accounts without having to go to each email account separately.
• Add-ons Manager: The new add-ons manager can help users find, download, and install hundreds of add-ons enabling them to customize Thunderbird 3 and add functionality or change the appearance.
• Gecko 1.9.1 Engine: The same Web page rendering engine and graphics infrastructure used in Firefox, provides the latest Web Standards support and security enhancements.
• Automated Updates: Thunderbird’s update system notifies users when a security update is available and automates the download and installation process to keep users safe.

Now, I am off to test this beast’s new features…

Related posts:

1. Organizing Mailing List messages with Evolution
2. Application Testing: Zero Free Space
3. Email Notifications from a Linux System

The second major change is that all feeds, except the RSS v0.92 feed, are now redirected to the my RSS v2.0 feed. No more Atom feed. No more RDF (RSS v1.0). As I had written in the past, despite my work on WordPress’ old Atom feed, RSS is my preferred feed type. There is nothing wrong with the other formats. I just do not need them. They add nothing to my website, but cause me trouble when I have to do any redirections or other manipulation of the syndicated content. I still consider the Atom format more advanced than RSS 2, but, since RSS was my choice for this blog, I find no reason to change it now.

I understand that redirecting the Atom or RDF feeds to the RSS one might cause some trouble to some feed readers (software). I apologize in advance for any such inconvenience, but this was a necessary change.

The following diagram shows the redirections that take place when any of the website’s feeds is requested:

Feed redirections on G-Loaded.eu

Some remarks:

• No matter which of the RSS2, Atom or RDF format you request, you always get the full content in RSS 2.0 format.
• If you just want a simple notification about the latest posts, you can select the RSS v0.92 feed. This feed delivers a very small summary of the content.
• There is a special exception for the Planet Feed Reader, which always gets the Atom feed. The amount of content this feed delivers is variable and is determined by the author of the post (me in this case). This is done using a custom plugin.

Getting the content

Here are the links to the G-Loaded Feeds:

• RSS 2.0 (full content)
• RSS 0.92 (summaries)

Please feel free to ask your questions or report any issues.

Related posts:

1. Status of content availability via feeds
2. Issue addressed: Author feeds deliver full content
3. And the winner is: RSS 2.0

[This post has been updated]

[Update #1]: This limitation also exists on the ping command, which runs with setuid access rights. (Thanks Stephane)

[Update #2]: After Chris Hallman reported that the original code actually works on Windows only, I made some changes to it and publish the updated code below.

The following code is a pure Python implementation of the ping command. I originally found it in the source code tree of pylucid in a subdirectory where the developers keep various code snippets.

I had originally tested the code under Microsoft Windows as this was the OS I had available at the moment. Chris Hallman noticed that the code does not work under GNU/Linux. After spending some hours trying to figure out what was wrong it, I realized that the part of the code that caused the failure on Linux was the use of the time.clock() function instead of time.time(). The clock() method works differently under Windows and Linux.

I fixed the 2007 implementation and hereby publish the updated code(as python-ping). Feel free to test it and report any issues.

Needed: test on Solaris.

#!/usr/bin/env python
 
"""
    A pure python ping implementation using raw socket.
 
 
    Note that ICMP messages can only be sent from processes running as root.
 
 
    Derived from ping.c distributed in Linux's netkit. That code is
    copyright (c) 1989 by The Regents of the University of California.
    That code is in turn derived from code written by Mike Muuss of the
    US Army Ballistic Research Laboratory in December, 1983 and
    placed in the public domain. They have my thanks.
 
    Bugs are naturally mine. I'd be glad to hear about them. There are
    certainly word - size dependenceies here.
 
    Copyright (c) Matthew Dixon Cowles, <http://www.visi.com/~mdc/>.
    Distributable under the terms of the GNU General Public License
    version 2. Provided with no warranties of any sort.
 
    Original Version from Matthew Dixon Cowles:
      -> ftp://ftp.visi.com/users/mdc/ping.py
 
    Rewrite by Jens Diemer:
      -> http://www.python-forum.de/post-69122.html#69122
 
    Rewrite by George Notaras:
      -> http://www.g-loaded.eu/2009/10/30/python-ping/
 
    Revision history
    ~~~~~~~~~~~~~~~~
 
    November 8, 2009
    ----------------
    Improved compatibility with GNU/Linux systems.
 
    Fixes by:
     * George Notaras -- http://www.g-loaded.eu
    Reported by:
     * Chris Hallman -- http://cdhallman.blogspot.com
 
    Changes in this release:
     - Re-use time.time() instead of time.clock(). The 2007 implementation
       worked only under Microsoft Windows. Failed on GNU/Linux.
       time.clock() behaves differently under the two OSes[1].
 
    [1] http://docs.python.org/library/time.html#time.clock
 
    May 30, 2007
    ------------
    little rewrite by Jens Diemer:
     -  change socket asterisk import to a normal import
     -  replace time.time() with time.clock()
     -  delete "return None" (or change to "return" only)
     -  in checksum() rename "str" to "source_string"
 
    November 22, 1997
    -----------------
    Initial hack. Doesn't do much, but rather than try to guess
    what features I (or others) will want in the future, I've only
    put in what I need now.
 
    December 16, 1997
    -----------------
    For some reason, the checksum bytes are in the wrong order when
    this is run under Solaris 2.X for SPARC but it works right under
    Linux x86. Since I don't know just what's wrong, I'll swap the
    bytes always and then do an htons().
 
    December 4, 2000
    ----------------
    Changed the struct.pack() calls to pack the checksum and ID as
    unsigned. My thanks to Jerome Poincheval for the fix.
 
 
    Last commit info:
    ~~~~~~~~~~~~~~~~~
    $LastChangedDate: $
    $Rev: $
    $Author: $
"""
 
 
import os, sys, socket, struct, select, time
 
# From /usr/include/linux/icmp.h; your milage may vary.
ICMP_ECHO_REQUEST = 8 # Seems to be the same on Solaris.
 
 
def checksum(source_string):
    """
    I'm not too confident that this is right but testing seems
    to suggest that it gives the same answers as in_cksum in ping.c
    """
    sum = 0
    countTo = (len(source_string)/2)*2
    count = 0
    while count<countTo:
        thisVal = ord(source_string[count + 1])*256 + ord(source_string[count])
        sum = sum + thisVal
        sum = sum & 0xffffffff # Necessary?
        count = count + 2
 
    if countTo<len(source_string):
        sum = sum + ord(source_string[len(source_string) - 1])
        sum = sum & 0xffffffff # Necessary?
 
    sum = (sum >> 16)  +  (sum & 0xffff)
    sum = sum + (sum >> 16)
    answer = ~sum
    answer = answer & 0xffff
 
    # Swap bytes. Bugger me if I know why.
    answer = answer >> 8 | (answer << 8 & 0xff00)
 
    return answer
 
 
def receive_one_ping(my_socket, ID, timeout):
    """
    receive the ping from the socket.
    """
    timeLeft = timeout
    while True:
        startedSelect = time.time()
        whatReady = select.select([my_socket], [], [], timeLeft)
        howLongInSelect = (time.time() - startedSelect)
        if whatReady[0] == []: # Timeout
            return
 
        timeReceived = time.time()
        recPacket, addr = my_socket.recvfrom(1024)
        icmpHeader = recPacket[20:28]
        type, code, checksum, packetID, sequence = struct.unpack(
            "bbHHh", icmpHeader
        )
        if packetID == ID:
            bytesInDouble = struct.calcsize("d")
            timeSent = struct.unpack("d", recPacket[28:28 + bytesInDouble])[0]
            return timeReceived - timeSent
 
        timeLeft = timeLeft - howLongInSelect
        if timeLeft <= 0:
            return
 
 
def send_one_ping(my_socket, dest_addr, ID):
    """
    Send one ping to the given >dest_addr<.
    """
    dest_addr  =  socket.gethostbyname(dest_addr)
 
    # Header is type (8), code (8), checksum (16), id (16), sequence (16)
    my_checksum = 0
 
    # Make a dummy heder with a 0 checksum.
    header = struct.pack("bbHHh", ICMP_ECHO_REQUEST, 0, my_checksum, ID, 1)
    bytesInDouble = struct.calcsize("d")
    data = (192 - bytesInDouble) * "Q"
    data = struct.pack("d", time.time()) + data
 
    # Calculate the checksum on the data and the dummy header.
    my_checksum = checksum(header + data)
 
    # Now that we have the right checksum, we put that in. It's just easier
    # to make up a new header than to stuff it into the dummy.
    header = struct.pack(
        "bbHHh", ICMP_ECHO_REQUEST, 0, socket.htons(my_checksum), ID, 1
    )
    packet = header + data
    my_socket.sendto(packet, (dest_addr, 1)) # Don't know about the 1
 
 
def do_one(dest_addr, timeout):
    """
    Returns either the delay (in seconds) or none on timeout.
    """
    icmp = socket.getprotobyname("icmp")
    try:
        my_socket = socket.socket(socket.AF_INET, socket.SOCK_RAW, icmp)
    except socket.error, (errno, msg):
        if errno == 1:
            # Operation not permitted
            msg = msg + (
                " - Note that ICMP messages can only be sent from processes"
                " running as root."
            )
            raise socket.error(msg)
        raise # raise the original error
 
    my_ID = os.getpid() & 0xFFFF
 
    send_one_ping(my_socket, dest_addr, my_ID)
    delay = receive_one_ping(my_socket, my_ID, timeout)
 
    my_socket.close()
    return delay
 
 
def verbose_ping(dest_addr, timeout = 2, count = 4):
    """
    Send >count< ping to >dest_addr< with the given >timeout< and display
    the result.
    """
    for i in xrange(count):
        print "ping %s..." % dest_addr,
        try:
            delay  =  do_one(dest_addr, timeout)
        except socket.gaierror, e:
            print "failed. (socket error: '%s')" % e[1]
            break
 
        if delay  ==  None:
            print "failed. (timeout within %ssec.)" % timeout
        else:
            delay  =  delay * 1000
            print "get ping in %0.4fms" % delay
    print
 
 
if __name__ == '__main__':
    verbose_ping("heise.de")
    verbose_ping("google.com")
    verbose_ping("a-test-url-taht-is-not-available.com")
    verbose_ping("192.168.1.1")

I decided not to publish the updated code on CodeTRAX, where I normally publish any programming-related stuff, but leave it on G-Loaded instead.

Related posts:

1. Sockets Programming In Python
2. Python IRC Bot
3. Python Crash Course

The permissive mode exists mainly for testing. In this mode, the auditing mechanism generates notices (AVC Denials) about the action/event that was blocked, but without actually blocking that action/event. It is just a way to check what would have happened if SELinux had been running in enforcing mode. There are two ways to switch SELinux to permissive mode. One is to configure it through its configuration file (/etc/selinux/config on EL) to start in permissive mode on boot and the other is to use the aforementioned utility, setenforce, to switch modes while the system is live.

When performing tests on a live system, I usually take all precautions to minimize the risk of failure and usually, if possible, I am around to monitor the progress of the test. But, this time I had to use the custom SELinux policy for some days to see if it is actually effective or it needs further fine-tuning.

The Mistake: I used setenforce to set SELinux into permissive mode (/usr/sbin/setenforce 0). setenforce is for temporary changes and it is definitely not suitable for testing the policy for a long period of time. This is because if for any reason the system reboots, SELinux will be set back into the mode that is defined into its configuration file. The mode set by setenforce does not survive a reboot. So, if the custom policy happens to be incomplete, it will block the server’s normal operation after the reboot. And such reboots can happen…

This is exactly what I experienced yesterday. The datacenter on which my virtual server is hosted had problems with its main power supply. This caused one hour of downtime. But G-Loaded’s outage was a lot greater, because my custom SELinux policy was incomplete. SELinux was started in enforcing mode, so my faulty policy blocked my www service.

This should be a lesson to anyone who performs tests on live systems. The Right Thing™ was simple: boot the server into permissive mode and do as many tests as desired. I can blame the datacenter for the one-hour downtime, but for the 7-hour unavailability of G-Loaded I am the only one to blame.

Related posts:

1. SELinux audit reports script
2. Using a switch to prevent system shutdown/reboot/suspend
3. Server upgraded to Fedora 6