So, TOR can provide anonymity as far as the user’s ISP is concerned, but is it a secure way to communicate with remote services? If no extra encryption is used, then it is quite obvious that using remote services through the TOR network is totally insecure. Here is why.

The TOR exit-node is a key point in the communication between the user and the remote service. This is where the user’s data exits the TOR tunnel and continues its way to the remote service through the 3rd party ISP’s network. It is also the place where data from the remote service leaves the 3rd party ISP’s network and enters the TOR tunnel in order to reach the end user. If no encryption is used, it is possible for the exit-node operator to sniff this network traffic. This means that it is technically possible for an evil exit-node operator to:

• know which web pages the user visits
• read the messages the user exchanges through unencrypted IM networks
• read the emails the user sends
• if the user authenticates to any services without encryption, the evil exit-node operator could for example find out his mailbox or FTP account password or the passwords the user uses for authentication to web sites
• even if the authentication to a web service has taken place through an encrypted SSL tunnel, if the rest of the communication with this specific web service is not encrypted, the evil operator could grab a copy of the user’s session cookie for this service and access it pretending to be him

These are some of the nasty things that can happen when you access remote services through a proxy server which you do not control.

Is there any guarantee that exit node operators do not sniff network traffic?

Even if the exit-node operators are cool, who can guarantee that the network traffic is not monitored within the 3rd party ISP‘s network? If the user accesses personalized services without encryption, then, even if the user’s real IP and thus his real name is not known, various pieces of collected data can be combined together and possibly reveal his real identity. This process is widely known as re-identification.

Is there any guarantee that the ISP providing internet access to a TOR exit node does not collect and sell information to “marketers and identity thieves”?

I consider the TOR project quite important. But, since typical internet users are urged to use the TOR network in order to browse the internet, the involved risks have to be explained in detail.

On the other hand, I’d like to urge internet users to spend some time to familiarize themselves with the basics of the HTTP protocol, the concepts of HTTP authentication and cookie based authentication and the importance of encrypted HTTP connections through SSL or TLS tunnels. Since the internet has become part of your lives, regardless of your profession, you need to be educated about these things, so as to be able to realize when your communication with the various internet services is vulnerable. You don’t have to be gurus, but rather get an idea of what is going on.

So far, it is quite clear that the only way to stay on the safe side while using anonymizing proxies on which you usually do not have full control, like TOR, is to connect to any remote services using encrypted connections only, usually through SSL or TLS tunnels. Personally, I never use anonymizing networks or third party proxies. This is because I never really had the need to hide my real location. Furthermore, I find it pointless as I don’t believe that such a thing as anonymity is really feasible. If I had to use TOR, I would try to find a way to connect to the remote service over an encrypted connection. In general, whenever I need a secure SOCKS proxy, for example when I have to use a public network to access personalized internet services, which do not offer full SSL access, I use OpenSSH client’s -D switch while logging in to a SSH server which I own and fully control and thus I have all the security I need.

Related Articles

• Auto-closing SSH tunnels
• Set up the VNC Server in Fedora
• SSH Tunnels Headaches
• Netcat – a couple of useful examples
• Using SSH for networking

DD-WRT is a Linux based alternative Open Source firmware suitable for a great variety of WLAN routers and embedded systems. The main emphasis lies on providing the easiest possible handling while at the same time supporting a great number of functionalities within the framework of the respective hardware platform used.

You can check whether a device is supported by DD-WRT or not by checking the device’s model name in the database of supported routers. If you are like me and prefer to read an overview of the wireless N devices instead of querying a database, there is a page in the project’s wiki that lists all these devices.

After checking the technical specifications of several routers on their manufacturer web sites, it is quite clear that at this moment there are not many wireless N routers & ADSL2+ modem combos that are fully supported by DD-WRT. In my case that won’t be a big problem, but I’d really like the device to have a DSL port so as to keep the number of devices around the house at a minimum. But anyway, this is not a big deal.

Some other things I’d consider a plus for such a device are:

1. Support for Wireless Client Mode by the manufacturer’s firmware
2. Support for a VPN Server by the manufacturer’s firmware

I know that these functions are available anyway if the DD-WRT firmware is used, but, if the default firmware supported these functions, it would definitely make a difference for me. Another useful feature would be the existence of a USB port, but this would greatly limit the available supported devices, so this is not a mandatory feature.

So, if anyone has a suggestion for such a router or if you own one, please let me know about your experience with it. I am particularly interested for devices that are 100% supported by the DD-WRT open source firmware.

Related Articles

• Viceo Backend for SANE with libusb support
• Software that detects violations of open source licenses
• Creative PC-CAM 750 on Fedora 10
• Creative PC-CAM Series webcams in linux
• Howto: DHCP Server (dhcpd) Configuration

Is a caching nameserver really important?

There is some controversy about the real benefits of using a caching name server in a system, either desktop or server. In this article we keep controversy out of the discussion and focus on the performance improvement the caching of DNS information can offer to a system while performing specific tasks. For instance, a caching nameserver allows a web browser to acquire DNS information from the local DNS cache, provided that this information has already been cached, without the need to access any public DNS servers, which results in faster web browsing. Similarly, in a server environment, services like spam filters often need to perform many DNS queries for the same hostnames. The latency of the communication with the remote nameserver may add up to the total time of email processing.

BIND vs dnsmasq

BIND is the flagship of DNS servers with large deployments around the globe. I have used BIND for many years as a caching nameserver, even on my desktop, until I realized it is overkill to use BIND this way. There are lighter solutions, even all-in-one software like dnsmasq, that seem to be more suitable for setting up local DNS caching.

System preparation

So, let’s get started with the system preparation before going into the details of the dnsmasq configuration.

First of all, we need to install dnsmasq:

yum install dnsmasq

dnsmasq, when run as root, is designed to drop privileges and run as an unprivileged user. By default, this user is nobody. We use a dedicated system user to run dnsmasq.

Run the following commands as root to create such an unprivileged system user and group named dnsmasq:

groupadd -r dnsmasq
useradd -r -g dnsmasq dnsmasq

The above should be enough.

Configuration

All dnsmasq configuration options go into /etc/dnsmasq.conf. Here we write this file from scratch, so if you need to keep a copy of the original that ships with your distribution, do so with:

cp /etc/dnsmasq.conf /etc/dnsmasq.conf.orig

Now, let’s get started with adding our own dnsmasq configuration in /etc/dnsmasq.conf.

First of all, we set some options regarding the basic server operation like the interface and port on which it should bind, the unprivileged user that should run the service and a PID file:

listen-address=127.0.0.1
port=53
bind-interfaces
user=dnsmasq
group=dnsmasq
pid-file=/var/run/dnsmasq.pid

The bind-interfaces directive instructs dnsmasq to bind only to the network interface specified in the listen-address directive.

Next comes logging.

By default, dnsmasq sends its log messages to the DAEMON syslog facility (LOCAL0 when operating in debug mode). We go with the defaults here, but keep in mind that a separate log file can be set as it is shown in the configuration snippet below (currently commented out):

#log-facility=/var/log/dnsmasq.log
#log-queries

Logging to file requires some extra configuration for proper log rotation. For more information, please read Appendix II.

Finally, we set the options that configure dnsmasq’s name resolution and caching operations.

The following directives prevent dnsmasq from forwarding plain names (without any dots) or addresses in the non-routed address space to the parent nameservers.

domain-needed
bogus-priv

The no-hosts directive also instructs dnsmasq not to read any hostnames from /etc/hosts. In most systems, /etc/hosts is queried before a DNS service is used by the system for name lookups. So, all plain name to private IP mappings should normally be added in /etc/hosts. If this is not what you want, then take a look at the expand-hosts and domain directives.

no-hosts

Set the maximum number of concurrent DNS queries. The default value is 150. Adjust to your needs.

dns-forward-max=150

Set the size of the dnsmasq cache. The default is to keep 150 hostnames. By setting the cache size to 0 disables the feature (this is not what we really want). Again, adjust this value according to your needs.

cache-size=1000

The following directive controls whether negative caching should be enabled or not. Negative caching allows dnsmasq to remember “no such domain” answers from the parent nameservers, so it does not query for the same non-existent hostnames again and again. This is probably useful for spam filters or MTA services. By default, negative caching is enabled. To disable, un-comment the following directive.

#no-negcache

The neg-ttl directive sets a default TTL value to add to negative replies from the parent nameservers, in case these replies do not contain TTL information. If neg-ttl is not set and a negative reply from a parent DNS server does not contain TTL information, then dnsmasq will not cache the reply. Here we set the default TTL to 3600 seconds. Again, adjust to your specific needs.

neg-ttl=3600

Here we use a separate file where dnsmasq reads the IPs of the parent nameservers from. The syntax is the same as in /etc/resolv.conf. We do this to facilitate the manipulation of the parent nameservers that should be used by dnsmasq by using, for example, an external script. The filename we use here is resolv.dnsmasq, but this can be changed to your liking. We also set the no-poll directive here to prevent dnsmasq from polling the ‘resolv’ file for changes.

resolv-file=/etc/resolv.dnsmasq
no-poll

A full configuration file containing all the above configuration, which can can be used as a drop-in replacement of the default /etc/dnsmasq.conf, can be found in Appendix I.

Upstream Nameservers

We have used a separate file to store the IPs of the parent nameservers; that is /etc/resolv.dnsmasq. Using the same syntax as in /etc/resolv.conf add the nameserver IP addresses in resolv.dnsmasq. For example:

nameserver 192.168.0.252
nameserver 192.168.0.253
nameserver 192.168.0.254

Note that we still need to make a change in /etc/resolv.conf before the system starts using dnsmasq for domain name lookups. Read on…

Starting dnsmasq

In order to start dnsmasq, run as root:

/etc/init.d/dnsmasq start

Check the syslog or the dnsmasq logfile (if used) for any error messages.

If everything seems to be OK, set the dnsmasq service to start on boot:

chkconfig dnsmasq on

This command might be Red-Hat specific, so consult your distribution’s documentation about how to set services to start on boot.

Switch name resolution to dnsmasq

What we have done so far is set up the dnsmasq service. For hostnames that do not exist in /etc/hosts the system still uses the nameserver inside /etc/resolv.conf for name resolution.

To start using dnsmasq, edit /etc/resolv.conf, remove all nameservers and add only the IP of our dnsmasq service:

nameserver 127.0.0.1

From now on, the system will use dnsmasq for domain name resolution. You can un-comment the log-queries option in order to confirm the dnsmasq operation.

Appendix I – Full configuration file

This is the complete configuration file containing the configuration that has been discussed in this article. Note that it can be used as is to replace the default /etc/dnsmasq.conf.

#
# Configuration file for dnsmasq acting as a caching nameserver.
#
# Format is one option per line, legal options are the same
# as the long options legal on the command line. See
# "/usr/sbin/dnsmasq --help" or "man 8 dnsmasq" for details.
#
# Updated versions of this configuration file may be available at:
#
#   http://www.g-loaded.eu/2010/09/18/caching-nameserver-using-dnsmasq/
#
#
# Basic server configuration
#
listen-address=127.0.0.1
port=53
bind-interfaces
user=dnsmasq
group=dnsmasq
pid-file=/var/run/dnsmasq.pid
#
# Logging
#
#log-facility=/var/log/dnsmasq.log
#log-queries
#
# Name resolution options
#
domain-needed
bogus-priv
no-hosts
dns-forward-max=150
cache-size=1000
#no-negcache
neg-ttl=3600
resolv-file=/etc/resolv.dnsmasq
no-poll

This file is meant to be used both on servers and desktops.

Appendix II – Logging to file

Before dnsmasq starts logging to file it is required to set the path to the logfile in the log-facility option inside /etc/dnsmasq.conf.

log-facility=/var/log/dnsmasq.log

To ensure proper rotation of the log file you should use the following logrotate configuration:

/var/log/dnsmasq.log {
    monthly
    missingok
    notifempty
    delaycompress
    sharedscripts
    postrotate
        [ ! -f /var/run/dnsmasq.pid ] || kill -USR2 `cat /var/run/dnsmasq.pid`
    endscript
    create 0640 dnsmasq dnsmasq
}

Save the above configuration in /etc/logrotate.d/dnsmasq. Also, adjust the log filename or the path to the PID file in case you have used custom names, but make sure you do not change the USR2 signal that is sent to the dnsmasq process in the post-rotation script.

Final Thoughts

dnsmasq is a very lightweight service. Therefore, you can run it on any system, either server or desktop without any noticeable impact on system resources. In this guide we used it as an internal system service bound to the loopback interface, without permitting direct access from the outside. This along with the fact that dnsmasq is mature software that has been around for several years makes our setup rather secure.

Several people might argue that the performance improvement a local caching nameserver offers in terms of name lookup speed is insignificant. This might be true in some cases, but there are times that this performance improvement is noticeable, especially when the quality of the network connectivity between the current machine and the upstream nameserver is an issue, or when the upstream name server is overloaded. On the other hand, it is almost certain that a local caching DNS server can in no way make name resolution slower, unless perhaps a huge cache is being used. Generally, I find keeping such a service operational a good idea.

In this article we discussed about one of the dnsmasq features: DNS caching. dnsmasq is a lot more than just that. Check the whole feature set in the dnsmasq homepage. Perhaps, in the future, more guides covering other features of this software are published. Until then, enjoy local DNS caching!!!

Related Articles

• Set up an anonymous FTP server with vsftpd in less than a minute
• How to integrate seaudit-report in logwatch
• Assign Virtual IPs to your NIC
• Use wget or curl to download from RapidShare Premium
• SSL-enabled Name-based Apache Virtual Hosts with mod_gnutls

[This post has been updated]

[Update #1]: This limitation also exists on the ping command, which runs with setuid access rights. (Thanks Stephane)

[Update #2]: After Chris Hallman reported that the original code actually works on Windows only, I made some changes to it and publish the updated code below.

The following code is a pure Python implementation of the ping command. I originally found it in the source code tree of pylucid in a subdirectory where the developers keep various code snippets.

I had originally tested the code under Microsoft Windows as this was the OS I had available at the moment. Chris Hallman noticed that the code does not work under GNU/Linux. After spending some hours trying to figure out what was wrong it, I realized that the part of the code that caused the failure on Linux was the use of the time.clock() function instead of time.time(). The clock() method works differently under Windows and Linux.

I fixed the 2007 implementation and hereby publish the updated code(as python-ping). Feel free to test it and report any issues.

Needed: test on Solaris.

#!/usr/bin/env python
 
"""
    A pure python ping implementation using raw socket.
 
 
    Note that ICMP messages can only be sent from processes running as root.
 
 
    Derived from ping.c distributed in Linux's netkit. That code is
    copyright (c) 1989 by The Regents of the University of California.
    That code is in turn derived from code written by Mike Muuss of the
    US Army Ballistic Research Laboratory in December, 1983 and
    placed in the public domain. They have my thanks.
 
    Bugs are naturally mine. I'd be glad to hear about them. There are
    certainly word - size dependenceies here.
 
    Copyright (c) Matthew Dixon Cowles, <http://www.visi.com/~mdc/>.
    Distributable under the terms of the GNU General Public License
    version 2. Provided with no warranties of any sort.
 
    Original Version from Matthew Dixon Cowles:
      -> ftp://ftp.visi.com/users/mdc/ping.py
 
    Rewrite by Jens Diemer:
      -> http://www.python-forum.de/post-69122.html#69122
 
    Rewrite by George Notaras:
      -> http://www.g-loaded.eu/2009/10/30/python-ping/
 
    Revision history
    ~~~~~~~~~~~~~~~~
 
    November 8, 2009
    ----------------
    Improved compatibility with GNU/Linux systems.
 
    Fixes by:
     * George Notaras -- http://www.g-loaded.eu
    Reported by:
     * Chris Hallman -- http://cdhallman.blogspot.com
 
    Changes in this release:
     - Re-use time.time() instead of time.clock(). The 2007 implementation
       worked only under Microsoft Windows. Failed on GNU/Linux.
       time.clock() behaves differently under the two OSes[1].
 
    [1] http://docs.python.org/library/time.html#time.clock
 
    May 30, 2007
    ------------
    little rewrite by Jens Diemer:
     -  change socket asterisk import to a normal import
     -  replace time.time() with time.clock()
     -  delete "return None" (or change to "return" only)
     -  in checksum() rename "str" to "source_string"
 
    November 22, 1997
    -----------------
    Initial hack. Doesn't do much, but rather than try to guess
    what features I (or others) will want in the future, I've only
    put in what I need now.
 
    December 16, 1997
    -----------------
    For some reason, the checksum bytes are in the wrong order when
    this is run under Solaris 2.X for SPARC but it works right under
    Linux x86. Since I don't know just what's wrong, I'll swap the
    bytes always and then do an htons().
 
    December 4, 2000
    ----------------
    Changed the struct.pack() calls to pack the checksum and ID as
    unsigned. My thanks to Jerome Poincheval for the fix.
 
 
    Last commit info:
    ~~~~~~~~~~~~~~~~~
    $LastChangedDate: $
    $Rev: $
    $Author: $
"""
 
 
import os, sys, socket, struct, select, time
 
# From /usr/include/linux/icmp.h; your milage may vary.
ICMP_ECHO_REQUEST = 8 # Seems to be the same on Solaris.
 
 
def checksum(source_string):
    """
    I'm not too confident that this is right but testing seems
    to suggest that it gives the same answers as in_cksum in ping.c
    """
    sum = 0
    countTo = (len(source_string)/2)*2
    count = 0
    while count<countTo:
        thisVal = ord(source_string[count + 1])*256 + ord(source_string[count])
        sum = sum + thisVal
        sum = sum & 0xffffffff # Necessary?
        count = count + 2
 
    if countTo<len(source_string):
        sum = sum + ord(source_string[len(source_string) - 1])
        sum = sum & 0xffffffff # Necessary?
 
    sum = (sum >> 16)  +  (sum & 0xffff)
    sum = sum + (sum >> 16)
    answer = ~sum
    answer = answer & 0xffff
 
    # Swap bytes. Bugger me if I know why.
    answer = answer >> 8 | (answer << 8 & 0xff00)
 
    return answer
 
 
def receive_one_ping(my_socket, ID, timeout):
    """
    receive the ping from the socket.
    """
    timeLeft = timeout
    while True:
        startedSelect = time.time()
        whatReady = select.select([my_socket], [], [], timeLeft)
        howLongInSelect = (time.time() - startedSelect)
        if whatReady[0] == []: # Timeout
            return
 
        timeReceived = time.time()
        recPacket, addr = my_socket.recvfrom(1024)
        icmpHeader = recPacket[20:28]
        type, code, checksum, packetID, sequence = struct.unpack(
            "bbHHh", icmpHeader
        )
        if packetID == ID:
            bytesInDouble = struct.calcsize("d")
            timeSent = struct.unpack("d", recPacket[28:28 + bytesInDouble])[0]
            return timeReceived - timeSent
 
        timeLeft = timeLeft - howLongInSelect
        if timeLeft <= 0:
            return
 
 
def send_one_ping(my_socket, dest_addr, ID):
    """
    Send one ping to the given >dest_addr<.
    """
    dest_addr  =  socket.gethostbyname(dest_addr)
 
    # Header is type (8), code (8), checksum (16), id (16), sequence (16)
    my_checksum = 0
 
    # Make a dummy heder with a 0 checksum.
    header = struct.pack("bbHHh", ICMP_ECHO_REQUEST, 0, my_checksum, ID, 1)
    bytesInDouble = struct.calcsize("d")
    data = (192 - bytesInDouble) * "Q"
    data = struct.pack("d", time.time()) + data
 
    # Calculate the checksum on the data and the dummy header.
    my_checksum = checksum(header + data)
 
    # Now that we have the right checksum, we put that in. It's just easier
    # to make up a new header than to stuff it into the dummy.
    header = struct.pack(
        "bbHHh", ICMP_ECHO_REQUEST, 0, socket.htons(my_checksum), ID, 1
    )
    packet = header + data
    my_socket.sendto(packet, (dest_addr, 1)) # Don't know about the 1
 
 
def do_one(dest_addr, timeout):
    """
    Returns either the delay (in seconds) or none on timeout.
    """
    icmp = socket.getprotobyname("icmp")
    try:
        my_socket = socket.socket(socket.AF_INET, socket.SOCK_RAW, icmp)
    except socket.error, (errno, msg):
        if errno == 1:
            # Operation not permitted
            msg = msg + (
                " - Note that ICMP messages can only be sent from processes"
                " running as root."
            )
            raise socket.error(msg)
        raise # raise the original error
 
    my_ID = os.getpid() & 0xFFFF
 
    send_one_ping(my_socket, dest_addr, my_ID)
    delay = receive_one_ping(my_socket, my_ID, timeout)
 
    my_socket.close()
    return delay
 
 
def verbose_ping(dest_addr, timeout = 2, count = 4):
    """
    Send >count< ping to >dest_addr< with the given >timeout< and display
    the result.
    """
    for i in xrange(count):
        print "ping %s..." % dest_addr,
        try:
            delay  =  do_one(dest_addr, timeout)
        except socket.gaierror, e:
            print "failed. (socket error: '%s')" % e[1]
            break
 
        if delay  ==  None:
            print "failed. (timeout within %ssec.)" % timeout
        else:
            delay  =  delay * 1000
            print "get ping in %0.4fms" % delay
    print
 
 
if __name__ == '__main__':
    verbose_ping("heise.de")
    verbose_ping("google.com")
    verbose_ping("a-test-url-taht-is-not-available.com")
    verbose_ping("192.168.1.1")

I decided not to publish the updated code on CodeTRAX, where I normally publish any programming-related stuff, but leave it on G-Loaded instead.

Related Articles

• Sockets Programming In Python
• Descramble Passwords from gftp Bookmarks using Python
• Python IRC Bot
• Use Python to get the web page data in Epiphany
• Python Crash Course

While I was performing the operating system installations, everything seemed to work as expected. Each virtual machine obtained a dynamic IP address and the DNS records were updated correctly. Glad that things work, I turned off Dom0 after the tests.

At a later time, when I switched Dom0 on again, I noticed that I could not reach any of the virtual machines. Their fully qualified domain names that had been allocated to them by the DHCP service would not resolve to the proper IP. To make it worse, it seemed that the virtual machines had no active leases at all!

Soon I discovered that xendomains, the service responsible for starting and stopping the unprivileged Xen domains (DomU), whenever Domain Zero was shut down, it just saved the domain U’s state for later restoration instead of initiating the DomU’s shutdown procedure. When Dom0 was brought up again, xendomains restored the saved DomU states instead of forcing the domains to boot up normally.

I assume this is like suspending a laptop, which has obtained its IP automatically from a DHCP service within LAN A, moving it to a LAN B, and then bringing it up again within that LAN. It is almost certain that for an arbitrary amount of time you would experience connectivity problems.

In both cases above, at the time the machine resumes operations, the dhclient script (or any other similar script) is not run in order to renew the machine’s IP address.

Fortunately, this behaviour of the xendomains service is configurable. In order to force all DomUs to shutdown, instead of suspending, during Domain 0′s shutdown, all you have to do is make following changes in /etc/sysconfig/xendomains:

Leave empty the XENDOMAINS_SAVE variable. By default, it uses /var/lib/xen/save as the directory where the states of the DomUs are saved. By leaving it empty, the states of the virtual machines are not saved, but they are shut down as usual.

XENDOMAINS_SAVE=""

The second modification involves setting the XENDOMAINS_RESTORE variable to false. This determines whether saved domains will be restored from the directory set in the XENDOMAINS_SAVE variable during the Dom0 startup:

XENDOMAINS_RESTORE=false

From now on, whenever Dom0 starts up, it will force all U domains to boot up normally. Contrariwise, whenever Dom0 shuts down, it will force all DomU to shut down as well. No domain state saving or restoration happens any more.

This means that the network service on each DomU, hence the dhclient script, will run every time the domain U starts up or shuts down, ensuring proper allocation of dynamic IP addresses and hostnames by the DHCP service. Of course, this adds much overhead to both the boot and power off process of Domain Zero (Dom0), but it seems that there is no other way to overcome this problem. If you know of a better way, please let me know.

Related Articles

• Howto: DHCP Server (dhcpd) Configuration
• Assign Virtual IPs to your NIC
• Set up the VNC Server in Fedora
• Partition images with Partimage and Partimaged
• Almost saying goodbye to innovation

First, is the cipler-none patch that adds none as a valid argument to the -c command line option. By using it, the transferred data is not encrypted. Pros: eliminates the data encryption overhead. Cons: totally insecure method of transferring sensitive data.

cipher-none-patch

Note: the OpenSSH server, even if it has been patched with this code, does not accept unencrypted connections by default. This has to be enabled explicitly in the sshd configuration (sshd_config) by adding the none “cipher” to the list of the accepted ciphers:

Ciphers aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,arcfour,aes192-cbc,aes256-cbc,rijndael-cbc@lysator.liu.se,aes128-ctr,aes192-ctr,aes256-ctr,none

Note2: here is step-by-step guide how to configure the SSH authentication using public keys.

Second, is a set of patches, created at the Pittsburgh Supercomputing Center, which eliminate the bottlenecks caused by some of the internal buffers that control data flow in network connections through OpenSSH. This project is called High Performance SSH/SCP (aka HPN-SSH). Benchmarks show that even encrypted network connections using HPN-SSH perform extraordinarily better than the usual encrypted OpenSSH connections.

OK, this info exists here for completeness, as those HPN-SSH patches have not been designed for home networks! Possibly it might make no difference if you use the regular OpenSSH or HPN-SSH in your home LAN.

Note: the HPN-SSH patches also contain code that adds the none cipher, similar to the cipher-none patch, so, if you intend to use HPN-SSH, it is not required to apply both on the OpenSSH source.

Finally, apart from all these, if you ever decide to use SSH for networking seriously consider using SSHfs (see also SSHfsFAQ).

Related Articles

• How to configure mod_gnutls to use the RC4 cipher to mitigate the SSL/TLS vulnerability
• Assign Virtual IPs to your NIC
• Setup the SSH server to use keys for authentication
• Encrypt devices using dm-crypt and LUKS
• TrueCrypt on 2.6.18 kernels

Check if the module is loaded

IPv6 functionality is being made available to the system by the ipv6 kernel module. To check if this module is currently loaded in your system, issue the following command as root:

lsmod | grep ipv6

If you see ipv6 in its output, then the module is loaded.

Performing this check is absolutely not necessary. It is included in this article for completeness.

Disable IPv6

You can prevent a module from being inserted into the kernel by either blacklisting it or by completely disabling it.

In this case, since you will most probably turn off the IPv6 firewall (ip6tables) as well, it is highly recommended to completely disable the ipv6 module, to avoid any accidental loading of the IPv6 stack without any firewall protection at the same time.

How the module blacklist works

This information about blacklisting a kernel module exists here for educational purposes. It has been mentioned above that for ipv6 it is important to completely disable it.

From the modprobe.conf man page:

Modules can contain their own aliases: usually these are aliases describing the devices they support, such as “pci:123…”. These “internal” aliases can be overridden by normal “alias” keywords, but there are cases where two or more modules both support the same devices, or a module invalidly claims to support a device: the blacklist keyword indicates that all of that particular module’s internal aliases are to be ignored.

So, blacklist indicates that a module’s aliases should be ignored. But, what happens if an application requires to load that specific module or if root uses modprobe to load it on demand? Let’s test it…

To blacklist the module, simply save the following line in a file inside /etc/modprobe.d:

blacklist ipv6

Next, disable any services that use IPv6, eg ip6tables or any IPv6-enabled network interfaces and reboot (mandatory).

After you’ve logged-in again, try, for example, to load the ipv6 module with the modprobe command (as root):

[root@centos]# modprobe -v ipv6
insmod /lib/modules/2.6.18-53.1.14.el5/kernel/net/ipv6/ipv6.ko
[root@centos]# lsmod | grep v6
ipv6                  251393  8

The blacklisted module has been loaded. This is what happens if it is needed by a system service, regardless of the fact that it has been blacklisted. In the case of ipv6 this could be a security risk, provided that the ipv6 firewall has been turned off but some network interfaces still use IPv6. So, frankly, it is suggested to read on how to disable the module more aggressively…

Completely disable the ipv6 module

To completely disable IPv6 in your system, all you have to do is save the following line in a file inside /etc/modprobe.d/.

install ipv6 /bin/true

The above line means: whenever the system needs to load the ipv6 kernel module, it is forced to execute the command true instead of actually loading the module. Since /bin/true, does absolutely nothing, the module never gets loaded.

Again, it is required to reboot for the changes to take effect.

It is obvious that this is an aggressive method to disable kernel modules, but it guarantees that the module never gets loaded.

This is the recommended way to disable IPv6.

Other Configuration Tasks

Since the IPv6 functionality has been disabled, you can disable the ip6tables service (IPv6 Firewall). Issue the following command as root:

chkconfig ip6tables off

It is also a good idea, since the ip6tables service has been turned off, to disable any IPv6-related functionality in the network interface configuration. Even if you do not do this, the IPv6 stack will not be initialized because the ipv6 module cannot be loaded. But, generally, you could set the following options to “no” inside your network interface scripts, for example: /etc/sysconfig/network-scripts/ifcfg-eth0

IPV6INIT=no
IPV6_AUTOCONF=no

Finally, In fedora 8 or newer you can safely remove the following option from the /etc/sysconfig/network file, if it exists:

NETWORKING_IPV6=no

Final Thoughts

Using the instructions above, you can completely disable IPv6 in your system. On the other hand, you should understand that IPv6 is not an evil thing… It exists in order to address certain issues. If you ever think about actually trying to configure and use it instead of just disabling it every time you install your Linux operating system, here is a good place to start…

Related Articles

• Fedora Server vs CentOS
• The Complete Fedora Kernel Headers
• Creative PC-CAM 750 on Fedora 10
• High CPU usage while running CentOS as guest on Virtualbox or VMware
• Set up the VNC Server in Fedora

Theory

Rapidshare uses cookie-based authentication. This means that every time you log into the service, a cookie containing information which identifies you as a registered user is stored in your browser’s cookie cache. Both wget and curl support saving and loading cookies, so before using them to download any files, you should save such a cookie. Having done this, then the only required action in order download from RapidShare is to load the cookie, so that wget or curl can use it to authenticate you on the RapidShare server. This is pretty much the same you would do with a graphical download manager. The difference now is that you do it on the command line.

Below you will find examples about how to perform these actions using both wget and curl.

IMPORTANT: Please note that in order to use these command-line utilities or any other download managers with RapidShare, you will have to check the Direct Downloads option in your account’s options page.

Save your RapidShare Premium Account Cookie

Saving your RapidShare cookie is a procedure that needs to be done once.

The login page is located at:

https://ssl.rapidshare.com/cgi-bin/premiumzone.cgi

The login form requires two fields: login and password. These are pretty self-explanatory.

In the following examples, the RapidShare username is shown as USERNAME and the password as PASSWORD.

Using wget

In order to save your cookie using wget, run the following:

wget \
    --save-cookies ~/.cookies/rapidshare \
    --post-data "login=USERNAME&password=PASSWORD" \
    -O - \
    https://ssl.rapidshare.com/cgi-bin/premiumzone.cgi \
    > /dev/null

–save-cookies : Saves the cookie to a file called rapidshare under the ~/.cookies directory (let’s assume that you store your cookies there)
–post-data : is the POST payload of the request. In other words it contains the data you would enter in the login form.
-O - : downloads the HTML data to the standard output. Since the above command is run only in order to obtain the cookie, this option prints the HTML data to stdout (Standard Output) and then discards it by redirecting stdout to /dev/null. If you don’t do this, wget will save the HTML data in a file called premiumzone.cgi in the current directory. This is just the Rapidshare HTML page, which is absolutely not needed.

Using curl

In order to save your cookie using curl, run the following:

curl \
    --cookie-jar ~/.cookies/rapidshare \
    --data "login=USERNAME&password=PASSWORD" \
    https://ssl.rapidshare.com/cgi-bin/premiumzone.cgi \
    > /dev/null

–cookie-jar : Saves the cookie to a file called rapidshare under the ~/.cookies directory (it has been assumed previously that cookies are stored there)
–data : contains the data you would enter in the login form.
Curl prints the downloaded page data to stdout by default. This is discarded by sending it to /dev/null.

Download files using your RapidShare Premium Account Cookie

Having saved your cookie, downloading files from RapidShare is as easy as telling wget/curl to load the cookie everytime you use them to download a file.

Downloading with wget

In order to download a file with wget, run the following:

wget -c --load-cookies ~/.cookies/rapidshare <URL>

-c : this is used in order to resume downloading of the file if it already exists in the current directory and is incomplete.
–load-cookies : loads your cookie.

Downloading with curl

In the same manner, in order to download a file with curl, run the following:

curl -L -O --cookie ~/.cookies/rapidshare <URL>

-L : Follows all redirections until the final destination page is found. This switch is almost always required as curl won’t follow redirects by default (read about how to check the server http headers with curl).
-O : By using this switch you instruct curl to save the downloaded data to a file in the current directory. The filename of the remote file is used. This switch is also required or else curl will print the data to stdout, which is something you won’t probably like.
–cookie : loads your Rapidshare account’s cookie.

Setting up a Download Server

Although most users would be satisfied with the above, I wouldn’t be surprised if you would want to go a bit further and try to setup a little service for your downloading pleasure. Here is a very primitive implementation of such a service. All you will need is standard command line tools.

This primitive server consists of the following:

1. A named pipe, called “dlbasket“. You will feed the server with URLs through this pipe. Another approach would be to use a listening TCP socket with NetCat.
2. A script, which, among others, contains the main server loop. This loop reads one URL at a time from dlbasket and starts a wget/curl process in order to download the file. If dlbasket is empty, the server should just stay there waiting.

So, in short, the service would be the following:

cat <> dlbasket | ( while ... done )

All credit for the “cat <> dlbasket |” magic goes to Zart, who kindly helped me out at the #fedora IRC channel.

So, let’s create that service. The following assume that a user named “downloader” exists in the system and the home directory is /var/lib/downloader/. Of course you can set this up as you like, but make sure you adjust the following commands and the script’s configuration options accordingly.

First, create the named pipe:

mkfifo -m 0700 /var/lib/downloader/dlbasket

If it does not exist, create a bin directory in the user’s home:

mkdir -p /var/lib/downloader/bin

Also, create a directory where the downloaded files will be saved:

mkdir -p /var/lib/downloader/downloads

The following is a quick and dirty script I wrote which actually implements the service. Save it as rsgetd.sh inside the user’s bin directory:

#! /usr/bin/env bash  
 
#  rsgetd.sh - Download Service
 
#  Version 0.3
 
#  Copyright (C) 2007 George Notaras (http://www.g-loaded.eu/)
#
#  This program is free software; you can redistribute it and/or modify
#  it under the terms of the GNU General Public License version 2 as
#  published by the Free Software Foundation.
#
#  This program is distributed in the hope that it will be useful,
#  but WITHOUT ANY WARRANTY; without even the implied warranty of
#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#  GNU General Public License for more details.
 
# Special thanks to 'Zart' from the #fedora channel on FreeNode
 
# v0.3 - Oct 31 2009:
# - corrected error: "./bin/rsgetd.sh: line 37: [: argument expected"
 
# CONFIG START 
HOMEDIR="/var/lib/downloader"
DLBASKET="$HOMEDIR/dlbasket"
DLDIR="$HOMEDIR/downloads/"
LOGFILE="$HOMEDIR/.downloads_log"
CACHEFILE="$HOMEDIR/.downloads_cache"
LIMIT="25k"
WGETBIN="/usr/bin/wget"
# Rapidshare Login Cookie  
RSCOOKIE="$HOMEDIR/cookies/.rapidshare"
# CONFIG END
 
DATETIME="`date '+%Y-%m-%d %H:%M:%S'`"
 
cat <> $DLBASKET | (
        while read url ; do
                # First, check the cache if the file has been already downloaded
                if [ -f "$CACHEFILE" -a -n "$(grep -i $(basename $url) $CACHEFILE)" ] ; then
                       echo "$DATETIME File exists in cache. Already downloaded - Skipping: $url" >> $LOGFILE
                else
                        echo "$DATETIME Starting with rate $LIMIT/s: $url" >> $LOGFILE
                        if [ $(expr match "$url" '[rapidshare.com]') = 1 ] ; then
                                # If it is a Rapidshare.com link, load the RS cookie 
                                echo "RAPIDSHARE LINK"
                                $WGETBIN -c --limit-rate=$LIMIT --directory-prefix=$DLDIR --load-cookies $RSCOOKIE $url
                        else
                                $WGETBIN -c --limit-rate=$LIMIT --directory-prefix=$DLDIR $url
                        fi
                        echo "$DATETIME Finished: $url" >> $LOGFILE
                        echo $url >> $CACHEFILE
                fi
        done )
 
exit 0

As you might have already noticed, two extra files are created inside the home directory: .downloads_cache and .downloads_log. The first contains a list of all the urls that have been downloaded. Each new download is checked against this list, so that the particular URL is not processed if the file has already been downloaded. The latter file is a usual logfile stating the start and end times of each download. Feel free to adjust the script to your needs.

Here is some info about how you should start the service:

-1- You can simply start the script as a background process and then feed URLs to it. For example:

rsgetd.sh &
echo "<URL>" > /var/lib/downloader/dlbasket

-2- Use screen in order to run the script in the background but still be able to see its output by connecting to a screen session. Although this is not a screen howto, here is an example:

Create a new screen session and attach to it:

screen -S rs_downloads

While being in the session, run rsgetd.sh

rsgetd.sh

From another terminal feed the download basket (dlbasket) with urls:

echo "<URL>" > /var/lib/downloader/dlbasket
cat url_list.txt > /var/lib/downloader/dlbasket

Watch the files in the screen window as they are being downloaded.

Detach from the screen session by hitting the following:

Ctrl-a   d

Re-attach to the session by running:

screen -r

Note that you do not need to be attached to the screen session in order to add URLs.

Feeding the basket with URLs remotely

Assuming that a SSH server is running on the machine that runs rsgetd.sh, you can feed URLs to it by running the following from a remote machine:

ssh downloader@server.example.org cat \> /var/lib/downloader/dlbasket

Note that the > needs to be escaped so that it is considered as part of the command that will be executed on the remote server.

Now, feel free to add as many URLs as you like. After you hit the [Enter] key the url will be added to the download queue. When you are finished, just press Ctrl-D to end the URL submission.

Conclusion

This article provides all the information you need in order to use wget or curl to download files from your RapidShare Premium account. Also, information on how to set up a service that will assist you in order to commence downloads on your home server from a remote location has been covered.

The same information applies in all cases that wget and curl need to be used with websites that use cookie-based authentication.

Related Articles

• rsapiget downloads files using the new Rapidshare API
• Check Server HTTP Headers with CURL
• Mass download
• Partition images with Partimage and Partimaged
• Script for Apache Error Report

Related Articles

• dircproxy IRC Proxy
• Creative Commons v3.0 Licenses Launched
• CC Configurator plugin version 1.0 is out!
• Bot-Allow-Content and CC-Configurator plugin updates
• Best Practices of Software Licensing

…and I run across dircproxy, which seems to be exactly what I need. dircproxy is an IRC Proxy Server. When the IRC client, eg XChat, connects to the actual IRC server through dircproxy for the first time, dircproxy creates a connection class for the proxied connection. Even if the user disconnects its IRC client, dircproxy will continue to log channel and private messages and other events. Whenever the IRC client reconnects through dircproxy and provides a pre-defined password, the previous session is resumed and the client may get all the logged messages. So, to sum it up, the advantages of using dircproxy are:

1. Even if you disconnect the IRC client, no private or channel messages are lost.
2. You can connect through your IRC proxy with any client and from any workstation and see all the messages you have missed.

Besides, being always connected to an IRC network involves some risks, so one might wonder if dircproxy works with anonymous networks like tor. As it is stated in the FAQ, it works.

Definitely, very interesting software!

So, the task I need this for (message logging) does not make dircproxy much different than an IRC bot. However, I hope that the use of dircproxy is completely transparent, so I will not have to ask for any special permissions in order to connect to the IRC network through it.

I have only read the information that is available on the dircproxy website. I have not actually tested the software yet. I’ll report back when I do.

Related Articles

• BIP IRC Proxy
• How secure is the TOR network for everyday internet browsing?
• Send a text file as SMS with a Sony-Ericsson mobile
• Setup the SSH server to use keys for authentication
• Netcat – a couple of useful examples