<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	xmlns:creativeCommons="http://backend.userland.com/creativeCommonsRssModule"
>
<channel>
	<title>G-Loaded Journal &#187; Programming</title>
	<atom:link href="http://www.g-loaded.eu/tag/programming/feed/" rel="self" type="application/rss+xml" />
	<link>http://www.g-loaded.eu</link>
	<description>An open-source software and technology related journal</description>
	<lastBuildDate>Mon, 05 Dec 2011 19:55:24 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<creativeCommons:license>http://creativecommons.org/licenses/by-nc-sa/3.0/</creativeCommons:license>
		<item>
		<title>Issues with the feeds are now resolved</title>
		<link>http://www.g-loaded.eu/2010/03/28/issues-with-the-feeds-are-now-resolved/</link>
		<comments>http://www.g-loaded.eu/2010/03/28/issues-with-the-feeds-are-now-resolved/#comments</comments>
		<pubDate>Sun, 28 Mar 2010 06:06:46 +0000</pubDate>
		<dc:creator>George Notaras</dc:creator>
				<category><![CDATA[Linux]]></category>
		<category><![CDATA[Administration]]></category>
		<category><![CDATA[Errors]]></category>
		<category><![CDATA[PHP]]></category>
		<category><![CDATA[Programming]]></category>
		<category><![CDATA[Servers]]></category>
		<category><![CDATA[Snippet]]></category>
		<category><![CDATA[System]]></category>
		<category><![CDATA[Web Applications]]></category>
		<category><![CDATA[Wordpress]]></category>
		<guid isPermaLink="false">http://www.g-loaded.eu/?p=1634</guid>
		<description><![CDATA[This is just a quick notice that during the last five days there was a problem with the website feeds. The web server returned a 500 Internal Server Error to almost all requests for /feed/ URLs. Also, there was a big increase of the server&#8217;s CPU load behind the scenes, which was caused by php-cgi [...]]]></description>
			<content:encoded><![CDATA[<p>This is just a quick notice that during the last five days there was a problem with the website feeds. The web server returned a <em>500 Internal Server Error</em> to almost all requests for <code>/feed/</code> URLs. Also, there was a big increase of the server&#8217;s <strong>CPU load</strong> behind the scenes, which was caused by <em>php-cgi</em> processes. It turned out that a custom plugin (unpublished) I had written for WordPress in the past was the cause of all the trouble, but, before coming to that conclusion, the fact that the problems had started without me doing anything on the blog during the last weeks together with a series of other random observations led me do a thorough examination of the whole server just in case.<br />
<span id="more-1634"></span><br />
So, after spending two days in front of a Linux terminal, I came to the conclusion that the following code had been problematic from the beginning and it was like a time-bomb for which the time had come to cause issues to the web site and the server that hosted it.</p>
<div class="wp_syntax"><div class="code"><pre class="php" style="font-family:monospace;"><span style="color: #000000; font-weight: bold;">function</span> rsse_add_related_posts<span style="color: #009900;">&#40;</span><span style="color: #000088;">$PostBody</span><span style="color: #009900;">&#41;</span> <span style="color: #009900;">&#123;</span>
  <span style="color: #b1b100;">if</span> <span style="color: #009900;">&#40;</span><span style="color: #339933;">!</span>rsse_verify_feed<span style="color: #009900;">&#40;</span><span style="color: #009900;">&#41;</span><span style="color: #009900;">&#41;</span> <span style="color: #009900;">&#123;</span> <span style="color: #b1b100;">return</span> <span style="color: #000088;">$PostBody</span><span style="color: #339933;">;</span> <span style="color: #009900;">&#125;</span>
  <span style="color: #b1b100;">if</span> <span style="color: #009900;">&#40;</span><span style="color: #990000;">function_exists</span><span style="color: #009900;">&#40;</span><span style="color: #0000ff;">'yarpp_related'</span><span style="color: #009900;">&#41;</span><span style="color: #009900;">&#41;</span> <span style="color: #009900;">&#123;</span>
    <span style="color: #000088;">$PostBody</span> <span style="color: #339933;">.=</span> yarpp_related<span style="color: #009900;">&#40;</span><span style="color: #990000;">array</span><span style="color: #009900;">&#40;</span><span style="color: #0000ff;">'post'</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">,</span><span style="color: #990000;">array</span><span style="color: #009900;">&#40;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">,</span><span style="color: #009900; font-weight: bold;">false</span><span style="color: #339933;">,</span><span style="color: #009900; font-weight: bold;">false</span><span style="color: #339933;">,</span><span style="color: #0000ff;">'rss'</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
  <span style="color: #009900;">&#125;</span>
  <span style="color: #b1b100;">return</span> <span style="color: #000088;">$PostBody</span><span style="color: #339933;">;</span>
<span style="color: #009900;">&#125;</span>
&nbsp;
add_filter<span style="color: #009900;">&#40;</span><span style="color: #0000ff;">'the_content'</span><span style="color: #339933;">,</span> <span style="color: #0000ff;">'rsse_add_related_posts'</span><span style="color: #339933;">,</span> <span style="color: #cc66cc;">252</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span></pre></div></div>
<p>This code used a function of another plugin to attach a &#8216;<em>related posts</em>&#8216; section at the end of the full content of each feed entry. As soon as I commented out the above filter and started attaching the related-posts section to the feed content through the <a href="http://mitcho.com/code/yarpp/">YARPP</a> plugin&#8217;s administration interface, everything was normal again. No more php-cgi processes timing-out or over-consuming the CPU cycles.</p>
<p>The following graph shows clearly the problematic period:</p>
<div id="attachment_1639" class="wp-caption aligncenter" style="width: 650px"><a href="http://www.g-loaded.eu/wp-content/uploads/phpcgi-cpu-time.png"><img src="http://www.g-loaded.eu/wp-content/uploads/phpcgi-cpu-time.png" alt="php-cgi processes consuming the CPU" title="phpcgi-cpu-time" width="640" class="size-full wp-image-1639" /></a><p class="wp-caption-text">php-cgi processes consuming the CPU</p></div>
<p>The httpd error_log included many errors like the following:</p>
<pre class="codesnp">
[...]
Allowed memory size of 33554432 bytes exhausted (tried to allocate 18893 bytes)
[...]
mod_fcgid: process ... exit(communication error), terminated by calling exit(), return code: 1
[...]
[warn] (104)Connection reset by peer: mod_fcgid: read data from fastcgi server error.
[...]
[warn] mod_fcgid: stderr: PHP Fatal error:  Allowed memory size of 33554432 bytes exhausted (tried to allocate 4864 bytes)
[...]
Premature end of script headers: index.php
[...]
</pre>
<p>Note that these errors might appear in several occasions.</p>
<p>In my case, finding that faulty piece of code was a huge relief, I can tell you.</p>
<div class="cc-block"><em><a href="http://www.g-loaded.eu/2010/03/28/issues-with-the-feeds-are-now-resolved/">Issues with the feeds are now resolved</a></em>, unless otherwise expressly stated, is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/3.0/">Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License</a>. Terms and conditions beyond the scope of this license may be available at <a href="http://www.g-loaded.eu/about/disclaimer-and-license/">www.g-loaded.eu</a>.</div>
<h4>Related Articles</h4>
<ul><li><a href="http://www.g-loaded.eu/2007/01/12/wordpress-206-feed-issues-resolved/" rel="bookmark">WordPress 2.0.6 &#8211; Feed Issues Resolved</a></li>
<li><a href="http://www.g-loaded.eu/2009/11/19/g-loaded-feeds/" rel="bookmark">G-Loaded Feeds</a></li>
<li><a href="http://www.g-loaded.eu/2009/10/08/redmine-deployment-delayed/" rel="bookmark">Redmine deployment delayed</a></li>
<li><a href="http://www.g-loaded.eu/2007/07/12/issue-addressed-author-feeds-deliver-full-content/" rel="bookmark">Issue addressed: Author feeds deliver full content</a></li>
<li><a href="http://www.g-loaded.eu/2008/10/17/status-of-content-availability-via-feeds/" rel="bookmark">Status of content availability via feeds</a></li></ul>]]></content:encoded>
			<wfw:commentRss>http://www.g-loaded.eu/2010/03/28/issues-with-the-feeds-are-now-resolved/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
	<creativeCommons:license>http://creativecommons.org/licenses/by-nc-sa/3.0/</creativeCommons:license>
	</item>
		<item>
		<title>Backslashes inside pre HTML tags in WordPress</title>
		<link>http://www.g-loaded.eu/2007/11/04/backslashes-inside-pre-html-tags-in-wordpress/</link>
		<comments>http://www.g-loaded.eu/2007/11/04/backslashes-inside-pre-html-tags-in-wordpress/#comments</comments>
		<pubDate>Sun, 04 Nov 2007 11:33:38 +0000</pubDate>
		<dc:creator>George Notaras</dc:creator>
				<category><![CDATA[Web Applications]]></category>
		<category><![CDATA[Maintenance]]></category>
		<category><![CDATA[Programming]]></category>
		<category><![CDATA[Publishing]]></category>
		<category><![CDATA[Resolved]]></category>
		<category><![CDATA[Tips]]></category>
		<category><![CDATA[Wordpress]]></category>
		<guid isPermaLink="false">http://www.g-loaded.eu/2007/11/04/backslashes-inside-pre-html-tags-in-wordpress/</guid>
		<description><![CDATA[Today, I noticed that it is no longer required to escape the backslash (\), known as the &#8220;escape character&#8221; on *nix systems, inside the pre HTML tag in order not to be removed by WordPress&#8217; HTML filters. This bug has lived long enough to be considered as a WordPress feature, but the devs have suddenly [...]]]></description>
			<content:encoded><![CDATA[<p>Today, I noticed that it is no longer required to <em>escape the backslash</em> (<code>\</code>), known as the &#8220;<em>escape character</em>&#8221; on *nix systems, inside the <strong>pre HTML tag</strong> in order <strong>not to be</strong> removed by WordPress&#8217; HTML filters. This bug has lived long enough to be considered as a WordPress feature, but the devs have suddenly decided to address it. So, no more pain when using the <em>escape character</em> inside pre tags. Good!! But, what happens to the code that has already been published with escaped backslashes? WordPress is one of those free software projects which leave some really difficult homework to their users from time to time. So, here follows my solution.<br />
<span id="more-456"></span><br />
After you have upgraded to the latest WordPress version, 2.3.1 at the time of writing, create a <strong>dump</strong> of your WordPress database:</p>
<pre class="console">mysqldump -u wpuser -p --opt --databases mywpdb > mywpdb.sql</pre>
<p>After the database has been dumped, those backslash pairs in your posts&#8217; data would appear as four (4) backslashes in the MySQL database dump. This is because this file contains the <strong>raw strings</strong> of your data.</p>
<p>To get an idea about what a <em>raw string</em> is, launch the <strong>python interpreter</strong> and test the following:</p>
<pre class="console">
>>> "string with escaped backslash \\"
'string with escaped backslash \\'
>>> r"raw string with escaped backslash \\"
'raw string with escaped backslash \\\\'
</pre>
<p><strong>All the above python-thing is not required</strong> at all, but it is good to know what really happens. As you can see, the escaped backslash is represented by <strong>four backslashes</strong> in the raw string and this is how it has been written by mysqldump in the <code>mywpdb.sql</code> file.</p>
<p>All you have to do in order to <strong>repair</strong> all that code, while leaving all non-escaped backslashes in place, is to use a good text-stream editor, like <strong>sed</strong> in the way presented below.</p>
<p>Assuming that you have kept a <strong>backup of the dump</strong>, use sed to repair it <strong>in-place</strong>:</p>
<pre class="console">
sed -i 's/\\\\/\\/g' mywpdb.sql
</pre>
<p>You can now <strong>import</strong> the repaired dump back into the MySQL server and replace your data with the corrected one.</p>
<pre class="console">mysql -u wpuser -p mywpdb &lt; mywpdb.sql</pre>
<p>Your previously escaped backslashes should now appear as single backslashes in your posts, but displayed correctly by WordPress.</p>
<p>I haven&#8217;t noticed any problems after performing the above operation. It might not work for you though. Use at your own risk.</p>
<p>If I miss anything, please let me know.</p>
<div class="cc-block"><em><a href="http://www.g-loaded.eu/2007/11/04/backslashes-inside-pre-html-tags-in-wordpress/">Backslashes inside pre HTML tags in WordPress</a></em>, unless otherwise expressly stated, is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/3.0/">Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License</a>. Terms and conditions beyond the scope of this license may be available at <a href="http://www.g-loaded.eu/about/disclaimer-and-license/">www.g-loaded.eu</a>.</div>
<h4>Related Articles</h4>
<ul><li><a href="http://www.g-loaded.eu/2008/03/31/how-to-fix-the-wordpress-databases-character-set-issue/" rel="bookmark">How to fix the WordPress database&#8217;s character-set issue</a></li>
<li><a href="http://www.g-loaded.eu/2006/05/05/modifying-your-name-in-the-wordpress-comments/" rel="bookmark">Modifying Your Name In The WordPress Comments</a></li>
<li><a href="http://www.g-loaded.eu/2007/09/27/add-meta-tags-and-wordpress-23/" rel="bookmark">Add-Meta-Tags and WordPress 2.3</a></li>
<li><a href="http://www.g-loaded.eu/2006/01/05/add-meta-tags-wordpress-plugin/" rel="bookmark">Add-Meta-Tags WordPress Plugin</a></li>
<li><a href="http://www.g-loaded.eu/2007/01/31/wordpress-meta-tags-plugin-stable-release/" rel="bookmark">WordPress Meta Tags plugin stable release</a></li></ul>]]></content:encoded>
			<wfw:commentRss>http://www.g-loaded.eu/2007/11/04/backslashes-inside-pre-html-tags-in-wordpress/feed/</wfw:commentRss>
		<slash:comments>4</slash:comments>
	<creativeCommons:license>http://creativecommons.org/licenses/by-nc-sa/3.0/</creativeCommons:license>
	</item>
		<item>
		<title>Dictionary Lookups Anywhere</title>
		<link>http://www.g-loaded.eu/2007/10/18/dictionary-lookups-anywhere/</link>
		<comments>http://www.g-loaded.eu/2007/10/18/dictionary-lookups-anywhere/#comments</comments>
		<pubDate>Thu, 18 Oct 2007 17:40:22 +0000</pubDate>
		<dc:creator>George Notaras</dc:creator>
				<category><![CDATA[Linux]]></category>
		<category><![CDATA[Desktop]]></category>
		<category><![CDATA[Dictionaries]]></category>
		<category><![CDATA[GNOME]]></category>
		<category><![CDATA[HOWTO]]></category>
		<category><![CDATA[Productivity]]></category>
		<category><![CDATA[Programming]]></category>
		<category><![CDATA[Resolved]]></category>
		<category><![CDATA[Scripts]]></category>
		<category><![CDATA[Snippet]]></category>
		<guid isPermaLink="false">http://www.g-loaded.eu/2007/10/18/dictionary-lookups-anywhere/</guid>
		<description><![CDATA[I consider ditionaries, either in printed or electronic form, as one of the most useful sources of valuable information and an indispensable companion when reading. Lately, I&#8217;ve been reading lots of stuff, which involved scientific terminology in the English language, so it was very often required to lookup words on various online dictionaries. I did [...]]]></description>
			<content:encoded><![CDATA[<p>I consider ditionaries, either in printed or electronic form, as one of the most useful sources of valuable information and an indispensable companion when reading. Lately, I&#8217;ve been reading lots of stuff, which involved scientific terminology in the English language, so it was very often required to lookup words on various online dictionaries. I did most of the reading using <a href="http://www.gnome.org/projects/epiphany/">epiphany</a> or <a href="http://www.gnome.org/projects/evince/">evince</a>. Epiphany has a very handy feature which can send the selected text to a <a href="http://www.g-loaded.eu/2007/05/11/smart-bookmarks-in-epiphany/">smart bookmark</a> or even to <a href="http://en.wikipedia.org/wiki/Gnome-dictionary">gnome-dictionary</a>. Unfortunately, evince lacks such features, so I had to manually copy and paste words to the web browser in order to look them up on the web. This, combined with evince&#8217;s performance issues (version 0.6.0 &#8211; Fedora Core 6), led me to look for a more universal method of looking up words or phrases, regardless of the document viewer. And, usually, when your sole ambition becomes to overcome an annoying situation, a solution is on the way.<br />
<span id="more-446"></span><br />
What I needed was the following:</p>
<ol>
<li>Select some text with the mouse (easy :P)</li>
<li>Trigger a dictionary lookup for the selected text.</li>
<li>Get the word&#8217;s definitions in a browser window.</li>
</ol>
<p>After some research on the web, I came up with a decent solution. I discovered that when some text is selected under X, it is temporarily stored in a location called &#8220;<em>Primary Clipboard</em>&#8220;. The text could be somehow retrieved from that location and could be sent to an online dictionary by pressing a specific keyboard combination. This is totally independent of the application which the text had been selected from. This method also makes <strong>dictionary lookups as easy and fast as it can get</strong>.</p>
<p>But, this was just in theory. I needed a way of retrieving the selected text from the primary clipboard programmatically. This was the most difficult part, as X or GTK programming is out of my league&#8230;</p>
<p>Fortunately, I was told that this exact functionality exists in the <a href="http://raphael.slinckx.net/deskbar/">deskbar applet</a>. The user can select some text, then press <strong>Alt-F3</strong> and the deskbar applet displays a list of pre-defined places (epiphany smart bookmarks included) to search for the selected text, which also includes gnome-dictionary in order to look up the word on a dictionary server.</p>
<p>Using this deskbar applet&#8217;s functionality for the dictionary lookups could be the perfect solution. But, on the other hand, being able to specify different key combinations for different dictionaries and not having to pick a search location from the deskbar&#8217;s list would be more than perfect. Grepping into deskbar applet&#8217;s source code I soon found what I was looking for in the first place: that is <em>the way to programmatically retrieve the selected text from the primary clipboard</em>. Of course, I shamelessly stole that small piece of code :P</p>
<p>Provided that some text has been selected, its retrieval is possible with the following python snippet:</p>
<div class="wp_syntax"><div class="code"><pre class="python" style="font-family:monospace;"><span style="color: #ff7700;font-weight:bold;">import</span> gtk
clipboard = gtk.<span style="color: black;">clipboard_get</span><span style="color: black;">&#40;</span>selection = <span style="color: #483d8b;">&quot;PRIMARY&quot;</span><span style="color: black;">&#41;</span>
text = clipboard.<span style="color: black;">wait_for_text</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
<span style="color: #ff7700;font-weight:bold;">print</span> text</pre></div></div>
<p>The rest was easy. What was needed was a key combination to trigger the execution of the above script. Unfortunately, a custom command cannot be set in the <strong>gnome-keybinding-properties</strong> application. So, a workaround was needed even for that!! After some research, it turned out that one can set a <strong>custom command</strong> and assign a keyboard shortcut for it in GNOME&#8217;s window manager, <a href="http://en.wikipedia.org/wiki/Metacity">metacity</a>.</p>
<p>So, finally, it worked! Read on for detailed info&#8230;</p>
<h4>How to set things up</h4>
<p>First of all, create a small script:</p>
<div class="wp_syntax"><div class="code"><pre class="python" style="font-family:monospace;"><span style="color: #808080; font-style: italic;">#! /usr/bin/env python</span>
<span style="color: #808080; font-style: italic;"># -*- coding: utf-8 -*-</span>
&nbsp;
command = <span style="color: #483d8b;">&quot;epiphany -n http://en.wiktionary.org/wiki/%s&quot;</span>
<span style="color: #808080; font-style: italic;">#command = &quot;epiphany -n http://dictionary.reference.com/search?q=%s&quot;</span>
&nbsp;
<span style="color: #ff7700;font-weight:bold;">import</span> gtk
<span style="color: #ff7700;font-weight:bold;">import</span> <span style="color: #dc143c;">os</span>
<span style="color: #ff7700;font-weight:bold;">from</span> <span style="color: #dc143c;">urllib</span> <span style="color: #ff7700;font-weight:bold;">import</span> quote
&nbsp;
clipboard = gtk.<span style="color: black;">clipboard_get</span><span style="color: black;">&#40;</span>selection = <span style="color: #483d8b;">&quot;PRIMARY&quot;</span><span style="color: black;">&#41;</span>
text = clipboard.<span style="color: black;">wait_for_text</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
text = text.<span style="color: black;">strip</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
<span style="color: #ff7700;font-weight:bold;">if</span> text:
	text = quote<span style="color: black;">&#40;</span>text<span style="color: black;">&#41;</span>	<span style="color: #808080; font-style: italic;"># This is basically for multiword selections</span>
	<span style="color: #dc143c;">os</span>.<span style="color: black;">popen</span><span style="color: black;">&#40;</span>command <span style="color: #66cc66;">%</span> text<span style="color: black;">&#41;</span></pre></div></div>
<p>Save the script to a directory of your choice. The rest assumes that the script has been saved to:</p>
<p><strong>/home/johndoe/bin/selectionlookup.py</strong></p>
<p>What this does is to retrieve the selected text from the &#8220;<em>primary clipboard</em>&#8221; and execute the specified command. In this case, the command is actually a system call to the epiphany browser which is instructed to load a page with the possible definitions for the selected text. The English version of <a href="http://en.wiktionary.org/">Wiktionary</a> or <a href="http://dictionary.reference.com/">Dictionary.com</a> (whichever is uncommented) is used for the lookup. Note that <strong>multiword selections are supported</strong> by both the script above and the aforementioned online dictionaries.</p>
<p><a name="customkeybinding"></a><br />
Second of all, a <strong>global keyboard shortcut</strong> needs to be set in metacity for the reasons I explained previously.</p>
<p>First, the custom command needs to be set. Open the <strong>gconf-editor</strong> and browse to the key:</p>
<pre class="codesnp">
/apps/metacity/keybinding_commands/command_1
</pre>
<p>Of course use any command key you wish. Here <code>command_1</code> is used. Set the following value:</p>
<pre class="codesnp">
python /home/johndoe/bin/selectionlookup.py
</pre>
<p>Then set up the keybinding for that command. Browse to the key:</p>
<pre class="codesnp">
/apps/metacity/global_keybindings/run_command_1
</pre>
<p>And set the combination (the following is an example):</p>
<pre class="codesnp">
&lt;control&gt;&lt;alt&gt;g
</pre>
<p>Test it by selecting some text anywhere and by pressing <code>Ctrl-Alt-g</code>. A new tab will appear in the browser window with a page with the selected text&#8217;s definitions (if any).</p>
<h4>Appendix I: Using a dictionary server</h4>
<p>Instead of looking the selected text up on a web-based dictionary, you can retrieve definitions from a dictionary server. Adjust the script accordingly. For example:</p>
<div class="wp_syntax"><div class="code"><pre class="python" style="font-family:monospace;"><span style="color: #808080; font-style: italic;">#! /usr/bin/env python</span>
<span style="color: #808080; font-style: italic;"># -*- coding: utf-8 -*-</span>
&nbsp;
command = <span style="color: #483d8b;">&quot;echo 'DEFINE * <span style="color: #000099; font-weight: bold;">\&quot;</span>%s<span style="color: #000099; font-weight: bold;">\&quot;</span>' | nc dict.org 2628 | zenity --text-info --title='Dict Lookup' --width=500 --height=500&quot;</span>
&nbsp;
<span style="color: #ff7700;font-weight:bold;">import</span> gtk
<span style="color: #ff7700;font-weight:bold;">import</span> <span style="color: #dc143c;">os</span>
&nbsp;
clipboard = gtk.<span style="color: black;">clipboard_get</span><span style="color: black;">&#40;</span>selection = <span style="color: #483d8b;">&quot;PRIMARY&quot;</span><span style="color: black;">&#41;</span>
text = clipboard.<span style="color: black;">wait_for_text</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
text = text.<span style="color: black;">strip</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
<span style="color: #ff7700;font-weight:bold;">if</span> text:
	<span style="color: #dc143c;">os</span>.<span style="color: black;">popen</span><span style="color: black;">&#40;</span>command <span style="color: #66cc66;">%</span> text<span style="color: black;">&#41;</span></pre></div></div>
<p>The following happens when the keyboard shortcut is pressed:</p>
<ol>
<li>The following dictionary protocol specific command: <strong>DEFINE * &#8220;selectedtext&#8221;</strong> is piped to netcat (see some <a href="http://www.g-loaded.eu/2006/11/06/netcat-a-couple-of-useful-examples/">netcat examples</a>).</li>
<li>Netcat opens a connection to the <code>dict.org</code> dictionary server, sends the command and retrieves the results.</li>
<li>The results are piped to <a href="http://freshmeat.net/projects/zenity">zenity</a> and are displayed in an information textbox.</li>
</ol>
<p>Obviously, in order to successfully run the above command you will need <strong>netcat</strong> (<strong>nc</strong>) and <strong>zenity</strong>.</p>
<h4>Final Thoughts</h4>
<p>A whole dictionary application can be built around the selected text that is stored at the primary clipboard. The above is the fastest and most effective way of retrieving word/phrase definitions from any possible dictionary. It is independent of the application from which the text is selected and no time is wasted in meaningless copy-pasting. If you know a better way of looking up words, please let me know.</p>
<div class="cc-block"><em><a href="http://www.g-loaded.eu/2007/10/18/dictionary-lookups-anywhere/">Dictionary Lookups Anywhere</a></em>, unless otherwise expressly stated, is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/3.0/">Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License</a>. Terms and conditions beyond the scope of this license may be available at <a href="http://www.g-loaded.eu/about/disclaimer-and-license/">www.g-loaded.eu</a>.</div>
<h4>Related Articles</h4>
<ul><li><a href="http://www.g-loaded.eu/2006/05/17/epiphany-python-console-open-new-tab/" rel="bookmark">Epiphany Python Console &#8211; Open New Tab</a></li>
<li><a href="http://www.g-loaded.eu/2006/03/16/dictexpress/" rel="bookmark">DictExpress</a></li>
<li><a href="http://www.g-loaded.eu/2006/04/05/epiphany-browser-review/" rel="bookmark">Epiphany Browser Review</a></li>
<li><a href="http://www.g-loaded.eu/2006/09/23/use-python-to-get-the-web-page-data-in-epiphany/" rel="bookmark">Use Python to get the web page data in Epiphany</a></li>
<li><a href="http://www.g-loaded.eu/2007/05/11/smart-bookmarks-in-epiphany/" rel="bookmark">Smart Bookmarks in Epiphany</a></li></ul>]]></content:encoded>
			<wfw:commentRss>http://www.g-loaded.eu/2007/10/18/dictionary-lookups-anywhere/feed/</wfw:commentRss>
		<slash:comments>5</slash:comments>
	<creativeCommons:license>http://creativecommons.org/licenses/by-nc-sa/3.0/</creativeCommons:license>
	</item>
		<item>
		<title>Use wget or curl to download from RapidShare Premium</title>
		<link>http://www.g-loaded.eu/2007/09/15/use-wget-or-curl-to-download-from-rapidshare-premium/</link>
		<comments>http://www.g-loaded.eu/2007/09/15/use-wget-or-curl-to-download-from-rapidshare-premium/#comments</comments>
		<pubDate>Sat, 15 Sep 2007 08:55:17 +0000</pubDate>
		<dc:creator>George Notaras</dc:creator>
				<category><![CDATA[Computers]]></category>
		<category><![CDATA[Linux]]></category>
		<category><![CDATA[Windows]]></category>
		<category><![CDATA[BASH]]></category>
		<category><![CDATA[HOWTO]]></category>
		<category><![CDATA[Networking]]></category>
		<category><![CDATA[Productivity]]></category>
		<category><![CDATA[Programming]]></category>
		<category><![CDATA[Scripts]]></category>
		<category><![CDATA[Servers]]></category>
		<category><![CDATA[Shell]]></category>
		<category><![CDATA[Snippet]]></category>
		<guid isPermaLink="false">http://www.g-loaded.eu/2007/09/15/use-wget-or-curl-to-download-from-rapidshare-premium/</guid>
		<description><![CDATA[This article describes how to use the command line download managers wget and curl in order to download files from your Rapidshare Premium account.]]></description>
			<content:encoded><![CDATA[<p>The last days I needed to download a bunch of medical videos which have been uploaded to RapidShare by many other people. Although RapidShare (and all the other 1-click file-hosting services) is very convenient, it has some strict rules for free accounts, for example a guest has to wait for 120 seconds per 1 MB of downloaded data and &#8211; to make it worse &#8211; no download managers are allowed. Since &#8220;waiting&#8221; is not a game I like and since I intended to use either <a href="http://www.gnu.org/software/wget/">wget</a> or <a href="http://curl.haxx.se/">curl</a> to download the files, I decided to sign up for a <a href="http://www.rapidshare.com/">RapidShare</a> Premium account and then figure out how to use the aforementioned tools. Fortunately, registered users are permitted to use download managers and, as you will read in the following article, the Linux command line downloaders work flawlessly with a Premier account.<br />
<span id="more-443"></span></p>
<h4>Theory</h4>
<p>Rapidshare uses cookie-based authentication. This means that every time you log into the service, a cookie containing information which identifies you as a registered user is stored in your browser&#8217;s cookie cache. Both <code>wget</code> and <code>curl</code> support saving and loading cookies, so before using them to download any files, you should save such a cookie. Having done this, then the only required action in order download from RapidShare is to load the cookie, so that wget or curl can use it to authenticate you on the RapidShare server. This is pretty much the same you would do with a graphical download manager. The difference now is that you do it on the command line.</p>
<p>Below you will find examples about how to perform these actions using both wget and curl.</p>
<p><strong>IMPORTANT</strong>: Please note that in order to use these command-line utilities or any other download managers with RapidShare, you will have to check the <strong>Direct Downloads option</strong> in your account&#8217;s <strong>options page</strong>.</p>
<h4>Save your RapidShare Premium Account Cookie</h4>
<p>Saving your RapidShare cookie is a procedure that needs to be done <strong>once</strong>.</p>
<p>The login page is located at:</p>
<pre class="codesnp">https://ssl.rapidshare.com/cgi-bin/premiumzone.cgi</pre>
<p>The login form requires two fields: <strong>login</strong> and <strong>password</strong>. These are pretty self-explanatory.</p>
<p>In the following examples, the RapidShare username is shown as <strong>USERNAME</strong> and the password as <strong>PASSWORD</strong>.</p>
<h5>Using wget</h5>
<p>In order to save your cookie using wget, run the following:</p>
<pre class="console">
wget \
    --save-cookies ~/.cookies/rapidshare \
    --post-data "login=USERNAME&#038;password=PASSWORD" \
    -O - \
    https://ssl.rapidshare.com/cgi-bin/premiumzone.cgi \
    > /dev/null
</pre>
<p><strong>&#8211;save-cookies</strong> : Saves the cookie to a file called <code>rapidshare</code> under the <code>~/.cookies</code> directory (let&#8217;s assume that you store your cookies there)<br />
<strong>&#8211;post-data</strong> : is the POST payload of the request. In other words it contains the data you would enter in the login form.<br />
<strong>-O -</strong> : downloads the HTML data to the standard output. Since the above command is run only in order to obtain the cookie, this option prints the HTML data to stdout (Standard Output) and then discards it by redirecting stdout to <code>/dev/null</code>. If you don&#8217;t do this, wget will save the HTML data in a file called <code>premiumzone.cgi</code> in the current directory. This is just the Rapidshare HTML page, which is absolutely not needed.</p>
<h5>Using curl</h5>
<p>In order to save your cookie using curl, run the following:</p>
<pre class="console">
curl \
    --cookie-jar ~/.cookies/rapidshare \
    --data "login=USERNAME&#038;password=PASSWORD" \
    https://ssl.rapidshare.com/cgi-bin/premiumzone.cgi \
    > /dev/null
</pre>
<p><strong>&#8211;cookie-jar</strong> : Saves the cookie to a file called <code>rapidshare</code> under the <code>~/.cookies</code> directory (it has been assumed previously that cookies are stored there)<br />
<strong>&#8211;data</strong> : contains the data you would enter in the login form.<br />
Curl prints the downloaded page data to stdout by default. This is discarded by sending it to <code>/dev/null</code>.</p>
<h4>Download files using your RapidShare Premium Account Cookie</h4>
<p>Having saved your cookie, downloading files from RapidShare is as easy as telling wget/curl to load the cookie everytime you use them to download a file.</p>
<h5>Downloading with wget</h5>
<p>In order to download a file with wget, run the following:</p>
<pre class="console">
wget -c --load-cookies ~/.cookies/rapidshare &lt;URL&gt;
</pre>
<p><strong>-c </strong>: this is used in order to resume downloading of the file if it already exists in the current directory and is incomplete.<br />
<strong>&#8211;load-cookies</strong> : loads your cookie.</p>
<h5>Downloading with curl</h5>
<p>In the same manner, in order to download a file with curl, run the following:</p>
<pre class="console">
curl -L -O --cookie ~/.cookies/rapidshare &lt;URL&gt;
</pre>
<p><strong>-L</strong> : Follows all redirections until the final destination page is found. This switch is almost always required as curl won&#8217;t follow redirects by default (read about how to check the server <a href="http://www.g-loaded.eu/2006/10/06/check-server-http-headers-with-curl/">http headers</a> with curl).<br />
<strong>-O</strong> : By using this switch you instruct curl to save the downloaded data to a file in the current directory. The filename of the remote file is used. This switch is also required or else curl will print the data to stdout, which is something you won&#8217;t probably like.<br />
<strong>&#8211;cookie</strong> : loads your Rapidshare account&#8217;s cookie.</p>
<h4>Setting up a Download Server</h4>
<p>Although most users would be satisfied with the above, I wouldn&#8217;t be surprised if you would want to go a bit further and try to setup a little service for your downloading pleasure. Here is a very primitive implementation of such a service. All you will need is standard command line tools.</p>
<p>This primitive server consists of the following:</p>
<ol>
<li>A <a href="http://en.wikipedia.org/wiki/Named_pipe">named pipe</a>, called &#8220;<strong>dlbasket</strong>&#8220;. You will feed the server with URLs through this pipe. Another approach would be to use a listening TCP socket with <a href="http://www.g-loaded.eu/2006/11/06/netcat-a-couple-of-useful-examples/">NetCat</a>.</li>
<li>A script, which, among others, contains the main server loop. This loop reads one URL at a time from dlbasket and starts a wget/curl process in order to download the file. If dlbasket is empty, the server should just stay there waiting.</li>
</ol>
<p>So, in short, the service would be the following:</p>
<pre class="codesnp">
cat &lt;&gt; dlbasket | ( while ... done )
</pre>
<p>All credit for the &#8220;<strong>cat &lt;&gt; dlbasket |</strong>&#8221; magic goes to <strong><em>Zart</em></strong>, who kindly helped me out at the #fedora IRC channel.</p>
<p>So, let&#8217;s create that service. The following assume that a user named &#8220;<strong>downloader</strong>&#8221; exists in the system and the home directory is <code>/var/lib/downloader/</code>. Of course you can set this up as you like, but make sure you adjust the following commands and the script&#8217;s configuration options accordingly.</p>
<p>First, create the named pipe:</p>
<pre class="console">
mkfifo -m 0700 /var/lib/downloader/dlbasket
</pre>
<p>If it does not exist, create a <code>bin</code> directory in the user&#8217;s home:</p>
<pre class="console">
mkdir -p /var/lib/downloader/bin
</pre>
<p>Also, create a directory where the downloaded files will be saved:</p>
<pre class="console">
mkdir -p /var/lib/downloader/downloads
</pre>
<p>The following is a quick and dirty script I wrote which actually implements the service. Save it as <strong>rsgetd.sh</strong> inside the user&#8217;s <code>bin</code> directory:</p>
<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #666666; font-style: italic;">#! /usr/bin/env bash  </span>
&nbsp;
<span style="color: #666666; font-style: italic;">#  rsgetd.sh - Download Service</span>
&nbsp;
<span style="color: #666666; font-style: italic;">#  Version 0.3</span>
&nbsp;
<span style="color: #666666; font-style: italic;">#  Copyright (C) 2007 George Notaras (http://www.g-loaded.eu/)</span>
<span style="color: #666666; font-style: italic;">#</span>
<span style="color: #666666; font-style: italic;">#  This program is free software; you can redistribute it and/or modify</span>
<span style="color: #666666; font-style: italic;">#  it under the terms of the GNU General Public License version 2 as</span>
<span style="color: #666666; font-style: italic;">#  published by the Free Software Foundation.</span>
<span style="color: #666666; font-style: italic;">#</span>
<span style="color: #666666; font-style: italic;">#  This program is distributed in the hope that it will be useful,</span>
<span style="color: #666666; font-style: italic;">#  but WITHOUT ANY WARRANTY; without even the implied warranty of</span>
<span style="color: #666666; font-style: italic;">#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the</span>
<span style="color: #666666; font-style: italic;">#  GNU General Public License for more details.</span>
&nbsp;
<span style="color: #666666; font-style: italic;"># Special thanks to 'Zart' from the #fedora channel on FreeNode</span>
&nbsp;
<span style="color: #666666; font-style: italic;"># v0.3 - Oct 31 2009:</span>
<span style="color: #666666; font-style: italic;"># - corrected error: &quot;./bin/rsgetd.sh: line 37: [: argument expected&quot;</span>
&nbsp;
<span style="color: #666666; font-style: italic;"># CONFIG START </span>
<span style="color: #007800;">HOMEDIR</span>=<span style="color: #ff0000;">&quot;/var/lib/downloader&quot;</span>
<span style="color: #007800;">DLBASKET</span>=<span style="color: #ff0000;">&quot;<span style="color: #007800;">$HOMEDIR</span>/dlbasket&quot;</span>
<span style="color: #007800;">DLDIR</span>=<span style="color: #ff0000;">&quot;<span style="color: #007800;">$HOMEDIR</span>/downloads/&quot;</span>
<span style="color: #007800;">LOGFILE</span>=<span style="color: #ff0000;">&quot;<span style="color: #007800;">$HOMEDIR</span>/.downloads_log&quot;</span>
<span style="color: #007800;">CACHEFILE</span>=<span style="color: #ff0000;">&quot;<span style="color: #007800;">$HOMEDIR</span>/.downloads_cache&quot;</span>
<span style="color: #007800;">LIMIT</span>=<span style="color: #ff0000;">&quot;25k&quot;</span>
<span style="color: #007800;">WGETBIN</span>=<span style="color: #ff0000;">&quot;/usr/bin/wget&quot;</span>
<span style="color: #666666; font-style: italic;"># Rapidshare Login Cookie  </span>
<span style="color: #007800;">RSCOOKIE</span>=<span style="color: #ff0000;">&quot;<span style="color: #007800;">$HOMEDIR</span>/cookies/.rapidshare&quot;</span>
<span style="color: #666666; font-style: italic;"># CONFIG END</span>
&nbsp;
<span style="color: #007800;">DATETIME</span>=<span style="color: #ff0000;">&quot;<span style="color: #780078;">`date '+%Y-%m-%d %H:%M:%S'`</span>&quot;</span>
&nbsp;
<span style="color: #c20cb9; font-weight: bold;">cat</span> <span style="color: #000000; font-weight: bold;">&amp;</span>lt;<span style="color: #000000; font-weight: bold;">&amp;</span>gt; <span style="color: #007800;">$DLBASKET</span> <span style="color: #000000; font-weight: bold;">|</span> <span style="color: #7a0874; font-weight: bold;">&#40;</span>
        <span style="color: #000000; font-weight: bold;">while</span> <span style="color: #c20cb9; font-weight: bold;">read</span> url ; <span style="color: #000000; font-weight: bold;">do</span>
                <span style="color: #666666; font-style: italic;"># First, check the cache if the file has been already downloaded</span>
                <span style="color: #000000; font-weight: bold;">if</span> <span style="color: #7a0874; font-weight: bold;">&#91;</span> <span style="color: #660033;">-f</span> <span style="color: #ff0000;">&quot;<span style="color: #007800;">$CACHEFILE</span>&quot;</span> <span style="color: #660033;">-a</span> <span style="color: #660033;">-n</span> <span style="color: #ff0000;">&quot;<span style="color: #007800;">$(grep -i $(basename $url)</span> <span style="color: #007800;">$CACHEFILE</span>)&quot;</span> <span style="color: #7a0874; font-weight: bold;">&#93;</span> ; <span style="color: #000000; font-weight: bold;">then</span>
                       <span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;<span style="color: #007800;">$DATETIME</span> File exists in cache. Already downloaded - Skipping: <span style="color: #007800;">$url</span>&quot;</span> <span style="color: #000000; font-weight: bold;">&gt;&gt;</span> <span style="color: #007800;">$LOGFILE</span>
                <span style="color: #000000; font-weight: bold;">else</span>
                        <span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;<span style="color: #007800;">$DATETIME</span> Starting with rate <span style="color: #007800;">$LIMIT</span>/s: <span style="color: #007800;">$url</span>&quot;</span> <span style="color: #000000; font-weight: bold;">&gt;&gt;</span> <span style="color: #007800;">$LOGFILE</span>
                        <span style="color: #000000; font-weight: bold;">if</span> <span style="color: #7a0874; font-weight: bold;">&#91;</span> $<span style="color: #7a0874; font-weight: bold;">&#40;</span><span style="color: #c20cb9; font-weight: bold;">expr</span> match <span style="color: #ff0000;">&quot;<span style="color: #007800;">$url</span>&quot;</span> <span style="color: #ff0000;">'[rapidshare.com]'</span><span style="color: #7a0874; font-weight: bold;">&#41;</span> = <span style="color: #000000;">1</span> <span style="color: #7a0874; font-weight: bold;">&#93;</span> ; <span style="color: #000000; font-weight: bold;">then</span>
                                <span style="color: #666666; font-style: italic;"># If it is a Rapidshare.com link, load the RS cookie </span>
                                <span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;RAPIDSHARE LINK&quot;</span>
                                <span style="color: #007800;">$WGETBIN</span> <span style="color: #660033;">-c</span> <span style="color: #660033;">--limit-rate</span>=<span style="color: #007800;">$LIMIT</span> <span style="color: #660033;">--directory-prefix</span>=<span style="color: #007800;">$DLDIR</span> <span style="color: #660033;">--load-cookies</span> <span style="color: #007800;">$RSCOOKIE</span> <span style="color: #007800;">$url</span>
                        <span style="color: #000000; font-weight: bold;">else</span>
                                <span style="color: #007800;">$WGETBIN</span> <span style="color: #660033;">-c</span> <span style="color: #660033;">--limit-rate</span>=<span style="color: #007800;">$LIMIT</span> <span style="color: #660033;">--directory-prefix</span>=<span style="color: #007800;">$DLDIR</span> <span style="color: #007800;">$url</span>
                        <span style="color: #000000; font-weight: bold;">fi</span>
                        <span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;<span style="color: #007800;">$DATETIME</span> Finished: <span style="color: #007800;">$url</span>&quot;</span> <span style="color: #000000; font-weight: bold;">&gt;&gt;</span> <span style="color: #007800;">$LOGFILE</span>
                        <span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #007800;">$url</span> <span style="color: #000000; font-weight: bold;">&gt;&gt;</span> <span style="color: #007800;">$CACHEFILE</span>
                <span style="color: #000000; font-weight: bold;">fi</span>
        <span style="color: #000000; font-weight: bold;">done</span> <span style="color: #7a0874; font-weight: bold;">&#41;</span>
&nbsp;
<span style="color: #7a0874; font-weight: bold;">exit</span> <span style="color: #000000;">0</span></pre></div></div>
<p>As you might have already noticed, two extra files are created inside the home directory: <code>.downloads_cache</code> and <code>.downloads_log</code>. The first contains a list of all the urls that have been downloaded. Each new download is checked against this list, so that the particular URL is not processed if the file has already been downloaded. The latter file is a usual logfile stating the start and end times of each download. Feel free to adjust the script to your needs.</p>
<p>Here is some info about how you should start the service:</p>
<p><strong>-1-</strong> You can simply start the script as a background process and then feed URLs to it. For example:</p>
<pre class="console">
rsgetd.sh &#038;
echo "&lt;URL&gt;" &gt; /var/lib/downloader/dlbasket
</pre>
<p><strong>-2-</strong> Use <a href="http://www.gnu.org/software/screen/">screen</a> in order to run the script in the background but still be able to see its output by connecting to a screen session. Although this is not a screen howto, here is an example:</p>
<p>Create a new screen session and attach to it:</p>
<pre class="console">
screen -S rs_downloads
</pre>
<p>While being in the session, run rsgetd.sh</p>
<pre class="console">
rsgetd.sh
</pre>
<p>From another terminal feed the download basket (dlbasket) with urls:</p>
<pre class="console">
echo "&lt;URL&gt;" &gt; /var/lib/downloader/dlbasket
cat url_list.txt &gt; /var/lib/downloader/dlbasket
</pre>
<p>Watch the files in the screen window as they are being downloaded.</p>
<p>Detach from the screen session by hitting the following:</p>
<pre class="codesnp">
Ctrl-a   d
</pre>
<p>Re-attach to the session by running:</p>
<pre class="console">
screen -r
</pre>
<p>Note that you do not need to be attached to the screen session in order to add URLs.</p>
<h5>Feeding the basket with URLs remotely</h5>
<p>Assuming that a SSH server is running on the machine that runs rsgetd.sh, you can feed URLs to it by running the following from a remote machine:</p>
<pre class="console">
ssh downloader@server.example.org cat \&gt; /var/lib/downloader/dlbasket
</pre>
<p>Note that the <strong>&gt;</strong> needs to be escaped so that it is considered as part of the command that will be executed on the remote server.</p>
<p>Now, feel free to <strong>add</strong> as many <strong>URLs</strong> as you like. After you hit the <strong>[Enter]</strong> key the url will be added to the download queue. When you are finished, just press <strong>Ctrl-D</strong> to end the URL submission.</p>
<h4>Conclusion</h4>
<p>This article provides all the information you need in order to use wget or curl to download files from your RapidShare Premium account. Also, information on how to set up a service that will assist you in order to commence downloads on your home server from a remote location has been covered.</p>
<p>The same information applies in all cases that wget and curl need to be used with websites that use cookie-based authentication.</p>
<div class="cc-block"><em><a href="http://www.g-loaded.eu/2007/09/15/use-wget-or-curl-to-download-from-rapidshare-premium/">Use wget or curl to download from RapidShare Premium</a></em>, unless otherwise expressly stated, is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/3.0/">Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License</a>. Terms and conditions beyond the scope of this license may be available at <a href="http://www.g-loaded.eu/about/disclaimer-and-license/">www.g-loaded.eu</a>.</div>
<h4>Related Articles</h4>
<ul><li><a href="http://www.g-loaded.eu/2010/10/04/rsapiget-download-rapidshare-api/" rel="bookmark">rsapiget downloads files using the new Rapidshare API</a></li>
<li><a href="http://www.g-loaded.eu/2006/10/06/check-server-http-headers-with-curl/" rel="bookmark">Check Server HTTP Headers with CURL</a></li>
<li><a href="http://www.g-loaded.eu/2005/11/10/mass-download/" rel="bookmark">Mass download</a></li>
<li><a href="http://www.g-loaded.eu/2006/01/06/partition-images-with-partimage-and-partimaged/" rel="bookmark">Partition images with Partimage and Partimaged</a></li>
<li><a href="http://www.g-loaded.eu/2010/03/28/script-apache-error-report/" rel="bookmark">Script for Apache Error Report</a></li></ul>]]></content:encoded>
			<wfw:commentRss>http://www.g-loaded.eu/2007/09/15/use-wget-or-curl-to-download-from-rapidshare-premium/feed/</wfw:commentRss>
		<slash:comments>4</slash:comments>
	<creativeCommons:license>http://creativecommons.org/licenses/by-nc-sa/3.0/</creativeCommons:license>
	</item>
		<item>
		<title>File and Directory diff in color in Midnight Commander</title>
		<link>http://www.g-loaded.eu/2007/08/06/file-and-directory-diff-in-color-in-midnight-commander/</link>
		<comments>http://www.g-loaded.eu/2007/08/06/file-and-directory-diff-in-color-in-midnight-commander/#comments</comments>
		<pubDate>Mon, 06 Aug 2007 15:51:07 +0000</pubDate>
		<dc:creator>George Notaras</dc:creator>
				<category><![CDATA[Linux]]></category>
		<category><![CDATA[Productivity]]></category>
		<category><![CDATA[Programming]]></category>
		<category><![CDATA[Shell]]></category>
		<category><![CDATA[System]]></category>
		<category><![CDATA[Tips]]></category>
		<guid isPermaLink="false">http://www.g-loaded.eu/2007/08/06/file-and-directory-diff-in-color-in-midnight-commander/</guid>
		<description><![CDATA[Midnight Commander (MC) is a lean, but powerful, two-panel file manager that, admittedly, promotes productivity while working from a console. Among other features, it implements a user menu with pre-defined actions that can be performed on the selected files or directories. This menu can be further customized by the user on a directory-, user- or [...]]]></description>
			<content:encoded><![CDATA[<p><a href="http://www.ibiblio.org/mc/">Midnight Commander</a> (MC) is a lean, but powerful, two-panel file manager that, admittedly, promotes productivity while working from a console. Among other features, it implements a <strong>user menu</strong> with pre-defined actions that can be performed on the selected files or directories. This menu can be further customized by the user on a directory-, user- or system-basis. While searching for ready-made actions for <abbr title="Midnight Commander">MC</abbr>&#8216;s user menu, I came across an excellent set, which adds the ability to display the differences (diff output) between two versions of a file or directory in color.<br />
<span id="more-435"></span></p>
<h4>Add the necessary code</h4>
<p>The code that needs to be added to the user menu is the following:</p>
<pre class="codesnp">
+ t r &#038; ! t t
d       Diff against file of same name in other directory
        if [ "%d" = "%D" ]; then
          echo "The two directores must be different"
          exit 1
        fi
        if [ -f %D/%f ]; then        # if two of them, then
          diff -up %f %D/%f | sed -e 's/\(^-.*\)/\x1b[1;31m\1\x1b[0m/g' \
                                  -e 's/\(^\+.*\)/\x1b[1;32m\1\x1b[0m/g' \
                                  -e 's/\(^@.*\)/\x1b[36m\1\x1b[0m/g' | less -R
        else
          echo %f: No copy in %D/%f
        fi
D       Diff current directory against other directory
        if [ "%d" = "%D" ]; then
          echo "The two directores must be different"
          exit 1
        fi
        diff -up %d %D | sed -e 's/\(^-.*\)/\x1b[1;31m\1\x1b[0m/g' \
                             -e 's/\(^\+.*\)/\x1b[1;32m\1\x1b[0m/g' \
                             -e 's/\(^@.*\)/\x1b[36m\1\x1b[0m/g' | less -R
        fi
</pre>
<p>The code snippet is a contribution by <cite>James Ogley</cite> to this <a href="http://en.opensuse.org/Midnight_Commander/Tips">OpenSUSE wiki page</a> about Midnight Commander and is released under the <a href="http://www.gnu.org/licenses/fdl.txt">GFDL 1.2 license</a>.</p>
<p>Here are the instructions on how to add this code to the user menu:</p>
<ol>
<li>Having launched Midnight Commander, press <strong>F9</strong></li>
<li>Go to the <strong>Command menu</strong> and hit the <strong>Edit menu file</strong></li>
<li>MC lets you customize the user menu for the current directory [<code>Local</code>], current user [<code>Home</code>] or make the modification system-wide. You will need to be <em>root</em> to change the latter. Most probably the <strong>Home</strong> choice is what you need.</li>
<li>You can now paste the code block between two other user-defined commands.</li>
</ol>
<h4>How to use these actions</h4>
<p>In order to show the differences between two versions of the same file, browse to the directories that contain each version, so that the directory in <em>panel A</em> contains the <em>A version</em> of <code>some.file</code> and the directory in <em>panel B</em> contains the <em>B version</em> of <code>some.file</code>. Then press <strong>F2</strong> and lowercase <strong>d</strong>.</p>
<p>In order to show the differences between the contents of the directories in the two panels, just press <strong>F2</strong> and then uppercase <strong>D</strong>.</p>
<div class="cc-block"><em><a href="http://www.g-loaded.eu/2007/08/06/file-and-directory-diff-in-color-in-midnight-commander/">File and Directory diff in color in Midnight Commander</a></em>, unless otherwise expressly stated, is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/3.0/">Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License</a>. Terms and conditions beyond the scope of this license may be available at <a href="http://www.g-loaded.eu/about/disclaimer-and-license/">www.g-loaded.eu</a>.</div>
<h4>Related Articles</h4>
<ul><li><a href="http://www.g-loaded.eu/2006/09/10/spec-file-directory/" rel="bookmark">SPEC File Directory</a></li>
<li><a href="http://www.g-loaded.eu/2005/11/06/meld/" rel="bookmark">Meld&#8230;</a></li>
<li><a href="http://www.g-loaded.eu/2008/12/09/making-a-directory-writable-by-the-webserver/" rel="bookmark">Making a directory writable by the webserver</a></li>
<li><a href="http://www.g-loaded.eu/2007/09/15/use-wget-or-curl-to-download-from-rapidshare-premium/" rel="bookmark">Use wget or curl to download from RapidShare Premium</a></li>
<li><a href="http://www.g-loaded.eu/2007/07/28/send-a-text-file-as-sms-with-a-sony-ericsson-mobile/" rel="bookmark">Send a text file as SMS with a Sony-Ericsson mobile</a></li></ul>]]></content:encoded>
			<wfw:commentRss>http://www.g-loaded.eu/2007/08/06/file-and-directory-diff-in-color-in-midnight-commander/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
	<creativeCommons:license>http://creativecommons.org/licenses/by-nc-sa/3.0/</creativeCommons:license>
	</item>
		<item>
		<title>Best Practices of Software Licensing</title>
		<link>http://www.g-loaded.eu/2007/07/29/best-practices-of-software-licensing/</link>
		<comments>http://www.g-loaded.eu/2007/07/29/best-practices-of-software-licensing/#comments</comments>
		<pubDate>Sun, 29 Jul 2007 14:04:25 +0000</pubDate>
		<dc:creator>George Notaras</dc:creator>
				<category><![CDATA[Computers]]></category>
		<category><![CDATA[HOWTO]]></category>
		<category><![CDATA[License]]></category>
		<category><![CDATA[Maintenance]]></category>
		<category><![CDATA[Opinion]]></category>
		<category><![CDATA[Programming]]></category>
		<category><![CDATA[Publishing]]></category>
		<category><![CDATA[Software]]></category>
		<guid isPermaLink="false">http://www.g-loaded.eu/2007/07/29/best-practices-of-software-licensing/</guid>
		<description><![CDATA[I am nowhere near being an authority on software licensing. However, the recent release of GPLv3 with all the online discussions, all the attempts to exploit statements of the old version 2 of the license in order to achieve automatic license renewal for all the projects that were released under GPLv2 made me re-evaluate, not [...]]]></description>
			<content:encoded><![CDATA[<p>I am nowhere near being an authority on software licensing. However, the recent release of GPLv3 with all the online discussions, all the attempts to exploit statements of the old version 2 of the license in order to achieve automatic license renewal for all the projects that were released under GPLv2 made me <a href="http://www.g-loaded.eu/2007/07/01/gplv3-is-out-the-g-loadedeu-views/">re-evaluate</a>, not only the license itself, but also the methodology that should be used when licensing even a single line of code.<br />
<span id="more-427"></span><br />
Here folows a list of what I believe are the best practices when licensing software:</p>
<ol>
<li><strong>Study</strong> the license&#8217;s legal code; not just read it, study it. It contains the terms under which your software will be released. Never let trends interfere with the selection of the license and, of course, make sure you do not choose a license just because you think you cannot find a more suitable one. Note that, when it comes to software licenses, the <em>point of no return</em> is the release of the software. If you change your mind about the license after a version of the software has been released, then it is impossible to invalidate previously released versions. Anyone, who has obtained a copy of the software with the old license, will be able to use it according to the terms of that license.</li>
<li><strong>Include</strong> a copy of the license inside the distribution package of your software. Also, keep a <strong>local copy</strong> of the license on a web site that you fully control and link to it when necessary. If you have to link to the license&#8217;s original web page, make sure you use a <strong>URI</strong> that includes at least the name and version of the license, eg <code>.../gpl-v2.html</code>. Avoid linking to URLs that might be redirected to different documents over time. For example, a URI like <code>.../gpl.html</code> will probably be redirected to the latest version of the license whenever the latter is released, so it might bring confusion to the users of your software or to developers who wish to use the code. If the organization that issues the license has taken care, so that the license URI contains the license name, version and the general rights &#8211; like it happens with the Creative Commons version 3 licenses -, then I guess it is safe enough to use such URLs for linking.</li>
<li>Keep in mind that <strong>you</strong>, the developer of the software, are the one who will decide the terms under which the software will be released. If one of the existing licenses is satisfactory enough, then it is highly recommended that you use it. Otherwise, feel free to <strong>modify</strong> one of those licenses by adding or removing conditions in order to create a license that suits your software&#8217;s needs.</li>
<li>It is highly recommended that you <strong>remove</strong> any statements that expressly state or imply the <em>automatic renewal</em> of the license when a new version of it is released. Such statements can lead to serious trouble and confusion. Automatic license renewal is irrational when it comes to computer software. [<em>Update</em>: it is suggested to include a notice which indicates the removal of such statements inside the software's distribution.]</li>
<li>Software that is released under the General Public License (GPL) is free open-source software. On the other hand, software does not have to be released under the GPL in order to be <abbr title="Free Software / Open Source Software">FS/OSS</abbr>. If your software is meant to be FS/OSS, there are numerous licenses that can be used in order to give your users the necessary rights to use, copy, distribute, build on it etc. See the <a href="http://www.opensource.org/licenses/category">OSI approved licenses</a> by category.</li>
</ol>
<p>This list may not be complete, but it outlines some general rules that can be followed in order to avoid getting into trouble with your software&#8217;s license.</p>
<div class="cc-block"><em><a href="http://www.g-loaded.eu/2007/07/29/best-practices-of-software-licensing/">Best Practices of Software Licensing</a></em>, unless otherwise expressly stated, is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/3.0/">Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License</a>. Terms and conditions beyond the scope of this license may be available at <a href="http://www.g-loaded.eu/about/disclaimer-and-license/">www.g-loaded.eu</a>.</div>
<h4>Related Articles</h4>
<ul><li><a href="http://www.g-loaded.eu/2011/03/01/forking-apache-licensed-software-on-github-and-bitbucket/" rel="bookmark">Forking Apache-licensed software on Github and Bitbucket</a></li>
<li><a href="http://www.g-loaded.eu/2007/07/01/gplv3-is-out-the-g-loadedeu-views/" rel="bookmark">GPLv3 is out &#8211; The G-Loaded.eu views</a></li>
<li><a href="http://www.g-loaded.eu/2010/04/16/software-that-detects-violations-of-open-source-licenses/" rel="bookmark">Software that detects violations of open source licenses</a></li>
<li><a href="http://www.g-loaded.eu/2006/12/18/free-software-inc/" rel="bookmark">Free Software Inc.</a></li>
<li><a href="http://www.g-loaded.eu/2009/01/31/microsoft-releases-part-of-web-sandbox-code-under-the-apache-license/" rel="bookmark">Microsoft releases part of Web Sandbox code under the Apache License</a></li></ul>]]></content:encoded>
			<wfw:commentRss>http://www.g-loaded.eu/2007/07/29/best-practices-of-software-licensing/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
	<creativeCommons:license>http://creativecommons.org/licenses/by-nc-sa/3.0/</creativeCommons:license>
	</item>
		<item>
		<title>Application Testing: Zero Free Space</title>
		<link>http://www.g-loaded.eu/2007/02/20/application-testing-zero-free-space/</link>
		<comments>http://www.g-loaded.eu/2007/02/20/application-testing-zero-free-space/#comments</comments>
		<pubDate>Tue, 20 Feb 2007 05:41:54 +0000</pubDate>
		<dc:creator>George Notaras</dc:creator>
				<category><![CDATA[Linux]]></category>
		<category><![CDATA[Errors]]></category>
		<category><![CDATA[Opinion]]></category>
		<category><![CDATA[Optimization]]></category>
		<category><![CDATA[Programming]]></category>
		<category><![CDATA[Software]]></category>
		<guid isPermaLink="false">http://www.g-loaded.eu/2007/02/20/application-testing-zero-free-space/</guid>
		<description><![CDATA[I should clarify from the beginning that this is not a technical article containing any kind of debugging information - I don't have the knowledge anyway - or any other type of proof that indicates that some commonly used applications have bugs. This is just about some observations, or better an unofficial user report, of "<em>what went wrong when I run out of free space</em>".]]></description>
			<content:encoded><![CDATA[<p>I should clarify from the beginning that this is not a technical article containing any kind of debugging information &#8211; I don&#8217;t have the knowledge anyway &#8211; or any other type of proof that indicates that some commonly used applications have bugs. This is just about some observations, or better an unofficial user report, of &#8220;<em>what went wrong when I ran out of free space</em>&#8220;.<br />
<span id="more-342"></span><br />
I had written a script which recorded information to a log file the other day. I had forgotten to turn it off and the result was that my home directory became full of data, leaving zero free space. This came at a moment while I was using <a href="http://www.mozilla.org/">firefox</a> (v2.0.0.1) and <a href="http://www.gftp.org/">gftp</a> (v2.0.18) on a <a href="http://www.gnome.org/">GNOME</a> (v2.16.3) <a href="http://fedoraproject.org/">Fedora</a> 6 desktop. Of course, as soon as I realised that my home was full of data, I deleted that huge log file. In the next moments, I noticed that the things I outline below had gone wrong during this incident. I have not tried to reproduce these errors, so I haven&#8217;t filed any bugs. Feel free to do so, if you are able to reproduce them.</p>
<p>First, I do not remember being warned about running out of free space. <strong>GNOME</strong> is supposed to do this, but, in my case, either it did not raise that warning message or I was too focused on my work so I didn&#8217;t notice it. I&#8217;d bet that the latter is what actually happened.</p>
<p>Second, I have setup several profiles in <strong>firefox</strong> and I have adjusted its launcher in the GNOME menu to start the profile manager each time I want to use the browser. Firefox keeps the profile information in the <code>~/.mozilla/firefox/profiles.ini</code> file. Under the &#8220;<em>no free space</em>&#8221; conditions, something had caused the file to be re-created as a zero-size file, losing all of my profile information. Fortunately, this was the only file that was affected. My profile directories were in tact, so it was rather easy to recover from this situation by setting my profiles again with the profile manager.</p>
<p>The third observation has to do with the FTP client, <strong>gftp</strong>. As I have mentioned earlier I was also working with this client when I run out of free space. It is strange to me how some things happen sometimes, but at that moment I was adding some ftp sites to the gftp bookmarks. Well, it seems that if you happen to save your bookmarks or settings in gftp while there is no free space in the partition that holds gftp&#8217;s config files, you are LOST! All the files in <code>~/.gftp/</code> are replaced with zero-size files and all your settings and bookmarks are lost, unless you have a recent backup of them. Well, I didn&#8217;t have one and that&#8217;s why I was angry (against myself). This was fixed &#8211; but not my bookmarks :-( &#8211; by completely removing the <code>~/.gftp/</code> directory and getting it recreated when launching gftp.</p>
<p>After this incident, I kept wondering if there were more configuration files in my user account&#8217;s home that were affected. I guess it is impossible to know in advance. Time will show.</p>
<p>An application which acted absolutely professionally and actually was the one that made me realize that there was something wrong with the free space is <a href="http://www.gnome.org/projects/evolution/">Evolution</a>. It displayed proper warning messages about not being able to write the newly fetched email and news messages to the disk and, generally, the problematic situation had absolutely no effect on it. <strong>I take my hat off to Evolution</strong> (v2.8.3).</p>
<p>This incident made me think if software developers ever test how their software would behave if there is no free space available, especially if its usage involves reading/writing/appending from/to files. I would bet that other programs might have problems under such conditions too, but I guess these situations are rare. On the other hand, if it happens and the software is not designed to work well, then your personal data could be at risk. At this point, I am rather happy I was not running my password manager at that critical period of time! ;-)</p>
<p>Anyway, as I have already written earlier, <strong>I blame noone</strong>. Unfortunately, I do not have the required free time or will to reproduce those errors in a test environment, but I guess I will do this at some later time. All the above might just be wrong conclusions and something else might was the cause of the problem. I really do not know, but I felt like writing about that experience.</p>
<div class="cc-block"><em><a href="http://www.g-loaded.eu/2007/02/20/application-testing-zero-free-space/">Application Testing: Zero Free Space</a></em>, unless otherwise expressly stated, is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/3.0/">Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License</a>. Terms and conditions beyond the scope of this license may be available at <a href="http://www.g-loaded.eu/about/disclaimer-and-license/">www.g-loaded.eu</a>.</div>
<h4>Related Articles</h4>
<ul><li><a href="http://www.g-loaded.eu/2007/09/14/featured-epiphany-plugins/" rel="bookmark">Featured Epiphany Plugins</a></li>
<li><a href="http://www.g-loaded.eu/2009/05/07/descramble-passwords-from-gftp-bookmarks-using-python/" rel="bookmark">Descramble Passwords from gftp Bookmarks using Python</a></li>
<li><a href="http://www.g-loaded.eu/2005/09/10/testing-gnome-blog-poster/" rel="bookmark">Testing gnome-blog-poster&#8230;</a></li>
<li><a href="http://www.g-loaded.eu/2006/04/05/epiphany-browser-review/" rel="bookmark">Epiphany Browser Review</a></li>
<li><a href="http://www.g-loaded.eu/2008/12/08/access-gvfs-mounts-from-the-command-line/" rel="bookmark">Access gvfs mounts from the command line</a></li></ul>]]></content:encoded>
			<wfw:commentRss>http://www.g-loaded.eu/2007/02/20/application-testing-zero-free-space/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
	<creativeCommons:license>http://creativecommons.org/licenses/by-nc-sa/3.0/</creativeCommons:license>
	</item>
		<item>
		<title>Optimize and Compress CSS Files</title>
		<link>http://www.g-loaded.eu/2006/12/04/optimize-and-compress-css-files/</link>
		<comments>http://www.g-loaded.eu/2006/12/04/optimize-and-compress-css-files/#comments</comments>
		<pubDate>Mon, 04 Dec 2006 21:39:02 +0000</pubDate>
		<dc:creator>George Notaras</dc:creator>
				<category><![CDATA[Programming]]></category>
		<category><![CDATA[Optimization]]></category>
		<category><![CDATA[Performance]]></category>
		<category><![CDATA[Tips]]></category>
		<category><![CDATA[Web]]></category>
		<guid isPermaLink="false">http://www.g-loaded.eu/2006/12/04/optimize-and-compress-css-files/</guid>
		<description><![CDATA[Usually, when writing or modifying a CSS file, the author adds comments and excessive indentation to the code in order to preserve its readability and to simplify maintenance. Although this might be a good habit, all those extra bits stored into the CSS file increase its filesize, often resulting in unnecessary waste of bandwidth, especially if such a file is used in a production web site. Today, I decided to search for tools that can perform compression and optimization of a CSS file.]]></description>
			<content:encoded><![CDATA[<p>Usually, when writing or modifying a <abbr title="Cascading Style Sheets">CSS</abbr> file, the author adds comments and excessive indentation to the code in order to preserve its readability and to simplify maintenance. Although this might be a good habit, all those extra bits stored into the CSS file increase its filesize, often resulting in unnecessary waste of bandwidth, especially if such a file is used in a production web site. Today, I decided to search for tools that can perform <strong>compression</strong> and <strong>optimization</strong> of a CSS file.<br />
<span id="more-315"></span><br />
A command line utility, called <a href="http://csstidy.sourceforge.net/">CSSTidy</a>, seems to be one of the best out there. It&#8217;s <em>open-source</em> software, released under the GPL, and, judging by the optimization results on this web site&#8217;s CSS, I can say it does a good job. Although the source code is available for download, I used the pre-compiled Linux binary.</p>
<p>Before proceeding, make sure you have backed up your current non-optimized style-sheet. CSSTidy will not touch your original file, but taking some precautions is a good habit.</p>
<p>Launch csstidy without arguments for a rather short description of the command line switches:</p>
<pre class="console">csstidy</pre>
<p>Although it is possible to define each parameter individually, some pre-defined templates are what you will most probably use. These include: <code>low</code>|<code>high</code>|<code>highest</code>. The higher the optimization level, the smaller the CSS filesize. More on templates <a href="http://csstidy.sourceforge.net/templates.php">here</a>.</p>
<p>I tried the <code>high</code> and <code>highest</code> templates. The latter produces a style sheet that contains all code in a single line, but that file would not <a href="http://jigsaw.w3.org/css-validator/">validate</a>, so I used the <code>high</code> template:</p>
<pre class="console">csstidy style.css --template=high style.css.out</pre>
<p>The compression ratio for my file was <code>34.86%</code> which was quite satisfactory. Also, the final file validated properly.</p>
<p>CSSTidy does a good job either you want to optimize the stylesheet for readability or for filesize. On the other hand, the CLI interface does not provide enough information about each of the CLI switches. They are pretty self-explanatory, but I guess a little description for each one of them wouldn&#8217;t hurt anyone. Also, I could not make the &#8220;<code>--remove_last_;</code>&#8221; option to work.</p>
<p>The only difference I could notice in my stylesheets between the <code>high</code> and the <code>highest</code> templates was that the whole code was squeezed in a single line when <code>highest</code> was used. Probably, this template performs some more optimizations than just deleting the line-feed characters, but it seems that there was nothing else that could be optimized in my CSS file.</p>
<p>Apart from this command line utility, there are also a lot of online CSS optimizers. I didn&#8217;t have the time to try them all, but I think I can recommend the following:</p>
<ul>
<li><a href="http://www.cleancss.com/">Clean CSS</a> &#8211; this seems to be the PHP version of CSSTidy. Everything seems to work as expected and the results are the same as before.</li>
<li><a href="http://iceyboard.no-ip.org/projects/css_compressor">Icey CSS Compressor</a> &#8211; This is another web-based CSS optimizer which performs very well. By using the default settings you get a single line with highly optimized CSS code. The compression ratio is similar to CSSTidy&#8217;s. In order to separate each selector with a line-feed character, so that the final CSS validates, you can pass the optimized CSS through <code>sed</code>:
<pre class="console">sed -i 's/}/}\n/g' style.css.out</pre>
</li>
</ul>
<p>If anyone knows about any other utility I would be very glad to know about.</p>
<div class="cc-block"><em><a href="http://www.g-loaded.eu/2006/12/04/optimize-and-compress-css-files/">Optimize and Compress CSS Files</a></em>, unless otherwise expressly stated, is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/3.0/">Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License</a>. Terms and conditions beyond the scope of this license may be available at <a href="http://www.g-loaded.eu/about/disclaimer-and-license/">www.g-loaded.eu</a>.</div>
<h4>Related Articles</h4>
<ul><li><a href="http://www.g-loaded.eu/2008/05/10/use-mod_deflate-to-compress-web-content-delivered-by-apache/" rel="bookmark">Use mod_deflate to Compress Web Content delivered by Apache</a></li>
<li><a href="http://www.g-loaded.eu/2007/12/01/veritar-verify-checksums-of-files-within-a-tar-archive/" rel="bookmark">VeriTAR &#8211; Verify checksums of files within a TAR archive</a></li>
<li><a href="http://www.g-loaded.eu/2008/05/03/how-to-annotate-pdf-files-in-linux-using-xournal/" rel="bookmark">How to annotate PDF files in Linux using Xournal</a></li>
<li><a href="http://www.g-loaded.eu/2006/01/14/creative-commons-configurator-wordpress-plugin/" rel="bookmark">Creative-Commons-Configurator WordPress Plugin</a></li>
<li><a href="http://www.g-loaded.eu/2007/07/10/wordpress-221-bugs/" rel="bookmark">WordPress 2.2.1 bugs</a></li></ul>]]></content:encoded>
			<wfw:commentRss>http://www.g-loaded.eu/2006/12/04/optimize-and-compress-css-files/feed/</wfw:commentRss>
		<slash:comments>4</slash:comments>
	<creativeCommons:license>http://creativecommons.org/licenses/by-nc-sa/3.0/</creativeCommons:license>
	</item>
		<item>
		<title>pdf2email CUPS Backend</title>
		<link>http://www.g-loaded.eu/2006/12/03/pdf2email-cups-backend/</link>
		<comments>http://www.g-loaded.eu/2006/12/03/pdf2email-cups-backend/#comments</comments>
		<pubDate>Sun, 03 Dec 2006 14:22:09 +0000</pubDate>
		<dc:creator>George Notaras</dc:creator>
				<category><![CDATA[Linux]]></category>
		<category><![CDATA[Printing]]></category>
		<category><![CDATA[Programming]]></category>
		<category><![CDATA[Python]]></category>
		<category><![CDATA[Scripts]]></category>
		<category><![CDATA[Software]]></category>
		<category><![CDATA[System]]></category>
		<guid isPermaLink="false">http://www.g-loaded.eu/2006/12/03/pdf2email-cups-backend/</guid>
		<description><![CDATA[pdf2email is a CUPS backend that uses GhostScript to print a document to PDF and sends the final file to the user that requested the print via email. This software is written in Python. I had written this backend a few months ago, it has worked fine for me, so I decided to release it. Your feedback is welcome.]]></description>
			<content:encoded><![CDATA[<p><strong>pdf2email</strong> is a <abbr title="Common UNIX Printing System">CUPS</abbr> backend that uses GhostScript to print a document to <strong>PDF</strong> and sends the final file to the user that requested the print via email. This software is written in Python. I had written this backend a few months ago, it has worked fine for me, so I decided to release it. Your feedback is welcome.<br />
<span id="more-314"></span></p>
<h4>Requirements</h4>
<p>The following software is required so that this backend functions correctly:</p>
<ol>
<li>Python (<a href="http://www.python.org/">http://www.python.org</a>)</li>
<li>CUPS server (<a href="http://www.cups.org/">http://www.cups.org/</a>)</li>
<li>Ghostscript (<a href="http://www.cs.wisc.edu/~ghost/">http://www.cs.wisc.edu/~ghost/</a>)</li>
<li>a postscript printer&#8217;s PPD file. (see <a href="#installation">Installation</a> section)</li>
<li>Email server, on which usernames from the LAN workstations either match the users on the email server or proper mappings (aliases) have been set.</li>
</ol>
<p><a id="installation"></a></p>
<h4>Installation</h4>
<p>Put the &#8220;<code>pdf2email</code>&#8221; file in <code>/usr/lib/cups/backend/</code> and set the executable bit:</p>
<pre class="console">chmod +x /usr/lib/cups/backend/pdf2email</pre>
<h4>Configuration</h4>
<p>Open the script in a text editor and set a couple of settings:</p>
<p>1. Set the email server&#8217;s hostname:</p>
<pre class="codesnp">SMTP_Server = "mail.example.org"</pre>
<p>2. Set the email address of the sender as it will appear in the email:</p>
<pre class="codesnp">print_admin = "admin@example.org"</pre>
<p>3. (<em>Optional</em>) Set the path to a text file that contains PDFmarks. If you don&#8217;t know what PDFmarks are, you <strong>should</strong> leave this variable empty. (see some more info in the PDFmarks section):</p>
<pre class="codesnp">pdfmarks_file = ""</pre>
<h4>Configure a CUPS Printer</h4>
<p>This section describes in brief all the required actions in order to add a printer that uses this backend to CUPS.</p>
<ul>
<li>Decide the printer&#8217;s name, eg &#8220;TestPDFprinter&#8221;.</li>
<li>Decide the directory, where the PDF files are saved temporarily. This directory must be writable by the users. Usually, <code>/tmp</code> is perfect for this. Note, that no PDF files are left in this directory. This piece of information is used in the <strong>Device URI</strong> of your CUPS printer. For example, provided that the <code>/tmp</code> directory will be used, the Device URI would be: <code>pdf2email:/tmp</code></li>
<li>Get a postscript printer&#8217;s PPD file. You can find such files at:
<ul>
<li>Your printer <strong>manufacturer</strong>&#8216;s web site.</li>
<li><a href="http://www.linuxprinting.org/show_driver.cgi?driver=Postscript">Linux Printing</a></li>
<li><a href="http://www.adobe.com/support/downloads/">Adobe&#8217;s download center</a></li>
</ul>
</li>
</ul>
<blockquote><p><strong>WARNING</strong>: It is your exclusive responsibility to examine the printer driver&#8217;s license prior to using the PPD file. <strong>NO PPD FILES ARE SHIPPED WITH THE pdf2email BACKEND</strong>.</p></blockquote>
<p>Finally, add a new printer that uses this backend, either by using lpadmin from the command line or by using the web-based administration interface of CUPS or by using any other CUPS Server management software.</p>
<h5>Using lpadmin from the shell</h5>
<p>As root issue the following command:</p>
<pre class="console">lpadmin -p TestPDFprinter -E -v pdf2email:/tmp -P /path/to/myprinter.ppd</pre>
<h5>Using CUPS web-based interface or other software</h5>
<p>Use the provided &#8220;<strong>Add Printer</strong>&#8221; facility. Some notes that might be useful:</p>
<ul>
<li>In the device dropdown box, select the following:<br /> <code>Direct PDF Printing/Delivery to user email</code></li>
<li>In the device URI box use the writable directory, eg:<br /> <code>pdf2email:/tmp</code></li>
<li>If asked about the type of the print queue, use &#8220;<code>RAW</code>&#8220;.</li>
</ul>
<h4>USAGE</h4>
<p>Make this printer available to the LAN through the IPP, SAMBA or any other protocol and add it to your printer list in your client machines. Whatever you send to that printer will be converted to a PDF document and will be emailed to your user&#8217;s email address.</p>
<h4>Trouble?</h4>
<p>If any error occurs during the PDF creation process, there will be a failure notice in your inbox and a message in syslog (<code>/var/log/messages</code>).</p>
<p>In case you receive no email after requesting a print, check syslog for any error messages. If there aren&#8217;t any, the check CUPS error log.</p>
<p>Finally, note that if you use a security layer like SELinux, then you might need to further adjust it so it lets 3rd party CUPS backends to function. This is beyond the scope of this document.</p>
<h4>PDFmarks</h4>
<p>If you do not know what pdfmarks are, please do a web search about them. This backend can include PDFmarks in the final PDF document. Do not expect anything spectacular, as this script cannot process the document for bookmarks, links etc. What it can do is to add pre-defined pdfmarks to all PDF documents.</p>
<p>For example, if, for example,  you want to fill some of the document properties, save the following in a file:</p>
<pre class="codesnp">
[/Title ()
/Author ()
/Subject (blah blah)
/Keywords (keyword1, keyword2, blah)
/DOCINFO pdfmark
</pre>
<p>By default the author&#8217;s property is filled with the username that requested the print.</p>
<p>Finally, add the path to the pdfmarks file in the configuration section of the backend, as described in the configuration section:</p>
<pre class="codesnp">pdfmarks_file = "/path/to/pdfmarks.file"</pre>
<p>This feature has not been tested thoroughly.</p>
<h4>License</h4>
<p>This project is released under the terms of the <a href="http://www.g-loaded.eu/licenses/GPL">GNU General Public License</a>.</p>
<h4>Download</h4>
<p>All versions of the software, including the latest stable release, are available from the development web site&#8217;s <a href="http://www.codetrax.org/projects/pdf2email/files">download area</a>.</p>
<h4>Donate</h4>
<p>This software is released as <strong>free software</strong>. Nevertheless, its development requires time and effort. A small donation, as a sign of appreciation of the effort, is welcome. Please, use the following button to visit the <em>Donations</em> page. Thank you.</p>
<p><a href="http://www.g-loaded.eu/about/donate" title="Donate"><img src="/images/donations_button.png" alt="Donations Button" /></a></p>
<h4>Support</h4>
<p>This software is released as <em>free software</em> without any warranties or official support. You can still get first class <strong>support</strong> from the <a href="http://www.codetrax.org/projects/pdf2email/boards">community of users</a>.</p>
<h4>Bugs and Feature Requests</h4>
<p>Found a bug? Want to suggest a new feature for the upcoming release? Feel free to file your requests in the <a href="http://www.codetrax.org/projects/pdf2email/issues">issue database</a> on the <a href="http://www.codetrax.org/projects/pdf2email">development website</a>.</p>
<div class="cc-block"><em><a href="http://www.g-loaded.eu/2006/12/03/pdf2email-cups-backend/">pdf2email CUPS Backend</a></em>, unless otherwise expressly stated, is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/3.0/">Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License</a>. Terms and conditions beyond the scope of this license may be available at <a href="http://www.g-loaded.eu/about/disclaimer-and-license/">www.g-loaded.eu</a>.</div>
<h4>Related Articles</h4>
<ul><li><a href="http://www.g-loaded.eu/2005/11/10/print-to-cups-printer-instances/" rel="bookmark">Print to CUPS printer instances</a></li>
<li><a href="http://www.g-loaded.eu/2005/11/10/using-a-cups-printer-from-command-line/" rel="bookmark">Using a CUPS printer from command line</a></li>
<li><a href="http://www.g-loaded.eu/2007/03/05/print-a-man-page/" rel="bookmark">Print a Man Page</a></li>
<li><a href="http://www.g-loaded.eu/2008/01/24/viceo-backend-for-sane-with-libusb-support/" rel="bookmark">Viceo Backend for SANE with libusb support</a></li>
<li><a href="http://www.g-loaded.eu/2010/10/29/sphinx-documentation-single-file/" rel="bookmark">How to create a single file of Sphinx based documentation</a></li></ul>]]></content:encoded>
			<wfw:commentRss>http://www.g-loaded.eu/2006/12/03/pdf2email-cups-backend/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
	<creativeCommons:license>http://creativecommons.org/licenses/by-nc-sa/3.0/</creativeCommons:license>
	</item>
		<item>
		<title>Use Python to get the web page data in Epiphany</title>
		<link>http://www.g-loaded.eu/2006/09/23/use-python-to-get-the-web-page-data-in-epiphany/</link>
		<comments>http://www.g-loaded.eu/2006/09/23/use-python-to-get-the-web-page-data-in-epiphany/#comments</comments>
		<pubDate>Sat, 23 Sep 2006 06:47:11 +0000</pubDate>
		<dc:creator>George Notaras</dc:creator>
				<category><![CDATA[Programming]]></category>
		<category><![CDATA[Epiphany]]></category>
		<category><![CDATA[GNOME]]></category>
		<category><![CDATA[Python]]></category>
		<category><![CDATA[Tips]]></category>
		<guid isPermaLink="false">http://www.g-loaded.eu/2006/09/23/use-puthon-to-get-the-web-page-data-in-epiphany/</guid>
		<description><![CDATA[Sometimes, things are not that straightforward as one might think. Yesterday, I spent over two hours on the Epiphany Python Console checking almost all of the available functions in order to find a way to store the displayed page&#8217;s HTML data in a variable. Before quitting, I decided to get some help over at the [...]]]></description>
			<content:encoded><![CDATA[<p>Sometimes, things are not that straightforward as one might think. Yesterday, I spent over two hours on the Epiphany Python Console checking almost all of the available functions in order to find a way to store the displayed page&#8217;s HTML data in a variable. Before quitting, I decided to get some help over at the #Epiphany IRC channel. All credit for the following Python code goes to <cite>JFR</cite>.<br />
<span id="more-259"></span><br />
So, assuming that a web page is loaded in an epiphany tab, launch the Python Console from within the browser.</p>
<p>We will need the <code>epiphany</code> module, so import it:</p>
<pre class="codesnp">import epiphany</pre>
<p>Next we assign the active tab and its respective embed to variables:</p>
<pre class="codesnp">
tab = window.get_active_tab()
embed = tab.get_embed()
</pre>
<p>And now the critical part of getting the page&#8217;s HTML code:</p>
<pre class="codesnp">
persist = epiphany.ephy_embed_factory_new_object( epiphany.EmbedPersist )
persist.set_flags( epiphany.EMBED_PERSIST_NO_VIEW | epiphany.EMBED_PERSIST_COPY_PAGE )
persist.set_embed( embed )
page = persist.to_string()
</pre>
<p>Then, you can just print the page code in the console, save it to files or filter it to get the info you want.</p>
<div class="cc-block"><em><a href="http://www.g-loaded.eu/2006/09/23/use-python-to-get-the-web-page-data-in-epiphany/">Use Python to get the web page data in Epiphany</a></em>, unless otherwise expressly stated, is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/3.0/">Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License</a>. Terms and conditions beyond the scope of this license may be available at <a href="http://www.g-loaded.eu/about/disclaimer-and-license/">www.g-loaded.eu</a>.</div>
<h4>Related Articles</h4>
<ul><li><a href="http://www.g-loaded.eu/2006/05/17/epiphany-python-console-open-new-tab/" rel="bookmark">Epiphany Python Console &#8211; Open New Tab</a></li>
<li><a href="http://www.g-loaded.eu/2006/05/17/epiphany-python-console-documentation/" rel="bookmark">Epiphany Python Console &#8211; Documentation</a></li>
<li><a href="http://www.g-loaded.eu/2006/05/16/tab-session-management-extension-for-epiphany/" rel="bookmark">Tab Session Management extension for Epiphany</a></li>
<li><a href="http://www.g-loaded.eu/2006/05/06/a-note-about-the-epiphany-extensions-on-fedora/" rel="bookmark">A Note About The Epiphany Extensions on Fedora</a></li>
<li><a href="http://www.g-loaded.eu/2007/02/28/tab-links-extension-for-the-epiphany-browser/" rel="bookmark">Tab Links extension for the Epiphany browser</a></li></ul>]]></content:encoded>
			<wfw:commentRss>http://www.g-loaded.eu/2006/09/23/use-python-to-get-the-web-page-data-in-epiphany/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
	<creativeCommons:license>http://creativecommons.org/licenses/by-nc-sa/3.0/</creativeCommons:license>
	</item>
	</channel>
</rss>

