<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Villa Road</title>
	<atom:link href="http://villaroad.com/feed/" rel="self" type="application/rss+xml" />
	<link>http://villaroad.com</link>
	<description></description>
	<lastBuildDate>Thu, 28 Mar 2013 17:47:53 +0000</lastBuildDate>
	<language>en-US</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.5.1</generator>
		<item>
		<title>Installing PIL</title>
		<link>http://villaroad.com/installing-pil/</link>
		<comments>http://villaroad.com/installing-pil/#comments</comments>
		<pubDate>Tue, 08 Feb 2011 20:29:06 +0000</pubDate>
		<dc:creator>Alfred</dc:creator>
				<category><![CDATA[Python]]></category>
		<category><![CDATA[Software]]></category>
		<category><![CDATA[Tools]]></category>
		<category><![CDATA[custom pil]]></category>
		<category><![CDATA[installing pil]]></category>
		<category><![CDATA[jpg]]></category>
		<category><![CDATA[PIL]]></category>
		<category><![CDATA[python]]></category>

		<guid isPermaLink="false">http://villaroad.com/?p=386</guid>
		<description><![CDATA[<p>While to some it may seem trivial to roll your own PIL (Python Imaging Library) it can be quite annoying for others. Especially if you have followed my previous tutorial on installing Python 2.7 from source. On request from a &#8230;</p>]]></description>
				<content:encoded><![CDATA[<p>While to some it may seem trivial to roll your own PIL (Python Imaging Library) it can be quite annoying for others. Especially if you have followed my previous tutorial on installing Python 2.7 from source. On request from a visitor of the site, I am posting a simple and efficient way to get the latest build of PIL up and rolling on your box. This should allow you to do all sorts of neat things with images.</p>
<p>One more thing before we begin, the methodology used in this tutorial can be applied to most any package that you wish to install from source.</p>
<p><strong>Step 1</strong><br />
Lets login to your box and ensure that we are inside the root home directory:</p>

<div class="wp_syntax"><table><tr><td class="line_numbers"><pre>1
2
</pre></td><td class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #c20cb9; font-weight: bold;">ssh</span> root<span style="color: #000000; font-weight: bold;">@</span>yourbox.com
<span style="color: #7a0874; font-weight: bold;">cd</span></pre></td></tr></table></div>

<p><strong>Step 2</strong><br />
We will install the necessary build tools and dependencies for PIl. If you have followed previous tutorials on this site then some of these may have already been installed. Don&#8217;t fret, yum will simply understand them to exist, update them if necessary and install only what is missing:</p>

<div class="wp_syntax"><table><tr><td class="line_numbers"><pre>1
2
</pre></td><td class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #c20cb9; font-weight: bold;">yum</span> <span style="color: #660033;">-y</span> groupinstall <span style="color: #ff0000;">'Development Tools'</span>
<span style="color: #c20cb9; font-weight: bold;">yum</span> <span style="color: #660033;">-y</span> <span style="color: #c20cb9; font-weight: bold;">install</span> bzip<span style="color: #000000; font-weight: bold;">*</span> zlib<span style="color: #000000; font-weight: bold;">*</span> libtif<span style="color: #000000; font-weight: bold;">*</span> libjpeg<span style="color: #000000; font-weight: bold;">*</span> freetype<span style="color: #000000; font-weight: bold;">*</span> tkinter</pre></td></tr></table></div>

<p><strong>Step 3</strong><br />
Now that all the prerequistes are out of the way, it&#8217;s time to get move on. Here we will download and unpack PIL:</p>

<div class="wp_syntax"><table><tr><td class="line_numbers"><pre>1
2
</pre></td><td class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #c20cb9; font-weight: bold;">wget</span> http:<span style="color: #000000; font-weight: bold;">//</span>effbot.org<span style="color: #000000; font-weight: bold;">/</span>downloads<span style="color: #000000; font-weight: bold;">/</span>Imaging-1.1.7.tar.gz
<span style="color: #c20cb9; font-weight: bold;">tar</span> xfz Imaging-1.1.7.tar.gz <span style="color: #000000; font-weight: bold;">&amp;&amp;</span> <span style="color: #c20cb9; font-weight: bold;">rm</span> <span style="color: #660033;">-f</span> Imaging-1.1.7.tar.gz</pre></td></tr></table></div>

<p><strong>Step 4</strong><br />
Ready to smoke this cohiba? Let&#8217;s do it. . . . In this step we will run the included python scripts to install PIL. Take note that we are simply calling the python interpreter from the command line. This will ask bash which python interpreter to use to build the PIL package and which site-packages folder to place the final product in. It reads what the current alias for &#8216;python&#8217; is. If you have followed the previous python 2.7 tutorial then you should be all set. If you are not sure which is alias&#8217;d, simply type <b>python</b>, press return and the interactive console will launch stating the current version.</p>
<p>Once you have confirmed your interpreter version, lets proceed:</p>

<div class="wp_syntax"><table><tr><td class="line_numbers"><pre>1
2
</pre></td><td class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #7a0874; font-weight: bold;">cd</span> Imaging-1.1.7
python setup.py build_ext <span style="color: #660033;">-i</span></pre></td></tr></table></div>

<p><strong>Step 5</strong><br />
In the final step, we will run the included <i>self test</i> application. This will attempt to run PIL pre-installation and print to screen the results of the installation. If everything is groovy you should notice that most compatibility is present. If there is something missing that you need (ie: openCMS, etc) then simply install said software using <b>yum</b> and rerun Step 4. After we have been made to feel good about building PIL, running setup.py with install will place a compiled version of the package inside of pythons site-packages folder making it accessible to python. Yatzee!</p>

<div class="wp_syntax"><table><tr><td class="line_numbers"><pre>1
2
</pre></td><td class="code"><pre class="bash" style="font-family:monospace;">python selftest.py
python setup.py <span style="color: #c20cb9; font-weight: bold;">install</span></pre></td></tr></table></div>

]]></content:encoded>
			<wfw:commentRss>http://villaroad.com/installing-pil/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Remote Desktop to VPS on CentOS 5.5</title>
		<link>http://villaroad.com/remote-desktop-to-centos-5-5/</link>
		<comments>http://villaroad.com/remote-desktop-to-centos-5-5/#comments</comments>
		<pubDate>Mon, 25 Oct 2010 21:55:16 +0000</pubDate>
		<dc:creator>Alfred</dc:creator>
				<category><![CDATA[Software]]></category>
		<category><![CDATA[Sys Admin]]></category>
		<category><![CDATA[Tools]]></category>

		<guid isPermaLink="false">http://villaroad.com/?p=341</guid>
		<description><![CDATA[<p>So lets say that you have a VPS or two from a company like <a href="http://slicehost.com" target="_blank">slicehost</a> or <a href="http://linode.com" target="_blank">Linode</a>. Cool, right? Yep. Powerful and affordable. You have a server at your finger tips for as little as $20/month with no contract. &#8230;</p>]]></description>
				<content:encoded><![CDATA[<p>So lets say that you have a VPS or two from a company like <a href="http://slicehost.com" target="_blank">slicehost</a> or <a href="http://linode.com" target="_blank">Linode</a>. Cool, right? Yep. Powerful and affordable. You have a server at your finger tips for as little as $20/month with no contract. Even better you may say. . . . I would agree with you sir.</p>
<p>Now, this type of setup fits a <a href="http://karsty.net/africa/IMG_4179-DungHeapStatue.jpg" target="_blank">sh*t ton</a> of use cases there is sometimes a need to have a graphical environment. Most VPS providers don&#8217;t offer virtual KVM (Keyboard Video Mouse) solutions for their customers, but that doesn&#8217;t mean we can&#8217;t eat the cake too.</p>
<p>While I understand there are many different solutions to this problem, I feel the way I am about to outline is the easiest and the path of least resistance. We will use <a href="http://freenx.berlios.de/" target="_blank">FreeNX</a>. FreeNX is an open source version of No Machine (commercial) and it allows for Remote Desktop style connections over SSH and other protocols.</p>
<p>Some notes before we begin:</p>
<ul>
<li>we will be using a fresh slicehost 256mb instance, CentOS 5.5 x86_64</li>
<li>nothing was installed or configured that is not listed below</li>
</ul>
<p><strong>Step 1</strong><br />
Lets login to your box as the root user and create a user of choice:</p>

<div class="wp_syntax"><table><tr><td class="line_numbers"><pre>1
2
3
4
</pre></td><td class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #c20cb9; font-weight: bold;">ssh</span> root<span style="color: #000000; font-weight: bold;">@</span>yourbox.com
<span style="color: #7a0874; font-weight: bold;">cd</span>
useradd sally
<span style="color: #c20cb9; font-weight: bold;">passwd</span> sally</pre></td></tr></table></div>

<p><strong>Step 2</strong><br />
Download/Install the latest <a href="http://www.rpmrepo.org/RPMforge/FAQ#What_is_RPMforge.3F" target="_blank">RPMforge</a> .rpm package. This will give us access to the latest and greatest software packages that are not included in the CentOS repos.</p>

<div class="wp_syntax"><table><tr><td class="line_numbers"><pre>1
2
</pre></td><td class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #c20cb9; font-weight: bold;">wget</span> http:<span style="color: #000000; font-weight: bold;">//</span>packages.sw.be<span style="color: #000000; font-weight: bold;">/</span>rpmforge-release<span style="color: #000000; font-weight: bold;">/</span>rpmforge-release-0.5.1-<span style="color: #000000;">1</span>.el5.rf.x86_64.rpm
rpm <span style="color: #660033;">-Uhv</span> rpmforge-release-0.5.1-<span style="color: #000000;">1</span>.el5.rf.x86_64.rpm</pre></td></tr></table></div>

<p><strong>Step 3</strong><br />
Update all of your existing packages with <b>yum</b>. Notice the <b>-y</b> flag. This means that yum will assume the answer is yes to any question it has for the specific operation.</p>

<div class="wp_syntax"><table><tr><td class="line_numbers"><pre>1
</pre></td><td class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #c20cb9; font-weight: bold;">yum</span> <span style="color: #660033;">-y</span> update</pre></td></tr></table></div>

<p><strong>Step 4</strong><br />
Install FreeNX</p>

<div class="wp_syntax"><table><tr><td class="line_numbers"><pre>1
</pre></td><td class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #c20cb9; font-weight: bold;">yum install</span> <span style="color: #660033;">-y</span> freenx</pre></td></tr></table></div>

<p><strong>Step 5</strong><br />
Install the Gnome Desktop Manager. Yes, you can choose something other than Gnome, but I love me some Gah-Know-M <img src='http://villaroad.com/wp-includes/images/smilies/icon_smile.gif' alt=':)' class='wp-smiley' /> </p>

<div class="wp_syntax"><table><tr><td class="line_numbers"><pre>1
</pre></td><td class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #c20cb9; font-weight: bold;">yum</span> <span style="color: #660033;">-y</span> groupinstall <span style="color: #ff0000;">&quot;X Window System&quot;</span> <span style="color: #ff0000;">&quot;GNOME Desktop Environment&quot;</span></pre></td></tr></table></div>

<p><strong>Step 6</strong><br />
Now that your server is all setup, yep; that was all you had to do, we need to setup a way to access an instance of the servers graphical environment. To do this, we will use a NoMachine client software. Go <a href="http://www.nomachine.com/download.php">here</a> and download the &#8220;NX Client&#8221; for your specific OS. <i>NOTE</i>: if you are are OSX, you will need to install Rosetta in order to run the NX Client for OSX. I know &#8230; I know, but they haven&#8217;t updated it yet.</p>
<p>Once you have your software installed open the &#8220;NX Connection Wizard&#8221; and fill out the &#8220;Session&#8221; and &#8220;Host&#8221; the click &#8220;Next&#8221;:<br />
<img alt="" src="http://fedoranews.org/contributors/rick_stout/freenx/connect2.png" title="Connection Wizard" class="aligncenter" width="450" height="350" /></p>
<p>Select &#8220;Unix&#8221;, a Window Manager and a resolution. Make sure that &#8220;Enable SSL encryption on all traffic&#8221; is checked and click &#8220;Next&#8221;. Check &#8220;Show the Advanced Configuration dialog&#8221; box and click &#8220;Finish&#8221;:<br />
<img alt="" src="http://fedoranews.org/contributors/rick_stout/freenx/connect3.png" title="Connection Wizard" class="aligncenter" width="450" height="350" /><br />
<img alt="" src="http://fedoranews.org/contributors/rick_stout/freenx/connect5.png" title="Connection Wizard" class="aligncenter" width="450" height="350" /></p>
<p>Click &#8220;Key&#8230;&#8221;, &#8220;Import&#8221;, Select the key file and click &#8220;Open&#8221;. This file, client.id_dsa.key can be found on the server either in the <b>/etc/nxserver/</b> or <b>/var/lib/nxserver/home/.ssh/</b> directories. Click &#8220;Save&#8221;, &#8220;Save&#8221; and &#8220;Ok&#8221;.<br />
<img alt="" src="http://fedoranews.org/contributors/rick_stout/freenx/connect4.png" title="Connection Wizard" class="aligncenter" width="355" height="400" /><br />
<img alt="" src="http://fedoranews.org/contributors/rick_stout/freenx/key1.png" title="Connection Wizard" class="aligncenter" width="380" height="414" /></p>
<p><strong>Step 7</strong><br />
Ready set go. Now enter your chosen username and password that you created in <i>Step 1</i> and go for it chief!</p>
<p><img alt="" src="http://fedoranews.org/contributors/rick_stout/freenx/client1.png" title="Connection Wizard" class="aligncenter" width="342" height="265" /></p>
<p><i>most of this tutorial is a reference from <a href="http://fedoranews.org/contributors/rick_stout/freenx/">Rick Stouts</a> guide.</i></p>
]]></content:encoded>
			<wfw:commentRss>http://villaroad.com/remote-desktop-to-centos-5-5/feed/</wfw:commentRss>
		<slash:comments>4</slash:comments>
		</item>
		<item>
		<title>Resource Cache</title>
		<link>http://villaroad.com/resource-cache/</link>
		<comments>http://villaroad.com/resource-cache/#comments</comments>
		<pubDate>Thu, 21 Oct 2010 00:31:12 +0000</pubDate>
		<dc:creator>Alfred</dc:creator>
				<category><![CDATA[Python]]></category>

		<guid isPermaLink="false">http://villaroad.com/?p=12</guid>
		<description><![CDATA[<p>The following is a generational cache object, which can be used to cache resources that take a long time to load or allocate. This is particularly useful for caching images or large files that are used often to prevent frequent &#8230;</p>]]></description>
				<content:encoded><![CDATA[<p>The following is a generational cache object, which can be used to cache resources that take a long time to load or allocate. This is particularly useful for caching images or large files that are used often to prevent frequent disk access.</p>
<p>As a warning, the following is practically untested. I have never used this as production code and have no current plans to do so. The concept, however, is sound. This is an almost exact port to Python of a Java program which I wrote which <span>is</span> tested and <span>is</span> used in production code.</p>
<p>The differences between this Python version and the Java version are:</p>
<ol>
<li>In Python I used dictionaries for the generations, while in Java I used trees. There are benefits to both approaches, but I won&#8217;t get into what those are here, so you&#8217;ll have to figure that out yourself. (:</li>
<li>In Python I subclassed the ager and set it by default; whereas in Java I had it as a completely separate class. This makes it easier to use the cache class because you don&#8217;t have to explicitly define the ager, but this also encourages people to use the default ager, which for many situations is not a good decision (it&#8217;s at least one you should devote some thought and experimentation to).</li>
<li>The Java interface has a get() method for getting resources; I think overloading the [] operators was a much cleaner way to do this in Python. In fact, the Java version is written in Java 1.0, so the whole thing really is a godawful mess compared to the Python version.</li>
</ol>
<p>Enjoy.</p>

<div class="wp_syntax"><table><tr><td class="line_numbers"><pre>1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
</pre></td><td class="code"><pre class="python" style="font-family:monospace;"><span style="color: #ff7700;font-weight:bold;">class</span> Cache<span style="color: black;">&#40;</span><span style="color: #008000;">object</span><span style="color: black;">&#41;</span>:
   &amp;quot<span style="color: #66cc66;">;</span>&amp;quot<span style="color: #66cc66;">;</span>&amp;quot<span style="color: #66cc66;">;</span>
   Used to cache resources which take a <span style="color: #008000;">long</span> <span style="color: #dc143c;">time</span> to load <span style="color: #ff7700;font-weight:bold;">or</span> allocate.  It
   uses a &amp;quot<span style="color: #66cc66;">;</span>most-recently-used&amp;quot<span style="color: #66cc66;">;</span> method to determine what remains <span style="color: #ff7700;font-weight:bold;">in</span> cache.
   Whenever something <span style="color: #ff7700;font-weight:bold;">is</span> used<span style="color: #66cc66;">,</span> it <span style="color: #ff7700;font-weight:bold;">is</span> promoted to the youngest generation.
   Every &amp;quot<span style="color: #66cc66;">;</span>frequency&amp;quot<span style="color: #66cc66;">;</span> calls to the cache<span style="color: #66cc66;">,</span> the oldest generation <span style="color: #ff7700;font-weight:bold;">is</span> dropped
   <span style="color: #ff7700;font-weight:bold;">and</span> <span style="color: #008000;">all</span> generations are aged by one.
&nbsp;
   The limitation of this approach <span style="color: #ff7700;font-weight:bold;">is</span> that it does <span style="color: #ff7700;font-weight:bold;">not</span> deal well <span style="color: #ff7700;font-weight:bold;">with</span>
   resources of highly variable size <span style="color: #ff7700;font-weight:bold;">or</span> load speed:
   * Since cache size <span style="color: #ff7700;font-weight:bold;">is</span> limited by the number of uses rather than
     directly by cache size<span style="color: #66cc66;">,</span> a very large <span style="color: #dc143c;">resource</span> might create a larger
     cache than <span style="color: #ff7700;font-weight:bold;">is</span> desireable.  As a result<span style="color: #66cc66;">,</span> the frequency must be <span style="color: #008000;">set</span>
     low to account <span style="color: #ff7700;font-weight:bold;">for</span> even a single large <span style="color: #dc143c;">resource</span><span style="color: #66cc66;">,</span> causing
     undercaching.
   * Resources which take longer to load should generally be kept <span style="color: #ff7700;font-weight:bold;">in</span> cache
     longer because caching them provides the greatest benefit.  This cache
     does <span style="color: #ff7700;font-weight:bold;">not</span> account <span style="color: #ff7700;font-weight:bold;">for</span> the <span style="color: #dc143c;">time</span> it takes to load at <span style="color: #008000;">all</span>.
&nbsp;
   To deal <span style="color: #ff7700;font-weight:bold;">with</span> these limitations<span style="color: #66cc66;">,</span> the <span style="color: #dc143c;">user</span> can define their own Ager<span style="color: #66cc66;">,</span> an
   <span style="color: #008000;">object</span> that determines when to age the cache.  This must define an
   age<span style="color: black;">&#40;</span>cache<span style="color: black;">&#41;</span> method which takes a cache <span style="color: #ff7700;font-weight:bold;">and</span> returns <span style="color: #008000;">True</span> <span style="color: #ff7700;font-weight:bold;">if</span> it needs to
   be cached<span style="color: #66cc66;">,</span> <span style="color: #ff7700;font-weight:bold;">and</span> a reset<span style="color: black;">&#40;</span>cache<span style="color: black;">&#41;</span> method which resets the ager after the
   cache has been aged <span style="color: #ff7700;font-weight:bold;">or</span> cleared.
&nbsp;
   cache.<span style="color: black;">age</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span> can be called to manually age the cache<span style="color: #66cc66;">,</span> <span style="color: #ff7700;font-weight:bold;">and</span> cache.<span style="color: black;">clear</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
   can be called to clear it.
   &amp;quot<span style="color: #66cc66;">;</span>&amp;quot<span style="color: #66cc66;">;</span>&amp;quot<span style="color: #66cc66;">;</span>
   <span style="color: #ff7700;font-weight:bold;">class</span> Ager<span style="color: black;">&#40;</span><span style="color: #008000;">object</span><span style="color: black;">&#41;</span>:
      <span style="color: #ff7700;font-weight:bold;">def</span> <span style="color: #0000cd;">__init__</span><span style="color: black;">&#40;</span><span style="color: #008000;">self</span><span style="color: #66cc66;">,</span>frequency <span style="color: #66cc66;">=</span> <span style="color: #ff4500;">25</span><span style="color: black;">&#41;</span>:
         <span style="color: #008000;">self</span>.<span style="color: black;">frequency</span> <span style="color: #66cc66;">=</span> frequency
         <span style="color: #008000;">self</span>.<span style="color: black;">count</span> <span style="color: #66cc66;">=</span> <span style="color: #ff4500;">0</span>
      <span style="color: #ff7700;font-weight:bold;">def</span> age<span style="color: black;">&#40;</span><span style="color: #008000;">self</span><span style="color: #66cc66;">,</span>cache<span style="color: black;">&#41;</span>:
         <span style="color: #008000;">self</span>.<span style="color: black;">count</span> +<span style="color: #66cc66;">=</span> <span style="color: #ff4500;">1</span>
         <span style="color: #ff7700;font-weight:bold;">if</span> <span style="color: #008000;">self</span>.<span style="color: black;">count</span> <span style="color: #66cc66;">==</span> <span style="color: #008000;">self</span>.<span style="color: black;">frequency</span>:
            <span style="color: #008000;">self</span>.<span style="color: black;">count</span> <span style="color: #66cc66;">=</span> <span style="color: #ff4500;">0</span>
            <span style="color: #ff7700;font-weight:bold;">return</span> <span style="color: #008000;">True</span>
         <span style="color: #ff7700;font-weight:bold;">return</span> <span style="color: #008000;">False</span>
      <span style="color: #ff7700;font-weight:bold;">def</span> reset<span style="color: black;">&#40;</span><span style="color: #008000;">self</span><span style="color: #66cc66;">,</span>cache<span style="color: black;">&#41;</span>:
         <span style="color: #008000;">self</span>.<span style="color: black;">count</span> <span style="color: #66cc66;">=</span> <span style="color: #ff4500;">0</span>
   <span style="color: #ff7700;font-weight:bold;">def</span> <span style="color: #0000cd;">__init__</span><span style="color: black;">&#40;</span><span style="color: #008000;">self</span><span style="color: #66cc66;">,</span>load<span style="color: #66cc66;">,</span>ager <span style="color: #66cc66;">=</span> Cache.<span style="color: black;">Ager</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span><span style="color: black;">&#41;</span>:
      <span style="color: #008000;">self</span>.<span style="color: black;">load</span> <span style="color: #66cc66;">=</span> load
      <span style="color: #008000;">self</span>.<span style="color: black;">cache</span> <span style="color: #66cc66;">=</span> <span style="color: black;">&#91;</span><span style="color: black;">&#123;</span><span style="color: black;">&#125;</span><span style="color: black;">&#93;</span> * <span style="color: #ff4500;">5</span>
      <span style="color: #008000;">self</span>.<span style="color: black;">ager</span> <span style="color: #66cc66;">=</span> ager
   <span style="color: #ff7700;font-weight:bold;">def</span> <span style="color: #0000cd;">__getitem__</span><span style="color: black;">&#40;</span><span style="color: #008000;">self</span><span style="color: #66cc66;">,</span>key<span style="color: black;">&#41;</span>:
      <span style="color: #008000;">self</span>.<span style="color: black;">count</span> +<span style="color: #66cc66;">=</span> <span style="color: #ff4500;">1</span>
      <span style="color: #ff7700;font-weight:bold;">for</span> generation <span style="color: #ff7700;font-weight:bold;">in</span> <span style="color: #008000;">self</span>.<span style="color: black;">cache</span>:
         <span style="color: #ff7700;font-weight:bold;">if</span> key <span style="color: #ff7700;font-weight:bold;">in</span> generation:
            value <span style="color: #66cc66;">=</span> generation<span style="color: black;">&#91;</span>key<span style="color: black;">&#93;</span>
            <span style="color: #ff7700;font-weight:bold;">if</span> <span style="color: #ff7700;font-weight:bold;">not</span> generation <span style="color: #ff7700;font-weight:bold;">is</span> <span style="color: #008000;">self</span>.<span style="color: black;">cache</span><span style="color: black;">&#91;</span><span style="color: #ff4500;">0</span><span style="color: black;">&#93;</span>:
               <span style="color: #008000;">self</span>.<span style="color: black;">cache</span><span style="color: black;">&#91;</span><span style="color: #ff4500;">0</span><span style="color: black;">&#93;</span><span style="color: black;">&#91;</span>key<span style="color: black;">&#93;</span> <span style="color: #66cc66;">=</span> value
            <span style="color: #ff7700;font-weight:bold;">if</span> <span style="color: #008000;">self</span>.<span style="color: black;">ager</span>.<span style="color: black;">age</span><span style="color: black;">&#40;</span><span style="color: #008000;">self</span><span style="color: black;">&#41;</span>:
               <span style="color: #008000;">self</span>.<span style="color: black;">age</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
            <span style="color: #ff7700;font-weight:bold;">return</span> value
      value <span style="color: #66cc66;">=</span> load<span style="color: black;">&#40;</span>key<span style="color: black;">&#41;</span>
      <span style="color: #008000;">self</span>.<span style="color: black;">cache</span><span style="color: black;">&#91;</span><span style="color: #ff4500;">0</span><span style="color: black;">&#93;</span><span style="color: black;">&#91;</span>key<span style="color: black;">&#93;</span> <span style="color: #66cc66;">=</span> value
      <span style="color: #ff7700;font-weight:bold;">if</span> <span style="color: #008000;">self</span>.<span style="color: black;">ager</span>.<span style="color: black;">age</span><span style="color: black;">&#40;</span><span style="color: #008000;">self</span><span style="color: black;">&#41;</span>:
         <span style="color: #008000;">self</span>.<span style="color: black;">age</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
      <span style="color: #ff7700;font-weight:bold;">return</span> value
   <span style="color: #ff7700;font-weight:bold;">def</span> age<span style="color: black;">&#40;</span><span style="color: #008000;">self</span><span style="color: black;">&#41;</span>:
      <span style="color: #008000;">self</span>.<span style="color: black;">cache</span> <span style="color: #66cc66;">=</span> <span style="color: black;">&#91;</span><span style="color: black;">&#123;</span><span style="color: black;">&#125;</span><span style="color: black;">&#93;</span> + <span style="color: #008000;">self</span>.<span style="color: black;">cache</span><span style="color: black;">&#91;</span>:-<span style="color: #ff4500;">1</span><span style="color: black;">&#93;</span>
      <span style="color: #008000;">self</span>.<span style="color: black;">ager</span>.<span style="color: black;">reset</span><span style="color: black;">&#40;</span><span style="color: #008000;">self</span><span style="color: black;">&#41;</span>
   <span style="color: #ff7700;font-weight:bold;">def</span> clear<span style="color: black;">&#40;</span><span style="color: #008000;">self</span><span style="color: black;">&#41;</span>:
      <span style="color: #008000;">self</span>.<span style="color: black;">cache</span> <span style="color: #66cc66;">=</span> <span style="color: black;">&#91;</span><span style="color: black;">&#123;</span><span style="color: black;">&#125;</span><span style="color: black;">&#93;</span> * <span style="color: #008000;">len</span><span style="color: black;">&#40;</span><span style="color: #008000;">self</span>.<span style="color: black;">cache</span><span style="color: black;">&#41;</span>
      <span style="color: #008000;">self</span>.<span style="color: black;">ager</span>.<span style="color: black;">reset</span><span style="color: black;">&#40;</span><span style="color: #008000;">self</span><span style="color: black;">&#41;</span></pre></td></tr></table></div>

]]></content:encoded>
			<wfw:commentRss>http://villaroad.com/resource-cache/feed/</wfw:commentRss>
		<slash:comments>2</slash:comments>
		</item>
		<item>
		<title>Rolling Python 2.7 On CentOS 5.5</title>
		<link>http://villaroad.com/rolling-python-2-6-2-on-centos-5-3/</link>
		<comments>http://villaroad.com/rolling-python-2-6-2-on-centos-5-3/#comments</comments>
		<pubDate>Thu, 21 Oct 2010 00:23:40 +0000</pubDate>
		<dc:creator>Alfred</dc:creator>
				<category><![CDATA[Python]]></category>
		<category><![CDATA[Sys Admin]]></category>
		<category><![CDATA[centos]]></category>
		<category><![CDATA[python]]></category>

		<guid isPermaLink="false">http://villaroad.com/?p=128</guid>
		<description><![CDATA[<p>So, we have all heard the horror stories of replacing the 2.4 interpreter with a newer one that is not offered in any of the maintained repos. Clouds part, log files build, mutt inboxes overflow and yum (among many other &#8230;</p>]]></description>
				<content:encoded><![CDATA[<p>So, we have all heard the horror stories of replacing the 2.4 interpreter with a newer one that is not offered in any of the maintained repos. Clouds part, log files build, mutt inboxes overflow and yum (among many other binaries) basically die. It doesn&#8217;t sound very appealing at first glance, but maybe you need some cool modules that just aren&#8217;t compatible with 2.4x or possibly there have been some standard library additions that are absolutely necessary. Whatever the reason, replacing the 2.4x interpreter is f&#8217;n spooky and bad juju no matter how you smoke it.</p>
<p>But wait, does this mean that CentOS is just going to be limited to using such an old build of Python? Nope. We just need to setup a second interpreter and ensure it won&#8217;t interfere with the 2.4x version that seemingly everything else <em>builtin</em> to CentOS uses. It can take some searching and sifting to find the proper way to do this, but I found a slightly outdated method written by <a href="http://www.visionaryrenesis.com/">Matt Reiferson</a>. I took this method and updated it for CentOS 5.5.</p>
<p>Some notes before we begin:</p>
<ul>
<li>we will be using a fresh install of CentOS 5.5 with no packages selected at installation</li>
<li>we will be using the x86_64 build</li>
</ul>
<p><strong>Step 1</strong><br />
Lets login to your box as the root user:</p>

<div class="wp_syntax"><table><tr><td class="line_numbers"><pre>1
2
</pre></td><td class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #c20cb9; font-weight: bold;">ssh</span> root<span style="color: #000000; font-weight: bold;">@</span>yourbox.com
<span style="color: #7a0874; font-weight: bold;">cd</span></pre></td></tr></table></div>

<p><strong>Step 2</strong><br />
Go ahead and install epel (extra packages for enterprise linux). This will add the epel repo to yum and give us a running start at some normally unavailable packages. It also provides some updates to existing repo packages:</p>

<div class="wp_syntax"><table><tr><td class="line_numbers"><pre>1
2
3
</pre></td><td class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #7a0874; font-weight: bold;">cd</span>
<span style="color: #c20cb9; font-weight: bold;">wget</span> http:<span style="color: #000000; font-weight: bold;">//</span>packages.sw.be<span style="color: #000000; font-weight: bold;">/</span>rpmforge-release<span style="color: #000000; font-weight: bold;">/</span>rpmforge-release-0.5.1-<span style="color: #000000;">1</span>.el5.rf.x86_64.rpm
rpm <span style="color: #660033;">-Uhv</span> rpmforge-release-0.5.1-<span style="color: #000000;">1</span>.el5.rf.x86_64.rpm</pre></td></tr></table></div>

<p><strong>Step 3</strong><br />
Install development tools, ssl and zlib. These development tools will allow us to build python properly as well as aide setuptools when using <strong>easy_install</strong> later in this tutorial. The ssl and zlib libraries will be found automatically when we build python and in turn become incorporated into the interpreter:</p>

<div class="wp_syntax"><table><tr><td class="line_numbers"><pre>1
2
</pre></td><td class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #c20cb9; font-weight: bold;">yum</span> <span style="color: #660033;">-y</span> groupinstall <span style="color: #ff0000;">'Development Tools'</span>
yum-y  <span style="color: #c20cb9; font-weight: bold;">install</span> openssl-devel<span style="color: #000000; font-weight: bold;">*</span> zlib<span style="color: #000000; font-weight: bold;">*</span>.x86_64</pre></td></tr></table></div>

<p><strong>Step 4</strong><br />
Download all the files that we will need; sqlite, python and setuptools:</p>

<div class="wp_syntax"><table><tr><td class="line_numbers"><pre>1
2
3
</pre></td><td class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #c20cb9; font-weight: bold;">wget</span> http:<span style="color: #000000; font-weight: bold;">//</span>sqlite.org<span style="color: #000000; font-weight: bold;">/</span>sqlite-amalgamation-3.7.3.tar.gz
<span style="color: #c20cb9; font-weight: bold;">wget</span> http:<span style="color: #000000; font-weight: bold;">//</span>python.org<span style="color: #000000; font-weight: bold;">/</span>ftp<span style="color: #000000; font-weight: bold;">/</span>python<span style="color: #000000; font-weight: bold;">/</span><span style="color: #000000;">2.7</span><span style="color: #000000; font-weight: bold;">/</span>Python-<span style="color: #000000;">2.7</span>.tgz
<span style="color: #c20cb9; font-weight: bold;">wget</span> http:<span style="color: #000000; font-weight: bold;">//</span>pypi.python.org<span style="color: #000000; font-weight: bold;">/</span>packages<span style="color: #000000; font-weight: bold;">/</span><span style="color: #000000;">2.7</span><span style="color: #000000; font-weight: bold;">/</span>s<span style="color: #000000; font-weight: bold;">/</span>setuptools<span style="color: #000000; font-weight: bold;">/</span>setuptools-0.6c11-py2.7.egg</pre></td></tr></table></div>

<p><strong>Step 5</strong><br />
Build and install SQLite:</p>

<div class="wp_syntax"><table><tr><td class="line_numbers"><pre>1
2
3
4
5
6
</pre></td><td class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #7a0874; font-weight: bold;">cd</span>
<span style="color: #c20cb9; font-weight: bold;">tar</span> xfz sqlite-amalgamation-3.7.3.tar.gz
<span style="color: #7a0874; font-weight: bold;">cd</span> sqlite-3.7.3<span style="color: #000000; font-weight: bold;">/</span>
.<span style="color: #000000; font-weight: bold;">/</span>configure
<span style="color: #c20cb9; font-weight: bold;">make</span>
<span style="color: #c20cb9; font-weight: bold;">make</span> <span style="color: #c20cb9; font-weight: bold;">install</span></pre></td></tr></table></div>

<p><strong>Step 6</strong><br />
Build and install Python 2.7. Now, there are some important things to discuss here. First and foremost we have given the option <strong>&#8211;prefix=/opt/python2.7</strong>. This option installs the python binaries and the python library in /opt/python2.7 (it will make the dir for us) rather than in /usr/local/ which would, as we stated above, replace the standard python interpreter and inherently be bad juju. The <strong>/opt</strong> directory in redhat based distributions is a directory provides a home for larger, mostly custom built, binaries and applications.</p>
<p>Also, we made sure that the interpreter is going to make use of multiple threads by adding the <strong>&#8211;with-threads</strong> option. I believe that by default, with-threads is true, but better to be safe than sorry. Finally, the <strong>&#8211;enable-shared</strong> option just allows python to be embedded into other apps:</p>

<div class="wp_syntax"><table><tr><td class="line_numbers"><pre>1
2
3
4
5
6
</pre></td><td class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #7a0874; font-weight: bold;">cd</span>
<span style="color: #c20cb9; font-weight: bold;">tar</span> xfz Python-<span style="color: #000000;">2.7</span>.tgz
<span style="color: #7a0874; font-weight: bold;">cd</span> Python-<span style="color: #000000;">2.7</span>
.<span style="color: #000000; font-weight: bold;">/</span>configure <span style="color: #660033;">--prefix</span>=<span style="color: #000000; font-weight: bold;">/</span>opt<span style="color: #000000; font-weight: bold;">/</span>python2.7 <span style="color: #660033;">--with-threads</span> <span style="color: #660033;">--enable-shared</span>
<span style="color: #c20cb9; font-weight: bold;">make</span>
<span style="color: #c20cb9; font-weight: bold;">make</span> <span style="color: #c20cb9; font-weight: bold;">install</span></pre></td></tr></table></div>

<p><strong>Step 7</strong><br />
We need to now make sure that all the users of the system access the new interpreter when <strong>python</strong> is typed into standard in. To do this, we will need to add a couple of aliases and an addidtion to the $PATH to each users <strong>.bash_profile</strong>. This file is kept in the home directory of each user (ie: <strong>/home/usera/.bash_profile</strong>):</p>

<div class="wp_syntax"><table><tr><td class="line_numbers"><pre>1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
</pre></td><td class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #c20cb9; font-weight: bold;">su</span> - root
<span style="color: #7a0874; font-weight: bold;">cd</span>
<span style="color: #c20cb9; font-weight: bold;">nano</span> .bash_profile
<span style="color: #666666; font-style: italic;"># add the following lines to the bottom of the file</span>
<span style="color: #7a0874; font-weight: bold;">alias</span> <span style="color: #007800;">python</span>=<span style="color: #ff0000;">'/opt/python2.7/bin/python'</span>
<span style="color: #7a0874; font-weight: bold;">alias</span> python2.7=<span style="color: #ff0000;">'/opt/python2.7/bin/python'</span>
<span style="color: #007800;">PATH</span>=<span style="color: #007800;">$PATH</span>:<span style="color: #000000; font-weight: bold;">/</span>opt<span style="color: #000000; font-weight: bold;">/</span>python2.7<span style="color: #000000; font-weight: bold;">/</span>bin
<span style="color: #666666; font-style: italic;"># 'ctrl + o' to save the file and 'ctrl+x' to close the file</span>
<span style="color: #666666; font-style: italic;"># now do the same for every other user, like this:</span>
<span style="color: #c20cb9; font-weight: bold;">nano</span> <span style="color: #000000; font-weight: bold;">/</span>home<span style="color: #000000; font-weight: bold;">/</span>usera<span style="color: #000000; font-weight: bold;">/</span>.bash_profile
<span style="color: #666666; font-style: italic;"># add the following lines to the bottom of the file</span>
<span style="color: #7a0874; font-weight: bold;">alias</span> <span style="color: #007800;">python</span>=<span style="color: #ff0000;">'/opt/python2.7/bin/python'</span>
<span style="color: #7a0874; font-weight: bold;">alias</span> python2.7=<span style="color: #ff0000;">'/opt/python2.7/bin/python'</span>
<span style="color: #007800;">PATH</span>=<span style="color: #007800;">$PATH</span>:<span style="color: #000000; font-weight: bold;">/</span>opt<span style="color: #000000; font-weight: bold;">/</span>python2.7<span style="color: #000000; font-weight: bold;">/</span>bin
<span style="color: #666666; font-style: italic;"># 'ctrl + o' to save the file and 'ctrl+x' to close the file</span></pre></td></tr></table></div>

<p><strong>Step 8</strong><br />
Now we need to update BASH so that it knows about the new shared libraries that we have put on the system. Lets create a symlink to them and then reload the cache of the shared libraries (don&#8217;t actually type &#8211;&gt; below. It is just to show that you are on a new line):</p>

<div class="wp_syntax"><table><tr><td class="line_numbers"><pre>1
2
3
4
5
</pre></td><td class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #c20cb9; font-weight: bold;">su</span> - root
<span style="color: #7a0874; font-weight: bold;">cd</span>
<span style="color: #c20cb9; font-weight: bold;">cat</span> <span style="color: #000000; font-weight: bold;">&amp;</span>gt;<span style="color: #000000; font-weight: bold;">&amp;</span>gt; <span style="color: #000000; font-weight: bold;">/</span>etc<span style="color: #000000; font-weight: bold;">/</span>ld.so.conf.d<span style="color: #000000; font-weight: bold;">/</span>opt-python2.7.conf
--<span style="color: #000000; font-weight: bold;">&amp;</span>gt;<span style="color: #000000; font-weight: bold;">/</span>opt<span style="color: #000000; font-weight: bold;">/</span>python2.7<span style="color: #000000; font-weight: bold;">/</span>lib	<span style="color: #666666; font-style: italic;">#hit 'enter' and then 'ctrl+d'</span>
ldconfig</pre></td></tr></table></div>

<p><strong>Step 9</strong><br />
Now lets roll up some setup tools. This will also give us our conduit to the cheese shop (aka: pypi) which I am a complete fan of, despite the nay sayers. Also, we will add some more symlinks :</p>

<div class="wp_syntax"><table><tr><td class="line_numbers"><pre>1
2
3
4
</pre></td><td class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #7a0874; font-weight: bold;">cd</span>
<span style="color: #c20cb9; font-weight: bold;">sh</span> setuptools-0.6c11-py2.7.egg
<span style="color: #7a0874; font-weight: bold;">cd</span> <span style="color: #000000; font-weight: bold;">/</span>opt<span style="color: #000000; font-weight: bold;">/</span>python2.7<span style="color: #000000; font-weight: bold;">/</span>lib<span style="color: #000000; font-weight: bold;">/</span>python2.7<span style="color: #000000; font-weight: bold;">/</span>config
<span style="color: #c20cb9; font-weight: bold;">ln</span> <span style="color: #660033;">-s</span> ..<span style="color: #000000; font-weight: bold;">/</span>..<span style="color: #000000; font-weight: bold;">/</span>libpython2.7.so .</pre></td></tr></table></div>

<p><strong>Step 10</strong><br />
Ignore Bridgett Cherry.</p>
<p><strong>Step 11</strong><br />
You are done senior! Logout and log back in, to get the new bash profile stuff and continue on your way!</p>
]]></content:encoded>
			<wfw:commentRss>http://villaroad.com/rolling-python-2-6-2-on-centos-5-3/feed/</wfw:commentRss>
		<slash:comments>64</slash:comments>
		</item>
		<item>
		<title>Monitor your django methods</title>
		<link>http://villaroad.com/monitor-your-django-methods/</link>
		<comments>http://villaroad.com/monitor-your-django-methods/#comments</comments>
		<pubDate>Tue, 20 Jul 2010 09:04:32 +0000</pubDate>
		<dc:creator>Alfred</dc:creator>
				<category><![CDATA[Django]]></category>
		<category><![CDATA[Python]]></category>
		<category><![CDATA[Web]]></category>

		<guid isPermaLink="false">http://villaroad.com/?p=267</guid>
		<description><![CDATA[<p>So . . . it occurred to me that there wasn&#8217;t a real way to monitor the internals of a django app/project. Sure you can run munin (or any flavor of monitoring app) and watch apache response times, db hits, &#8230;</p>]]></description>
				<content:encoded><![CDATA[<p>So . . . it occurred to me that there wasn&#8217;t a real way to monitor the internals of a django app/project. Sure you can run munin (or any flavor of monitoring app) and watch apache response times, db hits, network lag and so on. All great, right? Well it turns out that if you are in the market for something &#8220;out of the box&#8221; to profile the actual methods inside your django shiz, then you are &#8220;out of luck&#8221;.</p>
<p>So, I browsed the interwebs and found some old 0.96 compliant code that showed promise inside the django docs. Using this inside your django projects will display hotshot results and let you know just what python modules/files/groups are taking what time to respond. Not sure about you, but this is <i>amazing</i> to me. Made minor adjustments to make the code 1.2 compliant. Thanks to the original 0.96 <a href="http://djangosnippets.org/users/dogada/">dev</a> for this awesome code:</p>

<div class="wp_syntax"><table><tr><td class="line_numbers"><pre>1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
</pre></td><td class="code"><pre class="python" style="font-family:monospace;"><span style="color: #ff7700;font-weight:bold;">import</span> <span style="color: #dc143c;">sys</span>
<span style="color: #ff7700;font-weight:bold;">import</span> <span style="color: #dc143c;">os</span>
<span style="color: #ff7700;font-weight:bold;">import</span> <span style="color: #dc143c;">re</span>
<span style="color: #ff7700;font-weight:bold;">import</span> <span style="color: #dc143c;">hotshot</span><span style="color: #66cc66;">,</span> <span style="color: #dc143c;">hotshot</span>.<span style="color: black;">stats</span>
<span style="color: #ff7700;font-weight:bold;">import</span> <span style="color: #dc143c;">tempfile</span>
<span style="color: #ff7700;font-weight:bold;">import</span> <span style="color: #dc143c;">StringIO</span>
&nbsp;
<span style="color: #ff7700;font-weight:bold;">from</span> django.<span style="color: black;">conf</span> <span style="color: #ff7700;font-weight:bold;">import</span> settings
&nbsp;
&nbsp;
words_re <span style="color: #66cc66;">=</span> <span style="color: #dc143c;">re</span>.<span style="color: #008000;">compile</span><span style="color: black;">&#40;</span> r<span style="color: #483d8b;">'<span style="color: #000099; font-weight: bold;">\s</span>+'</span> <span style="color: black;">&#41;</span>
&nbsp;
group_prefix_re <span style="color: #66cc66;">=</span> <span style="color: black;">&#91;</span>
    <span style="color: #dc143c;">re</span>.<span style="color: #008000;">compile</span><span style="color: black;">&#40;</span> &amp;quot<span style="color: #66cc66;">;</span>^.*/django/<span style="color: black;">&#91;</span>^/<span style="color: black;">&#93;</span>+&amp;quot<span style="color: #66cc66;">;</span> <span style="color: black;">&#41;</span><span style="color: #66cc66;">,</span>
    <span style="color: #dc143c;">re</span>.<span style="color: #008000;">compile</span><span style="color: black;">&#40;</span> &amp;quot<span style="color: #66cc66;">;</span>^<span style="color: black;">&#40;</span>.*<span style="color: black;">&#41;</span>/<span style="color: black;">&#91;</span>^/<span style="color: black;">&#93;</span>+$&amp;quot<span style="color: #66cc66;">;</span> <span style="color: black;">&#41;</span><span style="color: #66cc66;">,</span>
    <span style="color: #dc143c;">re</span>.<span style="color: #008000;">compile</span><span style="color: black;">&#40;</span> &amp;quot<span style="color: #66cc66;">;</span>.*&amp;quot<span style="color: #66cc66;">;</span> <span style="color: black;">&#41;</span><span style="color: #66cc66;">,</span>
<span style="color: black;">&#93;</span>
&nbsp;
<span style="color: #ff7700;font-weight:bold;">class</span> ProfileMiddleware<span style="color: black;">&#40;</span><span style="color: #008000;">object</span><span style="color: black;">&#41;</span>:
    <span style="color: #ff7700;font-weight:bold;">def</span> process_request<span style="color: black;">&#40;</span><span style="color: #008000;">self</span><span style="color: #66cc66;">,</span> request<span style="color: black;">&#41;</span>:
        <span style="color: #ff7700;font-weight:bold;">if</span> <span style="color: black;">&#40;</span>settings.<span style="color: black;">DEBUG</span> <span style="color: #ff7700;font-weight:bold;">or</span> request.<span style="color: #dc143c;">user</span>.<span style="color: black;">is_superuser</span><span style="color: black;">&#41;</span> <span style="color: #ff7700;font-weight:bold;">and</span> <span style="color: #483d8b;">'lookie'</span> <span style="color: #ff7700;font-weight:bold;">in</span> request.<span style="color: black;">GET</span>:
            <span style="color: #008000;">self</span>.<span style="color: black;">tmpfile</span> <span style="color: #66cc66;">=</span> <span style="color: #dc143c;">tempfile</span>.<span style="color: black;">mktemp</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
            <span style="color: #008000;">self</span>.<span style="color: black;">prof</span> <span style="color: #66cc66;">=</span> <span style="color: #dc143c;">hotshot</span>.<span style="color: black;">Profile</span><span style="color: black;">&#40;</span><span style="color: #008000;">self</span>.<span style="color: black;">tmpfile</span><span style="color: black;">&#41;</span>
&nbsp;
    <span style="color: #ff7700;font-weight:bold;">def</span> process_view<span style="color: black;">&#40;</span><span style="color: #008000;">self</span><span style="color: #66cc66;">,</span> request<span style="color: #66cc66;">,</span> callback<span style="color: #66cc66;">,</span> callback_args<span style="color: #66cc66;">,</span> callback_kwargs<span style="color: black;">&#41;</span>:
        <span style="color: #ff7700;font-weight:bold;">if</span> <span style="color: black;">&#40;</span>settings.<span style="color: black;">DEBUG</span> <span style="color: #ff7700;font-weight:bold;">or</span> request.<span style="color: #dc143c;">user</span>.<span style="color: black;">is_superuser</span><span style="color: black;">&#41;</span> <span style="color: #ff7700;font-weight:bold;">and</span> <span style="color: #483d8b;">'lookie'</span> <span style="color: #ff7700;font-weight:bold;">in</span> request.<span style="color: black;">GET</span>:
            <span style="color: #ff7700;font-weight:bold;">return</span> <span style="color: #008000;">self</span>.<span style="color: black;">prof</span>.<span style="color: black;">runcall</span><span style="color: black;">&#40;</span>callback<span style="color: #66cc66;">,</span> request<span style="color: #66cc66;">,</span> *callback_args<span style="color: #66cc66;">,</span> **callback_kwargs<span style="color: black;">&#41;</span>
&nbsp;
    <span style="color: #ff7700;font-weight:bold;">def</span> get_group<span style="color: black;">&#40;</span><span style="color: #008000;">self</span><span style="color: #66cc66;">,</span> <span style="color: #008000;">file</span><span style="color: black;">&#41;</span>:
        <span style="color: #ff7700;font-weight:bold;">for</span> g <span style="color: #ff7700;font-weight:bold;">in</span> group_prefix_re:
            name <span style="color: #66cc66;">=</span> g.<span style="color: black;">findall</span><span style="color: black;">&#40;</span> <span style="color: #008000;">file</span> <span style="color: black;">&#41;</span>
            <span style="color: #ff7700;font-weight:bold;">if</span> name:
                <span style="color: #ff7700;font-weight:bold;">return</span> name<span style="color: black;">&#91;</span><span style="color: #ff4500;">0</span><span style="color: black;">&#93;</span>
&nbsp;
    <span style="color: #ff7700;font-weight:bold;">def</span> get_summary<span style="color: black;">&#40;</span><span style="color: #008000;">self</span><span style="color: #66cc66;">,</span> results_dict<span style="color: #66cc66;">,</span> <span style="color: #008000;">sum</span><span style="color: black;">&#41;</span>:
        <span style="color: #008000;">list</span> <span style="color: #66cc66;">=</span> <span style="color: black;">&#91;</span> <span style="color: black;">&#40;</span>item<span style="color: black;">&#91;</span><span style="color: #ff4500;">1</span><span style="color: black;">&#93;</span><span style="color: #66cc66;">,</span> item<span style="color: black;">&#91;</span><span style="color: #ff4500;">0</span><span style="color: black;">&#93;</span><span style="color: black;">&#41;</span> <span style="color: #ff7700;font-weight:bold;">for</span> item <span style="color: #ff7700;font-weight:bold;">in</span> results_dict.<span style="color: black;">items</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span> <span style="color: black;">&#93;</span>
        <span style="color: #008000;">list</span>.<span style="color: black;">sort</span><span style="color: black;">&#40;</span> reverse <span style="color: #66cc66;">=</span> <span style="color: #008000;">True</span> <span style="color: black;">&#41;</span>
        <span style="color: #008000;">list</span> <span style="color: #66cc66;">=</span> <span style="color: #008000;">list</span><span style="color: black;">&#91;</span>:<span style="color: #ff4500;">40</span><span style="color: black;">&#93;</span>
&nbsp;
        res <span style="color: #66cc66;">=</span> &amp;quot<span style="color: #66cc66;">;</span>      tottime\n&amp;quot<span style="color: #66cc66;">;</span>
        <span style="color: #ff7700;font-weight:bold;">for</span> item <span style="color: #ff7700;font-weight:bold;">in</span> <span style="color: #008000;">list</span>:
            res +<span style="color: #66cc66;">=</span> &amp;quot<span style="color: #66cc66;">;</span>%<span style="color: #ff4500;">4.1f</span>%% %<span style="color: #ff4500;">7.3f</span> %s\n&amp;quot<span style="color: #66cc66;">;</span> % <span style="color: black;">&#40;</span> <span style="color: #ff4500;">100</span>*item<span style="color: black;">&#91;</span><span style="color: #ff4500;">0</span><span style="color: black;">&#93;</span>/<span style="color: #008000;">sum</span> <span style="color: #ff7700;font-weight:bold;">if</span> <span style="color: #008000;">sum</span> <span style="color: #ff7700;font-weight:bold;">else</span> <span style="color: #ff4500;">0</span><span style="color: #66cc66;">,</span> item<span style="color: black;">&#91;</span><span style="color: #ff4500;">0</span><span style="color: black;">&#93;</span><span style="color: #66cc66;">,</span> item<span style="color: black;">&#91;</span><span style="color: #ff4500;">1</span><span style="color: black;">&#93;</span> <span style="color: black;">&#41;</span>
&nbsp;
        <span style="color: #ff7700;font-weight:bold;">return</span> res
&nbsp;
    <span style="color: #ff7700;font-weight:bold;">def</span> summary_for_files<span style="color: black;">&#40;</span><span style="color: #008000;">self</span><span style="color: #66cc66;">,</span> stats_str<span style="color: black;">&#41;</span>:
        stats_str <span style="color: #66cc66;">=</span> stats_str.<span style="color: black;">split</span><span style="color: black;">&#40;</span>&amp;quot<span style="color: #66cc66;">;</span>\n&amp;quot<span style="color: #66cc66;">;</span><span style="color: black;">&#41;</span><span style="color: black;">&#91;</span><span style="color: #ff4500;">5</span>:<span style="color: black;">&#93;</span>
&nbsp;
        mystats <span style="color: #66cc66;">=</span> <span style="color: black;">&#123;</span><span style="color: black;">&#125;</span>
        mygroups <span style="color: #66cc66;">=</span> <span style="color: black;">&#123;</span><span style="color: black;">&#125;</span>
&nbsp;
        <span style="color: #008000;">sum</span> <span style="color: #66cc66;">=</span> <span style="color: #ff4500;">0</span>
&nbsp;
        <span style="color: #ff7700;font-weight:bold;">for</span> s <span style="color: #ff7700;font-weight:bold;">in</span> stats_str:
            fields <span style="color: #66cc66;">=</span> words_re.<span style="color: black;">split</span><span style="color: black;">&#40;</span>s<span style="color: black;">&#41;</span><span style="color: #66cc66;">;</span>
            <span style="color: #ff7700;font-weight:bold;">if</span> <span style="color: #008000;">len</span><span style="color: black;">&#40;</span>fields<span style="color: black;">&#41;</span> <span style="color: #66cc66;">==</span> <span style="color: #ff4500;">7</span>:
                <span style="color: #dc143c;">time</span> <span style="color: #66cc66;">=</span> <span style="color: #008000;">float</span><span style="color: black;">&#40;</span>fields<span style="color: black;">&#91;</span><span style="color: #ff4500;">2</span><span style="color: black;">&#93;</span><span style="color: black;">&#41;</span>
                <span style="color: #008000;">sum</span> +<span style="color: #66cc66;">=</span> <span style="color: #dc143c;">time</span>
                <span style="color: #008000;">file</span> <span style="color: #66cc66;">=</span> fields<span style="color: black;">&#91;</span><span style="color: #ff4500;">6</span><span style="color: black;">&#93;</span>.<span style="color: black;">split</span><span style="color: black;">&#40;</span>&amp;quot<span style="color: #66cc66;">;</span>:&amp;quot<span style="color: #66cc66;">;</span><span style="color: black;">&#41;</span><span style="color: black;">&#91;</span><span style="color: #ff4500;">0</span><span style="color: black;">&#93;</span>
&nbsp;
                <span style="color: #ff7700;font-weight:bold;">if</span> <span style="color: #ff7700;font-weight:bold;">not</span> <span style="color: #008000;">file</span> <span style="color: #ff7700;font-weight:bold;">in</span> mystats:
                    mystats<span style="color: black;">&#91;</span><span style="color: #008000;">file</span><span style="color: black;">&#93;</span> <span style="color: #66cc66;">=</span> <span style="color: #ff4500;">0</span>
                mystats<span style="color: black;">&#91;</span><span style="color: #008000;">file</span><span style="color: black;">&#93;</span> +<span style="color: #66cc66;">=</span> <span style="color: #dc143c;">time</span>
&nbsp;
                group <span style="color: #66cc66;">=</span> <span style="color: #008000;">self</span>.<span style="color: black;">get_group</span><span style="color: black;">&#40;</span><span style="color: #008000;">file</span><span style="color: black;">&#41;</span>
                <span style="color: #ff7700;font-weight:bold;">if</span> <span style="color: #ff7700;font-weight:bold;">not</span> group <span style="color: #ff7700;font-weight:bold;">in</span> mygroups:
                    mygroups<span style="color: black;">&#91;</span> group <span style="color: black;">&#93;</span> <span style="color: #66cc66;">=</span> <span style="color: #ff4500;">0</span>
                mygroups<span style="color: black;">&#91;</span> group <span style="color: black;">&#93;</span> +<span style="color: #66cc66;">=</span> <span style="color: #dc143c;">time</span>
&nbsp;
        <span style="color: #ff7700;font-weight:bold;">return</span> &amp;quot<span style="color: #66cc66;">;</span>&amp;lt<span style="color: #66cc66;">;</span>pre&amp;gt<span style="color: #66cc66;">;</span>&amp;quot<span style="color: #66cc66;">;</span> + \
               &amp;quot<span style="color: #66cc66;">;</span> ---- By <span style="color: #008000;">file</span> ----\n\n&amp;quot<span style="color: #66cc66;">;</span> + <span style="color: #008000;">self</span>.<span style="color: black;">get_summary</span><span style="color: black;">&#40;</span>mystats<span style="color: #66cc66;">,</span><span style="color: #008000;">sum</span><span style="color: black;">&#41;</span> + &amp;quot<span style="color: #66cc66;">;</span>\n&amp;quot<span style="color: #66cc66;">;</span> + \
               &amp;quot<span style="color: #66cc66;">;</span> ---- By group ---\n\n&amp;quot<span style="color: #66cc66;">;</span> + <span style="color: #008000;">self</span>.<span style="color: black;">get_summary</span><span style="color: black;">&#40;</span>mygroups<span style="color: #66cc66;">,</span><span style="color: #008000;">sum</span><span style="color: black;">&#41;</span> + \
               &amp;quot<span style="color: #66cc66;">;</span>&amp;lt<span style="color: #66cc66;">;</span>/pre&amp;gt<span style="color: #66cc66;">;</span>&amp;quot<span style="color: #66cc66;">;</span>
&nbsp;
    <span style="color: #ff7700;font-weight:bold;">def</span> process_response<span style="color: black;">&#40;</span><span style="color: #008000;">self</span><span style="color: #66cc66;">,</span> request<span style="color: #66cc66;">,</span> response<span style="color: black;">&#41;</span>:
        <span style="color: #ff7700;font-weight:bold;">if</span> <span style="color: black;">&#40;</span>settings.<span style="color: black;">DEBUG</span> <span style="color: #ff7700;font-weight:bold;">or</span> request.<span style="color: #dc143c;">user</span>.<span style="color: black;">is_superuser</span><span style="color: black;">&#41;</span> <span style="color: #ff7700;font-weight:bold;">and</span> <span style="color: #483d8b;">'lookie'</span> <span style="color: #ff7700;font-weight:bold;">in</span> request.<span style="color: black;">GET</span>:
            <span style="color: #008000;">self</span>.<span style="color: black;">prof</span>.<span style="color: black;">close</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
&nbsp;
            out <span style="color: #66cc66;">=</span> <span style="color: #dc143c;">StringIO</span>.<span style="color: #dc143c;">StringIO</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
            old_stdout <span style="color: #66cc66;">=</span> <span style="color: #dc143c;">sys</span>.<span style="color: black;">stdout</span>
            <span style="color: #dc143c;">sys</span>.<span style="color: black;">stdout</span> <span style="color: #66cc66;">=</span> out
&nbsp;
            stats <span style="color: #66cc66;">=</span> <span style="color: #dc143c;">hotshot</span>.<span style="color: black;">stats</span>.<span style="color: black;">load</span><span style="color: black;">&#40;</span><span style="color: #008000;">self</span>.<span style="color: black;">tmpfile</span><span style="color: black;">&#41;</span>
            stats.<span style="color: black;">sort_stats</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">'time'</span><span style="color: #66cc66;">,</span> <span style="color: #483d8b;">'calls'</span><span style="color: black;">&#41;</span>
            stats.<span style="color: black;">print_stats</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
&nbsp;
            <span style="color: #dc143c;">sys</span>.<span style="color: black;">stdout</span> <span style="color: #66cc66;">=</span> old_stdout
            stats_str <span style="color: #66cc66;">=</span> out.<span style="color: black;">getvalue</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
&nbsp;
            <span style="color: #ff7700;font-weight:bold;">if</span> response <span style="color: #ff7700;font-weight:bold;">and</span> response.<span style="color: black;">content</span> <span style="color: #ff7700;font-weight:bold;">and</span> stats_str:
                response.<span style="color: black;">content</span> <span style="color: #66cc66;">=</span> &amp;quot<span style="color: #66cc66;">;</span>&amp;lt<span style="color: #66cc66;">;</span>pre&amp;gt<span style="color: #66cc66;">;</span>&amp;quot<span style="color: #66cc66;">;</span> + stats_str + &amp;quot<span style="color: #66cc66;">;</span>&amp;lt<span style="color: #66cc66;">;</span>/pre&amp;gt<span style="color: #66cc66;">;</span>&amp;quot<span style="color: #66cc66;">;</span>
&nbsp;
            response.<span style="color: black;">content</span> <span style="color: #66cc66;">=</span> &amp;quot<span style="color: #66cc66;">;</span>\n&amp;quot<span style="color: #66cc66;">;</span>.<span style="color: black;">join</span><span style="color: black;">&#40;</span>response.<span style="color: black;">content</span>.<span style="color: black;">split</span><span style="color: black;">&#40;</span>&amp;quot<span style="color: #66cc66;">;</span>\n&amp;quot<span style="color: #66cc66;">;</span><span style="color: black;">&#41;</span><span style="color: black;">&#91;</span>:<span style="color: #ff4500;">40</span><span style="color: black;">&#93;</span><span style="color: black;">&#41;</span>
&nbsp;
            response.<span style="color: black;">content</span> +<span style="color: #66cc66;">=</span> <span style="color: #008000;">self</span>.<span style="color: black;">summary_for_files</span><span style="color: black;">&#40;</span>stats_str<span style="color: black;">&#41;</span>
&nbsp;
            <span style="color: #dc143c;">os</span>.<span style="color: black;">unlink</span><span style="color: black;">&#40;</span><span style="color: #008000;">self</span>.<span style="color: black;">tmpfile</span><span style="color: black;">&#41;</span>
&nbsp;
        <span style="color: #ff7700;font-weight:bold;">return</span> response</pre></td></tr></table></div>

<p>To use this middleware you will first have to install it inside of your django project and add it to the <strong>MIDDLEWARE_CLASSES</strong> tuple. Once this is in place all requests will travel through the middleware like so <img alt="" src="http://docs.djangoproject.com/en/dev/_images/middleware.png" title="Django Middleware Traffic" class="aligncenter" width="502" height="417" />.</p>
<p>Now that every request is traveling through the newly created middleware, we want to see results right? To see a profile of any route inside your project simply type <strong>?lookie</strong> at the end of the route:<br />
<a href="http://villaroad.com/wp-content/uploads/2010/07/Screen-shot-2010-07-20-at-4.56.22-AM.png"><img src="http://villaroad.com/wp-content/uploads/2010/07/Screen-shot-2010-07-20-at-4.56.22-AM.png" alt="Lookie" title="Lookie" width="450" height="190" class="aligncenter size-full wp-image-268" /></a><br />
<a href="http://villaroad.com/wp-content/uploads/2010/07/Screen-shot-2010-07-20-at-4.56.50-AM.png"><img src="http://villaroad.com/wp-content/uploads/2010/07/Screen-shot-2010-07-20-at-4.56.50-AM.png" alt="Lookie" title="Lookie" width="450" height="190" class="aligncenter size-full wp-image-269" /></a></p>
<p>Once you do this your browser will be filled with lines of hotshot profiles on each method that was called to make your magic request . . . magical:<br />
<a href="http://villaroad.com/wp-content/uploads/2010/07/Screen-shot-2010-07-20-at-5.00.23-AM.png"><img src="http://villaroad.com/wp-content/uploads/2010/07/Screen-shot-2010-07-20-at-5.00.23-AM.png" alt="" title="Shazzam!" width="450" height="190" class="aligncenter size-full wp-image-270" /></a></p>
<p>Notice that it even profiles the methods that make django work <img src='http://villaroad.com/wp-includes/images/smilies/icon_wink.gif' alt=';)' class='wp-smiley' /> . Hotshot is magical.</p>
<p>I wouldn&#8217;t reccomend using this on anything live for too long as the middleware overheard might be too much for high traffic sites. You can always leave the code and just not include it inside the <strong>MIDDLEWARE_CLASSES</strong> tuple. Put it in place when things seem slow or you just want to test it out. Have fun and be safe people of the interweb.</p>
]]></content:encoded>
			<wfw:commentRss>http://villaroad.com/monitor-your-django-methods/feed/</wfw:commentRss>
		<slash:comments>8</slash:comments>
		</item>
		<item>
		<title>Sanitize an abused list</title>
		<link>http://villaroad.com/sanitize-an-abused-list/</link>
		<comments>http://villaroad.com/sanitize-an-abused-list/#comments</comments>
		<pubDate>Tue, 06 Jul 2010 01:16:26 +0000</pubDate>
		<dc:creator>Alfred</dc:creator>
				<category><![CDATA[Python]]></category>

		<guid isPermaLink="false">http://villaroad.com/?p=262</guid>
		<description><![CDATA[<p>It is often necessary to sanitize lists after substantial use and whilst using mixed sources. This is one of many ways you can achieve this with minimal effort. Apologies for the line return in the WP template. Update: no more &#8230;</p>]]></description>
				<content:encoded><![CDATA[<p>It is often necessary to sanitize lists after substantial use and whilst using mixed sources. This is one of many ways you can achieve this with minimal effort. Apologies for the line return in the WP template. Update: no more apologies for the line return as this should be fixed. I recently update the template with a new style. Hope you like the template and the abused list function.</p>

<div class="wp_syntax"><table><tr><td class="line_numbers"><pre>1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
</pre></td><td class="code"><pre class="python" style="font-family:monospace;">    <span style="color: #ff7700;font-weight:bold;">def</span> flattenlist<span style="color: black;">&#40;</span>L<span style="color: black;">&#41;</span>:
        <span style="color: #ff7700;font-weight:bold;">import</span> <span style="color: #dc143c;">types</span>
        WhiteTypes <span style="color: #66cc66;">=</span> <span style="color: black;">&#40;</span><span style="color: #483d8b;">'StringType'</span><span style="color: #66cc66;">,</span> <span style="color: #483d8b;">'UnicodeType'</span><span style="color: #66cc66;">,</span> <span style="color: #483d8b;">'StringTypes'</span><span style="color: #66cc66;">,</span> <span style="color: #483d8b;">'ListType'</span><span style="color: #66cc66;">,</span> <span style="color: #483d8b;">'ObjectType'</span><span style="color: #66cc66;">,</span> <span style="color: #483d8b;">'TupleType'</span><span style="color: black;">&#41;</span>
        BlackTypes<span style="color: #66cc66;">=</span> <span style="color: #008000;">tuple</span><span style="color: black;">&#40;</span> <span style="color: black;">&#91;</span><span style="color: #008000;">getattr</span><span style="color: black;">&#40;</span><span style="color: #dc143c;">types</span><span style="color: #66cc66;">,</span> x<span style="color: black;">&#41;</span> <span style="color: #ff7700;font-weight:bold;">for</span> x <span style="color: #ff7700;font-weight:bold;">in</span> <span style="color: #008000;">dir</span><span style="color: black;">&#40;</span><span style="color: #dc143c;">types</span><span style="color: black;">&#41;</span> <span style="color: #ff7700;font-weight:bold;">if</span> <span style="color: #ff7700;font-weight:bold;">not</span> x.<span style="color: black;">startswith</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">'_'</span><span style="color: black;">&#41;</span> <span style="color: #ff7700;font-weight:bold;">and</span> x <span style="color: #ff7700;font-weight:bold;">not</span> <span style="color: #ff7700;font-weight:bold;">in</span> whites<span style="color: black;">&#93;</span> <span style="color: black;">&#41;</span>
&nbsp;
        tmp <span style="color: #66cc66;">=</span> <span style="color: black;">&#91;</span><span style="color: black;">&#93;</span>
        <span style="color: #ff7700;font-weight:bold;">def</span> core<span style="color: black;">&#40;</span>L<span style="color: black;">&#41;</span>:
            <span style="color: #ff7700;font-weight:bold;">if</span>  <span style="color: #ff7700;font-weight:bold;">not</span> <span style="color: #008000;">hasattr</span><span style="color: black;">&#40;</span>L<span style="color: #66cc66;">,</span><span style="color: #483d8b;">'__iter__'</span><span style="color: black;">&#41;</span>:
                <span style="color: #ff7700;font-weight:bold;">return</span> <span style="color: black;">&#91;</span>L<span style="color: black;">&#93;</span>
            <span style="color: #ff7700;font-weight:bold;">else</span> :
                <span style="color: #ff7700;font-weight:bold;">for</span> i <span style="color: #ff7700;font-weight:bold;">in</span> L:
                    <span style="color: #ff7700;font-weight:bold;">if</span> <span style="color: #008000;">isinstance</span><span style="color: black;">&#40;</span>i<span style="color: #66cc66;">,</span>BlackTypes<span style="color: black;">&#41;</span>:
                        tmp.<span style="color: black;">append</span><span style="color: black;">&#40;</span>i<span style="color: black;">&#41;</span>
                        <span style="color: #ff7700;font-weight:bold;">continue</span>
                    <span style="color: #ff7700;font-weight:bold;">if</span> <span style="color: #008000;">type</span><span style="color: black;">&#40;</span>i<span style="color: black;">&#41;</span> <span style="color: #66cc66;">==</span> <span style="color: #008000;">type</span><span style="color: black;">&#40;</span><span style="color: #008000;">str</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span><span style="color: black;">&#41;</span>:
                        tmp.<span style="color: black;">append</span><span style="color: black;">&#40;</span>i<span style="color: black;">&#41;</span>
                    <span style="color: #ff7700;font-weight:bold;">else</span>:
                        core<span style="color: black;">&#40;</span>i<span style="color: black;">&#41;</span>
            <span style="color: #ff7700;font-weight:bold;">return</span> tmp
        <span style="color: #ff7700;font-weight:bold;">return</span> core<span style="color: black;">&#40;</span>L<span style="color: black;">&#41;</span></pre></td></tr></table></div>

]]></content:encoded>
			<wfw:commentRss>http://villaroad.com/sanitize-an-abused-list/feed/</wfw:commentRss>
		<slash:comments>3</slash:comments>
		</item>
		<item>
		<title>MongoDB ID&#8217;s and Django templates</title>
		<link>http://villaroad.com/return-mongodb-ids-inside-django/</link>
		<comments>http://villaroad.com/return-mongodb-ids-inside-django/#comments</comments>
		<pubDate>Tue, 23 Feb 2010 12:28:14 +0000</pubDate>
		<dc:creator>Alfred</dc:creator>
				<category><![CDATA[Data Storage]]></category>
		<category><![CDATA[Django]]></category>
		<category><![CDATA[MongoDB]]></category>
		<category><![CDATA[Python]]></category>

		<guid isPermaLink="false">http://villaroad.com/?p=168</guid>
		<description><![CDATA[<p>So. Ran into a little frustration using <a href="http://mongodb.org">MongoDB</a> with <a href="http://djangoproject.com">Django</a>; if you try to represent the _id field of a mongo object using something like {{ mongo_object._id }} or {{ mongo_object.id }}, you will get nothing but django barf. &#8230;</p>]]></description>
				<content:encoded><![CDATA[<p>So. Ran into a little frustration using <a href="http://mongodb.org">MongoDB</a> with <a href="http://djangoproject.com">Django</a>; if you try to represent the _id field of a mongo object using something like {{ mongo_object._id }} or {{ mongo_object.id }}, you will get nothing but django barf. To circumvent this you can simply create a template filter. Don&#8217;t be scared . . . it&#8217;s actually pretty simple. Let get to it:</p>
<ol>
<li>first off create the following folder and files:</li>

<div class="wp_syntax"><table><tr><td class="line_numbers"><pre>1
2
3
</pre></td><td class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #c20cb9; font-weight: bold;">mkdir</span> <span style="color: #660033;">-p</span> <span style="color: #000000; font-weight: bold;">/</span>path<span style="color: #000000; font-weight: bold;">/</span>to<span style="color: #000000; font-weight: bold;">/</span>project<span style="color: #000000; font-weight: bold;">/</span>appName<span style="color: #000000; font-weight: bold;">/</span>templatetags
<span style="color: #c20cb9; font-weight: bold;">touch</span> <span style="color: #000000; font-weight: bold;">/</span>path<span style="color: #000000; font-weight: bold;">/</span>to<span style="color: #000000; font-weight: bold;">/</span>project<span style="color: #000000; font-weight: bold;">/</span>an_app<span style="color: #000000; font-weight: bold;">/</span>templatetags<span style="color: #000000; font-weight: bold;">/</span>__init__.py
<span style="color: #c20cb9; font-weight: bold;">touch</span> <span style="color: #000000; font-weight: bold;">/</span>path<span style="color: #000000; font-weight: bold;">/</span>to<span style="color: #000000; font-weight: bold;">/</span>project<span style="color: #000000; font-weight: bold;">/</span>an_app<span style="color: #000000; font-weight: bold;">/</span>templatetags<span style="color: #000000; font-weight: bold;">/</span>appName_tags.py</pre></td></tr></table></div>

<li>It is important that you make the directory/files inside of an app folder and name it exactly <b>templatetags</b>. the file <strong>appName_tags.py</strong> can be named anything as long as it has a <strong>.py</strong> extension.</li>
<li>now paste the following code inside of <strong>appName_tags.py</strong>:</li>

<div class="wp_syntax"><table><tr><td class="line_numbers"><pre>1
2
3
4
5
6
</pre></td><td class="code"><pre class="python" style="font-family:monospace;"><span style="color: #ff7700;font-weight:bold;">from</span> django <span style="color: #ff7700;font-weight:bold;">import</span> template
register <span style="color: #66cc66;">=</span> template.<span style="color: black;">Library</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
&nbsp;
<span style="color: #66cc66;">@</span>register.<span style="color: #008000;">filter</span><span style="color: black;">&#40;</span>&amp;quot<span style="color: #66cc66;">;</span>mongo_id&amp;quot<span style="color: #66cc66;">;</span><span style="color: black;">&#41;</span>
<span style="color: #ff7700;font-weight:bold;">def</span> mongo_id<span style="color: black;">&#40;</span>value<span style="color: black;">&#41;</span>:
    <span style="color: #ff7700;font-weight:bold;">return</span> <span style="color: #008000;">str</span><span style="color: black;">&#40;</span>value._id<span style="color: black;">&#41;</span></pre></td></tr></table></div>

</li>
<li>all that is left is to utilize this filter. to do this, we simply load it inside of the corresponding template and viola:

<div class="wp_syntax"><table><tr><td class="line_numbers"><pre>1
2
3
4
5
6
</pre></td><td class="code"><pre class="html" style="font-family:monospace;">&lt;html&gt;
 &lt;body&gt;
  {% load appName_tags %}
  &lt;p&gt;here is your mongodb record id: {{ object|mongo_id }}&lt;/&gt;
 &lt;/body&gt;
&lt;/html&gt;</pre></td></tr></table></div>

</li>
</ol>
<p>
<em><b>tip</b>: custom filters can also take two arguments instead of the one shown above. check out this example below (referenced from <a href="http://docs.djangoproject.com/en/dev/howto/custom-template-tags/#writing-custom-template-filters">here</a>):</em></p>
]]></content:encoded>
			<wfw:commentRss>http://villaroad.com/return-mongodb-ids-inside-django/feed/</wfw:commentRss>
		<slash:comments>9</slash:comments>
		</item>
		<item>
		<title>Python and Tokyo Dystopia</title>
		<link>http://villaroad.com/python-and-tokyo-dystopia/</link>
		<comments>http://villaroad.com/python-and-tokyo-dystopia/#comments</comments>
		<pubDate>Tue, 29 Dec 2009 20:24:02 +0000</pubDate>
		<dc:creator>Alfred</dc:creator>
				<category><![CDATA[Data Storage]]></category>
		<category><![CDATA[Software]]></category>
		<category><![CDATA[Tools]]></category>
		<category><![CDATA[Web]]></category>

		<guid isPermaLink="false">http://villaroad.com/?p=158</guid>
		<description><![CDATA[<p>So . . . . after some back and forth with Qing, here is what is ready so far. I know it needs work, but I should be able to get some stuff ready for N-GRAM soon.</p>
<p><strong><em>update</em></strong>: (Feb &#8230;</p>]]></description>
				<content:encoded><![CDATA[<p>So . . . . after some back and forth with Qing, here is what is ready so far. I know it needs work, but I should be able to get some stuff ready for N-GRAM soon.</p>
<p><strong><em>update</em></strong>: (Feb 2011) I have chosen to abandon this project some time ago. All of the 1978th products are great and Qing and Mikio are still outstanding. They just no longer suit my use cases.</p>

<div class="wp_syntax"><table><tr><td class="line_numbers"><pre>1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
</pre></td><td class="code"><pre class="c" style="font-family:monospace;"><span style="color: #339933;">#include &quot;Python.h&quot;</span>
<span style="color: #339933;">#include &lt;dystopia.h&gt;</span>
<span style="color: #339933;">#include &lt;stdlib.h&gt;</span>
<span style="color: #339933;">#include &lt;stdbool.h&gt;</span>
<span style="color: #339933;">#include &lt;stdint.h&gt;</span>
&nbsp;
<span style="color: #993333;">static</span> PyObject <span style="color: #339933;">*</span>
put<span style="color: #009900;">&#40;</span>PyObject <span style="color: #339933;">*</span>self<span style="color: #339933;">,</span>PyObject <span style="color: #339933;">*</span>args<span style="color: #009900;">&#41;</span><span style="color: #009900;">&#123;</span>
    <span style="color: #993333;">const</span> <span style="color: #993333;">char</span> <span style="color: #339933;">*</span>dbname<span style="color: #339933;">;</span>
    <span style="color: #993333;">const</span> <span style="color: #993333;">char</span> <span style="color: #339933;">*</span>stext<span style="color: #339933;">;</span>
    <span style="color: #993333;">const</span> <span style="color: #993333;">int</span> <span style="color: #339933;">*</span>kid<span style="color: #339933;">;</span>
    <span style="color: #993333;">int</span> ecode<span style="color: #339933;">;</span>
    bool result<span style="color: #339933;">;</span>
    TCIDB <span style="color: #339933;">*</span>idb<span style="color: #339933;">;</span>
    <span style="color: #b1b100;">if</span> <span style="color: #009900;">&#40;</span><span style="color: #339933;">!</span>PyArg_ParseTuple<span style="color: #009900;">&#40;</span>args<span style="color: #339933;">,</span> <span style="color: #ff0000;">&quot;sis&quot;</span><span style="color: #339933;">,</span> <span style="color: #339933;">&amp;</span>amp<span style="color: #339933;">;</span>dbname<span style="color: #339933;">,</span> <span style="color: #339933;">&amp;</span>amp<span style="color: #339933;">;</span>kid<span style="color: #339933;">,</span> <span style="color: #339933;">&amp;</span>amp<span style="color: #339933;">;</span>stext<span style="color: #009900;">&#41;</span><span style="color: #009900;">&#41;</span>
        <span style="color: #b1b100;">return</span> NULL<span style="color: #339933;">;</span>
    <span style="color: #808080; font-style: italic;">/* create the object */</span>
    idb <span style="color: #339933;">=</span> tcidbnew<span style="color: #009900;">&#40;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
    <span style="color: #808080; font-style: italic;">/* open the database */</span>
    <span style="color: #b1b100;">if</span><span style="color: #009900;">&#40;</span><span style="color: #339933;">!</span>tcidbopen<span style="color: #009900;">&#40;</span>idb<span style="color: #339933;">,</span> dbname<span style="color: #339933;">,</span> IDBOCREAT <span style="color: #339933;">|</span> IDBOWRITER<span style="color: #009900;">&#41;</span><span style="color: #009900;">&#41;</span><span style="color: #009900;">&#123;</span>
        ecode <span style="color: #339933;">=</span> tcidbecode<span style="color: #009900;">&#40;</span>idb<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
        <span style="color: #000066;">fprintf</span><span style="color: #009900;">&#40;</span>stderr<span style="color: #339933;">,</span> <span style="color: #ff0000;">&quot;open error: %s<span style="color: #000099; font-weight: bold;">\n</span>&quot;</span><span style="color: #339933;">,</span> tcidberrmsg<span style="color: #009900;">&#40;</span>ecode<span style="color: #009900;">&#41;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
    <span style="color: #009900;">&#125;</span>
    result <span style="color: #339933;">=</span> tcidbput<span style="color: #009900;">&#40;</span>idb<span style="color: #339933;">,</span><span style="color: #009900;">&#40;</span><span style="color: #993333;">int64_t</span><span style="color: #009900;">&#41;</span>kid<span style="color: #339933;">,</span>stext<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
    <span style="color: #808080; font-style: italic;">/* close the database */</span>
    <span style="color: #b1b100;">if</span><span style="color: #009900;">&#40;</span><span style="color: #339933;">!</span>tcidbclose<span style="color: #009900;">&#40;</span>idb<span style="color: #009900;">&#41;</span><span style="color: #009900;">&#41;</span><span style="color: #009900;">&#123;</span>
        ecode <span style="color: #339933;">=</span> tcidbecode<span style="color: #009900;">&#40;</span>idb<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
        <span style="color: #000066;">fprintf</span><span style="color: #009900;">&#40;</span>stderr<span style="color: #339933;">,</span> <span style="color: #ff0000;">&quot;close error: %s<span style="color: #000099; font-weight: bold;">\n</span>&quot;</span><span style="color: #339933;">,</span> tcidberrmsg<span style="color: #009900;">&#40;</span>ecode<span style="color: #009900;">&#41;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
    <span style="color: #009900;">&#125;</span>
    <span style="color: #808080; font-style: italic;">/* delete the object */</span>
    tcidbdel<span style="color: #009900;">&#40;</span>idb<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
    <span style="color: #b1b100;">return</span> Py_BuildValue<span style="color: #009900;">&#40;</span><span style="color: #ff0000;">&quot;b&quot;</span><span style="color: #339933;">,</span>result<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
<span style="color: #009900;">&#125;</span>
&nbsp;
<span style="color: #993333;">static</span> PyObject <span style="color: #339933;">*</span>
search<span style="color: #009900;">&#40;</span>PyObject <span style="color: #339933;">*</span>self<span style="color: #339933;">,</span> PyObject <span style="color: #339933;">*</span>args<span style="color: #009900;">&#41;</span><span style="color: #009900;">&#123;</span>
    <span style="color: #993333;">const</span> <span style="color: #993333;">char</span> <span style="color: #339933;">*</span>stext<span style="color: #339933;">;</span>
    <span style="color: #993333;">const</span> <span style="color: #993333;">char</span> <span style="color: #339933;">*</span>dbname<span style="color: #339933;">;</span>
    TCIDB <span style="color: #339933;">*</span>idb<span style="color: #339933;">;</span>
    <span style="color: #993333;">int</span> ecode<span style="color: #339933;">,</span> rnum<span style="color: #339933;">,</span> i<span style="color: #339933;">;</span>
    <span style="color: #993333;">uint64_t</span> <span style="color: #339933;">*</span>result<span style="color: #339933;">;</span>
    <span style="color: #993333;">char</span> <span style="color: #339933;">*</span>text<span style="color: #339933;">;</span>
    PyObject<span style="color: #339933;">*</span> pList<span style="color: #339933;">;</span>
&nbsp;
    <span style="color: #b1b100;">if</span> <span style="color: #009900;">&#40;</span><span style="color: #339933;">!</span>PyArg_ParseTuple<span style="color: #009900;">&#40;</span>args<span style="color: #339933;">,</span> <span style="color: #ff0000;">&quot;ss&quot;</span><span style="color: #339933;">,</span> <span style="color: #339933;">&amp;</span>amp<span style="color: #339933;">;</span>dbname<span style="color: #339933;">,</span> <span style="color: #339933;">&amp;</span>amp<span style="color: #339933;">;</span>stext<span style="color: #009900;">&#41;</span><span style="color: #009900;">&#41;</span>
        <span style="color: #b1b100;">return</span> NULL<span style="color: #339933;">;</span>
&nbsp;
    <span style="color: #808080; font-style: italic;">/* create the object */</span>
    idb <span style="color: #339933;">=</span> tcidbnew<span style="color: #009900;">&#40;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
&nbsp;
    <span style="color: #808080; font-style: italic;">/* open the database */</span>
    <span style="color: #b1b100;">if</span><span style="color: #009900;">&#40;</span><span style="color: #339933;">!</span>tcidbopen<span style="color: #009900;">&#40;</span>idb<span style="color: #339933;">,</span> dbname<span style="color: #339933;">,</span> IDBOREADER <span style="color: #339933;">|</span> IDBONOLCK<span style="color: #009900;">&#41;</span><span style="color: #009900;">&#41;</span><span style="color: #009900;">&#123;</span>
        ecode <span style="color: #339933;">=</span> tcidbecode<span style="color: #009900;">&#40;</span>idb<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
        <span style="color: #000066;">fprintf</span><span style="color: #009900;">&#40;</span>stderr<span style="color: #339933;">,</span> <span style="color: #ff0000;">&quot;open error: %s<span style="color: #000099; font-weight: bold;">\n</span>&quot;</span><span style="color: #339933;">,</span> tcidberrmsg<span style="color: #009900;">&#40;</span>ecode<span style="color: #009900;">&#41;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
    <span style="color: #009900;">&#125;</span>
    <span style="color: #808080; font-style: italic;">/* search records */</span>
    result <span style="color: #339933;">=</span> tcidbsearch2<span style="color: #009900;">&#40;</span>idb<span style="color: #339933;">,</span> stext<span style="color: #339933;">,</span> <span style="color: #339933;">&amp;</span>amp<span style="color: #339933;">;</span>rnum<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
    pList <span style="color: #339933;">=</span> PyList_New<span style="color: #009900;">&#40;</span>rnum<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
    <span style="color: #b1b100;">if</span><span style="color: #009900;">&#40;</span>result<span style="color: #009900;">&#41;</span><span style="color: #009900;">&#123;</span>
        <span style="color: #b1b100;">for</span><span style="color: #009900;">&#40;</span>i <span style="color: #339933;">=</span> <span style="color: #0000dd;">0</span><span style="color: #339933;">;</span> i <span style="color: #339933;">&lt;</span> rnum<span style="color: #339933;">;</span> i<span style="color: #339933;">++</span><span style="color: #009900;">&#41;</span><span style="color: #009900;">&#123;</span>
            <span style="color: #666666; font-style: italic;">// printf(&quot;r[i]:%lld\n&quot;,result[i]);</span>
            PyList_SetItem<span style="color: #009900;">&#40;</span>pList<span style="color: #339933;">,</span> i<span style="color: #339933;">,</span> Py_BuildValue<span style="color: #009900;">&#40;</span><span style="color: #ff0000;">&quot;i&quot;</span><span style="color: #339933;">,</span> <span style="color: #009900;">&#40;</span><span style="color: #993333;">int</span><span style="color: #009900;">&#41;</span>result<span style="color: #009900;">&#91;</span>i<span style="color: #009900;">&#93;</span><span style="color: #009900;">&#41;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
        <span style="color: #009900;">&#125;</span>
        tcfree<span style="color: #009900;">&#40;</span>result<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
    <span style="color: #009900;">&#125;</span> <span style="color: #b1b100;">else</span> <span style="color: #009900;">&#123;</span>
        ecode <span style="color: #339933;">=</span> tcidbecode<span style="color: #009900;">&#40;</span>idb<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
        <span style="color: #000066;">fprintf</span><span style="color: #009900;">&#40;</span>stderr<span style="color: #339933;">,</span> <span style="color: #ff0000;">&quot;search error: %s<span style="color: #000099; font-weight: bold;">\n</span>&quot;</span><span style="color: #339933;">,</span> tcidberrmsg<span style="color: #009900;">&#40;</span>ecode<span style="color: #009900;">&#41;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
    <span style="color: #009900;">&#125;</span>
&nbsp;
    <span style="color: #808080; font-style: italic;">/* close the database */</span>
    <span style="color: #b1b100;">if</span><span style="color: #009900;">&#40;</span><span style="color: #339933;">!</span>tcidbclose<span style="color: #009900;">&#40;</span>idb<span style="color: #009900;">&#41;</span><span style="color: #009900;">&#41;</span><span style="color: #009900;">&#123;</span>
        ecode <span style="color: #339933;">=</span> tcidbecode<span style="color: #009900;">&#40;</span>idb<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
        <span style="color: #000066;">fprintf</span><span style="color: #009900;">&#40;</span>stderr<span style="color: #339933;">,</span> <span style="color: #ff0000;">&quot;close error: %s<span style="color: #000099; font-weight: bold;">\n</span>&quot;</span><span style="color: #339933;">,</span> tcidberrmsg<span style="color: #009900;">&#40;</span>ecode<span style="color: #009900;">&#41;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
    <span style="color: #009900;">&#125;</span>
&nbsp;
    <span style="color: #808080; font-style: italic;">/* delete the object */</span>
    tcidbdel<span style="color: #009900;">&#40;</span>idb<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
&nbsp;
    <span style="color: #b1b100;">return</span> Py_BuildValue<span style="color: #009900;">&#40;</span><span style="color: #ff0000;">&quot;O&quot;</span><span style="color: #339933;">,</span>pList<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
<span style="color: #009900;">&#125;</span>
&nbsp;
PyMethodDef methods<span style="color: #009900;">&#91;</span><span style="color: #009900;">&#93;</span> <span style="color: #339933;">=</span> <span style="color: #009900;">&#123;</span>
  <span style="color: #009900;">&#123;</span><span style="color: #ff0000;">&quot;search&quot;</span><span style="color: #339933;">,</span> search<span style="color: #339933;">,</span> METH_VARARGS<span style="color: #009900;">&#125;</span><span style="color: #339933;">,</span>
  <span style="color: #009900;">&#123;</span><span style="color: #ff0000;">&quot;put&quot;</span><span style="color: #339933;">,</span> put<span style="color: #339933;">,</span> METH_VARARGS<span style="color: #009900;">&#125;</span><span style="color: #339933;">,</span>
  <span style="color: #009900;">&#123;</span>NULL<span style="color: #339933;">,</span> NULL<span style="color: #009900;">&#125;</span><span style="color: #339933;">,</span>
<span style="color: #009900;">&#125;</span><span style="color: #339933;">;</span>
&nbsp;
<span style="color: #993333;">void</span> initpykhufu<span style="color: #009900;">&#40;</span><span style="color: #009900;">&#41;</span><span style="color: #009900;">&#123;</span>
    PyObject<span style="color: #339933;">*</span> m<span style="color: #339933;">;</span>
    m <span style="color: #339933;">=</span> Py_InitModule<span style="color: #009900;">&#40;</span><span style="color: #ff0000;">&quot;pykhufu&quot;</span><span style="color: #339933;">,</span> methods<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
<span style="color: #009900;">&#125;</span></pre></td></tr></table></div>

]]></content:encoded>
			<wfw:commentRss>http://villaroad.com/python-and-tokyo-dystopia/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Monitor NAS Space</title>
		<link>http://villaroad.com/monitor-nas-space/</link>
		<comments>http://villaroad.com/monitor-nas-space/#comments</comments>
		<pubDate>Wed, 22 Jul 2009 21:47:08 +0000</pubDate>
		<dc:creator>Alfred</dc:creator>
				<category><![CDATA[BASH]]></category>

		<guid isPermaLink="false">http://villaroad.com/?p=122</guid>
		<description><![CDATA[<p>This handy script can be run daily to monitor how much storage you have left. You will find this most useful when you are leasing space in a VPS environment such as slice host or the like. I added email &#8230;</p>]]></description>
				<content:encoded><![CDATA[<p>This handy script can be run daily to monitor how much storage you have left. You will find this most useful when you are leasing space in a VPS environment such as slice host or the like. I added email to the script to send an alert when things go awry <img src='http://villaroad.com/wp-includes/images/smilies/icon_wink.gif' alt=';)' class='wp-smiley' /> </p>

<div class="wp_syntax"><table><tr><td class="line_numbers"><pre>1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
</pre></td><td class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #666666; font-style: italic;">#!/bin/bash</span>
<span style="color: #666666; font-style: italic;">#*** SET ME FIRST ***#</span>
<span style="color: #007800;">NASUSER</span>=<span style="color: #000000; font-weight: bold;">&amp;</span>quot;Your-User-Name<span style="color: #000000; font-weight: bold;">&amp;</span>quot;
<span style="color: #007800;">NASPASS</span>=<span style="color: #000000; font-weight: bold;">&amp;</span>quot;Your-Password<span style="color: #000000; font-weight: bold;">&amp;</span>quot;
<span style="color: #007800;">NASIP</span>=<span style="color: #000000; font-weight: bold;">&amp;</span>quot;nas.yourcorp.com<span style="color: #000000; font-weight: bold;">&amp;</span>quot;
<span style="color: #007800;">NASROOT</span>=<span style="color: #000000; font-weight: bold;">&amp;</span>quot;<span style="color: #000000; font-weight: bold;">/</span>username<span style="color: #000000; font-weight: bold;">&amp;</span>quot;
<span style="color: #007800;">NASMNTPOINT</span>=<span style="color: #000000; font-weight: bold;">&amp;</span>quot;<span style="color: #000000; font-weight: bold;">/</span>mnt<span style="color: #000000; font-weight: bold;">/</span>nas<span style="color: #000000; font-weight: bold;">&amp;</span>quot;
<span style="color: #007800;">EMAILID</span>=<span style="color: #000000; font-weight: bold;">&amp;</span>quot;admin<span style="color: #000000; font-weight: bold;">@</span>yourcorp.com<span style="color: #000000; font-weight: bold;">&amp;</span>quot;
<span style="color: #666666; font-style: italic;">#*** END SET ME ***#</span>
&nbsp;
<span style="color: #007800;">GETNASIP</span>=$<span style="color: #7a0874; font-weight: bold;">&#40;</span>host <span style="color: #800000;">${NASIP}</span> <span style="color: #000000; font-weight: bold;">|</span> <span style="color: #c20cb9; font-weight: bold;">awk</span> <span style="color: #ff0000;">'{ print $4}'</span><span style="color: #7a0874; font-weight: bold;">&#41;</span>
&nbsp;
<span style="color: #666666; font-style: italic;"># Default warning limit is set to 17GiB</span>
<span style="color: #007800;">LIMIT</span>=<span style="color: #000000; font-weight: bold;">&amp;</span>quot;<span style="color: #000000;">17</span><span style="color: #000000; font-weight: bold;">&amp;</span>quot;
&nbsp;
<span style="color: #666666; font-style: italic;"># Failsafe</span>
<span style="color: #7a0874; font-weight: bold;">&#91;</span> <span style="color: #000000; font-weight: bold;">!</span> <span style="color: #660033;">-d</span> <span style="color: #800000;">${NASMNTPOINT}</span> <span style="color: #7a0874; font-weight: bold;">&#93;</span> <span style="color: #000000; font-weight: bold;">&amp;</span>amp;<span style="color: #000000; font-weight: bold;">&amp;</span>amp; <span style="color: #c20cb9; font-weight: bold;">mkdir</span> <span style="color: #660033;">-p</span> <span style="color: #800000;">${NASMNTPOINT}</span>
<span style="color: #c20cb9; font-weight: bold;">mount</span> <span style="color: #000000; font-weight: bold;">|</span> <span style="color: #c20cb9; font-weight: bold;">grep</span> <span style="color: #000000; font-weight: bold;">//</span><span style="color: #800000;">${GETNASIP}</span><span style="color: #000000; font-weight: bold;">/</span><span style="color: #800000;">${NASUSER}</span>
&nbsp;
<span style="color: #666666; font-style: italic;"># if not mounted, just mount nas</span>
<span style="color: #7a0874; font-weight: bold;">&#91;</span> <span style="color: #007800;">$?</span> <span style="color: #660033;">-eq</span> <span style="color: #000000;">0</span> <span style="color: #7a0874; font-weight: bold;">&#93;</span> <span style="color: #000000; font-weight: bold;">&amp;</span>amp;<span style="color: #000000; font-weight: bold;">&amp;</span>amp; : <span style="color: #000000; font-weight: bold;">||</span> <span style="color: #c20cb9; font-weight: bold;">mount</span> <span style="color: #660033;">-t</span> cifs <span style="color: #000000; font-weight: bold;">//</span><span style="color: #800000;">${NASIP}</span><span style="color: #000000; font-weight: bold;">/</span><span style="color: #800000;">${NASUSER}</span> <span style="color: #660033;">-o</span> <span style="color: #007800;">username</span>=<span style="color: #800000;">${NASUSER}</span>,<span style="color: #007800;">password</span>=<span style="color: #800000;">${NASPASS}</span> <span style="color: #800000;">${NASMNTPOINT}</span>
<span style="color: #7a0874; font-weight: bold;">cd</span> <span style="color: #800000;">${NASMNTPOINT}</span>
&nbsp;
<span style="color: #666666; font-style: italic;"># get NAS disk space</span>
<span style="color: #007800;">nSPACE</span>=$<span style="color: #7a0874; font-weight: bold;">&#40;</span><span style="color: #c20cb9; font-weight: bold;">du</span> <span style="color: #660033;">-hs</span><span style="color: #000000; font-weight: bold;">|</span><span style="color: #c20cb9; font-weight: bold;">cut</span> <span style="color: #660033;">-d</span><span style="color: #ff0000;">'G'</span> -f1<span style="color: #7a0874; font-weight: bold;">&#41;</span>
<span style="color: #666666; font-style: italic;"># Bug fix</span>
<span style="color: #666666; font-style: italic;"># get around floating point by rounding off e.g 5.7G stored in $nSPACE</span>
<span style="color: #666666; font-style: italic;"># as shell cannot do floating point</span>
<span style="color: #007800;">SPACE</span>=$<span style="color: #7a0874; font-weight: bold;">&#40;</span><span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #007800;">$nSPACE</span> <span style="color: #000000; font-weight: bold;">|</span> <span style="color: #c20cb9; font-weight: bold;">cut</span> -d. -f1<span style="color: #7a0874; font-weight: bold;">&#41;</span>
&nbsp;
<span style="color: #7a0874; font-weight: bold;">cd</span> <span style="color: #000000; font-weight: bold;">/</span>
<span style="color: #c20cb9; font-weight: bold;">umount</span> <span style="color: #800000;">${NASMNTPOINT}</span>
&nbsp;
<span style="color: #666666; font-style: italic;"># compare and send an email</span>
<span style="color: #000000; font-weight: bold;">if</span> <span style="color: #7a0874; font-weight: bold;">&#91;</span> <span style="color: #007800;">$SPACE</span> <span style="color: #660033;">-ge</span> <span style="color: #007800;">$LIMIT</span> <span style="color: #7a0874; font-weight: bold;">&#93;</span>
<span style="color: #000000; font-weight: bold;">then</span>
        logger <span style="color: #000000; font-weight: bold;">&amp;</span>quot;Warning: NAS Running Out Of Disk Space <span style="color: #7a0874; font-weight: bold;">&#91;</span><span style="color: #800000;">${SPACE}</span> G<span style="color: #7a0874; font-weight: bold;">&#93;</span><span style="color: #000000; font-weight: bold;">&amp;</span>quot;
        mail <span style="color: #660033;">-s</span> <span style="color: #ff0000;">'NAS Server Disk Space'</span> <span style="color: #800000;">${EMAILID}</span> <span style="color: #000000; font-weight: bold;">&amp;</span>lt;<span style="color: #000000; font-weight: bold;">&amp;</span>lt;EOF
NAS server <span style="color: #7a0874; font-weight: bold;">&#91;</span> mounted at $<span style="color: #7a0874; font-weight: bold;">&#40;</span><span style="color: #c20cb9; font-weight: bold;">hostname</span><span style="color: #7a0874; font-weight: bold;">&#41;</span> <span style="color: #7a0874; font-weight: bold;">&#93;</span> is running out of disk space<span style="color: #000000; font-weight: bold;">!!!</span>
Current allocation <span style="color: #800000;">${SPACE}</span>G <span style="color: #000000; font-weight: bold;">@</span> $<span style="color: #7a0874; font-weight: bold;">&#40;</span><span style="color: #c20cb9; font-weight: bold;">date</span><span style="color: #7a0874; font-weight: bold;">&#41;</span>
EOF
<span style="color: #000000; font-weight: bold;">else</span>
    logger <span style="color: #000000; font-weight: bold;">&amp;</span>quot;$<span style="color: #7a0874; font-weight: bold;">&#40;</span><span style="color: #c20cb9; font-weight: bold;">basename</span> <span style="color: #007800;">$0</span><span style="color: #7a0874; font-weight: bold;">&#41;</span> ~ NAS server <span style="color: #800000;">${NASIP}</span> has sufficent disk space <span style="color: #000000; font-weight: bold;">for</span> backup<span style="color: #000000; font-weight: bold;">!&amp;</span>quot;
<span style="color: #000000; font-weight: bold;">fi</span></pre></td></tr></table></div>

]]></content:encoded>
			<wfw:commentRss>http://villaroad.com/monitor-nas-space/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Port Forwarding</title>
		<link>http://villaroad.com/port-forwarding/</link>
		<comments>http://villaroad.com/port-forwarding/#comments</comments>
		<pubDate>Sat, 23 May 2009 00:53:57 +0000</pubDate>
		<dc:creator>Alfred</dc:creator>
				<category><![CDATA[Python]]></category>

		<guid isPermaLink="false">http://villaroad.com/?p=108</guid>
		<description><![CDATA[<p>To continue the security talks I decided to show you all how to create unpartisan port forwarded. This daemon will allow eleven constant active connections. Just make sure that your destination is able to interrupt the data properly. Also, I &#8230;</p>]]></description>
				<content:encoded><![CDATA[<p>To continue the security talks I decided to show you all how to create unpartisan port forwarded. This daemon will allow eleven constant active connections. Just make sure that your destination is able to interrupt the data properly. Also, I should note that you should use this via spawning a new process and kill it by killing the process id, PID. This is the best way. If you are a windows muchacho, then I am not sure how to help you :/</p>

<div class="wp_syntax"><table><tr><td class="line_numbers"><pre>1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
</pre></td><td class="code"><pre class="python" style="font-family:monospace;"><span style="color: #808080; font-style: italic;">#!/usr/bin/env python</span>
&nbsp;
<span style="color: #808080; font-style: italic;">#</span>
<span style="color: #808080; font-style: italic;">#     all modules in the stdlib</span>
<span style="color: #808080; font-style: italic;">#</span>
&nbsp;
<span style="color: #ff7700;font-weight:bold;">import</span> <span style="color: #dc143c;">socket</span>
<span style="color: #ff7700;font-weight:bold;">import</span> <span style="color: #dc143c;">sys</span>
<span style="color: #ff7700;font-weight:bold;">import</span> <span style="color: #dc143c;">thread</span>
&nbsp;
<span style="color: #ff7700;font-weight:bold;">def</span> main<span style="color: black;">&#40;</span>setup<span style="color: #66cc66;">,</span> error<span style="color: black;">&#41;</span>:
    <span style="color: #dc143c;">sys</span>.<span style="color: black;">stderr</span> <span style="color: #66cc66;">=</span> <span style="color: #008000;">file</span><span style="color: black;">&#40;</span>error<span style="color: #66cc66;">,</span> <span style="color: #483d8b;">'a'</span><span style="color: black;">&#41;</span>
    <span style="color: #ff7700;font-weight:bold;">for</span> settings <span style="color: #ff7700;font-weight:bold;">in</span> parse<span style="color: black;">&#40;</span>setup<span style="color: black;">&#41;</span>:
        <span style="color: #dc143c;">thread</span>.<span style="color: black;">start_new_thread</span><span style="color: black;">&#40;</span>server<span style="color: #66cc66;">,</span> settings<span style="color: black;">&#41;</span>
    lock <span style="color: #66cc66;">=</span> <span style="color: #dc143c;">thread</span>.<span style="color: black;">allocate_lock</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
    lock.<span style="color: black;">acquire</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
    lock.<span style="color: black;">acquire</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
&nbsp;
<span style="color: #ff7700;font-weight:bold;">def</span> parse<span style="color: black;">&#40;</span>setup<span style="color: black;">&#41;</span>:
    settings <span style="color: #66cc66;">=</span> <span style="color: #008000;">list</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
    <span style="color: #ff7700;font-weight:bold;">for</span> line <span style="color: #ff7700;font-weight:bold;">in</span> <span style="color: #008000;">file</span><span style="color: black;">&#40;</span>setup<span style="color: black;">&#41;</span>:
        parts <span style="color: #66cc66;">=</span> line.<span style="color: black;">split</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
        settings.<span style="color: black;">append</span><span style="color: black;">&#40;</span><span style="color: black;">&#40;</span>parts<span style="color: black;">&#91;</span><span style="color: #ff4500;">0</span><span style="color: black;">&#93;</span><span style="color: #66cc66;">,</span> <span style="color: #008000;">int</span><span style="color: black;">&#40;</span>parts<span style="color: black;">&#91;</span><span style="color: #ff4500;">1</span><span style="color: black;">&#93;</span><span style="color: black;">&#41;</span><span style="color: #66cc66;">,</span> <span style="color: #008000;">int</span><span style="color: black;">&#40;</span>parts<span style="color: black;">&#91;</span><span style="color: #ff4500;">2</span><span style="color: black;">&#93;</span><span style="color: black;">&#41;</span><span style="color: black;">&#41;</span><span style="color: black;">&#41;</span>
    <span style="color: #ff7700;font-weight:bold;">return</span> settings
&nbsp;
<span style="color: #ff7700;font-weight:bold;">def</span> server<span style="color: black;">&#40;</span>*settings<span style="color: black;">&#41;</span>:
    <span style="color: #ff7700;font-weight:bold;">try</span>:
        dock_socket <span style="color: #66cc66;">=</span> <span style="color: #dc143c;">socket</span>.<span style="color: #dc143c;">socket</span><span style="color: black;">&#40;</span><span style="color: #dc143c;">socket</span>.<span style="color: black;">AF_INET</span><span style="color: #66cc66;">,</span> <span style="color: #dc143c;">socket</span>.<span style="color: black;">SOCK_STREAM</span><span style="color: black;">&#41;</span>
        dock_socket.<span style="color: black;">bind</span><span style="color: black;">&#40;</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">''</span><span style="color: #66cc66;">,</span> settings<span style="color: black;">&#91;</span><span style="color: #ff4500;">2</span><span style="color: black;">&#93;</span><span style="color: black;">&#41;</span><span style="color: black;">&#41;</span>
        dock_socket.<span style="color: black;">listen</span><span style="color: black;">&#40;</span><span style="color: #ff4500;">5</span><span style="color: black;">&#41;</span>
        <span style="color: #ff7700;font-weight:bold;">while</span> <span style="color: #008000;">True</span>:
            client_socket <span style="color: #66cc66;">=</span> dock_socket.<span style="color: black;">accept</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span><span style="color: black;">&#91;</span><span style="color: #ff4500;">0</span><span style="color: black;">&#93;</span>
            server_socket <span style="color: #66cc66;">=</span> <span style="color: #dc143c;">socket</span>.<span style="color: #dc143c;">socket</span><span style="color: black;">&#40;</span><span style="color: #dc143c;">socket</span>.<span style="color: black;">AF_INET</span><span style="color: #66cc66;">,</span> <span style="color: #dc143c;">socket</span>.<span style="color: black;">SOCK_STREAM</span><span style="color: black;">&#41;</span>
            server_socket.<span style="color: black;">connect</span><span style="color: black;">&#40;</span><span style="color: black;">&#40;</span>settings<span style="color: black;">&#91;</span><span style="color: #ff4500;">0</span><span style="color: black;">&#93;</span><span style="color: #66cc66;">,</span> settings<span style="color: black;">&#91;</span><span style="color: #ff4500;">1</span><span style="color: black;">&#93;</span><span style="color: black;">&#41;</span><span style="color: black;">&#41;</span>
            <span style="color: #dc143c;">thread</span>.<span style="color: black;">start_new_thread</span><span style="color: black;">&#40;</span>forward<span style="color: #66cc66;">,</span> <span style="color: black;">&#40;</span>client_socket<span style="color: #66cc66;">,</span> server_socket<span style="color: black;">&#41;</span><span style="color: black;">&#41;</span>
            <span style="color: #dc143c;">thread</span>.<span style="color: black;">start_new_thread</span><span style="color: black;">&#40;</span>forward<span style="color: #66cc66;">,</span> <span style="color: black;">&#40;</span>server_socket<span style="color: #66cc66;">,</span> client_socket<span style="color: black;">&#41;</span><span style="color: black;">&#41;</span>
    <span style="color: #ff7700;font-weight:bold;">finally</span>:
        <span style="color: #dc143c;">thread</span>.<span style="color: black;">start_new_thread</span><span style="color: black;">&#40;</span>server<span style="color: #66cc66;">,</span> settings<span style="color: black;">&#41;</span>
&nbsp;
<span style="color: #ff7700;font-weight:bold;">def</span> forward<span style="color: black;">&#40;</span>source<span style="color: #66cc66;">,</span> destination<span style="color: black;">&#41;</span>:
    <span style="color: #dc143c;">string</span> <span style="color: #66cc66;">=</span> <span style="color: #483d8b;">' '</span>
    <span style="color: #ff7700;font-weight:bold;">while</span> <span style="color: #dc143c;">string</span>:
        <span style="color: #dc143c;">string</span> <span style="color: #66cc66;">=</span> source.<span style="color: black;">recv</span><span style="color: black;">&#40;</span><span style="color: #ff4500;">1024</span><span style="color: black;">&#41;</span>
        <span style="color: #ff7700;font-weight:bold;">if</span> <span style="color: #dc143c;">string</span>:
            destination.<span style="color: black;">sendall</span><span style="color: black;">&#40;</span><span style="color: #dc143c;">string</span><span style="color: black;">&#41;</span>
        <span style="color: #ff7700;font-weight:bold;">else</span>:
            source.<span style="color: black;">shutdown</span><span style="color: black;">&#40;</span><span style="color: #dc143c;">socket</span>.<span style="color: black;">SHUT_RD</span><span style="color: black;">&#41;</span>
            destination.<span style="color: black;">shutdown</span><span style="color: black;">&#40;</span><span style="color: #dc143c;">socket</span>.<span style="color: black;">SHUT_WR</span><span style="color: black;">&#41;</span>
&nbsp;
<span style="color: #ff7700;font-weight:bold;">if</span> __name__ <span style="color: #66cc66;">==</span> <span style="color: #483d8b;">'__main__'</span>:
    main<span style="color: black;">&#40;</span><span style="color: #483d8b;">'proxy.cfg'</span><span style="color: #66cc66;">,</span> <span style="color: #483d8b;">'error.log'</span><span style="color: black;">&#41;</span></pre></td></tr></table></div>

]]></content:encoded>
			<wfw:commentRss>http://villaroad.com/port-forwarding/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
	</channel>
</rss>
