<?xml version="1.0" encoding="UTF-8"?>
<!--Generated by Squarespace Site Server v5.11.81 (http://www.squarespace.com/) on Sun, 20 May 2012 07:43:59 GMT--><rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:wfw="http://wellformedweb.org/CommentAPI/" xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd" xmlns:dc="http://purl.org/dc/elements/1.1/" version="2.0"><channel><title>Posts</title><link>http://www.gridfire.com/posts/</link><description></description><lastBuildDate>Tue, 27 Sep 2011 21:44:39 +0000</lastBuildDate><copyright></copyright><language>en-GB</language><generator>Squarespace Site Server v5.11.81 (http://www.squarespace.com/)</generator><item><title>Postgresql - Finding unused indexes</title><dc:creator>Matthew Cooke</dc:creator><pubDate>Sat, 24 Sep 2011 19:39:47 +0000</pubDate><link>http://www.gridfire.com/posts/2011/9/24/postgresql-finding-unused-indexes.html</link><guid isPermaLink="false">309582:3212659:12970415</guid><description><![CDATA[<p>Database indexes are very useful, but the down side is that updating or adding rows to tables becomes more intensive as all the indexes also need updating.</p>
<p>If you are using postgresql then you probably already have access to information on index usage as this information is usually collected by default - if not, you can find more information here in the <a href="http://www.postgresql.org/docs/9.0/static/monitoring-stats.html">postgresql docs on statistics collection</a>.</p>
<p>This query (tested on PG9) gives some information on index usage and index size:</p>
<pre>SELECT t.tablename, 
  indexname, 
  c.reltuples AS num_rows, 
  pg_size_pretty ( pg_relation_size ( t.tablename::text ) ) AS table_size, 
  pg_size_pretty ( pg_relation_size ( indexrelname::text ) ) AS index_size, 
  CASE 
    WHEN x.is_unique = 1 
    THEN 'Y' 
    ELSE 'N' 
  END AS UNIQUE, 
  idx_scan AS number_of_scans, 
  idx_tup_read AS tuples_read, 
  idx_tup_fetch AS tuples_fetched 
FROM pg_tables t 
LEFT 
OUTER JOIN pg_class c 
  ON t.tablename = c.relname 
LEFT 
OUTER JOIN ( 
      SELECT indrelid, 
        max ( CAST ( indisunique AS integer ) ) AS is_unique 
      FROM pg_index 
      GROUP BY indrelid ) x 
  ON c.oid = x.indrelid 
LEFT 
OUTER JOIN ( 
      SELECT c.relname AS ctablename, 
        ipg.relname AS indexname, 
        x.indnatts AS number_of_columns, 
        idx_scan, 
        idx_tup_read, 
        idx_tup_fetch, 
        indexrelname 
      FROM pg_index x JOIN pg_class c 
        ON c.oid = x.indrelid JOIN pg_class ipg 
        ON ipg.oid = x.indexrelid JOIN pg_stat_all_indexes psai 
        ON x.indexrelid = psai.indexrelid ) AS foo 
  ON t.tablename = foo.ctablename 
WHERE t.schemaname = 'public' 
ORDER BY number_of_scans desc; </pre>
<p>If you assume that all indexes that are needed have been used at least once, this next query tells you how big all the unused indexes are:</p>
<pre>SELECT pg_size_pretty ( sum ( pg_relation_size ( indexrelname::text ) ) ::bigint ) AS index_size 
FROM pg_tables t 
LEFT 
OUTER JOIN pg_class c 
  ON t.tablename = c.relname 
LEFT 
OUTER JOIN ( 
      SELECT indrelid, 
        max ( CAST ( indisunique AS integer ) ) AS is_unique 
      FROM pg_index 
      GROUP BY indrelid ) x 
  ON c.oid = x.indrelid 
LEFT 
OUTER JOIN ( 
      SELECT c.relname AS ctablename, 
        ipg.relname AS indexname, 
        x.indnatts AS number_of_columns, 
        idx_scan, 
        idx_tup_read, 
        idx_tup_fetch, 
        indexrelname 
      FROM pg_index x JOIN pg_class c 
        ON c.oid = x.indrelid JOIN pg_class ipg 
        ON ipg.oid = x.indexrelid JOIN pg_stat_all_indexes psai 
        ON x.indexrelid = psai.indexrelid ) AS foo 
  ON t.tablename = foo.ctablename 
WHERE t.schemaname = 'public' 
  AND idx_scan =0; 
</pre>
<p>You can inverse that last query to get the total size of all the in-use indexes by changing the idx_scan=0 to idx_scan&gt;0. If you are heavily reliant on the database fitting into physical RAM (as I usually am) then you may want to keep an eye on the index size, along with the size of your tables.</p>
<p>If you are looking to reduce the size of the indexes you could look at some of your biggest indexes and think about the queries that actually run against the table - do you really need to index the *whole* table for your queries? If not, you could look at changing the index to a <a href="http://www.postgresql.org/docs/9.0/static/indexes-partial.html">partial index</a>&nbsp;which is basically an index which only covers part of a table based on a where clause.</p>
<p>There are a few other things that can help get down index size , ensure the auto vacuum setting are aggressive enough, dump-restore your database periodically,&nbsp;delete rows in the underlying table, etc, :)</p>]]></description><wfw:commentRss>http://www.gridfire.com/posts/rss-comments-entry-12970415.xml</wfw:commentRss></item><item><title>Using XCode with SVN - some Gotchas</title><dc:creator>Matthew Cooke</dc:creator><pubDate>Sat, 08 Aug 2009 11:01:21 +0000</pubDate><link>http://www.gridfire.com/posts/2009/8/8/using-xcode-with-svn-some-gotchas.html</link><guid isPermaLink="false">309582:3212659:4844504</guid><description><![CDATA[<p>I mainly do Java development using Eclipse with SVN but recently I've been playing around with a bit of iphone development using XCode.</p>
<p>I've included some below some Gotchas about using XCode with SVN in the hope that it might save someone else making the same mistakes as me...</p>
<ul>
<li>The left hand navigator pane in xcode labelled files and groups does not show a filesystem view of the folder containing the project files.</li>
<li>XCode projects need files adding in manually - the project structures does not have to match file system directory structures at all.</li>
<li>You will probably want to exclude the build directory - XCode will not do this for you.</li>
<li>Be aware that's it's easy to accidentily create links to files in the projects rather than putting the files into the project folder itself. Make sure when you drop a file into the project that you are copying it into the actual project and not just linking to an external file.</li>
<li>Committing the files to version control. When you are committing you will probably also want to commit the project file itself, you may this file if you aren't looking for it but it should show up in the SCM section. This file contains the list of all the files that are actually in the project, so if you don't commit this then the files will all be under version control but when you checkout the project on a fresh machine the files won't be included in the project, probably leaving the project with breakages.</li>
<li>New files that are placed in the project will not be added to version control even if the parent folder is already under version control and the entire project was checked out from an SCM such as SVN originally.</li>
<li>When using XCode to check out from a repository - you probably then want to tell XCode that the project is from that repository.</li>
<li>There is no way to get a graphical project level diff within Xcode, such as the syncronization view in Eclipse which helps you to merge and rollback changes prior to doing a commit. File level diffs can be done and to get an idea what files in the project have changes you can right-click the groups and files header on the left hand navigation panel and select SCM, this put a letter next to files depending on their state.</li>
<li>You may want to exclude your user files inside foo.xcodeproj/blah.pbxuser as this has user specific settings</li>
</ul>]]></description><wfw:commentRss>http://www.gridfire.com/posts/rss-comments-entry-4844504.xml</wfw:commentRss></item><item><title>Resizing a Fedora 8 Vmware Fusion image</title><dc:creator>Matthew Cooke</dc:creator><pubDate>Sat, 08 Dec 2007 17:29:00 +0000</pubDate><link>http://www.gridfire.com/posts/2007/12/8/resizing-a-fedora-8-vmware-fusion-image.html</link><guid isPermaLink="false">309582:3212659:2902553</guid><description><![CDATA[<p>This is how I managed to resize a VMware image for Fedora 8.</p>
<p>I started with a VMware Fedora 8 from thoughtpolice and a copy of VMWare Fusion.</p>
<h3>Step 1 Resize the virtual disk</h3>
<p>Under OS X run:<br />
<code>/Applications/VMware Fusion.app/Contents/MacOSdiskTool -X 40Gb fedora-8-i386.vmdk</code></p>
<p>(where  fedora-8-i386.vmdk is the VMWare disk image file)</p>
<h3>Step 2 Create an Ext3 partition in the free space.</h3>
<p>Download the latest GParted ISO.Connect it to Vmware Fusion (VirtualMachine | CD/DVD | Choose Disk Image)</p>
<p>Restart Vmware but during the initial boot go into the virtual bios settings and change the boot order to make CD boot before hard disk.</p>
<p>Restart VMware to boot up GParted. (You can not resize the partition as it shows as unknown - but do not cry at this point!)<br />
Select the free space and turn it into an ext3 partition.</p>
<p>Shutdown and disconnect the Virtual GParted CD Image.</p>
<h3>Step 3 Adding the new partition into the Logical Volume.</h3>
<p>I booted up into fedora to change the Logical Volume size - this may not be the best way to do it but it worked for me. then as root i executed the following.</p>
<p><code>pvcreate /dev/sda3</code></p>
<p><code>vgextend VolGroup00 /dev/sda3</code><br />
<code>vgdisplay -v VolGroup00</code></p>
<p>Somewhere it will display a number for Free PE on /dev/sda3</p>
<p>Now enter<br />
<code>lvextend -l xxxx /dev/VolGroup00/LogVol00</code></p>
<p>where xxxx is the Free PE number.</p>
<p>Finally the last step is a bit scary it resizes the partition to make use of the larger volume whilst the filesystem is mounted. But it did seem to work for me!</p>
<p><code>resize2fs /dev/VolGroup00/LogVol00</code></p>
<p>After a while the result should show something like:</p>
<p><code>resize2fs 1.40.2 (12-Jul-2007)<br />
Filesystem at /dev/VolGroup00/LogVol00 is mounted on /; on-line resizing required<br />
old desc_blocks = 1, new_desc_blocks = 2<br />
Performing an on-line resize of /dev/VolGroup00/LogVol00 to 8380416 (4k) blocks.<br />
The filesystem on /dev/VolGroup00/LogVol00 is now 8380416 blocks long.</code></p>
<p>And bingo my disk size has gone up from 7gig to 30Gig</p>
<p><code> [root@localhost dev]# df -h<br />
Filesystem            Size  Used Avail Use% Mounted on<br />
/dev/mapper/VolGroup00-LogVol00<br />
31G  3.1G   27G  11% /<br />
/dev/sda1             190M   13M  169M   7% /boot<br />
tmpfs                 125M   12K  125M   1% /dev/shm </code></p>
<p align="right"><em>“In the middle of difficulty lies opportunity.”</em></p>]]></description><wfw:commentRss>http://www.gridfire.com/posts/rss-comments-entry-2902553.xml</wfw:commentRss></item><item><title>Why Jotspot is not a good wiki</title><category>Rants</category><dc:creator>Matthew Cooke</dc:creator><pubDate>Wed, 06 Jun 2007 10:30:00 +0000</pubDate><link>http://www.gridfire.com/posts/2007/6/6/why-jotspot-is-not-a-good-wiki.html</link><guid isPermaLink="false">309582:3212659:2979670</guid><description><![CDATA[<p>Let's start off with the basic principals of a wiki. The original C2 wiki covers the basics of <a href="http://c2.com/cgi/wiki?WhyWikiWorks">why wikis work</a> pretty well. Of particular note is the following:</p>
<p><em>"Wiki is not <a href="http://c2.com/cgi/wiki?WysiWyg">WysiWyg</a>. It's an intelligence test of sorts to be able to edit a wiki page. It's not rocket science, but it doesn't appeal to the <a href="http://c2.com/cgi/wiki?VideoAddict">VideoAddict</a>s. If it doesn't appeal, they don't participate, which leaves those of us who read and write to get on with rational discourse."</em></p>
<p>So when I tell you that jotspot is a so called WYSIWYG wiki you would be right to be suspicious! After all ,we all know the kind of crap that WYSIWYG generators generally produce (see MS Frontpage!). But I was willing to give Jotspot a go. Well, here are some of the improvements jotspot has made to the concept of the wiki:</p>
<ul>
<li>Quick and simple to create new page.  
<ul>
<li>Jotspot improves this experience by assuming you perhaps want to create a spreadsheet every time you try to create a new page.</li>
</ul>
</li>
</ul>
<ul>
<li>Wiki-syntax is not WYSIWYG  
<ul>
<li>Jotspot improves on this experience by defaulting you to WYSIWYG rather than wiki syntax this enables lazy users to mess up your wiki markup very quickly.</li>
<li>Better still, using the default WYSIWYG often actually breaks the pages so they can no longer be edited in normal markup mode. Ever. Genius.</li>
</ul>
</li>
</ul>
<ul>
<li>Original wikis used a simple textarea for input.  
<ul>
<li>Jotspot improves on this with Web2.0 goodies such as pages that keep refreshing for no reason and sucking all your CPU.</li>
<li>Jotspot also uses a javascript pulldown menu instead of a button, just to edit in markup mode! This is both annoying and broken in some browsers.</li>
<li>The usual form page doesn't appear in the history, so no navigating back and fore. Instead when you click back you get an annoying javascript popup telling you maybe you didn't want to move off the page because you will lose everything. Which indeed you will.</li>
</ul>
</li>
</ul>
<p>Finally I'd just like to share some of my favourite random jotspot errors:<br /> <strong>Additional information:<br /> com.s3.script.lib.JotLibException: function filterBadHTML: parameter html: no value specified.</strong></p>
<p>After which jotspot becomes unuseable
<script type="text/javascript"></script>
</p>]]></description><wfw:commentRss>http://www.gridfire.com/posts/rss-comments-entry-2979670.xml</wfw:commentRss></item><item><title>HTTP Proxy Caching Vs Application Caches</title><dc:creator>Matthew Cooke</dc:creator><pubDate>Thu, 10 May 2007 16:06:00 +0000</pubDate><link>http://www.gridfire.com/posts/2007/5/10/http-proxy-caching-vs-application-caches.html</link><guid isPermaLink="false">309582:3212659:2902497</guid><description><![CDATA[<p>Developers love redeveloping the wheel and writing caching code is a typical example of this. I have seen several examples of developers writing custom caches inside their web applications without considering if they would be better off using standard proxy caching.</p>
<p>Sometimes you do need to write your own caching code. One case you might need to write your own cache (or wrap <a href="http://ehcache.sourceforge.net/">ehcache</a>) is if you need partial page caching - but for a lot of cases using standard HTTP proxy is a much better option.</p>
<p>Here are some of the many advantages of using standard HTTP proxy caching:</p>
<ul>
<li>Faster pages since resources can be served from the Web-browser cache, ISP caches, office caches and various other caches on the internet which are closer to the client than your server.</li>
<li>Save bandwidth since you have to serve less content from your origin server(s).</li>
<li>You can add caching directly on your server by installing one of several free high performance reverse-proxy caches.</li>
<li>Configure expiry times for resources in one place on your server for *all* the caches, and have all these caches work in a hierarchy right back to your server cache - totally transparently.</li>
</ul>
<p>That is the beauty of using standard proxy caching.</p>
<p>So, how does it work and what do you need to do?<br /> Let's look at the steps involved to make use of all the caches between your server and the client and to add an additional cache on your server (a cache of last resort!).</p>
<h3>Step 1 - Set Correct Expiry Headers for Web Pages you Serve. </h3>
<p>For each resource you serve that you want cached, you need to set caching header that the server will use in the HTTP response. There are various headers related to caching but one that you can use is the <a href="http://www.mnot.net/cache_docs/#CACHE-CONTROL">cache-control</a> with the max-age directive. You can google to find out how to set this for a resource on your particular application server or web server.</p>
<h3>Step 2 - Test the Expiry Headers are Working.<br /> </h3>
<p>There are many ways you can test if the headers are being set successfully. The easiest may be to check in the browser cache to see when the content expires. If the content is not being cached properly you probably want to check that the server does actually send the headers correctly that you tried to configure in step 1. You can install <a href="http://livehttpheaders.mozdev.org/">livehttpheader</a> to see headers in firefox or you can use <a href="http://www.ethereal.com/">ethereal</a> to scan the network traffic and get the headers from that (ethereal can take a while to get to grips with but it is really neat).</p>
<h3>Step 3 (Optional) - Adding a Reverse  Proxy server</h3>
<p>If you want additional caching on your server to further offload resources from your application server you can install a reverse proxy cache. This <a href="http://www.visolve.com/squid/whitepapers/reverseproxy.php">article</a> explains how to configure squid as a reverse proxy, I have done this for <a href="http://www.viralvideochart.com">viralvideochart.com</a> and it works nicely acting a cache that sits in front of our application server.<br /> There are many other reverse proxy caches around (the full version of the <a href="http://www.caucho.com/">Resin</a> application server has one built in for example).</p>
<h3>Step 4 - Enjoy Blistering Fast Serving Speeds!</h3>
<p>That's right with  the correct HTTP headers set and a reverse proxy configured on your server, as the load on the cacheable resources goes up a reasonable reverse-proxy should be able to approach static file serving speeds! Thus offloading a lot of needless work from your application and database server without the need to write yet another application cache.</p>
<p align="right"><em>"...when this baby hits eighty-eight miles per hour... you're gonna see some serious shit."</em></p>
<p align="right">&nbsp;</p>
<p align="right">&nbsp;</p>]]></description><wfw:commentRss>http://www.gridfire.com/posts/rss-comments-entry-2902497.xml</wfw:commentRss></item></channel></rss>
