Using Meta-Programming for Performance in Ruby
Normally we use meta-programming in Ruby for our own convenience as developers, and we swallow the speed hit it gives us as a reasonable trade-off. The way Rak is implemented turns this on its head.
Rak compiles its line matching code on the fly, to speed up searching. Its not as horrendous as it sounds. Here’s a very simplified version of the line matching code without the optimization:
def line_match(filename, regex, line)
if options[:invert_match]
unless line =~ regex
if options[:print_filename]
puts filename
end
puts line
end
else
if line =~ regex
if options[:print_filename]
puts filename
end
puts line
end
end
end
We notice that the options hash never changes once the searching has begun. But we are doing an awful lot of work checking the values in the hash for every single line we are matching against. So we replace it with this:
def compile_line_match(filename, regex, line)
code = []
code << %{def line_match(filename, regex, line) }
if options[:invert_match]
code << %{ unless line =~ regex }
if options[:print_filename]
code << %{ puts filename }
end
code << %{ puts line }
code << %{ end }
else
code << %{ if line =~ regex }
if options[:print_filename]
code << %{ puts filename }
end
code << %{ puts line }
code << %{ end }
end
code << %{end }
module_eval code.join("\\n")
end
I’ve preserved the indenting so you can see the logic of the method. Here all the option tests have been factored out and will only be done once, at startup. Assuming we are not inverting the match and are printing filenames, the final compiled method looks like this:
def line_match(filename, regex, line)
if line =~ regex
puts filename
puts line
end
end
Much smaller. This method is 140 lines long in the full version, so its a much more impressive win there. All told this sped up Rak by about two times when I tested it searching on my Phd repository (3500 files).
There’s more work to do optimizing Rak. It’s still not as fast as Ack, so you might want to use that unless you are a Ruby fanboy like me :). But I have some more ideas….
Update 20/04/08: thankyou to redditer tomel for pointing out a WTF in the above code :).
Really, really slow graphics in Gutsy?
Me too. I was using an ATI card, without any 3d accel. It worked fine on Feisty, but when I upgraded to Gutsy I found I could watch the windows redrawing.
It turns out* that I had xserver-xgl installed from a previous (failed) attempt to get 3d acceleration working in Ubuntu. Feisty didn’t care, but Gutsy loads XGL automatically if it is installed. And XGL with no acceleration is really really slow.
* Anyone who has used linux/ubuntu knows what getting to ‘it turns out’ entails.
Rak, a grep replacement in pure-Ruby
Rak is a tool for searching directories for files matching a regexp, like a more convenient grep. It gives you pretty highlighted output, and uses the Ruby regexp syntax. To install type gem install rak.
Rak is implemented in pure Ruby, so it should work on all platforms. Windows folk will probably find they need to use the –nocolour option (I will fix this next time I boot into Windows for any reason).
NB. Rak is an almost perfect clone of the Perl tool Ack by Andy Lester. He should get all the credit for the idea.
Migrating from Subversion to git
You have a public svn repository served by svnserve on your server. You would like to migrate to a public git server. Here’s how.
Importing your svn repository
First we have to get your subversion repository into git. Let’s say your repository is in /home/dbl/svn/rak (this is totally made up - obviously). I would like to create a git repository in /home/dbl/git/rak.
$ cd /home/dbl/git
$ git-svnimport -v -C rak1 file:///home/dbl/svn/rak
This will initialize the git repository and import the svn history. The -v simply means I can watch the progress. -C rak1 is the directory the repository will be in - note that I put it into rak1 rather than rak. This is because we are now going to move it.
The git-svnimport command checks out a working copy into the “rak1″ directory. If you use the server as a working area and not just a server, that’s ok, but I just want to use it as a server so I have to get rid of the working copy. We do this by creating a ‘bare’ repository:
/home/dbl/git $ git clone --bare rak1 rak
If you look into rak/ now, you will see the contents of rak1/.git/. This is now a repository without a working copy, which is exactly what we want on a server.
Using git-daemon
You can remotely clone and use this repository right away with ssh, but I like to change my ssh port for security and that’s no good if you want your repository to be public. So let’s start git-daemon. The git-daemon listens on 9418, so you’ll need to make sure that is open. There are lots of options for this, but this works for me:
git-daemon --base-path=/home/dbl/git
You should probably add this to your cron to start on reboot. To get it working right away I actually typed: nohup git-daemon –base-path=/home/dbl/git &.
Now we have to tell the daemon which repos it’s ok to serve. git-daemon checks to see if the git-daemon-export-ok file exists, so:
touch /home/dbl/git/rak/git-daemon-export-ok
You can now clone your repository from your local machine with:
git clone git://www.yourserver.com/rak
The git-daemon doesn’t support push, so to push your changes back up to the server you’ll have to use ssh as in the git User Manual.
Setting up gitweb
If you would like a fancy repository browser like git.donttreadonme.co.uk, you could do worse than checking these instructions.
And that’s it!
About
Daniel Lucraft, London, UK. Interested in Ruby, Prolog and software.