30 Mar

Resque, Why you should try

Installing redis (a crazy fast key/value store) for a website, I had a look to Resque, and give here the why I moved from the good DelayedJob. Here a short answer:

  • Replacing a dirty cron / curl / controller setup to launch scheduled tasks
  • Having a easy control on the scheduled tasks and failed jobs

Resque

Resque is a Redis-backed Ruby library for creating background jobs, placing those jobs on multiple queues, and processing them later.

For the application, the big differences are you cannot send ruby objects and you need to specify a queue.

Resque scheduler, a light-weight job scheduling system built on top of resque.

Quick installation

For full details: Resque on github

Redis:

curl -O http://redis.googlecode.com/files/redis-2.2.2.tar.gz
tar xzf redis-2.2.2.tar.gz
cd redis-2.2.2
make
sudo make install

Add to you Gemfile:

gem 'hiredis'
gem 'redis'
gem 'redis-namespace' 
gem 'yajl-ruby' 
gem 'vegas'
gem 'sinatra'
gem 'resque', '1.15'
gem 'resque-scheduler'

Add to your Rakefile:

require 'resque/tasks'
require 'resque_scheduler/tasks'

Configuration and authentication

Initializer for resque config/initializers/resque.rb:

Schedule configuration file config/resque_schedule.yml:

Let’s start

Resque workers:

COUNT=5 QUEUE=* rake resque:workers

Resque web interface:

bundle exec resque-web -e production -s thin -L config/initializers/resque.rb

And finally the resque scheduler worker:

rake resque:scheduler INITIALIZER_PATH=config/initializers/resque.rb

To kill the resque-web:

bundle exec resque-web -K

Async

If you use DelayedJob and want a replacement for the delay/send_later method, here a simple async method:

Deployment

see the update.

Conclusion
With this setup I get a more robust system to manage jobs and scheduled tasks:

  • The scheduler plugin, multiple queues
  • Web interface, make easy to follow exception and what’s going on (DJ gets admin)
  • Faster to insert thousand jobs without slowing the website db (DJ can manage a mongodb now)

Links:

update (02/04/2011): I ve just found a good post with full setup and capistrano deployment – setting up resque

tags: resque (1) redis (1) delayedjob (1)