ActiveLabs blog

Merb is a (future) great framework. Radiant is good CMS. And they have so much in common. I’m curious about porting Radiant to Merb then I started the challenge. For people interested in the same challenge, I set up a repository. Then please feel free to participate.

[update]

I got invited on github then I cloned the git repository to http://github.com/sbusso/merb_radiant/tree/master

Action plan:

Merb is a framework equivalent to Action Pack. It can use Active Record. For generator we can use Rubigen, it s an extract of RailsGenerator.

For this we will use:

• Radiant trunk r742
• Merb (core and more) 0.9.0
• Merb helpers 0.9.0
• Merb ActiveRecord 0.9.0
• ActiveRecord 2.0.2
• Rubigen 1.2.0

The repository tree was build with:

• Hoe (modified) generated skeleton
• Merb application
• Radiant source

1. Move Radiant base:

• use plugins as gem:

* radius 0.5.1
* highline 1.4.0
* rubypants 0.2.0

• use other plugins as ruby libraries:

* bluecloth

• using active record: nothing to do
• radius and other ruby libs: nothing to do
• replace RailsGenerator with Rubigen (Radiant bin)
• work on initializer
• desactivate extension dependencies (first step we move only the base)
• manage Controller and libraries using controllers
• check routes management
• work on views to replace Rails specific code

2. Replace extension system with gems:

• first, activate gem with only code features
• second, deal with extension providing assets

Following the next version of Merb, I’m a bit surprised by the way this new version came out:

Merb 0.9.0

Help of developers is needed for going on 1.0, then install it and try it.

Included :

• merb-core
• merb-action-args
• merb-assets
• merb-gen
• merb-haml
• merb-mailer
• merb-parts
• merb-more
• merb

then

merb-gen myapp

Classical way, download following archives and compile it in the order. Xcode has to be installed.

tar xzf *.tar.gz
cd 
./configure
make 
sudo make install

1. lib need for server: lib event. download
2. the server: memcached. download
3. the client lib written in c: libmemcached. download
4. the ruby binding:

   sudo gem install memcached

   or with leopard:

   sudo env ARCHFLAGS=“-arch i386” gem install memcached

$ PATH=/usr/local/mysql/bin:$PATH sudo gem install do_mysql

sudo gem install do_postgres — —with-pgsql-dir=/Library/PostgreSQL8/

If like me you have an error after installing scrubyt 0.3.4 (in console and launching mongrel):

can't activate RubyInline (= 3.6.3), already activated RubyInline-3.6.4]

I spent a lot of time to find the solution, and so easy solution … anyway could be useful for people googling about this. You just need to unsinstall the lastest installed version of RubyInline:

sudo gem uninstall RubyInline -v 3.6.4

Everything should be fine with this, even Scrubyt requiring version 3.6.3.

This the second part for a summary about solutions to help better computing of your rails app (1st part).

Application

To help your application to focus on replying fast to user, you can use a distributed queueing to give the job to other processes:

• Daemons
• Backgroundrb

And second thing, you can cache things you don’t need to compute for each request, this can incredibly increase the response of your server, but need to modify your code and it s recommend for very big sites:

• Memcache client, memcached server
• acts_as_cached

Back end

DB

For your database you can find some sql proxy, but I haven’t tried it yet. You have also the way to replicate your databse and serve data from several servers.

• sql relay
• use jruby with a jdbc pool connection and caching

Important things you have to check about your databse is if you have the right indexed fields. You should find some tools on mysql helping you to find where indexes missing. Check about cache too.

A last way is to serialize some of your data in a single object.

Mysql

Otherwise you can configure your mysql with some optimized configuration (mysql.ini):

• skip-name-resolve
• query_cache_size = 52428800;
• query_cache_type = 1

And in your ActiveRecord object you can use the :include parameter.

Filesystem

Optimizing the filesystem, in other sense than choose a good OS, can been done with not overloading the management of files by using some rules:

• max 10K files / subdir
• 16 top level / 256 sub / 10K per sub dir
• use a hasher to manage files

OS

As we have seen, file and memory management are keys of good performances. And these are managed by OS. The two best OSs mentionned are

• Sunsolaris
• FreeBSD (can be use for a all in one stack), avoid the version 5

And upcoming, last part of this article, some home made benchmarks for my own purpose.

One of the most interesting topic in the RailsConf was Scaling your Rails app. There are 2 ways of doing it:

• adapt your code
• adapt your servers

But the second one will not help to make your code faster. It will just help to give maximum resources to compute it right.

Summaries from the conference (sorry for the so few litterature it will be only a summarize):

Some theories

These theories are for big websites with millions of hits every month. This can involve thousand of servers, different location, … BTW some tips can be use to help smaller configurations.

The 7 layers to be considered, a la OSI.

• DNS
• frontend
• proxy
• application server
• application
• backend (Db , filesystem)
• OS

DNS

Nice website to check your DNS configuration: dnsstuff.com
Good tool to optimize your DNS: speed, dispatching, and so on…: powerdns

DNS can be quite fast to address the right server from a request, scripts allow to manage complex configurations. The tip is to use 1 DNS entry per 1 controller, then managing part of the routing.

Frontend and Proxy

Load balancing is about dispatching requests taking in account the charge of each cluster.

Load balancers:
Varnish
HAProxy

Both look promising and HAProxy seems to be more and more popular.

Front end with load balancer:
Nginx : fast, low memory usage, http server with load balancing, easy configuration, very good in front of mongrel cluster.
Apache 2.2 + mod_proxy_balancer, you can find how to setup it in previous post: apapche, mongrel cluster, rails on a load balancer

Application server

• mongrel server: enough for a personnal server
  gem install mongrel
  mongrel_rails start (more information)
• mongrel_cluster: useful to scale up your configuration, need at least a multi-processor or multi server environnment.
  gem install mongrel_cluster
  mongrel_rails cluster::configure …
  mongrel_rails cluster::start
• Evented Mongrel (swiftiply): it’s a mongrel based an event triggering
• jruby / glassfish: the technology java comes with some good environment to speed up part of your application like connection to database (jdbc, pool connection)

to be continued (application, backend, OS, and some home benchmark)

Nginx is a ‘small’ efficient http, load balancer server. More and more popular in Rails community, it’s a good front end for mongrel clusters. It’s seems to be a good replacement for apache, at least for the memory usage (on my test: 1 process use 1.3M against the more than 9M taken by 1 apache process). Benchmark on one single server are quite similar.

The process has been tested on debian 4.0r1 and should work on Ubuntu.

Installation

apt-get install nginx

or, if you want an up to date product:

1/ Prepare your system

apt-get install zlib1g-dev libgcrypt11-dev libpcre3-dev libssl-dev

2/ Download source

tar xzf nginx-0.6.12.tar.gz
cd nginx-0.6.12

3/ Compile it

./configure --pid-path=/usr/local/nginx/logs/nginx.pid \
	--sbin-path=/usr/local/sbin/nginx \
	--with-md5=/usr/lib \
	--with-sha1=/usr/lib \
	--with-http_ssl_module \
	--with-http_dav_module --prefix=/usr \
    --conf-path=/etc/nginx/nginx.conf \
    --pid-path=/var/run/nginx.pid \
    --lock-path=/var/run/nginx.lock \
    --error-log-path=/var/log/nginx/error.log \
    --http-log-path=/var/log/nginx/access.log 

make

sudo make install

Configuration

The following configuration is for 1 nginx server with load balancing 3 mongrel in cluster:

1/ Mongrel cluster configuration

gem install mongrel_cluster -y

cd /var/www/my-app-path/current/

configures 3 mongrel in cluster, in production mode, starting at port 5000 and listening on localhost

sudo mongrel_rails cluster::configure -e production \
    -p 5000 -N 3 -c /var/www/my-app-path/current/ -a 127.0.0.1 \
	--user www-data

mongrel_rails cluster::start

2/ Nginx

nano /etc/nginx.conf

Use the nginx.conf file from brainspl.at website, change the 2 or 4 (if you use ssl) references to the path of your application (/var/www/my-app-path/current/)

/etc/init.d/nginx start

All these information come from many website (thanks to all), I put them together for my own usage, hope it’s helpfull.