Category: Ruby on Rails ++

Tagging with Redis

Its been a long time since my last post and this one is about Redis. So I’ve been working on this project for a bit now and I had this story to implement where I had to associate posts with other posts which were identified to be similar based on a tagging system that already existed. The trouble here was that the existing tagging system was closely tied to a lot of other functionalities and couldn’t be easily (quickly) re-written. The feature needed to be rolled out quickly for several reasons which I won’t get into.

The Problem

The tags associated to each posts were poorly organized where one record in the Tag model would hold all the tags
associated to the post as a whitespace separated string (ouch!)

posts.tags.first.name #business money finance investment

So to find posts that had 2 tags in common from a database of approximately 2000 posts with at least
4 tags each took a significant amount of time. Here is a basic benchmark of just finding the matches
on each request.

Benchmark.measure do
 similar_posts = []
 Post.tagged_posts.each do |post|
   similar_tags = 
   (post.tags.first.name.split & current_post.tags.first.name.split)
   similar_posts << post if similar_tags.size <= 2
  end
end

Here is what benchmark returns

 
=> #Benchmark::Tms:0x111fd8cb8 @cstime=0.0, @total=2.25,
@cutime=0.0, @label="", @stime=0.19, @real=4.79089307785034, @utime=2.06

Not Great.

So the next option was to pre-populate the related posts for every post and store it in the database as similar_posts. So all that was required was to fetch the post with its ‘similar_posts’ at runtime. This seemed like an acceptable solution considering the tags were not changed on a regular basis but if changed it would require the similar_posts table to be rebuilt again(which took a long time). Here is the benchmark for fetching the pre-populated data from the database

Benchmark.measure { p.similar_posts }
=> #Benchmark::Tms:0x104d153f8 @cstime=0.0, @total=0.0100000000000007,
@cutime=0.0, @label="", @stime=0.0, @real=0.0333230495452881, 
@utime=0.0100000000000007

Nice! But this came at the cost of having to rebuild the similar_videos every time something had to be changed with tags or videos.

Redis

Redis is this awesome in memory key-value, data-structure store which is really fast. Though it would be wrong to think of it as a silver bullet it does a lot things which are really awesome. One is the ability to store data structures and perform operations on them, PubSub and a lot of other cool stuff. It even allows you to persist the information using a snapshotting technique which takes periodic snapshots of the dataset or by “append only that take place and recreates the dataset in case of failure.

In my case I didn’t need to persist the data but maintain a snapshot of it in memory. So assuming some basic understanding of Redis and the redis gem I’ll describe the approach.

We created a SET with each tag name as key in REDIS so that every tag contains a set of the post_ids of all posts that have that tag. Inorder to identify a post having two tags in common all that was needed was the intersection of tags sets and REDIS provides built methods for these operations. Thats it!

{"communication" => ['12','219', .. , '1027']}  #sample SET

Fetching the similar posts

def find_similar_posts
  similar_posts = []
  if self.tags.present? && self.tags.first.present?
    tags = self.tags.first.name.split
    tags_clone = tags.clone
    tags_clone.shift
    tags.each do |tag|
      tags_clone.each do |tag_clone|
        similar_posts << REDIS_API.sinter(tag.to_s, tag_clone.to_s)
  end
      tags_clone.shift
     end
  else
    puts "No matching posts."
   end
 similar_posts -= [self.id.to_s]

 Post.find(similar_posts)

Benchmark

 
>> Benchmark.measure { p.find_similar_posts }
Benchmark::Tms:0x1100636f8 @cstime=0.0, @total=0.0100000000000007, 
@cutime=0.0, @label="",
@stime=0.0, @real=0.163993120193481, @utime=0.0100000000000007

Which is pretty good considering that we do all the computation within this request and nothing is pre-populated.

FullText search with Sunspot

Recently I had to implement the full text search functionality in my project.Had couple of options in my hand, as there are some good gems available in the rails community.

listed some of them below

1. sunspot + solr (https://github.com/outoftime/sunspot)
2. acts_as_indexed (https://github.com/dougal/acts_as_indexed)
3. thinking-sphinx (https://github.com/freelancing-god/thinking-sphinx)
4. tire + elasticsearch (https://github.com/karmi/tire)

All of the gems listed above are widely used and it won’t be good to make any comparisons on there usage and performance. Here
i show you how i integrated Sunspot in my rails project.

Sunspot is a Ruby library for expressive, powerful interaction with the Solr search engine. So when integrating it with the rails project the only thing we need to require is the sunspot_rails gem as it also comes with the solr libraries and we don’t have to explicitly install it. The key strengths of sunspot would be the extensibility and to scale up to larger data better.

1. Installing sunspot_rails gem.

Include the sunspot_rails gem in your gem file.

gem 'sunspot_rails'

and run bundle install .

2. Generating the sunspot.yml file.

rails generate sunspot_rails:install

3. Start the Solr server.

rake sunspot:solr:start

to stop the server

rake sunspot:solr:stop

4. Add the searchable block to the model we want to search on.

inside the block we have to define the attributes to search by.

so inside the model we would have something like this

class Item < ActiveRecord::Base
  belongs_to :category
  searchable do
    text :name
    text :description, :stored => true
    integer :category_id
    float :price
  end
end

5. Reindex the existing records in the database.

rake sunspot:reindex

this command would reindex the existing records in the database.We don’t have to worry about the new records that are been added as Sunspot would automatically indexed them for us.

6. Setting the search variable inside the controller method.

def index
  search = Item.search do
    fulltext params[:search_query]
    paginate :page => params[:page], :per_page => 5
  end
  @items = search.results
end

here inside the search block we can use different methods to perform the search.for e.g. the fulltext method with the params parameter.

here search.results would return the array of all the items.

7. Inside the index.html.erb

:get do %>

8. Implementing the Faceting search

this is one of the most powerful feature of Solr that helps to filter the search criteria according to certain conditions.for example if we want to list all the items according to the categories they fall into.

for implementing the faceted search we need to make some changes in our index method and view, which will make our method and view to look something like this

inside controller

def index
  search = Item.search do
    fulltext params[:search_query]
    paginate :page => params[:page], :per_page => 5
    facet(:category_id)
    with(:category_id, params[:category]) if params[:category].present?
  end
  @items = search.results
  @search=[]
  search.facet(:category_id).rows.each do |facet|
    @search << Category.fetch_category_name(facet.value)
  end
end

inside view

Categories

  • search.id %>

(defining fetch_category_name method inside the category model)

class Category < ActiveRecord::Base
  has_many :items
  def self.fetch_category_name(value)
    find_by_id(value)
  end
end

9. Starting the Solr server in Production.

If you are in production its recommended that you don’t use the rake task to start the Solr server, instead you can fire this command which will start a daemon process.

sunspot-solr start -- -p 8983 -d data/solr/myapp

now if you check your sunspot.yml file that you generated inside /config directory.you would find this 8983 port defined and the other configuration settings.

There are lot more options that we can add to the search functionality with Sunspot Solr and i would recommend to take a look at the Sunspot documentation.

Delayed Job In rails3

There are times when you have to run certain tasks/jobs in background for your rails application.it can be as simple as sending emails, Http downloads, image resizing and much more.

Commonly used gems for Background processing

1. delayed_job (https://github.com/collectiveidea/delayed_job)
2. resque (https://github.com/defunkt/resque)
3. beanstalkd and stalker (http://http://kr.github.com/beanstalkd)

(https://github.com/adamwiggins/ stalker)

It totally depends on your needs and choice on which gem to choose for background job.But if your job queue is small with less failures i would suggest using delayed_job.

Here i would show you how i integrated delayed_job in my application for sending emails in background.

1. Installing delayed_job gem

Include the delayed_job gem in your gem file.

gem 'delayed_job'

and run bundle install .

2. We need to run the generator

this would create a delayed_job migration.
Run this command on your terminal in your project directory.

generate delayed_job

after this we need to run the migration.

rake db:migrate

3. Queuing jobs

Now we need to call the delay method on any object to run the job in the background.

this is how my controller looks

class ContactsController < ApplicationController   
  def create     
    @contact= Contact.new(params[:contact])     
    if @contact.save       
      UserMailer.delay.contact_information(@contact)       
      UserMailer.delay.contact_confirmation(@contact)       
      redirect_to root_path     
    else       
      render :action => 'new'
    end
  end
end

Here you can see that I have called the delay method on UserMailer.So what we have done here is we have replaced the deliver method that we normally use for sending mail with delay method.

so instead of using UserMailer.contact_information(@contact).deliver

we use UserMailer.delay.contact_information(@contact)

4. Start the worker

In Development we can start this with this simple command

rake jobs:work

what this command will do is it will start the worker and start processing the jobs in the queue.

Great.but this won’t work in production and to make it working we need to run the delayed_job
script that is provided when we install the delayed_job gem. Its inside the /script directory.
Just run this command and it will start the worker in the background.
to start – ruby script/delayed_job start
to stop – ruby script/delayed_job stop

If you want to delete all the jobs in the queue. just run this command.

rake jobs:clear

There’s a lot more that you can do in delayed_job but i have tried to keep this short. Kindly check the documentation for more options.

I hope you find this information useful when using delayed_job in your project.

Creating PDF using Prawn in Ruby on Rails

Recently i was working on a client project and i realized that i need to create a PDF for generating a invoice on the fly.
I wanted something that could be fast and easy to use.As i started searching my way for options, i found out that there are lot of gems and plugins available in the rails community for generating PDF.

There were two options for me which were widely used for generating PDF.

1. Prawnhttp://prawn.majesticseacreature.com/

2. PDFkithttps://github.com/jdpace/PDFKit

So i chose Prawn for this project and believe me adding this to my rails application wasn’t that difficult.

1. First thing that we need to do here is to add the prawn gem inside our gemfile and run the bundle command.

gem 'prawn'

2. Second thing that we need to do is that we need to create a PDF Mime::Type inside config/initializers/mime_types.rb that is because we need to notify rails about the PDF mime type.

Mime::Type.register “application/pdf”, :pdf

(Now when there is a pdf request our application can respond to it. )

3. Then we have to do couple of changes inside our controller action for which we need to return a pdf version.

This is how my Controller looks like.

class InvoicesController < ApplicationController

  before_filter :authenticate_customer!, :only => [:index, :show]

  def index
    @invoices = Invoice.all_invoices(current_customer)
  end

  def show
    @invoice = Invoice.find(params[:id])
    respond_to do |format|
      format.html
      format.pdf do
        pdf = InvoicePdf.new(@invoice, view_context)
        send_data pdf.render, filename: 
        "invoice_#{@invoice.created_at.strftime("%d/%m/%Y")}.pdf",
        type: "application/pdf"
      end
    end
  end
end

Here u can see inside my show controller i have created a respond block with html and pdf format.
And inside the format.pdf block there is a send_data method we are calling

send_data pdf.render, filename: 
"invoice_#{@invoice.created_at.strftime("%d/%m/%Y")}.pdf",
type: "application/pdf"

And what it does is, that it sends the data for the pdf document. we can have couple of options passed to this send data method. Like here i have passed a filename option to name the pdf document. and the type option.

One more option you can use here is to pass the disposition: “inline” after type and what this will do is instead of downloading a pdf it would open in the browser.

Now to work more on our PDF document, we will create a new class.
app/pdfs/inovice_pdf

Good way would be creating a new directory called pdfs inside our apps folder and then creating a new file called inovice_pdf (this is what i have created) and then create a class inside that called InvoicePdf and inherit that with a prawn document.

class InvoicePdf < Prawn::Document

end

then inside that we can have a initialize method

class InvoicePdf < Prawn::Document

  def initialize(invoice, view)
    super()
    text "This is an order invoice"
  end
end

here we gave a call to a super method and display the string that we need to show on our PDF.

4. Now remember the format.pdf block inside the show action in Invoices controller. inside that we have a line

pdf = InvoicePdf.new(@invoice, view_context)

Here we have instantiated the InvoicePdf. here @invoice is the invoice instance so it can be accessible inside our pdf document and to access to all the view helpers in our PDF document we would pass view_context as the argument.

5. Restart your application.

And we are ready to go. now we have to make the changes inside the InvoicePdf class for the styling that we need to apply to our PDF document.

6. Now inside our InvoicePdf class we need to set the @invoice and view_context.

def initialize(invoice, view)
  super()
  @invoice = invoice
  @view = view
  text "Invoice #{@invoice.id}"
end

7. Now you can create different method inside your InvoicePdf class as per what you want to show on your pdf.

For example to show your logo on the pdf we can have a logo method.

def logo
  logopath =  "#{Rails.root}/app/assets/images/logo.png"
  image logopath, :width => 197, :height => 91
end

and then give a call to that logo method inside your initialize method.and wow logo shows up on the pdf.

and this is how we can have separate method for separate section of our pdf.

For more styling options you can refer to http://prawn.majesticseacreature.com/manual.pdf .

I have created a standard invoice and have pasted the code below.You can modify it as you want your PDF to look like.

class InvoicePdf < Prawn::Document

  def initialize(invoice, view)
    super()
    @invoice = invoice
    @view = view
    logo
    thanks_message
    subscription_date
    subscription_details
    subscription_amount
    regards_message
  end

  def logo
    logopath =  "#{Rails.root}/app/assets/images/logo.png"
    image logopath, :width => 197, :height => 91
    move_down 10
    draw_text "Receipt", :at => [220, 575], size: 22
  end

  def thanks_message
    move_down 80
    text "Hello #{@invoice.customer.profile.first_name.capitalize},"
    move_down 15
    text "Thank you for your order.Print this receipt as 
    confirmation of your order.",
    :indent_paragraphs => 40, :size => 13
  end

  def subscription_date
    move_down 40
    text "Subscription start date: 
    #{@invoice.start_date.strftime("%d/%m/%Y")} ", :size => 13
    move_down 20
    text "Subscription end date :  
    #{@invoice.end_date.strftime("%d/%m/%Y")}", :size => 13
  end

  def subscription_details
    move_down 40
    table subscription_item_rows, :width => 500 do
      row(0).font_style = :bold
      columns(1..3).align = :right
      self.header = true
      self.column_widths = {0 => 200, 1 => 100, 2 => 100, 3 => 100}
    end
  end

  def subscription_amount
    subscription_amount = @invoice.calculate_subscription_amount
    vat = @invoice.calculated_vat
    delivery_charges = @invoice.calculated_delivery_charges
    sales_tax =  @invoice.calculated_sales_tax
    table ([["Vat (12.5% of Amount)", "", "", "#{precision(vat)}"] ,
    ["Sales Tax (10.3% of half the Amount)", "", "",
    "#{precision(sales_tax)}"]   ,
    ["Delivery charges", "", "", "#{precision(delivery_charges)}  "],
    ["", "", "Total Amount", "#{precision(@invoice.total_amount) }  "]]), 
    :width => 500 do
      columns(2).align = :left
      columns(3).align = :right
      self.header = true
      self.column_widths = {0 => 200, 1 => 100, 2 => 100, 3 => 100}
      columns(2).font_style = :bold
    end
  end

  def subscription_item_rows
    [["Description", "Quantity", "Rate", "Amount"]] +
    @invoice.subscriptions.map do |subscribe|
      [ "#{subscribe.description} ", subscribe.quantity, 
      "#{precision(subscribe.rate)}  ",  
      "#{precision(subscribe.quantity  * subscribe.rate)}" ]
    end
  end

  def precision(num)
    @view.number_with_precision(num, :precision => 2)
  end

  def regards_message
    move_down 50
    text "Thank You," ,:indent_paragraphs => 400
    move_down 6
    text "XYZ",
    :indent_paragraphs => 370, :size => 14, style:  :bold
  end

end

Now if we go to the address bar and type the url for the show action.

http://localhost:3000/invoices/1.pdf (press enter)

or you can have a link like this in your view
<%= link_to “Download invoice”, invoice_path(invoice.id, :format => “pdf”) %>;

the invoice would be downloaded. Remember at top i told you if you dont want to download and just show up in your browser just add this option to the send_data disposition: “inline.

send_data pdf.render, filename: 
"invoice_#{@invoice.created_at.strftime("%d/%m/%Y")}.pdf",
type: "application/pdf", disposition: "inline

I hope this helps you all who are looking out for quick generation of pdf. There is lot more to Prawn and i have tried to provide all information that i did while generating pdf.
That’s all there is to it.

Capistrano on EC2

This blog is about deploying a ruby on rails application on an amazon instance using capistrano.

The biggest issue that I was facing was configuring the ssh options for cap as the amazon box require’s identity file for you to ssh in. I also had one more issue that; I had not dealt in the past which was using rvm local, staging servers and the server did not have rvm but a standard installation. So, to get started with here are the things you will need.

1) Add capistrano and capistrano-ext to your to the ‘development’ group of your Gemfile. For example.

   
   group :development do
    ...
    gem 'capistrano', '2.8.0'
    gem 'capistrano-ext', '1.2.1' 
    ...
  end

2) bundle install

3) Open config/deploy.rb with your favorite editor.
here is how config/deploy/development.rb looks:

   
    server "localhost", :app, :web, :memcached, :sphnx, :db, :primary => 
    true

   set :user, "developer"
   set :rails_env, 'development'
   set :application, "mydiffenvsettings"
   set :repository,  "your_version_control_system"
   set :deploy_to, "/usr/local/xxx"

The ‘rails_env’ config is the only important thing.

4) I started with the intention that I will get going with a deployment locally followed by Staging and then to Prodution not remembering that the production ENV did not rvm. There are few dependencies that come to get cap working with rvm which will be indicated to you by a simple google search but still listing it below:

  
  $:.unshift(File.expand_path('./lib', ENV['rvm_path']))
  require "rvm/capistrano"
  set :rvm_ruby_string, "1.8.7-p330@rails3"
  set :rvm_type, :user
  set :use_sudo, false

but the above bombed in production so, below is the hack I put and not sure if its the best way to do it.

#only include rvm dependencies if the machine has rvm.
if ENV['rvm_path'] != nil && ENV['rvm_path'] != ''
  $:.unshift(File.expand_path('./lib', ENV['rvm_path']))
  require "rvm/capistrano"
  set :rvm_ruby_string, "1.8.7-p330@rails3"
  set :rvm_type, :user
  set :use_sudo, false
end

5) You will then prepare with your standard config type stuff

set :stages, %w(development staging production)
set :default_stage, "development"

require 'capistrano/ext/multistage'

set :application, "mydiffenvsettings"
set :repository,  ""
set :scm, "your_version_control_system"
set :scm_username, "my"
set :scm_password, "credentials"
set :user, "ec2user"
set :use_sudo, false
set :deploy_to, "/usr/local/xxx"
set :deploy_via, :remote_cache
set :repository_cache, "copy_of_your_checkout_instance"
set :keep_releases, 3

6) This is the biggest challenge I ran into #ssh options… Below is how I did it:

#set the ssh options
default_run_options[:pty] = true
ssh_options[:forward_agent] = true
#Configure SSH options for ec2
ssh_options[:keys] = [File.join(ENV["HOME"], ".ssh", "")] 
unless 'development' == rails_env

Yes those are the ssh options that you need to get it working on an EC2 instance.
Heh I found that funny as I was expecting I will have to look into some recipe someone wrote with a bunch of options and stuff and I’ll never be to get it working in a short period of time.

7) Then comes all the standard stuff… making calls to the before and after hooks for the 3 standard cap tasks.

I would be surely interested to know if there is a better way to do what I am doing. Thanks for taking the time to read the blog hoping it helps.

MultiUser Chat using XMPP and Orbited (Using Ruby-on-Rails)

One of the things that I wanted to understand and build since I first learned to program was to build a chat client. Something that would allow people to communicate and I am extremely thankful to Rishav Rastogi for introducing me to XMPP.

I never really understood all the moving parts very clearly during my first interaction with the technologies but with some time on my hands now I decided to revisit the entire process of building a web chat client. While there are a few well documented resources that cover how to build a simple web chat client the information is mostly directed towards using XMPP and building a one-on-one chat.
Though the requirements for building a multiuser chat aren’t significantly different there are subtle differences that exist.

A brief introduction to XMPP, Ejabberd (our XMPP server) and Orbited

XMPP
Ejabberd is an XMPP server that I used to build my chat client with.

The Extensible Messaging and Presence Protocol (XMPP) is an open technology for real-time communication, which powers a wide range of applications including instant messaging, presence, multi-party chat, voice and video calls, collaboration, lightweight middleware, content syndication, and generalized routing of XML data.Xmpp.org

The technology was initially called Jabber and hence both Jabber and XMPP are used interchangeably on several posts. – Wikipedia

Ejabberd
Ejabberd is a Jabber/XMPP server built using Erlang and is open-source and we would be using Ejabberd in this example. Another popular alternative for Ejabberd is OpenfireEjabberd

Xmpp4r
Since this post would be using Rails we use the xmpp4r gem which is a wrapper over the standard XML that XMPP/Jabber/Ejabberd uses, thus allowing us to work with Ruby rather than generate XML. For those using Ruby 1.9.2 the gem installation may throw up some errors while installing the Rdoc so I’d recommend you either skip the Rdoc installation or ignore the error. The online documentation for Xmpp4r is pretty good and the gem comes with some useful examples that could help you get started.

Orbited
Orbited provides a pure JavaScript/HTML socket in the browser. It is a web router and firewall that allows you to integrate web applications with arbitrary back-end systems.Orbited

Why do we need Orbited?
With our existing arrangement (once we install Ejabberd and xmpp4r gem) we could get a basic messaging system ready. We could have users send messages and receive messages. The problem would be to receive those messages on the browser. There is no way we can display those messages without having to poll our server to fetch this information and we know polling could cause scalability issues. Orbited fills this void by acting as a web router that routes the incoming messages to the appropriate user’s browser using a technique called as long-polling. And long-polling is more scalable than polling.

Long-Polling
Comet is a broad term used for technologies like Long-Polling and streaming. While traditional polling requires periodic requests to be sent to the server and then return with the response, in long-polling a connection is established with the server which persists until a response is provided (or the request times out). Once the response is provided the connection is closed and a new one is step up waiting for the next response from the server. Similarly a new connection is set up on timeout. In Streaming the connection persists between the client and the server while the information is transferred.

According to HTTP 1.1 a browser is allowed to have only 2 connections to the server one of which is used here for real time communication, though I am not fully clear if this is exactly the way the connection is setup. Apparently IE 8 allows 6 connections per host so I shall look forward to any clarifications on this.

Orbited comes with support for technologies such as STOMP, IRC and XMPP so its a handy tool to get started with.

Installation

Installing Xmpp4r
This is the easiest part especially with Rails 3. The following is a snippet of my gemfile

source 'http://rubygems.org'

gem 'rails', '3.0.3'

# Bundle edge Rails instead:
# gem 'rails', :git => 'git://github.com/rails/rails.git'

gem 'mysql2'
gem 'xmpp4r'
gem 'authlogic'
gem 'rails3-generators'

group :development do
 gem 'rspec-rails', '2.3.0'
 gem 'mongrel', '1.2.0.pre2'
 gem 'cgi_multipart_eof_fix'
 gem 'fastthread'
end

group :test do
 gem 'rspec', '2.3.0'
 gem 'webrat', '0.7.1'
end

bundle install and your ready.

Installing Ejabberd

You can download the installer from here. At the time of this tutorial the lastest version was 2.1.6.

The installer guides you on how to setup the xmpp server. Here are some of the questions you would have to provide answers to
Domain: siddharth-ravichandrans-macbook-pro.local This is simply a name (domain name) that you would want your server to be known by. In production this could be chat.example.com or jabber.example.com. For development the default is good. Its important that you note down the domain name somewhere as you will be using this a lot.

Cluster Installation: NO

Admin: siddharth This could be any name that you choose. This provides a way to access the ejabberd web administration interface
Admin password : siddharth

Thats it, you have your ejabberd server installed. Now open the folder you installed it in and navigate to the bin folder.

 
./ejabberdctl start
 
 SIDDHARTH-RAVICHANDRANs-MacBook-Pro:bin SIDDHARTH$ 
./ejabberdctl status
The node ejabberd@localhost is started with status: started
ejabberd 2.1.6 is running in that node

The will let you know if the server is working

Now that we have confirmation that our server is running log onto http://localhost:5280/admin to access you admin interface.

You may log in as [AdminUser]@[domain] followed by the password.
In my case ‘Siddharth@siddharth-ravichandrans-macbook-pro.local’ with the password ‘Siddharth’

You should now be able to see a web console for the administrator

Installing Orbited

The version of Orbited that I used was 0.7.10 which is available here. Ensure that you have python 2.5 or higher installed in your system. Most linux and OS X systems come with Python pre-installed. You can check by

$ python 
Python 2.6.1 (r261:67515, Jun 24 2010, 21:47:49) 
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()

Twisted is installed as dependency for Orbited 0.7.10 so it need not be installed explicitly but incase you face some errors these are the steps of installation

Ensure that the orbited.cfg file is placed in the /etc folder which is where orbited automatically checks for the configuration file or else it may be supplied as an argument

 sudo orbited --config=/Users/SIDDHARTH/orbited.cfg

Once your done open the configuration file on your favorite editor

[global]
reactor=select
#reactor=kqueue
# reactor=epoll
session.ping_interval = 40
session.ping_timeout = 30
# once the sockets are open, orbited will drop its privileges to this user.
user=SIDDHARTH

For the reactor epoll would the one to select on Linux machines and Kqueue for OS X but I noticed that Kqueue has not been maintained and throws errors so using select is the last resort. Though select has scalability issues its okay to use it for development.

Set the user to the user that you would want orbited to run as.
The access section identifies how orbited will communicate with Ejabberd
Orbited will listen to all incoming requests at port 8000 and communicate with port 5222 with XMPP (Ejabberd uses 5269 for server to server communication)
Therefore

localhost:8000 -> localhost:5222.

In production this could look like

localhost:8000 -> example.com:5222

So our access section would look like

[access]
#localhost:8000 - > irc.freenode.net:6667
localhost:8000 - > localhost:5222
* -> localhost:4747
#* -> localhost:61613

Thats it, orbited is ready. Give it a go by typing ‘orbited’ in the console. You should see the server start.

Beginning with some XMPP programming
I will be working on some basics of using xmpp4r which are explained beautifully in François Lamontagne’s two part tutorial on using Jabber with xmpp4r

Once you’ve conquered the basics of user subscription and sending messages lets take a look at the Multi User Client support provided in Xmpp4r.

Registering our users to the Jabber server. Ideally this would be after a user registers to your site, so an after_create operation.

require 'xmpp4r'
require 'xmpp4r/muc'
require 'xmpp4r/roster'
require 'xmpp4r/client'
# getting done with all the requires so you can try this on the console
 client = 
   Jabber::Client.new(Jabber::JID.new
('first_user@siddharth-ravichandrans-macbook-pro.local'))
 client.connect
 client.register('password')

# do the same for another user with 
#full_jiid = 'second_user@siddharth-ravichandrans-macbook-pro.local'

Logging into the server

require 'xmpp4r'
require 'xmpp4r/muc'
require 'xmpp4r/roster'
require 'xmpp4r/client'
# getting done with all the requires so you can try this on the console
 client = 
 Jabber::Client.new(Jabber::JID.new
('first_user@siddharth-ravichandrans-macbook-pro.local'))
 client.connect
 client.auth('password')

# Don't forget to log both users in

Logging into a room/ creating a room
The MUC Client is a multi User chat Client. The XMPP4R gem provides support for MUC too.

Create a new client

muc = Jabber::MUC::MUCClient.new(client)

The MUC is not to be confused with the room. Its simply a client that serves as an interface for the user in a particular room.

Joining/Creating a room

muc.join(Jabber::JID::new
('chatroom@conference.siddharth-ravichandrans-macbook-pro.local' 
+ client.jid.node))

This lets the user join a room called chatroom and the user is logged in to the room as client.jid.node which evaluates to first_user in our case.

The domain appends the word conference by default to all multi user chat rooms and can be changed by editing the configuration file. The JID for a room can be split as ROOM_NAME + @ + conference.domain_name/user_nick

Setting up callbacks for the client

    
muc.add_join_callback do |m|
      puts "[NEW MEMBER JOINED] " + m.to.jid.node
    end

    muc.add_message_callback do |m|
      puts "[NEW MESSAGE]" + m.body
    end

    muc.add_leave_callback do |m|
      puts "[MEMBER LEFT] " + m.to.jid.node
    end

The callbacks like the one described earlier in François Lamontagne’s two part tutorial get called when a new user joins the chat room, sends a message to the room or leaves the chat room. The MUC chat is actually very similar to the one – on – one chat example described in François Lamontagne’s example except that when a message is directed to the room it relays the message to all of the members in the room. So if you look at the xml you will notice that a message directed to the room is eventually directed to each user in the chatroom. The only difference is the send method which belongs to muc object takes care of the relaying or you may query the roster (I will come to this in a moment) to identify the members in a room and post a message to each member.

Sending a message to the room

muc.send(Jabber::Message.new
('chatroom@conference.siddharth-ravichandrans-macbook-pro.local',
 'Pink Floyd is the greatest band ever'))

The message type for a message sent to a chatroom is automatically set to the type :groupchat. (Jabber::Message is explained here. Lets have a look at the associated xml that is sent to each member

In order to view xml generated set the Jabber::debug to true

 
Jabber::debug = true

The roster describes the subscriptions or the buddies on a one-on-one chat but in a chatroom the muc client has a roster that identifies the number of users in a chatroom along with their presence

muc.roster

would yield something like this

The MUC roster is extremely useful and allows you to set callbacks too.

This pretty much wraps my example using Ejabberd and XMPP4R. The next part of my post will briefly describe how we can use orbited and have this information flow through the browser.

Starting Orbited

orbited

Would get orbited up and running if you placed the orbited.cfg in the /etc folder. Once orbited is running you can log onto http://localhost:8000/static where you would be able to the see the javascript files that Orbited provides you with. You will notice Orbited.js and a static folder. Jump into the static folder -> then protocols -> Xmpp -> to find the xmpp.js file. We will be working primarily with these two files.

So first lets make these two files available to our application by putting them in a layout file.

You will notice two partials at the bottom of my layout file called _tcpsocket and xmpp_client (both poorly named).

Before we begin try running this snippet obtained from Micheal Carter’s Sockets in the Browserarticle on CometDaily.com. Add this snippet to the _tcpsocket.html.erb partial that is included in the layout.

Load a view page (which includes the layout containing this parital). It could be any scaffold generated code block.

 
$(document).ready(function(){
  var conn  = new Orbited.TCPSocket();
   conn.open('localhost', 5222);
   conn.onopen = function(){ alert('connection opened');
   // conn.send('Hello World');
   }
   conn.onread  = function(data){ alert('RECIEVE DATA' + data ); }
   conn.onclose   = function(data){ alert('connection closed'); }

});

This would be a helpful example to understand better what Orbited does. All it does is opens a tcp socket on localhost and connects to port 5222 . The onopen callback is called when the connection is opened and sends a piece of text which is read by the onread callback and the connection close callback is called.

Basically reading the data whenever something is sent by the server while waiting for it with an open socket connection is what we do.

Looking at our _tcpsocket partial

 

  document.domain  = document.domain;
  Orbited.settings.port  = 8000;
  Orbited.settings.hostname   = 'localhost';
  TCPSocket  = Orbited.TCPSocket;

You may ignore the document.domain = document.domain code for now. Here we include the Orbited.js code along with the Xmpp.js javascript files provided by Orbited. We also specify the port we would be listening to and the hostname.

The xmpp.js file is where all the magic (not really) happens.

The xmpp.js contains (yet another) javascript based interface to XMPP methods, thus allowing us to perform all the XMPP operations right from the browser. The existing xmpp.js file comes with partial support for MUC operations. A poorly and hastily hacked xmpp.js file to suit basic MUC operations is available on my github account.

CONNECT = [""];
REGISTER = ["","",""];
LOGIN = ["","","Orbited"];
ROSTER = [""];
MSG = ["",""];
PRESENCE = [""];
EXT_PRESENCE = [];
GROUPCHAT_MSG = ["",""];

....

XMPPClient = function() {
    var self = this;
    var host = null;
    var port = null;
    var conn = null;
    var user = null;
    var domain = null;
    var bare_jid = null;
    var full_jid = null;
    var success = null;
    var failure = null;
    var parser = new XMLReader();
    self.onPresence = function(ntype, from) {}
    self.onMessage = function(jid, username, text) {}
    self.onSocketConnect = function() {}
    self.onUnknownNode = function(node) {}
    self.sendSubscribed = function(jid, me_return) {
        self.send(construct(PRESENCE, [me_return, jid, "subscribed"]));
    }
    self.connect = function(h, p) {
        host = h;
        port = p;
        reconnect();
    }
    self.msg = function(to, content) {
        self.send(construct(MSG, [full_jid, to, content]));
    }
    self.unsubscribe = function(buddy) {
        self.send(construct(PRESENCE, [full_jid, buddy.slice(0, 
        buddy.indexOf('/')), "unsubscribe"]));
    }
    self.subscribe = function(buddy) {
        self.send(construct(PRESENCE, [full_jid, buddy, "subscribe"]));
    }
    self.send = function(s) {.....

If you notice these methods end up generating the exact same XML code (converted to utf8) and sent to the ejabberd. So no real magic there.

Our goal is to now use this API to perform the same operations on the browser. Here is a basic MUC chat javascript. Add this to the _xmpp_client.js on the layout file. The snippet contains some missing text so use the code here

console.log('XmppClient partial loaded');
  var hostname                = 'localhost';
  var domain                  = 'siddharth-ravichandrans-macbook-pro.local';
  var bare_jid                =  ''; 
  var password                = ''; 
  var chatroom_domain         = 'conference.' + domain;
 // var username                = bare_jid + '@' + domain;

  console.log('xmpp client connect request posted');

  function loginSuccess(){
   alert('Login Successful');
  // xmpp_client.set_presence('available');
   alert(typeof ROOM_NICK);
     if(typeof ROOM_NICK != 'undefined'){
     xmpp_client.join_room(ROOM_NICK, chatroom_domain, bare_jid, 
                                 'available', null);   
     console.log('JOIN ROOM Called');
   } 
  }

  function loginFailure(){
   console.log('Login Failed');
  }

  function serverConnectSuccess(){
   alert('Server Connection Success');
   $('.presence-status').html('('+ 'Server Connected'+')');
   xmpp_client.login(bare_jid, password, loginSuccess, loginFailure);

  }

  function serverConnectFailure(){
   alert('Server Connection Failed');
  }

  var xmpp_client                            = new XMPPClient;
  xmpp_client.connect('localhost', 5222);

  xmpp_client.onSocketConnect                = function(){
   $('.presence-status').html('('+ 'On Socket Connected'+')');
   xmpp_client.connectServer(domain, serverConnectSuccess, 
                        serverConnectFailure);
   console.log('After COnnect Server is called'); 
   xmpp_client.login(bare_jid, password, loginSuccess, loginFailure); 

  }

  xmpp_client.onPresence = function(ntype, from) {
   var username  = bare_jid + '@' + domain + '/Orbited'; 
   if(from == username){
    if (ntype == null){
     $('.presence-status').html('(available)');
    }
    else{
     $('.presence-status').html('(' + ntype + ')');
    }
   }    
  }

  xmpp_client.onMessage = function(jid, username, text) {
  $('.conversation-box').append('');
‘ + username.split(“/”)[1] + ‘ says : ‘ + text + ‘
   alert('JID' +  jid.to_s + ' Username ' + username + ' Text' + text);
  }

 $(document).ready(function(){
  $('.send-message-button').click(function(){

   alert('Incoming message');
   var message                               = $('#message').val();   
  // xmpp_client.msg('007@conference.' + domain, message);
   xmpp_client.groupchat_msg(message, chatroom_domain);
   return false;
  });
 });

Note that the xmpp.js file has been modified slightly from what Orbited provides us and the _xmpp_client.html.erb uses this modified api hence the method parameters may appear strange when compared with the original xmpp.js file.

The ROOM_NICK parameter is defined in the view using a content_for :js block and would be available inside a chat room.

I hope this is useful and please let me know of errors or misinformation in my article. In case you are interested in having a detailed write up on the installation of all the software, add a comment and I will send you the write up as soon as possible. I have tried my best to attribute most references to their original authors and sources but in case I have forgotten any I would be glad to update it anytime.

 

Final chat screenshot

EC2 backup strategy on S3 – Ubuntu

It is recommended to backup only the database and the mysql files on s3. You do not want to backup the entire server image as that is expensive in terms of space and hence price. Below is a step by step instruction on how you could do that:

Setup s3sync

  1. Login to your EC2 instance
  2. cd ~/
  3. wget http://s3.amazonaws.com/ServEdge_pub/s3sync/s3sync.tar.gz
  4. tar xzvf s3sync.tar.gz; cd s3sync
  5. mkdir certs; cd certs
  6. wget http://mirbsd.mirsolutions.de/cvs.cgi/~checkout~/src/etc/ssl.certs.shar
  7. sh ssl.certs.shar
  8. cd ..

Edit s3config.rb

  1. vi s3congig.rb
    Replace confpath using below (highlighted change in red)
    confpath = [“./”,”#{ENV[‘S3CONF’]}”, “#{ENV[‘HOME’]}/.s3conf”, “/etc/s3conf”]

Locate your s3 credentials

  1. Login to http://aws.amazon.com/
  2. Click on accounts
  3. Click on Security credentials
  4. Locate access credentials section and click on Access Key Tab
  5. You will notice your access key there and show button to see the secret

aws-account

access-credentials1

Configure s3sync with s3 credentials

  1. cp s3config.yml.example s3config.yml
  2. vi s3config.yml and modify the below
    aws_access_key_id: 11111111111111111111111
    aws_secret_access_key: 222222222222222222222
    ssl_cert_dir: /home/user/s3sync/certs

Create a bucket in your s3 account

  1. Log into https://console.aws.amazon.com/s3/
  2. Create bucket <good_name_for>

Create your shell script in your ec2 instance

The below sample is to backup a rails application known as redmine. It backs up the database and uploaded attachments which lives in <app_root>/files = redmine/files directory.

  1. cd ~/s3sync
  2. mkdir s3backup
  3. vi s3backup.sh and you can replace and use the below
    cd ~/
    BUCKET=redmine_archives

    DBNAME=redmine
    DBPWD=admin
    DBUSER=root
    NOW=$(date +_%b_%d_%y)
    echo ‘compressing /usr/local/apps/redmine/files’
    tar czf files$NOW.tar.gz /usr/local/apps/redmine/files
    mv files$NOW.tar.gz s3sync/s3backup
    cd s3sync/s3backup
    echo ‘creating database dump for tb_production schema’
    touch $DBNAME.backup$NOW.sql.gz
    mysqldump -u $DBUSER -p$DBPWD $DBNAME | gzip -9 > $DBNAME.backup$NOW.sql.gz
    echo ‘creating a compressed file for application and the database dumps’
    tar czf server_backup$NOW.tar.gz $DBNAME.backup$NOW.sql.gz files$NOW.tar.gz
    rm -f $DBNAME.backup$NOW.sql.gz files$NOW.tar.gz
    cd ..
    echo ‘uploading to s3’
    ruby s3sync.rb -r –ssl s3backup/ $BUCKET:
    echo ‘cleaning up files created by this operation’
    cd ~/s3sync/s3backup
    rm -f ~/s3sync/s3backup/*
  4. sudo chmod +x s3backup.sh

NOTE: ruby s3sync.rb -r –ssl s3backup/ $BUCKET: it is minus minus ssl. When you copy the script as is, it messes up the — into dash.

See it in action

  1. cd ~/s3sync
  2. ./s3backup.sh
  3. Login to your s3 bucket and see the tar file created.

Automate

Below makes the backup run once every day.

  1. crontab -e
  2. 0 0 * * * /root/s3sync/s3backup.sh

Subscribe To Our Blog

Get access to proven marketing ideas, latest trends and best practices.

Next up home

Contact

Lets build cool stuff

Share your contact information & we will get in touch!

I want (Tell us more about your dream project)