Docker and Sidekiq

Sidekiq is well-known for memory bloat (see this), in the past I was using Monit to restart Sidekiq when it uses too much memory. As we transitioning to Docker, I want to figure out a cleaner solution to it without using Monit. I tried god first but turned out that it does not work in the container because the netlink does not work in Docker container. So instead, I went old school and wrote a bash script:

#!/usr/bin/env bash

num_workers=${SIDEKIQ_WORKERS:=4}
max_memory=${SIDEKIQ_MAX_MEMORY:=3000000} # 3GB
sidekiq_config=${SIDEKIQ_CONFIG:=config/sidekiq.yml}

function finish {
		echo "Shutting down Sidekiq workers..."
		for i in `seq 1 $num_workers`; do
	    bundle exec sidekiqctl stop tmp/pids/sidekiq-$i.pid 15 &
		done
    wait
    exit 0
}
trap finish SIGHUP SIGINT SIGTERM

echo "Starting Sidekiq workers..."
for i in `seq 1 $num_workers`; do
	bundle exec sidekiq -C $sidekiq_config -L log/sidekiq.log -i $i -P tmp/pids/sidekiq-$i.pid -d
done

while true; do
	sleep 60 &
	wait

	for i in `seq 1 $num_workers`; do
		pid=$(cat tmp/pids/sidekiq-${i}.pid)
		rss=$(cat /proc/${pid}/status | grep VmRSS | awk '{print $2}')
		now=$(date +"%T")
		echo "[$now] Sidekiq:$i memory usage: $rss KB"
		if [ $rss ] && [ $rss -gt $max_memory ]; then
			bundle exec sidekiqctl stop tmp/pids/sidekiq-$i.pid 30
			sleep 1
			echo "Starting Sidekiq:$i..."
			bundle exec sidekiq -C $sidekiq_config -L log/sidekiq.log -i $i -P tmp/pids/sidekiq-$i.pid -d
		fi
	done
done

There is one very tricky thing, in your Docker file, make sure you do the following or your script will never trap the signal:

...
ADD ./docker/sidekiq/run_sidekiq.sh ./run_sidekiq.sh

RUN chmod o+x run_sidekiq.sh

CMD ["./run_sidekiq.sh"]

Here are what I achieve with the scripts above:

  1. 1. Sidekiq workers will be restarted if they use too much memory.
  2. When EC2 instance gets terminated, Sidekiq workers will shut down gracefully.
  3. I sleep better.
Advertisements

Rails Script

Sometimes I want to write a ruby script that uses the models or other things in the Rails application. The simplest way to do it is to write a little script and executes it with Rails script runner. But what if the script is more than a few lines of code and needs to be able to handle multiple arguments? Let’s say I want to build a thrift server that uses Rails models and put it under Rails.root/script folder:

#!/usr/bin/env ruby
require 'optparse'

# Default options
app_config = { :environment => "development" }

# Parses options
OptionParser.new do |opts|
  opts.on("-e", "--environment=name", String,
          "Specifies the environment for the server to operate under (test/development/production).",
          "Default: development") { |v| app_config[:environment] = v }
  opts.on("-p", "--port=name", Integer,
          "Runs thrift server on the specific port.",
          "Default: 9090") { |v| app_config[:port] = v }
end.parse!

ENV["RAILS_ENV"] ||= app_config[:environment]

STDOUT.sync = true

# Loads Rails environment
require File.expand_path('../../config/boot',  __FILE__)
require File.expand_path("../../config/environment", __FILE__)

app_config = YAML::load(ERB.new(IO.read(File.join(Rails.root,
  'config/app.yml'))).result)[Rails.env].with_indifferent_access.merge(app_config)

port = app_config[:port] || 9090
thread_count = app_config[:thread] || 1

# Includes thrift-generated code
$:.push("#{Rails.root}/app/thrift")

class AppServiceHandler
  
  def initialize(options = {})
    ...
  end
  
  ...

end

handler = AppServiceHandler.new(app_config)
processor = AppService::Processor.new(handler)
transport = Thrift::ServerSocket.new(port)
transportFactory = Thrift::BufferedTransportFactory.new()
protocolFactory = Thrift::BinaryProtocolFactory.new()
server = Thrift::ThreadPoolServer.new(processor, transport, transportFactory, protocolFactory, thread_count)
server.serve()

JavaScript Alert Dialog is ugly

Rails uses JavaScript alert by default for confirmations, and developers love using alert() because it is easy. Downside? I think it feels cheap. There are tons of solution out there, jQuery UI, ColorBox. In case you are using Twitter’s Bootstrap (isn’t it the best thing ever?), it’s pretty easy to replace using native JavaScript alert dialog with Twitter modal dialog.

Here is how I replace the alert() function using CoffeeScript:

# Public: Shows a modal dialog in replacement of JavaScript alert.
#
# title - Title on the dialog (required).
# message - Message on the dialog (required).
# label - Label for the dismiss button (optional).
#
# Returns jQuery Object for the modal dialog.
#
YourApplication.alert = (title, message, label) ->
  $alert = $('#application_alert')
  if $alert.length == 0
    html =  """
            <div class="modal hide fade" id="application_alert">
              <div class="modal-header">
                <button type="button" class="close" data-dismiss="modal">&times;</button>
                <h3></h3>
              </div>
              <div class="modal-body">
                <p></p>
              </div>
              <div class="modal-footer">
                <a href="#" class="btn" data-dismiss="modal"></a>
              </div>
            </div>
            """
    $alert = $(html).appendTo($('body'))
  $('h3', $alert).text(title)
  $('p', $alert).text(message)
  $('a', $alert).text(label || 'Close')
  $alert.modal('show')

So instead of calling alert(…), you can call YourApplication.alert(…).

Next thing is I want to replace JavaScript confirm with Bootstrap modal dialog:

# Extends Rails JavaScript to use modal dialog for confirm.
$.rails.allowAction = (element) ->
  message = element.data('confirm')
  title = element.data('confirm-title') || 'Are you sure?'
  ok = element.data('confirm-ok') || 'OK'
  # If there's no message, there's no data-confirm attribute,
  # which means there's nothing to confirm
  return true unless message
  $('#confirmModal').remove()
  # Clone the clicked element (probably a delete link) so we can use it in the dialog box.
  $link = element.clone()
    # We don't necessarily want the same styling as the original link/button.
    .removeAttr('class')
    # We don't want to pop up another confirmation (recursion)
    .removeAttr('data-confirm')
    # We want a button
    .addClass('btn').addClass('btn-danger')
    # We want it to sound confirmy
    .html(ok)

  # Create the modal box with the message
  modal_html = """
               <div class="modal fade" id="confirmModal">
                 <div class="modal-header">
                   <a class="close" data-dismiss="modal">×</a>
                   <h3>#{title}</h3>
                 </div>
                 <div class="modal-body">
                   <p>#{message}</p>
                 </div>
                 <div class="modal-footer">
                   <a data-dismiss="modal" class="btn">Cancel</a>
                 </div>
               </div>
               """
  $modal_html = $(modal_html)
  # Add the new button to the modal box
  $modal_html.find('.modal-footer').append($link)
  # Pop it up
  $modal_html.modal()
  # Prevent the original link from working
  return false

So now you can use the following in your erb file:

 <%= link_to "Jump", @jump,
      :confirm => "Are you sure you want to jump off the cliff?",
      :data => { :confirm_title => "Think Twice" , :confirm_ok => "Jump" } %>

I hope this helps.

Retrieves all friends information with Twitter gem

This might be trivial but since I do not find an example on the Github page, I would like to give a sample implementation.

# Public: Retrieves Twitter User IDs for the followers.
#
# Returns Array of Twitter User IDs.
#
def get_follower_ids
  follower_ids = []
  next_cursor = -1
  while next_cursor != 0
    cursor = client.follower_ids(:cursor => next_cursor)
    follower_ids.concat cursor.collection
    next_cursor = cursor.next_cursor
  end
  follower_ids
end

# Public: Retrieves Twitter Users the user is following.
#
# Returns Array of Twitter::User.
#
def get_friends
  friends = []
  get_friend_ids.each_slice(100) do |ids|
    friends.concat client.users(ids)
  end
  friends
end

This example shows you how to use Twitter::Cursor and with Twitter API, you can only get 100 user information at a time (that’s why I have each_slice(100) ).

Click here to find out more about Twitter gem.

Guard with OS X Lion Notification

Since Lion has notification center built-in, I do not want to install Growl again. It would be great if I don’t have to switch between editor and terminal to see if all of my tests passed whenever I saved a file. Turned out, with the latest guard with Terminal notification, I got what I wished.

In case you don’t know what guard is, you should check out Ryan Bate’s Railscast on Guard.

Update your guard with bundle update guard, and add the following in your Gemfile:

group :development do
  ...
  gem 'terminal-notifier'
end

And… MONEY!

 

Testing Resque background jobs with RSpec

Let’s say you have a model which involves a background job upon save, or you want to schedule a background job in the controller, how to do you make sure the background will run correctly? One way is to use expectation mock like Resque.should_receive(:enqueue).with(…), but a simpler/better way is just put the following in your before…end block:

Resque.inline = true

That will run your background job right away.

Remember me, devise, and omniauth

This is how you do remember_me with Devise and Omniauthable, this is the OmniauthCallbacksController in one of my apps:

class OmniauthCallbacksController < Devise::OmniauthCallbacksController
  def facebook
    auth(:facebook)
  end

  def twitter
    auth(:twitter)
  end

protected
  def auth(provider)
    auth_hash = request.env['omniauth.auth']
    @user = User.find_for_omniauth(provider, auth_hash, current_user)

    if @user.persisted?
      flash[:notice] = I18n.t "devise.omniauth_callbacks.success", :kind => provider
      @user.remember_me = true
      sign_in_and_redirect @user, :event => :authentication
    else
      session['devise.omniauth'] = auth_hash.except('extra')
      redirect_to new_user_registration_url
    end
  end
end