Apache Passenger

This is a small entry which got me and Abhijith confused. We started Apache with Passenger installed as a module after configuring an app from a railscasts episode. Added the entry using the Passenger Pane. But it showed an access denied. Apache was running as a separate user. I gave permissions for the application directory, but still the result was the same. Abhijat then added permissions to the parent directory as well and the applicaion started running. Dont knwo why Apache needs the permissions for the parent directory as well.

Its written here ...

Ajaxified Will Paginate

I added the will_paginate plug-in for my rails 2.3.2 application but quickly found that there was no way I could ajaxify it. So I made a renderer of my own by modifyind some code. It was fun and effective.

The will_paginate.rb file inside config/initializers look like this..

class PaginationListLinkRenderer < WillPaginate::LinkRenderer

  def to_html
    links = @options[:page_links] ? windowed_links : []

    links.unshift(page_link_or_span(@collection.previous_page, 'previous', @options[:previous_label]))
    links.push(page_link_or_span(@collection.next_page, 'next', @options[:next_label]))

    html = links.join(@options[:separator])
    @options[:container] ? @template.content_tag(:ul, html, html_attributes) : html
  end

protected

  def windowed_links
    visible_page_numbers.map { |n| page_link_or_span(n, (n == current_page ? 'current' : nil)) }
  end

  def page_link_or_span(page, span_class, text = nil)
    text ||= page.to_s
    if page && page != current_page
      page_link(page, text, :class => span_class)
    else
      page_span(page, text, :class => span_class)
    end
  end

  def page_link(page, text, attributes = {})
    if @options[:ajax].nil?
      return @template.content_tag(:li, @template.link_to(text, url_for(page)), attributes)
    else
      return @template.content_tag(:li, @template.link_to(text,'#', :onclick => ajax_url_for(page)), attributes) 
    end
  end
  
  def ajax_url_for(page)
    return @options[:fn_name]+'("'+ (page.to_s) +'")'
  end

  def page_span(page, text, attributes = {})
    @template.content_tag(:li, text, attributes)
  end

end

WillPaginate::ViewHelpers.pagination_options[:renderer] = 'PaginationListLinkRenderer'
WillPaginate::ViewHelpers.pagination_options[:previous_label] = 'Prev'
WillPaginate::ViewHelpers.pagination_options[:next_label] = 'Next'

The ajax_url_for page gives the javascript to be called for the link. The options added are

:ajax => true (to signify that it is an Ajax request)
:fn_name => some_function_name (The Javascipt function which has the Ajax call)

The view is a partial which requires these local variables to be passed

A collection of paginated records using will_paginate's Model.paginate(:page => page)
A base_url which has the url to which the page parameter should be added.
The container which should be populated after the Ajax call.

The partial looks like this ...

<script type="text/javascript"><br />var access_page_for_paginate = function(page_no){<br /> <%= remote_function(:update => container, :url => base_url, :with => '"page="+page_no') %><br />}<br /></script>
<div class="pagination_container">
<%= will_paginate collection, :ajax => true, :base => base_url,
:container => container, :fn_name => 'access_page_for_paginate' %>
</div>

And the partial is called like this ..

<%= render :partial => 'partials/ajax_paginate',
:locals => {:collection => @feeds, :base_url => show_service_url, :container => 'profile_all'} %>

Post Production Stress

Two weeks before we went live with one of our products. A product where I worked personally for 4 months. I still do not feel I did justice to the product as the actual problems still glare at me when I see the application. It was a whole lot of work within 4 months and a more work when it went live. As the product was the first one to go live from us, we where a little tense. Some glitches here and there and I was all stressed out.

Its a whole big deal making an application live or into production. Especially if the application is not owned completely by you. The task of closely monitoring it, making sure that nothing breaks, maintaining the backups, ensuring that binding is done quickly and effectively if something breaks is completely done by the team. This is very tense. And I deeply appreciate all those managers who sympathize and stay beside their team during those stressful times. If you are answerable then you better have pretty good answers.

The build technique that was made is my main area of concentration this time around. Indraneel was the person in charge and I sat with him one time to deploy the code live. The scripts written via capistrano were excellent. The deployment can be to the test server or the live server. This requires issuing of different commands and writing separate tasks. The commands issued are remote commands and they usually are stop and start commands for mongrel. This application had also ap4r in it for asynchronous processing and therefore required a restart for that as well.

Monit has also been installed in this server and ap4r and mongrel has been configured to restart once stopped. The mod proxy balancer has been put along with apache so that mongrel clustering can be implemented. The balancer member written inside the proxy tags has an ip associated to it. This is where the actual transition in case of updates for the live application comes into picture. Using apache we could have something called a graceful restart using the 'apachectl graceful' command. Normally when a new code is deployed with all those changes and upgrades the server (mongrel in this case) must be restarted. But with graceful restart Indraneel just has to do minimal stuff. He first deploys the new code onto a server started with a different port, say mirror. Now what he does is change the balancer member port number to the mirror port and does a graceful restart. And wallah .. a work well done. This could also be automated if there were 2 virtual host files and according to the live/mirror deployment, the file is overwritten by the specific files.

Great work !!

What a week !

This week was a long one. Lots of work to be done, or atleast was asked from us. Amit and Abhilash had to put in extra hours to get it done and the work still continues as I write this post. My work is some what done so I am calmly sitting at a corner so that no one notices me.

The bugs that I got were all bugs which arose as a result of some miscommunication. Sometimes I understood it wrong, otherwise they interpreted it differently. 'They' are obviously NOT wrong. The major bug that took most my time was the one related to the Sortable list not working in IE7. I tried juggling with those classes but reached no where. At last a simple ' ' solved my problem :). The next major one was that the auto-completer in scriptaculous kept on disappearing when the container div's scroll was clicked. This was obviously because all the browsers except Firefox interpreted this as a valid auto-complete text field blur. The work around done was to entirely take off the onBlur event and be entirely dependent on the keypress(ESC key) and the selection of the element inside the autocomplete list.

Coding for the browser was also something we learned here. Amit had lots of problems uploading flv files via Safari. As I understand Safari gives a String Object instead of a StringIO object and therefore checking the content type becomes impossible. The attachment_fu plug-in therefore bombs and gives errors. Kudos to Amit for solving this bug.

To top it all up, I was to travel home today and my flight got canceled. Can't still believe it. This guy calls me up early in the morning to say that the flight has been canceled. He asks me the mode of refund and pleasantly says thanks. I had to spend a thousand rupees more to get a ticket home and am 'supposed' to go tomorrow. Let's hope that works out.

Asynchronous access via Ruby On Rails

Over the last month we have been pondering upon a major problem that needed to be implemented in a Ruby On Rails application. The statement of the problem is simple. I have a procedure/algorithm which takes two input sets, gets each element in the set and using a combination of the elements in the sets calculates a rating. As the problem it self suggests, this is a tedious process. Database access is evident and the procedure is also lengthy. I decided to write the algorithm implementation in the lib folder, so that it is accessible inside the application, and then call it inside the controllers.

If the algorithm was called inside the controller, the response would stall until the whole algorithm was completed. This is NOT what I wanted. The user should not be affected by the time the algorithm took to complete. The requirement here was to run this process as another thread/process at another place. To replicate the application environment in another server dedicated to this did not seem to be a great idea.

Thats when background process was suggested. Running the algorithm as another thread was another option. As the algorithm only required the sets, it could be run as a different thread. But the environment that the server provides must still be loaded. Ap4r came to the rescue here. Asynchronous Processing For Ruby could be used by any rails application to route all those tedious and time consuming accesses. This I thought was a smart way of doing a background process, although it is NOT a background process. It is just another url access. The Gem must be installed along with the plugin and the server must be started and wallah !! You are in ! Simple as that.

Next comes the trigger, ie the part where you actually need to start the background process. I had two places which acted as triggers. One was in the controller part and the other was from a model. Triggering from the controller part was easy as the plug-in's init file actually added the ap4r method with the Action Controller. Triggering from the model was tricky. As far as I know there is no way to actually call a url/controller method from a model. Theoretically this is wrong as the controller is supposed to be the base and the model does not need to know the controller which is accessing it, if it is accessing it.

I took the easy way out of this by firing a system command which did a wget to the url. This method (corresponding to the url) written in the controller does an asynchronous access thus returning immediately. Thus the system command does not stall the execution of the model method when it is called. The ap4r server is still called and the algorithm is run asynchronously thus serving our need.

The algorithm had many stages of executions/ filters along with database accesses. I managed to use message passing by calling the send method for the objects and lots of array operations. I thought I did a pretty good job in writing the algorithm. The vast thought that has been gone in developing a language like ruby must be praised here. Kudos to Yukihiro “matz” Matsumoto.

A lot to do ..

I have started enjoying some of my free time so much that I now make free time to enjoy. Staring with playing badminton, I will get better at this, to learning how to play table tennis. The carom board has been taken over by some people from another project and we could not play because of this. I have started going to the gym as well. But work has to be finished within the time frame and I strive to finish all the important work. This post is all about Rails.

Before Rails I knew what an MVC was and how to implement it. But never got an entire framework bundled with all the features. I am a relative starter in this field but it took little time to understand how great is rails when considered as a rapid application development platform. The tests also was a pro as I never had written a single test when I was doing JAVA [:)].

The usage of helpers like link_to_remote and form_for has been great as it greatly reduced application development time. I also found it better to write all the configurations in a yml and load it via the environment file that to write static classes which returned all these. And the plug-ins that it gave. Just awesome. Think about not writing a single piece of code for the authentication and tagging features. It even has Lucene plug-ins. I have also used the paginating_find plug-in.

The stuff that I hear most from my friends is the problems that ActiveRecord has. As per Abhijat, it took 13 secs to execute a simple query with limit and 3-4 secs without limit. I also understand that loading with include is actually transformed into join statements which is bound to up the query time and object creation time. 1+N problem ! And their effort. I also liked wrting rake tasks as they are so ruby ! :)

I also see that even though twitter is having so much problems extending, a lot of rails sites have come. Slide Share is one of them. I also saw this site called Crazzyegg. They use nginx and mongrel servers. I have not seen a site use mod-rails as of now.

Iteration's Over

4 days of mayhem. Sleepless nights and total work. Now I understand what a 'working professional' feels like. Most of the problems where HTML based. We, me and Amit, did all we could, from fixing HTML to Functional and Unit testing. Even after having worked so hard, we got errors during the demo. These errors were quickly sorted out, but pressurized us. But all that is well, ends well (I am a pessimist by the way).

Ruby On Rails coding has been great up until now. Yesterday I had a party along with Vivek, Rohan, Abhilash, Abhijat and Amit, and then Vivek told me the problem he faced when a query fired via find_by_sql took 5 secs to execute. He tells that the query is a simple one with a where clause, no 'in' statements, but fetches 20000 odd records. The same query run via php-myadmin runs in .00x odd secs. Is the Active Record a bad ORM then? I also find many people stating that not using the Active Record in Rails will solve a whole lot of problems.

It was also interesting to find that the ':include' option in 'find' will return you the object to which it has Foreign Key - Primary Key relationship. But it doesnot work if there is a select clause. But why ? If i have a credit table and a user table, then I cant write a find statement, with :include, which gives me the credit amount and the user data? It is a join at the end of the day and the populated stuff, which I understand is bad, could be populated along with the user hash too. This is also the case if I iterate through an array with records which were fetched with along with a select statement. Could not decide if this were logical.

We also put some validations along with the code. Live Validation was introduced to me for the first time and I loved it. I also understand there is a RoR plug-in for this which directly hooks up to the model and then writes its own Javascript code according to the validations given at the model level. Thats just 'WOW' !

I out of ideas as it has been a long time now that I have written something.

But as Arnie says 'I'll Be Back !'

Alternate Comet Ruby On Rails

I happened to have a talk about the previous post to Raja. The implementation had its own problems and changes.

Firstly : The Service Layer of the app, the part which communicates with the queue, must be separated from the controller. We don't want the controller to have a heavy load. The service layer will not know the part of the application that communicates with it and the controller will not know who its contacts. This ensures that even if the service layer has implementation changes, it will not affect the controller thus making it entirely a different entity. I guess this kind of distinction will only come with age and experience. The Facade Pattern is the base.

Secondly: The application is better off with cache implementations. Mainly because the queues are designed to serve a purpose. Communication between apps. The Cache will also be maintained by a service layer and the access via the controller will either be fetched from the cache or if expired from the database. To have a uniformity between the kind of platforms accessing the application (say Adobe AIR and the browser accessing the Mongrel server), we could have an identifying parameter parameter passed along with the url which tells the controller the type of client and the resultant format of output (say XML/ JSON/RHTML).

I feel I have lifted myself a step now !! Thanks Raja .

Comet Ruby On Rails

The stuff that we do in the company is quite cool. It lets me, personally, to go through many technologies and implement many ideas. Concepts in theory is not great unless you start to implement those. A new thing that we are trying to accomplish is the functionality that a Comet Server gives, an HTTP push. This must be implemented in Ruby On Rails. Now the point here is that as HTTP is a connectionless protocol, we cannot pass stuff from the server to the client (unless the client requests). To have this functionality, we must either do a polling technique, where in the client constantly accesses the server for information or a live connection (may be a flash connection). Now another techniques which goes against this HTTP connectionless concept is the one that Comet deals with. A long lasting HTTP connection, either streaming or long.

Juggernaut is a Ruby On Rails plug-in which has this kind of functionality. It uses a flash open socket connection to get this server side push. Now this is great but the main problem that I felt is the firewall problem. Most of the firewalls do allows connections from other ports to go through them. Juggernaut also say that we could connect through the 443 port (http ssl) and that would solve the firewall issue. For a general need this is great.

My requirement was a little specific and I needed more stuff to get this stuff working. That was when I got to see the Active Messaging plug-in that Ruby On Rails had. A plug-in which could take in data from an Event Driven Architecture based product (say Queue / Messaging) and consume results in a Provider/ Subscriber model. The application also needed a lot of database optimization ( it will have to survive a lot of select statements). So an intermediated queue is perfect.

This is what it might look like.






















The only problem that I am facing is that by default the Active MQ is supposed to be a first Come First Serve queue. Now this does not serve my purpose as there might be many clients subscribing to the out queue. What I need is durable subscribers and that problem has been addressed in their thread.

This is a great way of accessing data (messages) as it is inherently the publisher/subscriber pattern (which I need) and the interoperability between different application could also be possible (a flex app accessing not HTTP information ?). Load balancing could also be done on queues !!