Bucket Maker

After some days of research and work, I managed to create a gem which focused on Creating buckets for objects and querying them when needed

Have a look at https://github.com/dinks/bucket_maker which is a gem for Rails 3.2 -> 4 .

Comments most appreciated !

That Great Feeling

Words can't express that feeling when you upload something to a community that everyone can use !

https://rubygems.org/profiles/dvasudevan

:)

Module extensions 1.9 and 2.0

I did not know that this threw TypeError on 1.9 but passes for 2.0

x = Module.new { def foo; end } 
Module.new { define_method :bar, x.instance_method(:foo) }

From Rails !

Fibonacci - Golden Ratio

In mathematics, the Fibonacci numbers or Fibonacci series or Fibonacci sequence are the numbers in the following integer sequence: 0, 1, 1,2, 3, 5, 8, 13, 21, 34, 55, 89, 144 ...

I have always done the one with recursion (or iteration). But mathematics says we could do it with the Golden Ratio ..

Dom Mutation Observers and other Topics

A great video about Dom Mutation Observers


Some notes to read as well

http://infrequently.org/2012/04/bedrock/
http://smus.com/how-the-web-should-work/
http://www.x-tags.org/
http://www.polymer-project.org/

Js for Sublime

Setting up a build system for Sublime Text for js

Let your interpreter be Node. If you don't have Node, I would recommend installing it from here. It uses the Google V8 Engine.

Open Tools -> Build System and select a New Build System

Name it whatever you want it to be (this will show up in the Build Options)

Add the following to it

{ 
 "cmd": ["node", "$file", "$file_base_name"], 
 "working_dir": "${project_path:${folder}}", 
 "selector": "*.js" 
}

The syntax could be understood from this link.

Restart sublime and open the file. Goto Tools -> Build System and set the build system to the name you gave.

Run Command + B and it will work !

All the build files go into /Library/Application Support/Sublime Text 2/Packages/User

:)

Statsd

During the last couple of days, I was trying to find myself around Statsd from Etsy. It seems to be a wonderful tool which lets users send stats (or tracking) to a server which in turn could be visualized using tools like Graphite.

The set up is quite simple. Add the Statsd gem to your application's Gemfile and then write a Rack Middleware which sends out information to the predefined server. Before sending out information, we need to setup the Statsd NodeJS Server and give the IP and port for the Gem to work.

Matt Aimonetti has written a great post about setting up the service.

I have found it damn hard to set up Graphite.

The download is easy. git clone the 3 repositories and then run check_dependencies.

Install all the required libraries using pip. Now comes the hard part. A 2D graphics library called Cairo is a dependency for Graphite and a library called py2cairo (pycairo if python >2.7) is to be installed.

I would suggest installing cairo using

sudo brew install cairo

Homebrew is better than MacPorts.

Install pycairo using pip

sudo pip install pycairo

Or

easy_install pycairo

You might get errors when importing the module for cairo. This might be because you installed it via easy_install.

If it does not work (it did not because I have 2 versions of python), symlink from the site-packages/Cairo to the place where python searches for site-packages.

Then run check_dependencies again and you are good to go. I still have to setup Graphite. But I hope I'll have less obstacles after this.

Experimenting with Rails 4

I had some time this week and I had to try out the Rails 4 and Ruby 2. That's exactly what I did !

I did not try out Key based Cache Expirations or Etag stuff. But I did try out some Server Sent Events stuff. And it went pretty well.

I started out installing Ruby 2 with rvm (I use rvm and don't use to rbenv for some reason). I got through installing 2.0, but when I tried to use a specific gem_set for a test application, I got an error from Rubinius for some odd reason. It said that rbx was to be installed. Somehow it messed up the whole setup and I had to do away with gem_sets. I also had to remove the rbx* installation directory because it was bursting warnings to the console every time the rvm was set.

You must upgrade RVM :)

This benchmark is a great read for 2.0

I wrote the version to the .ruby-version file and that was the start of the project.

I wanted to do something which had some real time communication. Tic Tac Toe seemed to be a nice choice.

There was then this weird problem with sqlite3. It, for some odd reason, could not find the native binding of sqlite3. The path was correctly set but it just could not find the path. I had to spend some hours of debugging to find this out and a lot of googling !

The error was

/usr/local/share/gems/gems/sqlite3-1.3.7/lib/sqlite3.rb:6:in `require': cannot load such file -- sqlite3/sqlite3_ruby (LoadError)

This post saved me at last. Seems like we have to change the actual file of the gem to get this thing running.

In the case of rvm, the file would be under

~/.rvm/gems/ruby-2.0.0-p247/gems/sqlite3-1.3.8/lib/sqlite3.rb

and you will have to change the loading locations relative like

~/.rvm/gems/ruby-2.0.0-p247/gems/sqlite3-X-X-X/ext/sqlite3/sqlite3_native


Once this is done, rails finds it and loads it. This, I think, is NOT good for a Continous Integration Server.

Server Side Events came next. I went through this excellent post on how to get started with SSE. I managed to add Redis pub/sub with SSE after looking through this post. And ultimately used some Thread Processing to get it less Non Blocking as per this post.

Another problem I had was with the servers. I started with Unicorn which was a bad choice. Unicorn kills connections because it is meant for fast processing. For persistent connections, we need to have Puma or Rainbows!

I started using Puma but I did the mistake of not putting

config.preload_frameworks = true 
config.allow_concurrency = true

in the application.rb file. Only one request was permitted because of this flaw. This is to be removed later (I have no idea why this is not default ?).

But that did not solve my problems. Puma constantly gave me errors like

ThreadError: Attempt to unlock a mutex which is locked by another thread

which is not good. It never called the ensure block (don't know why) and I seriously doubt the sockets to be open :|

Thats when I tried Rainbows! I added the rainbow.rb file for server config and started the server. It worked great for me !

I also understood that attr_accesible is no more for rails and its all about Strong Parameters.

I tried adding rails-api to the project (this was my first time). It was only after some time that I realized that the stack for rails-api removes the Rack for Session handling because it aims for the Processing to be as less as possible

Rack::Session::Cookie

I was working with an existing Rails application and I did not want this feature. I tried adding

config.api_only = false

to the application.rb file. But that did not help me.

I had to add the particular require for the api for this to work in the Gemfile

gem 'rails-api', require: 'rails-api/action_controller/api' 

I wanted to try my bit with the state machine and I managed to write this inside Concerns.

The POC is still incomplete, but I loved doing this after a long time !

Feeling Little

Today I managed to catch up on some reading and it actually made me feel so tiny and insignificant.

Like the action is elsewhere. And looking up some of the high profiles in LinkedIn, I feel that to do something significant one needs at least a PhD/MS.

Some links

http://gigaom.com/2013/09/04/new-open-source-tech-marathon-wants-to-make-your-data-center-run-like-googles/

http://gigaom.com/2013/07/29/airbnb-is-engineering-itself-into-a-data-driven-company/

https://blog.twitter.com/2011/engineering-behind-twitter%E2%80%99s-new-search-experience

https://blog.twitter.com/2011/twitter-search-now-3x-faster

https://blog.twitter.com/2010/twitters-new-search-architecture

https://github.com/twitter

http://mesos.apache.org/

http://gigaom.com/2013/06/14/a-real-time-bonanza-facebooks-wormhole-and-yahoos-streaming-hadoop/

http://developer.yahoo.com/blogs/ydn/storm-yarn-released-open-source-143745133.html

https://www.facebook.com/notes/facebook-engineering/wormhole-pubsub-system-moving-data-through-space-and-time/10151504075843920

https://amplab.cs.berkeley.edu/projects/spark-lightning-fast-cluster-computing/

http://nerds.airbnb.com/introducing-chronos/

http://www.mesosphere.io/about

Vienna to Innsbruck on Rails

We had a short get away the past weekend to Innsbruck. We had decided to board a train from Vienna to Innsbruck after we landed by plane in Vienna. I had looked up the prices and found it to be reasonable.

We reached from the airport to the railway station on time and had to buy the tickets from the vending machine. The tickets cost me about 58 per person. We hoped on the Rail Jet train, in second class and found the cabins to be almost full. I and Reshma managed to get hold of a seat which was made for children. There was a small screen in front of us where all the animation movies were run. So time flew ! We watched clips of Donald, Mickey, Goofy, UP! including others.

The train stopped at St Polten, Linz, Salzburg to reach Innsbruck. It would be too less if I were to describe the scenic beauty of this route. Mountains on the left and right with luscious green fields, cattle and watch dogs, trees matriculately placed all cut and clean. What a view ! A breath taking treat to the eyes. The houses were all so beautifully placed as though a great painter had portrayed his dreams.

If you are plan to go to Vienna try this route ! You won't be sad.

Trip To Kerala

It took me more than a year to find some time to go to my home place. The wait was worth it. I managed to enjoy the Monsoons to the fullest extend, spend quality time with my Mom and Dad, visit Kumarakom with Friends and meet up some NITC batchmates. At the end, 2 weeks was not enough time.

The next time around I will go for more than 3 weeks. :)

Content Security Policy

I had not heard of this before a couple of days. I was trying to run something from the firebug console on github and it gave me an error of this kind.

Error: call to eval() blocked by CSP

Nothing ran. And even more confusingly, similar errors were thrown on page load. 

It seems like CSP is a great advancement in reducing the number of XSS (Cross Side Scripting) attacks.   By adding a header to the response, we add a list of white listed sites where the scripts are allowed to run and the rest encounter errors similar to the one seen before. Naice

For Chrome,
The header is X-WebKit-CSP
And the value looks like "default-src 'self'; img-src *; script-src 'self'; style-src 'self' 'unsafe-inline'; report-uri /Home/Report"

For Firefox,
The header is X-Content-Security-Policy
And the value looks like "default-src 'self'; img-src *; script-src 'self'; style-src 'self' 'unsafe-inline'; report-uri /Home/Report"

Less Caching

We came across this unusual problem when developing. So for starters, there is a Gem that we use and a test application reside in the test folder of the Gem. The Gem uses only Javascript in the app/assets folder and has a minimal Ruby code.

The Gem uses Bootstrap's less version to set the CSS. We use the mixins by Bootstrap and overwrite variables and stuff. The Gem still used SASS and we felt a little odd with writing mixed CSS. Therefore we moved all the stuff to less files and put it all in assets/stylesheets. Nice and organized. 

Now we had the problem. We made a change to the less files and the application did not load them. We had put all our imports in a main file and any change to the main file reflected in the test app. but not if the change were done inside the individual imports.

We wanted to analyze if the problem was with Sprockets or with less-rails. After some probe and hunt we concluded that the problem indeed was with less-rails. The Gem caches the imports which prevent any compilation after the import to the base file (first level).

I tried adding options like

config.consider_all_requests_local = true config.action_controller.perform_caching = false 

inside the environment files, but alas! no change what so ever.

So we had to do the inevitable. Look at files and trigger a restart and cache clear.

The process was to be something like :

if Change? 
  rake tmp:clear 
  powder restart (or touch tmp/restart) [we use pow]

Guard was perfect for this.

2 Guard plugins were used for this :
  guard-process => To run the rake process
  guard-pow => To restart Pow

The file looked something like this

# RUN bundle exec guard -p -i -w ../../ 
guard 'pow' do 
 watch(%r{app/assets/stylesheets/**/.*\.less$}) 
end 

guard 'process', :name => 'ClearCache', :command => 'rake tmp:clear' do
 watch(%r{app/assets/stylesheets/**/.*\.less$}) 
end 

The guard command needed to run with
  -w because we were looking at a change from another directory
  -p for polling
  -i for no interactions

phew..