A New Zoho Invoice Ruby Gem

With all the choices for invoicing systems out there, making a decision on which invoicing system to use can be challenging. In my company's case the decision really only boiled down to one feature, the ability to send invoices via mail. That being said, there weren't any good choices out there for Ruby gems, so I figured I'd build my own. Most of the inspiration for setting it up came from the official Zendesk ruby gem.

I drew my inspiration from two main design aspects of the Zendesk gem. First is the concept of a client and that client gets passed to all of the associated resources in the system. The main benefit of that pattern is that it allows the developer to inject a mock client if they so choose for testing purposes, which I think is absolutely awesome. The second reason is the ability to to do ActiveResource style scoping through the client so that you don't have to pass a client around if you don't want to. I believe that between these two patterns it makes for interacting with an API a breeze. For example:

To get started, download the latest version on rubygems.org and checkout the README. At the time of this writing, the gem is at version 0.3.1 and its not quite a full representation of the API. I plan on 1.0 being that milestone in which you can interact with all API endpoints. If you have any issues with the gem, please feel free to file an issue in the github repo.

Precompile Static HTML Pages with Rails 3 Asset Pipeline

At CarWoo we recently upgraded all the way from Rails 2.3.8 to 3.2 and with that brought the asset pipeline. We probably could have skipped the asset pipeline but we figured we should throw in it for good measure. One of the problems that we ran into was including some of the compiled javascript and stylesheet files from the assets directory into our 404 and 500 error pages. In Rails 2.3, we just created static html pages and threw them in the public folder and didn't think about them ever again. Unfortunately, since the Rails 3 asset pipeline adds and MD5 hash to the js and css files names, statically linking them wasn't going to work.

I wish I could take credit for this, but the honorable @sshconnection recommended that we just precompile 404 and 500 pages as well. And wouldn't you know that, it was way easier than you think it could be.

First get your config in place:

Just like any Rails convention, add the extensions to want to use to precompile. In my case, I used erb in my 404 page:

This probably should be labelled as a quick tip, but it was a little a-ha moment for me and I figured I'd share it.

Start Using Shogun Toolbox in Ruby

Shogun Toolbox is the ultimate in machine learning. It provides a number of different machine learning models like support vector machines (SVM) and hidden Markov models just to name a few. So why use Shogun toolbox? Well, when you look at the SVM landscape, for example, you'll find many different publicly available implementations. Let's say you needed to switch your code to use OCAS instead of LibSVM, the semantics of interacting with each implementation could vary wildly. As a result the authors of Shogun Toolbox state, "the motivation for creating a machine learning toolbox was to provide an easy, unified way for solving certain types of machine learning problems." This is great for us because that means we'll get access to many more machine learning models without having to find or build C extensions for Ruby that interface with the SVM implementation we want to use.

What are we trying to solve with Shogun Toolbox?

According to the wikipedia page for Shogun Toolbox, it was developed with bioinformatics in mind. With an industry like that, you should expect lots of data, and do I mean lots. We're talking about millions and millions of data points. Ok... so it can handle a lot of data, but what else is it good for? The real sweet spot are two- and multiclass classification and regression problems, according to the overview paper. There are some other things packaged with the toolbox but know that if you're looking to do classification this is the package for you.

Compiling Shogun Toolbox

The easiest way to get started would be to download the source from github. Once downloaded, make your way to the src directory so that you can compile the toolbox for your machine.

It's as simple as that. Theres a whole host of options that you can configure when you're compiling the program. I would encourage you to check them out as you use the toolbox. Since we're really interested in doing this for Ruby, we need to do a couple more things before we get the toolbox working. First, install the ruby gem narray to your system. Its pretty simple:

Then when you configure the toolbox you'll need to specify that you also want the ruby_modular interface:

There are other interfaces for languages like Octave, Java and Python. Feel free to include them at this time. You should be good to go from this point, unless you're an RVM user.

Gotchas for RVM users

Unfortunately at this time, when you configure the library, it's not smart enough to know that you need to point to your current ruby instead of the system ruby. You'll see a weird error in the build process like Undefined Symbol: _rb_str_new_cstr. You'll need to collect two different pieces of information.

  • The first is where your ruby lib lives. You can issue rvm info and find the environment variable MY_RUBY_HOME.
  • The second is where your platform specific include directory. You can issue ruby -e "puts $:" at the command line and that will show you all of the directories ruby will look to include code. Make sure you've switched to the ruby you want to use! I recommend you put the compiled modshogun lib in the path with site_ruby and the platform specific folder.
Once you get that, all you need to do is slap those on the end of your configuration options:

If you didn't notice when you add the ruby dir to install to, modshogun will be available to all gemsets of the ruby you compiled against. If you need to namespace modshogun to a particular gemset, let me know if you have any ideas.

Running through some examples

Let's go through one of the classifier examples packaged with the source code, namely classifier_libsvm_minimal_modular.rb. I've added my extra notes in the comments:


If you're new to machine learning and/or classification, I would recommend checking out some other articles to get up to speed on the subject before jumping into just a big piece of software like Shogun Toolbox:

Mounting Resque Web in Rails 3 using Warden, Devise and the Asset Pipeline

Really the title of this post should be, "Don't fear middleware". So here's the set up, I had a Rails 3.1 app that was utilizing the Asset Pipeline, Resque and Devise. Like most people, I wanted to just mount Resque Server into the Rails app behind the authorization that I was already using and be done with it. The easiest and most elegant way of doing so is to do it in your routes. What's so awesome about this solution is that you get authorization and paths pointing to Resque Web just like that. Unfortunately, if you're using the asset pipeline like I was, you'll be unable to get any of the Resque Web assets served by the application in staging and production. By "using the asset pipeline", I mean setting the action_dispatch.x_sendfile_header = 'X-Sendfile' and serve_static_assets = false in the config file. Plus the solution given above for serving the assets from Resque is to copy them to your assets folder. BARF!

To The Middleware

Let's delve into the rackup file. If you're like most Rails projects you probably have a rackup file that looks like this:

Well thats going to have to change. First off you'll need to map the routes and then tell Resque Web how to handle authentication within the scope of rack. What you end up with is a file like this:

The only thing that's really left is creating a new instance of Resque::Server. I decided to do it via Sinatra rather than creating another rack middleware. I have an initializer in config/initializers/resque.rb like this:

There you have it, you can now serve Resque Web without having to copy any assets or do anything funky to the gem. This feels a little bit better to me since we're not really messing with the internal workings of the resque gem.

Text Classification using Support Vector Machines in Ruby

Support Vector Machines represent a group of supervised learning classifiers that will try to predict the outcome of a feature set. I don't want to bore you with the details but I would recommend that you read Ilya's post about SVMs. It's very informative and it will give you the basics of what you need to understand to get started.

In my case, I've been working on a fairly large scale text classification problem at work when I realized that SVMs would be perfect for the job. Unfortunately, for our needs the current state of Ruby support for SVMs wasn't where we needed it to be.

I am happy to introduce rb-libsvm to the world!

While I can't take credit for the entire gem myself, I did help clean up some pieces of the codebase and gemify the code to make it possible to install from Rubygems.

Basic Usage Example:

As for the future plans for this gem, one of the biggest features I can think of building would be the Marshaling of the Libsvm::Model object. This would be a big performance win since you wouldn't have to re-train the classifier. If you have any other ideas for features, I'd be happy to hear about them.

Configuring Your Rails 3 Plugin

With Railties, I've found writing plugins for Rails 3 to be much easier than before. Instead of writing a post about how easy it is to create plugins, I figured I would share what I learned about configuring your recently created plugin. After poking through the Rails code, I came across some interesting things that can really clean up and tighten the code in your Rails projects.

If you're looking for more of an introduction on Railties and Rails 3 plugins, I recommend that you read a couple of posts before you continue with mine. These posts should be enough to get you started.

Environment-specific Configuration

In the past, if you needed any type of configuration to be done in your app you'd more than likely create a yaml file and stick what you needed in there. With Rails 3 and Railties, you can move all of your plugin configuration into the environment specific files (application.rb / development.rb / test.rb / production.rb) within your Rails project. I like this so much more because all of the configuration stays in the same place and you don't have to litter your project with a bunch of yaml files.

Throughout this post, I'll refer to a library I created to use MongoDB as a FIFO cache store. Let's start with a snippet of development.rb in one of my current projects.

The important line in the above code is config.mongo_fifo. Did you know that you can create any variable that you want within the config object? The cool thing is that the config object is actually an instance of Rails::Railtie::Configuration and when you specify a method that doesn't exist, that class will actually go through method_missing and add the new method to a hash and assign whatever you wanted to it. So in the example above, mongo_fifo was added to the application's config hash with a new instance of MongoFifo. If you need a little more configurability with your methods, like action_mailer above, you can do that too. So let's say I wanted mongo_fifo to have a slew of config options, I could do this in my Railties class to add that ability:

Coincidentally, that's how ActiveRecord, ActionMailer and ActiveSupport allow for "namespaced" configuration in your Rails app.


Now that we moved the configuration into the specific environment files, we'll be able to access it via our initializers. I really wish I had more to write here but it's just as easy as accessing a hash. So in my MongoFifo code, I've got one initializer that will create a global variable within the Rails module so that anywhere within my app, I can read and write to my MongoDB cache. Here's the code from MongoFifo:

Get going!

The beauty of Railties is that you don't necessarily need to develop a whole gem to get the benefits of it's modularity. In my case, I wrote some code that's a slimmed down version of an ActiveSupport cache store that works with MongoDb's capped collections. I plopped this file in my /lib directory. I'm able to put the MongoDB connection configuration in the specific environment (development / test / production) file and keep my overall Rails project that much cleaner. Give it a go!

Using Allocate to Create ActiveRecord Objects When Stubbing find_by_sql

find_by_sql. What a method. I love the flexibility it gives me when I need to get information from my database with as little fuss as possible. Yet when I want to stub find_by_sql and return real objects in my rspec tests, a little part of me gets frustrated. The real pain comes in when you want to have an object with attributes that aren't part of the original class and typically you're going to run into that when you use find_by_sql.

In my particular use case, I had two different tables, the first was a Shop table and the second was a Job table. In terms of the association, a Shop could have many jobs. The find_by_sql method was aggregating data from the two tables and creating each instance of the result as Job object. So it looked something like this:

I'm sure that you're thinking to yourself that I could have just created some factories or fixtures and then had my tests just query the database for the records. That probably would have been the easy route, but I was wondering if I could pull this off without the database call. So how do we do this?

First, we need to understand how ActiveRecord actually creates new objects. The post from Peter Bui is a great first step in explaining what goes on when ActiveRecord queries a database and creates a new object. The important thing to note in his article is Ruby's ability to create an object without calling the initialize method via allocate.

From there it was time to look at the core ActiveRecord code to see what is going on. Just to note, since Peter's article, ActiveRecord::Base has changed a little bit. First I started with the code for find_by_sql:

You can see in this piece of code that ActiveRecord is hitting the database with your custom SQL and then for each result creating a new instance. Now let's see how that instance is created:

The first line of code is looking for the appropriate class to use to create the object via find_sti_class. Then it allocates an object based upon the class name. The next line is really where the meat is. init_with will actually then create the object with the attributes that you specify. Sweet! I think we struck gold.

Now that we know about the init_with and allocate methods we can now create real objects to use in our tests if we need attributes outside of what the object provides. So in the example that I provided above, I could do something like this:

Excellent! Now I can stub the find_by_sql call and I can create objects that will be in the correct class that I need.

Do note that what I did above is for edge rails. If you're on 3.0.1 or below you're going to have to do this to create your objects:

Also, pay attention to the way the hash was created. You'll need to use strings for the keys otherwise you'll get a bunch of errors.