This is part 5 of my ongoing series on Ruby on Rails performance. Today’s topic is a bit more complicated that some of the previous parts.
I’m going to explore an alternative to using the method ActiveRecord::Base.to_xml.
Before I get into that, I probably should explain why. After all, ActiveRecord::Base.to_xml is really easy to use. The alternative I’m going to demonstrate isn’t very easy to use.
The problem is that ActiveRecord::Base.to_xml can be really slow—even after making the optimizations I’ve explored in the previous parts to this series. For fairly simple AR objects, this won’t be a problem. However, when you have a deep object hierarchy and need substantial portions of it to be included in the XML response, it becomes a problem.
One instance where I encountered this was a nested has_many tree. That is, the object tree looks something like this:
ParentObject has_many ChildObjects
Each ChildObject has_many OtherObjects
The XML output is often 100k or more, and representing a few dozen or more total objects.
This is a quick one, but if your Rails app uses ActiveResource as a client, it can be really useful to know something about each ARes request.
ActiveResource actually has some built-in logging of what requests are made, the response size, and how long it took. But, it’s not enabled. The good news is, it’s easy to turn it on.
I like to make ActiveResource’s logging match the rest of my environment — that is, it will log when in development mode and not in production mode.
Add the following to a file in config/initializers/:
ActiveResource::Base.logger = ActionController::Base.logger
Obviously, if you want it to go to a different log file, or function at a different log level, you can create a new Logger instance instead.
The logging output will look something like this:
GET http://api-server.com/resources.xml
--> 200 OK (1526 69ms)In part 3 of my Ruby on Rails Performance Series we’ll look at how to improve the parsing speed of XML content.
Rails will automatically parse XML in two key places:
1. When XML content is sent to a controller/action pair via POST or PUT. (Your app is an API server.)
2. When XML content is received from an ActiveResource request. (Your app is an API client.)
The default XML parser is REXML. It’s written in pure Ruby which makes it very portable. And slow. Very slow.
So, we need to replace REXML with something faster. The key is the libxml-ruby gem which is just a binding to the very fast libxml2 C library.
libxml-ruby has a different set of methods from REXML and is not drop in compatible. If you’re planning on moving your own REXML parsing code to libxml-ruby, it will take some work. However, it’s worth it. In one particular project, I had about a 100x speed improvement!
Rails, as it often does, makes the switch to libxml-ruby super easy. First, install the gem (requires a functioning compiler, the libxml2 library, and headers for libxml2):
gem install libxml-ruby
Then, just add a new file to config/initializers/. Let’s call it config/initializers/xml_config.rb. In that file, put this:
ActiveSupport::XmlMini.backend = 'LibXML'
This was only added in recent versions of Rails, I believe in 2.3.
Restart your Rails app server(s) and off you go!
The standard way to build XML in Ruby is with the Builder gem.
Builder iterates over each character in the output text to determine if it needs to escape it. When generating larger XML documents, this can be quite slow.
Thankfully, the solution is already available via the fast_xs gem. This gem does compile an extension, so you’ll need a working compiler.
gem install fast_xs
Then just require it after Builder:
require 'builder'
require 'fast_xs'It will be used automatically by Builder going forward.
If your app is built on Rails, it’s even easier. Just install the gem—that’s it! Recent versions of Rails will auto-detect the existence of fast_xs and enable it.
Lately I’ve been doing a lot of API type work in Rails — both client and server. It mostly uses Rails’ default XML generation and consumption.
Along the way I’ve discovered that the default performance of XML and related things on Ruby and Rails is, shall we say, less than amazing. On the bright side, there are a number of ways to improve matters.
So, I’m going to do a series on improving the performance of Rails’ API-related stack.
I’ll start off with serialized columns in ActiveRecord.
I bought something on ebay this morning (a 60-year old book). That’s not particularly exciting. But in the process of trying to pay for it, I had several observations.
1. Finding and buying the item on ebay was fairly straight-forward. I’m not sure why the item didn’t show up in the main search results, but several acceptable options did show up in the “Here’s what people have in their stores” section. Good enough.
2. ebay continues to try to force everything through Paypal. I was going to do that this time anyway, but the lack of choice is below average.
3. ebay and Paypal are still run as separate sites. So when I click on checkout from ebay, it sends me over to Paypal. That’s the theory anyway. Instead I get a giant error message telling me that Paypal has experienced an error and that I should wait and try again.
So I wait. And try again. And it fails again.
I wait some more. Same error. “Odd,” I think.
Months ago (years maybe) I started configuring my web browsers to only allow cookies back to the source domain. This is mostly a privacy protection attempt as it keeps volumes of advertising cookies from being accepted.
On occasion some site exhibits bad behavior by relying on these cross-domain cookies. Wondering if ebay/Paypal is one of these errant sites, I dig into my browser’s cookies.
Sure enough, cookies that are shared between ebay.com and paypal.com have been refused. I enable just the select cookie and try the checkout again. Now it works.
While reviewing all the cookies, there were more than a dozen various advertising cookies that ebay also tried to set. This merely justifies my general decision to block such things.
The failure here is really two-fold. One is not having a better way to hand off the session data between domains — something in the url would seem appropriate enough.
The second is having an unhelpful, misleading error message.
4. Lastly, Paypal still, after all these years, refuses to let me set a credit card as my default payment method, instead forcing me to go through several clicks and multiple pages to override it for each purchase.
I’m reminding yet again why I buy from ebay and use Paypal so infrequently. It’s just a hassle.
One of the common questions in Rails land is how to disable the funny numbers at the end of images and CSS stylesheets. Let’s explore what exactly those numbers are, why they’re sometimes helpful, and how to turn them off when they’re not.
Just for fun, I decided to try Yahoo’s search advertising for a little while. It’s been running for a few weeks now.
Here’s my take on a series of aspects of their service.
First, the initial reason I thought I’d try it: the keyword bid rates are lower. I figured they were lower for a reason, but thought it couldn’t be that bad. Boy was I wrong.
Just a quick note to introduce a few services I’ve had a hand in developing.
Zerigo Pods are high-performance, easy to use virtual private servers (VPS’s). Zerigo Watchdog is a simple (and free!) server monitoring service. Zerigo NS is a straightforward DNS service (also free!) with a REST API.
I have a Xen 3.2.1 server and the guests (domU’s), after a few months of uptime, are all behind about 45 seconds. Interestingly, the dom0’s clock is fine.
It seems that the hypervisor’s clock is what’s off as all the guests are off by the same amount. A reboot would likely fix it, but what fun is that?