Tuesday, April 10, 2012

Fedora Packages, Ruby Gems And Patching Sources

I'm the Fedora package maintainer for several Ruby language gems. One new package I'm preparing to release is the Ruby language bindings for the Qpid messaging framework.

One of the challenges I had to overcome was to apply some patches to the release that overcome some blocking I/O issues in the underlying codebase. We decided that the first release, for Fedora 16, would be based on our 0.16 release of Qpid, which doesn't contain the fixes I've written to provide non-blocking I/O functionality. (long story short, Fedora 16 provides Ruby 1.8, Fedora 17 introduces Ruby 1.9, and Ruby 1.9 has a better threading model). I've proposed an RPM based on our 0.16 code and needed to apply a set of patches on top of that for code that's not going to be a part of the upstream codebase for 0.16.

So the challenge was, how do I apply these patches on top of a gem when creating a the RPM? I'll need to unpack the gem, apply the patches and then rebuild the gem before continuing. Not an easy task, to say the least. But one that, with a little ingenuity, was overcome with ease. And I owe thanks to my buddy Ashcrow for help with a few issues.

Part 1: Unpacking The Gem

We have a set of seven patches that need to be applied to the base gem to provide our non-blocking I/O functionality. They are defined higher up in the spec:

Patch1: 0001-Ruby-extensions-to-use-the-non-blocking-I-O-commands.patch
Patch2: 0002-Updated-the-Rakefile-to-build-the-nonblockio-code.patch
Patch3: 0003-Modified-the-Qpid-Ruby-code-to-load-the-non-blocking.patch
Patch4: 0004-Modified-the-testing-environment-to-accomodate-non-b.patch
Patch5: 0005-Updated-the-spout-and-drain-ruby-examples.patch
Patch6: 0006-Cleaned-up-the-Ruby-bindings-documentation.patch
Patch7: 0007-More-cleanups-on-the-Ruby-documentation.patch

The first thing you need to do is open up the gem in order to apply the patches to the source code. In the spec file in the %setup section we have:

%setup -q -c -T

pushd ..
gem unpack %{SOURCE0}

pushd %{gemname}-%{version}
gem spec %{SOURCE0} -l --ruby > %{gemname}.gemspec

The spec exits the buildroot (holding an anchor so it can return) and unpacks the gem, specified by %{SOURCE0}. It then enters the subdirectory that gets created (again, holding an anchor) and generates a gemspec file based on the contents of the directory. This is necessary in order to repackage the gem once we've done applying the patches.

Part 2: Applying The Patches

This is the easiest part of the whole process, but it was also the spot where Ashcrow needed to set my head straight.

In our source tree for Qpid, the Ruby language code extists at qpid/cpp/bindings/qpid/ruby. But when you're inside of the unpacked gem, you're already at the bottom level of that tree. All of the patches that get applied, though, assume you're starting above the qpid directory.

To apply the patches, you have to tell the patch macro to ignore the first n levels of nesting in the patch files:

%patch1  -p6
%patch2  -p6
%patch3  -p6
%patch4  -p6
%patch5  -p6
%patch6  -p6
%patch7  -p6

Here we it to ignore the first 6 levels of directories when applying the patches, since they're generated with the prefix of "{a,b}/"; i.e., a/qpid/cpp/bindings/qpid/ruby/lib/qpid/encoding.rb is one example of a referenced file.

Part 3: Repackaging The Gem

One of the changes in this set of patches included deleting two source files, the aforementioned lib/qpid/encoding.rb and spec/qpid/encoding_spec.rb. When the repacking step occurs the gem command choked on the missing files. So we had to fix this by removing those entries from the list of files to be packed into the gem.


# eliminate the encoding-related entries in the gemspec
sed 's/\,\ \"spec\/qpid\/encoding_spec.rb\"//;s/\,\ \"lib\/qpid\/encoding.rb\"//' %{gemname}.gemspec > %{gemname}.gemspec-1
cp -f %{gemname}.gemspec-1 %{gemname}.gemspec


gem build %{gemname}.gemspec
cp -f %{gemname}-%{version}.gem %{SOURCE0}
popd
popd

The first step uses sed in order to remove references to the two files that were removed while copying the gemspec to a backup file, and then overwrite the original gemspec with the newly updated one.

The next step rebuilds the gem and then copies it back over the original sources.

BE VERY CAREFUL HERE!

If you're building the RPM locally this will REPLACE your source gem with the modified version. So you'll want to replace the source gem after each local rpmbuild session or else you'll hit complaints from patch about trying to apply changes that apparently have already been applied.

To avoid overwriting the original SOURCE0, use the following line to install the gem:


gem install --local --install-dir .%{gemdir} \
            -V \
            --force ../%{gemname}-%{version}/%{gemname}-%{version}.gem


At this point the gem installation process proceeds as normal. The patched sources are now ready to be installed when the RPM is installed.

No comments:

Post a Comment