Inside Gemfile - intense dive in

Gemfile is a well-known file even for not experienced Ruby developers. While for most people, it’s evident that the purpose of that file and the bundler library, fewer of them know how bundler works and what happens from the time you declare the gem in the Gemfile to the moment when it is installed in your system.

It is time for another deep dive into Ruby’s code internals to see how the Gemfile works with bundler. Before we start, ensure that you have Ruby and bundler installed in your system.

The code we will be working with

Create an empty directory and the Gemfile with the following contents:

source 'https://rubygems.org'

gem 'redis'

puts 'Hello from Gemfile!'

You can declare any gem you want; I used Redis because it came first to my mind. I also put some greeting in the code to show you that the Gemfile is just a Ruby file to use the ordinary Ruby code there.

Run the bundle install command, and you will notice that the greeting is rendered twice:

Hello from Gemfile!
Hello from Gemfile!
Using bundler 1.17.3
Using redis 4.2.5
Bundle complete! 1 Gemfile dependency, 2 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.

That’s interesting! It’s a perfect starting point for our investigation. During this article, we will be working mostly with the following two elements:

  • Gemfile - a file where you declare the gems
  • bundle install - a command that installs gems listed in the Gemfile

What happens when you hit bundle install

First of all, the bundler verifies if the library is installed along with the module for the command line processing. If everything is fine with that part, the following code is invoked from the exe/bundle executable file:

help_flags = %w[--help -h]
help_flag_used = ARGV.any? {|a| help_flags.include? a }
args = help_flag_used ? Bundler::CLI.reformatted_help_args(ARGV) : ARGV
Bundler::CLI.start(args, :debug => true)

If you would like to install gems from the console, you can run the following command:

require 'bundler'
require 'bundler/cli'

Bundler::CLI.start(['install'], debug: true)

The Thor’s hammer

To process the command line arguments with grace, Bundler is using the excellent Thor library. When you invoke the start method from the Bundler::CLI module, it invokes Thor’s start method.

The Thor verifies if the given command, in our case install, is handled by the library. The investigation showed that with the newest version of bundler, 32 commands are available to use! When the command is defined, the extra arguments are parsed, and the Invocation module invokes the command by calling the run method on the Bundler::Thor::Command instance.

Further investigation shows that the install method is invoked then on the Bundler::CLI class. We just returned to Bundler’s CLI class after Thor verified that the method can be called.

Bundler’s DSL

The class responsible for the gem’s installation is Bundler::CLI::Install with the run method that invokes the Bundler::Plugin.gemfile_install method.

If you would like to check from the console level where the Gemfile is located, you can call:

Bundler.default_gemfile

At this point, the Gemfile is finally parsed. The code inside the Gemfile uses a particular domain-specific language (DSL) that allows us to create gems configuration easily. If you are not familiarized with the DSL concept, the same thing is used in RSpec tests. DSL allows us to create highly readable and maintainable files.

We can parse our Gemfile from the console level as well:

builder = Bundler::Dsl.new
builder.eval_gemfile(Bundler.default_gemfile)
=> [<Bundler::Dependency type=:runtime name="redis" requirements=">= 0">]

We have just one dependency right now.

Installation

When the bundler confirms that we have any dependencies listed in our Gemfile, it begins the installation process. Depending on the configuration we would pass to the bundle install command, Bundler runs installation in parallel or single worker.

If you have ever build a ruby gem, you had to define the gem_name.gemspec file. It’s a configuration file for every gem that lists down the gem’s information like dependencies, authors, and other essential attributes. Bundler pulls that information from the source you defined in the Gemfile (in our case, it’s Rubygems.org). Thanks to that information, we can download and install the gem and its dependencies. Bundler will look for already installed gems in the first place before hitting the defined source’s API.

Things that you can do when knowing how bundler works

During the deep-dive, I also noticed that bundler has built-in support for hooks. You can use, for example, before-install-all or after-install-all hooks to perform given action before installing gems or after gems are installed.

Since Bundler is built with Thor, an excellent library for creating command-line applications, you can also easily extend it by providing your commands or extending existing ones. For more information, you can reach the official Bundler’s documentation that explains how you can build a plugin - https://bundler.io/guides/bundler_plugins.html.

There is a lot more than that

The knowledge of how the gems are installed with bundler is just a beginning. There is a lot more to explore about bundler. Also, many other developers wrote about managing the Gemfile or invoking less-known commands from Bundler.