At the beginning of the internet age, websites were much simpler and not very interactive. With the advancement of technology, devices, and programming languages, they became more complex and consisted of several files, including assets like images and CSS stylesheets.

The more interactive your website is, the more JavaScript code you have to use. To use such code, you have to include all HTML code files using the script tag. Such an approach is error-prone because you have to remember to include every single file and keep the correct order, otherwise, your code may not work. With the Webpack tool, these problems go away. Webpacker is a bridge between Webpack and a Rails application.

This article takes a deep dive into Webpacker and offers a detailed explanation that will enable you to understand how this tool works under the hood. To make the content as valuable as possible, I decided to divide it into the following sections:

• High-level overview — before we do the deep dive, I will explain the high-level overview of both Webpack and Webpacker. You can treat it as the preparation for more advanced topics.
• Structure explanation — since we'll explore multiple files, it is good to know why the particular structure was used and how it responds to the general purpose.
• Anatomy explanation — this part covers the investigation of the most important files of the Webpack setup. Knowing how they work is essential for the effective and hassle-free development process.
• Environment specific setup — the last part explains how Webpacker works in development and production environments.

After reading this article, you'll know what Webpack and Webpacker are and why we need them in our Rails application. The deep dive into Webpacker’s internals will help you understand how the tool connects with Webpack and how it communicates with the Rails application.

Before Diving In

Before going out into deep water, we first need to prepare. A high-level overview is a great way to start working with any technology as it helps us understand the purpose of a given tool and the problems it solves without going deep into its internals.

Webpack

I already mentioned that Webpack helps us organize our JavaScript file code to avoid errors and improve a website’s performance. It organizes the code into bundles.

A bundle is a file where multiple modules are intelligently placed, respecting the dependency graph that is first created. Thanks to this process, we can be sure that our code will work as expected and before we invoke a given library’s code, it will have been loaded.

Webpack isn't the only tool that is used by default in the newest version of Rails. When the project is generated, the configuration files for Babel and PostCSS are created as well. PostCSS is a tool for transforming CSS with JavaScript, and Babel is a JavaScript compiler that enables us to write modern JavaScript without worrying about browser support.

Webpacker

Webpacker is a tool that integrates Webpack with a Rails application. It makes it easy to configure and develop JavaScript-like applications and optimize them for the production environment.

The source code is available as a gem, and the library comes with a development server that makes the development of the app very fast as you don’t have to stop and start the server to see the changes introduced in the JavaScript files.

The Deep Immersion

Now that you know why Webpack was created and what role Wepacker plays in a Rails application, we can focus on Webpacker’s internals to see how it organizes files, speeds up the development process, and optimizes files for the production environment.

Webpacker is available out-of-the-box in the newest version of Rails. Before creating a new project, ensure that the Node version is greater or equal to 10.17.0 as the webpacker:install command will be automatically invoked with rails new command.

Webpacker File Structure

After the install command is executed, the following files are created:

• config/webpacker.yml — the main configuration file that contains the default configuration and configs for specific environments
• config/webpacker/ — the directory where the JavaScript configuration files for particular environments are created
• bin/webpack — an executable file that invokes webpack to create bundles
• bin/webpack-dev-server — an executable file that starts the development server, which reloads webpack every time you make a change inside the JavaScript files included in the bundle

The Anatomy of the Configuration File

The main configuration file for Webpacker is located under the config directory, and it’s named webpacker.yml. Unlike Webpacker, configuration for Webpack is stored separately for each environment under the config/webpack directory.

Webpacker

By default, the configuration file contains many entries — default ones, and sections for each environment where you can overwrite specific settings. Let’s take a look at the most important settings:

• source_path — the primary source of javascript files in your application. It’s set to app/javascript by default, and usually, there's no need to change this value.
• source_entry_path — the name of the directory under the source_path where you keep the pack files, aka entry points. By default, this setting is set to the packs’ directory.
• public_root_path — path to the directory in your application that is accessible from a browser. In a typical Rails application, it’s a public directory
• public_output_path — when Webpacker compiles your files, it will put all compiled files in this directory under the public_root_path. By default, the directory is named packs, but you can call it whatever you want.
• webpack_compile_output — if the flag is set to true, then output messages are displayed when files are compiled. It’s useful when compilation fails, as you'll be aware of that and can take action.

It is worth mentioning that the development section also contains configuration for the dev server that is used to compile files in the development environment without the need for restarting the server.

If you would like to access the configuration from the rails console or code level, you can call Webpacker.manifest.config, which will return the configuration class instance.

Webpack

Each environment has its configuration file, but in every file, the main environment file is imported: config/webpack/environment.js. In the environment configuration file, there is a place for loading custom plugins that will modify the default behavior of Webpack. We can also add custom rules for compiling our files.

The Anatomy of the Pack File

Packs are located under the app/javascript/packs directory. Each pack file is treated by Webpacker as an entry point when the compilation process starts.

The Default Pack File

When you generate a new Rails project, a default pack file named application.js is created with the following content:

1import Rails from "@rails/ujs";
2import Turbolinks from "turbolinks";
3import * as ActiveStorage from "@rails/activestorage";
4import "channels";
5
6Rails.start();
7Turbolinks.start();
8ActiveStorage.start();

As you can see, the first step is to import the given library and then call its initialization method when it’s needed. When you call import, the system searches for a given node module or a local library. With the standard approach, you don’t have to explicitly initialize the library as it’s initialized when you include it in the page source with the script tag.

Good Practices for a Pack File

While you can put regular JavaScript code inside the pack file and execute it, it is not recommended to keep JavaScript code inside the pack file. The best approach is to keep the files clean and only import and initialize libraries here and keep other logic outside the packs directory.

Including Pack Files in the Application

Pack files are not automatically included in the website’s source. Just like in the asset pipeline case, we have to use a special tag inside our views:

1<%= javascript_packs_with_chunks_tag 'application' %>

How does this method work? Under the hood, it calls the Webpacker configuration that holds the data you put in the config/webpacker.yml file. It looks for an application.js file inside the entry points directory, which, in our case, is app/javascript/packs. You can verify it by calling the following code:

1Webpacker.manifest.config.source_entry_path

When entries are found, Webpacker calls javascript_include_tag method, a helper from the ActionView library available by default in Rails. The method accepts one or more sources and returns script tags that you can apply straight to your view.

The Development Server

Webpacker development server’s entry point is the executable file placed inside the bin folder and named webpack-dev-server. The process of running the server consists of five steps:

• Setting environment — the environment name for Node and Rails is set. When the environment is not specified, the program assumes that the server will be executed in the development environment.
• Loading configuration — the file config/webpacker.yml is loaded with settings for the environment set in the previous step.
• Validating command options — the program checks if we passed appropriate options to the server command. For example, it will throw an error when the --https option is given, but we didn’t specify it in the config/webpacker.yml file.
• Verifying port’s availability — the program checks if the given port is available to use. If another program is already using it, it will throw an error letting you know that you have to update the webpack server configuration inside the config/webpacker.yml file.
• Executing webpack serve command — when the node modules directory exists, the program executes the command within the directory, otherwise, it executes it with yarn. The program passes the configuration option to the command that points to one of the files placed inside the config/webpack/ directory.

The server is now running, and it compiles your code on the fly, so you don’t have to restart the server to see the changes and check if webpack was able to create the bundle with the new code.

The development server’s code is quite simple. It only loads the config from the configuration file in YAML format and passes the proper config in JavaScript format directly to the webpack serve command. You can also run it by hand, but you don’t have to take care of proper command arguments with the built-in command.

Compiling Files for the Production Environment

Suppose you want to deploy the application that's using webpacker. In that case, you can simply invoke the assets:precompile task as Webpacker automatically hooks up the task webpacker:compile to it.

How does the webpacker:compile task work under the hood? Let’s dig into it and see. It does the following things:

• It wraps the main call into two blocks ensuring that the NODE environment is set and Webpacker logs are output to the standard out.
• It calls Webpack and then parses its logs to determine if the action was successful or not.

You can now visit the public/packs/js directory to see the compiled files. They should be deployed to the server.

Summary

We have just learned how Webpacker works under the hood. You can treat it as a bridge between the Webpack library and a Rails application that allows you to configure Webpack with Ruby and easily use javascript code inside your application.

To summarize the whole article, let’s recall the main elements of Webpacker again:

• Configuration — it’s placed inside the config/webpacker.yml file and allows you to point Webpacker to the directory where you keep your javascript files and packs
• Pack — it’s a single entry point from which Webpacker begins compilation. Each library imported and initialized in that file will be automatically compiled.
• Development server — a simple script that allows you to compile files on the fly, so you don’t have to restart the server each time you modify JavaScript files
• Compile task — a simple rake task that invokes Webpack and compiles files under the public directory so you can deploy them to the production server and make them available for end-users
• View helper — helper that allows you to include compiled files inside the views

For many developers without much experience, Wepacker seems to be a little magical, but in fact, it’s a simple library that helps us use Webpack in a Ruby way.

P.S. If you'd like to read Ruby Magic posts as soon as they get off the press, subscribe to our Ruby Magic newsletter and never miss a single post!