We hope you’ve got your stroopwafels warmed on top of your coffee because today we’re gluing things up with sticky stroop (the syrup that makes the two halves of a stroopwafel stick together). In the first two parts of our series, we baked a Lexer and a Parser and now, we’re adding the Interpreter and gluing things together by pouring stroop over them.

Ingredients

Alright! Let’s get the kitchen ready for baking and put our ingredients on the table. Our interpreter needs two ingredients or pieces of information to do its job: the previously generated Abstract Syntax Tree (AST) and the data we want to embed into the template. We’ll call this data the environment.

To traverse the AST, we’ll implement the interpreter using the visitor pattern. A visitor (and therefore our interpreter) implements a generic visit method that accepts a node as a parameter, processes this node and potentially calls the visit method again with some (or all) of the node’s children, depending on what makes sense for the current node.

1module Magicbars
2  class Interpreter
3    attr_reader :root, :environment
4
5    def self.render(root, environment = {})
6      new(root, environment).render
7    end
8
9    def initialize(root, environment = {})
10      @root = root
11      @environment = environment
12    end
13
14    def render
15      visit(root)
16    end
17
18    def visit(node)
19      # Process node
20    end
21  end
22end

Before continuing, let’s also create a small Magicbars.render method that accepts a template and an environment and outputs the rendered template.

1module Magicbars
2  def self.render(template, environment = {})
3    tokens = Lexer.tokenize(template)
4    ast = Parser.parse(tokens)
5    Interpreter.render(ast, environment)
6  end
7end

With this in place, we’ll be able to test the interpreter without having to construct the AST by hand.

1Magicbars.render('Welcome to {{name}}', name: 'Ruby Magic')
2# => nil

To no surprise, it currently doesn’t return anything. So let’s start implementing the visit method. As a quick reminder, here’s what the AST for this template looks like.

For this template, we’ll have to process four different node types: Template, Content, Expression, and Identifier. To do this, we could just put a huge case statement inside our visit method. However, this will become unreadable pretty quickly. Instead, let’s make use of Ruby’s metaprogramming capabilities to keep our code a bit more organized and readable.

1module Magicbars
2  class Interpreter
3    # ...
4
5    def visit(node)
6      short_name = node.class.to_s.split('::').last
7      send("visit_#{short_name}", node)
8    end
9  end
10end

The method accepts a node, gets its class name, and removes any modules from it (check out our article on cleaning strings if you’re interested in different ways of doing this). Afterward, we use send to call a method that handles this specific type of node. The method name for each type is made up of the demodulized class name and the visit_ prefix. It’s a bit unusual to have capital letters in method names, but it makes the method’s intent pretty clear.

1module Magicbars
2  class Interpreter
3    # ...
4
5    def visit_Template(node)
6      # Process template nodes
7    end
8
9    def visit_Content(node)
10      # Process content nodes
11    end
12
13    def visit_Expression(node)
14      # Process expression nodes
15    end
16
17    def visit_Identifier(node)
18      # Process identifier nodes
19    end
20  end
21end

Let’s start by implementing the visit_Template method. It should just process all the statements of the node and join the results.

1def visit_Template(node)
2  node.statements.map { |statement| visit(statement) }.join
3end

Next, let’s look at the visit_Content method. As a content node just wraps a string, the method is as simple as it gets.

1def visit_Content(node)
2  node.content
3end

Now, let’s move on to the visit_Expression method where the substitution of the placeholder with the real value happens.

1def visit_Expression(node)
2  key = visit(node.identifier)
3  environment.fetch(key, '')
4end

And finally, for the visit_Expression method to know what key to fetch from the environment, let’s implement the visit_Identifier method.

1def visit_Identifier(node)
2  node.value
3end

With these four methods in place, we get the desired result when we try to render the template again.

1Magicbars.render('Welcome to {{name}}', name: 'Ruby Magic')
2# => Welcome to Ruby Magic

Interpreting Block Expressions

We wrote a lot of code to implement what a simple gsub could do. So let’s move on to a more complex example.

1Welcome to {{name}}!
2
3{{#if subscribed}}
4  Thank you for subscribing to our mailing list.
5{{else}}
6  Please sign up for our mailing list to be notified about new articles!
7{{/if}}
8
9Your friends at {{company_name}}

As a reminder, here’s what the corresponding AST looks like.

There’s only one node type that we don’t yet handle. It’s the visit_BlockExpression node. In a way, it is similar to the visit_Expression node, but depending on the value it either continues to process the statements or the inverse_statements of the BlockExpression node.

1def visit_BlockExpression(node)
2  key = visit(node.identifier)
3
4  if environment[key]
5    node.statements.map { |statement| visit(statement) }.join
6  else
7    node.inverse_statements.map { |statement| visit(statement) }.join
8  end
9end

Looking at the method, we notice that the two branches are very similar, and they also look similar to the visit_Template method. They all handle the visiting of all nodes of an Array, so let’s extract a visit_Array method to clean things up a bit.

1def visit_Array(nodes)
2  nodes.map { |node| visit(node) }
3end

With the new method in place, we can remove some code from the visit_Template and visit_BlockExpression methods.

1def visit_Template(node)
2  visit(node.statements).join
3end
4
5def visit_BlockExpression(node)
6  key = visit(node.identifier)
7
8  if environment[key]
9    visit(node.statements).join
10  else
11    visit(node.inverse_statements).join
12  end
13end

Now that our interpreter handles all node types, let’s try and render the complex template.

1Magicbars.render(template, { name: 'Ruby Magic', subscribed: true, company_name: 'AppSignal' })
2# => Welcome to Ruby Magic!
3#
4#
5#  Please sign up for our mailing list to be notified about new articles!
6#
7#
8# Your friends at AppSignal

That almost looks right. But on a closer look, we notice that the message is prompting us to sign up for the mailing list, even though we provided subscribed: true in the environment. That doesn’t seem right…

Adding Support for Helper Methods

Looking back at the template, we notice that there’s an if in the block expression. Instead of looking up the value of subscribed in the environment, the visit_BlockExpression is looking up the value of if. As it is not present in the environment, the call returns nil, which is false.

We could stop here and declare that we’re not trying to imitate Handlebars but Mustache, and get rid of the if in the template, which will give us the desired result.

1Welcome to {{name}}!
2
3{{#subscribed}}
4  Thank you for subscribing to our mailing list.
5{{else}}
6  Please sign up for our mailing list to be notified about new articles!
7{{/subscribed}}
8
9Your friends at {{company_name}}

But why stop when we are having fun? Let’s go the extra mile and implement helper methods. They might come in handy for other things as well.

Let’s start by adding a helper method support to simple expressions. We’ll add a reverse helper, that reverses strings passed to it. In addition, we’ll add a debug method that tells us the class name of a given value.

1def helpers
2  @helpers ||= {
3    reverse: ->(value) { value.to_s.reverse },
4    debug: ->(value) { value.class }
5  }
6end

We use simple lambdas to implement these helpers and store them in a hash so that we can look them up by their name.

Next, let’s modify visit_Expression to perform a helper lookup before trying a value lookup in the environment.

1def visit_Expression(node)
2  key = visit(node.identifier)
3
4  if helper = helpers[key]
5    arguments = visit(node.arguments).map { |k| environment[k] }
6
7    return helper.call(*arguments)
8  end
9
10  environment[key]
11end

If there is a helper matching the given identifier, the method will visit all the arguments and try to lookup values for them. Afterward, it will call the method and pass all the values as arguments.

1Magicbars.render('Welcome to {{reverse name}}', name: 'Ruby Magic')
2# => Welcome to cigaM ybuR
3
4Magicbars.render('Welcome to {{debug name}}', name: 'Ruby Magic')
5# => Welcome to String

With that in place, let’s finally implement an if and an unless helper. In addition to the arguments, we’ll pass two lambdas to them so that they can decide if we should continue interpreting the node’s statements or inverse_statements.

1def helpers
2  @helpers ||= {
3    if: ->(value, block:, inverse_block:) { value ? block.call : inverse_block.call },
4    unless: ->(value, block:, inverse_block:) { value ? inverse_block.call : block.call },
5    # ...
6  }
7end
8

The necessary changes to visit_BlockExpression are similar to what we did with visit_Expression, only this time, we also pass the two lambdas.

1def visit_BlockExpression(node)
2  key = visit(node.identifier)
3
4  if helper = helpers[key]
5    arguments = visit(node.arguments).map { |k| environment[k] }
6
7    return helper.call(
8      *arguments,
9      block: -> { visit(node.statements).join },
10      inverse_block: -> { visit(node.inverse_statements).join }
11    )
12  end
13
14  if environment[key]
15    visit(node.statements).join
16  else
17    visit(node.inverse_statements).join
18  end
19end

And with this, our baking is done! We can render the complex template that started this journey into the world of lexers, parsers, and interpreters.

1Magicbars.render(template, { name: 'Ruby Magic', subscribed: true, company_name: 'AppSignal' })
2# => Welcome to Ruby Magic!
3#
4#
5#  Thank you for subscribing to our mailing list.
6#
7#
8# Your friends at AppSignal

Only Scratching the Surface

In this three-part series, we covered the basics of creating a templating language. These concepts can also be used to create interpreted programming languages (like Ruby). Admittedly, we glossed over a couple of things (like proper error handling 🙀) and only scratched the surface of the underpinnings of today’s programming languages.

We hope you enjoyed the series and if you want more of that, subsribe to the Ruby Magic list. If you are now hungry for stroopwafels, drop us a line and we might be able to fuel you with those as well!