The Evolution of Angry Building Export Module

Hey 👋

Angry Building (the company I co-founded) helps manage a facility's finances and documents. This involves having the ability to export data out of the system in various formats. Our Export module handles those exports.

This module's structure is a good illustration of several useful patterns and techniques for modern web applications.

In this post, I will follow the timeline of the decision and the refactorings I did toward the current version of the Export module.

Angry Building uses Ruby on Rails, so all examples are in Ruby.

If you are not using Ruby on Rails, the techniques described in this post generally apply - see the "Outside of Ruby on Rails" section of this post

The Setup

My initial set of export-related features were:

• Generate a PDF of an invoice to be shared with the tenant.

• Export CSV of generated invoices to be imported into a 3rd party. Accounting software name Plus Minus (Used by our first major client).

• Generate an Excel file of unpaid taxes of a tenant; this is used as evidence for legal cases against debtors.

• Download a zip archive of all files attached to the building.

• ...and a lot more similar like this.

As with every rapidly developed product, those weren’t grouped as an single export feature. They came as sub-tasks of other features, I was working on.

Step 1 - Service object

The first pattern I used is what we call a "Service Object" in the Ruby world. Another name for this pattern is a "Transaction Script". This is the term, I prefer because "Service Object" is often mistaken for microservice 🤷‍♂️

"Service objects" are Plain Old Ruby Objects (PORO) designed to execute one single action in your domain logic. They help with code organization and make the code easier to test. 

• I name those service objects as verbs, because they represent actions.

  • I’m fine with long and descriptive names because they are easy to search across the system.

  • Examples: GeneratePaymentInvoice or ChangePaymentAmount.

• Most of my service objects are modules with extend self and a method named call, which is the entry point of the action. 

  • When the call method gets too big, I extract the private methods to increase its readability.

• The service objects follow a procedural coding style.

  • It is fine for me the implementation of a service object to be a bit messy, as long as is covered by tests and input / output is clear.

Here is an example of the service object the "Generate an Excel file of unpaid taxes" feature:

module ExportUnpaidTaxesToExcel
  extend self

  def call(taxes)
    # ... logic
    
    excel_file
  end 

  private

  # ... helper methods
end

Because they have simple input, clear output and obvious path of execution. Service object are easy to test and read.

In some cases, a module will have many shared arguments that must be passed around its private methods. In those cases, I use a class masquerading as an extend self module.

class GenerateInvoiceDocumentPdf
  # This is what the consumers of this service are going to use.
  # Everything below this is an "private" implementation detail.
  def self.call(document)
    new(document).call
  end

  attr_reader :document, :pdf

  def intialize(document)
    @document = document
    @pdf = Prawn::Document.new(page_size: 'LETTER')
  end
  
  def call 
    watermark
    print_header
    print_company_and_customer
    print_line_items
    print_payment_info
    print_powered_by
   
    pdf
  end

  private

  # ... helper methods
end

In this example, I use a class to document and pdf attributes in every helper method without passing them around.

Sometimes, I use class to memorize some of its methods.

I use this style cautiously because it can lead to having too many instance variables and short private methods. Then, you have to do a lot of reading back and forth to follow the logic.

Here is an example:

def call
  if some_condition?
    do_something
  else
    do_some_other_thing
  end
end


def some_condition?
  some_other_condition? || some_third_condition?
end

def some_other_condition?
  @record1.some_other_condition?
end

def some_third_condition?
  @record2.some_third_condition?
end

I call this the "pretend to be an interpreter" game. To understand a code like this, you must interpret the code and keep a stack in your head. 😵

The key here is that both extend self module and class behave in the same way for the outside user of the code. 

I have had many cases where a single service object has switched styles multiple times during development, often without needing to change their tests.

Step 2 - Namespacing

One of the strategies that help me manage a large Ruby on Rails codebase is to use modules as namespaces very widely. I use namespace for every domain and group of utilities. I often move objects/namespaces around as the system evolves.

Originally, I was putting the export service objects inside of the namespace of the domain they were related to. Following the Proximity Principle and my interpretation - "code that is related should be closer together".

• PDF of the invoice, to be shared with tenant → Invoicing::GeneratePdf

• CSV export to PlusMinus → Invoicing::ExportToPlusMinus

• Excel file of unpaid taxes of a tenant → Taxation::DebtorsLegalExport

• Download a zip archive of all files attached to building → Buildings::DownloadNotesZip

This worked for a while. However, I had mistaken what is "related" in this case. An exporter is more related to other exporters compared to what it is exporting. 😵‍💫

All exporters should be grouped in the Export namespace.

In Domain Drive Design terms, the Export module is a "Supporting sub-domain".

How did I get to this conclusion?

When creating a new exporter, I almost always copy an existing exporter and adjust it to fit the new feature. I was wasting time searching for exporters to copy. 🙈

This will get even harder when I start to hire developers to work with me at Angry Building.

An even bigger issue with this structure was that spotting code for extraction and refactoring was hard because objects were spread across multiple directories and not grouped.

All PDF exporters following the same structure was not an accident of me copying/pasting. It was because the logic for generating PDFs is similar.

It is the same for Zip, Excel, CSV, etc. But I couldn't see this when the code was spread around.

I decided to create the domain namespace module named Export (very clever name, I know). I moved all service objects to the new namespace:

• Export::InvoiceToPdf

• Export::InvoicesToPlusMinusCsv

• Export::DebtorsLegalExportExcel

• Export::NotesToZip

I noticed how messy the naming was when I saw all service objects in the same directory. When I name stuff, I often try to follow predictable patterns like

[namespace][object][action] or [namespace][thing][type] or etc. This creates symmetry when you list files in a directory or have an auto-complete suggestion in your editor.

In this case I decided to go with [namespace]::[export type]::[what is exported] :

• Export::Pdf::Invoice

• Export::Csv::PlusMinus

• Export::Excel::Debtors

• Export::Zip::Notes

Having those extra modules for Pdf / Csv / Excel / Zip modules gives me the following benefits:

1. It was easy to go through all related exporters and notice opportunities for refactoring

2. I had an obvious place to put shared utilities between each exporter

Example: I noticed that all PDFs were using the  A4 paper format, so I created an Export::Pdf.a4 which returns a Prawn::Document (the library that I use for PDFs)  instance with A4 size.

Step 3 - Value object

Next, I noticed that I often copied code in my controllers that sent the files to the user:

file = Export::Csv::PlusMinus.call(invoices)

send_data(
  file, 
  type: 'text/csv', 
  disposition: 'attachment', 
  filename: 'plus_minus.csv'
)

This pushed me to exact a value object named Export::File.

class Export::File
  attr_reader :filename, :type, :content, :disposition

  class << self
    # I was considering having classes for each file type.
    # Then I decided that having couple of factory methods is cleaner.
    def csv(name, csv_content)
      new("#{name}.csv", 'text/csv', csv_content)
    end

    # ... similar method for PDF, Excel, Zip and etc
  end

  def initialize(filename:, type:, content:, disposition: :attachment)
    @filename = filename
    @type = type
    @content = content
    @disposition = disposition
  end
end

Every exporter service object now returns this value object:

module Export::Csv::PlusMinus
  extend self

  def call(documents)
    # ... logic
    
    Export::File.csv('plus_minus', content)
  end 
end

The controller code became:

file = Export::Csv::PlusMinus.call(invoices)

send_data(
  file, 
  type: file.content_type, 
  disposition: file.disposition, 
  filename: file.filename
)

This push me to extract a simple controller helper method:

file = Export::Csv::PlusMinus.call(invoices)
download_export(file)

# ... in ApplicationController
def download_export(export_file)
  send_data(
    file, 
    type: file.content_type, 
    disposition: file.disposition, 
    filename: file.filename
  )
end

The value object can also be integrated with Active Storage:

file = Export::Pdf::Invoice.call(document)
document.file.attach(file)

Integration

Rails has this format feature, which allows a controller action to respond differently depending on the requested file format.  

I use this to the full extent. Here is what my invoices controller looks like:

class InvoicingDocumentsController < ApplicationController
  def index
    # Find an account that current user has access to.
    # I'll write about this authorized strategy in a future post.
    account = find_record Account, params[:account_id], authorize: :view

    # Search for documents based on search params
    # I'll write about SearchObject in a future post.
    @search = Invocing::DocumentsSearch.new(account, params)

    # URL: /account/[account_id]/invocing_documents.[format]
    # Depending on the URL's file format return HTML/ZIP/XLS/PDF/CSV.
    respond_to do |format|
      format.html
      format.zip do
        download_export(Export::Zip::Invoices.call(@search.all))
      end
      format.xls do
        download_export(Export::Excel::Invoices.call(@search.all))
      end
      format.pdf do
        download_export(Export::Pdf::InvoicesCombined.call(@search.all))
      end
      format.csv do
        download_export(Export::Csv::PlusMinus.call(@search.all))
      end
    end
  end
end

In the UI, I have a filter form for invoices, and when the user has filtered down to the invoices they want to export.

The ViewComponent for the menu, just".format" to the URL and return what was selected.

<%= render ActionМenuComponent.new(title: :download) do |menu| %>
  <% menu.action :button_zip, url(@search.account, format: :zip, **@search.params) %>
  <% menu.action :button_plus_minus, account_invoicing_documents_path(@search.account, format: :csv, **@search.params) %>
  <% menu.action :button_excel, account_invoicing_documents_path(@search.account, format: :xls, **@search.params) %>
  <% menu.action :button_pdf, account_invoicing_documents_path(@search.account, format: :pdf, **@search.params) %>
<% end %>

Future expansion

Work is never done.

A couple of features in the Angry Building roadmap will involve moving the file exporters to be performed in a background job because exporting more data on demand in a web request is not a good idea. I have a couple of ideas on how to morph the structure to handle this case. I'll sure write a follow-up post when this happens.

I use libraries directly in the service object to generate PDF and Excel files. I might create wrappers around those and use the builder pattern to make the exporters themselves simpler. Will see when I get there.

Outside of Ruby on Rails

At first sight, what I have described so far looks very Ruby on Rails-specific. 🤨

However, I have also used all the techniques from above in Node.js projects. 🧐

Let's imagine I have used Node.js and TypeScript for AngryBuilding. What will the export system look like? 🤔

• I will structure the directories as I did with the namespaces.

• All exporters will be functions instead of service objects;

• The file value object will be TypeScript type, created via factory methods.

Here is what the directory will look like:

lib/export/index.ts
lib/export/file.ts - factories for value object
lib/export/pdf/utils.ts - all utils for PDF will be
lib/export/pdf/invoice.ts - an exporter
lib/export/excel/utils.ts - all utils for excel will be
lib/export/excel/invoices.ts - an exporter

Here is how the utilities going to look like:

// lib/export/pdf/utils

// Forward those so exporters have fewer imports
export { filePdf, IFile } from '../file'; 

export function a4() {
   // ...
}

// ...

The exporter will look like something like this:

// lib/export/pdf/invoices
import { a4, filePdf, IFile } from './utils';

// IFile is optional since TypeScript can infer
export default function(document): IFile { 
 
   const content = a4();

   // ...

   return filePdf(`document-${document}`, content);
}

The exporter can used as this:

import generateInvoice from '@/lib/export/pdf/invoice'

As you can see the structure looks very similar.

Most of the differences will be in the glue of the system - libraries used to generate the various formats and the code sending them to the user.

Conclusion 

The Export modules is a good illustration, how I think about software design and system evolution .

In this post, I have outlined some the essential patterns for a modern Ruby on Rails application:

• Service objects

• Namespaces

• Value objects

The patterns, shared can be used not only in Ruby on Rails projects.

p.s. I’m experimenting with companion videos for my posts. 

I hope you find them useful.🤞

If you have any questions or comments, you can ping me on Threads, LinkedIn, Mastodon, Twitter or just leave a comment below 📭