has_aux_table/design_document.md

ActiveRecord Auxiliary Table Design Document

Overview

The ActiveRecord Auxiliary Table gem extends ActiveRecord's Single Table Inheritance (STI) capabilities by allowing developers to define auxiliary tables that are automatically joined to subclasses. This solves the fundamental RDBMS limitation where subclasses cannot have additional columns with their own indexes beyond what's defined in the parent table.

Problem Statement

Current STI Limitations

1. Schema Bloat: All subclass columns must exist in the parent table, leading to sparse tables with many NULL values
2. Index Inefficiency: Cannot create subclass-specific indexes on columns that don't apply to other subclasses
3. Query Performance: Queries against subclass-specific columns are less efficient due to shared table structure
4. Maintenance Overhead: Adding new subclass columns requires altering the parent table, affecting all subclasses

Example Problem

# Current STI approach - all columns in one table
class Vehicle < ActiveRecord::Base
  # Common columns: id, type, name, created_at, updated_at
  # Car-specific: engine_size, fuel_type, transmission
  # Boat-specific: hull_material, sail_area, draft
  # Plane-specific: wingspan, max_altitude, engine_count
end

class Car < Vehicle; end
class Boat < Vehicle; end
class Plane < Vehicle; end

This results in a sparse vehicles table with many NULL values and inability to create efficient indexes for subclass-specific queries.

Solution Architecture

Core Concept

The gem introduces auxiliary tables that are automatically joined to their corresponding STI subclasses. Each subclass can define its own auxiliary table with subclass-specific columns and indexes.

Database Schema

-- Parent STI table (minimal, common columns only)
CREATE TABLE vehicles (
  id BIGINT PRIMARY KEY,
  type VARCHAR(255) NOT NULL,
  name VARCHAR(255),
  created_at TIMESTAMP,
  updated_at TIMESTAMP
);

-- Auxiliary tables for each subclass
CREATE TABLE car_aux (
  vehicle_id BIGINT PRIMARY KEY REFERENCES vehicles(id),
  engine_size DECIMAL(3,1),
  fuel_type VARCHAR(50),
  transmission VARCHAR(50),
  created_at TIMESTAMP,
  updated_at TIMESTAMP
);

CREATE TABLE boat_aux (
  vehicle_id BIGINT PRIMARY KEY REFERENCES vehicles(id),
  hull_material VARCHAR(100),
  sail_area DECIMAL(6,2),
  draft DECIMAL(4,2),
  created_at TIMESTAMP,
  updated_at TIMESTAMP
);

CREATE TABLE plane_aux (
  vehicle_id BIGINT PRIMARY KEY REFERENCES vehicles(id),
  wingspan DECIMAL(6,2),
  max_altitude INTEGER,
  engine_count INTEGER,
  created_at TIMESTAMP,
  updated_at TIMESTAMP
);

-- Subclass-specific indexes
CREATE INDEX idx_car_aux_engine_size ON car_aux(engine_size);
CREATE INDEX idx_boat_aux_hull_material ON boat_aux(hull_material);
CREATE INDEX idx_plane_aux_wingspan ON plane_aux(wingspan);

API Design

Basic Configuration

class Vehicle < ActiveRecord::Base
  # Enable auxiliary table support
  include ActiveRecord::AuxTable
end

class Car < Vehicle
  # Define auxiliary table
  aux_table :car_aux do |t|
    t.decimal :engine_size, precision: 3, scale: 1
    t.string :fuel_type, limit: 50
    t.string :transmission, limit: 50
    t.timestamps
  end

  # Validations work on auxiliary columns
  validates :engine_size, presence: true, numericality: { greater_than: 0 }
  validates :fuel_type, inclusion: { in: %w[gasoline diesel hybrid electric] }
end

class Boat < Vehicle
  aux_table :boat_aux do |t|
    t.string :hull_material, limit: 100
    t.decimal :sail_area, precision: 6, scale: 2
    t.decimal :draft, precision: 4, scale: 2
    t.timestamps
  end

  validates :hull_material, presence: true
end

class Plane < Vehicle
  aux_table :plane_aux do |t|
    t.decimal :wingspan, precision: 6, scale: 2
    t.integer :max_altitude
    t.integer :engine_count
    t.timestamps
  end

  validates :wingspan, presence: true
  validates :engine_count, numericality: { greater_than: 0 }
end

Usage Examples

# Create with auxiliary attributes
car = Car.create!(
  name: "Toyota Camry",
  engine_size: 2.5,
  fuel_type: "gasoline",
  transmission: "automatic"
)

# Read auxiliary attributes transparently
puts car.engine_size  # => 2.5
puts car.fuel_type    # => "gasoline"

# Update auxiliary attributes
car.update!(engine_size: 3.0)

# Query with auxiliary attributes
fast_cars = Car.where(engine_size: 3.0..8.0)
hybrid_cars = Car.where(fuel_type: "hybrid")

# Joins are handled automatically
Car.joins(:aux_table).where(car_aux: { fuel_type: "electric" })

Advanced Features

class Car < Vehicle
  aux_table :car_aux do |t|
    t.decimal :engine_size, precision: 3, scale: 1
    t.string :fuel_type, limit: 50
    t.string :transmission, limit: 50
    t.timestamps

    # Define custom indexes
    t.index :engine_size
    t.index [:fuel_type, :transmission]
  end

  # Callbacks work with auxiliary attributes
  before_save :normalize_fuel_type

  # Scopes using auxiliary attributes
  scope :electric, -> { where(fuel_type: "electric") }
  scope :by_engine_size, ->(size) { where(engine_size: size) }

  private

  def normalize_fuel_type
    self.fuel_type = fuel_type.downcase if fuel_type
  end
end

Technical Implementation

Core Components

1. AuxTable Module: Main module that extends ActiveRecord::Base
2. AuxTableAssociation: Manages the relationship between parent and auxiliary tables
3. AuxTableProxy: Transparent proxy for auxiliary attribute access
4. AuxTableMigration: Migration helpers for creating auxiliary tables
5. AuxTableQuery: Query builder extensions for auxiliary table joins

Key Implementation Details

1. Automatic Association Setup

module ActiveRecord
  module AuxTable
    extend ActiveSupport::Concern

    class_methods do
      def aux_table(table_name, &block)
        # Create auxiliary model class
        aux_model_class = create_aux_model_class(table_name)

        # Set up associations
        has_one :aux_record, class_name: aux_model_class.name,
                foreign_key: "#{base_class.name.underscore}_id"

        # Define attribute accessors
        setup_aux_attribute_accessors(aux_model_class)

        # Set up query extensions
        extend_query_methods(aux_model_class)
      end
    end
  end
end

2. Transparent Attribute Access

def setup_aux_attribute_accessors(aux_model_class)
  aux_model_class.column_names.each do |column_name|
    next if %w[id created_at updated_at].include?(column_name)
    next if column_name.ends_with?("_id")

    define_method(column_name) do
      aux_record&.send(column_name)
    end

    define_method("#{column_name}=") do |value|
      build_aux_record unless aux_record
      aux_record.send("#{column_name}=", value)
    end
  end
end

3. Query Integration

def extend_query_methods(aux_model_class)
  # Override where to handle aux table columns
  singleton_class.prepend(Module.new do
    def where(*args)
      if args.first.is_a?(Hash) && contains_aux_columns?(args.first)
        super.joins(:aux_record).where(aux_record: args.first)
      else
        super
      end
    end
  end)
end

4. Migration Support

class AuxTableMigration < ActiveRecord::Migration[7.0]
  def self.create_aux_table(table_name, parent_class, &block)
    create_table table_name do |t|
      t.references parent_class.name.underscore, null: false,
                   foreign_key: { to_table: parent_class.table_name }

      yield t if block_given?

      t.timestamps
    end
  end
end

Transaction and Callback Handling

class AuxTableCallbacks
  def after_create(record)
    if record.aux_record && record.aux_record.changed?
      record.aux_record.save!
    end
  end

  def after_update(record)
    if record.aux_record && record.aux_record.changed?
      record.aux_record.save!
    end
  end

  def after_destroy(record)
    record.aux_record&.destroy!
  end
end

Performance Considerations

Automatic Joins

• Joins are only added when auxiliary columns are referenced in queries
• Lazy loading of auxiliary records to avoid N+1 queries
• Eager loading support: Car.includes(:aux_record)

Indexing Strategy

# Recommended indexes for auxiliary tables
class CreateCarAux < ActiveRecord::Migration[7.0]
  def change
    AuxTableMigration.create_aux_table :car_aux, Vehicle do |t|
      t.decimal :engine_size, precision: 3, scale: 1
      t.string :fuel_type, limit: 50
      t.string :transmission, limit: 50

      # Performance indexes
      t.index :engine_size
      t.index :fuel_type
      t.index [:fuel_type, :transmission]
    end
  end
end

Caching Strategy

• Auxiliary attributes are cached with the main record
• Cache invalidation on auxiliary record changes
• Support for Rails' built-in caching mechanisms

Migration Path

From Existing STI Tables

class SplitVehicleAuxiliaryData < ActiveRecord::Migration[7.0]
  def up
    # Create auxiliary tables
    AuxTableMigration.create_aux_table :car_aux, Vehicle do |t|
      t.decimal :engine_size, precision: 3, scale: 1
      t.string :fuel_type, limit: 50
      t.string :transmission, limit: 50
    end

    # Migrate existing data
    Vehicle.where(type: 'Car').find_each do |car|
      CarAux.create!(
        vehicle_id: car.id,
        engine_size: car.read_attribute(:engine_size),
        fuel_type: car.read_attribute(:fuel_type),
        transmission: car.read_attribute(:transmission)
      )
    end

    # Remove columns from main table
    remove_column :vehicles, :engine_size
    remove_column :vehicles, :fuel_type
    remove_column :vehicles, :transmission
  end

  def down
    # Reverse migration logic
  end
end

Testing Strategy

Unit Tests

RSpec.describe Car do
  describe "auxiliary table integration" do
    it "creates auxiliary record on create" do
      car = Car.create!(name: "Test Car", engine_size: 2.0)
      expect(car.aux_record).to be_present
      expect(car.aux_record.engine_size).to eq(2.0)
    end

    it "updates auxiliary attributes" do
      car = Car.create!(name: "Test Car", engine_size: 2.0)
      car.update!(engine_size: 3.0)
      expect(car.reload.engine_size).to eq(3.0)
    end

    it "queries auxiliary attributes" do
      car = Car.create!(name: "Test Car", engine_size: 2.0)
      results = Car.where(engine_size: 2.0)
      expect(results).to include(car)
    end
  end
end

Integration Tests

RSpec.describe "AuxTable Integration" do
  it "works with complex queries" do
    car1 = Car.create!(name: "Prius", fuel_type: "hybrid", engine_size: 1.8)
    car2 = Car.create!(name: "Camry", fuel_type: "gasoline", engine_size: 2.5)

    hybrids = Car.where(fuel_type: "hybrid")
    expect(hybrids).to eq([car1])

    small_engines = Car.where(engine_size: ..2.0)
    expect(small_engines).to eq([car1])
  end
end

Future Enhancements

Phase 2 Features

1. Polymorphic Auxiliary Tables: Support for auxiliary tables shared across multiple STI classes
2. Nested Auxiliary Tables: Support for auxiliary tables that reference other auxiliary tables
3. Dynamic Schema: Runtime addition of auxiliary columns without migrations
4. Composite Keys: Support for composite primary keys in auxiliary tables

Phase 3 Features

1. Sharding Support: Distribute auxiliary tables across multiple databases
2. Read Replicas: Route auxiliary table reads to replica databases
3. Materialized Views: Create materialized views combining main and auxiliary data
4. GraphQL Integration: Automatic GraphQL schema generation for auxiliary tables

Conclusion

The ActiveRecord Auxiliary Table gem provides a clean, performant solution to STI's limitations while maintaining ActiveRecord's familiar API. By separating subclass-specific data into auxiliary tables, it enables better schema design, improved query performance, and more maintainable code.

The implementation prioritizes developer experience through transparent attribute access, automatic query handling, and seamless integration with existing ActiveRecord features like validations, callbacks, and associations.