has_aux_table/design_document.md

ActiveRecord Auxiliary Table Design Document

Overview

The ActiveRecord Auxiliary Table gem extends ActiveRecord's Single Table Inheritance (STI) capabilities by allowing developers to define auxiliary tables that are automatically joined to subclasses. This solves the fundamental RDBMS limitation where subclasses cannot have additional columns with their own indexes beyond what's defined in the parent table.

Problem Statement

Current STI Limitations

1. Schema Bloat: All subclass columns must exist in the parent table, leading to sparse tables with many NULL values
2. Index Inefficiency: Cannot create subclass-specific indexes on columns that don't apply to other subclasses
3. Query Performance: Queries against subclass-specific columns are less efficient due to shared table structure
4. Maintenance Overhead: Adding new subclass columns requires altering the parent table, affecting all subclasses

Example Problem

# Current STI approach - all columns in one table
class Vehicle < ActiveRecord::Base
  # Common columns: id, type, name, created_at, updated_at
  # Car-specific: engine_size, fuel_type, transmission
  # Boat-specific: hull_material, sail_area, draft
  # Plane-specific: wingspan, max_altitude, engine_count
end

class Car < Vehicle; end
class Boat < Vehicle; end
class Plane < Vehicle; end

This results in a sparse vehicles table with many NULL values and inability to create efficient indexes for subclass-specific queries.

Solution Architecture

Core Concept

The gem introduces auxiliary tables that are automatically joined to their corresponding STI subclasses. Each subclass can define its own auxiliary table with subclass-specific columns and indexes.

Database Schema

-- Parent STI table (minimal, common columns only)
CREATE TABLE vehicles (
  id BIGINT PRIMARY KEY,
  type VARCHAR(255) NOT NULL,
  name VARCHAR(255),
  created_at TIMESTAMP,
  updated_at TIMESTAMP
);

-- Auxiliary tables for each subclass
CREATE TABLE car_aux (
  vehicle_id BIGINT PRIMARY KEY REFERENCES vehicles(id),
  engine_size DECIMAL(3,1),
  fuel_type VARCHAR(50),
  transmission VARCHAR(50),
  created_at TIMESTAMP,
  updated_at TIMESTAMP
);

CREATE TABLE boat_aux (
  vehicle_id BIGINT PRIMARY KEY REFERENCES vehicles(id),
  hull_material VARCHAR(100),
  sail_area DECIMAL(6,2),
  draft DECIMAL(4,2),
  created_at TIMESTAMP,
  updated_at TIMESTAMP
);

CREATE TABLE plane_aux (
  vehicle_id BIGINT PRIMARY KEY REFERENCES vehicles(id),
  wingspan DECIMAL(6,2),
  max_altitude INTEGER,
  engine_count INTEGER,
  created_at TIMESTAMP,
  updated_at TIMESTAMP
);

-- Subclass-specific indexes
CREATE INDEX idx_car_aux_engine_size ON car_aux(engine_size);
CREATE INDEX idx_boat_aux_hull_material ON boat_aux(hull_material);
CREATE INDEX idx_plane_aux_wingspan ON plane_aux(wingspan);

API Design

Basic Configuration

class Vehicle < ActiveRecord::Base
  # Enable auxiliary table support
  include ActiveRecord::AuxTable
end

class Car < Vehicle
  # Define auxiliary table
  aux_table :car_aux do |t|
    t.decimal :engine_size, precision: 3, scale: 1
    t.string :fuel_type, limit: 50
    t.string :transmission, limit: 50
    t.timestamps
  end

  # Validations work on auxiliary columns
  validates :engine_size, presence: true, numericality: { greater_than: 0 }
  validates :fuel_type, inclusion: { in: %w[gasoline diesel hybrid electric] }
end

class Boat < Vehicle
  aux_table :boat_aux do |t|
    t.string :hull_material, limit: 100
    t.decimal :sail_area, precision: 6, scale: 2
    t.decimal :draft, precision: 4, scale: 2
    t.timestamps
  end

  validates :hull_material, presence: true
end

class Plane < Vehicle
  aux_table :plane_aux do |t|
    t.decimal :wingspan, precision: 6, scale: 2
    t.integer :max_altitude
    t.integer :engine_count
    t.timestamps
  end

  validates :wingspan, presence: true
  validates :engine_count, numericality: { greater_than: 0 }
end

Usage Examples

# Create with auxiliary attributes
car = Car.create!(
  name: "Toyota Camry",
  engine_size: 2.5,
  fuel_type: "gasoline",
  transmission: "automatic"
)

# Read auxiliary attributes transparently
puts car.engine_size  # => 2.5
puts car.fuel_type    # => "gasoline"

# Update auxiliary attributes
car.update!(engine_size: 3.0)

# Query with auxiliary attributes
fast_cars = Car.where(engine_size: 3.0..8.0)
hybrid_cars = Car.where(fuel_type: "hybrid")

# Joins are handled automatically
Car.joins(:aux_table).where(car_aux: { fuel_type: "electric" })

Advanced Features

class Car < Vehicle
  # Simple table reference - columns discovered from database schema
  aux_table :car_aux

  # Callbacks work with auxiliary attributes
  before_save :normalize_fuel_type

  # Scopes using auxiliary attributes
  scope :electric, -> { where(fuel_type: "electric") }
  scope :by_engine_size, ->(size) { where(engine_size: size) }

  private

  def normalize_fuel_type
    self.fuel_type = fuel_type.downcase if fuel_type
  end
end

Updated Design: Unconditional Auxiliary Table Presence

Core Design Principle

Auxiliary tables are unconditionally present for STI subclasses. This means:

1. Automatic Creation: When a model is created, its auxiliary table record is automatically created
2. Automatic Loading: When a model is loaded, its auxiliary attributes are automatically loaded and merged
3. Single Query Loading: When loading STI subclasses directly (e.g., Car.find(id)), auxiliary attributes are loaded via JOIN in a single SELECT query
4. Transparent Access: Auxiliary attributes appear as native model attributes

Database Schema (Updated)

-- Parent STI table (minimal, common columns only)
CREATE TABLE vehicles (
  id BIGINT PRIMARY KEY,
  type VARCHAR(255) NOT NULL,
  name VARCHAR(255),
  created_at TIMESTAMP,
  updated_at TIMESTAMP
);

-- Auxiliary tables for each subclass (ALWAYS present)
CREATE TABLE car_aux (
  vehicle_id BIGINT PRIMARY KEY REFERENCES vehicles(id) ON DELETE CASCADE,
  engine_size DECIMAL(3,1) NOT NULL,
  fuel_type VARCHAR(50) NOT NULL,
  transmission VARCHAR(50) NOT NULL,
  created_at TIMESTAMP,
  updated_at TIMESTAMP
);

CREATE TABLE boat_aux (
  vehicle_id BIGINT PRIMARY KEY REFERENCES vehicles(id) ON DELETE CASCADE,
  hull_material VARCHAR(100) NOT NULL,
  sail_area DECIMAL(6,2) NOT NULL,
  draft DECIMAL(4,2) NOT NULL,
  created_at TIMESTAMP,
  updated_at TIMESTAMP
);

CREATE TABLE plane_aux (
  vehicle_id BIGINT PRIMARY KEY REFERENCES vehicles(id) ON DELETE CASCADE,
  wingspan DECIMAL(6,2) NOT NULL,
  max_altitude INTEGER NOT NULL,
  engine_count INTEGER NOT NULL,
  created_at TIMESTAMP,
  updated_at TIMESTAMP
);

Updated API Design

class Vehicle < ActiveRecord::Base
  include ActiveRecord::AuxTable
end

class Car < Vehicle
  # Simple table reference - columns are discovered from database schema
  aux_table :car_aux

  # Validations work on auxiliary columns as if they were native
  validates :engine_size, presence: true, numericality: { greater_than: 0 }
  validates :fuel_type, inclusion: { in: %w[gasoline diesel hybrid electric] }
end

Updated Usage Examples

# Create with auxiliary attributes - auxiliary record created automatically
car = Car.create!(
  name: "Toyota Camry",
  engine_size: 2.5,
  fuel_type: "gasoline",
  transmission: "automatic"
)

# Auxiliary record is automatically created
expect(car.engine_size).to eq(2.5)
expect(car.fuel_type).to eq("gasoline")

# Loading via single query with JOIN
car = Car.find(id)  # Issues: SELECT vehicles.*, car_aux.* FROM vehicles
                     #         LEFT JOIN car_aux ON vehicles.id = car_aux.vehicle_id
                     #         WHERE vehicles.id = ? AND vehicles.type = 'Car'

# All auxiliary attributes are available as native attributes
puts car.engine_size  # => 2.5
puts car.fuel_type    # => "gasoline"

# Updates work transparently
car.update!(engine_size: 3.0)

# Queries work on auxiliary attributes directly
fast_cars = Car.where(engine_size: 3.0..8.0)
hybrid_cars = Car.where(fuel_type: "hybrid")

# No need for explicit joins - they're automatic
electric_cars = Car.where(fuel_type: "electric")

Technical Implementation (Updated)

Core Components

1. AuxTable Module: Main module that extends ActiveRecord::Base
2. AuxTableAssociation: Manages the unconditional relationship between parent and auxiliary tables
3. AuxTableAttributeProxy: Transparent proxy that merges auxiliary attributes into main model
4. AuxTableQueryBuilder: Extends query methods to automatically include auxiliary table joins
5. AuxTableMigration: Migration helpers for creating auxiliary tables with proper constraints

Key Implementation Details

1. Automatic Association Setup

module ActiveRecord
  module AuxTable
    extend ActiveSupport::Concern

    class_methods do
      def aux_table(table_name)
        # Create auxiliary model class that discovers columns from database
        aux_model_class = create_aux_model_class(table_name)

        # Set up unconditional has_one association
        has_one :aux_record, class_name: aux_model_class.name,
                foreign_key: "#{base_class.name.underscore}_id",
                dependent: :destroy,
                autosave: true

        # Define attribute accessors that proxy to auxiliary record
        setup_aux_attribute_accessors(aux_model_class)

        # Extend query methods to automatically include joins
        extend_query_methods(aux_model_class)

        # Set up automatic auxiliary record creation
        after_create :ensure_aux_record_exists
      end
    end
  end
end

2. Transparent Attribute Access

def setup_aux_attribute_accessors(aux_model_class)
  aux_model_class.column_names.each do |column_name|
    next if %w[id created_at updated_at].include?(column_name)
    next if column_name.ends_with?("_id")

    # Define getter that proxies to auxiliary record
    define_method(column_name) do
      aux_record&.send(column_name)
    end

    # Define setter that ensures auxiliary record exists
    define_method("#{column_name}=") do |value|
      ensure_aux_record_exists
      aux_record.send("#{column_name}=", value)
    end

    # Define presence check
    define_method("#{column_name}?") do
      aux_record&.send(column_name).present?
    end
  end
end

# Ensure auxiliary record exists (called after_create and when setting attributes)
def ensure_aux_record_exists
  return if aux_record.present?

  aux_table_config = self.class.aux_table_configurations.values.first
  aux_record_class = aux_table_config.model_class

  build_aux_record(
    aux_record_class.new(
      "#{self.class.base_class.name.underscore}_id" => id
    )
  )
end

3. Query Integration

def extend_query_methods(aux_model_class)
  # Override find methods to automatically include auxiliary table joins
  singleton_class.prepend(Module.new do
    def find(*args)
      if has_aux_tables?
        joins(:aux_record).find(*args)
      else
        super
      end
    end

    def find_by(*args)
      if has_aux_tables?
        joins(:aux_record).find_by(*args)
      else
        super
      end
    end

    def where(*args)
      if has_aux_tables? && args.first.is_a?(Hash) && contains_aux_columns?(args.first)
        # Split conditions between main table and auxiliary table
        main_conditions = {}
        aux_conditions = {}

        args.first.each do |key, value|
          if aux_model_class.column_names.include?(key.to_s)
            aux_conditions[key] = value
          else
            main_conditions[key] = value
          end
        end

        query = joins(:aux_record)
        query = query.where(main_conditions) if main_conditions.any?
        query = query.where(aux_record: aux_conditions) if aux_conditions.any?
        query
      else
        super
      end
    end
  end)
end

# Check if query contains auxiliary table columns
def contains_aux_columns?(conditions)
  return false unless has_aux_tables?

  aux_table_config = aux_table_configurations.values.first
  aux_columns = aux_table_config.model_class.column_names

  conditions.keys.any? { |key| aux_columns.include?(key.to_s) }
end

4. Migration Support

class AuxTableMigration < ActiveRecord::Migration[7.0]
  def self.create_aux_table(table_name, parent_class, &block)
    create_table table_name do |t|
      t.references parent_class.name.underscore, null: false,
                   foreign_key: { to_table: parent_class.table_name }

      yield t if block_given?

      t.timestamps
    end
  end
end

Transaction and Callback Handling

# Automatic auxiliary record creation
def after_create
  ensure_aux_record_exists
  aux_record.save! if aux_record.changed?
end

# Automatic auxiliary record updates
def after_update
  aux_record.save! if aux_record&.changed?
end

# Automatic auxiliary record cleanup
def after_destroy
  aux_record&.destroy!
end

Performance Benefits

1. Single Query Loading: Car.find(id) loads auxiliary attributes in one query via JOIN
2. No N+1 Queries: Auxiliary records are loaded with the main record
3. Efficient Updates: Changes to auxiliary attributes are batched with main record updates
4. Index Optimization: Each auxiliary table can have optimized indexes for its specific columns

Migration Considerations

The auxiliary tables should be created via standard Rails migrations before using the aux_table method:

class CreateCarAuxTable < ActiveRecord::Migration[7.0]
  def change
    create_table :car_aux do |t|
      t.references :vehicle, null: false, foreign_key: { to_table: :vehicles, on_delete: :cascade }
      t.decimal :engine_size, precision: 3, scale: 1, null: false
      t.string :fuel_type, limit: 50, null: false
      t.string :transmission, limit: 50, null: false
      t.timestamps
    end

    # Add indexes for performance
    add_index :car_aux, :engine_size
    add_index :car_aux, :fuel_type
    add_index :car_aux, [:fuel_type, :transmission]
  end
end

Then in your model:

class Car < Vehicle
  aux_table :car_aux  # Columns are automatically discovered from the database
end

Performance Considerations

Automatic Joins

• Joins are only added when auxiliary columns are referenced in queries
• Lazy loading of auxiliary records to avoid N+1 queries
• Eager loading support: Car.includes(:aux_record)

Indexing Strategy

# Recommended indexes for auxiliary tables
class CreateCarAux < ActiveRecord::Migration[7.0]
  def change
    create_table :car_aux do |t|
      t.references :vehicle, null: false, foreign_key: { to_table: :vehicles, on_delete: :cascade }
      t.decimal :engine_size, precision: 3, scale: 1, null: false
      t.string :fuel_type, limit: 50, null: false
      t.string :transmission, limit: 50, null: false
      t.timestamps
    end

    # Performance indexes
    add_index :car_aux, :engine_size
    add_index :car_aux, :fuel_type
    add_index :car_aux, [:fuel_type, :transmission]
  end
end

Caching Strategy

• Auxiliary attributes are cached with the main record
• Cache invalidation on auxiliary record changes
• Support for Rails' built-in caching mechanisms

Migration Path

From Existing STI Tables

class SplitVehicleAuxiliaryData < ActiveRecord::Migration[7.0]
  def up
    # Create auxiliary tables using standard Rails migrations
    create_table :car_aux do |t|
      t.references :vehicle, null: false, foreign_key: { to_table: :vehicles, on_delete: :cascade }
      t.decimal :engine_size, precision: 3, scale: 1, null: false
      t.string :fuel_type, limit: 50, null: false
      t.string :transmission, limit: 50, null: false
      t.timestamps
    end

    # Add performance indexes
    add_index :car_aux, :engine_size
    add_index :car_aux, :fuel_type

    # Migrate existing data
    Vehicle.where(type: 'Car').find_each do |car|
      CarAux.create!(
        vehicle_id: car.id,
        engine_size: car.read_attribute(:engine_size),
        fuel_type: car.read_attribute(:fuel_type),
        transmission: car.read_attribute(:transmission)
      )
    end

    # Remove columns from main table
    remove_column :vehicles, :engine_size
    remove_column :vehicles, :fuel_type
    remove_column :vehicles, :transmission
  end

  def down
    # Reverse migration logic
  end
end

Then update your model:

class Car < Vehicle
  aux_table :car_aux  # Columns are automatically discovered
end

Testing Strategy (Updated)

Unit Tests

RSpec.describe Car do
  describe "unconditional auxiliary table integration" do
    it "automatically creates auxiliary record on create" do
      car = Car.create!(name: "Test Car", engine_size: 2.0, fuel_type: "gasoline")
      expect(car.aux_record).to be_present
      expect(car.engine_size).to eq(2.0)
      expect(car.fuel_type).to eq("gasoline")
    end

    it "loads auxiliary attributes via single query" do
      car = Car.create!(name: "Test Car", engine_size: 2.0, fuel_type: "gasoline")

      # Should use single query with JOIN
      loaded_car = Car.find(car.id)
      expect(loaded_car.engine_size).to eq(2.0)
      expect(loaded_car.fuel_type).to eq("gasoline")
    end

    it "updates auxiliary attributes transparently" do
      car = Car.create!(name: "Test Car", engine_size: 2.0, fuel_type: "gasoline")
      car.update!(engine_size: 3.0, fuel_type: "diesel")
      expect(car.reload.engine_size).to eq(3.0)
      expect(car.reload.fuel_type).to eq("diesel")
    end

    it "queries auxiliary attributes directly" do
      car1 = Car.create!(name: "Prius", engine_size: 1.8, fuel_type: "hybrid")
      car2 = Car.create!(name: "Camry", engine_size: 2.5, fuel_type: "gasoline")

      hybrids = Car.where(fuel_type: "hybrid")
      expect(hybrids).to eq([car1])

      small_engines = Car.where(engine_size: ..2.0)
      expect(small_engines).to eq([car1])
    end

    it "ensures auxiliary record exists when setting attributes" do
      car = Car.new(name: "Test Car")
      car.engine_size = 2.0
      car.save!

      expect(car.aux_record).to be_present
      expect(car.engine_size).to eq(2.0)
    end
  end
end

Integration Tests

RSpec.describe "AuxTable Integration" do
  it "works with complex queries" do
    car1 = Car.create!(name: "Prius", fuel_type: "hybrid", engine_size: 1.8)
    car2 = Car.create!(name: "Camry", fuel_type: "gasoline", engine_size: 2.5)

    hybrids = Car.where(fuel_type: "hybrid")
    expect(hybrids).to eq([car1])

    small_engines = Car.where(engine_size: ..2.0)
    expect(small_engines).to eq([car1])
  end
end

Future Enhancements

Phase 2 Features

1. Polymorphic Auxiliary Tables: Support for auxiliary tables shared across multiple STI classes
2. Nested Auxiliary Tables: Support for auxiliary tables that reference other auxiliary tables
3. Dynamic Schema: Runtime addition of auxiliary columns without migrations
4. Composite Keys: Support for composite primary keys in auxiliary tables

Phase 3 Features

1. Sharding Support: Distribute auxiliary tables across multiple databases
2. Read Replicas: Route auxiliary table reads to replica databases
3. Materialized Views: Create materialized views combining main and auxiliary data
4. GraphQL Integration: Automatic GraphQL schema generation for auxiliary tables

Conclusion

The updated ActiveRecord Auxiliary Table gem provides an unconditional auxiliary table presence approach that delivers superior performance and developer experience. Key benefits of this design:

Performance Advantages

• Single Query Loading: Auxiliary attributes are loaded via JOIN in one query
• No N+1 Queries: Eliminates the need for separate queries to load auxiliary data
• Efficient Updates: Changes are batched and handled in transactions
• Optimized Indexes: Each auxiliary table can have specialized indexes

Developer Experience

• Transparent Access: Auxiliary attributes appear as native model attributes
• Automatic Creation: Auxiliary records are created automatically when needed
• Familiar API: Works seamlessly with existing ActiveRecord patterns
• No Breaking Changes: Existing code continues to work without modification

Database Benefits

• Reduced Schema Bloat: Subclass-specific columns are isolated
• Better Index Performance: Optimized indexes for each subclass
• Improved Query Performance: Efficient queries against subclass-specific data
• Easier Maintenance: Adding new subclass columns doesn't affect other subclasses

The implementation prioritizes developer experience through transparent attribute access, automatic query handling, and seamless integration with existing ActiveRecord features like validations, callbacks, and associations.