Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for dumping only selected tables from database #14

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions README.markdown
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,19 @@ Further, there are tasks db:dump and db:load which do the entire database (the e
rake db:data:dump_dir -> Dump contents of database to curr_dir_name/tablename.extension (defaults to yaml)
rake db:data:load_dir -> Load contents of db/data_dir into database

Also, it is possible to dump only selected tables of your database:

rake db:data:dump_dir filter_tables=^table[0-5]$|other_table
rake db:data:dump filter_tables=^table[0-5]$|other_table

(Beware that the `filter_tables` paramater is evaluated as a regular expression
(with `Regexp.new(filter_tables)`), so:

rake db:data:dump_dir filter_tables=mytable

will dump `mytable`, but also `mytable_join_other_table`.


In addition, we have plugins whereby you can export your database to/from various formats. We only deal with yaml and csv right now, but you can easily write tools for your own formats (such as Excel or XML). To use another format, just load setting the "class" parameter to the class you are using. This defaults to "YamlDb::Helper" which is a refactoring of the old yaml_db code. We'll shorten this to use class nicknames in a little bit.

## Examples
Expand Down
11 changes: 9 additions & 2 deletions lib/serialization_helper.rb
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,11 @@ module SerializationHelper
class Base
attr_reader :extension

def initialize(helper)
def initialize(helper, filter_table_names = nil)
@dumper = helper.dumper
@loader = helper.loader
@extension = helper.extension
@dumper.filter_table_names = filter_table_names if filter_table_names
end

def dump(filename)
Expand Down Expand Up @@ -137,6 +138,10 @@ def self.quote_table(table)
end

class Dump
class << self
attr_accessor :filter_table_names
end

def self.before_table(io, table)

end
Expand All @@ -154,7 +159,9 @@ def self.after_table(io, table)
end

def self.tables
ActiveRecord::Base.connection.tables.reject { |table| ['schema_info', 'schema_migrations'].include?(table) }
all_tables = ActiveRecord::Base.connection.tables.reject { |table| ['schema_info', 'schema_migrations'].include?(table) }

filter_table_names ? all_tables.grep(Regexp.new(filter_table_names)) : all_tables
end

def self.dump_table(io, table)
Expand Down
6 changes: 4 additions & 2 deletions lib/tasks/yaml_db_tasks.rake
Original file line number Diff line number Diff line change
Expand Up @@ -18,14 +18,16 @@ namespace :db do
task :dump => :environment do
format_class = ENV['class'] || "YamlDb::Helper"
helper = format_class.constantize
SerializationHelper::Base.new(helper).dump db_dump_data_file helper.extension
filter = ENV['filter_tables']
SerializationHelper::Base.new(helper,filter).dump db_dump_data_file helper.extension
end

desc "Dump contents of database to curr_dir_name/tablename.extension (defaults to yaml)"
task :dump_dir => :environment do
format_class = ENV['class'] || "YamlDb::Helper"
dir = ENV['dir'] || "#{Time.now.to_s.gsub(/ /, '_')}"
SerializationHelper::Base.new(format_class.constantize).dump_to_dir dump_dir("/#{dir}")
filter = ENV['filter_tables']
SerializationHelper::Base.new(format_class.constantize, filter).dump_to_dir dump_dir("/#{dir}")
end

desc "Load contents of db/data.extension (defaults to yaml) into database"
Expand Down
5 changes: 5 additions & 0 deletions spec/serialization_helper_base_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,11 @@ def stub_helper!
SerializationHelper::Base.new(@helper).dump_to_dir "dir_name"
end

it "should be able to be configured such as it will dump only selected tables" do
@dumper.should_receive(:"filter_table_names=").with("table_filter")
SerializationHelper::Base.new(@helper, "table_filter").dump_to_dir "dir_name"
end

end

context "for multi-file loads" do
Expand Down
44 changes: 44 additions & 0 deletions spec/serialization_helper_dump_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -61,5 +61,49 @@
end


context "when configured to only dump selected tables" do
before do
ActiveRecord::Base.connection.stub!(:tables).and_return([ 'mytable1', 'mytable2', 'mytable3', 'schema_info', 'schema_migrations' ])
end

after do
# Check expected tables to dump
SerializationHelper::Dump.tables.sort.should == @expected_tables

if @expected_tables.size > 0
@expected_tables.each do |table|
# Check actual calls to dump_table with expected tables
SerializationHelper::Dump.should_receive(:dump_table).with(nil, table)
end
else
SerializationHelper::Dump.should_not_receive(:dump_table).with(anything)
end

SerializationHelper::Dump.dump(nil)

# Restore default behaviour
SerializationHelper::Dump.filter_table_names = nil
end

it "should dump every table if filter not set" do
SerializationHelper::Dump.filter_table_names = nil
@expected_tables = ['mytable1', 'mytable2', 'mytable3']
end

it "should dump matching tables if filter is set as a matching string" do
SerializationHelper::Dump.filter_table_names = "mytable"
@expected_tables = ['mytable1', 'mytable2', 'mytable3']
end

it "should dump matching tables if filter is set as a regular expression string" do
SerializationHelper::Dump.filter_table_names = "mytable[1-2]"
@expected_tables = ['mytable1', 'mytable2']
end

it "should not dump any table if filter does not match any table" do
SerializationHelper::Dump.filter_table_names = "wadustable"
@expected_tables = []
end

end
end