RailsProgramming today is a race between softwareProgramming today is a race between softwareengineers striving to build better and bigger idiot-proofengineers striving to build better and bigger idiot-proofprograms, and the Universe trying to produce biggerprograms, and the Universe trying to produce biggerand better idiots. So far, the Universe is winning. - Rickand better idiots. So far, the Universe is winning. - RickCookCook No, I'm not RickNo, I'm not Rick
Inventory Application
So, I was reading my email
• Tuesday morning• A fellow that reports to me came in to
talk about how we order and receivepcs and printers
• And I've been wanting to convert ourmachine move form to a web page
• Suddenly, a real project…
Requirements
• A form that can handle both acquisitionsand moves
• An email hook so we can create remedytickets as part of the process
• A mechanism for marking records asprocessed, and for moving data into ourmain inventory database
• This should be simple, following KISS
What we'll cover
• Migrations and more about Sqlite3• Validations
The usual• Create a rails
framework• Note backslashes• Also, use of
underscores• Some of this is bad
thinking…
rails inventoryrails inventorycd cd inventoryinventoryruby script/generate scaffold Machine \ruby script/generate scaffold Machine \ user_nameuser_name:string \:string \ date_submitteddate_submitted::datetime datetime \\ ticket_numberticket_number:integer \:integer \ from_locationfrom_location:string \:string \ to_locationto_location:string \:string \ from_entityfrom_entity:string \:string \ to_entityto_entity:string \:string \ old_machine_nameold_machine_name:string \:string \ new_machine_namenew_machine_name:string \:string \ serial_numberserial_number:string \:string \ unc_numberunc_number:string \:string \ comments:text \ comments:text \ status:string \ status:string \ exported_to_mainexported_to_main::boolean boolean \\ unneeded_fieldunneeded_field:decimal:decimal
Generation of the firstMigration
• The generation of the scaffold creates:– The view– A controller– A model– Also a database migration file in the db
directory, in this case:20081104182035_create_machines.rb
• Note the timestamp and the conventionalname
What does this file do?
• This file is a script, that contains aclass, with two defined methods
• One method– creates the database table– creates initial fields– sets types
• The other method– undoes everything the first one does
class class CreateMachines CreateMachines < < ActiveRecordActiveRecord::Migration::Migration def self.up def self.up create_table create_table :machines do |t|:machines do |t| t.string : t.string :user_nameuser_name t.t.datetime datetime ::date_submitteddate_submitted t.integer :t.integer :ticket_numberticket_number t.string :t.string :from_locationfrom_location t.string :t.string :to_locationto_location t.string :t.string :from_entityfrom_entity t.string :t.string :to_entityto_entity t.string :t.string :old_machine_nameold_machine_name t.string :t.string :new_machine_namenew_machine_name t.string :t.string :serial_numberserial_number t.string :t.string :unc_numberunc_number t.text :commentst.text :comments t.string :status t.string :status t. t.boolean boolean ::exported_to_mainexported_to_main t.decimal :t.decimal :unneeded_fieldunneeded_field t.timestampst.timestamps end end end end
1st Part• Class inherits
fromActiveRecord::Migration
• self.up is amethod appliedwhen a migrationis run
• A loop assignstype and names
def self.downdef self.down drop_table drop_table :machines:machines end endendend
2nd Part
• a second methodprovides a way toroll back themigration
• Done properly,this allows one tomove forward andback in database"versions" withoutaffecting otherstructures
Migrations
• You can modify this file before applyingit, adding additional options such asfield lengths, default values, etc
What's the point?
• Migrations allow manipulation of the databasewith some version control
• You could also manually manipulate thedatabase, but you'd have to keep track of thechanges
• But some migrations are irreversible, and ifyou don't define a good way back….
• To protect against that, backup! Or useversion control systems like cvs, subversion,git
schema.rb
• Stored in db/• This is the canonical representation of
the current state of the database• You could modify this--don't• Generated after each migration• You can use this with db:schema:load
to implement the same db structures onanother system
ActiveRecordActiveRecord::Schema.define :version => 20081105005808 do::Schema.define :version => 20081105005808 do
create_table create_table "machines", :force => true do |t|"machines", :force => true do |t| t.string t.string "user_name""user_name" t.t.datetime "date_submitted"datetime "date_submitted" t.integer t.integer "ticket_number""ticket_number" t.string t.string "from_location""from_location" t.string t.string "to_location""to_location" t.string t.string "from_entity""from_entity" t.string t.string "to_entity""to_entity" t.string t.string "old_machine_name""old_machine_name" t.string t.string "new_machine_name""new_machine_name" t.string t.string "serial_number""serial_number" t.integer t.integer "unc_number""unc_number", :limit => 255, :limit => 255 t.text "comments" t.text "comments" t.string "status" t.string "status" t. t.boolean "exported_to_main"boolean "exported_to_main" t.t.datetime "created_at"datetime "created_at" t.t.datetime "updated_at"datetime "updated_at" endend
But…
• We haven't run our first migration yet• rake db:migrate• This command applies all unapplied
migrations in the db/migrate dir• The timestamps for the migrations are
stored in a table in the database,schema_migrations (this is how railskeeps track of migrations)
What rake really does
• Analogous to make, it looks for a file to process inorder to build something
• rake db:migrate looks in the db/migrate folder, andfinds any of the migration files that have not yet beenapplied, and runs those
• Each time you want to make changes to the db, yougenerate a migration script, edit that, then use rake tomigrate the changes into the db
About the database
• By default, rails 2.1 uses sqlite3, otherdbs are also possible to use, like mysql
• sqlite3 databases are single files, eg.development.sqlite3
• We can look at the database directlylook, but don't touch!
Sqlite3 syntax
• Commands that manipulate the dbbegin with a period
• Sql commands don’t and must end witha semicolon
• Get help with .help, exit with .exit
Some sqlite3 commands.databases List names and files of attached databases.exit Exit this program.header s ON|OFF Turn display of headers on or off.help Show this message.output FILENAME Send output to FILENAME.output stdout Send output to the screen.quit Exit this program.read FILENAME Execute SQL in FILENAME.schema ?TABLE? Show the CREATE statements.separator STRING Change separator used by output mode and .import.show Show the current values for various settings
A Sample Sessionsqlite> .tablesmachines schema_migrationssqlite> .schema machinesCREATE TABLE "machines" "id" INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, "user_name" varchar 255 , "date_submitted" datetime, "ticket_number" integer, "from_location" varchar 255 , "to_location" varchar 255 , "from_entity" varchar 255 , "to_entity" varchar 255 , "old_machine_name" varchar 255 , "new_machine_name" varchar 255 , "serial_number" varchar 255 , "unc_number" varchar 255 , "comments" text, "status" varchar 255 , "exported_to_main" boolean, "unneeded_field" decimal, "created_at" datetime, "updated_at" datetime ;sqlite> .exit
You might have noticed
• There's a field named unneeded_field• I don't need this field, so we'll look at
dumping it• To do this, create a new migrationhays$ script/generate migration \ RemoveColumn_uneeded_field exists db/migrate create db/migrate/20081104181336_remove_column_uneeded_field.rb
A blank migration
• Now we have a blank migration file:20081104181336_remove_column_uneeded_field.rb file in db/migrate
• Yes the name is long, but it's explicitand helps us know what the migrationwas supposed to do
• We have to edit this file with thecommands we need to apply to thedatabase (rails, as good as it is, cannotread minds)
A blank migration• Again, a class with two methods, but
nothing in them
class class RemoveColumnUneededField RemoveColumnUneededField << \\ ActiveRecordActiveRecord::Migration::Migration def self.up def self.up end end
def self.down def self.down end endendend
Filling the empty migration• We'll use remove_column with the table
and field name, and add_column toundo the change in case we were wrong
class class RemoveColumnUneededField RemoveColumnUneededField << \\ ActiveRecordActiveRecord::Migration::Migration def self.up def self.up remove_column remove_column :machines, ::machines, :unneeded_fieldunneeded_field endend
def self.down def self.down add_column add_column :machines, ::machines, :unneeded_fieldunneeded_field, :decimal, :decimal end endend end
create_table name, options drop_table namerename_table old_name, new_nameadd_column table_name, column_name, type, optionsrename_column table_name, column_name, new_column_name change_column table_name, column_name, type, options remove_column table_name, column_name add_index table_name, column_names, options remove_index table_name, index_name
from http://api.rubyonrails.org/classes/ActiveRecord/Migration.html
Available migrationcommands
• These are the current commands you canuse
Up and down
• Use rake db:migrate to apply this nemigration (the assumption is that wewant to apply a new migration)
• Use rake db:migrate:downVERSION=xxxxxxxxxxxxxx wherexxxxxxxxxxxxxx is the timestamp of themigration file.
• So if we run rake db:migrate:downVERSION=20081104182506, we getthe column back
Running the Migration
• When you run it, if you don’t get anerror, you'll see something like this
hays$ rake db:migratehays$ rake db:migrate(in /Volumes/BIL/INLS672/samples/ruby/inventory)(in /Volumes/BIL/INLS672/samples/ruby/inventory)== 20081104182506 == 20081104182506 RemoveColumnUneededFieldRemoveColumnUneededField: migrating ===========: migrating ===========-- -- remove_columnremove_column(:machines, :(:machines, :unneeded_fieldunneeded_field)) -> 0.3480s -> 0.3480s== 20081104182506 == 20081104182506 RemoveColumnUneededFieldRemoveColumnUneededField: migrated (0.3494s) ===: migrated (0.3494s) ===
Resultshays$ sqlite3 db/development.sqlite3 hays$ sqlite3 db/development.sqlite3 SQLite SQLite version 3.4.0version 3.4.0Enter ".help" for instructionsEnter ".help" for instructionssqlitesqlite> .schema machines> .schema machinesCREATE TABLE "machines" ("id" INTEGER PRIMARYCREATE TABLE "machines" ("id" INTEGER PRIMARY KEYKEY AUTOINCREMENT NOT NULL, AUTOINCREMENT NOT NULL, "user_name" varchar"user_name" varchar(255),(255), "date_submitted" datetime"date_submitted" datetime, , "ticket_number" "ticket_number" integer, integer, "from_location" "from_location" varcharvarchar(255), (255), "to_location" varchar"to_location" varchar(255), (255), "from_entity" varchar"from_entity" varchar(255),(255), "to_entity" varchar"to_entity" varchar(255), (255), "old_machine_name" varchar"old_machine_name" varchar(255),(255), "new_machine_name" varchar"new_machine_name" varchar(255), (255), "serial_number" varchar"serial_number" varchar(255),(255), "unc_number" varchar"unc_number" varchar(255), "comments" text, "status" (255), "comments" text, "status" varcharvarchar(255),(255), "exported_to_main" boolean"exported_to_main" boolean, , "created_at" datetime"created_at" datetime, , "updated_at" "updated_at" datetimedatetime););sqlitesqlite> .exit> .exit
Rolling back
• We can get the column back byrunning:rake db:migrate:down VERSION=20081104182506
• But we can't get the data that was in thecolumn back
An aside
• Rail documentation– Tutorials for 2.1 are generally just how to get
started– API doc are the most complete, but not very
readable--seehttp://api.rubyonrails.org/
– Lots of code snippets out there, playing with thoseare a good was to learn new things--most of theseare unixy and terse
– Agile Web Development with Rails is the bestbook I've found. see: http://www.pragprog.com/
– Practical Rails Projects, seehttp://www.apress.com/book/view/9781590597811
An aside
• Go with the flow– Rails is complicated enough that it's best to
roll with it– This is partly due to the emphasis on
convention over configuration– The conundrum is where does the
knowledge lie• CLI versus GUI• DB versus Middleware versus Browser• Thick versus Thin clients
After all this…
• We've looked at migrations and thedatabase
• Migrations do not affect other parts ofthe applications, such as the model,controller(s), or any of the views
• We dropped a column after thescaffolding, so the views referenceunneeded_field
• So we get an error when we try to runthe pages…
The error
Easy peasy
• The error message references amethod, this is one way fields in the dbare accessed
• Also shows us source code around theoffense
Cleaning up
• In each of the views we need to removethe references
• For example, in new.html.erb andedit.html.erb:<p><%= f.label :unneeded_field %><br /><%= f.text_field :unneeded_field %></p>
Now it works
Validations
Simple validation
• Now that we have the db straighten out(yeah, right), time to add somevalidations
• These belong in the model, machine.rbin app/models
• Right now, that's just:class Machine < ActiveRecord::Baseend
Included validations
• Our class has access to classes and methodsfrom ActiveRecord::Validations
• The simplest is validates_presence_of• Usage:
# Validate that required fields are not emptyvalidates_presence_of :user_name, \ :date_submitted, :from_location, \ :from_entity, :to_location, :to_entity, :status
see http://api.rubyonrails.org/classes/ActiveRecord/Validations.html
Other Included Validationsvalidates_acceptance_ofvalidates_associatedvalidates_confirmation_ofvalidates_eachvalidates_exclusion_ofvalidates_format_ofvalidates_inclusion_ofvalidates_length_ofvalidates_numericality_ofvalidates_presence_ofvalidates_size_ofvalidates_uniqueness_of
from http://api.rubyonrails.org/classes/ActiveRecord/Validations/ClassMethods.html
Validations added to themodel
• These went easily:#Validates #Validates to_locationto_location, that should only 6 chars, we'll allow 10, that should only 6 chars, we'll allow 10validates_length_of validates_length_of ::to_locationto_location, :maximum=>15, :maximum=>15
#Validates fields that should not be more than 15 characters#Validates fields that should not be more than 15 charactersvalidates_length_of validates_length_of ::user_nameuser_name, :maximum=>15, :maximum=>15
#Validates fields that should not be more than 30 chars#Validates fields that should not be more than 30 charsvalidates_length_of validates_length_of ::from_locationfrom_location, :maximum=>30, :maximum=>30validates_length_of validates_length_of ::from_entityfrom_entity, :maximum=>30, :maximum=>30validates_length_of validates_length_of ::to_entityto_entity, :maximum=>30, :maximum=>30# None of these affect the database# None of these affect the database field lengthsfield lengths
A rabbit hole• And, also, I want to make sure that
unc_number is an integer, but it can beempty, so I try this:#Validates that unc number is a numbervalidates_numericality_of :unc_number,\ :allow_nil=>true, :only_integer=>true,
• But it fail--if the field is empty it throws anerror…
After digging
• Google is my friend, and I find:http://opensoul.org/2007/2/7/validations-on-empty-not-nil-attributes
• This suggests that the problem is thatunc_number is really a string, and thatan empty string is empty, not nil…
But where?
• HTML knows shinola about text/integer types• But no errors on stuffing the wrong kind of
data into a field (esp no ugly ones), so likelysqlite3 doesn't really care
• But the db type is involved since in themigration it was defined with:t.string :unc_number
• So rails knows it's a string
A hack• Brandon's approach is to define a
method that goes through all of theparams passed to the model forvalidation and if empty, set to nil….
def def clear_empty_attrsclear_empty_attrs @attributes.each do |key,value|@attributes.each do |key,value| self[key] = nil if value.blank? self[key] = nil if value.blank? end end end end
A hack• This method must be called before the
validation are run, so it goes to the top of themodel (not required, JAGI)
• This is done using before_validation• So before the validation sequence, all empties
are converted to nil• Does this seem like a good fix?
before_validation :clear_empty_attrs
A hack• One concern I had was this it hitting all of
the fields--and that way leads tomadness--so I limited it
protected def clear_empty_attrs # we don't need to loop through everything, so I'm # just calling the one field # @attributes.each do |key,value| #self[key] = nil if value.blank? self[:unc_number] = nil if unc_number.blank? # end end
Works, but….
• It is a hack• Using it, I'm working around an error
that's due to a bad choice I made--rarely a good idea, and these thingsmay take time to bite
• I'm also going against the flow• So what to do to fix it? Test, research,
change, rinse and repeat
Back to the beginning• As it turns out,
I did define aninteger in thedatabase,ticket_number
ruby script/generate scaffold Machine \ user_name:string \ date_submitted:datetime \ ticket_number:integer \ from_location:string \ to_location:string \ from_entity:string \ to_entity:string \ old_machine_name:string \ new_machine_name:string \ serial_number:string \ unc_number:string \ comments:text \ status:string \ exported_to_main:boolean \ unneeded_field:decimal
An easy test
• I try the same validation against thatfield and it works, so I confirm theproblem is the field type
• Note the error message…validates_numericality_of\validates_numericality_of\ ::ticket_numberticket_number,\,\ : :only_integer=only_integer=>true,\>true,\ : :allow_nil=allow_nil=>true,\>true,\ :message=>'must be an integer if not blank' :message=>'must be an integer if not blank'
A new migration
• So, I need to change the type of unc_number• Again leaving a way back….• This fixed it for realclass class ChangeTextToDecimalsUncNumberChangeTextToDecimalsUncNumber < < ActiveRecordActiveRecord::Migration::Migration def self.up def self.up change_columnchange_column(:machines, :(:machines, :unc_numberunc_number, :integer), :integer) end end def self.down def self.down change_columnchange_column(:machines, :(:machines, :unc_numberunc_number, :string), :string) end endendend
Custom Validations
• It's also easy to write custom validations– Define a method in the protected section
(since we don't need this outside our class)– Call that method as a symbol in the
validation section:validate :our_method
• As an example, we'll work withticket_number, even tho it's an artificialexample
A new method
• First, we'll check the value and make sure it'sabove 1000
• In testing this works ok, but obviously it won'taccept a nil value
def ticket_number_must_be_greater_than_1000def ticket_number_must_be_greater_than_1000 errors.add(: errors.add(:ticket_numberticket_number, 'must be greater than 1000')\, 'must be greater than 1000')\ if if ticket_number ticket_number < 1001< 1001 end end
Not a nil
• So we need to check for not nil and lessthan 1001
• Use a bang (!) to say notdef ticket_number_must_be_greater_than_1000def ticket_number_must_be_greater_than_1000 errors.add(: errors.add(:ticket_numberticket_number, 'must be greater than 1000')\, 'must be greater than 1000')\ if ! if !ticket_numberticket_number.nil? \.nil? \ && && ticket_number ticket_number < 1001< 1001endend
Time Validations
• We want the date_submitted to bereasonable
• Fortunately, rails understands time anddates
Time Methods• ago• day• days• fortnight• fortnights• from_now• hour• hours• minute• minutes
• month• months• second• seconds• since• until• week• weeks• year• years
Another Validation
# This validates that the date and time are # This validates that the date and time are resonable resonable valuesvalues def def date_submitted_must_be_sensible date_submitted_must_be_sensible errors.add(:errors.add(:date_submitteddate_submitted, \, \ 'Time and date cannot be in the past')\ 'Time and date cannot be in the past')\ if if date_submitted date_submitted < Time.now< Time.now errors.add(: errors.add(:date_submitteddate_submitted, \, \ 'Time and date cannot be more than 2 years in the future')\ 'Time and date cannot be more than 2 years in the future')\ if if date_submitted date_submitted > Time.now.advance(:years => 2)> Time.now.advance(:years => 2) # is equivalent to: # is equivalent to: #if #if date_submitted date_submitted > 2.years.> 2.years.from_nowfrom_now end end
Other minor changes
• Stripped down the index view, don'tneed that much detail
• This version is tarred and gzipped in thesamples dir as inventory01.gz.tar
Intelligence in the middleware
• Although sqlite3 types this field as a string, itdoesn't care what the contents are
• Rails does care though• So most all of the control mechanisms are in
rails• This is a common approach• Makes management of the system easier
Sources
• http://dizzy.co.uk/ruby_on_rails/cheatsheets/rails-migrations
• http://opensoul.org/2007/2/7/validations-on-empty-not-nil-attributes
• http://rubyonrailsprogrammingtips.com/• http://linuxgazette.net/109/chirico1.html