What is the specific process ActiveRecord generates attributes?

Question

I'm using Rails to connect to a remote MySQL database. When doing an ActiveRecord select, or even find_by_sql I noticed many columns were missing. The columns show up properly in Model.connection.select_all, so the issue is not the database sending them. Model.new.attributes.keys gives

["ORIG_ID", "DEST_ID", "CTRL_NUM", "\nTRL_NUM", "INTEGERM", "DTTEGERM", 
"TMTEGERM", "DOC_TYPE", "STEPTYPE", "\nTEPTYPE", "TEXTTYPE", "RT_SEQPE", 
"OSYSEQPE", "OVOLUMEE", "OSUBVOLE", "OFNAMELE", "\nFNAMELE", "TEXTMELE", 
"FVOLUMEE", "FSUBVOLE", "FFNAMELE", "ASYSMELE", "AVOLUMEE", "\nVOLUMEE", 
"TEXTUMEE", "AFNAMEEE", "CONTACTE", "INTSACTE", "GRPSACTE", "\nRPSACTE", 
"INTEGERE", "SEGSGERE", "BYTESERE", "IC_NUMRE", "MSG_NUME", "\nSG_NUME", 
"TEXTNUME", "ODT_DTME", "ODT_ORIG_ID", "ODT_DEST_ID", "\nDT_DEST_ID", 
"INTEGERT_ID", "DDT_DTRT_ID", "DDT_DEST_ID", "DDT_ORIG_ID", "\nDT_ORIG_ID", 
"INTEGERG_ID", "FORMAT_IDID", "AGENCY_IDID", "VERSIONIDID", "\nERSIONIDID", 
"TEXTIONIDID", "ASETIONIDID", "PTETIONIDID", "FILLER1IDID", "SESS_NUMDID"]

The only columns with correct names are ORIG_ID, DEST_ID, CTRL_NUM, DOC_TYPE, ODT_ORIG_ID, ODT_DEST_ID, DDT_ORIG_ID, DDT_DEST_ID The rest of the columns appear to be carrying over characters from the previous column, in addition to the /n causing the column type to be come an attribute and overwrite the next column

 ORIG_ID
 DEST_ID  Same length, success
 CTRL_NUM New max length, success
\nTRL_NUM First \n character, adds new column and replaces next with field type
 INTEGERM Should be TSTAMP, took type instead
 DTTEGERM Should be DT Shorter than previous, carries all extra characters
 TMTEGERM Should be TM
 DOC_TYPE Same length as previous, overwrites all characters successfully
 STEPTYPE Should be STEP
\nTEPTYPE
 TEXTTYPE Should be PNAME, took type instead
 RT_SEQPE Should be RT_SEQ
 OSYSEQPE etc..
 OVOLUMEE
 OSUBVOLE
 OFNAMELE
\nFNAMELE
 TEXTMELE Should be FSYS
 FVOLUMEE
 FSUBVOLE
 FFNAMELE
 ASYSMELE
 AVOLUMEE
\nVOLUMEE Should be ASUBVOL
 TEXTUMEE
 AFNAMEEE
 CONTACTE
 INTSACTE
 GRPSACTE
\nRPSACTE
 INTEGERE Should be SETS
 SEGSGERE
 BYTESERE
 IC_NUMRE
 MSG_NUME
\nSG_NUME
 TEXTNUME Should be CDT
 ODT_DTME
 ODT_ORIG_ID
 ODT_DEST_ID
\nDT_DEST_ID
 INTEGERT_ID Should be ODT_CTRL_NUM
 DDT_DTRT_ID
 DDT_DEST_ID
 DDT_ORIG_ID
\nDT_ORIG_ID
 INTEGERG_ID Should be DDT_CTRL_NUM
 FORMAT_IDID
 AGENCY_IDID
 VERSIONIDID
\nERSIONIDID
 TEXTIONIDID  Should be TSET, took type instead
 ASETIONIDID
 PTETIONIDID
 FILLER1IDID
 SESS_NUMDID

So what is the SQL that ActiveRecord executes to get the schema for the attributes? And what is the process it does to convert that result to the attributes?

My ActiveRecord version is 4.1.8, Rails is 4.1.8, MySql2 is 0.3.18, Ruby is 2.1.5p273

Rake:db:schema:dump gives the same thing

ActiveRecord::Schema.define() do

  create_table "STEPS", id: false, force: true do |t|
    t.string  "ORIG_ID",      limit: nil, default: "NULL"
    t.string  "DEST_ID",      limit: nil, default: "NULL"
    t.integer "CTRL_NUM",                 default: 0
    t.string  "\nTRL_NUM",    limit: nil, default: "NULL"
    t.string  "INTEGERM",     limit: nil, default: "NULL"
    t.string  "DTTEGERM",     limit: nil, default: "NULL"
    t.string  "TMTEGERM",     limit: nil, default: "NULL"
    t.string  "DOC_TYPE",     limit: nil, default: "NULL"
    t.string  "STEPTYPE",     limit: nil, default: "NULL"
    t.string  "\nTEPTYPE",    limit: nil, default: "NULL"
    t.string  "TEXTTYPE",     limit: nil, default: "NULL"
    t.integer "RT_SEQPE",                 default: 0
    t.string  "OSYSEQPE",     limit: nil, default: "NULL"
    t.string  "OVOLUMEE",     limit: nil, default: "NULL"
    t.string  "OSUBVOLE",     limit: nil, default: "NULL"
    t.string  "OFNAMELE",     limit: nil, default: "NULL"
    t.string  "\nFNAMELE",    limit: nil, default: "NULL"
    t.string  "TEXTMELE",     limit: nil, default: "NULL"
    t.string  "FVOLUMEE",     limit: nil, default: "NULL"
    t.string  "FSUBVOLE",     limit: nil, default: "NULL"
    t.string  "FFNAMELE",     limit: nil, default: "NULL"
    t.string  "ASYSMELE",     limit: nil, default: "NULL"
    t.string  "AVOLUMEE",     limit: nil, default: "NULL"
    t.string  "\nVOLUMEE",    limit: nil, default: "NULL"
    t.string  "TEXTUMEE",     limit: nil, default: "NULL"
    t.string  "AFNAMEEE",     limit: nil, default: "NULL"
    t.string  "CONTACTE",     limit: nil, default: "NULL"
    t.integer "INTSACTE",                 default: 0
    t.integer "GRPSACTE",                 default: 0
    t.string  "\nRPSACTE",    limit: nil, default: "NULL"
    t.string  "INTEGERE",     limit: nil, default: "NULL"
    t.integer "SEGSGERE",                 default: 0
    t.integer "BYTESERE",                 default: 0
    t.integer "IC_NUMRE",                 default: 0
    t.integer "MSG_NUME",                 default: 0
    t.string  "\nSG_NUME",    limit: nil, default: "NULL"
    t.string  "TEXTNUME",     limit: nil, default: "NULL"
    t.string  "ODT_DTME",     limit: nil, default: "NULL"
    t.string  "ODT_ORIG_ID",  limit: nil, default: "NULL"
    t.string  "ODT_DEST_ID",  limit: nil, default: "NULL"
    t.string  "\nDT_DEST_ID", limit: nil, default: "NULL"
    t.string  "INTEGERT_ID",  limit: nil, default: "NULL"
    t.string  "DDT_DTRT_ID",  limit: nil, default: "NULL"
    t.string  "DDT_DEST_ID",  limit: nil, default: "NULL"
    t.string  "DDT_ORIG_ID",  limit: nil, default: "NULL"
    t.string  "\nDT_ORIG_ID", limit: nil, default: "NULL"
    t.string  "INTEGERG_ID",  limit: nil, default: "NULL"
    t.string  "FORMAT_IDID",  limit: nil, default: "NULL"
    t.string  "AGENCY_IDID",  limit: nil, default: "NULL"
    t.string  "VERSIONIDID",  limit: nil, default: "NULL"
    t.string  "\nERSIONIDID", limit: nil, default: "NULL"
    t.string  "TEXTIONIDID",  limit: nil, default: "NULL"
    t.string  "ASETIONIDID",  limit: nil, default: "NULL"
    t.string  "PTETIONIDID",  limit: nil, default: "NULL"
    t.string  "FILLER1IDID",  limit: nil, default: "NULL"
    t.integer "SESS_NUMDID",              default: 0
  end

  add_index "STEPS", ["ECT"], name: "STEPS_05", length: {"ECT"=>"NULL"}, using: :btree
  add_index "STEPS", ["LECT"], name: "STEPS_01", length: {"LECT"=>"NULL"}, using: :btree
  add_index "STEPS", ["LECT"], name: "STEPS_02", length: {"LECT"=>"NULL"}, using: :btree
  add_index "STEPS", ["LECT"], name: "STEPS_03", length: {"LECT"=>"NULL"}, using: :btree
  add_index "STEPS", ["LECT"], name: "STEPS_04", length: {"LECT"=>"NULL"}, using: :btree
  add_index "STEPS", ["LECT"], name: "STEPS_PK", length: {"LECT"=>"NULL"}, using: :btree

end

Running Model.connection.select_all("SHOW CREATE TABLE STEPS").to_hash gives

CREATE_TABLE = CREATE TABLE STEPS (ORIG_ID TEXT,DEST_ID TEXT,CTRL_NUM INTEGER,TSTAMP INTEGER,DT TEXT,TM TEXT,DOC_TYPE TEXT,STEP TEXT,PNAME TEXT,RT_SEQ INTEGER,OBJECT TEXT,DATAFILE TEXT,ACTION TEXT,CONTACT TEXT,INTS INTEGER,GRPS INTEGER,SETS INTEGER,SEGS INTEGER,BYTES INTEGER,IC_NUM INTEGER,MSG_NUM INTEGER,CDT TEXT,FORMAT_ID TEXT,AGENCY TEXT,VERSION TEXT,TSET TEXT,ASET TEXT,PT TEXT,SESS_NUM INTEGER,SENDER TEXT, RECEIVER TEXT,GS_SENDER TEXT, GS_RECEIVER TEXT, GS_CTRL_NUM INTEGER)

score 0 · Answer 1 · edited May 23 '17 at 12:13

0

It seems like your database and your attributes in schema.rb aren't in sync.

You'll want to look at this answer to figure out what is shown in the model vs the database and look at this answer that shows what to do when they are out of sync.

edited May 23 '17 at 12:13

Community

1
1

answered Mar 31 '15 at 18:37

creativereason

1,524
1
11
20

Running rake db:schema:dump results in a schema containing the same thing that the attributes show. So they both use the same SQL to build the attributes, here is SHOW CREATE TABLE STEPS CREATE_TABLE = CREATE TABLE STEPS (ORIG_ID TEXT,DEST_ID TEXT,CTRL_NUM INTEGER,TSTAMP INTEGER,DT TEXT,TM TEXT,DOC_TYPE TEXT,STEP TEXT,PNAME TEXT,RT_SEQ INTEGER,OBJECT TEXT,DATAFILE TEXT,ACTION TEXT,CONTACT TEXT,INTS INTEGER,GRPS INTEGER,SETS INTEGER,SEGS INTEGER,BYTES INTEGER,IC_NUM INTEGER,MSG_NUM INTEGER,CDT TEXT,FORMAT_ID TEXT,AGENCY TEXT,VERSION TEXT,TSET TEXT,ASET TEXT,PT TEXT) A few more fields.. – Clark Mar 31 '15 at 18:59
Sorry about that messy comment, added the relevant parts to the question. Thanks for the suggestion. – Clark Mar 31 '15 at 19:22

score 0 · Accepted Answer · answered Apr 02 '15 at 14:17

I found what I believe is the SQL that ActiveRecord uses to get the column names, in the abstract_mysql_adapter.rb file for the ActiveRecord Gem. \Ruby2.1.0\lib\ruby\gems\2.1.0\gems\activerecord-4.1.8\lib\active_record\connection_adapters\abstract_mysql_adapter.rb

  def columns(table_name)#:nodoc:
    sql = "SHOW FULL FIELDS FROM #{quote_table_name(table_name)}"
    execute_and_free(sql, 'SCHEMA') do |result|
      each_hash(result).map do |field|
        field_name = set_field_encoding(field[:Field])
        new_column(field_name, field[:Default], field[:Type], field[:Null] == "YES", field[:Collation], field[:Extra])
      end
    end
  end

What is the specific process ActiveRecord generates attributes?

2 Answers2