RE: [coldbox:4886] performance on large caches

Can you store the hash in the database for each record? This will persist the hash value over server restarts. Also, this process may work better if you can find a way to load the spread sheet into a temp table in the DB, and then process the inserts and updates on the DB side. Then you can quickly determine whether or not a record has changed by hashing the records in the temp table and simply joining it to the real table where the hashes don’t match.


Storing the hashes in the db is a thought but I was trying to avoid
the db hits in the first place. Unfortunately, it isn't easy to do a
bulk insert into a temp table as the data in each row actually
contains two different objects that map to two different tables.
Still, if I added a hash field to each row, I could potentially bring
it down to a single select on a single field for each row and then an
update/insert/no action based on the result.