Skip to content

Commit

Permalink
Adds CSV loader for SearchEvents
Browse files Browse the repository at this point in the history
https://mitlibraries.atlassian.net/browse/TCO-35

This loader is not meant to be used in production, but instead be a
convenience for loading data into development environments to allow for
easier development of detection algorithms or reporting.

There is a Dataclip that exports data in the expected format, but it
is also easy to generate your own file as it is just `term` and
`timestamp` that is expected.

Currently, `source` is provided via CLI but if we find it would be
better to include the source in the CSV that would be an easy change
to make.
  • Loading branch information
JPrevost committed Jul 1, 2024
1 parent 3213b09 commit 1cf9848
Showing 1 changed file with 28 additions and 0 deletions.
28 changes: 28 additions & 0 deletions lib/tasks/loader.rake
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# frozen_string_literal: true

require 'csv'

# Loaders can bulk load data
namespace :loader do
# csv loader can bulk load SearchEvents and Terms
#
# @note the csv should be formated as `term phrase`, `timestamp`. A dataclip is available that can export in this
# format.
# @example
# bin/rails loader:csv['local_path_to_file.csv', 'some-source-to-use-for-all-loaded-records']
#
# @param path [String] local file path to a CSV file to load
# @param source [String] source name to load the data under
desc 'Load search_events from csv'
task :csv, %i[path source] => :environment do |_task, args|
raise ArgumentError.new('Path is required') unless args.path.present?
raise ArgumentError.new('Source is required') unless args.source.present?

Rails.logger.info("Loading data from #{args.path}")

CSV.foreach(args.path) do |row|
term = Term.create_or_find_by!(phrase: row.first)
term.search_events.create!(source: args.source, created_at: row.last)
end
end
end

0 comments on commit 1cf9848

Please sign in to comment.