Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CV2-5138: Data Dashboard: Write and analyze queries #2033

Merged
merged 22 commits into from
Sep 25, 2024
Merged
Changes from 8 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
9155140
CV2-5138: initiate a class for data points
melsawy Sep 12, 2024
a46574a
Merge branch 'develop' into CV2-5138-data-dashboard-write-and-analyze…
melsawy Sep 17, 2024
23f116c
CV2-5138: add extra data points
melsawy Sep 18, 2024
7602737
Merge branch 'develop' into CV2-5138-data-dashboard-write-and-analyze…
melsawy Sep 18, 2024
a856690
CV2-5138: fix typo
melsawy Sep 18, 2024
3601672
CV2-5138: remove private methods
melsawy Sep 18, 2024
8551fd1
CV2-5138: add newsletters_sent report
melsawy Sep 19, 2024
661b123
Merge branch 'develop' into CV2-5138-data-dashboard-write-and-analyze…
melsawy Sep 19, 2024
c704455
CV2-5138: add users report
melsawy Sep 22, 2024
e445195
Merge branch 'develop' into CV2-5138-data-dashboard-write-and-analyze…
melsawy Sep 22, 2024
d0d4b3c
CV2-5138: add top cluster and top media tags
melsawy Sep 23, 2024
15ddef8
CV2-5138: add users data
melsawy Sep 23, 2024
f4545e1
CV2-5138: more updates for user data
melsawy Sep 23, 2024
4471828
Merge branch 'develop' into CV2-5138-data-dashboard-write-and-analyze…
melsawy Sep 23, 2024
6c0655b
CV2-5138: apply PR comments
melsawy Sep 23, 2024
01628ef
CV2-5138: add tests
melsawy Sep 24, 2024
ce1980f
CV2-5138: add more index to improve performance
melsawy Sep 24, 2024
cc3a6b0
Merge branch 'develop' into CV2-5138-data-dashboard-write-and-analyze…
melsawy Sep 24, 2024
cb401ca
CV2-5138: update schema.rb file
melsawy Sep 24, 2024
bedbb83
CV2-5138: fix CC
melsawy Sep 24, 2024
9d36319
CV2-5138: test coverage
melsawy Sep 25, 2024
0cb8fdc
Merge branch 'develop' into CV2-5138-data-dashboard-write-and-analyze…
melsawy Sep 25, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 76 additions & 0 deletions lib/check_data_points.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
class CheckDataPoints
# 1) Number of tipline messages
def self.tipline_messages(team_id, start_date, end_date, granularity = nil)
start_date, end_date = self.parse_start_end_dates(start_date, end_date)
query = TiplineMessage.where(team_id: team_id, created_at: start_date..end_date)
self.query_based_on_granularity(query, granularity)
end

# 2) Number of tipline requests
def self.tipline_requests(team_id, start_date, end_date, granularity = nil)
start_date, end_date = self.parse_start_end_dates(start_date, end_date)
query = TiplineRequest.where(team_id: team_id, created_at: start_date..end_date)
self.query_based_on_granularity(query, granularity)
end

# 3) Number of tipline requests grouped by type of search result
def self.tipline_requests_by_search_type(team_id, start_date, end_date)
start_date, end_date = self.parse_start_end_dates(start_date, end_date)
TiplineRequest.where(
team_id: team_id,
smooch_request_type: ['relevant_search_result_requests', 'irrelevant_search_result_requests'],
melsawy marked this conversation as resolved.
Show resolved Hide resolved
created_at: start_date..end_date,
).group('smooch_request_type').count
end

# 4) Number of Subscribers
def self.tipline_subscriptions(team_id, start_date, end_date, granularity = nil)
start_date, end_date = self.parse_start_end_dates(start_date, end_date)
query = TiplineSubscription.where(team_id: team_id, created_at: start_date..end_date)
self.query_based_on_granularity(query, granularity)
end

# 5) Number of Newsletters sent
def self.newsletters_sent(team_id, start_date, end_date, granularity = nil)
start_date, end_date = self.parse_start_end_dates(start_date, end_date)
query = TiplineNewsletterDelivery
.joins("INNER JOIN tipline_newsletters tnl ON tipline_newsletter_deliveries.tipline_newsletter_id = tnl.id")
melsawy marked this conversation as resolved.
Show resolved Hide resolved
.where('tnl.team_id = ?', team_id)
.where(created_at: start_date..end_date)
self.query_based_on_granularity(query, granularity, 'tipline_newsletter_deliveries.created_at')
end

# 6) Number of Media received, by type
def self.media_received_by_type(team_id, start_date, end_date)
bot = BotUser.smooch_user
start_date, end_date = self.parse_start_end_dates(start_date, end_date)
query = ProjectMedia.where(team_id: team_id, user_id: bot.id, created_at: start_date..end_date)
.joins(:media).group('medias.type')
query.count
melsawy marked this conversation as resolved.
Show resolved Hide resolved
end

# Top clusters
def self.top_clusters(team_id, start_date, end_date)

end

caiosba marked this conversation as resolved.
Show resolved Hide resolved
def self.parse_start_end_dates(start_date, end_date)
# date format is `2023-08-23`
start_date = Time.parse(start_date)
end_date = Time.parse(end_date)
raise 'End date should be greater than start date' if start_date > end_date
melsawy marked this conversation as resolved.
Show resolved Hide resolved
return start_date, end_date
end

def self.query_based_on_granularity(query, granularity, col_name = 'created_at')
# For PG the allowed values for granularity can be one of the following
# [millennium, century, decade, year, quarter, month, week, day, hour,
# minute, second, milliseconds, microseconds]
# But I'll limit the value to the following [year, quarter, month, week, day]
if %w(year quarter month week day).include?(granularity)
query.group("date_trunc('#{granularity}', #{col_name})").count
github-advanced-security[bot] marked this conversation as resolved.
Fixed
Show resolved Hide resolved
Fixed Show fixed Hide fixed
else
query.count
end
end
end
Loading