Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CV2-5138: Data Dashboard: Write and analyze queries #2033

Merged
merged 22 commits into from
Sep 25, 2024

Conversation

melsawy
Copy link
Contributor

@melsawy melsawy commented Sep 12, 2024

Description

Add a class for data points

  • Number of tipline messages (optionally grouped by some granularity, like, by day)
  • Number of tipline requests (optionally grouped by some granularity, like, by day)
  • Number of tipline requests grouped by type of search result
  • Subscribers
  • Newsletters sent
  • Media received, by type
  • Top clusters
  • Top media tags
  • Articles sent: Total number can be retrieved by
  • Average response time
  • Number of users (new x returning)

References: CV2-5138

How has this been tested?

Implemented automated tests.

Things to pay attention to during code review

Please describe parts of the change that require extra attention during code review, for example:

  • File FFFF, line LL: This refactoring does this and this. Is it consistent with how it’s implemented elsewhere?
  • Etc.

Checklist

  • I have performed a self-review of my own code
  • I have added unit and feature tests, if the PR implements a new feature or otherwise would benefit from additional testing
  • I have added regression tests, if the PR fixes a bug
  • I have added logging, exception reporting, and custom tracing with any additional information required for debugging
  • I considered secure coding practices when writing this code. Any security concerns are noted above.
  • I have commented my code in hard-to-understand areas, if any
  • I have made needed changes to the README
  • My changes generate no new warnings
  • If I added a third party module, I included a rationale for doing so and followed our current guidelines

lib/check_data_points.rb Outdated Show resolved Hide resolved
lib/check_data_points.rb Outdated Show resolved Hide resolved
lib/check_data_points.rb Outdated Show resolved Hide resolved
lib/check_data_points.rb Outdated Show resolved Hide resolved
lib/check_data_points.rb Fixed Show resolved Hide resolved
lib/check_data_points.rb Outdated Show resolved Hide resolved
lib/check_data_points.rb Outdated Show resolved Hide resolved
lib/check_data_points.rb Outdated Show resolved Hide resolved
lib/check_data_points.rb Show resolved Hide resolved
lib/check_data_points.rb Outdated Show resolved Hide resolved
lib/check_data_points.rb Fixed Show resolved Hide resolved
lib/check_data_points.rb Fixed Show fixed Hide fixed
lib/check_data_points.rb Outdated Show resolved Hide resolved
smooch_request_type: SEARCH_RESULT_TYPES,
created_at: start_date..end_date,
).count
c2 = TiplineRequest
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: For clarity, please rename this variable and/or add a comment that explains that "this is the number of articles sent as reports".

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

created_at: start_date..end_date,
).count
c2 = TiplineRequest
.where(smooch_request_type: ["default_requests", "timeout_requests"])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@melsawy , I was thinking more about it, I think that we don't need this condition here. Because for a situation that a search result was returned but then an updated report was sent, I think we still want to count that.

@brianfleming , if you can confirm, please... imagine this situation:

  1. User sends a query
  2. The query returns a search result - this increments the statistics of "articles sent" by 1
  3. That same report is updated later on, and the user receives an update report - should this also increment "articles sent" by 1?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes @caiosba I agree. It matches with another discussion we had on analytics counting. starting with counting all the things now is a good idea. later if we need to get more specific we can

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Comment on lines 142 to 155
def elastic_search_top_items(team_id, start_date, end_date, limit, with_tags = false)
data = {}
query = {
range: {'created_at': { start_time: start_date, end_time: end_date } },
demand: { min: 1 },
sort: 'demand',
eslimit: limit
}
query[:tags_as_sentence] = { min: 1 } if with_tags
result = CheckSearch.new(query.to_json, nil, team_id)
result.medias.each{|pm| data[pm.id] = pm.demand }
data
end
end
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

Copy link
Contributor

@caiosba caiosba left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Sawy, looks great so far!

@melsawy melsawy marked this pull request as ready for review September 24, 2024 08:28
Copy link
Contributor

@caiosba caiosba left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, thanks Sawy! Just one thing: before merging, please re-generate the schema.rb file according to latest develop... right now it has more stuff than it needs.

@melsawy
Copy link
Contributor Author

melsawy commented Sep 24, 2024

Looks good to me, thanks Sawy! Just one thing: before merging, please re-generate the schema.rb file according to latest develop... right now it has more stuff than it needs.

Done

@melsawy melsawy merged commit 00f575f into develop Sep 25, 2024
10 checks passed
@melsawy melsawy deleted the CV2-5138-data-dashboard-write-and-analyze-queries branch September 25, 2024 05:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants