Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Media 2021 #2144

Closed
6 tasks done
rviscomi opened this issue Apr 27, 2021 · 39 comments · Fixed by #2717
Closed
6 tasks done

Media 2021 #2144

rviscomi opened this issue Apr 27, 2021 · 39 comments · Fixed by #2717
Assignees
Labels
2021 chapter Tracking issue for a 2021 chapter

Comments

@rviscomi
Copy link
Member

rviscomi commented Apr 27, 2021

Part I Chapter 5: Media

Media illustration

If you're interested in contributing to the Media chapter of the 2021 Web Almanac, please reply to this issue and indicate which role or roles best fit your interest and availability: author, reviewer, analyst, and/or editor.

Content team

Lead Authors Reviewers Analysts Editors Coordinator
@eeeps @eeeps @dougsillars @Navaneeth-akam @akshay-ranganath @tpiros @addyosmani @eeeps @dougsillars @akshay-ranganath - @rviscomi
Expand for more information about each role
  • The content team lead is the chapter owner and responsible for setting the scope of the chapter and managing contributors' day-to-day progress.
  • Authors are subject matter experts and lead the content direction for each chapter. Chapters typically have one or two authors. Authors are responsible for planning the outline of the chapter, analyzing stats and trends, and writing the annual report.
  • Reviewers are also subject matter experts and assist authors with technical reviews during the planning, analyzing, and writing phases.
  • Analysts are responsible for researching the stats and trends used throughout the Almanac. Analysts work closely with authors and reviewers during the planning phase to give direction on the types of stats that are possible from the dataset, and during the analyzing/writing phases to ensure that the stats are used correctly.
  • Editors are technical writers who have a penchant for both technical and non-technical content correctness. Editors have a mastery of the English language and work closely with authors to help wordsmith content and ensure that everything fits together as a cohesive unit.
  • The section coordinator is the overall owner for all chapters within a section like "User Experience" or "Page Content" and helps to keep each chapter on schedule.

Note: The time commitment for each role varies by the chapter's scope and complexity as well as the number of contributors.

For an overview of how the roles work together at each phase of the project, see the Chapter Lifecycle doc.

Milestone checklist

0. Form the content team

  • May 31: The content team has at least one author, reviewer, and analyst

1. Plan content

  • June 15 The content team has completed the chapter outline in the draft doc

2. Gather data

  • June 30: Analysts have added all necessary custom metrics and drafted a PR (example) to track query progress
  • July 1 - 31: HTTP Archive runs the July crawl

3. Validate results

  • September 30: Analysts have queried all metrics and saved the output to the results sheet

4. Draft content

  • October 31: The content team has written, reviewed, and edited the chapter in the doc

5. Publication

  • November 15: The completed chapter and all required metadata and figures are converted to markdown and submitted to GitHub
  • December 1: Target launch date 🚀

Chapter resources

Refer to these 2021 Media resources throughout the content creation process:

📄 Google Docs for outlining and drafting content
🔍 SQL files for committing the queries used during analysis
📊 Google Sheets for saving the results of queries
📝 Markdown file for publishing content and managing public metadata

@rviscomi rviscomi added 2021 chapter Tracking issue for a 2021 chapter help wanted Extra attention is needed labels Apr 27, 2021
@IsaacSichangi
Copy link

hello id like to take up an analyst role in the media chapter

@boosef
Copy link

boosef commented Apr 27, 2021

I can help review and/or edit! :)

@eeeps
Copy link
Contributor

eeeps commented Apr 28, 2021

I reviewed in 2019 and pitched in as a last-minute author in 2020. I would love to be involved at an earlier stage, this year, either as an author or reviewer.

@Navaneeth-akam
Copy link
Contributor

I was a reviewer for media in 2020 and would like to be reviewer for this year as well. Along with that, i would like to take up the editor role as well

@akshay-ranganath
Copy link
Contributor

Hi Rick,

I'd like to help build this chapter. I am a Solution Architect at Cloudinary and used to be a web performance consultant at Akamai prior to my current role.

Thanks!

@rviscomi
Copy link
Member Author

Thanks everyone! Feel free to start brainstorming content for this chapter by requesting edit access to the doc and adding your thoughts.

@rviscomi
Copy link
Member Author

@TrevorFSmith mentioned that WebXR might be an interesting topic to explore this year. As a follow-up to the question of what can be quantified, does anyone know of use counters that track WebXR usage? For example XRDOMOverlay seems like it may be relevant. The top chart shows the percent of page loads in Chrome that make use of that feature. The bottom chart is empty in this case, indicating that none of the pages in HTTP Archive use it, or that we're unable to trigger the conditions needed to use it (like if it depends on real user interactions).

@TrevorFSmith @Yonet would either of you be interested in authoring/reviewing this chapter? And would your focus strictly be WebXR or might you be interested in Media topics as a whole (images, video, etc)?

@TrevorFSmith
Copy link

I found a few relevant use counters:

Some of the old XR counters are for WebVR which is no longer shipping in Chromium-based browsers. WebXR is the actually standardized replacement for WebVR. The old API counters trend toward zero like XRRequestDevice.

I'm interested in editing the XR-related parts of the almanac (or co-editing with @Yonet).

@rviscomi
Copy link
Member Author

rviscomi commented May 3, 2021

Thanks @TrevorFSmith those use counters are a good place to start. Could you add a WebXR section to the chapter outline and list these use counters in the metrics section of the chapter doc?

Editing is a non-technical role—do you mean authoring or reviewing?

@rockeynebhwani
Copy link
Contributor

rockeynebhwani commented May 4, 2021

@TrevorFSmith @rviscomi - Last year, I had added detection of webAR in Wappalyzer to track usage on eCommerce sites but on eCommerce sites uses tend to be on Product Detail page which we don't crawl so I didn't end up covering this in the chapter.
https://github.com/AliasIO/wappalyzer/issues/3216

@rviscomi
Copy link
Member Author

rviscomi commented May 4, 2021

@eeeps thanks for your interest in authoring this chapter! As the content team lead, you'll be responsible for the scope and direction of the chapter and keeping it on schedule. We automatically monitor the staffing and progress of each chapter based on the state of the initial comment so please keep that updated as you add new contributors and meet each milestone.

We've created a Google Doc for this chapter, which you're encouraged to use to collaborate with the content team on the initial outline, metrics, and ultimately the final draft.

Next steps for this chapter are:

There's not currently a section coordinator for this chapter, so I'll be periodically checking in with you directly to make sure the chapter is staying on schedule. Reach out here in this issue if you have any questions about the process.

More information about the content team lead and author roles and responsibilities are available for reference in the wiki if needed.

To anyone else interested in contributing to this chapter, please comment below to join the team!

@rviscomi rviscomi removed the help wanted Extra attention is needed label May 4, 2021
@Yonet
Copy link

Yonet commented May 5, 2021

@rviscomi @TrevorFSmith and I want to be authors. I've asked for edit access for both of us for the Media chapter doc to add WebXR and metrics section.

If someone else would like to contribute to Media chapter as an author and I am happy to contribute as a reviewer as well.

@rviscomi
Copy link
Member Author

rviscomi commented May 5, 2021

Ok great. thanks @Yonet and @TrevorFSmith, I've given you both edit access to the doc.

@eeeps is now the content team lead and is in charge of assigning authors/reviewers as needed, so I'll defer to them on the final call. If WebXR will be a big part of this chapter I'd recommended adding one or both as coauthors. @eeeps could you update the contributor info in the top comment of this issue with your decision either way?

@rviscomi
Copy link
Member Author

rviscomi commented May 5, 2021

ℹ️ FYI 2019/2020 contributors: @dougsillars @colinbendell @ahmadawais @kornelski @tpiros @bseymour @nhoizey @smatei

This year's Media chapter is looking pretty good in terms of staffing, but I wanted to ping you all in case you'd still like to contribute as reviewers or analysts. It's great to have input from past years' contributors. Let us know!

@dougsillars
Copy link
Contributor

Rick, I'd be happy to focus on video as a part of the media chapter.

@tpiros
Copy link
Contributor

tpiros commented May 6, 2021

@rviscomi 👋 I'd be happy to sign up as a reviewer this year, if that's an option!

@rviscomi
Copy link
Member Author

rviscomi commented May 6, 2021

@dougsillars @tpiros great to have you both back! I'll defer to @eeeps as content team lead to loop you both in.

@rviscomi
Copy link
Member Author

rviscomi commented May 11, 2021

Hi @eeeps just checking in. Here are some tips to help keep the chapter on track:

  • Coordinate with the rest of the content team to request edit access to the doc and start brainstorming an outline for the chapter
  • Edit the top comment to keep the chapter metadata in sync with all coauthors/reviewers/analysts and also any completed milestones (looks like Milestone 0 is ready to be checked off and Tamas can be added as a reviewer—this is helpful for us to monitor progress at a glance in 2021 Chapter Progress #2179)

Also as an FYI for the content team, we have an existing #web-almanac-media channel on Slack from last year. So if you'd find it useful to coordinate on there, feel free to get that going again.

@rviscomi rviscomi mentioned this issue May 11, 2021
6 tasks
@jyrkialakuijala
Copy link

jyrkialakuijala commented Jul 18, 2021

Could we have a decomposition of image quality in the media analysis? This would give us guidance for developing and optimizing new image formats and make recommendations on their use. Currently there is a lot of confusion on what image qualities would actually matter on the web, and different teams are optimizing different ends of the quality spectrum.

Tracing what is the quality (or just bpp if we are unable to decide about quality) of an image byte in the internet would be one way to achieve this. Similar to this: https://www.spiedigitallibrary.org/ContentImages/Proceedings/11137/111370K/FigureImages/00163_PSISDG11137_111370K_page_3_1.jpg

Seeing the long trends over years on quality of images -- if it is increasing or decreasing, what are the differences in optimized vs. unoptimized sites, etc. would be interesting to me.

@eeeps
Copy link
Contributor

eeeps commented Jul 23, 2021

@jyrkialakuijala I was able to get a BPP custom metric in place (just) in time this year, so we'll be able to, for the first time, talk about the BPP of all images surveyed in this year's crawl!

Lacking a time machine, we do not have this information for previous years' crawls.

I've brainstormed the idea of taking a statistically significant random sample of crawled image URLs, re-downloading the actual image resources, and getting various quality metrics from those. That kind of analysis was way beyond what the crawler (and custom metric hooks) can handle, though. If you know anyone who could help with this – that would be amazing.

But -- I can't wait to dive into the BPP data, which should be fascinating all on its own.

@eeeps
Copy link
Contributor

eeeps commented Jul 23, 2021

@addyosmani:

Perhaps it would be interesting this year to look at whether there are correlations between the usage of deployed modern image formats (WebP, AVIF) and Largest Contentful Paint?

After talking with @siakaramalegos (who's heading up the Performance chapter), we decided a good rule of thumb for how to divide up subjects which deal with both images and performance was: if the topic is about how images are affecting whole page performance (e.g., as measured by CWV), Performance gets first dibs; if it's about how images themselves are performing, Media gets first dibs. With a healthy dose of cross-linking where appropriate.

If Performance doesn't tackle this correlation, I'd love to!

eeeps added a commit to eeeps/almanac.httparchive.org that referenced this issue Aug 13, 2021
@eeeps eeeps mentioned this issue Aug 13, 2021
29 tasks
@eeeps
Copy link
Contributor

eeeps commented Aug 13, 2021

August 13 update:

  • @dougsillars and I have (finally!) completed the Images and Video sections of the outline.
  • I haven't been able to get in touch with our analyst, @heymarkreeves, after a couple of weeks of trying. @dougsillars and I will start the analysis work ourselves, and keep an ear to the ground for any help we could get.
  • @TrevorFSmith let me know that he won't be able to contribute any WebXR content. I reached out to @Yonet to ask if they would be able to continue on this section solo, but haven't heard back. Not dropping them as an author quite yet.
  • I've drafted a (so far empty) PR to track progress on our queries, linked directly above this post.

If any of our reviewers (@boosef @Navaneeth-akam @akshay-ranganath @tpiros) want to take a gander at the outline and leave comments, please do!

Otherwise, time for @dougsillars and I (and anyone else who'd like to pitch in!) to get cracking on the queries.

@rviscomi
Copy link
Member Author

👋 Hi @eeeps @Yonet @dougsillars @IsaacSichangi @heymarkreeves, just checking in on the chapter progress. How is the analysis coming along?

@rviscomi rviscomi removed the ASAP This issue is blocking progress label Sep 20, 2021
@rviscomi rviscomi mentioned this issue Nov 24, 2021
29 tasks
rviscomi added a commit that referenced this issue Nov 24, 2021
* Media 2021 queries (#2144)

* Test query for 'encoding -> format use'

* Percent of pages with picture elements, and distribution of number of picture elements per page

* Simplify query, based on advice from Kevin Farrugia

* Basic script for counting no of images that use lazy loading

* Replaced JSON_EXTRACT_SCALAR with JSON_VALUE as the formor is depracated

* Adding the query to pull in details about the usage of , ,  and resposive dimension specification based on new custom metrics

* Copying and updating last year's media queries

* Adding a file from Performance chapter and tweaking it to report image dimension by industry vertical

* Updated the alt query to also count images that have  attribute

* Removing reference to "decode=lazy" from alt tag SQ; creating a new query for decode usage

* removing a condition that is not valid for this query

* correcting name of a output query field to indicate %

* Add extra totals columns, query July data

* Fix for loop; 'lazy' → 'async'

* Add average image query

* Deal with nulls and INT64 limits

* Adding a query to find the usage of image cdns

* Adding a SQL to count cross domain image requests

* Bits per pixel, by format

* A few fixes to BPP/format query

* Align on distribution percentiles

* Copying Colin's query for top media queries from 2019 SQL code base

* Chaged the description of the query

* Corrected the looping construct to make it more readable

* Two bytes and dimensions queries to rule them all

* Img elements with one and zero pixel resources
...excluded from other analysis

* Top aspect ratios query

* Remove unnecessary queries

* Portriat / aspect ratio / square query

* Comment out smallImageCount and bigImageCount constraint
...which was making it return no results?

* Most common sizes values

* Sizes implicit vs explicit, and parse errors

* .sql

* linting

* lint

* lint

* lint

* remaining queries

* lint

* 1x1 and 0x0

Co-authored-by: Eric Portis <[email protected]>
Co-authored-by: Akshay Ranganath <[email protected]>
@eeeps
Copy link
Contributor

eeeps commented Dec 1, 2021

Review team! @boosef @Navaneeth-akam @akshay-ranganath @tpiros

Sincere apologies for the late, and last-minute call for review, but: we finally have a full first draft of the chapter in the Google Doc, and it is finally in a reviewable state.

Due to my own missed deadlines, we're going to miss the launch tomorrow. But we're hoping to publish the chapter as soon as possible, so the sooner you have a chance to review, the better.

Thank you!

@tpiros
Copy link
Contributor

tpiros commented Dec 1, 2021 via email

@tpiros
Copy link
Contributor

tpiros commented Dec 1, 2021 via email

@Navaneeth-akam
Copy link
Contributor

Review team! @boosef @Navaneeth-akam @akshay-ranganath @tpiros

Sincere apologies for the late, and last-minute call for review, but: we finally have a full first draft of the chapter in the Google Doc, and it is finally in a reviewable state.

Due to my own missed deadlines, we're going to miss the launch tomorrow. But we're hoping to publish the chapter as soon as possible, so the sooner you have a chance to review, the better.

Thank you!

Will take a look at it and add comments

@tunetheweb
Copy link
Member

@eeeps @dougsillars @Navaneeth-akam @akshay-ranganath @tpiros @addyosmani

🎉 This chapter is fully written, reviewed, edited, and ready to be launched! Thank you to all of the contributors who put in the time and effort to make this a great chapter.

When you get 5 minutes, I'd really appreciate if you could fill out our contributor survey to tell us (the project leads) about your experience. It's super helpful to hear what went well or what could be improved for next time. 🙏

Congratulations and thank you all again. I'm excited for this to launch soon!

@tpiros
Copy link
Contributor

tpiros commented Dec 19, 2021

thank you @tunetheweb for all your help this year as well! ❤️ 🤗

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2021 chapter Tracking issue for a 2021 chapter
Projects
None yet
Development

Successfully merging a pull request may close this issue.