diff --git a/README.md b/README.md index c5977ab..6e061cc 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,8 @@ ## 📖 Introduction Java library which allows you to retrieve subtitles/transcripts for a YouTube video. -It supports manual and automatically generated subtitles and does not use headless browser for scraping. +It supports manual and automatically generated subtitles, bulk transcript retrieval for all videos in the playlist or +on the channel and does not use headless browser for scraping. Inspired by [Python library](https://github.com/jdepoix/youtube-transcript-api). ## 🤖 Features @@ -21,6 +22,8 @@ Inspired by [Python library](https://github.com/jdepoix/youtube-transcript-api). ✅ Automatically generated transcripts retrieval +✅ Bulk transcript retrieval for all videos in the playlist or channel + ✅ Transcript translation ✅ Transcript formatting @@ -79,7 +82,7 @@ TranscriptList transcriptList = youtubeTranscriptApi.listTranscripts("videoId"); // Iterate over transcript list for(Transcript transcript : transcriptList) { - System.out.println(transcript); + System.out.println(transcript); } // Find transcript in specific language @@ -143,6 +146,8 @@ TranscriptContent transcriptContent = youtubeTranscriptApi.listTranscripts("vide TranscriptContent transcriptContent = youtubeTranscriptApi.getTranscript("videoId"); ``` +For bulk transcript retrieval see [Bulk Transcript Retrieval](#bulk-transcript-retrieval). + ## 🔧 Detailed Usage ### Use fallback language @@ -241,7 +246,7 @@ TranscriptFormatter jsonFormatter = TranscriptFormatters.jsonFormatter(); String formattedContent = jsonFormatter.format(transcriptContent); ```` -### YoutubeClient customization +### YoutubeClient Customization By default, `YoutubeTranscriptApi` uses Java 11 HttpClient for making requests to YouTube, if you want to use a different client, @@ -275,6 +280,52 @@ TranscriptList transcriptList = youtubeTranscriptApi.listTranscriptsWithCookies( TranscriptContent transcriptContent = youtubeTranscriptApi.getTranscriptWithCookies("videoId", "path/to/cookies.txt", "en"); ``` +### Bulk Transcript Retrieval + +All bulk transcript retrieval operations are done via the `PlaylistsTranscriptApi` interface. Same as with the +`YoutubeTranscriptApi`, +you can create a new instance of the PlaylistsTranscriptApi by calling the `createDefaultPlaylistsApi` method of the +`TranscriptApiFactory`. +Playlists and channels information is retrieved from +the [YouTube V3 API](https://developers.google.com/youtube/v3/docs/), +so you will need to provide API key for all methods. + +```java +// Create a new default PlaylistsTranscriptApi instance +PlaylistsTranscriptApi playlistsTranscriptApi = TranscriptApiFactory.createDefaultPlaylistsApi(); + +// Retrieve all available transcripts for a given playlist +Map transcriptLists = playlistsTranscriptApi.listTranscriptsForPlaylist("playlistId", "apiKey", true); + +// Retrieve all available transcripts for a given channel +Map transcriptLists = playlistsTranscriptApi.listTranscriptsForChannel("channelName", "apiKey", true); +``` + +As you can see, there is also a boolean flag `continueOnError`, which tells whether to continue if transcript retrieval +fails for a video or not. For example, if it's set to `true`, all transcripts that could not be retrieved will be +skipped, if +it's set to `false`, operation will fail fast on the first error. + +All methods are also have overloaded versions which accept path to [cookies.txt](#cookies) file. + +```java +// Retrieve all available transcripts for a given playlist +Map transcriptLists = playlistsTranscriptApi.listTranscriptsForPlaylist( + "playlistId", + "apiKey", + true, + "path/to/cookies.txt" +); + +// Retrieve all available transcripts for a given channel +Map transcriptLists = playlistsTranscriptApi.listTranscriptsForChannel( + "channelName", + "apiKey", + true, + "path/to/cookies.txt" +); +``` + ## 🤓 How it works Within each YouTube video page, there exists JSON data containing all the transcript information, including an diff --git a/lib/src/main/java/io/github/thoroldvix/api/PlaylistsTranscriptApi.java b/lib/src/main/java/io/github/thoroldvix/api/PlaylistsTranscriptApi.java new file mode 100644 index 0000000..ca07a14 --- /dev/null +++ b/lib/src/main/java/io/github/thoroldvix/api/PlaylistsTranscriptApi.java @@ -0,0 +1,71 @@ +package io.github.thoroldvix.api; + +import io.github.thoroldvix.internal.TranscriptApiFactory; + +import java.util.Map; + +/** + * Retrieves transcripts for all videos in a playlist, or all videos for a specific channel. + *

+ * Playlists and channel videos are retrieved from the YouTube API, so you will need to have a valid api key to use this. + *

+ *

+ * To get implementation for this interface see {@link TranscriptApiFactory} + *

+ */ +public interface PlaylistsTranscriptApi { + + /** + * Retrieves transcript lists for all videos in the specified playlist using provided API key and cookies file from a specified path. + * + * @param playlistId The ID of the playlist + * @param apiKey API key for the YouTube V3 API (see Getting started) + * @param continueOnError Whether to continue if transcript retrieval fails for a video. If true, all transcripts that could not be retrieved will be skipped, + * otherwise an exception will be thrown. + * @param cookiesPath The file path to the text file containing the authentication cookies. Used in the case if some videos are age restricted see {Cookies} + * @return A map of video IDs to {@link TranscriptList} objects + * @throws TranscriptRetrievalException If the retrieval of the transcript lists fails + */ + Map listTranscriptsForPlaylist(String playlistId, String apiKey, String cookiesPath, boolean continueOnError) throws TranscriptRetrievalException; + + + /** + * Retrieves transcript lists for all videos in the specified playlist using provided API key. + * + * @param playlistId The ID of the playlist + * @param apiKey API key for the YouTube V3 API (see Getting started) + * @param continueOnError Whether to continue if transcript retrieval fails for a video. If true, all transcripts that could not be retrieved will be skipped, + * otherwise an exception will be thrown. + * @return A map of video IDs to {@link TranscriptList} objects + * @throws TranscriptRetrievalException If the retrieval of the transcript lists fails + */ + Map listTranscriptsForPlaylist(String playlistId, String apiKey, boolean continueOnError) throws TranscriptRetrievalException; + + + /** + * Retrieves transcript lists for all videos for the specified channel using provided API key and cookies file from a specified path. + * + * @param channelName The name of the channel + * @param apiKey API key for the YouTube V3 API (see Getting started) + * @param cookiesPath The file path to the text file containing the authentication cookies. Used in the case if some videos are age restricted see {Cookies} + * @param continueOnError Whether to continue if transcript retrieval fails for a video. If true, all transcripts that could not be retrieved will be skipped, + * otherwise an exception will be thrown. + * @return A map of video IDs to {@link TranscriptList} objects + * @throws TranscriptRetrievalException If the retrieval of the transcript lists fails + * @throws TranscriptRetrievalException If the retrieval of the transcript lists fails + */ + Map listTranscriptsForChannel(String channelName, String apiKey, String cookiesPath, boolean continueOnError) throws TranscriptRetrievalException; + + + /** + * Retrieves transcript lists for all videos for the specified channel using provided API key. + * + * @param channelName The name of the channel + * @param apiKey API key for the YouTube V3 API (see Getting started) + * @param continueOnError Whether to continue if transcript retrieval fails for a video. If true, all transcripts that could not be retrieved will be skipped, + * otherwise an exception will be thrown. + * @return A map of video IDs to {@link TranscriptList} objects + * @throws TranscriptRetrievalException If the retrieval of the transcript lists fails + */ + Map listTranscriptsForChannel(String channelName, String apiKey, boolean continueOnError) throws TranscriptRetrievalException; +} diff --git a/lib/src/main/java/io/github/thoroldvix/api/TranscriptList.java b/lib/src/main/java/io/github/thoroldvix/api/TranscriptList.java index e14205d..4ce3e34 100644 --- a/lib/src/main/java/io/github/thoroldvix/api/TranscriptList.java +++ b/lib/src/main/java/io/github/thoroldvix/api/TranscriptList.java @@ -56,6 +56,13 @@ public interface TranscriptList extends Iterable { */ Transcript findManualTranscript(String... languageCodes) throws TranscriptRetrievalException; + /** + * Retrieves the ID of the video to which transcript was retrieved. + * + * @return The video ID. + */ + String getVideoId(); + @Override default void forEach(Consumer action) { Iterable.super.forEach(action); diff --git a/lib/src/main/java/io/github/thoroldvix/api/TranscriptRetrievalException.java b/lib/src/main/java/io/github/thoroldvix/api/TranscriptRetrievalException.java index d7d64e9..9a5c956 100644 --- a/lib/src/main/java/io/github/thoroldvix/api/TranscriptRetrievalException.java +++ b/lib/src/main/java/io/github/thoroldvix/api/TranscriptRetrievalException.java @@ -10,7 +10,7 @@ public class TranscriptRetrievalException extends Exception { private static final String ERROR_MESSAGE = "Could not retrieve transcript for the video: %s.\nReason: %s"; private static final String YOUTUBE_WATCH_URL = "https://www.youtube.com/watch?v="; - private final String videoId; + private String videoId; /** * Constructs a new exception with the specified detail message and cause. @@ -36,10 +36,22 @@ public TranscriptRetrievalException(String videoId, String message) { } /** - * @return The ID of the video for which the transcript retrieval failed. + * Constructs a new exception with the specified detail message and cause. + * + * @param message The detail message explaining the reason for the failure. + * @param cause The cause of the failure (which is saved for later retrieval by the {@link Throwable#getCause()} method). */ - public String getVideoId() { - return videoId; + public TranscriptRetrievalException(String message, Throwable cause) { + super(message, cause); + } + + /** + * Constructs a new exception with the specified detail message. + * + * @param message The detail message explaining the reason for the failure. + */ + public TranscriptRetrievalException(String message) { + super(message); } /** @@ -53,5 +65,12 @@ private static String buildErrorMessage(String videoId, String message) { String videoUrl = YOUTUBE_WATCH_URL + videoId; return String.format(ERROR_MESSAGE, videoUrl, message); } + + /** + * @return The ID of the video for which the transcript retrieval failed. + */ + public String getVideoId() { + return videoId; + } } diff --git a/lib/src/main/java/io/github/thoroldvix/api/YoutubeClient.java b/lib/src/main/java/io/github/thoroldvix/api/YoutubeClient.java index 23893a7..f9ca255 100644 --- a/lib/src/main/java/io/github/thoroldvix/api/YoutubeClient.java +++ b/lib/src/main/java/io/github/thoroldvix/api/YoutubeClient.java @@ -6,7 +6,6 @@ /** * Responsible for sending GET requests to YouTube. */ -@FunctionalInterface public interface YoutubeClient { /** @@ -18,5 +17,16 @@ public interface YoutubeClient { * @throws TranscriptRetrievalException If the request to YouTube fails. */ String get(String url, Map headers) throws TranscriptRetrievalException; + + + /** + * Sends a GET request to the specified endpoint and returns the response body. + * + * @param endpoint The endpoint to which the GET request is made. + * @param params A map of parameters to include in the request. + * @return The body of the response as a {@link String}. + * @throws TranscriptRetrievalException If the request to YouTube fails. + */ + String get(YtApiV3Endpoint endpoint, Map params) throws TranscriptRetrievalException; } diff --git a/lib/src/main/java/io/github/thoroldvix/api/YtApiV3Endpoint.java b/lib/src/main/java/io/github/thoroldvix/api/YtApiV3Endpoint.java new file mode 100644 index 0000000..bd2273e --- /dev/null +++ b/lib/src/main/java/io/github/thoroldvix/api/YtApiV3Endpoint.java @@ -0,0 +1,28 @@ +package io.github.thoroldvix.api; + +/** + * The YouTube API V3 endpoints. Used by the {@link YoutubeClient}. + */ +public enum YtApiV3Endpoint { + PLAYLIST_ITEMS("playlistItems"), + SEARCH("search"), + CHANNELS("channels"); + private final static String YOUTUBE_API_V3_BASE_URL = "https://www.googleapis.com/youtube/v3/"; + + private final String resource; + private final String url; + + YtApiV3Endpoint(String resource) { + this.url = YOUTUBE_API_V3_BASE_URL + resource; + this.resource = resource; + } + + public String url() { + return url; + } + + @Override + public String toString() { + return resource; + } +} diff --git a/lib/src/main/java/io/github/thoroldvix/internal/DefaultPlaylistsTranscriptApi.java b/lib/src/main/java/io/github/thoroldvix/internal/DefaultPlaylistsTranscriptApi.java new file mode 100644 index 0000000..503fa8d --- /dev/null +++ b/lib/src/main/java/io/github/thoroldvix/internal/DefaultPlaylistsTranscriptApi.java @@ -0,0 +1,194 @@ +package io.github.thoroldvix.internal; + +import com.fasterxml.jackson.core.JsonProcessingException; +import com.fasterxml.jackson.databind.JsonNode; +import com.fasterxml.jackson.databind.ObjectMapper; +import io.github.thoroldvix.api.*; + +import java.util.ArrayList; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.concurrent.ExecutionException; +import java.util.concurrent.ExecutorService; +import java.util.concurrent.Executors; +import java.util.concurrent.Future; + +import static io.github.thoroldvix.api.YtApiV3Endpoint.*; + +/** + * Default implementation of {@link PlaylistsTranscriptApi} + */ +class DefaultPlaylistsTranscriptApi implements PlaylistsTranscriptApi { + private final YoutubeClient client; + private final YoutubeTranscriptApi youtubeTranscriptApi; + private final ObjectMapper objectMapper; + + DefaultPlaylistsTranscriptApi(YoutubeClient client, YoutubeTranscriptApi youtubeTranscriptApi) { + this.client = client; + this.objectMapper = new ObjectMapper(); + this.youtubeTranscriptApi = youtubeTranscriptApi; + } + + @Override + public Map listTranscriptsForPlaylist(String playlistId, String apiKey, String cookiesPath, boolean continueOnError) throws TranscriptRetrievalException { + Map transcriptLists = new HashMap<>(); + List videoIds = getVideoIds(playlistId, apiKey); + ExecutorService executor = Executors.newCachedThreadPool(); + + List> futures = new ArrayList<>(); + + for (String videoId : videoIds) { + futures.add(executor.submit(() -> { + try { + return getTranscriptList(videoId, cookiesPath); + } catch (TranscriptRetrievalException e) { + if (!continueOnError) throw e; + return null; + } + })); + } + + try { + for (Future future : futures) { + try { + TranscriptList transcriptList = future.get(); + if (transcriptList != null) { + transcriptLists.put(transcriptList.getVideoId(), transcriptList); + } + } catch (ExecutionException e) { + if (!continueOnError) { + executor.shutdownNow(); + throw new TranscriptRetrievalException("Failed to retrieve transcripts for playlist: " + playlistId, e); + } + } catch (InterruptedException e) { + Thread.currentThread().interrupt(); + executor.shutdownNow(); + throw new TranscriptRetrievalException("Failed to retrieve transcripts for playlist: " + playlistId, e); + } + } + } finally { + executor.shutdownNow(); + } + + return transcriptLists; + } + + private TranscriptList getTranscriptList(String videoId, String cookiesPath) throws TranscriptRetrievalException { + if (cookiesPath != null) { + return youtubeTranscriptApi.listTranscriptsWithCookies(videoId, cookiesPath); + } + return youtubeTranscriptApi.listTranscripts(videoId); + } + + @Override + public Map listTranscriptsForPlaylist(String playlistId, String apiKey, boolean continueOnError) throws TranscriptRetrievalException { + return listTranscriptsForPlaylist(playlistId, apiKey, null, continueOnError); + } + + @Override + public Map listTranscriptsForChannel(String channelName, String apiKey, String cookiesPath, boolean continueOnError) throws TranscriptRetrievalException { + String channelId = getChannelId(channelName, apiKey); + String channelPlaylistId = getChannelPlaylistId(channelId, apiKey); + return listTranscriptsForPlaylist(channelPlaylistId, apiKey, cookiesPath, continueOnError); + } + + @Override + public Map listTranscriptsForChannel(String channelName, String apiKey, boolean continueOnError) throws TranscriptRetrievalException { + return listTranscriptsForChannel(channelName, apiKey, null, continueOnError); + } + + + private String getChannelPlaylistId(String channelId, String apiKey) throws TranscriptRetrievalException { + Map params = createParams( + "key", apiKey, + "part", "contentDetails", + "id", channelId + ); + String channelJson = client.get(CHANNELS, params); + + JsonNode jsonNode = parseJson(channelJson, + "Could not parse channel JSON for the channel with id: " + channelId); + + JsonNode channel = jsonNode.get("items").get(0); + + return channel.get("contentDetails").get("relatedPlaylists").get("uploads").asText(); + } + + private String getChannelId(String channelName, String apiKey) throws TranscriptRetrievalException { + Map params = createParams( + "key", apiKey, + "q", channelName, + "part", "snippet", + "type", "channel" + ); + + String searchJson = client.get(SEARCH, params); + + JsonNode jsonNode = parseJson(searchJson, + "Could not parse search JSON for the channel: " + channelName); + + for (JsonNode item : jsonNode.get("items")) { + JsonNode snippet = item.get("snippet"); + if (snippet.get("title").asText().equals(channelName)) { + return snippet.get("channelId").asText(); + } + } + + throw new TranscriptRetrievalException("Could not find channel with the name: " + channelName); + } + + + private List getVideoIds(String playlistId, String apiKey) throws TranscriptRetrievalException { + Map params = createParams( + "key", apiKey, + "playlistId", playlistId, + "part", "snippet", + "maxResults", "50" + ); + + List videoIds = new ArrayList<>(); + + while (true) { + String playlistJson = client.get(PLAYLIST_ITEMS, params); + JsonNode jsonNode = parseJson(playlistJson, + "Could not parse playlist JSON for the playlist: " + playlistId); + extractVideoId(jsonNode, videoIds); + JsonNode nextPageToken = jsonNode.get("nextPageToken"); + + if (nextPageToken == null) { + break; + } + + params.put("pageToken", nextPageToken.asText()); + } + + return videoIds; + } + + private void extractVideoId(JsonNode jsonNode, List videoIds) { + jsonNode.get("items").forEach(item -> { + String videoId = item.get("snippet") + .get("resourceId") + .get("videoId") + .asText(); + videoIds.add(videoId); + }); + } + + private Map createParams(String... params) { + Map map = new HashMap<>(params.length / 2); + for (int i = 0; i < params.length; i += 2) { + map.put(params[i], params[i + 1]); + } + return map; + } + + private JsonNode parseJson(String json, String errorMessage) throws TranscriptRetrievalException { + try { + return objectMapper.readTree(json); + } catch (JsonProcessingException e) { + throw new TranscriptRetrievalException(errorMessage, e); + } + } +} diff --git a/lib/src/main/java/io/github/thoroldvix/internal/DefaultTranscript.java b/lib/src/main/java/io/github/thoroldvix/internal/DefaultTranscript.java index 31f2f7d..3056338 100644 --- a/lib/src/main/java/io/github/thoroldvix/internal/DefaultTranscript.java +++ b/lib/src/main/java/io/github/thoroldvix/internal/DefaultTranscript.java @@ -115,10 +115,10 @@ public boolean equals(Object o) { if (o == null || getClass() != o.getClass()) return false; DefaultTranscript that = (DefaultTranscript) o; return isGenerated == that.isGenerated && isTranslatable == that.isTranslatable && - Objects.equals(client, that.client) && Objects.equals(videoId, that.videoId) && - Objects.equals(apiUrl, that.apiUrl) && Objects.equals(language, that.language) && - Objects.equals(languageCode, that.languageCode) && - Objects.equals(translationLanguages, that.translationLanguages); + Objects.equals(client, that.client) && Objects.equals(videoId, that.videoId) && + Objects.equals(apiUrl, that.apiUrl) && Objects.equals(language, that.language) && + Objects.equals(languageCode, that.languageCode) && + Objects.equals(translationLanguages, that.translationLanguages); } @Override @@ -129,10 +129,10 @@ public int hashCode() { @Override public String toString() { String template = "Transcript for video with id: %s.\n" + - "Language: %s\n" + - "Language code: %s\n" + - "API URL for retrieving content: %s\n" + - "Available translation languages: %s"; + "Language: %s\n" + + "Language code: %s\n" + + "API URL for retrieving content: %s\n" + + "Available translation languages: %s"; return String.format(template, videoId, diff --git a/lib/src/main/java/io/github/thoroldvix/internal/DefaultTranscriptContent.java b/lib/src/main/java/io/github/thoroldvix/internal/DefaultTranscriptContent.java index b5715f6..f58ae3c 100644 --- a/lib/src/main/java/io/github/thoroldvix/internal/DefaultTranscriptContent.java +++ b/lib/src/main/java/io/github/thoroldvix/internal/DefaultTranscriptContent.java @@ -102,10 +102,10 @@ public int hashCode() { @Override public String toString() { return "{" + - "text='" + text + '\'' + - ", start=" + start + - ", dur=" + dur + - '}'; + "text='" + text + '\'' + + ", start=" + start + + ", dur=" + dur + + '}'; } } } diff --git a/lib/src/main/java/io/github/thoroldvix/internal/DefaultTranscriptList.java b/lib/src/main/java/io/github/thoroldvix/internal/DefaultTranscriptList.java index e5649bd..18ac989 100644 --- a/lib/src/main/java/io/github/thoroldvix/internal/DefaultTranscriptList.java +++ b/lib/src/main/java/io/github/thoroldvix/internal/DefaultTranscriptList.java @@ -26,6 +26,21 @@ final class DefaultTranscriptList implements TranscriptList { this.translationLanguages = translationLanguages; } + private static String[] getDefault(String[] languageCodes) { + return languageCodes.length == 0 ? new String[]{"en"} : languageCodes; + } + + private static void validateLanguageCodes(String... languageCodes) { + for (String languageCode : languageCodes) { + if (languageCode == null) { + throw new IllegalArgumentException("Language codes cannot be null"); + } + if (languageCode.isBlank()) { + throw new IllegalArgumentException("Language codes cannot be blank"); + } + } + } + @Override public Transcript findTranscript(String... languageCodes) throws TranscriptRetrievalException { try { @@ -50,15 +65,16 @@ private Transcript findTranscript(Map transcripts, String... throw new TranscriptRetrievalException(videoId, String.format("No transcripts were found for any of the requested language codes: %s. %s.", Arrays.toString(languageCodes), this)); } - private static String[] getDefault(String[] languageCodes) { - return languageCodes.length == 0 ? new String[]{"en"} : languageCodes; - } - @Override public Transcript findGeneratedTranscript(String... languageCodes) throws TranscriptRetrievalException { return findTranscript(generatedTranscripts, getDefault(languageCodes)); } + @Override + public String getVideoId() { + return videoId; + } + @Override public Iterator iterator() { return new Iterator<>() { @@ -80,17 +96,6 @@ public Transcript next() { }; } - private static void validateLanguageCodes(String... languageCodes) { - for (String languageCode : languageCodes) { - if (languageCode == null) { - throw new IllegalArgumentException("Language codes cannot be null"); - } - if (languageCode.isBlank()) { - throw new IllegalArgumentException("Language codes cannot be blank"); - } - } - } - @Override public boolean equals(Object o) { if (this == o) return true; @@ -107,12 +112,12 @@ public int hashCode() { @Override public String toString() { String template = "For video with ID (%s) transcripts are available in the following languages:\n" + - "Manually created: " + - "%s\n" + - "Automatically generated: " + - "%s\n" + - "Available translation languages: " + - "%s"; + "Manually created: " + + "%s\n" + + "Automatically generated: " + + "%s\n" + + "Available translation languages: " + + "%s"; return String.format(template, videoId, diff --git a/lib/src/main/java/io/github/thoroldvix/internal/DefaultYoutubeClient.java b/lib/src/main/java/io/github/thoroldvix/internal/DefaultYoutubeClient.java index 711362b..6c420cb 100644 --- a/lib/src/main/java/io/github/thoroldvix/internal/DefaultYoutubeClient.java +++ b/lib/src/main/java/io/github/thoroldvix/internal/DefaultYoutubeClient.java @@ -2,6 +2,7 @@ import io.github.thoroldvix.api.TranscriptRetrievalException; import io.github.thoroldvix.api.YoutubeClient; +import io.github.thoroldvix.api.YtApiV3Endpoint; import java.io.IOException; import java.net.URI; @@ -15,8 +16,6 @@ */ final class DefaultYoutubeClient implements YoutubeClient { - private static final String YOUTUBE_REQUEST_FAILED = "Request to YouTube failed."; - private final HttpClient httpClient; DefaultYoutubeClient() { @@ -30,31 +29,68 @@ final class DefaultYoutubeClient implements YoutubeClient { @Override public String get(String url, Map headers) throws TranscriptRetrievalException { String videoId = url.split("=")[1]; + String errorMessage = "Request to YouTube failed."; String[] headersArray = createHeaders(headers); - HttpRequest request = createRequest(url, headersArray); - HttpResponse response = send(videoId, request); + HttpRequest request = HttpRequest.newBuilder() + .uri(URI.create(url)) + .headers(headersArray) + .build(); + + HttpResponse response; + try { + response = httpClient.send(request, HttpResponse.BodyHandlers.ofString()); + } catch (IOException e) { + throw new TranscriptRetrievalException(videoId, errorMessage, e); + } catch (InterruptedException e) { + Thread.currentThread().interrupt(); + throw new TranscriptRetrievalException(videoId, errorMessage, e); + } + if (response.statusCode() != 200) { - throw new TranscriptRetrievalException(videoId, YOUTUBE_REQUEST_FAILED + " Status code: " + response.statusCode()); + throw new TranscriptRetrievalException(videoId, errorMessage + " Status code: " + response.statusCode()); } return response.body(); } - private HttpResponse send(String videoId, HttpRequest request) throws TranscriptRetrievalException { + @Override + public String get(YtApiV3Endpoint endpoint, Map params) throws TranscriptRetrievalException { + String paramsString = createParamsString(params); + String errorMessage = String.format("Request to YouTube '%s' endpoint failed.", endpoint); + HttpRequest request = HttpRequest.newBuilder() + .uri(URI.create(endpoint.url() + "?" + paramsString)) + .build(); + + HttpResponse response; try { - return httpClient.send(request, HttpResponse.BodyHandlers.ofString()); + response = httpClient.send(request, HttpResponse.BodyHandlers.ofString()); } catch (IOException e) { - throw new TranscriptRetrievalException(videoId, YOUTUBE_REQUEST_FAILED, e); + throw new TranscriptRetrievalException(String.format(errorMessage, endpoint), e); } catch (InterruptedException e) { Thread.currentThread().interrupt(); - throw new TranscriptRetrievalException(videoId, YOUTUBE_REQUEST_FAILED, e); + throw new TranscriptRetrievalException(errorMessage, e); } + + if (response.statusCode() != 200) { + throw new TranscriptRetrievalException(errorMessage + " Status code: " + response.statusCode()); + } + + return response.body(); } - private static HttpRequest createRequest(String url, String[] headersArray) { - return HttpRequest.newBuilder() - .uri(URI.create(url)) - .headers(headersArray) - .build(); + private String createParamsString(Map params) { + StringBuilder paramString = new StringBuilder(); + + for (Map.Entry entry : params.entrySet()) { + String value = formatValue(entry.getValue()); + paramString.append(entry.getKey()).append("=").append(value).append("&"); + } + + paramString.deleteCharAt(paramString.length() - 1); + return paramString.toString(); + } + + private String formatValue(String value) { + return value.replaceAll(" ", "%20"); } private String[] createHeaders(Map headers) { diff --git a/lib/src/main/java/io/github/thoroldvix/internal/DefaultYoutubeTranscriptApi.java b/lib/src/main/java/io/github/thoroldvix/internal/DefaultYoutubeTranscriptApi.java index 29d590d..5eb6336 100644 --- a/lib/src/main/java/io/github/thoroldvix/internal/DefaultYoutubeTranscriptApi.java +++ b/lib/src/main/java/io/github/thoroldvix/internal/DefaultYoutubeTranscriptApi.java @@ -29,6 +29,38 @@ final class DefaultYoutubeTranscriptApi implements YoutubeTranscriptApi { this.fileLinesReader = fileLinesReader; } + private static HttpCookie createCookie(String[] parts) { + String domain = parts[0]; + boolean secure = Boolean.parseBoolean(parts[1]); + String path = parts[2]; + boolean httpOnly = Boolean.parseBoolean(parts[3]); + long expiration = Long.parseLong(parts[4]); + String name = parts[5]; + String value = parts[6]; + + HttpCookie cookie = new HttpCookie(name, value); + cookie.setDomain(domain); + cookie.setPath(path); + cookie.setSecure(secure); + cookie.setHttpOnly(httpOnly); + cookie.setMaxAge(expiration); + return cookie; + } + + private static boolean containsConsentPage(String videoPageHtml) { + String consentPagePattern = "action=\"https://consent.youtube.com/s\""; + return videoPageHtml.contains(consentPagePattern); + } + + private static String extractConsentCookie(String videoId, String html) throws TranscriptRetrievalException { + Pattern consentCookiePattern = Pattern.compile("name=\"v\" value=\"(.*?)\""); + Matcher matcher = consentCookiePattern.matcher(html); + if (!matcher.find()) { + throw new TranscriptRetrievalException(videoId, FAILED_TO_GIVE_COOKIES_CONSENT); + } + return String.format("CONSENT=YES+%s", matcher.group(1)); + } + @Override public TranscriptList listTranscriptsWithCookies(String videoId, String cookiesPath) throws TranscriptRetrievalException { validateVideoId(videoId); @@ -62,24 +94,6 @@ private List loadCookies(String videoId, String cookiesPath) throws } } - private static HttpCookie createCookie(String[] parts) { - String domain = parts[0]; - boolean secure = Boolean.parseBoolean(parts[1]); - String path = parts[2]; - boolean httpOnly = Boolean.parseBoolean(parts[3]); - long expiration = Long.parseLong(parts[4]); - String name = parts[5]; - String value = parts[6]; - - HttpCookie cookie = new HttpCookie(name, value); - cookie.setDomain(domain); - cookie.setPath(path); - cookie.setSecure(secure); - cookie.setHttpOnly(httpOnly); - cookie.setMaxAge(expiration); - return cookie; - } - private String fetchVideoPageHtml(String videoId, String cookieHeader) throws TranscriptRetrievalException { Map requestHeaders = createRequestHeaders(cookieHeader); return client.get(YOUTUBE_WATCH_URL + videoId, requestHeaders); @@ -105,11 +119,6 @@ public TranscriptList listTranscripts(String videoId) throws TranscriptRetrieval .transcriptList(); } - private static boolean containsConsentPage(String videoPageHtml) { - String consentPagePattern = "action=\"https://consent.youtube.com/s\""; - return videoPageHtml.contains(consentPagePattern); - } - private String retryWithConsentCookie(String videoId, String videoPageHtml) throws TranscriptRetrievalException { String consentCookie = extractConsentCookie(videoId, videoPageHtml); Map requestHeaders = createRequestHeaders(consentCookie); @@ -133,13 +142,4 @@ public TranscriptContent getTranscript(String videoId, String... languageCodes) .findTranscript(languageCodes) .fetch(); } - - private static String extractConsentCookie(String videoId, String html) throws TranscriptRetrievalException { - Pattern consentCookiePattern = Pattern.compile("name=\"v\" value=\"(.*?)\""); - Matcher matcher = consentCookiePattern.matcher(html); - if (!matcher.find()) { - throw new TranscriptRetrievalException(videoId, FAILED_TO_GIVE_COOKIES_CONSENT); - } - return String.format("CONSENT=YES+%s", matcher.group(1)); - } } diff --git a/lib/src/main/java/io/github/thoroldvix/internal/TranscriptApiFactory.java b/lib/src/main/java/io/github/thoroldvix/internal/TranscriptApiFactory.java index b785ce6..7c67491 100644 --- a/lib/src/main/java/io/github/thoroldvix/internal/TranscriptApiFactory.java +++ b/lib/src/main/java/io/github/thoroldvix/internal/TranscriptApiFactory.java @@ -1,5 +1,6 @@ package io.github.thoroldvix.internal; +import io.github.thoroldvix.api.PlaylistsTranscriptApi; import io.github.thoroldvix.api.YoutubeClient; import io.github.thoroldvix.api.YoutubeTranscriptApi; @@ -7,7 +8,7 @@ import java.nio.file.Path; /** - * Class for creating instances of {@link YoutubeTranscriptApi}. + * Responsible for creating instances of {@link YoutubeTranscriptApi} and {@link PlaylistsTranscriptApi}. */ public final class TranscriptApiFactory { @@ -23,6 +24,25 @@ public static YoutubeTranscriptApi createDefault() { return createWithClient(new DefaultYoutubeClient()); } + /** + * Creates a new instance of {@link PlaylistsTranscriptApi} using the specified {@link YoutubeClient}. + * + * @param client The {@link YoutubeClient} to be used for YouTube interactions + * @return A new instance of {@link PlaylistsTranscriptApi} + */ + public static PlaylistsTranscriptApi createDefaultPlaylistsApi(YoutubeClient client) { + return new DefaultPlaylistsTranscriptApi(client, createDefault()); + } + + /** + * Creates a new instance of {@link PlaylistsTranscriptApi} using the default YouTube client. + * + * @return A new instance of {@link PlaylistsTranscriptApi} + */ + public static PlaylistsTranscriptApi createDefaultPlaylistsApi() { + return createDefaultPlaylistsApi(new DefaultYoutubeClient()); + } + /** * Creates a new instance of {@link YoutubeTranscriptApi} using the specified {@link YoutubeClient}. * diff --git a/lib/src/main/java/io/github/thoroldvix/internal/TranscriptContentXML.java b/lib/src/main/java/io/github/thoroldvix/internal/TranscriptContentXML.java index d2eceb3..62a5f6f 100644 --- a/lib/src/main/java/io/github/thoroldvix/internal/TranscriptContentXML.java +++ b/lib/src/main/java/io/github/thoroldvix/internal/TranscriptContentXML.java @@ -27,13 +27,6 @@ final class TranscriptContentXML { this.videoId = videoId; } - TranscriptContent transcriptContent() throws TranscriptRetrievalException { - List fragments = parseFragments(); - List content = formatFragments(fragments); - - return new DefaultTranscriptContent(content); - } - private static List formatFragments(List fragments) { return fragments.stream() .filter(TranscriptContentXML::isValidTranscriptFragment) @@ -42,15 +35,6 @@ private static List formatFragments(List fragments) { .collect(Collectors.toList()); } - private List parseFragments() throws TranscriptRetrievalException { - try { - return xmlMapper.readValue(xml, new TypeReference<>() { - }); - } catch (JsonProcessingException e) { - throw new TranscriptRetrievalException(videoId, "Failed to parse transcript content XML.", e); - } - } - private static Fragment unescapeXmlTags(Fragment fragment) { String formattedValue = StringEscapeUtils.unescapeXml(fragment.getText()); return new Fragment(formattedValue, fragment.getStart(), fragment.getDur()); @@ -65,4 +49,20 @@ private static Fragment removeHtmlTags(Fragment fragment) { private static boolean isValidTranscriptFragment(Fragment fragment) { return fragment.getText() != null && !fragment.getText().isBlank(); } + + TranscriptContent transcriptContent() throws TranscriptRetrievalException { + List fragments = parseFragments(); + List content = formatFragments(fragments); + + return new DefaultTranscriptContent(content); + } + + private List parseFragments() throws TranscriptRetrievalException { + try { + return xmlMapper.readValue(xml, new TypeReference<>() { + }); + } catch (JsonProcessingException e) { + throw new TranscriptRetrievalException(videoId, "Failed to parse transcript content XML.", e); + } + } } diff --git a/lib/src/main/java/io/github/thoroldvix/internal/TranscriptListJSON.java b/lib/src/main/java/io/github/thoroldvix/internal/TranscriptListJSON.java index 0e904d3..d3c5ef3 100644 --- a/lib/src/main/java/io/github/thoroldvix/internal/TranscriptListJSON.java +++ b/lib/src/main/java/io/github/thoroldvix/internal/TranscriptListJSON.java @@ -21,12 +21,12 @@ final class TranscriptListJSON { private static final String TOO_MANY_REQUESTS = "YouTube is receiving too many requests from this IP and now requires solving a captcha to continue. " + - "One of the following things can be done to work around this:\n" + - "- Manually solve the captcha in a browser and export the cookie. " + - "Read here how to use that cookie with " + - "youtube-transcript-api: https://github.com/thoroldvix/youtube-transcript-api#cookies\n" + - "- Use a different IP address\n" + - "- Wait until the ban on your IP has been lifted"; + "One of the following things can be done to work around this:\n" + + "- Manually solve the captcha in a browser and export the cookie. " + + "Read here how to use that cookie with " + + "youtube-transcript-api: https://github.com/thoroldvix/youtube-transcript-api#cookies\n" + + "- Use a different IP address\n" + + "- Wait until the ban on your IP has been lifted"; private static final String TRANSCRIPTS_DISABLED = "Transcripts are disabled for this video."; private final JsonNode json; @@ -39,10 +39,6 @@ private TranscriptListJSON(JsonNode json, YoutubeClient client, String videoId) this.videoId = videoId; } - TranscriptList transcriptList() { - return new DefaultTranscriptList(videoId, getManualTranscripts(), getGeneratedTranscripts(), getTranslationLanguages()); - } - static TranscriptListJSON from(String videoPageHtml, YoutubeClient client, String videoId) throws TranscriptRetrievalException { String json = getJsonFromHtml(videoPageHtml, videoId); JsonNode parsedJson = parseJson(json, videoId); @@ -91,6 +87,10 @@ private static void checkIfTranscriptsDisabled(String videoId, JsonNode parsedJs } } + TranscriptList transcriptList() { + return new DefaultTranscriptList(videoId, getManualTranscripts(), getGeneratedTranscripts(), getTranslationLanguages()); + } + private Map getTranslationLanguages() { if (!json.has("translationLanguages")) { return Collections.emptyMap(); @@ -115,7 +115,11 @@ private Map getTranscripts(YoutubeClient client, Predicate getTranscript(client, jsonNode, translationLanguages)) - .collect(Collectors.toMap(Transcript::getLanguageCode, transcript -> transcript)); + .collect(Collectors.toMap( + Transcript::getLanguageCode, + transcript -> transcript, + (existing, replacement) -> existing) + ); } private Transcript getTranscript(YoutubeClient client, JsonNode jsonNode, Map translationLanguages) { diff --git a/lib/src/test/java/io/github/thoroldvix/internal/DefaultPlaylistsTranscriptApiTest.java b/lib/src/test/java/io/github/thoroldvix/internal/DefaultPlaylistsTranscriptApiTest.java new file mode 100644 index 0000000..8e59625 --- /dev/null +++ b/lib/src/test/java/io/github/thoroldvix/internal/DefaultPlaylistsTranscriptApiTest.java @@ -0,0 +1,171 @@ +package io.github.thoroldvix.internal; + +import io.github.thoroldvix.api.*; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; +import org.mockito.ArgumentCaptor; +import org.mockito.Captor; +import org.mockito.junit.jupiter.MockitoExtension; + +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Paths; +import java.util.Collections; +import java.util.Map; + +import static io.github.thoroldvix.api.YtApiV3Endpoint.*; +import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.assertThatThrownBy; +import static org.mockito.ArgumentMatchers.anyMap; +import static org.mockito.ArgumentMatchers.eq; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; + +@ExtendWith(MockitoExtension.class) +class DefaultPlaylistsTranscriptApiTest { + private final String VIDEO_ID_1 = "8idr1WZ1A7Q"; + private final String VIDEO_ID_2 = "ZA4JkHKZM50"; + private static final String API_KEY = "apiKey"; + private static final String PLAYLIST_ID = "playlistId"; + private YoutubeClient client; + + private YoutubeTranscriptApi api; + + private PlaylistsTranscriptApi playlistApi; + + private static String PLAYLIST_SINGLE_PAGE; + + private static String CHANNEL_SEARCH_RESPONSE; + + private static final String API_RESPONSES_DIR = "src/test/resources/api_v3_responses"; + + @BeforeAll + static void beforeAll() throws IOException { + PLAYLIST_SINGLE_PAGE = Files.readString(Paths.get(API_RESPONSES_DIR, "playlist_single_page.json")); + CHANNEL_SEARCH_RESPONSE = Files.readString(Paths.get(API_RESPONSES_DIR, "channel_search_response.json")); + } + + @BeforeEach + void setUp() { + client = mock(YoutubeClient.class); + api = mock(YoutubeTranscriptApi.class); + playlistApi = new DefaultPlaylistsTranscriptApi( + client, + api + ); + } + + @Test + void listTranscriptsForPlaylistWithCookies() throws TranscriptRetrievalException { + TranscriptList transcriptList1 = createTranscriptList(VIDEO_ID_1); + TranscriptList transcriptList2 = createTranscriptList(VIDEO_ID_2); + + + when(client.get(eq(PLAYLIST_ITEMS), anyMap())).thenReturn(PLAYLIST_SINGLE_PAGE); + when(api.listTranscriptsWithCookies(VIDEO_ID_1, "cookiePath")).thenReturn(transcriptList1); + when(api.listTranscriptsWithCookies(VIDEO_ID_2, "cookiePath")).thenReturn(transcriptList2); + + Map result = playlistApi.listTranscriptsForPlaylist(PLAYLIST_ID, API_KEY, "cookiePath", false); + + assertThat(result.keySet()).containsExactlyInAnyOrder(VIDEO_ID_1, VIDEO_ID_2); + assertThat(result.get(VIDEO_ID_1)).isEqualTo(transcriptList1); + assertThat(result.get(VIDEO_ID_2)).isEqualTo(transcriptList2); + } + + private static DefaultTranscriptList createTranscriptList(String videoId) { + return new DefaultTranscriptList(videoId, Collections.emptyMap(), Collections.emptyMap(), Collections.emptyMap()); + } + + @Test + void listTranscriptsForPlaylistContinueOnError() throws TranscriptRetrievalException { + TranscriptList transcriptList = createTranscriptList(VIDEO_ID_2); + + when(client.get(eq(PLAYLIST_ITEMS), anyMap())).thenReturn(PLAYLIST_SINGLE_PAGE); + when(api.listTranscripts(VIDEO_ID_2)).thenReturn(transcriptList); + when(api.listTranscripts(VIDEO_ID_1)).thenThrow(new TranscriptRetrievalException(VIDEO_ID_1, "Error")); + + assertThatThrownBy(() -> playlistApi.listTranscriptsForPlaylist(PLAYLIST_ID, API_KEY, false)) + .isInstanceOf(TranscriptRetrievalException.class); + + assertThat(playlistApi.listTranscriptsForPlaylist(PLAYLIST_ID, API_KEY, true)).containsKey(VIDEO_ID_2); + } + + @Test + void listTranscriptsForPlaylistGetsNextPage(@Captor ArgumentCaptor> paramsCaptor) throws TranscriptRetrievalException, + IOException { + String firstPageResponse = Files.readString(Paths.get(API_RESPONSES_DIR, "playlist_page_one.json")); + String secondPageResponse = Files.readString(Paths.get(API_RESPONSES_DIR, "playlist_page_two.json")); + + when(client.get(eq(PLAYLIST_ITEMS), paramsCaptor.capture())) + .thenReturn(firstPageResponse) + .thenReturn(secondPageResponse); + + playlistApi.listTranscriptsForPlaylist(PLAYLIST_ID, API_KEY, false); + + assertThat(paramsCaptor.getValue()).containsEntry("pageToken", + "EAAajgFQVDpDQUVpRURJNE9VWTBRVFEyUkVZd1FUTXdSRElvQVVpN3BfbVAwY3FHQTFBQldrVWlRMmx" + + "LVVZSR2NFbFZWVGxwVkRGa1ZWVlZVbEJoYlRGMlRURnJNbEZWVW5STlJrNXFWak" + + "JHYzFKV2FHMU1WMXAzUldkM1NYRkxlVTl6ZDFsUkxVbFRkWGgzU1NJ"); + } + + @Test + void listTranscriptsForPlaylistThrowsExceptionIfCannotParsePlaylistJson() throws TranscriptRetrievalException { + when(client.get(eq(PLAYLIST_ITEMS), anyMap())).thenReturn("error"); + + assertThatThrownBy(() -> playlistApi.listTranscriptsForPlaylist(PLAYLIST_ID, API_KEY, false)) + .isInstanceOf(TranscriptRetrievalException.class); + } + + @Test + void listTranscriptsForChannelWithCookies(@Captor ArgumentCaptor> paramsCaptor) throws Exception { + TranscriptList transcriptList1 = createTranscriptList(VIDEO_ID_1); + TranscriptList transcriptList2 = createTranscriptList(VIDEO_ID_2); + + String channelResponse = Files.readString(Paths.get(API_RESPONSES_DIR, "channel_response.json")); + + + when(client.get(eq(SEARCH), anyMap())).thenReturn(CHANNEL_SEARCH_RESPONSE); + when(client.get(eq(CHANNELS), paramsCaptor.capture())).thenReturn(channelResponse); + when(client.get(eq(PLAYLIST_ITEMS), anyMap())).thenReturn(PLAYLIST_SINGLE_PAGE); + + when(api.listTranscriptsWithCookies(VIDEO_ID_1, "cookiePath")).thenReturn(transcriptList1); + when(api.listTranscriptsWithCookies(VIDEO_ID_2, "cookiePath")).thenReturn(transcriptList2); + + Map result = playlistApi.listTranscriptsForChannel("3Blue1Brown", API_KEY, "cookiePath", false); + + assertThat(result.keySet()).containsExactlyInAnyOrder(VIDEO_ID_1, VIDEO_ID_2); + assertThat(result.get(VIDEO_ID_1)).isEqualTo(transcriptList1); + assertThat(result.get(VIDEO_ID_2)).isEqualTo(transcriptList2); + assertThat(paramsCaptor.getValue()).containsEntry("id", "UCYO_jab_esuFRV4b17AJtAw"); + } + + @Test + void listTranscriptsForChannelThrowsExceptionWhenChannelNotFound() throws TranscriptRetrievalException, IOException { + String searchNoMatchResponse = Files.readString(Paths.get(API_RESPONSES_DIR, "channel_search_no_match.json")); + when(client.get(eq(SEARCH), anyMap())).thenReturn(searchNoMatchResponse); + + assertThatThrownBy(() -> playlistApi.listTranscriptsForChannel("3Blue1Brown", API_KEY, false)) + .isInstanceOf(TranscriptRetrievalException.class); + } + + @Test + void listTranscriptsForChannelThrowsExceptionIfCannotParseChannelSearchJson() throws Exception { + when(client.get(eq(SEARCH), anyMap())).thenReturn("error"); + + assertThatThrownBy(() -> playlistApi.listTranscriptsForChannel("3Blue1Brown", API_KEY, false)) + .isInstanceOf(TranscriptRetrievalException.class); + } + + @Test + void listTranscriptsForChannelThrowsExceptionIfCannotParseChannelJson() throws Exception { + when(client.get(eq(SEARCH), anyMap())).thenReturn(CHANNEL_SEARCH_RESPONSE); + when(client.get(eq(CHANNELS), anyMap())).thenReturn("error"); + + assertThatThrownBy(() -> playlistApi.listTranscriptsForChannel("3Blue1Brown", API_KEY, false)) + .isInstanceOf(TranscriptRetrievalException.class); + } + + +} \ No newline at end of file diff --git a/lib/src/test/java/io/github/thoroldvix/internal/DefaultYoutubeClientTest.java b/lib/src/test/java/io/github/thoroldvix/internal/DefaultYoutubeClientTest.java index 9fd18ff..f964ff2 100644 --- a/lib/src/test/java/io/github/thoroldvix/internal/DefaultYoutubeClientTest.java +++ b/lib/src/test/java/io/github/thoroldvix/internal/DefaultYoutubeClientTest.java @@ -19,6 +19,7 @@ import java.net.http.HttpResponse; import java.util.Map; +import static io.github.thoroldvix.api.YtApiV3Endpoint.PLAYLIST_ITEMS; import static org.assertj.core.api.Assertions.assertThat; import static org.assertj.core.api.Assertions.assertThatThrownBy; import static org.mockito.ArgumentMatchers.any; @@ -30,7 +31,7 @@ class DefaultYoutubeClientTest { private static final String VIDEO_URL = "https://www.youtube.com/watch?v=dQw4w9WgXcQ"; private static final Map HEADERS = Map.of("Accept-Language", "en-US"); - + private static final Map PARAMS = Map.of("key", "test", "part", "snippet"); @Mock private HttpResponse response; @Mock @@ -91,4 +92,48 @@ void getThrowsExceptionWhenInterruptedExceptionOccurs() throws Exception { assertThatThrownBy(() -> youtubeClient.get(VIDEO_URL, HEADERS)) .isInstanceOf(TranscriptRetrievalException.class); } + + @Test + void getToApiEndpoint() throws Exception { + String expected = "expected response"; + + when(httpClient.send(requestCaptor.capture(), any(HttpResponse.BodyHandlers.ofString().getClass()))).thenReturn(response); + when(response.statusCode()).thenReturn(200); + when(response.body()).thenReturn(expected); + + String actual = youtubeClient.get(PLAYLIST_ITEMS, PARAMS); + + HttpRequest request = requestCaptor.getValue(); + + assertThat(actual).isEqualTo(expected); + assertThat(request.uri().toString()).contains(PLAYLIST_ITEMS.url() + "?"); + assertThat(request.uri().toString()).contains("key=test"); + assertThat(request.uri().toString()).contains("part=snippet"); + } + + @ParameterizedTest + @ValueSource(ints = {500, 404}) + void getToApiEndpointThrowsExceptionIfResponseIsNotOk(int statusCode) throws Exception { + when(httpClient.send(any(HttpRequest.class), any(HttpResponse.BodyHandlers.ofString().getClass()))).thenReturn(response); + when(response.statusCode()).thenReturn(statusCode); + + assertThatThrownBy(() -> youtubeClient.get(PLAYLIST_ITEMS, PARAMS)) + .isInstanceOf(TranscriptRetrievalException.class); + } + + @Test + void getToApiEndpointThrowsExceptionWhenIOExceptionOccurs() throws Exception { + when(httpClient.send(any(), any())).thenThrow(new IOException()); + + assertThatThrownBy(() -> youtubeClient.get(PLAYLIST_ITEMS, PARAMS)) + .isInstanceOf(TranscriptRetrievalException.class); + } + + @Test + void getToApiEndpointThrowsExceptionWhenInterruptedExceptionOccurs() throws Exception { + when(httpClient.send(any(), any())).thenThrow(new InterruptedException()); + + assertThatThrownBy(() -> youtubeClient.get(PLAYLIST_ITEMS, PARAMS)) + .isInstanceOf(TranscriptRetrievalException.class); + } } \ No newline at end of file diff --git a/lib/src/test/resources/api_v3_responses/channel_response.json b/lib/src/test/resources/api_v3_responses/channel_response.json new file mode 100644 index 0000000..ba65f83 --- /dev/null +++ b/lib/src/test/resources/api_v3_responses/channel_response.json @@ -0,0 +1,49 @@ +{ + "kind": "youtube#channelListResponse", + "etag": "_miJc-8nYvbt63SiDPtLibVPmiY", + "pageInfo": { + "totalResults": 1, + "resultsPerPage": 5 + }, + "items": [ + { + "kind": "youtube#channel", + "etag": "yUhB4AYzJoEGPItn1HGuNbwH5aQ", + "id": "UCYO_jab_esuFRV4b17AJtAw", + "snippet": { + "title": "3Blue1Brown", + "description": "My name is Grant Sanderson. Videos here cover a variety of topics in math, or adjacent fields like physics and CS, all with an emphasis on visualizing the core ideas. The goal is to use animation to help elucidate and motivate otherwise tricky topics, and for difficult problems to be made simple with changes in perspective.\n\nFor more information, other projects, FAQs, and inquiries see the website: https://www.3blue1brown.com", + "customUrl": "@3blue1brown", + "publishedAt": "2015-03-03T23:11:55Z", + "thumbnails": { + "default": { + "url": "https://yt3.ggpht.com/ytc/AIdro_nFzZFPLxPZRHcE3SSwzdrbuWqfoWYwLAu0_2iO6blQYAU=s88-c-k-c0x00ffffff-no-rj", + "width": 88, + "height": 88 + }, + "medium": { + "url": "https://yt3.ggpht.com/ytc/AIdro_nFzZFPLxPZRHcE3SSwzdrbuWqfoWYwLAu0_2iO6blQYAU=s240-c-k-c0x00ffffff-no-rj", + "width": 240, + "height": 240 + }, + "high": { + "url": "https://yt3.ggpht.com/ytc/AIdro_nFzZFPLxPZRHcE3SSwzdrbuWqfoWYwLAu0_2iO6blQYAU=s800-c-k-c0x00ffffff-no-rj", + "width": 800, + "height": 800 + } + }, + "localized": { + "title": "3Blue1Brown", + "description": "My name is Grant Sanderson. Videos here cover a variety of topics in math, or adjacent fields like physics and CS, all with an emphasis on visualizing the core ideas. The goal is to use animation to help elucidate and motivate otherwise tricky topics, and for difficult problems to be made simple with changes in perspective.\n\nFor more information, other projects, FAQs, and inquiries see the website: https://www.3blue1brown.com" + }, + "country": "US" + }, + "contentDetails": { + "relatedPlaylists": { + "likes": "", + "uploads": "UUYO_jab_esuFRV4b17AJtAw" + } + } + } + ] +} \ No newline at end of file diff --git a/lib/src/test/resources/api_v3_responses/channel_search_no_match.json b/lib/src/test/resources/api_v3_responses/channel_search_no_match.json new file mode 100644 index 0000000..57b6ac9 --- /dev/null +++ b/lib/src/test/resources/api_v3_responses/channel_search_no_match.json @@ -0,0 +1,124 @@ +{ + "kind": "youtube#searchListResponse", + "etag": "Si8hxIyH7Dn2SgY0dQ12mnVDFGE", + "nextPageToken": "CAUQAA", + "regionCode": "UA", + "pageInfo": { + "totalResults": 208, + "resultsPerPage": 5 + }, + "items": [ + { + "kind": "youtube#searchResult", + "etag": "KwpU6cbvPGOz30J490cU4B2S1cs", + "id": { + "kind": "youtube#channel", + "channelId": "UCBevyiJ2ierZY-0yZhfLrmQ" + }, + "snippet": { + "publishedAt": "2020-09-23T22:37:55Z", + "channelId": "UCBevyiJ2ierZY-0yZhfLrmQ", + "title": "3Blue1BrownJapan", + "description": "3Blue1Brownの日本語版公式チャンネルです。東京大学の学生有志団体が本家3Blue1Brownの公式ライセンスのもと、動画を日本 ...", + "thumbnails": { + "default": { + "url": "https://yt3.ggpht.com/kkBN740CKMSMPdjWOrKn8sbZQvEEHiHmO5jT5dt0AVF3iePocVolv8IwfHml85v1kOlRDwg2DQ=s88-c-k-c0xffffffff-no-rj-mo" + }, + "medium": { + "url": "https://yt3.ggpht.com/kkBN740CKMSMPdjWOrKn8sbZQvEEHiHmO5jT5dt0AVF3iePocVolv8IwfHml85v1kOlRDwg2DQ=s240-c-k-c0xffffffff-no-rj-mo" + }, + "high": { + "url": "https://yt3.ggpht.com/kkBN740CKMSMPdjWOrKn8sbZQvEEHiHmO5jT5dt0AVF3iePocVolv8IwfHml85v1kOlRDwg2DQ=s800-c-k-c0xffffffff-no-rj-mo" + } + }, + "channelTitle": "3Blue1BrownJapan", + "liveBroadcastContent": "none", + "publishTime": "2020-09-23T22:37:55Z" + } + }, + { + "kind": "youtube#searchResult", + "etag": "X41COJVPJ0YgI0XQ1E1LxwUtAzM", + "id": { + "kind": "youtube#channel", + "channelId": "UC6hAYNOWMmuqOBvFOuAFKwA" + }, + "snippet": { + "publishedAt": "2018-03-26T18:54:06Z", + "channelId": "UC6hAYNOWMmuqOBvFOuAFKwA", + "title": "3Blue1Brown Русский", + "description": "3Blue1Brown, созданный Грантом Сандэрсоном (Grant Sanderson), Это комбинация Математики и развлечения - в ...", + "thumbnails": { + "default": { + "url": "https://yt3.ggpht.com/ytc/AIdro_km1q4dyRhqPHfRXTuQg_B5iLVCeXhJtHyO4FVxXPB4xIU=s88-c-k-c0xffffffff-no-rj-mo" + }, + "medium": { + "url": "https://yt3.ggpht.com/ytc/AIdro_km1q4dyRhqPHfRXTuQg_B5iLVCeXhJtHyO4FVxXPB4xIU=s240-c-k-c0xffffffff-no-rj-mo" + }, + "high": { + "url": "https://yt3.ggpht.com/ytc/AIdro_km1q4dyRhqPHfRXTuQg_B5iLVCeXhJtHyO4FVxXPB4xIU=s800-c-k-c0xffffffff-no-rj-mo" + } + }, + "channelTitle": "3Blue1Brown Русский", + "liveBroadcastContent": "none", + "publishTime": "2018-03-26T18:54:06Z" + } + }, + { + "kind": "youtube#searchResult", + "etag": "4sE5zUzu6QtK5xmmU4ZdFa3m9kQ", + "id": { + "kind": "youtube#channel", + "channelId": "UCCbgOIWdmYncvYMbl3LjvBQ" + }, + "snippet": { + "publishedAt": "2019-01-22T22:33:40Z", + "channelId": "UCCbgOIWdmYncvYMbl3LjvBQ", + "title": "3Blue1Brown translated by Sciberia", + "description": "Наши переводы MIT и Stanford: youtube.com/c/sciberia_science.", + "thumbnails": { + "default": { + "url": "https://yt3.ggpht.com/ytc/AIdro_kupS03sbv7wxGaGwMbIUOIo0pDuSG1NlvcC-MmIOE4gA=s88-c-k-c0xffffffff-no-rj-mo" + }, + "medium": { + "url": "https://yt3.ggpht.com/ytc/AIdro_kupS03sbv7wxGaGwMbIUOIo0pDuSG1NlvcC-MmIOE4gA=s240-c-k-c0xffffffff-no-rj-mo" + }, + "high": { + "url": "https://yt3.ggpht.com/ytc/AIdro_kupS03sbv7wxGaGwMbIUOIo0pDuSG1NlvcC-MmIOE4gA=s800-c-k-c0xffffffff-no-rj-mo" + } + }, + "channelTitle": "3Blue1Brown translated by Sciberia", + "liveBroadcastContent": "none", + "publishTime": "2019-01-22T22:33:40Z" + } + }, + { + "kind": "youtube#searchResult", + "etag": "rEjOIyCKfTE_GRT-nahPaewhvWs", + "id": { + "kind": "youtube#channel", + "channelId": "UC4kAnPtYF9ulAaC7Dap9BDA" + }, + "snippet": { + "publishedAt": "2022-07-29T13:22:15Z", + "channelId": "UC4kAnPtYF9ulAaC7Dap9BDA", + "title": "3Blue1Brown UA", + "description": "3Blue1Brown — канал, створений Грантом Сандероном і є певною комбінацією математики та розважального контенту, ...", + "thumbnails": { + "default": { + "url": "https://yt3.ggpht.com/DdRS6NY1DtMRo3WagAfz_AEflwCHJlyc1b2WMc-SaVUoQ20pBr115-lauL26BQeDWQnVH8SWow=s88-c-k-c0xffffffff-no-rj-mo" + }, + "medium": { + "url": "https://yt3.ggpht.com/DdRS6NY1DtMRo3WagAfz_AEflwCHJlyc1b2WMc-SaVUoQ20pBr115-lauL26BQeDWQnVH8SWow=s240-c-k-c0xffffffff-no-rj-mo" + }, + "high": { + "url": "https://yt3.ggpht.com/DdRS6NY1DtMRo3WagAfz_AEflwCHJlyc1b2WMc-SaVUoQ20pBr115-lauL26BQeDWQnVH8SWow=s800-c-k-c0xffffffff-no-rj-mo" + } + }, + "channelTitle": "3Blue1Brown UA", + "liveBroadcastContent": "none", + "publishTime": "2022-07-29T13:22:15Z" + } + } + ] +} \ No newline at end of file diff --git a/lib/src/test/resources/api_v3_responses/channel_search_response.json b/lib/src/test/resources/api_v3_responses/channel_search_response.json new file mode 100644 index 0000000..3de0489 --- /dev/null +++ b/lib/src/test/resources/api_v3_responses/channel_search_response.json @@ -0,0 +1,152 @@ +{ + "kind": "youtube#searchListResponse", + "etag": "Si8hxIyH7Dn2SgY0dQ12mnVDFGE", + "nextPageToken": "CAUQAA", + "regionCode": "UA", + "pageInfo": { + "totalResults": 208, + "resultsPerPage": 5 + }, + "items": [ + { + "kind": "youtube#searchResult", + "etag": "6WQE07OoKqJA-3J_041337KI8Xs", + "id": { + "kind": "youtube#channel", + "channelId": "UCYO_jab_esuFRV4b17AJtAw" + }, + "snippet": { + "publishedAt": "2015-03-03T23:11:55Z", + "channelId": "UCYO_jab_esuFRV4b17AJtAw", + "title": "3Blue1Brown", + "description": "My name is Grant Sanderson. Videos here cover a variety of topics in math, or adjacent fields like physics and CS, all with an ...", + "thumbnails": { + "default": { + "url": "https://yt3.ggpht.com/ytc/AIdro_nFzZFPLxPZRHcE3SSwzdrbuWqfoWYwLAu0_2iO6blQYAU=s88-c-k-c0xffffffff-no-rj-mo" + }, + "medium": { + "url": "https://yt3.ggpht.com/ytc/AIdro_nFzZFPLxPZRHcE3SSwzdrbuWqfoWYwLAu0_2iO6blQYAU=s240-c-k-c0xffffffff-no-rj-mo" + }, + "high": { + "url": "https://yt3.ggpht.com/ytc/AIdro_nFzZFPLxPZRHcE3SSwzdrbuWqfoWYwLAu0_2iO6blQYAU=s800-c-k-c0xffffffff-no-rj-mo" + } + }, + "channelTitle": "3Blue1Brown", + "liveBroadcastContent": "none", + "publishTime": "2015-03-03T23:11:55Z" + } + }, + { + "kind": "youtube#searchResult", + "etag": "KwpU6cbvPGOz30J490cU4B2S1cs", + "id": { + "kind": "youtube#channel", + "channelId": "UCBevyiJ2ierZY-0yZhfLrmQ" + }, + "snippet": { + "publishedAt": "2020-09-23T22:37:55Z", + "channelId": "UCBevyiJ2ierZY-0yZhfLrmQ", + "title": "3Blue1BrownJapan", + "description": "3Blue1Brownの日本語版公式チャンネルです。東京大学の学生有志団体が本家3Blue1Brownの公式ライセンスのもと、動画を日本 ...", + "thumbnails": { + "default": { + "url": "https://yt3.ggpht.com/kkBN740CKMSMPdjWOrKn8sbZQvEEHiHmO5jT5dt0AVF3iePocVolv8IwfHml85v1kOlRDwg2DQ=s88-c-k-c0xffffffff-no-rj-mo" + }, + "medium": { + "url": "https://yt3.ggpht.com/kkBN740CKMSMPdjWOrKn8sbZQvEEHiHmO5jT5dt0AVF3iePocVolv8IwfHml85v1kOlRDwg2DQ=s240-c-k-c0xffffffff-no-rj-mo" + }, + "high": { + "url": "https://yt3.ggpht.com/kkBN740CKMSMPdjWOrKn8sbZQvEEHiHmO5jT5dt0AVF3iePocVolv8IwfHml85v1kOlRDwg2DQ=s800-c-k-c0xffffffff-no-rj-mo" + } + }, + "channelTitle": "3Blue1BrownJapan", + "liveBroadcastContent": "none", + "publishTime": "2020-09-23T22:37:55Z" + } + }, + { + "kind": "youtube#searchResult", + "etag": "X41COJVPJ0YgI0XQ1E1LxwUtAzM", + "id": { + "kind": "youtube#channel", + "channelId": "UC6hAYNOWMmuqOBvFOuAFKwA" + }, + "snippet": { + "publishedAt": "2018-03-26T18:54:06Z", + "channelId": "UC6hAYNOWMmuqOBvFOuAFKwA", + "title": "3Blue1Brown Русский", + "description": "3Blue1Brown, созданный Грантом Сандэрсоном (Grant Sanderson), Это комбинация Математики и развлечения - в ...", + "thumbnails": { + "default": { + "url": "https://yt3.ggpht.com/ytc/AIdro_km1q4dyRhqPHfRXTuQg_B5iLVCeXhJtHyO4FVxXPB4xIU=s88-c-k-c0xffffffff-no-rj-mo" + }, + "medium": { + "url": "https://yt3.ggpht.com/ytc/AIdro_km1q4dyRhqPHfRXTuQg_B5iLVCeXhJtHyO4FVxXPB4xIU=s240-c-k-c0xffffffff-no-rj-mo" + }, + "high": { + "url": "https://yt3.ggpht.com/ytc/AIdro_km1q4dyRhqPHfRXTuQg_B5iLVCeXhJtHyO4FVxXPB4xIU=s800-c-k-c0xffffffff-no-rj-mo" + } + }, + "channelTitle": "3Blue1Brown Русский", + "liveBroadcastContent": "none", + "publishTime": "2018-03-26T18:54:06Z" + } + }, + { + "kind": "youtube#searchResult", + "etag": "4sE5zUzu6QtK5xmmU4ZdFa3m9kQ", + "id": { + "kind": "youtube#channel", + "channelId": "UCCbgOIWdmYncvYMbl3LjvBQ" + }, + "snippet": { + "publishedAt": "2019-01-22T22:33:40Z", + "channelId": "UCCbgOIWdmYncvYMbl3LjvBQ", + "title": "3Blue1Brown translated by Sciberia", + "description": "Наши переводы MIT и Stanford: youtube.com/c/sciberia_science.", + "thumbnails": { + "default": { + "url": "https://yt3.ggpht.com/ytc/AIdro_kupS03sbv7wxGaGwMbIUOIo0pDuSG1NlvcC-MmIOE4gA=s88-c-k-c0xffffffff-no-rj-mo" + }, + "medium": { + "url": "https://yt3.ggpht.com/ytc/AIdro_kupS03sbv7wxGaGwMbIUOIo0pDuSG1NlvcC-MmIOE4gA=s240-c-k-c0xffffffff-no-rj-mo" + }, + "high": { + "url": "https://yt3.ggpht.com/ytc/AIdro_kupS03sbv7wxGaGwMbIUOIo0pDuSG1NlvcC-MmIOE4gA=s800-c-k-c0xffffffff-no-rj-mo" + } + }, + "channelTitle": "3Blue1Brown translated by Sciberia", + "liveBroadcastContent": "none", + "publishTime": "2019-01-22T22:33:40Z" + } + }, + { + "kind": "youtube#searchResult", + "etag": "rEjOIyCKfTE_GRT-nahPaewhvWs", + "id": { + "kind": "youtube#channel", + "channelId": "UC4kAnPtYF9ulAaC7Dap9BDA" + }, + "snippet": { + "publishedAt": "2022-07-29T13:22:15Z", + "channelId": "UC4kAnPtYF9ulAaC7Dap9BDA", + "title": "3Blue1Brown UA", + "description": "3Blue1Brown — канал, створений Грантом Сандероном і є певною комбінацією математики та розважального контенту, ...", + "thumbnails": { + "default": { + "url": "https://yt3.ggpht.com/DdRS6NY1DtMRo3WagAfz_AEflwCHJlyc1b2WMc-SaVUoQ20pBr115-lauL26BQeDWQnVH8SWow=s88-c-k-c0xffffffff-no-rj-mo" + }, + "medium": { + "url": "https://yt3.ggpht.com/DdRS6NY1DtMRo3WagAfz_AEflwCHJlyc1b2WMc-SaVUoQ20pBr115-lauL26BQeDWQnVH8SWow=s240-c-k-c0xffffffff-no-rj-mo" + }, + "high": { + "url": "https://yt3.ggpht.com/DdRS6NY1DtMRo3WagAfz_AEflwCHJlyc1b2WMc-SaVUoQ20pBr115-lauL26BQeDWQnVH8SWow=s800-c-k-c0xffffffff-no-rj-mo" + } + }, + "channelTitle": "3Blue1Brown UA", + "liveBroadcastContent": "none", + "publishTime": "2022-07-29T13:22:15Z" + } + } + ] +} \ No newline at end of file diff --git a/lib/src/test/resources/api_v3_responses/playlist_page_one.json b/lib/src/test/resources/api_v3_responses/playlist_page_one.json new file mode 100644 index 0000000..6d4aa56 --- /dev/null +++ b/lib/src/test/resources/api_v3_responses/playlist_page_one.json @@ -0,0 +1,58 @@ +{ + "kind": "youtube#playlistItemListResponse", + "etag": "nseEjC2LxBDCPe0ex6AlDjFiIak", + "nextPageToken": "EAAajgFQVDpDQUVpRURJNE9VWTBRVFEyUkVZd1FUTXdSRElvQVVpN3BfbVAwY3FHQTFBQldrVWlRMmxLVVZSR2NFbFZWVGxwVkRGa1ZWVlZVbEJoYlRGMlRURnJNbEZWVW5STlJrNXFWakJHYzFKV2FHMU1WMXAzUldkM1NYRkxlVTl6ZDFsUkxVbFRkWGgzU1NJ", + "items": [ + { + "kind": "youtube#playlistItem", + "etag": "DHYqYNlNsbiH6IOb_kgcyfzxxos", + "id": "UExaSFFPYk9XVFFET2ptbzNZNkFEbTBTY1dBbEVYZi1mcC41NkI0NEY2RDEwNTU3Q0M2", + "snippet": { + "publishedAt": "2020-04-12T18:07:57Z", + "channelId": "UCYO_jab_esuFRV4b17AJtAw", + "title": "Binomial distributions | Probabilities of probabilities, part 1", + "description": "Part 2: https://youtu.be/ZA4JkHKZM50\nHelp fund future projects: https://www.patreon.com/3blue1brown\nAn equally valuable form of support is to simply share some of the videos.\nSpecial thanks to these supporters: http://3b1b.co/beta1-thanks\n\nJohn Cook post: https://www.johndcook.com/blog/2011/09/27/bayesian-amazon/\n\n------------------\n\nThese animations are largely made using manim, a scrappy open-source python library: https://github.com/3b1b/manim\n\nIf you want to check it out, I feel compelled to warn you that it's not the most well-documented tool, and it has many other quirks you might expect in a library someone wrote with only their own use in mind.\n\nMusic by Vincent Rubinetti.\nDownload the music on Bandcamp:\nhttps://vincerubinetti.bandcamp.com/album/the-music-of-3blue1brown\n\nStream the music on Spotify:\nhttps://open.spotify.com/album/1dVyjwS8FBqXhRunaG5W5u\n\nIf you want to contribute translated subtitles or to help review those that have already been made by others and need approval, you can click the gear icon in the video and go to subtitles/cc, then \"add subtitles/cc\". I really appreciate those who do this, as it helps make the lessons accessible to more people.\n\n------------------\n\n3blue1brown is a channel about animating math, in all senses of the word animate. And you know the drill with YouTube, if you want to stay posted on new videos, subscribe: http://3b1b.co/subscribe\n\nVarious social media stuffs:\nWebsite: https://www.3blue1brown.com\nTwitter: https://twitter.com/3blue1brown\nReddit: https://www.reddit.com/r/3blue1brown\nInstagram: https://www.instagram.com/3blue1brown_animations/\nPatreon: https://patreon.com/3blue1brown\nFacebook: https://www.facebook.com/3blue1brown", + "thumbnails": { + "default": { + "url": "https://i.ytimg.com/vi/8idr1WZ1A7Q/default.jpg", + "width": 120, + "height": 90 + }, + "medium": { + "url": "https://i.ytimg.com/vi/8idr1WZ1A7Q/mqdefault.jpg", + "width": 320, + "height": 180 + }, + "high": { + "url": "https://i.ytimg.com/vi/8idr1WZ1A7Q/hqdefault.jpg", + "width": 480, + "height": 360 + }, + "standard": { + "url": "https://i.ytimg.com/vi/8idr1WZ1A7Q/sddefault.jpg", + "width": 640, + "height": 480 + }, + "maxres": { + "url": "https://i.ytimg.com/vi/8idr1WZ1A7Q/maxresdefault.jpg", + "width": 1280, + "height": 720 + } + }, + "channelTitle": "3Blue1Brown", + "playlistId": "PLZHQObOWTQDOjmo3Y6ADm0ScWAlEXf-fp", + "position": 0, + "resourceId": { + "kind": "youtube#video", + "videoId": "8idr1WZ1A7Q" + }, + "videoOwnerChannelTitle": "3Blue1Brown", + "videoOwnerChannelId": "UCYO_jab_esuFRV4b17AJtAw" + } + } + ], + "pageInfo": { + "totalResults": 2, + "resultsPerPage": 1 + } +} \ No newline at end of file diff --git a/lib/src/test/resources/api_v3_responses/playlist_page_two.json b/lib/src/test/resources/api_v3_responses/playlist_page_two.json new file mode 100644 index 0000000..bffce33 --- /dev/null +++ b/lib/src/test/resources/api_v3_responses/playlist_page_two.json @@ -0,0 +1,58 @@ +{ + "kind": "youtube#playlistItemListResponse", + "etag": "Sb9EayDoodATYTMAN6O2wIJ4IgE", + "prevPageToken": "EAEajgFQVDpDQUVpRURJNE9VWTBRVFEyUkVZd1FUTXdSRElvQVVpN3BfbVAwY3FHQTFBQVdrVWlRMmxLVVZSR2NFbFZWVGxwVkRGa1ZWVlZVbEJoYlRGMlRURnJNbEZWVW5STlJrNXFWakJHYzFKV2FHMU1WMXAzUldkM1NYRkxlVTl6ZDFsUkxVbFRkWGgzU1NJ", + "items": [ + { + "kind": "youtube#playlistItem", + "etag": "vo2tv_GL8bfm3DnjKJCmtyC-2pA", + "id": "UExaSFFPYk9XVFFET2ptbzNZNkFEbTBTY1dBbEVYZi1mcC4yODlGNEE0NkRGMEEzMEQy", + "snippet": { + "publishedAt": "2020-04-12T18:07:57Z", + "channelId": "UCYO_jab_esuFRV4b17AJtAw", + "title": "Why “probability of 0” does not mean “impossible” | Probabilities of probabilities, part 2", + "description": "An introduction to probability density functions\nHelp fund future projects: https://www.patreon.com/3blue1brown\nAn equally valuable form of support is to simply share some of the videos.\nSpecial thanks to these supporters: http://3b1b.co/thanks\r\n\r\nCurious about measure theory? This does require some background in real analysis, but if you want to dig in, here is a textbook by the always-great Terence Tao.\nhttps://terrytao.files.wordpress.com/2012/12/gsm-126-tao5-measure-book.pdf\n\nAlso, for the real analysis buffs among you, there was one statement I made in this video that is a rather nice puzzle. Namely, if the probabilities for each value in a given range (of the real number line) are all non-zero, no matter how small, their sum will be infinite. This isn't immediately obvious, given that you can have convergent sums of countable infinitely many values, but if you're up for it see if you can prove that the sum of any uncountable infinite collection of positive values must blow up to infinity.\n\nThanks to these viewers for their contributions to translations\nHebrew: Omer Tuchfeld\n\n------------------\r\n\r\nThese animations are largely made using manim, a scrappy open source python library: https://github.com/3b1b/manim\r\n\r\nIf you want to check it out, I feel compelled to warn you that it's not the most well-documented tool, and it has many other quirks you might expect in a library someone wrote with only their own use in mind.\r\n\r\nMusic by Vincent Rubinetti.\r\nDownload the music on Bandcamp:\r\nhttps://vincerubinetti.bandcamp.com/album/the-music-of-3blue1brown\r\n\r\nStream the music on Spotify:\r\nhttps://open.spotify.com/album/1dVyjwS8FBqXhRunaG5W5u\r\n\r\nIf you want to contribute translated subtitles or to help review those that have already been made by others and need approval, you can click the gear icon in the video and go to subtitles/cc, then \"add subtitles/cc\". I really appreciate those who do this, as it helps make the lessons accessible to more people.\r\n\r\n------------------\r\n\r\n3blue1brown is a channel about animating math, in all senses of the word animate. And you know the drill with YouTube, if you want to stay posted on new videos, subscribe: http://3b1b.co/subscribe\r\n\r\nVarious social media stuffs:\r\nWebsite: https://www.3blue1brown.com\r\nTwitter: https://twitter.com/3blue1brown\r\nReddit: https://www.reddit.com/r/3blue1brown\r\nInstagram: https://www.instagram.com/3blue1brown_animations/\r\nPatreon: https://patreon.com/3blue1brown\r\nFacebook: https://www.facebook.com/3blue1brown", + "thumbnails": { + "default": { + "url": "https://i.ytimg.com/vi/ZA4JkHKZM50/default.jpg", + "width": 120, + "height": 90 + }, + "medium": { + "url": "https://i.ytimg.com/vi/ZA4JkHKZM50/mqdefault.jpg", + "width": 320, + "height": 180 + }, + "high": { + "url": "https://i.ytimg.com/vi/ZA4JkHKZM50/hqdefault.jpg", + "width": 480, + "height": 360 + }, + "standard": { + "url": "https://i.ytimg.com/vi/ZA4JkHKZM50/sddefault.jpg", + "width": 640, + "height": 480 + }, + "maxres": { + "url": "https://i.ytimg.com/vi/ZA4JkHKZM50/maxresdefault.jpg", + "width": 1280, + "height": 720 + } + }, + "channelTitle": "3Blue1Brown", + "playlistId": "PLZHQObOWTQDOjmo3Y6ADm0ScWAlEXf-fp", + "position": 1, + "resourceId": { + "kind": "youtube#video", + "videoId": "ZA4JkHKZM50" + }, + "videoOwnerChannelTitle": "3Blue1Brown", + "videoOwnerChannelId": "UCYO_jab_esuFRV4b17AJtAw" + } + } + ], + "pageInfo": { + "totalResults": 2, + "resultsPerPage": 1 + } +} \ No newline at end of file diff --git a/lib/src/test/resources/api_v3_responses/playlist_single_page.json b/lib/src/test/resources/api_v3_responses/playlist_single_page.json new file mode 100644 index 0000000..2c37fb3 --- /dev/null +++ b/lib/src/test/resources/api_v3_responses/playlist_single_page.json @@ -0,0 +1,104 @@ +{ + "kind": "youtube#playlistItemListResponse", + "etag": "YmC5Lm32yhnylJYtX50x5vJ3Sx8", + "items": [ + { + "kind": "youtube#playlistItem", + "etag": "DHYqYNlNsbiH6IOb_kgcyfzxxos", + "id": "UExaSFFPYk9XVFFET2ptbzNZNkFEbTBTY1dBbEVYZi1mcC41NkI0NEY2RDEwNTU3Q0M2", + "snippet": { + "publishedAt": "2020-04-12T18:07:57Z", + "channelId": "UCYO_jab_esuFRV4b17AJtAw", + "title": "Binomial distributions | Probabilities of probabilities, part 1", + "description": "Part 2: https://youtu.be/ZA4JkHKZM50\nHelp fund future projects: https://www.patreon.com/3blue1brown\nAn equally valuable form of support is to simply share some of the videos.\nSpecial thanks to these supporters: http://3b1b.co/beta1-thanks\n\nJohn Cook post: https://www.johndcook.com/blog/2011/09/27/bayesian-amazon/\n\n------------------\n\nThese animations are largely made using manim, a scrappy open-source python library: https://github.com/3b1b/manim\n\nIf you want to check it out, I feel compelled to warn you that it's not the most well-documented tool, and it has many other quirks you might expect in a library someone wrote with only their own use in mind.\n\nMusic by Vincent Rubinetti.\nDownload the music on Bandcamp:\nhttps://vincerubinetti.bandcamp.com/album/the-music-of-3blue1brown\n\nStream the music on Spotify:\nhttps://open.spotify.com/album/1dVyjwS8FBqXhRunaG5W5u\n\nIf you want to contribute translated subtitles or to help review those that have already been made by others and need approval, you can click the gear icon in the video and go to subtitles/cc, then \"add subtitles/cc\". I really appreciate those who do this, as it helps make the lessons accessible to more people.\n\n------------------\n\n3blue1brown is a channel about animating math, in all senses of the word animate. And you know the drill with YouTube, if you want to stay posted on new videos, subscribe: http://3b1b.co/subscribe\n\nVarious social media stuffs:\nWebsite: https://www.3blue1brown.com\nTwitter: https://twitter.com/3blue1brown\nReddit: https://www.reddit.com/r/3blue1brown\nInstagram: https://www.instagram.com/3blue1brown_animations/\nPatreon: https://patreon.com/3blue1brown\nFacebook: https://www.facebook.com/3blue1brown", + "thumbnails": { + "default": { + "url": "https://i.ytimg.com/vi/8idr1WZ1A7Q/default.jpg", + "width": 120, + "height": 90 + }, + "medium": { + "url": "https://i.ytimg.com/vi/8idr1WZ1A7Q/mqdefault.jpg", + "width": 320, + "height": 180 + }, + "high": { + "url": "https://i.ytimg.com/vi/8idr1WZ1A7Q/hqdefault.jpg", + "width": 480, + "height": 360 + }, + "standard": { + "url": "https://i.ytimg.com/vi/8idr1WZ1A7Q/sddefault.jpg", + "width": 640, + "height": 480 + }, + "maxres": { + "url": "https://i.ytimg.com/vi/8idr1WZ1A7Q/maxresdefault.jpg", + "width": 1280, + "height": 720 + } + }, + "channelTitle": "3Blue1Brown", + "playlistId": "PLZHQObOWTQDOjmo3Y6ADm0ScWAlEXf-fp", + "position": 0, + "resourceId": { + "kind": "youtube#video", + "videoId": "8idr1WZ1A7Q" + }, + "videoOwnerChannelTitle": "3Blue1Brown", + "videoOwnerChannelId": "UCYO_jab_esuFRV4b17AJtAw" + } + }, + { + "kind": "youtube#playlistItem", + "etag": "vo2tv_GL8bfm3DnjKJCmtyC-2pA", + "id": "UExaSFFPYk9XVFFET2ptbzNZNkFEbTBTY1dBbEVYZi1mcC4yODlGNEE0NkRGMEEzMEQy", + "snippet": { + "publishedAt": "2020-04-12T18:07:57Z", + "channelId": "UCYO_jab_esuFRV4b17AJtAw", + "title": "Why “probability of 0” does not mean “impossible” | Probabilities of probabilities, part 2", + "description": "An introduction to probability density functions\nHelp fund future projects: https://www.patreon.com/3blue1brown\nAn equally valuable form of support is to simply share some of the videos.\nSpecial thanks to these supporters: http://3b1b.co/thanks\r\n\r\nCurious about measure theory? This does require some background in real analysis, but if you want to dig in, here is a textbook by the always-great Terence Tao.\nhttps://terrytao.files.wordpress.com/2012/12/gsm-126-tao5-measure-book.pdf\n\nAlso, for the real analysis buffs among you, there was one statement I made in this video that is a rather nice puzzle. Namely, if the probabilities for each value in a given range (of the real number line) are all non-zero, no matter how small, their sum will be infinite. This isn't immediately obvious, given that you can have convergent sums of countable infinitely many values, but if you're up for it see if you can prove that the sum of any uncountable infinite collection of positive values must blow up to infinity.\n\nThanks to these viewers for their contributions to translations\nHebrew: Omer Tuchfeld\n\n------------------\r\n\r\nThese animations are largely made using manim, a scrappy open source python library: https://github.com/3b1b/manim\r\n\r\nIf you want to check it out, I feel compelled to warn you that it's not the most well-documented tool, and it has many other quirks you might expect in a library someone wrote with only their own use in mind.\r\n\r\nMusic by Vincent Rubinetti.\r\nDownload the music on Bandcamp:\r\nhttps://vincerubinetti.bandcamp.com/album/the-music-of-3blue1brown\r\n\r\nStream the music on Spotify:\r\nhttps://open.spotify.com/album/1dVyjwS8FBqXhRunaG5W5u\r\n\r\nIf you want to contribute translated subtitles or to help review those that have already been made by others and need approval, you can click the gear icon in the video and go to subtitles/cc, then \"add subtitles/cc\". I really appreciate those who do this, as it helps make the lessons accessible to more people.\r\n\r\n------------------\r\n\r\n3blue1brown is a channel about animating math, in all senses of the word animate. And you know the drill with YouTube, if you want to stay posted on new videos, subscribe: http://3b1b.co/subscribe\r\n\r\nVarious social media stuffs:\r\nWebsite: https://www.3blue1brown.com\r\nTwitter: https://twitter.com/3blue1brown\r\nReddit: https://www.reddit.com/r/3blue1brown\r\nInstagram: https://www.instagram.com/3blue1brown_animations/\r\nPatreon: https://patreon.com/3blue1brown\r\nFacebook: https://www.facebook.com/3blue1brown", + "thumbnails": { + "default": { + "url": "https://i.ytimg.com/vi/ZA4JkHKZM50/default.jpg", + "width": 120, + "height": 90 + }, + "medium": { + "url": "https://i.ytimg.com/vi/ZA4JkHKZM50/mqdefault.jpg", + "width": 320, + "height": 180 + }, + "high": { + "url": "https://i.ytimg.com/vi/ZA4JkHKZM50/hqdefault.jpg", + "width": 480, + "height": 360 + }, + "standard": { + "url": "https://i.ytimg.com/vi/ZA4JkHKZM50/sddefault.jpg", + "width": 640, + "height": 480 + }, + "maxres": { + "url": "https://i.ytimg.com/vi/ZA4JkHKZM50/maxresdefault.jpg", + "width": 1280, + "height": 720 + } + }, + "channelTitle": "3Blue1Brown", + "playlistId": "PLZHQObOWTQDOjmo3Y6ADm0ScWAlEXf-fp", + "position": 1, + "resourceId": { + "kind": "youtube#video", + "videoId": "ZA4JkHKZM50" + }, + "videoOwnerChannelTitle": "3Blue1Brown", + "videoOwnerChannelId": "UCYO_jab_esuFRV4b17AJtAw" + } + } + ], + "pageInfo": { + "totalResults": 2, + "resultsPerPage": 2 + } +} \ No newline at end of file