(Chihiro) Generating the List

The core functionality of PSDLE, the bread and butter, revolves around its ability to create the list of content and manipulate it to create the final result.

The way PSDLE has done this has evolved over time and will all be explained.
There will also be two major parts to this, the evolution and then the actual current implementation.

PSDLE's Evolution

The Old Days (original PSDLE)

The first iteration of list generation used the store's list that had already been generated and shown to the user. This required PSDLE to run on the actual download list page where it would read each item's HTML, grab what it can, and then change to the next page.

This was painfully slow and limited. But it was good enough for PSDLE to be in a proof-of-concept stage.

Aside from being slow and limited, it was actually really slow. Go to the restroom slow. The store was having to generate each page used before PSDLE could read it and this could lead to desync issues if it was going too slow. Switch to another tab? Desync. Random computer hiccup? Desync. This also obviously required PSDLE to be run on the download list, and nowhere else.

As of April 18th 2016, this concept still works for the most part and you can try it on the original Download List page using this version of PSDLE.

Age of Discovery

After watching the store's network requests while making sure PSDLE wasn't dying, I noticed a certain network request to a URL involving "Entitlements". Not knowing what for, I looked into the request and noticed it contained a paginated (N or 100 items each) list of an accounts... entitlements. Games, videos, subscriptions.

The prominent thing that stood out was how much information it provided. A lot more than the original download list. By using this new information, PS+ support was swiftly added. The main advantage of this was it was no longer required to run PSDLE on the download list page and instead literally anywhere (even technically not on the store, but that never happened).

Around this time, I also stumbled upon a part of the Chihiro API which provided actual information about the content (ffdcaf...) and incorporated it into PSDLE's content list for categories. This usage of the Chihiro API would eventually be referred to as Catalog by PSDLE.

Golden Time

However, after now tapping into the Entitlements and Catalog there would almost be no progress left. Fortunately, that was not the case.

I came to a realization that the store was always ready to go, and PSDLE was wasting resources by fetching the entitlements separately. Lo and behold, the store itself was keeping a copy of the entitlements around! After discovering this, generating the standard list took only a couple hundred milliseconds since there was now no waiting on URL requests. Going from potentially several minute long generation times to less than a second for the same thing changed the game, vastly improved accuracy, and paved the way for other time-consuming functions (like Catalog) to not be a chore on top of it.

Throughout, PSDLE was still spitting out the same results having just acquired them via alternative means.

Pulling it all Together

The first step of generating the download list, is to actually acquire it. As above, PSDLE has reached most likely the pinnacle of this by using the store's local copy.

In short, entitlement data contains the bare minimum information required to get off the ground. That information includes the product ID, item/unique ID, item name, date of purchase, and size among a few other things. Combining and using these these properly would achieve basic parity with the download list, barring queue support.

However, PSDLE does not stop there. Before it actually does anything with that newly acquired information there are a few more additional data sources to check first:

• Injected entitlements.
• A debug feature allowing users to export their entitlements for injection and act like a normal entitlement for anybody else.
• Primarily used for providing problematic entries to me for debugging locally.
• Combined with original download list before proceeding.
• The Catalog. (if enabled)
• Looks up individual/unique game IDs to store additional information like category, descriptions, genres, media, and more.
• PS TV support. (if enabled, en-us only)
• Simple lookup on the store's PS TV compatibility list.

PSDLE utilizes two tables/lists to function. A "master" table that contains everything and a "working" or "current" table that contains volatile entries. Every time a system or category is filtered or a search is ran PSDLE empties the working table and copies over valid entries from the master table. In reality, the master table is never actually used for looking up or showing information except to be copied over to the working table.

When the time comes for PSDLE to generate the displayed list, it uses the working table (even if it's an exact replica of the master!). Exporting also uses the working table, since it's already good to go and does not require additional changes.

---

Further reading:

• Enhanced Accuracy (liberties PSDLE takes to enhance accuracy (not features!))
• Download Queue (how PSDLE queues items)
• Determining Platforms (using Entitlement data)