Play module directory

[gmethvin]

Now that "modules" are a thing people are interested in in 2.x, we might want to have the /modules page use/reference the module directory we have at https://www.playframework.com/documentation/2.4.x/ModuleDirectory and the modules documentation at https://www.playframework.com/documentation/2.4.x/Modules.

I would actually prefer to move the 1.x modules page somewhere else altogether and maybe link to it from the new /modules page.

[jroper]

One thing we have to be careful of is that Play 1 depends on several REST endpoints served by playframework.com, they have to remain.

But, something that we should have done long ago is build a Play 2 module directory in playframework.com. It's never too late - I wonder if anyone in the community would be interested in taking on this project? Here's some use cases

• A user can login using GitHub
• A module developer can submit a new module for approval
• A module developer can edit modules once approved
• A module developer can add other GitHub logins to be able to edit the modules
• A module developer can specify the name, description, author, GitHub repo, ivy organization/artifactId and required resolvers for a module
• A module developer can create/update/delete multiple versions for a module
• A module developer can specify the version and compatible Play versions for a module version
• An admin can approve modules
• An admin can delete modules
• An admin can do everything a module developer can do
• An end user can browse modules
• An end user can search for modules
• An end user can view a module, the current version, and its version history

That probably covers it. Some non functional requirements:

• The code must not pull in any dependencies that transitively depend on Play but are not maintained by the Play team. This is because plafyramework.com is used to dogfood milestones of Play etc, but if third party Play modules are used, that will prevent this. This, for example, means that play-reactivemongo is out of the question.

[gmethvin]

Also it should probably mention on the existing module pages prominently that the module is for 1.x. If linked from a google search or another page a user could easily be confused.

Also, I was exploring modules for RabbitMQ and noticed that https://www.rabbitmq.com/devtools.html links to a 1.x module (https://www.playframework.com/modules/rabbitmq-0.0.9/home) under the "Scala" section. Their documentation is obviously misleading, but we can at least help by being explicit on our end.

[jroper]

I'm going to offer this as an Outreachy project for Round 11.

[Shruti9520]

Hi, @gmethvin @jroper

I want suggestions regarding first four use cases mentioned above-

The user can sign in using Github. We can implement this using OAuth authentication ( Java OAuth). Once the user is logged in, it will be redirected to the main page of website and there we can provide a UI for the user to upload their developed module from their native system/computer to the website. Now, after submission, we can generate a token ( exclusively for the developer who just uploaded his/her module) and pass it to the admin of the website ( for the validation of that token). Once the token is validated, we can set a boolean flag and use that to allow the module developer to edit module.
Next, we can create a form or search bar kind of thing where user can enter GitHub usernames of other users ( who have already logged in to the website) and invite them to edit his/her module.

Is this a good approach or should I think in another way?

Thank you.

[Dulya]

In the current implementation, I think there might be some modifications to the existing structure at the database level and will have to perform a data migration as well if we would go with the same structure. It may break the existing 1.x modules directory pages as well.
Considering the above requirements for the implementation of 2.x module directory, I think it's better to go with a new structure from the database level.

Do we need the new system mentioned above to support existing Play 1.x modules as well?

[gmethvin]

1.x modules currently use a simple system that loads from the filesystem: https://github.com/playframework/playframework.com/blob/master/app/services/modules/ModulesLookup.scala#L35. This could easily be moved to a separate "1.x modules" section. @xael-fry any opinion?

Flat files stored on GitHub might not be a bad idea, as long as we can satisfy the above use cases.

[Dulya]

I generalized and classified tasks listed above in my forked repo.
https://github.com/Dulya/playframework.com/wiki/Module-Directory---Implementation-Flow

[xael-fry]

@gmethvin Play 1 use the website and the module section to download/install module. So we could organize it a different way but url should be the same, or at least prepare a migration and update of play 1 to handle the new organisation

[marcospereira]

Play Modules Directory

Here is a compilation of ideas to module directory.

What I would like to know about a module

I think there we can use Github and a file (.play.yml) as the information source for the modules directory. The file is a nice to have and we can add it later. Github should be the primary information source here.

Information from Github

1. Basic information:
   1. Name
   2. Documentation URL
   3. Owner/Author
   4. Basic description
   5. Latest release
   6. License
2. How popular it is: this is important because more popular modules usually have more users, which means they are more battle tested. We can use Github stars, watchers and forks to say how popular a module is.
3. How well maintained it is: How many contributors (maybe who are the contributors too), releases and when was the last commit.
4. Topics: Repository topics that could be useful to later filter modules.

Although Github API has a lot of repository information, we should use only what is really needed. This way we can later add other source control repository hosting services (Bitbucket, Gitlab, etc) in the future and there is a bigger chance they also have all the necessary information.

Information from .play.yml

Developers can use this file to add information that is more specific to Play:

1. Which Play versions are supported by the module
2. Relevant links:
   1. Mailing list
   2. Gitter or Slack channel
   3. Build (Travis, CircleCI, etc)

Keeping information up-to-date

We can use Github Webhooks to retrieve the updated information every time a relevant event (new commit, new release, new tag, etc) occurs on module directory.

Modules directory and Play website

Modules directory would live in https://modules.playframework.com.

Modules directory should be another application. The advantage here is that we can move faster, there is no risk of a change in modules directory impacting Play website, we can deploy it separately and we can keep both applications as simple as possible. Given that, I'm not sure which kind of persistence we should use, but it would be good to have a database as a service so that we won't need to maintain (install, update, monitoring, etc) it.

We can also think about deploying to Heroku which has managed data store services.

Integration with Scaladex

I'm not sure if Scaladex has an API that we can use to push and pull modules data.

Some nice modules/plugins directories

It could be useful to look at these services to see if they have something interesting we want for Play Modules Directory:

• https://index.scala-lang.org/
• https://plugins.jenkins.io/
• https://crates.io/

[gmethvin]

@marcospereira Yeah, I totally agree that a separate modules site would be the best way to go, and having a .play.yml would be nice since it doesn't require any user interface on our end for providing that info.