add RFC for HKTs

[erip]

Rendered

[srenatus]

Could there be another set of standard-lib-like modules instead? I'm thinking of scala's cats stuff; not necessarily as the one true way to go, but as a means to reduce this risk, and letting users opt into this.

[erip]

Yes and no, I think. If the standard lib supported HKTs, I could definitely see libraries to support more functional style things (like Cats as you mentioned). I actually opened this because I was hoping to create such a lib. The ultimate blocker is that there's no such language feature for HKTs.

[jemc]

We'll have to consider how this would be implemented.

There has been some somewhat related discussion around this RFC: #123

[SeanTAllen]

From sync: This dovetails a little with #123. We had a discussion of this during sync and decided to pick this up again for more conversation next week after some research from @jemc and @sylvanc into how Scala handles HKTs.

[erip]

FWIW, I started a little project here, which could see some awesome things happen if HKTs were implemented. If supported, this would satisfy @srenatus's comment.

[shelby3]

I wasn’t aware of this thread then I posted to the mailing list asking if there’s interest in HKTs. Note my pertinent comment at the mailing list about type-level lambdas.

[jemc]

Some reading material that @sylvanc shared for us on this topic from the work on the Dotty language:

• http://guillaume.martres.me/publications/dotty-hk.pdf

And here's the user documentation for HKTs in Dotty:

• https://dotty.epfl.ch/docs/internals/higher-kinded-v2.html

[erip]

I'd love to help, but I'm very new to Pony. If someone would be willing to help me up front, I would be happy to try to make some contributions to this.

[jemc]

I have a standing offer to do some calls to synchronously knowledge share and answer questions for anyone who is interested in becoming a serious contributor to the compiler. I don't know everything about the whole codebase, but I'm happy to share what I know and get you pointed in the right direction if you have some time to commit to this (or any other compiler bug/feature ticket you want to pick up)

[erip]

I'm playing with the compiler on this branch and hopefully that's where HKTs will land... for now, I'm not planning anything fancy with implicits; those are mostly a nice-to-have and not a blocker.

It seems like the first step is to change the parser rules to allow optional [ followed by comma-delimited underscores followed by ] in DEF(typeparam) in parser.c.

[mfelsche]

During discussion on slack with @erip and @jemc the question about interaction of HKTs with iftype came up.

Given the following example for more context:

trait Functor[F[_]]
  fun map[A, B](fa: F[A], f: {(A): B} box): F[B]

type Option[A] is (A | None)

primitive OptionFunctor is Functor[Option[_]]
  fun map[A, B](fa: Option[A], f: {(A): B} box): Option[B] => None

The F[T] type-param in Functor[F[_]] denotes a HKT. They can be applied with types that accept typeargs itself, in this case with only 1 type-arg. Option[_] is an example of such a HKT type-arg. This type-arg can only be used in code, if it is provided the missing type-arg (see Option[A] and Option[B in OptionFunctor.map).

Should it be possible, e.g. in the body of OptionFunctor.map to issue an iftype on the HKT-type-param Option like?:

iftype Option[_] <: String then
  ...
end

That is, Option is a HKT, a type-arg to a HKT-type-param, but cannot be fully reified yet. It needs to be provided with 1 (in this case, could be more) type-arg to be reifiable. But String is a possible reification/instance of Option[_]. Should this be allowed?
Or should we only allow the following? That is to only allow using iftype on the HKT with a given type-arg:

iftype Option[A] <: String then
  ...
end

What would the resulting logic for delaying the reification be? It should have all information at the call-side of OptionFunctor.map, right? And there it should be reified.
In the example above, OptionFunctor needs to be reified differently for each application, considering both type-args on the class and the method: OptionFunctor[HKT].map[T1, T2]. So we get different reifications for the whole class for each combination of HKT and T1 and T2, right? On reification a different iftype branch is chosen based on the concrete type, we reify for, right? But only Option[A] and Option[B] get actually reified, not Option[_] as it can have > 1 instantiation. So, it sounds to me like we should only allow the second iftype case above, but i wanted to make sure, we don't go down a wrong path.

[erip]

I prefer just passing the unparameterizered type arg into the (e.g.) Functor, but if this facilitates implementation or maintenance, I'm all for putting in a [_] after the type. The reason I prefer the unparameterized version is purely philosophical. The ID of the HKT (F in F[_]) is technically the type that is higher-kinded; F will be the type that satisfies the requirement that it takes arguments (a single one in the case of Functor). The way to think about this is that any type ID can be passed as a type argument:

• a nullary type constructable argument (*) (e.g., numbers, strings, etc.)
• a unary type constructable argument (* -> *) (e.g., lists, sets, etc.)
• a binary type constructable argument (* -> * -> *) (e.g., functions, maps, etc.)
• ...

In the above formulation, just the type (the nullary constructable) which would be constructed needs to be passed (F or whatever).

Again, this is nuts and bolts -- I'm happy to move forward in whichever direction is most tractable.

---

Another reason for prefering just the type parameter: when expressing a constraint, the following would be quite cumbersome and ugly:

trait Something[F[_]: Functor[_]]
    // ...

[erip]

@mfelsche and @jemc, do either of you have thoughts?

[mfelsche]

I am fine either way for the syntax of how to pass typeargs for HKT params for now. The internals of implementing it in the compiler would differ, but that is of lesser importance to me.

I'd suggest describing the intended approach for syntax and implementation in greater detail in this RFC document to get some initial feedback before investing too much work on the implementation. The concept should be solid.

[erip]

@mfelsche that semes like a reasonable idea. I'll add a note here so I can remember to add it to the formal proposal:

Need to include:

• Syntax for defining a class with an HKT
  • interface Functor[F[_]]
• Syntax for defining an instance which nominalizes an HKT
  • primitive OptionFunctor is Functor[Option]
• Syntax for applying a constrait where the constraint is a class with an HKT
  • fun foo[F[_]: Functor, A](f: F[A])
• Syntax for returning a reified HKT:
  • interface Functor[F[_]] fun map[A, B](f: {(A): B} box, fa: F[A]): F[B]
• Syntax for using an HKT in iftype
  • tbd

Probably more