-
-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refactor: Split Request, Query and Response interfaces from App #1831
Conversation
11c37bd
to
cde8645
Compare
@wolfgangwalther it's becoming obvious that #1814 (comment) was a lie... This splits up pieces from |
a30feb6
to
31fb447
Compare
All good. I was a bit bored by #1830 already.
Hm. I had hoped we could get to a point where all IO handling was in Do you think that's possible / would make things easier? My hope is that unit testing of query generation would be easier to do. Also what strikes me as odd is the fact, that I really like those interfaces: updateResponse :: ApiRequest -> WriteQueryResult -> Wai.Response
singleUpsertResponse :: ApiRequest -> WriteQueryResult -> Wai.Response
deleteResponse :: ApiRequest -> WriteQueryResult -> Wai.Response Those are still a bit heavy: readResponse :: Bool -> QualifiedIdentifier -> AppConfig -> ApiRequest -> Statements.ResultsWithCount -> Maybe Int64 -> [GucHeader] -> Maybe HTTP.Status -> Wai.Response
createResponse :: QualifiedIdentifier -> DbStructure -> ApiRequest -> WriteQueryResult -> Wai.Response
invokeResponse :: InvokeMethod -> ApiRequest -> Statements.ProcResults -> [GucHeader] -> Maybe HTTP.Status -> Wai.Response This looks like we need to either extend |
Thought about this also, and how it would be nice to chain one set of three functions for all cases. I think this way seems a bit less elegant, but makes more impossible states impossible: It's obvious that no
Yes, definitely - the signatures of both the |
Hm. Everything from |
We are pretty close actually!
A DbHandler is a Hasql.Transaction whose execution (and theoretically creation, need to check whether we have such cases left) can fail with an Error. We should be able to split this out further within Query, but as a first step this makes sense I think. |
|
2b6ec9f
to
a22d914
Compare
Was not on purpose! Fixed. I think that the whole |
I think we should make a cut here! The dispatch in |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed "everything" except Query.hs
, Request.hs
and Response.hs
so far. So... basically nothing, yet. :D
src/PostgREST/Auth.hs
Outdated
JWT.verifyClaimsAt validation secret time =<< JWT.decodeCompact payload | ||
liftEither . mapLeft jwtClaimsError $ claimsMap configJwtRoleClaimKey <$> eitherClaims | ||
AppConfig -> Request -> UTCTime -> ExceptT Error m JWTClaims | ||
jwtClaims AppConfig{..} request time = |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need some kind of naming convention for PostgREST.Request
s and Wai.Request
s. In App.hs
it was request
and req
. Now request
is the other one.
I wonder whether we should maybe not use Request
and Response
at all, given that those clash with wai like that.
What about RestRequest
and RestResponse
?
And then consistently use request
for Wai.Request
and restRequest
for PostgREST.RestRequest
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hm, I think it would be cleanest to use PostgREST.Request
and PostgREST.Response
if possible in our codebase. The most impacted module is PostgREST.App
, let me see if I can make it clearer there.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
request
and waiRequest
for names then?
9d14113
to
6e79239
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This second review is mostly about the interfaces between App
, Request
, Query
and Response
. I haven't really looked into the 3 modules in detail, yet - that's something I will do in a third round, to make sure that no unrelated changes sneaked in.
From what I can tell right now, I really like how things have been split up and where the different parts ended up.
However, I don't like the interface between those modules, too much, yet. Specifically:
App
currently needs to know about a few implementation details, namely:- The explicit pass-through of
xxxRequestInfo
. This type is purely internal actually and just a workaround to the partiality problem for constructors with different record fields. - The mere fact that each "path" (read, create, ...) is separated in each module. Whether some of those paths are handled by the same function internally or not, should be up to the module. It would be better to have one entrypoint per module - just like
Request.parse
.
- The explicit pass-through of
- On the other side some part of the interface are hidden, because they are passed on through the
xxxRequestInfo
andxxxQueryResult
types: config, version, structure in both cases, and the request inQueryResult
. The response function's interface should really be something likerespond :: Context -> Request -> Result -> Wai.Response
. Right now each step kind of takes their input and wraps it in the output. Strange!
I know that this interface resulted from "making impossible states impossible", i.e. stronger typing - at compile time. That's an absolute positive.
But, I think we can get both. Generalize App
, make the interfaces really nice - and have strong typing. Here's how:
(Note: this includes suggestions in some of the other comments below, might need to read those first - mainly about naming etc.)
{-# LANGUAGE DataKinds #-}
{-# LANGUAGE GADTs #-}
{-# LANGUAGE KindSignatures #-}
data Context = Context
{ ctxConfig :: AppConfig
, ctxPgVersion :: PgVersion
, ctxDbStructure :: DbStructure
}
data RequestType
= Read
| Create
| Update
| Upsert
| Delete
| Invoke
| Info
| OpenApi
data Request (t :: RequestType) where
ReadRequest :: ApiRequest -> ReadTree -> Request 'Read
CreateRequest :: ApiRequest -> ReadTree -> MutateQuery -> Request Create
UpdateRequest :: ApiRequest -> ReadTree -> MutateQuery -> Request Update
UpsertRequest :: ApiRequest -> ReadTree -> MutateQuery -> Request Upsert
DeleteRequest :: ApiRequest -> ReadTree -> MutateQuery -> Request Delete
InvokeRequest :: ApiRequest -> ReadTree -> Request Invoke
InfoRequest :: ApiRequest -> Request Info
OpenApiRequest :: ApiRequest -> Request OpenApi
data Result (t :: RequestType) = Result
{ resBody :: ByteString
, resGucHeaders :: [GucHeader]
, resGucStatus :: Maybe HTTP.Status
, resQueryTotal :: Int64
, resTableTotal :: Maybe Int64
}
-- public interfaces (towards App)
Request.parse :: Context -> Wai.Request -> LBS.ByteString -> Either Error (Request t)
Query.build :: Context -> Request t -> DbHandler (Result t)
Response.send :: Context -> Request t -> DbHandler (Result t) -> Wai.Response
-- examples for internal interfaces
readQuery :: Context -> Request Read -> DbHandler (Result Read)
createQuery :: Context -> Request Create -> DbHandler (Result Create)
-- ...
readResponse :: Context -> Request Read -> DbHandler (Result Read) -> Wai.Response
createResponse :: Context -> Request Create -> DbHandler (Result Create) -> Wai.Response
-- ...
This would even be more type-safe than the current interface, where you could pass a MutateQueryResult
from e.g. createQuery
into updateResponse
or so...
We can still easily see from the public interfaces, that it's not possible to mix different request and response types (Request t -> Result t
). App
is completely generic and just puts the different parts together. None of the 3 modules is limited regarding internal refactoring, because they each have 1 entrypoint only.
waiBody <- lift $ Wai.strictRequestBody waiRequest | ||
request <- liftEither $ Request.parse conf pgVer dbStructure waiRequest waiBody |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think body
was fine. waiBody
makes me second-guess - is there something "in" there, that I am missing? It's just a lazy bytestring, isn't it?
data ReadRequestInfo = ReadRequestInfo | ||
{ rrConfig :: AppConfig | ||
, rrPgVersion :: PgVersion | ||
, rrDbStructure :: DbStructure | ||
, rrApiRequest :: ApiRequest | ||
, rrHeadersOnly :: Bool | ||
, rrIdentifier :: QualifiedIdentifier | ||
, rrReadRequest :: Types.ReadRequest | ||
, rrBinaryField :: BinaryField | ||
} | ||
|
||
data MutateRequestInfo = MutateRequestInfo | ||
{ mrConfig :: AppConfig | ||
, mrPgVersion :: PgVersion | ||
, mrDbStructure :: DbStructure | ||
, mrApiRequest :: ApiRequest | ||
, mrIdentifier :: QualifiedIdentifier | ||
, mrReadRequest :: Types.ReadRequest | ||
, mrMutateRequest :: Types.MutateRequest | ||
} | ||
|
||
data InvokeRequestInfo = InvokeRequestInfo | ||
{ irConfig :: AppConfig | ||
, irPgVersion :: PgVersion | ||
, irDbStructure :: DbStructure | ||
, irApiRequest :: ApiRequest | ||
, irInvokeMethod :: InvokeMethod | ||
, irProc :: ProcDescription | ||
, irReadRequest :: Types.ReadRequest | ||
, irBinaryField :: BinaryField | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(Might be irrelevant when generalizing the interface as mentioned in the opening comment)
I think it would make the code more readable to use DuplicateRecordFields
and have those different types use the same field prefix req
. All the names that match are exactly the same. rrConfig
, mrConfig
, irConfig
- all reqConfig
, etc.
Also, the two letter prefixes are a tad too short, I think. It's almost as iXX
with ApiRequest
. Although, that's a bit tougher, because I don't see any connection at all. req
is immediately clear.
data Request | ||
= ReadRequest ReadRequestInfo | ||
| CreateRequest MutateRequestInfo | ||
| UpdateRequest MutateRequestInfo | ||
| SingleUpsertRequest MutateRequestInfo | ||
| DeleteRequest MutateRequestInfo | ||
| InfoRequest DbStructure ApiRequest QualifiedIdentifier | ||
| InvokeRequest InvokeRequestInfo | ||
| OpenApiRequest AppConfig DbStructure ApiRequest Bool Schema |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(Might be irrelevant when generalizing the interface as mentioned in the opening comment)
Info seems off. It suggests some additional, non-essential information. But this is really the meat of everything.
I suggest to rename the XXXRequestInfo
to XXXRequestPayload
, XXXRequestContext
, XXXRequestFields
or something like that. I think I like "fields" most.
The additional type added here is really just to avoid partial application of accessors, but is otherwise not really exposed - at least I don't consider it a valuable part of the interface. If this wasn't a problem we would most likely add them as record fields directly on the Request
constructors, right? I think XXXRequestFields
matches that intent best.
ReadRequest readRequestInfo -> | ||
Response.readResponse <$> Query.readQuery readRequestInfo | ||
CreateRequest mutateRequestInfo -> | ||
Response.createResponse <$> Query.createQuery mutateRequestInfo | ||
UpdateRequest mutateRequestInfo -> | ||
Response.updateResponse <$> Query.updateQuery mutateRequestInfo | ||
SingleUpsertRequest mutateRequestInfo -> | ||
Response.singleUpsertResponse <$> Query.singleUpsertQuery mutateRequestInfo | ||
DeleteRequest mutateRequestInfo -> | ||
Response.deleteResponse <$> Query.deleteQuery mutateRequestInfo |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(Might be irrelevant when generalizing the interface as mentioned in the opening comment)
I liked the "same name for all info types" from before better. For me, the difference between those is really not important here. This is just about "passing through" the fields that should actually be part of Request
directly (see other comment).
data ReadQueryResult = ReadQueryResult | ||
{ rqRequest :: ReadRequestInfo | ||
, rqQueryTotal :: Int64 | ||
, rqBody :: BS8.ByteString | ||
, rqTableTotal :: Maybe Int64 | ||
, rqGucHeaders :: [GucHeader] | ||
, rqGucStatus :: Maybe HTTP.Status | ||
} | ||
|
||
data MutateQueryResult = MutateQueryResult | ||
{ resRequest :: MutateRequestInfo | ||
, resQueryTotal :: Int64 | ||
, resFields :: [ByteString] | ||
, resBody :: ByteString | ||
, resGucStatus :: Maybe HTTP.Status | ||
, resGucHeaders :: [GucHeader] | ||
} | ||
|
||
data InvokeQueryResult = InvokeQueryResult | ||
{ iqRequest :: InvokeRequestInfo | ||
, iqTableTotal :: Maybe Int64 | ||
, iqQueryTotal :: Int64 | ||
, iqBody :: BS8.ByteString | ||
, iqGucHeaders :: [GucHeader] | ||
, iqGucStatus :: Maybe HTTP.Status | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(Might be irrelevant when generalizing the interface as mentioned in the opening comment)
Same reasoning as with the request info types: Let's use the same prefix and DuplicateRecordFields
here.
MutateQueryResult
already has the correct prefix!
Actually, on second thought: Do we really need 3 types here? They are all the same - almost. resTableTotal
could easily be added to MutateQueryResult
- it's Maybe
anyway. And resRequest
... should just not be in here at all, I think.
It would be much nicer to pass request and result separately to the response functions.
data MutateRequestInfo = MutateRequestInfo | ||
{ mrConfig :: AppConfig | ||
, mrPgVersion :: PgVersion | ||
, mrDbStructure :: DbStructure | ||
, mrApiRequest :: ApiRequest | ||
, mrIdentifier :: QualifiedIdentifier | ||
, mrReadRequest :: Types.ReadRequest | ||
, mrMutateRequest :: Types.MutateRequest |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not really happy with the ambiguity of ReadRequest
, ReadRequestXXX
and MutateRequestXXX
on one side and Types.ReadRequest
and Types.MutateRequest
on the other.
I think we should rename the two Types.XXX
ones. Types.MutateRequest
is just an alias for MutateQuery
- let's use this. And Types.ReadRequest
could very well be type ReadTree = Tree ReadNode
.
, rrHeadersOnly :: Bool | ||
, rrIdentifier :: QualifiedIdentifier |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Those two are already part of ApiRequest
, right?
There are some in the other Info types, too. Removing those, and using everything from ApiRequest
, would streamline the interface a bit more.
a92280b
to
34cd707
Compare
I like the idea of a single entrypoint per module but I'm having trouble with getting GADTs to compile. Assuming the following reduced example, how would the {-# LANGUAGE DataKinds #-}
{-# LANGUAGE GADTs #-}
{-# LANGUAGE KindSignatures #-}
data RequestType
= Create
| Update
data Request (t :: RequestType) where
CreateRequest :: ApiRequest -> ReadRequest -> MutateRequest -> Request Create
UpdateRequest :: ApiRequest -> ReadRequest -> MutateRequest -> Request Update
-- single entrypoint for Request.hs
parse :: ApiRequest -> ReadRequest -> MutateRequest -> Request t
parse apiReq readReq mutateReq =
case iAction apiReq of
ActionMutate MutationUpdate -> UpdateRequest apiReq readReq mutateReq
_ -> CreateRequest apiReq readReq mutateReq
This produces
If the type is changed to data Request (t :: RequestType) where
CreateRequest :: ApiRequest -> ReadRequest -> MutateRequest -> Request t
UpdateRequest :: ApiRequest -> ReadRequest -> MutateRequest -> Request t It does compile but that would defeat the purpose of GADTs. |
It took me a few attempts to understand GADTs and DataKinds a little bit.. and I'm still not sure whether I fully grasp them. I will look at this in the next few days and answer a bit more detailed. |
I had exactly the same problem just a few days ago. Here's my current take on it. Polymorphic functions in haskell work much like they do in postgres: They need a polymorphic input type - from which the return type can be derived. This is not possible in the parse :: ApiRequest -> ReadRequest -> MutateRequest -> Request t No way for haskell to tell which type it should actually use as the return type at compile time. So we need a parametrized input type as well. The way I think about this is, that I need to be able to draw some kind path between both ends of everything that needs to be polymorphic. I need to tie the concrete types together, basically. I can tie one end to the The following won't exactly work, but you get the idea: {-# LANGUAGE DataKinds #-}
{-# LANGUAGE GADTs #-}
{-# LANGUAGE KindSignatures #-}
data RequestType
= Create
| Update
data Mutation (t :: RequestType) where
MutationCreate :: Mutation Create
MutationUpdate :: Mutation Update
data Action (t :: RequestType) where
ActionMutate :: Action (Mutation t)
data ApiRequest (t :: RequestType) where
ApiRequest :: { iAction :: Action t } -> ApiRequest t
data Request (t :: RequestType) where
CreateRequest :: ApiRequest -> ReadRequest -> MutateRequest -> Request Create
UpdateRequest :: ApiRequest -> ReadRequest -> MutateRequest -> Request Update
-- single entrypoint for Request.hs
parse :: ApiRequest t -> ReadRequest -> MutateRequest -> Request t
parse apiReq readReq mutateReq =
case iAction apiReq of
ActionMutate MutationUpdate -> UpdateRequest apiReq readReq mutateReq
_ -> CreateRequest apiReq readReq mutateReq Now, the other end is tied to the Now, you will still have a similar challenge for the For this to work, you need a container type for the ApiRequest, which itself is not parametrized. It kind of hides the parameter: data AnyApiRequest = forall any. AnyApiRequest (ApiRequest any)
userApiRequest :: whatever -> AnyApiRequest This function is not polymorphic anymore, so this works. You can then extract the actual ApiRequest from the AnyApiRequest in Let me know whether that makes any sense or where I should try to put it in different words. |
@wolfgangwalther Thanks for the explanation, It's clear now! I've linked your comment on #1804. I think that's better left for another PR since I'll close this one since |
Work in progress: Create a small API for all the request parsing we do - this will simplify
App
, enable more unit testing and allow us to simplify/refactorRequest
and it's submodules more easily.