Current backend problems, and what we need going forward

[Irev-Dev]

There are a few issues with the current implementation of the CAD part of the backend. However, it needs to change anyway to cover upcoming requirements, so it makes sense to address them together and have a rethink of the backend architecture.

Current stack:

First of all this is a Redwood app, and setup in a pretty standard fashion at least for all of the persistent user data, projects etc (I don't think there's much point going over this part of the app in detail, in a single sentence it's: Using prisma as an ORM, connecting to a postgres db and has a graphQL interface to the client). Where it departs from standard redwood is for the model rendering because the packages can't run in an off-the-self node lambda like the rest of the app. Docker was an obvious choice since it allows us the set up the environment however we need to integrate current or future CAD packages (currently CadQuery and OpenSCAD), before I explain more, I'll give a run down of how our IDE works as relevant context.

The aim from the beginning was to make an agnostic or universal CAD IDE, to allow us to keep adding integrations without being couple to any one package in particular. The IDE is made up of 4 pieces of UI; Editor, Console, 3D-viewer and the customizer.

The way that the CAD packages interact with the IDE is like so:

By adding a light wrapping to each or the CAD packages to give a uniform interface (inputs of code, and optionally customizer values and outputs of a 3D artifact, logs and optionally customizer params) has worked well. It's not entirely that clean, there are some important differences between the outputs of the different packages. CadQuery outputs STL files (hoping to support STEP file and tesselate in the browser soon), JSCAD outputs three.js primitives directly, and while OpenSCAD can output STLs, by default it instead sends preview images from the same perspective as the three.js camera because STL rendering can be much slower ( good example is this moving fish, that's able to send through a preview image relatively quickly, but takes 10 minutes to create an STL when ran locally on my machine)

How are the models rendered?

Like I said we're using docker but more specifically we're using AWS's container lambdas to stick with a serverless architecture. The flow is like:

1. The client requests a render and sends code
2. The 3d aritfact is generated and put into S3
3. A url to the aritfact is returned to the client
4. The client fetches the artifact from S3

One extra nuance here is that before the aritfact is generated, we check if it already exists in S3, which can be done because the key its stored under is a hash of the code used to generate it.

What are the problems with this current setup?

• Performance; focusing on OpenSCAD, even for simple parts it takes much longer to generate a preview image than it should. It could be from cold start of the lambda container, or the fact that it sends the image to S3 before the browser fetches it making it a double wait situation or other things I'm not accounting for?
• 30s execution limit; while the lambda itself can run for up to fifteen minutes, the way its currently setup with the AWS APIGateway means that the artifact needs to be able to generate in under 30s (including cold start time). Since it's common for OpenSCAD models to take minutes to generate, this is a problem.
• Disconnected persistence; Currently generated artifacts are not so much stored as they are cached, in that they are stored using a hash of the code that was used to generate them so that we can check if they are already in s3, but they are not directly connected to the postgres db at all (we're not storing the hash anywhere) further the artifacts are automatically deleted 5 days after being created.

What's needed going forward?

• Performance, I'm mentioning it again because it definitely needs improvement. A quick feed back loop in important and current we're not providing that.
• Background jobs We need the ability to kick off longer processing tasks. Take the example of the moving fish again, that previews in seconds but take many minutes to produce an STL, we still need to produce an STL to download but obviously this can't be done on demand. Instead it need to be done ahead of time and stored and the work most like needs to be qued in some fashion. For example if the user makes a number of changes as they are working on theit project, we don't want to kick off a STL-gen task every time they save, instead the process should be that once they change the project so that it's different from the last STL produced, we than invalidate the old STL, and schedule generating a new one, I'm open to different approaches to this, but perhaps a CRON job that kicks of STL generation for all projects that have changed recently, but not in the last hour. For OpenSCAD another benefit of having stored STLs is that for users just browsing parts, we can send the then the STL straight away for a proper 3d-viewing experience in the browser and falling back to preview image once they makes changes (code or customizer)
• Customizer renders, still related to background jobs, but because we want to allow users to use the customizer and then download the resulting STL, we need to kick off a task to produce this and notify them when its ready.

[Irev-Dev]

It's worth noting that I'm more of a front end dev, so backend architecture is not something I'm super comfortable with. Any input would be appreciated.

[itsgc]

Hi! This is a super interesting problem and while i don't have any specific expertise to offer i'd love to know more and help if i can.

Few things i'd personally look into without knowing nothing about the system and the people running it (so apologies if this is very basic)

Performance

• Lambda spinups shouldn't factor in this order of magnitude so i'd be surprised but there should be some slice and dice of cloudwatch metrics that should answer this conclusively (For context i trigger lambda executions on individual http requests through cloudfront at $dayjob, some sites are very hot so maybe we don't incur in cold starts that much but others aren't and if we had cold starts north of 500ms-1000ms our alerting would lit like a tree)
• Is sharding the job an option? this is possibly a openscad question so i'll go and read-up on it but just throwing it out there
• The other common gotcha is retries and exponential backoffs
• Fargate is not serverless but comes to my mind as a pretty hand-off alternative that would offer more CPU chunk if necessary

Background Jobs / Customizer renders

This sounds like a brand new service/component. How does AWS spend factor in in extending cadhub? The way this would be done can change a lot depending on that.

Some trickery with CodePipelines come to mind as a random idea to investigate for the latter two problem.

[Irev-Dev]

Hello @itsgc, thanks for the advice!

I've got to run to work, so I'll answer this properly later.

I definitely didn't go over the specifics of how things are set up, so I'm happy to answer any questions you have.

I don't know much about about fargate, or really AWS too much in general, but I'm not wedded to serverless.

Some places to look in the code base if you want to do some digging are:

• Main readme covers how the setup to app https://github.com/Irev-Dev/cadhub/blob/main/README.md, by default this uses the live model rendering endpoint to make getting started easy
• but to setup a local model rendering the place to look is: https://github.com/Irev-Dev/cadhub/blob/main/app/api/src/docker/README.md however this depends on S3 so you'll need DEV_AWS_SECRET_ACCESS_KEY="" DEV_AWS_ACCESS_KEY_ID="" in app/api/src/docker/.env.example you could setup your own bucket, or I can send you credentials, maybe via twitter DM?

[itsgc]

No need for creds thanks! i'll have a go at setting up my own instance :) Thanks for the pointers, i'll try and get to know this platform a bit better and hopefully i'll have better questions!

[Irev-Dev]

I'm not 100% if those readmes have stayed up to date with changes to the project, so let me know if something doesn't add up or isn't working.

[Irev-Dev]

Replying properly to you original comment @itsgc

• lambda cold starts, do you know if container lambdas have similar cold start to normal lambdas?
• I have no idea about sharding, I thought that was mostly a db thing.
• Whats Im reading fargate is serverless and it does sound like a good fit, I don't really know what compute resources is best suited for OpenSCAD but I'm sure it can be optimized.
• How would you investigate retries and exponential backoffs
• It's hard to give you anything super accurate when it comes to spend factor, but atm the user count is low so the cost is very low atm, I'll need to figure out some kind of revenue when the user base picks up.

[hrgdavor]

My personal opinion (without actual exploration of prices) is that all those cloud fancy stuff is just so you pay 10x for resources in exchange for their management, although at the start it can be cheaper and a good option, I would suggest you look into running workers on standalone server (can be your PC for test).

just look at what a beast you can rent for EUR 40 monthly (65 AUD)
https://www.hetzner.com/dedicated-rootserver

If you have a setup that can take-over part of the work on standalone server (or for test your PC), you can get better idea how much a standalone server can do and depending on scale you can even use both cloud and standalone options.

If you prepare yourself a system where you can combine lambdas, and standalone server, then you are free to offset work to which ever option is cheaper, or easier to maintain at certain point in time.

[Irev-Dev]

@mrhut10 Had a suggestion of using websockets to keep a persistent connection to the browser for longer running tasks.