General notes for getting up and running with distillery

[supernullset]

1. With phoenix, mix release by itself does not make sense to use, and in fact will not produce a runnable executable.
   • I believe this is due to the presence of phoenix-live-reload in development
2. To produce a release which functions, server: true must be set in the config/prod.exs file
3. MIX_ENV=prod mix release --env=prod builds a working release after the above steps are completed.
   • By default it looks like releases which are serving their own static assets default to serving uncompressed css/js (I think this is a phoenix side thing)
4. The hot upgrade path described in the current distillery documentation works with static assets; I have not tested against sockets yet

[supernullset]

@bitwalker I added you here to reduce noise round the documentation which we discussed here: bitwalker/distillery#2 . I will of course be following a normal PR workflow, but I needed a place to drop notes + have a simple example project that I could both refer to and use as extra documentation.

My ETA for a first PR draft is sometime tomorrow (Aug 16)

[supernullset]

Im having issues getting javascript assets to update in releases. I have setup a socket which I am planning to use for testing that sockets are held open during the upgrade.

Current versions:

0.0.1 -> default app from phoenix.new
0.0.2 -> Same application minus logo html and css for .logo class
0.0.3 -> Initialization of socket which emits an even number every 1000ms

Upcoming versions

0.0.4 -> Socket is modified to emit odd numbers every 1000ms

Current status

Upping from 0.0.1 -> 0.0.2 works great

0.0.2 -> 0.0.3 does not seem to be delivering the new js (possible cache issue?)

[bitwalker]

I suspect that's a caching issue, are you running mix phoenix.digest prior to the release?

[supernullset]

I am indeed, which is why it is curious. I will be poking this more tonight for sure

[bitwalker]

If the digest hash for the assets isn't changing after a refresh then there's definitely an issue, but if it is but things aren't updating appropriately, I'm not sure what would cause that.

[supernullset]

@bitwalker sorry for the delay on this; I have been trying to figure out what is happening to no avail. The js assets hash is definitely not changing for the 0.0.2 -> 0.0.3 release; which is odd, because between 0.0.1 -> 0.0.2 the css asset hash changed without issue.

This is the command I am running to build an update (including the asset build step) is:

./node_modules/brunch/bin/brunch b -p && MIX_ENV=prod mix phoenix.digest && MIX_ENV=prod mix release --upgrade --env=prod

if you look at 7cc39d1 , you can see that the socket.js file definitely changes.

As far as I can tell, the asset hash for my css is changing properly, the asset hash for the js is not.

[bitwalker]

I'll see if I can find some time to look at this today, with both our eyes on it, we can get it sorted :)

[supernullset]

Interesting development.

Looking at Plug.Static's Caching Options I set cache_control_for_vsn_requests to have a 0 for max_age and the js SHA updated immediately.

While I think this makes some sort of sense, ill be darned if that isn't a gotcha!

[supernullset]

And I can now confirm that hot uploading the socket code works gloriously. Coolest thing I have seen all week!

Now that I have a decent working example, I can get a draft of a phoenix oriented walkthrough together.

Im not entirely sure what happened with Plug.Static and specifically with the js updates; its oddly specific to me which is a little bit of a smell. I am going to gloss over this in the walkthrough for the moment until I can dig around in Plug a bit.

Thanks for offering to take a look @bitwalker!

[bitwalker]

@supernullset Interesting, it makes sense why it would serve the old version if the ?vsn= string didn't change, but I would assume that if the version did change, then it shouldn't be caching anything, and serving the latest assets.

In any case, I suppose it's at least something we can make a note of in the short term until we figure out exactly what's causing that behaviour - to the best of my knowledge though that appears to be a bug.

[vladshcherbin]

I've noticed in the phoenix docs:

Within config/prod.exs we need to make the following changes. We must configure our Endpoint to act as a server server: true. We must set the root to be . and we must set the version: Mix.Project.config[:version] to ensure that new versions cause the Endpoint assets cache to be reloaded.

There is also some info below, why this is needed for releases and hot reloading. Maybe this can somehow help.

[bitwalker]

Yeah all of those things still hold true, though if you'd missed those steps I don't think the upgrades would've worked at all, where in your case the assets partially updated.

[supernullset]

@vladshcherbin good catch and thanks for linking me. I did not set root or version, which seems like a likely culprit. I am surprised that the upgrades worked without either of those being set. Ill rollback and run through the test again with these in place

[supernullset]

@bitwalker @vladshcherbin Sorry for the delay on this; I have been out of town for a few days and could not sit down to wrap this up.

I have a doc draft of a walkthrough in progress but I have a few questions about the issue which @vladshcherbin raised. Setting root and version in the endpoint configuration 100% work, but I have ZERO idea why. Having dug through the documentation for Plug.Static, Phoenix.Endpoint and Phoenix.Endpoint.Adapter, I have found no reference to either of these configuration variables; nor have I found any reference to these within the source itself. So, including these options in the documentation is no problem, but I would like to figure out why they have to be included! Do either of you have any ideas? If I find out, of course I will relay that info.

[bitwalker]

When a new version is installed, config_change is called here. If the configuration has not changed, then the root configuration variable will not be re-evaluated (as you can see here).

root determines the root path of the application, and in a release should be set to ., which will expand to the application directory, under which priv is found, and priv is where your assets are. Phoenix expands this path when it loads it's config. When you do an upgrade, Phoenix must re-evaluate this configuration so that it expands to the new application root directory, and in order to trigger this re-evaluation, you have to tell Phoenix to reload it's config, hence the version configuration, which isn't used by anything, but will force Phoenix to refresh it's configuration.

Basically this is due to the fact that Phoenix manages it's own configuration cache, and because it must be told where to look for assets. I believe this latter one could probably be fixed, but I haven't had time to try and look into it myself. In any case, hopefully that clears up any confusion :)

[vladshcherbin]

@supernullset I'm actually very interested in this tutorial as the environment where you build your release has to be the same as on production server (if you build on os x and run on ubuntu - it won't work).

The official documentation also tells you that but omits this whole step leaving it for user to figure out. I'm curious, if your tutorial will have this step :)

[supernullset]

@bitwalker Thanks for that walkthrough; extremely useful explanation. I just found it odd that the knobs required for configuring a release don't seem to show up in any of the module docs! Understanding the details really helps.

@vladshcherbin I keep a Vagrant machine in each of my projects which is a mirror of the OS which I have running in production. Generally I have been doing active development in my local environment (OSX), then when I am ready for a release I fire up my Vagrant VM, build a release, and scp to my production remote.

The current Distillery Walkthrough does mention that the erts files need to be the same for build -> production here under the NOTE sub header. I am happy to include a brief synopsis of how I am building in a VM, but I believe that it is out of scope for Distillery documentation. How about I add a mini-walkthrough on using Vagrant (with ubuntu) into the README for this project? That way I can link out from the Phoenix Distillery Walkthrough to an example project where more detail on ensuing that the build server matches production.

[vladshcherbin]

@supernullset yep, a small guide somewhere would be awesome because if you want to do releases, you are basically forced to learn how docker or vagrant works.

[supernullset]

@vladshcherbin Ill put something together in the next 2 days!