No compression happening

[jplehmann]

I'm using django-pipeline with s3. I'm successfully using collectstatic to combined my Javascript files and store them in my s3 bucket, but they are not getting compressed for some reason (verified by looking at the file, its size, and its content-encoding). Otherwise things are working correctly with the combined scripts.js that is produced.

Here are the changes I made to use django-pipeline:

1. Added pipeline to installed apps.
2. Added 'pipeline.finders.PipelineFinder' to STATICFILES_FINDERS.
3. Set STATICFILES_STORAGE = 'mysite.custom_storages.S3PipelineManifestStorage' where this class is as defined in the documentation, as seen below.
4. Set PIPELINE_JS as seen below, which works but just isn't compressed.
5. PIPELINE_ENABLED = True since DEBUG = True and I'm running locally.
6. PIPELINE_JS_COMPRESSOR = 'pipeline.compressors.yuglify.YuglifyCompressor' even though this should be default.
7. Installed the Yuglify Compressor with npm -g install yuglify.
8. PIPELINE_YUGLIFY_BINARY = '/usr/local/bin/yuglify' even though the default with env should work.
9. Using the {% load pipeline %} and {% javascript 'scripts' %} which work.

More detail:

PIPELINE_JS = {
    'scripts': {
        'source_filenames': (
            'lib/jquery-1.11.1.min.js',
            ...            
        ),
        'output_filename': 'lib/scripts.js',
    }
}

class S3PipelineManifestStorage(PipelineMixin, ManifestFilesMixin, S3BotoStorage):
    location = settings.STATICFILES_LOCATION

As mentioned, collectstatic does produce scripts.js just not compressed. The output of that command includes:

Post-processed 'lib/scripts.js' as 'lib/scripts.js'

I'm using Django 1.8, django-pipeline 1.5.2, and django-storages 1.1.8.

Probably this is something dumb I'm doing, but possibly a bug or missing documentation. IF I have compression enabled, I'd expect an error at least.

[cyberdelia]

You have to explicitly use the GZIPMixin: http://django-pipeline.readthedocs.org/en/latest/storages.html#gzip-compression

[jplehmann]

Thank you Timothee, this is very helpful, I did miss this.

However even after making this change, and running collectstatic I still
don't see any compression happening. Any other ideas?

class S3PipelineManifestStorage(PipelineMixin, ManifestFilesMixin,
S3BotoStorage, GZIPMixin):
    location = settings.STATICFILES_LOCATION

On Tue, Jul 28, 2015 at 11:08 AM, Timothée Peignier <
[email protected]> wrote:

You have to explicitly use the GZIPMixin:
http://django-pipeline.readthedocs.org/en/latest/storages.html#gzip-compression

—
Reply to this email directly or view it on GitHub
#483 (comment)
.

[cyberdelia]

I'm afraid the mixins order is important here, try to put GZIPMixin first.

[jplehmann]

Thanks Timothee for the support. After reordering the mixins, it's
producing .gz's (scripts.js.gz). However, the template is still producing
references to "scripts.js" rather than the .gz version. What causes that?
(If I rm the scripts.js I get a broken link).

On Tue, Jul 28, 2015 at 11:29 AM, Timothée Peignier <
[email protected]> wrote:

I'm afraid the mixins order is important here, try to put GZIPMixin first.

—
Reply to this email directly or view it on GitHub
#483 (comment)
.

[jplehmann]

Is this the issue?

To make it work with S3, you would need to change the staticfiles storage url method to return .gz urls (and staticfiles/pipeline template tags depending if you care for clients that don't support gzip).
Also don't forget to setup the proper header on s3 to serve theses assets as being gzipped.
Pipeline GZIP support is fairly recent and limited, you seems to be the first users, so thanks for bearing with me here.
#312 (comment)

[cyberdelia]

Yes, that's the issue here.

[jplehmann]

Closing this issue, of which is basically a duplicate of:

• GZIP compression! template failed to detect .gz file! #312
• How to do a S3 + Cached + GZipped storage class? #339

See also my StackOverflow question which I answered as best I could with fragments from these conversations.

@cyberdelia your prompt responses were very appreciated. Also you may save yourself some time if you add a bit more to the README in the relevant sections. 1) It sounds like "compression" is on by default, so it wasn't clear to me that I needed the special gzip support enabled. Even though I tried to read it all I somehow missed it. A comment near the front might help. 2) That GZipMixin has to go first, and on the issues of rewriting URLs. 3) Perhaps a tips or troubleshooting section. I hope my SO issue will defray some future questions, although I feel that my approach is a bit of a hack and perhaps still not complete.

Also I'm not clear yet on the relationship of using AWS_IS_GZIPPED to the solution I've implemented.

[cyberdelia]

@jplehmann Feel free to propose a PR to improve the documentation.

I should probably go ahead and merge my gzip-url branch, since it sounds like most people expect that behavior.

[jplehmann]

and staticfiles/pipeline template tags depending ...

Can you explain how that's an "and" and not an "or"? And, can you provide any examples or links on how to do this? (change pipeline template tags to conditionally provide gz links based on headers?) It seems like it's pretty important to support that scenario.

[jplehmann]

I think part of the confusion for the uninitiated, is that "compression" on by default really means minification, not gzipping.

On the topic of compression, I got burned this morning by Yuglify's inability to handle a bang in the javadoc comment. Basically it fails to compress and produces malformed Javascript. Seems like a huge bug and maybe Yuglify isn't the best default/recommended compressor. But for me only happens when using terminal input the way Pipeline uses it.
yui/yuglify#19 (comment)