Delayed Job w/ s3, Missing required :bucket option error

[aaronwiggins]

I posted a question in SO here.
I'm able to get everything working in development perfectly, but the background processes don't work in production. I'm using Ubuntu 14.04 and deploying with Capistrano

Picture model
has_attached_file :uploaded_file, :styles => { original: "990x990#",large: "300x300#", medium: "200x200#", thumb: "100x100#"}, only_process: [:medium]

process_in_background :uploaded_file, queue: "project_image_resize", only_process: [:original, :large, :thumb]

Application.rb (also put in production.rb, made no difference)

`config.paperclip_defaults = {

  :storage => :s3,
  :url =>':s3_domain_url',
  :s3_protocol => 'https',
  :path => '/:class/:attachment/:id_partition/:style/:filename',
  :bucket => ENV['s3-bucket'],
  :s3_credentials => {
    :access_key_id => ENV['s3-id'],
    :secret_access_key => ENV['s3-key']
  }
}`

I've tried it with bucket inside s3_credentials hash, but it made no difference.
When the job fails and I look at the job last_error I get:
missing required :bucket option\n/home/username/project_folder/shared/bundle/ruby/2.2.0/gems/paperclip-4.3.0/lib/paperclip/storage/s3.rb:218:in...

The medium style uploads like it should, but the background styles never work because of the error. Also, photo uploads without delayed_paperclip work in development and production.

To me, it seems like delayed_paperclip can't get my s3 credentials, but the documentation never shows me where to add any. Please let me know if I can provide any other information or if I'm not understanding something.

[morgoth]

is the ENV['s3-bucket'] available on your worker instance?

[aaronwiggins]

How can I tell if they are? I haven't seen that done before.

[morgoth]

you specified this option by ENV variable (:bucket => ENV['s3-bucket']) and looks like it's missing, so I would first verify if it's indeed available.
It doesn't look like this gem issue

[aaronwiggins]

When I call for the variable in the console I get the correct output in development and production. When I post images to s3 w/ paperclip using those same credentials it's works correctly. When I create a background job to send emails it accesses my profile variables correctly.
But I'm not sure how to log the variable while the delayed_paperclip job is working, to check if it's there.

[aaronwiggins]

I also just typed in my bucket name instead of using the env variable, to test it, and I still get the same error.

[morgoth]

Not sure if I can help - I'm using delayed_paperclip with s3 and sidekiq aj adapter and it works for me.
I have:

    s3_credentials: {
      bucket:            Rails.application.secrets.s3_bucket,
      access_key_id:     Rails.application.secrets.s3_access_key_id,
      secret_access_key: Rails.application.secrets.s3_secret_access_key,
      s3_region:         "eu-west-1"
    }

[aaronwiggins]

Is that in your application.rb like mine?

[morgoth]

I have it in config/environments/production.rb but it shouldn't matter

[aaronwiggins]

Yeah, I can't figure out what's going on. I've used the secrets.yml file like you, and it still doesn't work. I've used Figaro and that doesn't work. I'm always able to get a test background job to log the value of the ENV variables, Rails.logger.info "BUCKET_NAME: #{Figaro.env.s3_bucket}" for example. But always the missing:bucket option in delayed_paperclip last_error

[morgoth]

You can also try to log Paperclip::Attachment.default_options - populated bucket option should be there

[aaronwiggins]

I just entereted that in rails c and s3_credentials are there