Missing required arguments: aws_access_key_id, aws_secret_access_key

[kotowick]

Hello,

I have implemented asset_sync into a Rails project. I am having an issue when trying to sync the assets. I have the config/asset_sync.yml file generated as well as the config/initializer/asset_sync.rb. The environment variables are being set... but still get an error:

[fog][WARNING] Unable to fetch credentials: end of file reached (EOFError)
rake aborted!
ArgumentError: Missing required arguments: aws_access_key_id, aws_secret_access_key
/Users/spencerkotowick/Projects/reports/vendor/bundle/ruby/2.3.0/gems/fog-core-1.43.0/lib/fog/core/service.rb:244:in `validate_options'
/Users/spencerkotowick/Projects/reports/vendor/bundle/ruby/2.3.0/gems/fog-core-1.43.0/lib/fog/core/service.rb:268:in `handle_settings'
/Users/spencerkotowick/Projects/reports/vendor/bundle/ruby/2.3.0/gems/fog-core-1.43.0/lib/fog/core/service.rb:98:in `new'
/Users/spencerkotowick/Projects/reports/vendor/bundle/ruby/2.3.0/gems/fog-core-1.43.0/lib/fog/core/services_mixin.rb:16:in `new'
/Users/spencerkotowick/Projects/reports/vendor/bundle/ruby/2.3.0/gems/fog-core-1.43.0/lib/fog/storage.rb:27:in `new'
/Users/spencerkotowick/Projects/reports/vendor/bundle/ruby/2.3.0/gems/asset_sync-2.0.0/lib/asset_sync/storage.rb:17:in `connection'
/Users/spencerkotowick/Projects/reports/vendor/bundle/ruby/2.3.0/gems/asset_sync-2.0.0/lib/asset_sync/storage.rb:22:in `bucket'
/Users/spencerkotowick/Projects/reports/vendor/bundle/ruby/2.3.0/gems/asset_sync-2.0.0/lib/asset_sync/storage.rb:87:in `get_remote_files'
/Users/spencerkotowick/Projects/reports/vendor/bundle/ruby/2.3.0/gems/asset_sync-2.0.0/lib/asset_sync/storage.rb:201:in `upload_files'
/Users/spencerkotowick/Projects/reports/vendor/bundle/ruby/2.3.0/gems/asset_sync-2.0.0/lib/asset_sync/storage.rb:223:in `sync'
/Users/spencerkotowick/Projects/reports/vendor/bundle/ruby/2.3.0/gems/asset_sync-2.0.0/lib/asset_sync/asset_sync.rb:29:in `block in sync'
/Users/spencerkotowick/Projects/reports/vendor/bundle/ruby/2.3.0/gems/asset_sync-2.0.0/lib/asset_sync/asset_sync.rb:51:in `with_config'
/Users/spencerkotowick/Projects/reports/vendor/bundle/ruby/2.3.0/gems/asset_sync-2.0.0/lib/asset_sync/asset_sync.rb:28:in `sync'
/Users/spencerkotowick/Projects/reports/vendor/bundle/ruby/2.3.0/gems/asset_sync-2.0.0/lib/tasks/asset_sync.rake:5:in `block (2 levels) in <top (required)>'
/Users/spencerkotowick/Projects/reports/vendor/bundle/ruby/2.3.0/gems/asset_sync-2.0.0/lib/tasks/asset_sync.rake:28:in `block in <top (required)>'
Tasks: TOP => assets:sync

[PikachuEXE]

You can try to debug with AssetSync::Config.new.inspect
According to spec
It should work fine

[Haseeb717]

@kotowick you fix this issue?
facing same issue @PikachuEXE

[PikachuEXE]

@Haseeb717
I can't help you unless there are more details
Let's start with:
How do you store your config?
What's the result of AssetSync::Config.new.inspect?

[markgandolfo]

I've received a similar problem .

>> AssetSync.sync
[fog][WARNING] Unable to fetch credentials: Connection reset by peer (Errno::ECONNRESET)
Traceback (most recent call last):
        1: from (irb):1
ArgumentError (Missing required arguments: aws_access_key_id, aws_secret_access_key)

Result of AssetSync::Config.new.inspect

>> AssetSync::Config.new.inspect
=> "#<AssetSync::Config:0x00007fb52a4f32b0 @fog_region=\"ap-southeast-2\",
 @fog_public=#<AssetSync::Config::FogPublicValue:0x00007fb52a4f3288 @value=true>, 
 @existing_remote_files=\"keep\", @gzip_compression=false, @manifest=false, @fail_silently=false, 
 @log_silently=true, @always_upload=[], @ignored_files=[], @custom_headers={}, @enabled=true, 
 @run_on_precompile=true, @cdn_distribution_id=nil, @invalidate=[], @cache_asset_regexps=[], 
 @include_manifest=false, @additional_local_file_paths_procs=[], @yml={\"fog_provider\"=>\"AWS\", 
 \"aws_access_key_id\"=>\"REDACTED\", \"aws_secret_access_key\"=>\"REDACTED\", \"aws_iam_roles\"=>true, 
 \"fog_region\"=>\"ap-southeast-2\", \"fog_directory\"=>\"customer-assets\", 
 \"existing_remote_files\"=>\"keep\", \"enabled\"=>true}, @fog_provider=\"AWS\", @fog_host=nil, 
 @fog_directory=\"customer-assets\", @fog_path_style=nil, @fog_scheme=nil, 
 @aws_access_key_id=\"REDACTED\", @aws_secret_access_key=\"REDACTED\", @aws_reduced_redundancy=nil, 
 @aws_iam_roles=true, @aws_signature_version=nil, @rackspace_username=nil, @rackspace_api_key=nil>"

If I hit fog-aws directly it works

aws_credentials = { :aws_access_key_id => 'REDACTED', :aws_secret_access_key => 'REDACTED' }

storage = Fog::Storage.new(aws_credentials.merge(:provider => 'AWS', region: 'ap-southeast-2'))
storage.get_bucket('customer-assets')

[PikachuEXE]

Can you try to run

begin; AssetSync.sync; rescue => ex; pp(ex.backtrace); end

To get the backtrace of the error?

[markgandolfo]

I'll try this tonight.
@PikachuEXE does the access key and secret need to be sent in the request?
It doesn't look like they are in the config.rb

        if aws_iam?
          options.merge!({
            :use_iam_profile => true
          })
        else
          options.merge!({
            :aws_access_key_id => aws_access_key_id,
            :aws_secret_access_key => aws_secret_access_key
          })

If I changed the above code to include the key/secret it started working for me. If you agree, I can submit a patch.

[PikachuEXE]

@markgandolfo
What's your value of aws_iam_roles though
If it's true, then either

• you are trying to use IAM role and access key/secret shouldn't be required
  (I might be wrong, I never use IAM role); or
• you set it by accident; or
• it's set to true by default (I don't think so?)

[xxx44yyy]

@PikachuEXE
IAM needs for asset_sync gem, but aws_access_key_id and aws_secret_access_key needs for fog-aws.

That is, asset_sync does not work with IAM if you use fog-aws.

[PikachuEXE]

I have almost zero knowledge about fog-aws or fog
If it fixes the issue I am happy to apply the change and release soon

But does that mean using IAM is impossible in asset_sync?

[PikachuEXE]

Looking at fog/fog-aws#499
It seems if something is wrong it still complains about
Missing required arguments: aws_access_key_id, aws_secret_access_key
But the actual causes could be IAM profile being invalid / fog-aws issue reading the credentials

You can start debugging with Fog::Storage.new(AssetSync.config.fog_options)
Then maybe modify the option until you got it working

fog_options = AssetSync.config.fog_options
# Modify `fog_options`...
Fog::Storage.new(fog_options)

Unless you can create Fog::Storage with use_iam_profile = true without key/secret
It should be fog/your setup issue