IP address is incorrect in Apache access logs

[cyril-bouthors]

After an upgrade from 0.6 to 0.8.4, the IP address is no longer correctly defined in the Apache access logs but is still correct in ErrorLog file and the server-status page: the IP address of the proxy is shown in the access logs, instead.

I'm running Debian/jessie with Apache 2.4.10-10 and can reproduce the issue on two distinct machines, both running the same environment.

mod_rpaf configuration:

<IfModule rpaf_module>
    RPAF_Enable On
    RPAF_ProxyIPs 127.0.0.1 ::1
</IfModule>

Apache logs configuration:

ErrorLog ${APACHE_LOG_DIR}/error.log
LogFormat "%v:%p %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" vhost_combined

Logs extract (10.10.0.254 is the client IP address and 127.0.0.1 is the proxy):

==> /var/log/apache2/webfinance.jessie.cyb.isvtec.com-error.log <==
[Thu Jun 04 14:39:50.587886 2015] [authz_core:error] [pid 24932] [client 10.10.0.254:48088] AH01630: client denied by server configuration: /home/cyb/Dropbox/Code/webfinance/htdocs/favicon.ico

==> /var/log/apache2/webfinance.jessie.cyb.isvtec.com-access.log <==
webfinance.jessie.cyb.isvtec.com:80 127.0.0.1 - - [04/Jun/2015:14:39:50 +0200] "GET /favicon.ico HTTP/1.1" 403 498 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.81 Safari/537.36"

server-status page:

# w3m http://localhost/server-status | grep fav
0-0 24929 0/3/ _ 0.01 406 0   0.0  0.00  0.00 10.10.0.254 webfinance.jessie.cyb.isvtec.co GET /favicon.ico HTTP/1.1

[nfillot]

Hello Cyril,

Did you try to replace the LogFormat "%h" value with "%a" ?

[cyril-bouthors]

Hi @nfillot,

I replaced %h with %a in all the LogFormat default directives and the IP address is correctly shown in the error log, the access log and the server-status.

I think it's better if mod_rpaf works with the default LogFormat value as it used to be working in previous releases.

Is there a way to avoid this regression?

Thanks.

[tersmitten]

That would be nice indeed.

[gnif]

Can you please verify the headers being passed from the reverse proxy? Your configuration for mod_rpaf doesnt explicity set it via RPAF_Header, perhaps you are using a different header.

[ghost]

My config:

RPAF_Enable On
RPAF_ProxyIPs X.X.X.X (proxy server)
RPAF_Header X-Forwarded-For

pcap'ed traffic on backend, traffic from X.X.X.X(proxy server):

X-Forwarded-For: Y.Y.Y.Y (my computer)

This is apache2.4 from debian jessie: https://packages.debian.org/jessie/apache2
Latest rpaf version 0.8.5-rc1 from github
My client IP appears in $_SERVER['REMOTE_ADDR'] using php5-fpm+mod fastcgi, which is correct
However the log appearing in apache logs, using the default logformat, is the proxy server IP

The funny thing is that i tried the latest version from here because i had the exact opposite result with the rpaf 0.6 version in Jessie, incorrect IP (proxy server ip) in php, and correct IP in logs (probably related to the ticket you got a few weeks ago:)
#33

Rpaf from jessie is here:
https://packages.debian.org/jessie/libapache2-mod-rpaf

[mpdude]

Is that %a or %h in the log config? (Both from memory, hope it makes sense)

[ghost]

It's %h in apache2.conf, at least in debian Jessie.

And using %a shows the correct IP (client IP Y.Y.Y.Y).

[tersmitten]

We have the same problen (in 0.8.4). %a is working, but %h is the proxy ip.

[ghost]

I seriously think now it is perfect like this, before there was no way to keep both the Proxy IP and the Client IP. Though the change was surprising, maybe it could be more documented.

Thank you!

[glensc]

i think this is regression, can there be added perhaps separate option to control this?

as changing log format in all vhosts and installations can be problematic, can surely still miss some place.

[glensc]

for reference adding doc what %a and %h mean:

Format String	Description
%a	Client IP address of the request (see the mod_remoteip module).
%h	Remote hostname. Will log the IP address if HostnameLookups is set to Off, which is the default. If it logs the hostname for only a few hosts, you probably have access control directives mentioning them by name. See the Require host documentation.

http://httpd.apache.org/docs/current/mod/mod_log_config.html#formats

[glensc]

ps: i tested that with php $_SERVER['REMOTE_ADDR'] and require ip x.y.z.z work properly 👍

if there's no plan to fix the %h value from mod_rpaf side, this should be described in release notes and readme that must configure LogFormat

[glensc]

tested mod_remoteip and seems behave identical to mod_rpaf:

LoadModule remoteip_module modules/mod_remoteip.so
RemoteIPHeader X-Forwarded-For
RemoteIPInternalProxy 127.0.0.1

%h prints ip from connection and only %a contains the client real ip

[glensc]

@gnif ping. please see discussion and my notes. what's your verdict?

[gnif]

This is not a viable fix for this, there is an issue with mod_rpaf as it is intended to make the reverse proxy transparent, there should be no need to reconfigure any logging. I will investigate this when I find some time.

[mpdude]

Also keep in mind that the Apache API changed in 2.4 to separate the client and connection/peer address. If historically %h was the connection IP and %a is a 2.4 addition, I don't know whether we need to fix this. If we did, how could I get the proxies IP address?

Behaving similar to mod_remoteip in this regard is a good thing IMO. And still, mod_remoteip cannot transparently handle SSL termination (which can happen on Amazon EC2 ELBs, for example).