Add ability to disable default log instrumentation through gradle

[ttsatsanis]

Summary

Nowadays, most applications do not depend on the default Android Log implementation. Instead they use SFL4J, Logback or other frameworks.
While the ability to upload logs is a great advantage, one cannot upload its specific logs in their desired format.

Desired Behavior

Have the ability to disable log instrumentation through new relic gradle config and be able to use my own NewRelic.logxxx functions.

Possible Solution

Maybe something like the following would be enough, allowing the gradle plugin to not run the log instrumentation (not an actual possible solution):

newRelic {
     ...
     setLogInstrumentationEnabled(boolean: true|false)
}

Additional context

Since NewRelic gives the ability to dynamically create columns and distribute data in each column, users must have the ability to enhance their search criteria based on their logging needs, with the support of their desired logging facility.

[ndesai-newrelic]

@ttsatsanis We'd like to understand your specific use case for disabling our logging instrumentation. Our solution currently instruments all major logging libraries, providing comprehensive coverage. Could you elaborate on the reasons behind your decision to turn off this feature? We're particularly interested in learning about any scenarios or requirements that our current logging instrumentation doesn't address.
This information will help us better understand your needs and potentially improve our offering. Are there particular aspects of our logging instrumentation that don't align with your requirements?

[sguptaruby]

Same question from my side how do I disable default log, I want track those loges which I have sent with the help of this function NewRelic.logAttributes.

[ttsatsanis]

@ndesai-newrelic sorry for the delayed response!!

I found out (which was obvious, but did not understood well) that I have been using a LogcatAppender for SLF4J and that's where NewRelic instrumented these functions and send logs to the platform. After not applying this, I got pure logs from NewRelic.logAttributes and not duplicate logs from instrumented Logcat and my logging implementation.
However, although my issue is solved, I think that the possible solution I proposed can allow a developer to maintain logcat logs (Log.x) and also apply its own logging implementation with the use of NewRelic.logxxxx without the need to applying or not applying the Logcat interface.

Let me explain this through code.

Here is a logging implementation with SLF4J and Logback Android:

private fun installLogger(
        enableLogReporting: Boolean,
        logLevel: Level = Level.INFO
    ) {
        val context = LoggerFactory.getILoggerFactory() as LoggerContext
        context.stop()

        val logDir = getExternalFilesDir(null)

        val rollingFileAppender = RollingFileAppender<ILoggingEvent>()
        rollingFileAppender.name = "file"
        rollingFileAppender.isAppend = true
        rollingFileAppender.context = context

        val rollingPolicy = SizeAndTimeBasedRollingPolicy<ILoggingEvent>()
        rollingPolicy.fileNamePattern = "$logDir/log-%d{yyyy-MM-dd}.%i.txt"
        rollingPolicy.maxHistory = 30
        rollingPolicy.setMaxFileSize(FileSize.valueOf("4700 kb"))
        rollingPolicy.setTotalSizeCap(FileSize.valueOf("400 mb"))
        rollingPolicy.setParent(rollingFileAppender)
        rollingPolicy.context = context
        rollingPolicy.start()

        rollingFileAppender.rollingPolicy = rollingPolicy

        val encoder = PatternLayoutEncoder()
        encoder.pattern = "%d{yyyy-MM-dd'T'HH:mm:ss.SSS} [%thread] %-5level %logger{10} - %msg%n"
        encoder.context = context
        encoder.start()

        rollingFileAppender.encoder = encoder
        rollingFileAppender.start()

        rootLogger.level = logLevel

        if (! enableLogReporting) {
            val logcatAppender = LogcatAppender()
            logcatAppender.name = "logcat"
            logcatAppender.context = context

            val tagEncoder = PatternLayoutEncoder()
            tagEncoder.pattern = "%logger{12}"
            tagEncoder.context = context

            logcatAppender.tagEncoder = tagEncoder
            tagEncoder.start()

            val msgEncoder = PatternLayoutEncoder()
            msgEncoder.pattern = "%d{yyyy-MM-dd'T'HH:mm:ss.SSS} [%thread] %-5level %logger{10} - %msg%n"
            msgEncoder.context = context

            logcatAppender.encoder = msgEncoder
            msgEncoder.start()

            logcatAppender.start()

            rootLogger.addAppender(logcatAppender)
        }
        
        if (enableLogReporting) {
            val newRelicAppender = NewRelicLogAppender(applicationContext)
            newRelicAppender.name = "newrelic"
            newRelicAppender.context = context
            newRelicAppender.start()

            rootLogger.addAppender(newRelicAppender)   
        }

        rootLogger.addAppender(rollingFileAppender)

        StatusPrinter.print(context)
    }

I have introduced a parameter which I configure in build time enableLogReporting. This, allow me to enable Logcat output for debug builds. If it is a release build, then I set this to false in order to disable the Logcat output, even though that it is already instrumented.
The problem is that I have to configure this parameter each time, in order not to accidentally log the instrumented logcat output in test-release builds.
By using my proposal, It would be easy for me to maintain Logcat output on a test-release through buildType options and define flavors where I can enable/disable the instrumentation, in order to avoid either logging duplicate logs (through Logcat instrumentation and Logback Appenders, accidentally create a release that will contain Logcat output (which in debug-test builds are quite useful) and also have the ability to introduce specific Log.x calls throughout my code.

Hope this explains what my proposal intents to do.

Also, @sguptaruby hope this also solves your issue.

[preetiandroid]

hi @ttsatsanis , I am using the NewRelic.logAttributes(Map<String, Object> attributes) method in my Android app, and it's working fine. However, I don't want to include other logs, such as API call logs or logs generated from exceptions, third party logs , lifecycle logs , background service logs and so on. I only want to capture logs from the NewRelic.logAttributes() method. I'm not using any external logging library; instead, I'm relying on the default Android logging mechanism. I also refer "https://docs.newrelic.com/docs/mobile-monitoring/mobile-monitoring-ui/mobile-logs/" but not working as expected. Can you please help me?

[ttsatsanis]

hi @ttsatsanis , I am using the NewRelic.logAttributes(Map<String, Object> attributes) method in my Android app, and it's working fine. However, I don't want to include other logs, such as API call logs or logs generated from exceptions, third party logs , lifecycle logs , background service logs and so on. I only want to capture logs from the NewRelic.logAttributes() method. I'm not using any external logging library; instead, I'm relying on the default Android logging mechanism. I also refer "https://docs.newrelic.com/docs/mobile-monitoring/mobile-monitoring-ui/mobile-logs/" but not working as expected. Can you please help me?

@preetiandroid since you are depending on Android logging Log.x, the NewRelic Gradle plugin will instrument this code and add also NewRelic.logx() after the Log.x functions (this happens on the build time and it is a beautiful feature!). So, I suspect, that this is the reason you see these logs.

So, either remove the Log.x from your code, or fully replace it with NewRelic.logx functions.

Your comment, also, is what depicted on the proposal! But the guys from NewRelic surely will give us more information :)

[preetiandroid]

Thank you for your response @ttsatsanis , However, we are concerned about exceeding our bucket size due to unwanted logs. we want that Log.x() in our code for some purpose. so we can't remove it.Do you have any other suggestions or solution?

[ttsatsanis]

@preetiandroid If your Logcat logs have a debugging purpose, then you could do something like I have done above:

@ndesai-newrelic sorry for the delayed response!!

I found out (which was obvious, but did not understood well) that I have been using a LogcatAppender for SLF4J and that's where NewRelic instrumented these functions and send logs to the platform. After not applying this, I got pure logs from NewRelic.logAttributes and not duplicate logs from instrumented Logcat and my logging implementation. However, although my issue is solved, I think that the possible solution I proposed can allow a developer to maintain logcat logs (Log.x) and also apply its own logging implementation with the use of NewRelic.logxxxx without the need to applying or not applying the Logcat interface.

Let me explain this through code.

Here is a logging implementation with SLF4J and Logback Android:

private fun installLogger(
        enableLogReporting: Boolean,
        logLevel: Level = Level.INFO
    ) {
        val context = LoggerFactory.getILoggerFactory() as LoggerContext
        context.stop()

        val logDir = getExternalFilesDir(null)

        val rollingFileAppender = RollingFileAppender<ILoggingEvent>()
        rollingFileAppender.name = "file"
        rollingFileAppender.isAppend = true
        rollingFileAppender.context = context

        val rollingPolicy = SizeAndTimeBasedRollingPolicy<ILoggingEvent>()
        rollingPolicy.fileNamePattern = "$logDir/log-%d{yyyy-MM-dd}.%i.txt"
        rollingPolicy.maxHistory = 30
        rollingPolicy.setMaxFileSize(FileSize.valueOf("4700 kb"))
        rollingPolicy.setTotalSizeCap(FileSize.valueOf("400 mb"))
        rollingPolicy.setParent(rollingFileAppender)
        rollingPolicy.context = context
        rollingPolicy.start()

        rollingFileAppender.rollingPolicy = rollingPolicy

        val encoder = PatternLayoutEncoder()
        encoder.pattern = "%d{yyyy-MM-dd'T'HH:mm:ss.SSS} [%thread] %-5level %logger{10} - %msg%n"
        encoder.context = context
        encoder.start()

        rollingFileAppender.encoder = encoder
        rollingFileAppender.start()

        rootLogger.level = logLevel

        if (! enableLogReporting) {
            val logcatAppender = LogcatAppender()
            logcatAppender.name = "logcat"
            logcatAppender.context = context

            val tagEncoder = PatternLayoutEncoder()
            tagEncoder.pattern = "%logger{12}"
            tagEncoder.context = context

            logcatAppender.tagEncoder = tagEncoder
            tagEncoder.start()

            val msgEncoder = PatternLayoutEncoder()
            msgEncoder.pattern = "%d{yyyy-MM-dd'T'HH:mm:ss.SSS} [%thread] %-5level %logger{10} - %msg%n"
            msgEncoder.context = context

            logcatAppender.encoder = msgEncoder
            msgEncoder.start()

            logcatAppender.start()

            rootLogger.addAppender(logcatAppender)
        }
        
        if (enableLogReporting) {
            val newRelicAppender = NewRelicLogAppender(applicationContext)
            newRelicAppender.name = "newrelic"
            newRelicAppender.context = context
            newRelicAppender.start()

            rootLogger.addAppender(newRelicAppender)   
        }

        rootLogger.addAppender(rollingFileAppender)

        StatusPrinter.print(context)
    }

I have introduced a parameter which I configure in build time enableLogReporting. This, allow me to enable Logcat output for debug builds. If it is a release build, then I set this to false in order to disable the Logcat output, even though that it is already instrumented. The problem is that I have to configure this parameter each time, in order not to accidentally log the instrumented logcat output in test-release builds. By using my proposal, It would be easy for me to maintain Logcat output on a test-release through buildType options and define flavors where I can enable/disable the instrumentation, in order to avoid either logging duplicate logs (through Logcat instrumentation and Logback Appenders, accidentally create a release that will contain Logcat output (which in debug-test builds are quite useful) and also have the ability to introduce specific Log.x calls throughout my code.

Hope this explains what my proposal intents to do.

Also, @sguptaruby hope this also solves your issue.

That, however, means that you should introduce a logging library to wrap all your logging functionality.

If it is only for debugging, you do not care about them in release and you also use ProGuard, you could consider to remove them through ProGuard obfuscation by using -assumenosideeffects.

Like in the example:

-assumenosideeffects class android.util.Log {
    public static *** d(...);
    public static *** w(...);
    public static *** v(...);
    public static *** i(...);
}

However, as I said before, it is better to get help or at least get some instructions from NewRelic people since they might find a better solution to your issue!

[ndesai-newrelic]

@sguptaruby we created Internal ticket for this enhancement.