OOM - with the multiple log files - expecting suggestion to the attached program

[pradnyagandhe]

Hi Log4j2 team,
I have written a sample Java program which creates multiple log files. I am attaching the program. Upon its execution generates a Out Of Memory exception. Please advise.
Multi_LogFile.txt
advise.
The code creates LoggerContext, Configuration, PatternLayout, FileAppender, creating logger, adding appender for every creation of object in a member method and removing appender, stopping appender and removing logger in its another method.

on Execution, get following exception

16022::Removing:16022
16023::Removing:16023
16024::Removing:16024
16025::Removing:16025
16026::Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
        at java.util.Arrays.copyOf(Unknown Source)
        at java.util.ArrayList.grow(Unknown Source)
        at java.util.ArrayList.ensureExplicitCapacity(Unknown Source)
        at java.util.ArrayList.ensureCapacityInternal(Unknown Source)
        at java.util.ArrayList.addAll(Unknown Source)
        at org.apache.logging.log4j.spi.LoggerRegistry.getLoggers(LoggerRegistry.java:134)
        at org.apache.logging.log4j.spi.LoggerRegistry.getLoggers(LoggerRegistry.java:129)
        at org.apache.logging.log4j.core.LoggerContext.updateLoggers(LoggerContext.java:739)
        at org.apache.logging.log4j.core.LoggerContext.updateLoggers(LoggerContext.java:729)
        at Multi_LogFile.initialize(Multi_LogFile.java:60)
        at Multi_LogFile.main(Multi_LogFile.java:106)

Thanks,
Pradnya Gandhe

[pradnyagandhe]

Hi Team,
Let me explain what I did in the attached program.
I have attempted to create appender and logger for every log file to generate programmatically.
In the class, I have initialize and unload method.
initialize() method does following->
```
// part 1
LoggerContext ctx = (LoggerContext) LogManager.getContext(false);
Configuration config = ctx.getConfiguration();

	PatternLayout layout = PatternLayout.newBuilder().withConfiguration(config).
                                                 withPattern("%d{HH:mm:ss.SSS} %level %msg%n").build();

	 String appenderName="programmaticFileAppender"+cnt;
    	
              appender = FileAppender.newBuilder()
	  .withConfiguration(ctx.getConfiguration())
	  .withName(appenderName).withLayout(layout)
	  .withFileName(log_fileName).build();
		
	appender.start();
	config.addAppender(appender);   

	// part 2
	AppenderRef ref = AppenderRef.createAppenderRef(appenderName, null, null);
	AppenderRef[] refs = new AppenderRef[] { ref };

	String loggerName="programmaticLogger"+cnt;
	LoggerConfig loggerConfig = LoggerConfig.createLogger(false, Level.DEBUG, loggerName, "name", refs, null, config, null);
	loggerConfig.addAppender(appender, null, null);
	config.addLogger(loggerName, loggerConfig);
	ctx.updateLoggers();


	// part 3
	Logger pLogger = LogManager.getLogger(loggerName);
	for (long i=1;i<=2000;i++){
         		pLogger.info("Programmatic Logger Message:info:"+cnt+"::"+i);		
	}

unload() method does following->

     ctx = (LoggerContext) LogManager.getContext(false);
    config = ctx.getConfiguration();

    // Remove the specific logger configuration
String loggerName="programmaticLogger"+i;
String appenderName="programmaticFileAppender"+i;
    LoggerConfig loggerConfig = config.getLoggerConfig(loggerName);
    loggerConfig.removeAppender(appenderName);

    // Stop the appender
    Appender appender = config.getAppender(appenderName);
    if (appender != null) {			
        appender.stop();
    }

    // Remove the logger
    config.removeLogger(loggerName);
    ctx.updateLoggers();

I am creating multiple objects of the class and calling initialize() and unload() method for every object.
Every object creates a single log file and logs data into it. I am expecting or assuming that unload methods uses removeAppender, stopping appender and removeLogger. So it should release the memory occupied by every object's log4j2 elements. But I don't see it that is happening.
For every object created and called intialize() method, memory gets allocated but after unload() method execution memory is not released.
memory usage goes on increasing and at certain point (though we increase java heap size) OOM happens.

Is there anything to modify in unload() method so that memory is released after logging of the data is complete?
Kindly advise.

Thanks
Thanks

[ppkarwasz]

Hi Log4j2 team, I have written a sample Java program which creates multiple log files. I am attaching the program. Upon its execution generates a Out Of Memory exception. Please advise.

Hi @pradnyagandhe,

Your code fails to remove unused appenders from the current configuration. Your unload method should look like this:

    public void unload(final long i) {
        ctx = (LoggerContext) LogManager.getContext(false);
        config = ctx.getConfiguration();
        // Remove the logger
        config.removeLogger("programmaticLogger" + i);
        // Remove the appender
        ((AbstractConfiguration) config).removeAppender("programmaticFileAppender" + i);

        ctx.updateLoggers();
    }

Remark: We don't recommend modifying the current Configuration programmatically. By using log4j-core directly in your code, your application can not be easily upgraded to newer versions of log4j-core.

If you are looking for a component that creates appenders dynamically, see Routing Appender. For your use case you also need to configure a purge policy.