Could not log "process.action_mailer" event. JSON::GeneratorError: source sequence is illegal/malformed utf-8

[srajan-dadhich]

Environment

Provide at least:

• Ruby Version.
  3.1.3
• Rails Version.
  7.0.4.3
• Semantic Logger Version.
  semantic_logger-4.15.0
• Rails Semantic Logger Version.
  rails_semantic_logger-4.14.0
• Other Application/framework names and versions (e.g. Puma, etc.).
• Rails configuration. Only need the settings related to Rails Semantic Logger and Semantic Logger.
  semantic_appender_options = { level: Rails.application.config.log_level, formatter: Rails.application.config.rails_semantic_logger.format, } case Rails.application.config.log_destination when :stdout SemanticLogger.add_appender(io: STDOUT, **semantic_appender_options) when :file path = Rails.root.join("log", "rails_logger_#{Rails.env}.json").to_s SemanticLogger.add_appender(file_name: path, **semantic_appender_options) else raise NotImplementedError("Invalid log_destination #{Rails.application.config.log_destination}") end Rails.logger.level = Rails.application.config.log_level
• Full Stack Trace, if an exception is being raised.

Could not log "process.action_mailer" event. JSON::GeneratorError: source sequence is illegal/malformed utf-8 ["/usr/local/bundle/ruby/3.1.0/gems/json-2.6.3/lib/json/common.rb:406:in generate'",
"/usr/local/bundle/ruby/3.1.0/gems/json-2.6.3/lib/json/common.rb:406:in pretty_generate'", "/usr/local/bundle/ruby/3.1.0/gems/rails_semantic_logger-4.14.0/lib/rails_semantic_logger/action_mailer/log_subscriber.rb:96:in formatted_args'",
"/usr/local/bundle/ruby/3.1.0/gems/rails_semantic_logger-4.14.0/lib/rails_semantic_logger/action_mailer/log_subscriber.rb:68:in block in payload'", "<internal:kernel>:90:in tap'", "/usr/local/bundle/ruby/3.1.0/gems/rails_semantic_logger-4.14.0/lib/rails_semantic_logger/action_mailer/log_subscriber.rb:55:in payload'", "/usr/local/bundle/ruby/3.1.0/gems/rails_semantic_logger-4.14.0/lib/rails_semantic_logger/action_mailer/log_subscriber.rb:125:in log_with_formatter'",
"/usr/local/bundle/ruby/3.1.0/gems/rails_semantic_logger-4.14.0/lib/rails_semantic_logger/action_mailer/log_subscriber.rb:37:in process'", "/usr/local/bundle/ruby/3.1.0/gems/activesupport-7.0.4.3/lib/active_support/subscriber.rb:149:in finish'",
"/usr/local/bundle/ruby/3.1.0/gems/activesupport-7.0.4.3/lib/active_support/log_subscriber.rb:115:in finish'", "/usr/local/bundle/ruby/3.1.0/gems/activesupport-7.0.4.3/lib/active_support/notifications/fanout.rb:211:in finish'",
"/usr/local/bundle/ruby/3.1.0/gems/activesupport-7.0.4.3/lib/active_support/notifications/fanout.rb:76:in block in finish'", "/usr/local/bundle/ruby/3.1.0/gems/activesupport-7.0.4.3/lib/active_support/notifications/fanout.rb:91:in block in iterate_guarding_exceptions'",
"/usr/local/bundle/ruby/3.1.0/gems/activesupport-7.0.4.3/lib/active_support/notifications/fanout.rb:90:in each'", "/usr/local/bundle/ruby/3.1.0/gems/activesupport-7.0.4.3/lib/active_support/notifications/fanout.rb:90:in iterate_guarding_exceptions'",
"/usr/local/bundle/ruby/3.1.0/gems/activesupport-7.0.4.3/lib/active_support/notifications/fanout.rb:76:in finish'", "/usr/local/bundle/ruby/3.1.0/gems/activesupport-7.0.4.3/lib/active_support/notifications/instrumenter.rb:49:in finish_with_state'",
"/usr/local/bundle/ruby/3.1.0/gems/activesupport-7.0.4.3/lib/active_support/notifications/instrumenter.rb:30:in instrument'", "/usr/local/bundle/ruby/3.1.0/gems/activesupport-7.0.4.3/lib/active_support/notifications.rb:206:in instrument'",
"/usr/local/bundle/ruby/3.1.0/gems/actionmailer-7.0.4.3/lib/action_mailer/base.rb:645:in process'", "/usr/local/bundle/ruby/3.1.0/gems/actionmailer-7.0.4.3/lib/action_mailer/message_delivery.rb:128:in block in processed_mailer'"....`

Note:

• Issues reported here should be related to monkey patches applied to Rails to make it use Semantic Logger.
• For other logging and appender related issues, please report the issue at [Semantic Logger](https://github.com/reidmorrison/semantic_logger/issues.

Expected Behavior

• Describe your expectation of how Semantic Logger should behave, perhaps by showing how the builtin Rails logger behaves.
• Provide a standalone Ruby script or a link to an example repository that helps reproduce the issue.

The format method should handle illegal/malformed UTF-8 input for String argument. For Rails application logs we have a patch to handle a similar behaviour,
`module SemanticLoggerExtensions
class CustomLogFormatter < SemanticLogger::Formatters::Json
def message
if log.message
encoded_message = encode_message(log.message)
log.message = encoded_message if encoded_message
hash[:message] = log.cleansed_message
end
end

  private

  def encode_message(message)
    return message unless message && (message.encoding != Encoding.default_internal)
    message.encode(Encoding.default_internal, invalid: :replace, undef: :replace)
  end
end

end`

Actual Behavior

• Describe or show the actual behavior.
• Provide text or screen capture showing the behavior.

Pull Request

• Consider submitting a Pull Request with a fix for the issue.
  • This is particularly helpful when running newer Rails versions, since we are not running it yet.
• Or, even a Pull request that only includes a test that reproduces the problem.

[reidmorrison]

Would be worthwhile to submit a pull request with this enhancement, along with a config option to disable the behavior? Enabling it by default sounds like a good option, allowing people to disable it if they need to.