You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
privatedefgetExtraPlugins:Seq[SparkPlugin] = {
valresourceName="spark-rapids-extra-plugins"valclassLoader=RapidsPluginUtils.getClass.getClassLoader
valresourceUrls= classLoader.getResources(resourceName)
valresourceUrlArray= resourceUrls.asScala.toArray
if (resourceUrlArray.isEmpty) {
logDebug(s"Could not find file $resourceName in the classpath, not loading extra plugins")
Seq.empty
} else {
valplugins= scala.collection.mutable.ListBuffer[SparkPlugin]()
for (resourceUrl <- resourceUrlArray) {
valsource= scala.io.Source.fromURL(resourceUrl)
valpluginClasses= source.getLines().toList
source.close()
plugins ++= loadExtensions(classOf[SparkPlugin], pluginClasses)
}
plugins.toSeq
}
}
The resourceUrls may contain duplicated urls for the same extra plugin to be loaded. And the a lot of extra plugins will go weirdly when being loaded multiple times.
The text was updated successfully, but these errors were encountered:
Fixes#12076
The PR is about to fix the multiple loading problem which is due to the
duplicated URL paths for single extra plugin.
---------
Signed-off-by: sperlingxx <[email protected]>
Describe the bug
(https://github.com/NVIDIA/spark-rapids/blob/branch-25.02/sql-plugin/src/main/scala/com/nvidia/spark/rapids/Plugin.scala#L356)
The
resourceUrls
may contain duplicated urls for the same extra plugin to be loaded. And the a lot of extra plugins will go weirdly when being loaded multiple times.The text was updated successfully, but these errors were encountered: