Skip to content

Commit

Permalink
[SPARK-50320][CORE] Make --remote an official option by removing `e…
Browse files Browse the repository at this point in the history
…xperimental` warning

### What changes were proposed in this pull request?

`spark-submit` has one experimental option, which is `--remote`. This PR aims to make `--remote` option be an official option by removing `experimental` warning from Apache Spark 4.0.0.

https://github.com/apache/spark/blob/0aee601dcc9875928fc526c821985d2890bdc42f/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala#L568-L572

### Why are the changes needed?

`--remote` has been used well so far and intuitively. It's unlikely for us to change it. We had better remove a warning message.

### Does this PR introduce _any_ user-facing change?

No behavior change.

### How was this patch tested?

Manual review.

**BEFORE (4.0.0-preview2)**
```
$ bin/spark-submit 2>&1 | grep xperimental | wc -l
       1
```

**AFTER**
```
$ bin/spark-submit 2>&1 | grep xperimental | wc -l
       0
```

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes apache#48850 from dongjoon-hyun/SPARK-50320.

Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
  • Loading branch information
dongjoon-hyun committed Nov 15, 2024
1 parent 77e006f commit 11e4706
Showing 1 changed file with 1 addition and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -568,8 +568,7 @@ private[deploy] class SparkSubmitArguments(args: Seq[String], env: Map[String, S
| Spark Connect only:
| --remote CONNECT_URL URL to connect to the server for Spark Connect, e.g.,
| sc://host:port. --master and --deploy-mode cannot be set
| together with this option. This option is experimental, and
| might change between minor releases.
| together with this option.
|
| Cluster deploy mode only:
| --driver-cores NUM Number of cores used by the driver, only in cluster mode
Expand Down

0 comments on commit 11e4706

Please sign in to comment.