Skip to content

Commit

Permalink
[SPARK-47285][SQL] AdaptiveSparkPlanExec should always use the contex…
Browse files Browse the repository at this point in the history
…t.session

### What changes were proposed in this pull request?

Use `context.session` instead of `session` to avoid potential issue. For example, a cached plan may re-instance `AdaptiveSparkPlanExec` with a different active session.

### Why are the changes needed?

avoid potential issue

### Does this PR introduce _any_ user-facing change?

avoid potential issue

### How was this patch tested?

avoid potential issue

### Was this patch authored or co-authored using generative AI tooling?

no

Closes apache#45388 from ulysses-you/aqe.

Authored-by: ulysses-you <[email protected]>
Signed-off-by: Kent Yao <[email protected]>
  • Loading branch information
ulysses-you authored and yaooqinn committed Mar 6, 2024
1 parent e0d7910 commit 36299f1
Showing 1 changed file with 3 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ case class AdaptiveSparkPlanExec(

// The logical plan optimizer for re-optimizing the current logical plan.
@transient private val optimizer = new AQEOptimizer(conf,
session.sessionState.adaptiveRulesHolder.runtimeOptimizerRules)
context.session.sessionState.adaptiveRulesHolder.runtimeOptimizerRules)

// `EnsureRequirements` may remove user-specified repartition and assume the query plan won't
// change its output partitioning. This assumption is not true in AQE. Here we check the
Expand All @@ -103,7 +103,8 @@ case class AdaptiveSparkPlanExec(

@transient private val costEvaluator =
conf.getConf(SQLConf.ADAPTIVE_CUSTOM_COST_EVALUATOR_CLASS) match {
case Some(className) => CostEvaluator.instantiate(className, session.sparkContext.getConf)
case Some(className) =>
CostEvaluator.instantiate(className, context.session.sparkContext.getConf)
case _ => SimpleCostEvaluator(conf.getConf(SQLConf.ADAPTIVE_FORCE_OPTIMIZE_SKEWED_JOIN))
}

Expand Down

0 comments on commit 36299f1

Please sign in to comment.