32 Pull requests opened by 24 people

• [SPARK-53309][SQL] Allow to propagate extra conf to executor

  #52071 opened Aug 19, 2025

• [SPARK-53329][CONNECT] Improve exception handling when adding artifacts

  #52073 opened Aug 19, 2025

• [WIP][SPARK-52090] retry fetch on ExecutorDeadException if block is found on another executor

  #52076 opened Aug 19, 2025

• [SPARK-53330][SQL][PYTHON] Fix Arrow UDF with DayTimeIntervalType (bounds != start/end)

  #52077 opened Aug 19, 2025

• [SPARK-53320] Make RegisterUserDefinedTableFunction in SparkConnectPlanner side effect free

  #52081 opened Aug 20, 2025

• [SPARK-53321] Make RegisterUserDefinedDataSource in SparkConnectPlanner side effect free

  #52082 opened Aug 20, 2025

• [SPARK-53339][CONNECT] Fix an issue which occurs when an operation in pending state is interrupted

  #52083 opened Aug 20, 2025

• [SPARK-53337][CORE] Ensure the application name get escaped.

  #52084 opened Aug 20, 2025

• [WIP] Preparser

  #52085 opened Aug 20, 2025

• [SPARK-53341][Core] Expand golden test coverage on multivariable DECLARE

  #52086 opened Aug 20, 2025

• [SPARK-53342][SQL] Fix Arrow converter to handle multiple record batches in single IPC stream

  #52090 opened Aug 21, 2025

• [SPARK-48547][DEPLOY] Add opt-in flag to have SparkSubmit automatically call System.exit after user code main method exits

  #52091 opened Aug 21, 2025

• [SPARK-52346][SDP] Fix counter initialization and increment logic for flow retries

  #52094 opened Aug 22, 2025

• [SPARK-53351][CORE] Use consistent string type blockId for block data access in external block handler

  #52095 opened Aug 22, 2025

• [SPARK-51348][BUILD][SQL] Upgrade Hive to 4.1

  #52099 opened Aug 22, 2025

• [SPARK-53356][PYTHON][DOCS] Small improvements to python data source docs

  #52101 opened Aug 22, 2025

• [SPARK-53355][PYTHON][SQL][INFRA] test python udf type behavior

  #52105 opened Aug 22, 2025

• Update supported_api_gen.py: remove an invalid escape sequence "\_" using a raw string

  #52107 opened Aug 24, 2025

• [SPARK-53347][PROTOBUF] Fix proto deserialization to allow false and null values

  #52108 opened Aug 24, 2025

• [SPARK-53361][SS] Optimizing JVM–Python Communication in TWS by Grouping Multiple Keys into One Arrow Batch

  #52109 opened Aug 24, 2025

• [DRAFT] JDBC Driver for Spark Connect Server

  #52110 opened Aug 25, 2025

• refactor: Remove python 2 code

  #52112 opened Aug 25, 2025

• [SPARK-53365] [SQL] Unify code for persisting of configs in views and UDFs

  #52113 opened Aug 25, 2025

• temp

  #52114 opened Aug 25, 2025

• [MINOR][SQL] Fix rCTE bug with malformed recursion

  #52115 opened Aug 25, 2025

• [SPARK-53367][PYTHON][SQL] add int to decimal coercion for Arrow UDFs

  #52117 opened Aug 25, 2025

• [SPARK-53369][PYTHON] Fix error message for UDFs with `CHAR/VARCHAR` return type

  #52118 opened Aug 25, 2025

• [SPARK-52346][SDP] Propagate partition columns from destination for BatchTableWrite

  #52119 opened Aug 25, 2025

• [WIP][SPARK-53372][SDP] End to End Testing Suite

  #52120 opened Aug 25, 2025

• [SPARK-53377][SDP] Tests for groupby and rollup

  #52121 opened Aug 25, 2025

• [SPARK-53378][PYTHON] Enable @udtf decorator to support returnType as first positional argument

  #52122 opened Aug 25, 2025

• [SPARK-53381][CONNECT] Avoid creating temporary collections in toCatalystStruct

  #52123 opened Aug 26, 2025

23 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• [SPARK-33737][K8S] Support getting pod state using Informers + Listers

  #51396 commented on Aug 26, 2025 • 18 new comments

• [SPARK-53294][SS] Enable StateDataSource with state checkpoint v2 (only batchId option)

  #52047 commented on Aug 23, 2025 • 16 new comments

• [SPARK-53293][SQL] Modify exprIdToOrdinal implementation for speedup on queries with wide tables

  #52046 commented on Aug 22, 2025 • 11 new comments

• [SPARK-53156][CORE] Track Driver Memory Metrics when the Application ends

  #51882 commented on Aug 25, 2025 • 3 new comments

• [SPARK-53318][SQL] Support the time type by make_timestamp_ltz()

  #52062 commented on Aug 26, 2025 • 2 new comments

• [SPARK-53289][SQL] Invoke cancelJobsWithTag before start passing the SparkListenerSQLExecutionEnd event to prevent execution of canceled tasks.

  #52039 commented on Aug 22, 2025 • 2 new comments

• [SPARK-53157][CORE] Decouple driver and executor heartbeat polling intervals

  #51885 commented on Aug 25, 2025 • 2 new comments

• [SPARK-53207][SDP] Send Pipeline Event to Client Asynchronously

  #51956 commented on Aug 25, 2025 • 1 new comment

• [SPARK-53113][SQL] Support the time type by try_make_timestamp()

  #51824 commented on Aug 25, 2025 • 1 new comment

• [SPARK-52777][SQL] Enable shuffle cleanup mode configuration in Spark SQL

  #51458 commented on Aug 21, 2025 • 1 new comment

• [SPARK-53109][SQL] Support TIME in the make_timestamp_ntz and try_make_timestamp_ntz functions in Scala

  #51828 commented on Aug 26, 2025 • 0 new comments

• [SPARK-53108][SQL] Implement the time_diff function in Scala

  #51826 commented on Aug 26, 2025 • 0 new comments

• [SPARK-53209][YARN] Add ActiveProcessorCount JVM option to YARN executor and AM

  #51948 commented on Aug 24, 2025 • 0 new comments

• [SPARK-52336][CORE] Prepend Spark identifier to GCS user agent

  #52027 commented on Aug 25, 2025 • 0 new comments

• [SPARK-52923][CORE] Allow ShuffleManager to control push merge during shuffle registration

  #51629 commented on Aug 25, 2025 • 0 new comments

• [SPARK-53283][CONNECT] Make CheckpointCommand/RemoveCachedRemoteRelationCommand in SparkConnectPlanner side effect free

  #52044 commented on Aug 19, 2025 • 0 new comments

• [DRAFT][DO-NOT-REVIEW][SPARK-51XXX][SQL] Enable implicit cast from STRING to TIME type

  #51583 commented on Aug 26, 2025 • 0 new comments

• [WIP][SPARK-51169] Set up a daily job for Python 3.14

  #51532 commented on Aug 25, 2025 • 0 new comments

• [SPARK-53296][CORE] ESS exit main thread in case boss thread exits

  #52050 commented on Aug 24, 2025 • 0 new comments

• [SPARK-53298][SQL] Make an isolation to control Shuffle partitionSizeInBytes converted from `REBALANCE` hint

  #52052 commented on Aug 24, 2025 • 0 new comments

• [SPARK-53319][SQL] Support the time type by try_make_timestamp_ltz()

  #52063 commented on Aug 26, 2025 • 0 new comments

• [WIP][SPARK-53292] Make CreateResourceProfileCommand in SparkConnectPlanner side effect free

  #52064 commented on Aug 20, 2025 • 0 new comments

• [SPARK-53335][K8S] Optionally capture diagnostics for jobs on Kubernetes

  #52068 commented on Aug 20, 2025 • 0 new comments