Spark on K8s: Failed to connect to spark-master:7077

发布于:2025-02-10 ⋅ 阅读:(97) ⋅ 点赞:(0)

异常信息如下:

spark 08:50:17.93
spark 08:50:17.94 Welcome to the Bitnami spark container
spark 08:50:17.94 Subscribe to project updates by watching https://github.com/bitnami/containers
spark 08:50:17.95 Submit issues and feature requests at https://github.com/bitnami/containers/issues
spark 08:50:17.95

24/12/27 08:50:21 INFO SparkContext: Running Spark version 3.3.1
24/12/27 08:50:21 INFO ResourceUtils: ==============================================================
24/12/27 08:50:21 INFO ResourceUtils: No custom resources configured for spark.driver.
24/12/27 08:50:21 INFO ResourceUtils: ==============================================================
24/12/27 08:50:21 INFO SparkContext: Submitted application: Spark Pi
24/12/27 08:50:22 INFO ResourceProfile: Default ResourceProfile created, executor resources: Map(cores -> name: cores, amount: 1, script: , vendor: , memory -> name: memory, amount: 1024, script: , vendor: , offHeap -> name: offHeap, amount: 0, script: , vendor: ), task resources: Map(cpus -> name: cpus, amount: 1.0)
24/12/27 08:50:22 INFO ResourceProfile: Limiting resource is cpus at 1 tasks per executor
24/12/27 08:50:22 INFO ResourceProfileManager: Added ResourceProfile id: 0
24/12/27 08:50:22 INFO SecurityManager: Changing view acls to: spark
24/12/27 08:50:22 INFO SecurityManager: Changing modify acls to: spark
24/12/27 08:50:22 INFO SecurityManager: Changing view acls groups to:
24/12/27 08:50:22 INFO SecurityManager: Changing modify acls groups to:
24/12/27 08:50:22 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(spark); groups with view permissions: Set(); users  with modify permissions: Set(spark); groups with modify permissions: Set()
24/12/27 08:50:22 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
24/12/27 08:50:23 INFO Utils: Successfully started service 'sparkDriver' on port 7078.
24/12/27 08:50:23 INFO SparkEnv: Registering MapOutputTracker
24/12/27 08:50:23 INFO SparkEnv: Registering BlockManagerMaster
24/12/27 08:50:23 INFO BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
24/12/27 08:50:23 INFO BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up
24/12/27 08:50:23 INFO SparkEnv: Registering BlockManagerMasterHeartbeat
24/12/27 08:50:23 INFO DiskBlockManager: Created local directory at /var/data/spark-3016eb8d-b215-46f9-b66a-fa05f3143941/blockmgr-5ad552e1-3ce2-4b39-b060-1a4cfb9df197
24/12/27 08:50:23 INFO MemoryStore: MemoryStore started with capacity 413.9 MiB
24/12/27 08:50:23 INFO SparkEnv: Registering OutputCommitCoordinator
24/12/27 08:50:23 INFO Utils: Successfully started service 'SparkUI' on port 4040.
24/12/27 08:50:23 INFO SparkContext: Added JAR local:///opt/bitnami/spark/examples/jars/spark-examples_2.12-3.3.1.jar at file:/opt/bitnami/spark/examples/jars/spark-examples_2.12-3.3.1.jar with timestamp 1735289421910
24/12/27 08:50:23 INFO SparkContext: The JAR local:///opt/bitnami/spark/examples/jars/spark-examples_2.12-3.3.1.jar at file:/opt/bi


网站公告

今日签到

点亮在社区的每一天
去签到