Train XGBoost with Spark

Train XGBoost with Spark

2021-01-16Edit: 2021-01-16Zhanhang (Matthew) ZENG a few seconds read (About 55 words) 0 visits

Big Data / Machine Learning / Spark / Statistical Learning Big Data,Spark,Machine Learning

# XGB training script
# run spark-shell on cluster

spark-shell --name xxx --num-executors 15 --executor-cores 4 --executor-memory 20G --jars /tmp/xgboost4j-0.82.jar,/tmp/xgboost4j-spark-0.82.jar --driver-class-path /tmp/xgboost4j-0.82.jar,/tmp/xgboost4j-spark-0.82.jar

# import dependencies
import org.apache.spark.sql.types._
import scala.collection.mutable.ArrayBuilder

Title: Train XGBoost with Spark
Author: Zhanhang (Matthew) ZENG
Link: https://zengzhanhang.com/2021/01/16/LearnSpark-XGBoost/
Released Date: 2021-01-16
Last update: 2021-01-16
Statement: All articles in this blog, unless otherwise stated, are based on the CC BY-NC-SA 4.0 license.

#Big Data,Spark,Machine Learning

Comments

Your browser is out-of-date!

Update your browser to view this website correctly.&npsb;Update my browser now