Can I add a function to one sentence with python(pyspark)for statement?

Original manually manipulated function

df2 = df1.withColumn("test1", col("data.test1"))\
            .withColumn("test2", col("data.test2"))\
            .withColumn("test3", col("data.test3"))\
            .withColumn("test4", col("data.test4"))\
            .drop("data")

The state of hope?

for col_name in ['test1', 'test2', 'test3', 'test4']:
    df1.withColumn(col_name, col(data.col_name)).drop("data")

Like this... Can I add the .withColumn function by returning the list of column names I want to manipulate to a repeat statement?

python function

2022-09-20 21:44

1 Answers

I think I just need to interpolate well.

for col_name in ['test1', 'test2', 'test3', 'test4']:
    df1.withColumn(col_name, col("data." + col_name)).drop("data")

2022-09-20 21:44

If you have any answers or tips

Popular Tags

python x 4647

android x 1593

java x 1494

javascript x 1427

c x 927

c++ x 878

ruby-on-rails x 696

php x 692

python3 x 685

html x 656