pyspark.sql.functions.schema_of_csv¶

pyspark.sql.functions.schema_of_csv(csv, options={})[source]¶

Parses a CSV string and infers its schema in DDL format.

New in version 3.0.0.

Parameters

csvColumn or str: a CSV string or a foldable string column containing a CSV string.
optionsdict, optional: options to control parsing. accepts the same options as the CSV datasource

Examples

>>> df = spark.range(1)
>>> df.select(schema_of_csv(lit('1|a'), {'sep':'|'}).alias("csv")).collect()
[Row(csv='STRUCT<`_c0`: INT, `_c1`: STRING>')]
>>> df.select(schema_of_csv('1|a', {'sep':'|'}).alias("csv")).collect()
[Row(csv='STRUCT<`_c0`: INT, `_c1`: STRING>')]

pyspark.sql.functions.rtrim pyspark.sql.functions.schema_of_json