Blog
guide
RSSUDFs in Krishiv: Python, pandas and Arrow, and RustA Krishiv Python UDF runs in your process: row by row with @udf, a whole pandas Series with @pandas_udf, a whole Arrow array with @arrow_udf. Measured on 200,000 rows, the pandas form is about 4 times and the Arrow form about 8 times faster than the row form, and the Arrow form is within about 15% of a built-in.
Configuring Krishiv: SET, conf and the settings that matterKrishiv has 119 settings, all named krishiv.*, that you read and change with SET, spark.conf and the session builder. A misspelled `krishiv.*` name or a bad value fails loudly. These are the ones you will touch first.