Polars

@pola.rs

1037 Followers 3 Following

Dataframes powered by a multithreaded, vectorized query engine, written in Rust.

Figure showing the speed-up of Polars compared to PySpark per query of the PDS-H benchmark.
Code example of the Decimal type
Free DataCamp course announcement - Learn Polars, the high-performance data processing library that engineers love. Interactive Introduction to Polars course available at no cost. Partnership between DataCamp and Polars.
Course title slide with dark blue background showing 'FREE COURSE' in green text, 'Introduction to Polars' as the main white heading, and logos for DataCamp and Polars at the bottom.
Diagram showing examples of applying horizontal operations to a dataframe with random numerical values. The example shows the expressions max_horizontal, sum_horizontal, mean_horizontal, and cum_sum_horizontal. The first three produce a numerical column and the expression cum_sum_horizontal produces a struct column with as many fields as there are input columns/series.
Diagram showing how the functions date_range, datetime_range, and time_range, can be used to generate series of consecutive temporal values. The interval between consecutive values can be changed with the parameter interval and the two endpoints may or may not be included in the generated range, depending on the value the parameter closed is set to.
Diagram showing how the use of the expression over impacts the result of an expression. Using rank on a dataframe will rank all swimmers by their time, but that means that the fastest swimmer of the second race type is actually ranked 3rd. By using over, the fastest swimmer of each race type gets the rank 1.
Diagram showing how filter works inside aggregations. We use an aggregation to compute the current record for different swimming race types and we use a filter to ignore unverified times, while also using information about unverified times to compute another column.
Diagram showing how the expression clip works. The diagram shows `clip` used with integers and with `datetime` objects. The diagram also shows the usage of literals, other columns, and arbitrary expressions, as the bounds.
Webinar banner with the NVIDIA and Polars logos and the webinar title "Inside the RAPIDS-Accelerated Polars GPU Engine" that is taking place on the 28th of January of 2025. On the side, you can see a digital/technological rendition of what could be a black hole or another space-themed object.
Diagram showing how the window boundaries of 10-year long windows align neatly with the decades and the decade halfpoints, 1980, 1985, 1990, etc, although the first datapoint is from 1981.
Diagram with a snippet of Polars code that creates a dataframe mapping each month to the number of days in that month. The stars of the snippet are the functions `date_range` and `group_by_dynamic`. The code: from datetime import date print( pl.date_range( start=date(2024, 1, 1), end=date(2025, 1, 1), interval="1d", closed="left", # Don't include `end` eager=True, ).to_frame("d") .group_by_dynamic("d", every="1mo") .agg(pl.len().alias("days_in_month")) .select( pl.col("d").dt.month().alias("month"), pl.col("days_in_month"), ) )
Diagram exemplifying how `join_where` performs inequality joins. We're joining two dataframes that have 3 rows each and each row has a corresponding row on the other dataframe because they have matching IDs. The dataframe on the right has a column `dt` with date values, which fall within the range created by the columns `start` and `end` of the other dataframe, except for the third row, in which `dt` falls outside the range. When using `join_where` with the predicate `pl.col("dt").is_between("start", "end")`, the third row of each dataframe will not be matched and won't show up in the join result, which only joined the first 2 rows of each dataframe.
Diagram establishing a connection between Polars data types and similar Python types. The connections are as follows: Boolean is most similar to bool Int8, Int16, Int32, and Int64 is most similar to int with restrictions UInt8, UInt16, UInt32, and UInt64 is most similar to int with restrictions Float32 and Float64 is most similar to float Decimal is most similar to decimal.Decimal String is most similar to str Binary is most similar to bytes Date is most similar to datetime.date Time is most similar to datetime.time Datetime is most similar to datetime.datetime Duration is most similar to datetime.timedelta Array is most similar to numpy.array List is most similar to list Categorical is most similar to enum.StrEnum Enum is most similar to enum.StrEnum Struct is most similar to typing.TypedDict Object is most similar to object Null is most similar to None
A diagram shows how to use str.split, list.first / list.last, cast, pl.int_ranges, and explode, all together, to turn a dataframe where a column may contain ranges like "3-5" into a similar dataframe where all ranges have been expanded, or exploded, across multiple rows. The full code is: range_start = pl.col("nrs").str.split("-").list.first().cast(pl.Int64) range_end = pl.col("nrs").str.split("-").list.last().cast(pl.Int64) df.with_columns(pl.int_ranges(range_start, range_end + 1)).explode("nrs")
Diagram showing how `value_counts` produces a column with struct values, mapping column values to their counts. We then show how to use `.struct.field` to extract a single field from the struct and how to use `.struct.unnest` to extract all fields into corresponding columns.