Alternative to Prefect for building Workflows
Both Windmill and Prefect support writing complex workflows and ETL with code and run them at scale. We highlight below the main differences between the 2 on the following axis:
- Open Source and Hosting
- Workflows and Code
- Triggers
- Execution Runtime and Coldstarts
- Observability and Monitoring
- Windmill AI
- Pricing
- Scripts and Apps
Prefect is a data workflow management system, designed to help teams build, run, and monitor data workflows. It provides tools for orchestrating data pipelines, managing their execution, and ensuring data integrity.
Open Source and Hosting
Windmill is fully open source. This implies that Windmill can be self-hosted with just a few commands, providing cost-effective and secure solutions.
Windmill offers a clean slate, allowing you to fully explore its platform before committing to the product. Windmill's open-source nature ensures that the platform is tested, approved, and continuously enhanced to meet the community's needs.
Windmill provides and integrates into its platform a public Community Hub where users share useful and proven scripts, flows, and applications.
Prefect has an Apache v2.0 License and can be self-hosted.
Prefect's monitoring is done through a cloud platform that requires subscription. Many of the features necessary for its proper functioning are only available under Prefect Cloud:
Workflows and Code
Prefect and Windmill Flow editor have a very similar sets of features. Below are the advanced features presented by Prefect and, in backlink, their counterpart on Windmill:
- scheduling
- retries
- logging
- convenient async functionality
- caching
- notifications
- observability
- event-based orchestration
We could also mention storage of configuration or error handling that are dealt with in a similar manner.
Prefect is heavily code-based. Everything is and can only be defined in Python. The steps and their configurations are defined by code:
Flow "Given a GitHub repository, logs the number of stargazers and contributors for that repo" on Prefect.
Windmill offers both low-code and code-based solutions.
Flow editor
In Windmill's Flow editor, the steps are also code-based but built from scripts (TypeScript, Python, Go, Bash, SQL ...). Users can write the code directly, choose a script from the workspace, the community library WindmillHub, or have it generated with AI. Each script can be saved with permissions set from the workspace.
The structuring of the flow and the configurations for each step are defined from a User Interface. With a code when you need mindset, the user can navigate the flow editor in low-code.
Flow "Given a GitHub repository, logs the number of stargazers and contributors for that repo" on Windmill.
In particular, Windmill generates automatically UIs for flows and steps and lets users test flows, steps or flow until a given step, from the UI.
Flows can be defined visually / YAML on a local environment. In particular, Windmill has a VS Code extension to edit from your code editor scripts flows YAML.
Workflows as code
Flows are not the only way to write distributed programs that execute distinct jobs. Another approach is to write a program that defines the jobs and their dependencies, and then execute that program. This is known as workflows as code.
Windmill supports defining workflows as code in a single script in both Python and TypeScript using intuitive and lightweight syntax.
Here is an example of a workflow as code with Prefect (from their documentation):
from prefect import flow, task
@task
def my_first_task(msg):
print(f"Hello, {msg}")
@task
def my_second_task(msg):
my_first_task.fn(msg)
@flow
def my_flow():
my_second_task("Trillian")
and the same example with Windmill (in Python):
from wmill import task
@task
def my_first_task(msg):
print(f"Hello, {msg}")
@task
def my_second_task(msg):
my_first_task(msg)
def main():
my_second_task("Trillian")
Triggers
Prefect uses a deployment system from the terminal to trigger and schedule its flows.
To manage its triggers from the Prefect UI, you have to go through a separate menu called Automations (available on the Prefect cloud only) and go through many actions before actually triggering the flow.
While Windmill also allows configuration and triggers from the terminal, the platform is designed to schedule, manually trigger, generate webhooks or UIs from the Windmill UI (cloud or self-hosted).
Example of a flow scheduled manually via the UI on Windmill.
Execution Runtime and Coldstarts
Windmill's architecture allows runnning each task on the current fleet of worker (which you can auto-scale automatically) and hence do not suffer from cold start. Isolation and creating dedicated dependency environment is the secret sauce that makes Windmill the fastest execution runtime for scripts (10ms cold start).
Windmill can run 26M tasks a month on a single worker costing 5$. Prefect does not provide figures on its average runtime and performance.
Also, Windmill has a transparent API, on which you could imagine launching 1m simulateously by API.
We have conducted benchmarks to measure our performance against our competitors.
For comparison, you will find below some tests between Prefect and Windmill for cold start, execution, and result of a flow shared by Prefect, and of a Python script.
Each instance is self-hosted.
Benchmark #1
Flow "Given a GitHub repository, logs the number of stargazers and contributors for that repo", shared by Prefect on their quickstart (at date 09.20.23):
- Prefect: Cold start + Execution + Results = 8.09s
- Windmill: Cold start + Execution + Results = 1.13s
Benchmark #2
Slowfunction that makes imports and loops from 11^7 down to 0. For each number i in that range:
-
Calculates the arctangent (math.atan(i)) of the number.
-
Calculates the tangent (math.tan(i)) of the number.
-
Multiplies the two results together and adds them to the result. And records the end time once the loop finishes and then calculates the elapsed time.
-
Prefect: Cold start + Execution + Results = 8.09s
-
Windmill: Cold start + Execution + Results = 12.33s
Trust but verify: to ensure the performance of Windmill's workers and measure their capabilities, we provide a benchmarking tool. This tool allows you to benchmark the execution of jobs and flows, providing insights into the performance metrics.
Observability and Monitoring
In terms of monitoring and observability, Windmill and Prefect's cloud app fulfill the same functions: audit logs, monitoring of runs and schedules, saving of flows, configurations (Blocks in Prefect, Resource Types in Windmill) and variables, allocation of workers.
The main difference is that Prefect's application seems to be conceived as a reliable source for reporting actions from Prefect, whereas with Windmill, the application provides more control as everything can be done from the Windmill app (creation of scripts, flows apps, triggers, monitoring, permissions, etc.)
Windmill AI
Windmill provides ways to have AI help you in your coding experience. From prompts, generate scripts that interact with your integrations, or flows where the AI manages the data transmission between steps, or even automatic error resolution.
Pricing
Windmill has a transparent pricing policy, with clear steps for implementation.
Windmill is cheaper at scale and can be used for free at any scale thanks to its open source nature.
As of September 20, 2023, Prefect does not disclose its Enterprise pricing (see below). Its Organization pricing is more expensive than Windmill's Team Plan.
Prefect pricing:
Scripts and Apps
This document has focused on the Flow editor since it is the product closest to what Prefect does. However, Windmill also includes Script and App editors.
The script editor allows users to build long-running heavy background jobs, script with complex dependencies, endpoints with high rpm or simple one-off tasks without any overhead. They can be triggered them from a webhook, auto-generated UIs, flows, apps etc.
The app editor goes beyond auto-generated UIs and allows you to create your own customized UIs using drag-and-drop from components powered by Windmill scripts and flows.
In conclusion, Prefect is an excellent tool for building workflows and heavy data ETL. From Windmill's perspective, Prefect's flows don't lack major features.
However, Windmill distinguishes itself from Prefect by providing more control for building and monitoring flows, as well as scripts and UIs, all from a single open-source platform.