Operational Database Management Systems
Prefect is an open source workflow orchestration platform for data pipelines and ML workflows. It provides a Python-native API for defining, scheduling, and monitoring workflows, with built-in support for retries, caching, parameterization, and observability....
dbt is an open source transformation framework that enables data analysts and engineers to transform raw data in their warehouse using SQL and Python. It brings software engineering best practices — version control, testing,...
Apache Polaris is an open source catalog for Apache Iceberg tables, providing a centralized metadata layer that enables interoperability across different data lakehouse query engines. It implements the Apache Iceberg REST Catalog specification and...
Apache Iceberg is an open source high-performance table format for large-scale analytic datasets. It brings ACID transactions, schema evolution, time travel, and partition evolution to data lakes, and is supported by all major query...
LangGraph is an open source library for building stateful, multi-actor AI applications and complex agent workflows. Built on top of LangChain, it uses a graph-based abstraction to model agent state and transitions, enabling the...
CrewAI is an open source framework for orchestrating role-playing, autonomous AI agents that collaborate on complex tasks. It allows developers to define agents with specific roles, goals, and tools, and to organize them into...
AutoGen is Microsoft’s open source framework for building multi-agent AI systems. It enables the creation of networks of AI agents that can converse, collaborate, and coordinate to execute complex tasks. AutoGen supports both fully...
DSPy is an open source framework from Stanford University for programming language models using optimizable, modular pipelines rather than brittle hand-crafted prompts. It introduces the concept of “signatures” and “modules” that can be automatically...
Haystack is an open source LLM orchestration framework by deepset for building production-ready RAG pipelines and AI-powered search applications. It features a component-based pipeline architecture, support for multiple LLM providers and document stores, and...
LangChain is an open source framework for building applications powered by large language models. It provides modular abstractions for chains, agents, memory, tools, and data retrieval, with integrations to dozens of LLM providers, vector...