MetricFlow 0.207.2

Changes from 0.207.2:

This release includes a number of a number of assorted fixes and updates that were backported from main. It includes some updates to enable CTEs by default.

MetricFlow 0.207.2 - February 21, 2025

Features

• Allow setting aliases for queried metrics (#1573)

Fixes

• Always treat metric_time and the agg_time_dimension the same in the JoinToTimeSpineNode. (#1541)
• Apply time constraints after time offsets to avoid filtering out values that will change later in the query. (#1544)
• Update mf tutorial flow to always create / use the sample dbt project. (#1651)

Contributors

• @courtneyholcomb (#1541, #1544)
• @plypaul (#1651)
• @serramatutu (#1573)

MetricFlow 0.207.1

Changes from 0.206.0:

Note: this release is compatible with / requires dbt-core >= 1.9.0, < 1.10.0 and is cut from 5bb13511d36c6122527a1bd12de9d6e0707a9ddc. The prior version 0.207.0 on Pypi depended on a version of dbt-semantic-interfaces that was later than the versions supported in the latest official release of dbt-core (1.9.1 as of this note).

Breaking Changes

• Changes MetricFlowQueryRequest.where_constraint to where_constraints and now accepts a list (#1431)
• Require Python >=3.9 (#1470)
• Require dbt-core >= 1.9.0, < 1.10.0 (#1615 )

Features

• Enable predicate pushdown for categorical dimensions (#1011)
• Support sub-daily granularity options in SQL rendering for all supported SQL engines. (#1258)
• Adds a new dataflow plan node to re-aggregate metrics using window functions. Needed to calculate cumulative metrics at non-default granularities. (#1274)
• Build dataflow plan for cumulative metrics queried with non-default granularity. (#1281)
• Remove restriction on querying non-default granularities with cumulative metrics. (#1282)
• Enable predicate pushdown optimization by default for all callers (#1011)
• Use Metric.time_granularity to resolve metric_time. (#1310)
• Support multiple time spines with different granularities. (#1348)
• Enable sub-daily queries without metrics. (#1359)
• Enable sub-daily time constraints. (#1367)
• Support combining AggregateMeasuresNodes where metric input measures have aliases, so long as there are no duplicates. (#1375)
• Handle custom granularities in DataflowPlan. (#1382)
• Support custom granularities in query parsing. (#1413)
• Support conversion metrics queried with custom granularities. (#1475)
• Basic support for join_to_timespine metrics with custom grain in the group by. (#1505)
• Use CTEs instead of sub-queries in generated SQL. (#1040)

Fixes

• When querying multiple agg time or metric time dimensions with a cumulative metric, select all of them from the time spine table. (#1271)
• Remove extraneous where filter subqueries added by predicate pushdown (#1011)
• Add Column header to the exported CSV from query command
• Allow metrics with matching input measures where one is cumulative and one is not. (#1374)
• Fixes bug where conversion metric query fails when filter with base semantic model's dimension is provided (#1210)
• Bug fix: ensure that granularity requested for non-additive dimension is respected. (#1383)
• Bug fix: when querying a join_to_timespine metric with a metric_time filter that is not included in the group by, unexpected output rows were included. (#1450)
• Remove unnecessary group bys that make queries less efficient. (#1453)
• Add new validation that checks for SCDs in the join path to make grouping by metric_time required in this case. (#1451)
• Make ID generation thread-safe (#1473)
• Prevent SourceScanOptimizer from combining nodes that use the same input metric alias in different derived metrics (#1494)
• Fix mf tutorial experience. (#1631)
• Compatibility Issue with dbt-core 1.9.0 and dbt-metricflow 0.7.1. (#1589)
• dbt-core dependency issue with metricflow==0.207.0. (#1632)

Under the Hood

• Add test coverage for more filter + join interactions (#1240)
• Add dataflow plan optimizer to replace build-time predicate pushdown (#1011)
• Added filtering for DWH validation tasks and saved query support (#1271)
• Refine subquery ID generation. This may result in changing subquery ids for rendered SQL. (#1280)
• Move categorical dimension predicate pushdown to DataflowPlanOptimizer (#1011)
• Consume cumulative-specific metric type params from new cumulative_type_params field. (#1293)

Dependencies

• Allow pydantic 2.x installations (#1299)

Contributors

• @WilliamDee (#1210, #1271)
• @WilliamDee,courtneyholcomb (#1431)
• @courtneyholcomb (#1258, #1274, #1281, #1282, #1310, #1348, #1359, #1367, #1375, #1382, #1413, #1475, #1505, #1271, #1374, #1383, #1450, #1453, #1293)
• @plypaul (#1470, #1040, #1473, #1631, #1589, #1632)
• @saurabh0402
• @serramatutu (#1451, #1494)
• @tlento (#1011, #1011, #1011, #1240, #1011, #1280, #1011, #1299)

MetricFlow 0.206.0

MetricFlow 0.206.0 - June 11, 2024

This release requires dbt-core version 1.8. The most important breaking change here will affect installation of the MetricFlow CLI - any MetricFlow CLI users should install dbt-metricflow to pick up the latest CLI updates. This was a necessary step in the process of decoupling the main MetricFlow library from dbt-core's minor version release cycle. Cloud CLI users are not affected.

Notable improvements include the ability to use metric values as filter expressions and some fixes for strange filter behaviors on time spine joins.

Breaking Changes

• Remove unsupported legacy MetricFlowClient APIs (#1094)
• Move MetricFlow CLI to dbt-metricflow package (#1090)

Features

• Allow metrics in filters for metrics and metric queries. (#740)
• Enable metric filters for distinct values queries. (#1107)
• List all dimensions for the semantic manifest. (#1142)
• Add a Dependency Resolver for Saved Queries (#1155)
• Add support for Python 3.12 (#1065)
• Update dbt-metricflow dependencies to use dbt* 1.8 packages (#1243)

Fixes

• Enable filtering by metrics in the same semantic model as queried metrics. (#1115)
• Improve error messages for dataflow plan failures. (#1118)
• Remove redundant entity names like 'listing__listing' from suggested group bys. (#1130)
• For metrics that join to time spine, apply post-join filters only for specs that are in the group by. (#1119)

Under the Hood

• Clean up files included in MetricFlow package distribution (#1092)
• Move MetricFlow tests to repository root (#1088)
• Create metricflow-semantics package. (#1150)
• Make source semantic models available from DataflowPlanNode instances (#1218)
• Cache functions related to parsing group by options in order to improve query performance. (#1229)

Dependencies

• Update Ruff to 0.3.3 (#769)
• Update dbt-semantic-interfaces minimum version to 0.5.1 (#1104)

Contributors

• @courtneyholcomb (#740, #1107, #1142, #1115, #1118, #1130, #1119, #1229)
• @plypaul (#1155, #1065, #1243, #1150)
• @tlento (#1094, #1090, #1092, #1088, #1218, #769, #1104)

MetricFlow 0.205.0

This release is an intermediate update to MetricFlow and can be used with existing dbt-core 1.7 installations. The breaking changes listed here will only affect users relying on any sql-comment tagging the MetricFlow CLI had been doing previously (e.g., the mf_rid_* comments).

Notable improvements include increased flexibility in querying the metric_time dimension and increased consistency in how we manage metrics requesting joins against a time spine for filling in missing values on a time axis.

MetricFlow 0.205.0 - February 29, 2024

Breaking Changes

• Remove SQL-Comment-Based Tags (#1034)

Features

• Enable querying metric_time without metrics. (#928)
• Enable querying cumulative metrics with their agg_time_dimension. (#1000)
• Enable offset metrics to be queried with agg_time_dimension. (#1006)
• Add Support for Consistent SQL Query Generation (#1020)

Fixes

• Validate that there are metrics or group by items in each query. (#1002)
• For measures that join to time spine, allow joining when agg_time_dimension is queried. (#1009)
• Join to time spine if requested for conversion metric input measures. (#1048)
• Enable querying offset metric with multiple agg_time_dimensions at once. Also fixes a bug when filtering by a different grain than the group by grain. (#1052, #1053)
• Bug fix: if measure joins to time spine, apply filters again after that join. (#1039)
• Improve error message for metrics/queries with missing inputs (#1051)

Docs

• change group-bys to group-by in the tutorial message

Under the Hood

• Add test for nested derived metric filter rendering bug fixed in 0.204.0 (#920)

Dependencies

• Remove unused numpy dependency (#984)
• Update Jinja2 past 3.1.3 to resolve CVE-2024-22195 (#1049)

Contributors

• @batou9150
• @courtneyholcomb (#928, #1000, #1006, #1002, #1009, #1048, #1052, #1053, #1039)
• @plypaul (#1034, #1020)
• @tlento (#1051, #920, #984, #1049)

MetricFlow 0.204.0

To take full advantage of this release, please upgrade to dbt-core 1.7.4 or later. Earlier versions of dbt-core 1.7.x will continue to work with any config that does not include Conversion Metrics.

Notable improvements include:

• Support for Conversion Metrics
• Support for Trino (huge thanks to @sarbmeetka for the contribution!)
• A variety of important fixes and adjustments, particularly around time offset metrics

MetricFlow 0.204.0 - January 11, 2024

Features

• Add Trino support to the MetricFlow. (#207)
• Implemented date_part in where filter. (#None)
• Resolve Ambiguous Group-By-Items (#887)
• Support for Conversion Metrics (#252)
• Add a Query Validation Rule for Repeated Metrics in a Query (#943)
• Expose label on Metric & Dimension for use in APIs. (#956)

Fixes

• Apply time offset for nested dervied & ratio metrics (#882)
• Fix Incorrect SQL Column Name Rendering for WhereConstraintNode (#908)
• Unable To Satisfy Query Error with Cumulative Metrics in Saved Queries (#917)
• Fixes a bug in dimension-only queries where the filter column is removed before the filter has been applied. (#923)
• Bug fix: Keep where constraint column until used for nested derived offset metric queries. (#930)
• Fixes incorrect time constraint applied to derived offset metrics. (#925)
• Remove default time constraint for queries with cumulative metrics. (#917)
• Return exit code 1 for failed validations (#867)
• Optimizer Does Not Deduplicate Common Metrics (#941)
• Duplicate input measures after combiner optimizer (#969)

Under the Hood

• Test to ensure Dimension and TimeDimension syntax are identical in the case of time dimensions
• Fixed typo in error message

Dependencies

• Remove unnecessary MarkupSafe dependency (#950)

Contributors

• @DevonFulcher
• @WilliamDee (#252, #867, #969)
• @courtneyholcomb (#956, #882, #923, #930, #925)
• @plypaul (#887, #943, #908, #917, #917, #941)
• @sarbmeetka (https://github.com/sarbmeetka) (#207)
• @tlento (#950)

MetricFlow 0.203.1

MetricFlow 0.203.1 - November 15, 2023

Fixes

• Fix error in cumulative metric output when start-time and end-time are specified (#869)

Dependencies

• Remove Rudderstack client and associated dependencies (#866)
• Relax version pin on typing extensions to allow >=4.4, <5 (#875)

Contributors

• @tlento (#869, #866, #875)

MetricFlow 0.203.0

This release requires an upgrade to dbt core 1.7 , which is now the minimum requirement for future minor version updates to MetricFlow.

Notable improvements include:

• Support for saved queries
• Fixes for issues where time granularity was not being applied to columns with finer-grained input than what was listed in the config. This may cause performance regressions for users of partitioned datasets in certain engines, which we are committed to resolving.
• Support for configuring a way to fill null value measure outputs to prevent implicit null filtering.

Complete changelog follows. Please take note of the breaking changes.

MetricFlow 0.203.0 - November 13, 2023

Breaking Changes

• Use FULL OUTER JOIN to combine input metrics for derived metrics. This is a change from using INNER JOIN and may result in changes in output. (#842)
• Update Dependencies to Use dbt-semantic-interfaces~=0.4.0 (#846)

Features

• Enable DATE PART aggregation for time dimensions (#770)
• Support Saved Queries in MetricFlow (#765)
• Support for sort order in query interface (#None)
• Support for the Dimension(...).grain(...) syntax for the where parameter (#None)
• Support querying dimensions without metrics. (#804)
• Join to time spine and fill nulls when requested on metric input measures. (#759)
• Fill nulls for multi-metric queries (#850)

Fixes

• Removing methods and reordering parameters for Query Interface. (#None)
• Coerce time granularity to configured value to prevent finer-grained timestamps from causing unexpected query behavior (#714)
• Prioritize source nodes based on correct cost (#801)
• Enables case insensitivity for various query params. (#802)
• Ensure extract calls return consistent results across engines (#792)
• The --order param was being dropped from CLI saved queries. (#835)
• Fix query validation for metric_time requirements (#825)
• Use FULL OUTER JOIN for dimension-only queries. (#863)

Under the Hood

• A simple update to make the where filter query parameter objects more accurate (#None)
• Expose underlying where clause error message (#None)
• Remove query interface and depend on DSI protocol instead (#None)
• re-categorize TypeErrors that arise from create_from_where_filter into InvalidQueryException (#None)
• Add the ability to use distinct select in sql nodes (#None)
• Removed DatePart Enum and change imports to depend on DSI version instead. (#None)

Dependencies

• Update to dbt-semantic-interfaces~=0.3.0. (#809)
• Update typing-extensions minimum version to 4.4 (#823)
• Update dbt dependencies to ~=1.7.0 (#860)

Contributors

• @DevonFulcher
• @WilliamDee
• @courtneyholcomb (#842, #770, #804, #759, #850, #801, #802, #835, #863)
• @plypaul (#846, #765, #809)
• @tlento (#714, #792, #825, #823, #860)

MetricFlow 0.202.0

This release adds support for granularity adjustments in window offsets and fixes an issue with BigQuery granularity adjustments for YEAR level granularities.

BigQuery users take note - YEAR granularity now truncates to calendar year start (1st January), which was the original intention, rather than ISO year start, which is something like the Monday of the week containing the first Thursday of the year. This is now consistent with other engines.

MetricFlow 0.202.0 - September 06, 2023

Features

• Adds the option for users to specify group by parameters with object syntax matching the where/filter expressions. (#717)
• Expose measures for metrics on MFEngine with agg_time_dimension (#735)

Fixes

• Update dataflow plan to support different granularities with time offset metrics (#726)
• Removes unneeded JoinOverTimeRangeNode step from dataflow plan. (#743)
• Update BigQuery YEAR granularity truncation to use January 1st instead of ISOYEAR start (#755)

Dependencies

• Allow tabulate versions >= 0.8.9 (#762)

MetricFlow 0.201.0

This release adds duckdb support and provides a partial fix for user-reported issues with partition pruning in BigQuery.

MetricFlow 0.201.0 - August 17, 2023

Features

• Add dbt-duckdb as a supported adapter and remove legacy DuckDB sql client (#583)

Fixes

• Remove barrier to partition pruning certain time partition filter predicates (#712)

Under the Hood

• Make duckdb the standard for all dev-env environment runs, including make test (#723)
• (#728)

Dependencies

• Update pandas to 1.5.x (#719)
• Relax version pins for MetricFlow dependencies (#720)

Contributors

• @courtneyholcomb (#728)
• @jwills (#583)
• @tlento (#712, #723, #719, #720)

v0.200.0

This release represents a complete overhaul of MetricFlow targeted at a first class integration with dbt! Complete changelog below:

MetricFlow 0.200.0 - August 02, 2023

Breaking Changes

• License Change - Version 0 to 0.140.0 was covered by the Affero GPL license. Version 0.150.0 and greater is covered by the BSL license. (#465)
• Removing time_format from DimensionTypeParams (#494)
• Use Templates For Defining Metric Filters (#505)
• Rename Metric.constraint to Metric.filter (#511)
• Deprecate and refactor CLI commands
• Removes async query and query cancel methods from SqlClient protocols (#577)
• Remove time spine introspection and table creation, which may break cumulative metric queries (#592)
• Remove SqlEngineAttributes construct from SqlClient interface in favor of dialect rendering and engine type properties (#577)
• CLI needs to be ran in a dbt project root directory
• Remove expr & ratio metrics and bundle with derived metrics. (#504)
• explain_get_dimension_values & get_dimension_values take a list of metrics parameters
• Remove MetricFlow config file - all future configuration must originate with the dbt project (#624)
• Update to dbt-semantic-interfaces==0.1.0.dev8. (#634)
• Changed the --group-bys option in mf query to be --group-by
• Add Support for primary_entity in Semantic Models (#694)

Features

• Added new entity calls to CLI/MetricFlowEngine
• Script to Generate Snapshots for Supported SQL Engines. (#609)
• Add dbt adapter support for postgres and enable it for tests (#578)
• Use dbt adapter to run queries and warehouse validations from MetricFlow CLI (#624)
• Enable Snowflake queries in dbt <-> MetricFlow CLI integration (#579)
• Refactor mf tutorial to work alongside a dbt project.
• New package dbt-metricflow which bundles dbt-core and metricflow and dbt-adapters
• Add Support for Python 3.10 / 3.11 (#659)
• Include metric_time in List Dimensions Output Where Appropriate (#673)
• Enable support for Redshift queries in dbt-metricflow integration (#582)
• Enable Databricks support for the dbt-metricflow integration (#580)
• Enable support for BigQuery for dbt metricflow integration users (#581)

Fixes

• Removes MySQL from SqlEngine and SqlDialect options since it is not supported. (#0)
• Derived metrics were not respecting the constraint defined in the original input metric's definition.
• Fixes type error in BigQuerySqlExpressionRenderer (#536)
• Fix broken type signature for log_call decorator
• Apply transformations to dbt-generated serialized model to fix issue with query generation (#624)
• Improve error message rendering in MetricFlow CLI (#646)
• Added --version and fix manifest transformer rules for dbt-core-=1.6.0b8 (#650)
• Include granularity suffix on time dimension name rendering for all time dimension granularities
• Clean up list dimensions outputs

Under the Hood

• Adding Changie (#457)
• Ensure use of ValidationIssue instead of ValidationIssueType. ValidationIssueType was from a time before ValidationIssue classes had proper inheritance, and it's continued use was become problematic for typing.
• Removing model from the ModelValidator.validate return type. The model isn't altered, and thus doesn't need to be returned.
• Moving AggregationType enum into dbt-semantic-interfaces
• Moving errors relevant to dbt_semantic_interfaces to dbt_semantic_interfaces
• Migrating to RapidFuzz (#470)
• Matching dbt-core issue templates (#457)
• Removing the transform CLA (#450)
• Pinning dbt-core to 1.4 (#475)
• Removing YamlLint (#472)
• Add ObjectToReference class in preparation of removing the .reference calls (##463,, ##464)
• Moving all *Reference objects to dbt-semantic-interfaces.
• Add pytest flag to use a persistent source schema for faster repeat testing. (#482)
• Renamed instances of and related to Identifiers to Entities. (#dbt-semantic-interfaces#9)
• Improves typechecking coverage by updgrading to MyPy 0.942 and removing blanket ignore all imports setting (#536)
• Push mypy to run using local environment packages in pre-commit. Developers should always use a clean virtual environment to ensure consistency with CI. (#530, #536)
• Update mypy to 1.3.0 (#546)
• Migrate from Poetry -> Hatch for Project / Package Management (#549)
• Enable the ability to return only dimensions requested in the query, specifically used for dimension values queries.
• Raising a UnsupportedEngineFeatureError instead of a generic RuntimeError when a data platform doesn't support a feature
• Remove SqlIsolationLevel constructs and other vestigial remnants of defunct SqlClient features (#577)
• Raise a more specific exception when a Metric isn't found during linking.
• Update test environment configuration to allow for more streamlined dependencies
• Remove DDL and other unused methods from SqlClient protocol

Dependencies

• Switches MetricFlow SemanticManifest dependencies from the local dbt semantic interfaces package to the initial dev release (#540)
• Clean up unused dependencies, relax tabulate version pin (#545)
• Update dbt dependencies to support development on the new integration (#571)
• Move SQLAlchemy and SQL engine dependencies out of the production package (#672)
• Update dependencies and attribution file in preparation for 0.200.0 release (#703)

Contributors

• @DevonFulcher
• @QMalcolm
• @WilliamDee
• @callum-mcdata
• @courtneyholcomb
• @nhandel
• @plypaul
• @tlento