OpenMetadata is the most trusted and reliable platform for data organizations worldwide. Our community thrives on the ease of metadata management offered by OpenMetadata, a unified all-in-one solution for comprehensive data needs. With the 1.4.0 release, we've enhanced the user experience to better serve the diverse needs of users and teams.

This release introduces some exciting features for Collate:

• Metadata Actions: Managing data at scale can be challenging and resource-intensive, requiring consistent documentation, glossary terms, tags, and ownership updates. OpenMetadata simplifies this with its API-based structure, enabling teams to automate these tasks seamlessly.

  In version 1.4.0, we introduce Automations—a no-code framework that lets users quickly build workflows directly from the UI. These workflows can add owners, tiers, domains, descriptions, glossary terms, and more, as well as propagate these attributes using column-level lineage.

• Bulk Upload Data Assets: OpenMetadata has made it easier for data governance teams by allowing the import of glossary terms. This feature helps manage glossaries within the platform more efficiently. In the latest release, Collate expanded these capabilities to also include importing databases, schemas, and tables.

  This enhancement simplifies updating descriptions, ownership details, tags, and other metadata for many tables, schemas, and databases directly from the user interface.

• Data Quality: OpenMetadata’s Data Quality feature includes a time series view of test case success and failure, helping users track failures and their timing. Each test also shows the number of failing rows, offering insights into issues.

  In version 1.4.0, we made debugging even more accessible by displaying captured samples of failed rows directly in the UI, allowing users to identify and address problems quickly.

• Connectors: We are introducing three new connectors: Qlik Cloud, Kafka Connect, and Alation.

• New Landing Page Widgets: The OpenMetadata Landing Page is customizable with widgets like Activity Feeds, My Data, and Data Insights, giving users the flexibility to view important information upon logging in. In the 1.4.0 release, we're adding a new Data Quality Widget, allowing users to see the performance of their test cases.

Check out the release blog for OpenMetadata 1.4 and dive into the open-source features!

Metadata Automations

The state of metadata management is constantly evolving. As teams and data platforms grow, the complexity grows exponentially, making it difficult to scale. In the 1.4 release, we have introduced Metadata Actions, a no-code solution to create and maintain high-quality metadata at scale using automation. It is built on top of OpenMetadata’s comprehensive APIs with complete UI control to automate operations where custom development is usually needed.

Metadata Automations streamline governance processes from ownership assignments to tagging, ensuring compliance and consistency. You can add and remove owner, tier, domain, tags, glossary terms, and descriptions in bulk and schedule the process to update your metadata.

You can select the Automation assets and filter by service, owner, domain, or any other supported property to narrow down the selection. The selected assets can be viewed with a single click, helping you validate which assets will be updated. Users can also apply Metadata Actions to the columns or fields for tables, data models, topics, and search indexes. Moreover, you can overwrite existing metadata or keep the present properties of the assets.

With the lineage propagation workflow, you can automatically annotate descriptions, tags, and glossary terms through column-level lineage based on a single source of truth.

PII tagging can be automated by linking assets to a glossary term, which in turn will assign the term’s tags to each piece of data. You can also tag PII data in bulk using the Automations or by letting our NLP models identify PII based on your column names and sample data.

Automations aim to democratize metadata management for non-technical users. Governance users can easily manage metadata and scale their data platform using rule-based automation.

You can watch this video to see them in action!

Bulk Upload Data Assets

OpenMetadata has made data governance teams easier by allowing the import of glossary terms. This feature helps manage glossaries within the platform more efficiently. In the latest release, OpenMetadata expanded its capabilities to include importing databases, schemas, and tables.

This enhancement simplifies updating descriptions, ownership details, tags, and other metadata for many tables, schemas, and databases directly from the user interface.

With these new features, users can more easily maintain accurate and up-to-date metadata across their data assets. If you have hundreds of tables that need new descriptions or updated ownership information, you can now handle these updates in bulk rather than individually.

This saves time and ensures consistency and accuracy across your data environment. The ability to manage these updates directly from the UI means you don't need to rely on complex scripts or external tools, making the process more accessible to a broader range of users within your organization.

Landing Page Widgets

The OpenMetadata Landing Page offers customization with various widgets, such as Activity Feed, My Data, and Data Insights. This customization allows users to see the most relevant and important information as soon as they log into OpenMetadata.

With the 1.4.0 release, we are enhancing this customization by introducing a new Data Quality Widget. This widget enables users to monitor the status of their test cases directly from the landing page. Users can quickly identify and address any issues by having immediate access to data quality metrics, ensuring their data remains reliable and accurate.

Data Quality - Sample Failed Rows

OpenMetadata’s Data Quality feature includes a time series view showing the success or failure of test cases. This makes it easier for users to track if and when a test case fails. Additionally, each test case displays the number of failing rows, providing valuable insights into why a test fails. Users can then query the data warehouse to identify which specific rows are causing the issue.

In version 1.4.0, debugging a failed test case becomes even simpler. While running the test case, OpenMetadata captures and samples the failed rows, presenting these results directly in the UI. This enhancement lets users quickly identify and address issues without running additional queries, streamlining the troubleshooting process without leaving OpenMetadata.

Conclusion

The team at Collate has been pushing a very extensive list of features, both for OpenMetadata in open-source and Collate-only features.

Book a demo with our team and see how you can get your Data Platform to the next level.