OpenMetadata 0.13.2 Release
Improved SQL Lineage, Glossary Bulk Upload, Unified Tag Category API, Mutually Exclusive Tags, Chrome Extension, and lots more.
We are glad to announce OpenMetadata’s Release — 0.13.2, which showcases some exciting features and improvements like Glossary bulk upload, improved SQL lineage, mutually exclusive tags, and a beta release of the OpenMetadata Chrome extension.
The OpenMetadata community recently hosted a Mini-Webinar on Custom Connectors, which discussed the core aspects of OpenMetadata, the ingestion process, and building your own custom connectors. Most organizations have their own in-house systems and other third-party services crucial to their business. OpenMetadata supports 60+ connectors, and by building a custom connector, you can have a holistic view of everything that happens in your Data Platform.
We plan to add additional features to the SaaS version in future releases, so stay tuned for more updates 🚀.
Community Update
Thanks to OpenMetadata’s amazing community. We’ve been receiving productive feedback and active code contributions over the past one and a half years. And, we deeply appreciate your support and feedback.
Crossed 1800+ GitHub stars
The Slack community reached 2200+ members
141 Open source GitHub developers
312 Commits were merged into the 0.13.2 release
OpenMetadata 0.13.2 Release Highlights
Glossary Import and Export
Glossary import and export
When the Glossary was introduced in the 0.9 release, we supported a manual approach to add the glossary terms one at a time. This can become time-consuming for organizations already maintaining a business glossary in a spreadsheet or similar tools. Based on community feedback, we’ve introduced the functionality to bulk upload glossary terms. Users can save time and effort by uploading a CSV with thousands of terms in one go. The import utility will validate the file and show you a preview of the elements that will be imported to OpenMetadata. Just as you can import the glossary terms, you can export the glossary data as a CSV file.
We take your feedback really seriously, and we are always excited to see community requests come to life!
Improved SQL Lineage
Open source thrives when communities grow together. We have been heavy users of sqllineage for our query parsing processes, and it’s been a great tool. Still, the growing number of connectors and community needs required us to improve our existing parsing capabilities.
We have worked together with Reata, the developer behind sqllineage, to migrate the library’s core to sqlfluff. We’d like to take this opportunity to thank both communities for merging our contributions and helping bring our lineage parsing to the next level.
If you are interested in learning more about how we handle Data Lineage in OpenMetadata, you can look at our webinar!
New Glossary UI
Previously, the Glossary and the Glossary terms were displayed together in a tree UI. Now, we’ve separated the Glossary and Glossary Terms within the UI, such that the Glossaries are displayed in the left navigation pane, while the glossary details are displayed on the right. The list of terms has been sorted alphabetically for better navigation. The improved UI changes will help you easily access the details with tags and descriptions in the cards.
From Tag Category to Classification
We have renamed Tag Categories to Classification as a more widely used term for expressing groups of tags. The team also took this chance to unify the Tags API to conform to the other APIs we have been building.
This change will make it easier for developers familiar with the existing APIs, and it simplified different internal flows. You can find more information about the exact modifications here.
Mutually Exclusive Tags from the UI
There are situations where mixing multiple tags from the same classification does not make sense. Imagine an asset being tagged both with PII Sensitive and PII Non-Sensitive!
When creating a Classification or a Glossary term, their elements can be mutually exclusive. If this configuration is enabled, you won’t be able to assign multiple tags from the same category or glossary to the same data asset.
Chrome Extension
OpenMetadata Chrome Extension
We are elated to announce the launch of the BETA version of the OpenMetadata Chrome extension. Feel free to explore it and share your feedback. We are focused on meeting the needs of the Data Community, by simplifying processes and making all the metadata ubiquitously accessible by providing the necessary context in all your tools.
Connectors
Databricks Pipeline Connector has been added as a community contribution!
Some important fixes have been made for the AWS QuickSight connector.
Improvements have been made to DB2 constraints and profiler.
The Oracle connector now ships with the Thick mode enabled.
Added support for Postgres Foreign Tables
Connectors like Redshift or Snowflake are way faster now, as we get the descriptions in batches.
Added support for Data Lake profiler row-based sampling.
Entity Names
The team introduced an enforcement of the Entity Names’ format using regex patterns. The intention is to better manage and harmonize Entity Names values and to allow users to form better expectations around them.
You can find more information about this change in the docs.
Planned for Next Release
Our next release will be the long-awaited 1.0 Release ✨
The team has already started developing amazing new features, such as:
Storage Service revamp to better represent your Datalakes (based on your feedback) with awesome community members!
NoSQL stores API definitions to prepare the ingestion for sources like Elasticsearch,
Automatic PII tagging during the profiler workflow,
Continuous improvement for the brand-new sqllineage package
And more! Stay tuned for any news, and let us know which features you’d like to see next!
Thanks to our Contributors
It’s been great to associate with the open-source community members for the last one and a half years. We’ve been receiving encouraging feedback, and great code contributions; all this has been a huge motivation to keep scaling new heights. Thanks to Aashish1221, Alina Valea Forter, Bigdata-spec, Frederico Cotrim, geoHeil, Gnomolio, Itai Sevitt, Laila Patel, Martin Trillhaas, Nithin Kumara N T, Paulo Ventura, Sérgio Passos for raising GitHub issues that made it to the 0.13.2 release.
Please contact us on Slack if you have questions about code, installation, and docs. For feature requests, you can file a GitHub issue or reach out to us on Slack. Interested in contributing code? Here are some good first issues to get you going.
Please give us a GitHub star if you like what we are doing. That’ll greatly help OpenMetadata in reaching a wider audience.
OpenMetadata 0.13.2 Release was originally published in OpenMetadata on Medium, where people are continuing the conversation by highlighting and responding to this story.