Automated Metadata Harmonization Using Entity Resolution & Contextual Embedding

17 Oct 2020  ·  Kunal Sawarkar, Meenkakshi Kodati ·

ML Data Curation process typically consist of heterogeneous & federated source systems with varied schema structures; requiring curation process to standardize metadata from different schemas to an inter-operable schema. This manual process of Metadata Harmonization & cataloging slows efficiency of ML-Ops lifecycle. We demonstrate automation of this step with the help of entity resolution methods & also by using Cogntive Database's Db2Vec embedding approach to capture hidden inter-column & intra-column relationships which detect similarity of metadata and then predict metadata columns from source schemas to any standardized schemas. Apart from matching schemas, we demonstrate that it can also infer the correct ontological structure of the target data model.

PDF Abstract

Code
Add Remove Mark official

No code implementations yet. Submit your code now

Datasets

  Add Datasets introduced or used in this paper

Results from the Paper
Add Remove

  Submit results from this paper to get state-of-the-art GitHub badges and help the community compare results to other papers.

Methods
Add Remove

No methods listed for this paper. Add relevant methods here
Contact us on: hello@paperswithcode.com . Papers With Code is a free resource with all data licensed under CC-BY-SA.
Terms Data policy Cookies policy