An introduction to Metadata Application Profiles

Editor's Notes

• #3: We' e all gotten the message about data silos and how bad they are. They prevent data sharing, they keep us from taking advantage of the work of others.
• #4: We think of data silos as being the result of using different standards or different data formats.
• #5: But just because we use the same basic standard, it doesn't mean that we are producing identical data. As anyone who has tried to consume data from another institution or project knows, there are always local variations – variations in which fields or terms are used, variations in how the data has been recorded. Even when we think we are not creating data in a silo, in many cases we are. It's difficult to avoid have your data be your data. I'm going to talk about how application profiles can help with data sharing and data reuse.
• #6: They are many things
• #8: Rather like the schema.org focus in terms of data providers = anyone with data on the web. These folks are not always coders, and may have experience limited to a only a few technologies. Any solution has to fit into their toolbox; we can’t require them to re-tool to use this solution.
• #11: Today there is no standard format for application profiles
• #12: Profiles exist today, and take many forms. This is an example of a profile based on the Dataset Catalog (DCAT) vocabulary. There a number of different application profiles based on this vocabulary, which is was developed to support government open data in the European Union, although it also used elsewhere. Many countries have created application profiles that are specific to their country because they each had some specific needs. Those choices are recorded in the application profiles which generally take the form of documents.
• #13: BIBFRAME has a number of profiles.
• #14: This is a profile based on the BIBFRAME vocabulary.
• #15: There is information about each term, including what type of value is expect (whether it will be a URI for a thing or will be text).
• #16: Dublin Core has had the ide aof application profiles since the late 1990s. This fits in well with the fact that DC is intended as a highly reusable vocabulary that can fit many different situations and needs.
• #17: First in 2007. In keeping with the use of the place name to name things, this was first presented in Singapore.
• #23: It's the picture that you have in your head that tells you what aspect of the world your data covers, what are the "parts" or entities that make up your description of that world, and how the parts fit together. Domain models can vary in their complexity and detail, and may have different levels of detail depending on the view that is needed at some moment in time.
• #27: Profiles generally are a reuse of vocabularies. Like BIBFRAME, the profile can be a selection from a single vocabulary. In this case the profile's message is: I am a profile of BIBFRAME. I use some but not all of the BIBFRAME-defined vocabulary. Profiles can make use of all or part of a vocabulary, but can add terms that they need but that are missing from the base vocabulary. This is the case with the DCAT profiles: they are based on the DCAT vocabulary, but in some cases the users of DCAT needed to add some information that was not covered by that vocabulary. If you have worked with the Europeana Data Model you may be familiar with profiles that have some Europeana elements but that also add their own terms or equivalents. The DCAT profiles have much overlap between them but each one has information that is not included in DCAT. But profiles can also be not related to any single vocabulary. They can be a mix and match that essentially creates a new vocabulary. Oftentimes these vocabularies are not treated as profiles, and the line between a new vocabulary made up of existing terms and a profile is not distinct. We can say, however, that a profile does not have to be primarily based on any one vocabulary. Even Europeana and DCAT vocabularies make use of terms from Dublin Core,
• #28: bibframe is an example of a profile that is a selection from a single vocabulary.
• #29: DCAT uses terms from its own vocabulary, from dublin core terms and foaf, among others.
• #30: You have to pay attention to how a term is defined before you reuse it.
• #31: When you have all of this together, then it is time to create your profile.
• #38: Examples
• #41: The problem is the validation code usually is pretty complex. This is an extremely simple example that say that my property "status" can be one of two things: "assigned" or "unassigned". So you an imagine how much code it takes to say something much more complicated. It isn't reasonable to assume that everyone who creates metadata is capable of writing the needed validation code.
• #43: There are a lot of things in data that cannot be validated, especially for those in the cultural heritage area where much of there metadata consists of text, and is based on decisions made by human beings, not on calculations. So although validation is important, validation alone won't describe a profile.
• #45: These are community decisions. If you do not include in your development of profiles the means to maintain and evolve the profiles, within a short time they cease being living solutions to your metadata needs. https://pro.europeana.eu/project/creation-and-governance-of-edm-mappings-profiles-and-extensions-task-force http://makxdekkers.com/DXWG/DCAT-AP.pdf
• #47: Just to mention here that I am co-chairing the W3C group representing DC. That group will be creating a kind of "best practices" document but nothing so specific as code. The Dublin Core work is more detailed, but has not yet yielded a usable schema for profiles. Hopefully that is in progress.
• #52: This looks like an entity-relation diagram at this point. But more is needed.
• #53: This looks like an entity-relation diagram at this point. But more is needed.
• #54: In a sense, an application profile for an application profile. Not yet "finished", but may demonstrate that we can use something as simple as a spreadsheet to allow people to easily create application profiles that can be converted to a form that would allow for ingest and validation. All without the metadata schema creator having to write code. Note in particular that very few of the elements here are required. (Those with an initial zero in the third column are optional.) An application profile could be simply a list of terms that are used for one or more resources, with a resource being a document, a person, a place, a subject. Whatever you want it to be.
• #55: Spreadsheets – which then become CSV files (comma separated files).
• #56: It would be ideal to be able to express all of the rules that would be necessary to validate your data. That can get to be quite complex.
• #57: Can we include in this simple view of profiles an easy way to include validation rules, or at least the most basic validation rules? That would go a long way to aiding interoperability of datasets.