Introduction:

Are you thinking of relying on AI to understand how your organization uses information? Well, the truth is, AI hasn’t reached that level of sophistication just yet. If you have any thoughts to share on this, please comment below.

The responsibility falls on you to uncover how your organization utilizes unstructured data. By understanding the utility derived from metadata, which includes classification, you can unlock the power of AI to classify and tag large volumes of documents effectively.

Why Metadata?

We don’t need to emphasize how much time information workers waste searching for unstructured information—it’s common knowledge. However, what’s not commonly understood is how to reliably determine your metadata requirements. Surprisingly, many organizations repeatedly attempt and fail in this area.

Full Text Indexing VS. Metadata or Database Search

Sure, search tools may improve with the help of AI in the coming years. But even with advancements, simply relying on search algorithms may not significantly reduce search times or improve reliability—unless something is done with your metadata.

I propose that there are two main categories of search:

• Process – Documents used in a defined workflow, typically in high volume and critical tasks. Metadata is critical.
• Non-Process – These documents or images are accessed in a more random or ad-hoc manner, or created and used in non-structured workflows. Metadata helps.

Process Searches

These searches involve documents or images used, referenced, updated, reviewed, or approved as part of a work process. In high-volume and critical processes, like aircraft manufacturing or healthcare, documents play a crucial role in the organization’s core functions. Wasting time searching for them can have a direct impact on the organization’s effectiveness and bottom line. Frustration and anger towards the technology responsible for search inefficiencies are often directed by business units facing these challenges.

To determine the metadata requirements for process-oriented information, you can analyze the lifecycle states of the information within the workflow. By asking questions like, “What key piece of data is used to determine the correct document for this task?” you can identify the document categories and essential metadata values. Later, with the help of AI, these metadata models can be used to train the system for accurate classification and tagging.

Automating the metadata tagging process within process tasks is crucial for deriving value from your ECM system. By integrating workflows with associated unstructured information and automatically tagging them as they go through assessment and approval processes, you eliminate the need for repetitive data entry and enhance efficiency. The result? Unambiguous search results that operators in critical process workflows can rely on, saving time and ensuring accuracy.

Non-Process Searches

These searches are more ad-hoc and less time-bound. While time is a factor, it has a lesser impact on the outcome. In this scenario, metadata still contributes to user satisfaction and overall efficiency, but contextual results from full-text index searches are more acceptable.

Full-text indexing combines file names, associated metadata, and the words found within the documents. However, organizations often have varying file naming conventions across different business units, leading to search results that heavily rely on context and file contents, which can vary from one author or business unit to another.

For non-process searches, an automated AI metadata tagging system becomes an efficient and productive way to tag documents. If there isn’t an automated workflow or process for tagging documents, AI remains the most viable method to improve search efficiency through effective tagging.

Conclusion

The bottom line is that, yes, AI can help you tag your content with metadata. However, for it to be relevant to your business, you need to categorize it and determine your metadata model. Once you do that, you can train your AI against it.

For process-centric content you can add the required search metadata either on ingestion or during a workflow process, if the volumes justify it. AI powered metadata tagging can also be used to augment the tagging done in the workflow process.

For non-process-centric content AI metadata tagging can be utilized to make existing content more findadable either during ingestion, or after the fact.

Keep in mind that folder and file naming conventions will also improve search results without the need for AI metadata tagging, while improving the site/folder navigation experience. More on site/folder structure in a later later post.

Summary

• AI can help tag content with metadata, but first you have to categorize the content files, E.g. contracts, invoices, assessments, design spcifications, marketing plans, business plans. Then, find out from the business how they search for the files, what keywords they use, and where they find them. Keep the required number of metadata fields under three for each category.
• Process-centric searches or just-in-time information delivery requires metadata. Full-text-indexing is not the correct solution due to the time constraints associated with service delivery. You can look for ways to automate the population of metadata during ingestion of files or during the process tasks or transitions. AI metadata tagging can help after-the-fact, if required.
• Non-process centric or ad-hoc document searches can be helped in full-text indexing cases by using standardized and enforced site/folder and file naming conventions. This requires change management and training. Automated file name generation helps by making it easy for users to conform to policy.
• AI metadata population is very useful for tagging content to be used to enhance search result reliability. Some full text search engines can be optimized to prioritize specific keywords or tags.
• AI platforms that can be used to tag content files.
  • Google Cloud AutoML
  • Microsoft Azure Cognitive Services

For more information, or if you’d like to chat, please reach out to me at mike.clarke at qtility dot com. (You will need to change the stop words to make it a proper email address. Sorry for the inconvenience, but you would not believe my spam folder.)