Two of the biggest trends in improving business operations these days are Artificial Intelligence (AI) and Machine Learning (ML). The way they are talked about in certain circles make them sound like solutions to literally every problem under the sun. 

But that isn’t entirely true. While AI and ML are amazing tools that can, in the right circumstances, improve efficiency and accuracy in your processes and workflows, they are not perfect pieces of tech that make humans obsolete.

In order to shape the functions of these intelligent tools into workable solutions, many hours and an intense amount of coding is required. At TEAM, we recently tracked the process of creating a new Topic Modeling tool for Syl Search that will be available later this calendar year. If our developers weren’t some of the best in the business, that tool would be nowhere near a reality.

Let’s take a look at the Topic Modeling creation process and then examine how AI and ML can work best in your organization’s day-to-day operations so that we can separate the hype from the reality.

What is the Difference Between AI and ML?

The relationship between AI and ML is a symbiotic one. AI is intelligent software that is programmed to make decisions like a person would. ML is a process by which AI develops its decision making parameters.

AI operates based on sets of rules that have been developed mostly through machine learning and sometimes through what is called deep learning— where the software uses models based on the structure of a human brain. 

A machine learning program will be fed test data sets with which to run its functions. Often, this has to be done several times until the software gets the results that are desired. For example, developers could feed an ML program data from old invoices to automate part of an invoice processing workflow.

The programmers would already know what a successful result looks like, so if they plug in the data and don’t get the result they need to classify the test as a success, they can look through the code to find the anomalies and refine them. 

Once the intelligent software has learned the parameters for its function, it can be incorporated into the actual business process that it was designed to automate. AI doesn’t think on its own. It learns a set of rules that it then applies to the data it is fed.

This is why fears that AI will completely replace human workers are unfounded. AI can make certain tasks go by faster, across greater volumes, and more accurately than a human could do them, but as far as abstract thought or critical thinking goes— that simply isn’t something AI is made for.

So essentially, demystifying AI and ML boils down to this: an AI program makes decisions based on the rules and parameters that get established via machine learning. Creating artificial intelligence software requires patience and deep knowledge.

Bringing Topic Modeling to Syl Searches

What we at TEAM wanted to do was improve the Syl Search product we offer by incorporating a topic modeling tool that would allow for topical searches to discover relevant assets among both your structured and unstructured documents and data.

This tool will allow you to add value to large data sets by displaying the main topics in that data set. But creating that topic modeling tool took much more human intervention than creating tools to automate processes that are more focused on mathematical operations or data entry.

Latent Dirichlet Allocation

In order to recognize sets of words as topics, we needed a generative statistical model that could be used to recognize common topics in our data sets. In natural speech and language processing, there is a statistical model known as the Latent Dirichlet Allocation (LDA).

LDA represents a document as a collection of topics or smaller collections of words within the greater whole— a hidden or latent layer. 

In our Topic Modeling tool development process, we found that the LDA was returning too many topics from our test data set to be useful when we first started. It required human intervention to introduce context and semantic analysis to be added to the LDA by human developers to make the results more meaningful.

Context and Coding

What our specialists found was that Topic Modeling is not so simple as running data sets through an algorithm. They found that using machine learning to teach the program to recognize context was the key to success.

The LDA our team was working with was very responsive to parameter adjustment, so a long process of coding and testing and coding again ensued. It was a surprisingly intense process that resulted in a Topic Modeling tool for Syl Searches that helps them find even more relevant results across your structured and unstructured data sets and documents.

The Hype vs The Reality

So what lessons can we pull from this story of AI, ML, and LDA? The moral of the story is pretty simple. AI is a valuable tool and is improving every day, but there is still a great deal of work that goes into the machine learning required to create viable AI tools and applications. 

The key for an organization that wants to get the most out of their AI and ML tools is to work with a team of experts that can do the complex stuff so that the end-user experience is smooth and simple.

It’s also important for organizations to set reasonable expectations. AI is designed to make the decisions that a human would make within certain parameters. You cannot expect complex, critical thought from your AI. Even with deep learning, it isn’t made to do that. 

The reality of AI and ML is that they can make time consuming, rules based tasks run much faster and more accurately than your human team members are capable of. But those humans are essential to your success because they can spot a rare error or give notes to developers to improve performance. They can also think outside the box to find solutions to issues that no amount of machine learning can equip your artificial intelligence to handle. 

The Right Tool for the Job

In the end, our team of AI developers was able to craft a Topic Modeling tool for Syl Search that works efficiently and produces meaningful results — which you will be able to see when it officially launches later this year. But the amount of dedication and elbow grease involved should not be overlooked.

Teaching a machine how to recognize something as complicated as a topic, let alone teaching it to pull topics from the bodies of documents and data sets, is extremely difficult but not impossible. 

At TEAM, we have decades of combined experience with AI and ML. That experience helped our developers stay patient and create a Topic Modeling tool that will make Syl Search even more effective at searching through your content to find relevant results and drive value.

About TEAM IM

TEAM IM is an experienced solution company that advises, develops, implements, supports, and manages enterprise grade process, information and content management systems. For more than twenty years, TEAM has acted as a trusted advisor to our clients through our offices in Australia, New Zealand, Europe and the United States. Our mission is to assist our client to get the most out of their investments in technology. Whether our clients are large government agencies or corporations, construction firms, accounting firms, heavy industry, or smaller organizations, we strive to deliver demonstrable business benefits and generate real return on investment for our clients.

Our products and services offer solutions to transform your business by automating and modernizing your operations. We work hand-in-hand with our clients to understand their goals and create and execute multi-year, continuous improvement plans. Our mission is to support and manage every solution we deliver, so we take care to design long term, future proof, maintainable solutions. We work with best-in-class technology partners that we have carefully selected to ensure we can execute our plan and achieve our clients continuous improvement goals.

Our products and solutions encompass Advisory Services, Implementation Services and Managed Support Services. We specialize in Business Process Automation and Optimization, Content Platforms and Content Services and we are also a leader in Mobile App/Field Services software development and Digital Workplaces. We have industry-specific solutions for the Construction and Accounting Services sectors, and cross-industry solutions for Accounts Payable, Contract Management, App Modernization, Field Services and File Sharing.

At TEAM IM we are passionate about delivering outstanding outcomes for you, our clients.