ML is a backbone in the field of artificial intelligence (AI). Just as adults make everyday decisions based on learning and experience, ML models need training and validation data to learn. Yet, we must follow proven principles and adhere to best practices.
Why do I need training data?
Initially, a new neural network might not know the difference between images of dogs and cats or be able to distinguish between a local bus and a luxury coach. However, after enough training, the machine will accurately analyze and sense the difference. In AI, this is the purpose of training data.
Training is essential, whatever the AI project and however straightforward the algorithm is. In the automation of complex or difficult tasks and the development of advanced neural networks, dataset size and quality come increasingly under the spotlight. Below, we offer guidelines.
Data is inevitable in learning
One of our innate characteristics is that previous learning and experiences shape our future perceptions. Over the years, memorized information and cognitive processes mean that usually, humans appear to make seamless decisions.
In the same way that children learn labels such as 'alphabet' and elements such as A–B–C–D quite intuitively, artificial neural networks need to understand the input they receive. In AI, we use data labeling to add detail to situations, instances, and events.
What types of data are necessary?
AI training data is clean, curated information. Be it to recognize individuals, detect vital signs, or differentiate between laughter, crying, and calls for help, training is the first stage of module development.
Using different types of input reduces bias and improves recognition performance. For instance, if scanning restaurant receipts, we use copies from multiple establishments to reflect varying layouts and styles. In linguistic annotation, dialects and accents are significant factors.
Similarly, we need to source a broad range of diverse samples when processing video – from medical diagnostics and exercise performance to retail footfall and traffic flow. Similar advice applies to the analysis of images, audio and geographic information from 3D sensors.
Notably, training data is different from big data. Whereas the former uses input data and algorithms to predict future results, the latter relates to extracting current information from thousands of records.
Validation and test datasets
Apart from training datasets, AI development also involves validation and test sets. Validation uses different samples, i.e. previously unseen by the model. At this stage, it is possible to tune and control the model using hyperparameters (more below on this).
The formal distinction between the training and validation functions may blur in smaller or relatively simple projects. Each dataset can play an additional part in developing and selecting optional features of the new ML model. The final result or fit is an accumulation of all the inputs.
The third type is test data, a separate sample for a final, unbiased evaluation. Although the input types are similar to those used in training and validation, the values are different. For more detail on the differences between training and testing data, please go here.
How much training data do I need?
The volume of data required depends on the method, input technique, and error tolerance, i.e. the proportion of negligible or acceptable mistakes in the output for the field or domain. It also depends on the complexity of the model concerned.
From controlling machinery to suggesting titles in video subscription services, AI applications require adequate training. As an analogy, visualize an orchestra rehearsing for an important performance. The more practice sessions the musicians complete, the better they will perform the musical piece, scale, or song.
In the same way, machines and systems built on AI need enough training to produce reliable responses. Training forms the most significant part of a development dataset, typically comprising 70 to 80 percent of all the input to train, validate and test a model.
Using multiple training cycles improves the accuracy of an algorithm. However, unlike validation and testing, training datasets often feature an even distribution of classes. This uniformity might not represent real-world situations in some fields or applications.
The accuracy of the final model depends on the quality and diversity of the data. While we need enough, we must avoid having too much: overfitting leads to biased responses.
If not enough data is available, the ML model might be unreliable. Likewise, if inputs are low in quality, the output will be inaccurate and, at worst, dangerous.
Within limits, the more training input used, the better the results. Let's consider a project to produce autonomous vehicles. The control systems must sense road conditions and drive safely by interpreting events and responding correctly – whether in everyday situations or unusual circumstances.
Apart from vehicle recognition and lane markings, the machine should interpret road hazard signs and adjust driving techniques in difficult conditions. Additionally, it ought to anticipate the behavior of humans near pedestrian and pelican crossings.
What is a human in the loop?
Human intelligence is instrumental in improving neural networks in AI. Humans in the loop refer to the judgment of specialists who work to gather data, select reliable attributes, and label the expected outcome.
With this input from humans, semi-supervised learning also involves the adjustment of hyperparameters to configure and control the learning performance of the models.
Returning to the previous example, driverless car systems need to adjust to instructions spoken in different ways and learn to recognize owners' voices. Their control modules need to understand and connect words, interpret the overall meaning, and follow commands. Typical instances are controlling audio systems, finding garages, or searching for recharging points.
In the above case, human interactions provide actual development inputs. Likewise, other AI systems identify individuals, interpret facial expressions and handle various languages or dialects.
How do you improve data standards?
High-quality ML models require training inputs that are uniform, comprehensive, consistent, and relevant. Experts usually consider formatted information from one source as high quality.
To be comprehensive and relevant, training should cover all the possible scenarios within the specification of a new system. Consistency refers to correct formatting and freedom from structural errors. Finally, diverse data means different types: computer vision image and video files and text inputs, PDF files, and voice recordings.
In short, a high-quality dataset has undergone rigorous cleaning and contains all the necessary values for a model to learn its task(s).
What affects training data quality?
High standards ensure optimum interaction with humans, be they consumers, site visitors, hotel guests, students, or patients. However, free data or web scraping from multiple sources can compromise quality.
In the example of driverless cars, AI sensors should discern between surrounding traffic traveling in the same direction versus vehicles approaching from the opposite direction. Are they in the other lane, i.e. offset to one side – or on a collision course? Similarly, the cameras must pick up on various road elements. Lorries, public transport vehicles, motorcycles, cycles, pedestrians, and stray animals form an important dataset.
What is labeled data?
Annotated or labeled data includes a field in each record that shows the target, i.e. the outcome that the model should predict. Labeling is sometimes known as data tagging or transcription.
If we train a machine learning model to sort incoming emails received from customers and forward them to the appropriate department, we might analyze the sentiment. For instance, we could search for 'problem' or 'issue' to detect complaints. Along with other characteristics that the software platform might help identify during labeling and validation, we could train the machine to predict which emails to escalate to a service team.
Each label has a score or weighting. How the algorithm manages borderline combinations will influence the model's accuracy. Thus, in ML model training, labeling expertise could be crucial.
Where do you source AI training data?
The three main ways to obtain training data for ML projects are to:
Use free options such as open datasets, forums and search engines.
Repurpose internal data.
Outsource training data services.
Below, we examine each of the above, along with data scraping.
While there are an increasing number of open data resources available online, zero or low-cost options may not suffice. Searches like Google are likely to find data sets intended for private or non-commercial use.
Often, cleansing or labeling is necessary. Experts estimate that data scientists spend around four-fifths of their time working on data preparation and enrichment. They are already busy enough!
Medium to larger businesses with comprehensive data archives may find they can repurpose some material. Imagine a company developing chatbots to handle online inquiries. Their customer service department might have stored chat logs or email threads, useful to train the model.
The information should still be relevant and represent current situations. Otherwise, results could be inaccurate or outdated, like students attending a course with obsolete textbooks.
A third option is to outsource data collection or annotation to a specialist company. For instance, if you were building a voice recognition model and needed samples of a hundred voices (or more), you could contract a company to record the latter.
This approach has several advantages. First, you set the guidelines. Then the data collection company staff handle the project management tasks, from locating and training contributors to checking the data. They also provide a platform to label the data. Significantly, you do not have the initial workload of setting up the project infrastructure.
Companies that specialize in data labeling also provide specialist knowledge and tools. One example is dimensionality reduction, an advantageous technique to reduce the number of input variables, simplify classification and improve performance.
Extracting information from web pages is relatively straightforward, but European guidelines require scrapers to obtain consent. In 2020, the French data protection authority CNIL ruled that publicly available data is still personal; its reuse requires the owner's permission.
To comply with GDPR legislation, individuals and companies that carry out web scraping from public websites must obtain unequivocal consent. Furthermore, the disclosure must be detailed yet clear.
CNIL also directed that providing or refusing consent ought to be equally simple. There is a minimum period before making repeated requests and the owner can withdraw consent at any time.
Open datasets: to use or not to use?
Most open datasets consist of information available publicly through government sites or social media. However, using open-source data for commercial purposes may infringe copyright, so checking is advisable.
When we factor in the cost of the working hours to locate and process or label open data, the price can be unexpectedly high. On occasions, its relevance and accuracy may be questionable, too.
Training data examples
Communication projects may involve categorizing text inputs from email sentences, short messages, speech-to-text conversions, or other documents. The target might be to determine the sentence's topic, such as a customer inquiry or complaint. Similarly, we classify the main subject in articles or more lengthy texts and then tag other content.
To train a model to detect spam, labels would state whether the text in the sample email and text messages is spam or not.
Finally, labels identify objects within the pictures or photographs in image recognition and definition.
More and more, we are likely to see more algorithms powering routine transactions. As a result, getting training data quantity and quality right will be increasingly important.
Kili is the complete solution to train your AI faster and ensure project success. With this innovative platform, you can better manage training thanks to rapid annotation, straightforward collaboration, simple quality control, and intuitive data management.
In conclusion, Kili enables you and your company to make the most of machine learning by bridging the gap between businesses and ML experts. To arrange a demonstration, please go here.