Category: Blog

Data Validation through the DataNeuron ALP

Teams in nearly all fields, spend a majority of their time on research and finding chunks of important information from the huge bulk of unfiltered data and documents that is present within the organization. This process is very time consuming and tedious.

In fields like data science and machine learning, getting annotated data is one of the biggest hurdles and one that the teams tend to spend the most time on.

Apart from this, data annotation can often prove to be expensive as well. Multiple human annotators might need to be hired and this can increase the overall cost of the project.

The DataNeuron platform enables organizations to get accurately annotated data, while minimizing the time, effort and cost expenditure.

DataNeuron’s Semi-Supervised Annotation

What does the platform provide?

The user is provided with an option to define a project structure, which is not limited to a flat classification hierarchy but can incorporate a multilevel hierarchical structure as well with indefinite levels of parent-child relationships between nodes.

This aids research, since the data is essentially divided into groups and further sub-groups depending on the user preference and defined structure which enables the team to adopt a “top-down” approach for getting to the desired data.

The platform takes a semi-supervised approach to data annotation in the sense that the user is required to annotate only about 5–10% of the entire data and the platform annotates the remaining data automatically for the user by detecting contextual similarity and patterns in the data.

How the semi-supervised approach works?

Even for the 5–10% of the total data that still needs to be annotated, the time and effort spent is reduced by a large margin by adopting a suggestion-based validation technique.

The platform provides auto-labeling to the users and suggests the paragraphs that are likely to belong to a specific class based on label heuristics and contextual filtering algorithm; users have to accept or reject at the validation stage.

The semi-supervised approach for validation is broken down into stages:

• In the first stage, the user is provided with suggestions based on an intelligent context-based filtering algorithm. The validations done by the user in the first stage are used to improve the accuracy of the filtering algorithm used to provide suggestions for validations.
• In the second stage the validation is then further broken down into ‘batches’. This process is repeated for each batch of the second stage, i.e. the validations done in each batch are used to increase the accuracy of the filtering algorithm for the succeeding batch.

This breaks down the problem of annotating a data point into a “one-vs-all” problem which makes it far easier for the user to arrive at an answer(annotation) than if they had to consider all the classes (which might be a huge number depending on the complexity of the problem) for making each individual annotation.

Our platform is a “No-Code” platform and anyone with basic knowledge of the domain they are working on can use the platform to its maximum potential.

Testing On Various Datasets

The platform chooses from among multiple models trained on the same training data, to provide the best possible results to the users.

The average K-Fold accuracy of the model is presented as the final accuracy of the trained model.

We incur a relatively small drop in accuracy as a result of the decreased size of the training data as highlighted. This dip in accuracy is within 12% and can be controlled by the user by annotating more data, or choosing to add seed paragraphs during the validation or feedback and review stage.

Comparisons with an In-House Project

Difference in Paragraphs Annotated. We observe it is possible to reduce annotation effort by up to 96%.

Difference in Time Required. We observe it is possible to reduce time required for annotation by up to 98%.

Difference in Accuracy.

We observe that the DataNeuron platform can decrease the annotation time up to 98%. This vastly decreases the time and effort spent annotating huge amounts of data, and allows teams to focus more on the task at hand.

Additionally it can also help reduce the Subject Matter Expert effort up to 96%, while incurring a marginal cost. Our platform also helps reduce the overall cost of the project by a significant margin, by nearly eradicating the need for data labeling/annotation teams.

In most cases, the need for appointing an SME is also diminished, as the process of annotation is made much simpler and easier and anyone with knowledge of the domain and the project they are working on can be able to perform the annotations through our platform.

The term Artificial General Intelligence (often abbreviated “AGI”) has no precise definition, but one of the most widely accepted ones is the capacity of an engineered system to display intelligence that is not tied to a highly specific set of tasks or generalize what it has learned, including generalization to contexts qualitatively very different from those it has seen before and take a broad view, interpret its tasks at hand in the context of the world at large and its relation thereto.

In essence, Artificial General Intelligence can be summarized as the ability of an intelligent agent to learn not just to do a highly specialized task but to use the skills it has learned to extract insight from data originating in multiple contexts or domains.

How does DataNeuron achieve Artificial General Intelligence?

The DataNeuron platform displays Artificial General Intelligence as it has the ability to perform well on:

• NLP tasks belonging to multiple domains.
• Text data originating from multiple contexts.

Masterlist: Machine Learning is not binary so we don’t rely on rules or predefined functions, we rely on the simpler structure which is the Masterlist where we allow classes to have overlap. Further, we support taxonomy or hierarchical ontologies on the Masterlist. The platform uses intelligent algorithms to assign paragraphs for each class making the data annotation process automated.

Advanced Masterlist: We are also launching Advanced Masterlist to support subjective labeling of datasets (where clear class distribution is missing).

Apart from the ability to perform auto-annotation on data, the platform also provides complete automation for model training including automatic data processing, feature engineering, model selection, hyperparameter optimization, and cross-validation of results.

The DataNeuron Platform automatically deploys the algorithm and provides APIs which can be integrated to build any application with real-time no-code prediction capabilities. It also provides a continuous feedback and retraining framework for updating the model for achieving the best performance. All these features make it one step closer to achieving Explainable AI.

The DataNeuron platform has produced exceptional results in extremely specialized domains like Document or Text classification in the Tax & Legal, Financial, and Life Sciences use cases, as well as general tasks like Document or Text Clustering in any given context. DataNeuron reduces the time and effort by ~95% required to label and create models, allowing users to extract up to ~99.98% insights. DataNeuron is an Advanced platform for complex data annotations, model training, prediction & lifecycle management. We have achieved a major breakthrough by fully automating data labeling with comparable accuracy to state-of-the-art solutions with just 2% of labeled data when compared to human-in-loop labeling on unseen data.

The impact created by DataNeuron’s General Intelligence

We observe that the DataNeuron platform can decrease the annotation time by up to ~98%. This vastly decreases the time and effort spent annotating huge amounts of data and allows teams to focus more on the task at hand by automating the process of data annotation and easing research.

Additionally, it can also help reduce the SME effort up to 96%, while incurring a fraction of the cost. Our platform also significantly reduces the overall cost of the project, by nearly eradicating the need for data labeling/annotation teams. In some cases, the need for an SME is also diminished as the process of annotation is much simpler and anyone with knowledge of the domain can be able to do it properly unless the project is too complex.

Results Visualized

The above visualizations showcase the platform’s ability to perform extraordinarily in different domains. As opposed to the specialized systems that tend to perform well on only one type of task or domain, the DataNeuron platform breaks boundaries by performing exceptionally for a diversified set of domains.

What does it mean for the Future of AI?

As AI adoption has picked up among enterprises, the need for labeled and structured data has dramatically increased in order to remove the bottleneck in developing the AI solutions.

DataNeuron, powered by a data-centric platform provides a complete end-to-end platform from training to Ensemble Model APIs for faster deployment of AI.

Our research continues to be focused on the area of Artificial General Intelligence and further automation of Data Labeling / Validation and provide better explainability of AI.