Enabling innovative accurate diagnostics for early bladder cancer detection

Enabling innovative accurate diagnostics for early bladder cancer detection

Key Impact

  • 70 percent

    Time reduced to label cytology slide

  • 3x

    Higher productivity with simpler collaboration

  • Cost efficiency and accuracy of data lebelling

Overview

VitaDX, a French healthcare company focusing on the field of bladder cancer diagnosis is developing a new AI-powered diagnostics model to provide a simple, reliable, non-invasive, and more efficient diagnostics tool to detect early bladder cancer as a complementary for urinary cytology. However, the company found it challenging to industrialize its data pre-processing to build the tool while managing the data accuracy in check – as accuracy is highly critical in medical diagnostics. VitaDX partnered with Kili Technology to enable not only a simplified process, but also a more structured, collaborative workflow.

Aiming for perfection: when accurate diagnosis could save people’s lives

Globally, almost 3 million people are currently diagnosed or treated for bladder cancer, with 400 thousand global new cases each year. Like almost all other cancer types, the survival rate of a bladder cancer patient increases when it is detected earlier. However, urinary cytology – the non-invasive diagnostic method available – has a poor performance on detecting early bladder cancer. As a company, VitaDX has a mission to build a new tool to significantly increase the capability to accurately diagnose early bladder cancer with a non-invasive method, using an image processing algorithm of urinary cytology slides, scanned in white light and in fluorescence.

Using artificial intelligence as the core of its diagnostic tool, the company understood perfectly that accuracy of data to train the model is highly critical. It was already clear that to be able to provide a precise diagnosis of bladder cancer, the AI model needs to be trained with clean, accurately labelled images of cytology slides. Thus, the image labelling process is essential as it would determine the precision level of the diagnosis, which will impact people’s lives.

We build an artificial intelligence model that will determine whether a person is sick. At the end of the day, our model will impact the life of a human being. This is why it needs to be accurate, quality should be the first to prioritize since the beginning – data preparation.

– Abderrahmane Data Project Manager.

The challenge to diagnose cytology slides at scale

Before partnering with Kili, the company relied on crowdsourcing to annotate cytology slides. On the face value, the cost of crowdsourcing seemed lower as all of the labelers worked part-time. The annotation process was also done without a dedicated resource in charge for data operations. Thus, each crowdsourced labeler had his own annotation stream and flow. However, VitaDX realized that when it comes to delivery, it was extra difficult to achieve the level of annotation quality required and also to manage the consistency of this quality. With no one in charge of data operations, monitoring these crowdsourced labelers was a challenging task.

Another challenge is that VitaDX understood that the medical image annotation process is a laborious, highly time-consuming work. Annotating cytology slides involves performing semantic segmentation to the images, and this requires a pixel-level precision. Then, each labeler needs to label carefully with laser-sharp focus, resulting in a long process to label one image. Reviewing the labelled cytology slides was also tiresome, as doctors need to sit together and discuss when there is a disagreement over a label of presumably cancer cells.

“Precision is very important, it determines the quality. The quality of annotation will impact on the quality of our machine learning model, and at the end the quality of the medical device where we are building.” continued Abderrahmane.

Why Kili Technology

Reflecting on these challenges, VitaDX realized that to perform cancer diagnosis at scale it needs to find a solution to structure and streamline the process of annotating cytology slides and the collaboration between doctors, machine learning team, and data labelers. Benchmarking against several top players in the market, the company found Kili offers the combination of features suitable to their needs. More importantly, Kili stood out when it comes to quality management of the annotations.

We found the quality management features – especially consensus – very useful, as it allows us (doctors) to gather up only when an asset has a low level of consensus, which means the cell image is very complex to identify. Otherwise, there is no need to review together, hence it saves us time.

– Abderrahmane Data Project Manager.

Partnering with Kili, VitaDX was able to structure the annotation process much more clearly. One person was required to be pointed as the person in charge of data operations for quality management feature on Kili, hence quality control is kept in check. Reviewing features on Kili also allows a more efficient collaboration for doctors, machine learning team, and data labelers, breaking the SILOs of departments that existed previously.

Moreover, the company found itself to be able to perform image labelling much more precisely at a way faster pace. Using online learning as a machine learning in the loop was enabled during the process, where VitaDX plugged in its machine learning model into Kili to automate the labelling process itself. This process allowed a pre-annotation process, where labelling predictions were produced, reducing the effort of the annotators. Kili was also flexible to use in devices suitable to deliver pixel-precise perfection quality.

“We speed up the process five times.”

Industrializing image annotation of cytology slides to detect bladder cancer enabled VitaDX to perform cancer diagnosis early, more accurately, and at scale. Streamlining the whole labelling process along with online learning as machine learning in the loop reduces the time taken to annotate images by 70%, and it improves the efficiency and the productivity of the team.

The walls between SILOs of department that existed previously, hindering collaboration and transparency of work flow, is also gone – drastically improving productivity and accelerating the diagnosis process. The overall cost to conduct cancer diagnosis is reduced.

If we don’t have high quality data, the process of our diagnostic device could have been much slower. The bonus is that we speed up the process five times

– Abderrahmane Data Project Manager.

Lesson learned

  • Accurately labelled images play a critical role in determining the accuracy of a cancer diagnostic tool, since it involves decisions that impact people’s lives. Hence investing time and effort in data labelling is of the utmost importance to ensure the success of the project.
  • Flexibility to apply machine learning in the loop during image annotation process is essential to enable improved speed for 70% time saving of data labelling
  • – It is important to select the suitable data annotation partner to develop a cancer detection tool. Factors such as robustness, meticulous quality management, and simplicity of collaboration are key.

Photo by Luis Melendez on Unsplash
Cancer Cells Vectors by Vecteezy