Independent Evaluation Of Medical Images Is Now Widely Used In Non-Cancer Clinical Trials

Lee Jeong-hyun, CEO of TI Image, an imaging clinical trial agency,

 "FDA allows the use of medical imaging for new drug approval... More and more regulatory agencies are accepting it."

Stay ahead of the industry with exclusive feature stories on the top companies, expert insights and the latest news delivered straight to your inbox. Subscribe today.

[Until the End HIT No. 6] ‘Independent imaging evaluation’, which was used as the primary efficacy endpoint in clinical trials for new anti-cancer drugs, is expanding its scope to non-cancer fields as well.

Independent image evaluation refers to an objective evaluation by a third person who does not participate in the clinical trial using only image data. It is used in various terms such as blind independent central review (BICR), central reading, and central review, so it can be confused with different meanings, but the meaning is the same.

At this time, the clinical trial information must be blinded, and depending on the characteristics of the clinical trial, specific clinical trial information such as the subject's gender, height, and weight may be required, so discussion with an independent imaging evaluation expert may be necessary before clinical design.

In the meantime, in anticancer drug clinical trials, the primary efficacy endpoints are usually △OS (overall survival) evaluated by the investigator △PFS (progression free survival) △EFS (event-free survival) free survival) and △pCR (pathologic response) have been used. Recently, independent image evaluation has been conducted using various image data such as computed tomography (CT), magnetic resonance imaging (MRI), and bone scan.

Recently, the U.S. Food and Drug Administration (FDA) has allowed the use of medical images as data for new drug approvals, and more and more regulatory agencies are accepting this. In addition, there are situations where pharmaceutical companies seeking approval for new drugs are requested to provide data through independent imaging evaluation.

In addition to the anti-cancer field, independent image evaluation using image data is also receiving attention in clinical trials for various disease groups. In clinical trials, the results of independent image evaluations using medical images used in various medical fields such as dermatology, plastic surgery, ophthalmology, and medical devices, as well as general photographs, are used as main endpoints.

met with Lee Jeong-hyeon, CEO of TI Image, an imaging contract research organization (CRO), to hear about the independent image evaluation procedure, its significance, and its necessity. TI Image is a company headquartered in Singapore that performs the operation, management, and evaluation tasks necessary for clinical trial image analysis.

“Independent imaging evaluation, unlike traditional evaluation, requires additional procedures and standardization”

CT and MRI comparison results showing clear 'progression' for the same tumor lesion with different measurement methods (Source: New response evaluation criteria in solid tumors: Revised RECIST guideline (version 1.1), European journal of cancer, first author EA Eisenhauer )

Typically, each efficacy endpoint in a clinical trial is evaluated by researchers, so introducing independent imaging evaluation as a key variable requires additional procedures. Independent imaging evaluation is in high demand in clinical trials in the field of new anti-cancer drugs, where clinical trial information such as treatment plans is designed to be blinded. Usually, the size of the tumor is measured using CT or MRI images, and when measuring the size of the tumor, the evaluation is conducted according to clear standards rather than simply measuring the size.

CEO Jeong-Hyeon Lee said, "When independent image evaluation is introduced as a primary or secondary endpoint, clinical trials must be conducted in accordance with the independent image evaluation procedures and standardization required by regulatory agencies, rather than simple image evaluation." He added, "We need to conduct efficient independent image evaluation." “For this reason, an image management system developed according to the characteristics of medical images must be introduced to ensure the reliability of images and evaluation data,” he explained.

CEO Lee continued, “In clinical trials for new anti-cancer drugs, the standards for evaluation are different depending on the drug’s mechanism or indication, so pharmaceutical companies need to consider evaluation methods from the clinical trial planning stage.” “We are able to derive objective evaluation results by conducting an evaluation using only video based on global evaluation standards appropriate for the company,” he added.

Although global evaluation standards exist for independent imaging evaluation, CEO Lee believes that there is a need to develop appropriate evaluation standards depending on the design and indications of the clinical trial. For example, it has recently been shown that PFS and BOR (Best Overall Response), which are set as primary and secondary efficacy endpoints, are being confirmed through independent image evaluation methods.

“ FDA prepares guidelines starting in 2018...emphasizes the importance of image quality and procedures “

As independent imaging evaluations become more important in clinical trials, guidelines from regulatory agencies are being rapidly updated. Independent imaging evaluations, which were introduced simply to provide objective tumor evaluation results, are now becoming necessary for multiple disease models based on technological advancements, regulatory agency guidelines, and the variability of various clinical studies. Accordingly, there is a lot of interest in the independent video evaluation process, and regulatory agencies are also checking whether the work was carried out fairly and objectively.

Representative Lee said, “In fact, the FDA has already been aware of the importance of imaging in clinical trials since 2011, and issued the final version of the ‘Clinical trial imaging endpoint process’ guideline in 2018.”“This guideline mentions the quality and standard procedures of image evaluation results that pharmaceutical companies use as primary efficacy endpoints.”

He added, “Not only the FDA but also the European Medicines Agency (EMA) emphasizes the importance of independent and unbiased evaluation results.”

FDA 

"A centralized image interpretation process, fully blinded, may greatly enhance the credibility of image assessments and better ensure consistency of image assessments."

- “Fully blinded independent imaging evaluation procedures can significantly improve the reliability of imaging evaluations and better ensure imaging evaluation consistency.”

EMA

“The analysis will be more convincing if prior therapy is chosen among clinically
appropriate options and if progression on both prior and experimental therapy is
independently adjudicated, eg, using blinded independent central review.”

- “The analysis is more persuasive when prior treatment is selected among clinically appropriate options and the progress of prior treatment and experimental therapy is determined independently, such as through blinded independent imaging evaluation.”

However, Korea's Ministry of Food and Drug Safety has not yet prepared this. In fact, independent evaluation is not the only way to evaluate clinical images. Since clinical institutions usually conduct imaging evaluations themselves, many people wonder why a separate, independent imaging evaluation is necessary.

What needs to be considered here is whether the imaging evaluations conducted by various clinical institutions can be derived objectively and consistently. Performing imaging evaluations at each institution where clinical trials are conducted means that evaluations are ultimately conducted by different evaluators, and the 'inter-observer variability' derived from this is expected to be at a level that cannot be ignored. This is Representative Lee’s point.

According to CEO Lee, video evaluation has inherent characteristics of qualitative evaluation, so even if the evaluation standard is the same, individual evaluators' viewpoints or judgment standards may differ. To prevent this, independent image evaluators must complete prior training, including the evaluation criteria and evaluation methods of the relevant clinical trial, before the actual evaluation to minimize variability.

For independent image evaluation, there must be a process such as developing an imaging protocol for standardization of work procedures and imaging devices, developing a clinical trial imaging plan, managing the institution, and establishing an imaging data management system. In addition, in order to improve the completeness of the work, clinical trial regulations and standardization of independent image evaluation procedures must be considered.

CEO Lee emphasized, “The more independent image evaluation that takes into account clinical trial regulations and standardization, the more helpful it is to identify and resolve errors, inconsistencies, or biases,” and “thereby ensuring good quality results.”

In addition, an important aspect is the exposure limit and timing of independent image evaluation results. This is a question that CEO Lee Jeong-hyun frequently hears while working with companies.

CEO Lee said, “In order to derive objective results, which is the purpose of independent video evaluation, the integrity and fairness of the evaluation process must be guaranteed,” and “The reason for not exposing the independent video evaluation results to the company is to avoid bias due to interests and the resulting influence.” “It’s about preventing it,” he explained.

In other words, this means that the objectivity and reliability of the data can be guaranteed as the independent image evaluation progresses without any external pressure or influence.

“Use of independent image evaluation not only in the anti-cancer field but also in the non-cancer field”

While independent imaging evaluation has been in the spotlight in clinical trials for new anticancer drugs over the past few years, there is now a trend to apply it to various indications as well. 

Representative Lee said that the easiest way to check this trend is to look at clinical trial information registered on 'clinicaltrials.gov', a site that provides clinical trial information at the U.S. National Institutes of Health (NIH).

He said, "For example, just as tumor evaluation is evaluated according to the 'RECIST1.1' standard, osteoarthritis clinical trials are also evaluated according to the 'Magnetic Resonance Imaging Osetoarthritis Knee Score (MOAKS)' standard using X-rays and MRI images. “He gave an example.

In addition, “Image evaluation using an endoscope is sometimes performed for diseases such as chronic gastritis, ulcerative colitis, and bowel preparation,” he said. “Images are also used to evaluate the safety and effectiveness of new drugs in ophthalmology, dermatology, plastic surgery, internal medicine, and medical devices.” “I started doing it,” he explained.
 

Glossary of Terms
RECIST 1.1 Guidelines

It is a standard approach to solid tumor measurement and definition to objectively assess changes in tumor size for use in adult and pediatric cancer clinical trials. This criterion applies to all trials in which subject response is the primary study endpoint, as well as trials in which analyzes of stable disease, tumor progression, or time to progression are performed, as all measurements are based on assessment of anatomical tumor burden and its changes. It is explained as being useful.

Magnetic Resonance Imaging Osetoarthritis Knee Score (MOAKS)

It measures the score for each part of the bone marrow, provides a description of each part and score for each part, and is a standard for subdividing cartilage, meniscus shape, and subluxation score elements.

As the indications for which independent image evaluation can be used are becoming more diverse, the type of image, evaluation procedure, standards, and standardization are also required according to the indication.

Prior confirmation is necessary because it may affect the effectiveness evaluation results of future clinical trials. In addition, as the area of ​​independent imaging evaluation appears to be being introduced into the exclusion criteria for selecting clinical trial subjects, interest is also focused on the selection of evaluators. Basically, independent imaging evaluators were often composed of radiologists, but recently, specialists in each indication have been selected as evaluators.

CEO Lee Jeong-hyeon said, "Over the past few years, the number of clinical trials selecting evaluators based on the experience and qualifications of evaluators has been increasing, both domestically and internationally. As we proceed with the work, we often receive inquiries about the number of evaluators when selecting evaluators." He said.

He added, “The appropriateness of the number of evaluators ultimately depends on the clinical trial design and the characteristics of the drug,” adding, “The evaluation system (reading paradigm), such as whether there is one evaluator or multiple evaluators, must also be designed.”

“The requirement for a suitable independent imaging evaluator is ‘experience’.”

So, are foreign evaluators, where independent video evaluation has been introduced as a system for a long time, unconditionally suitable and excellent evaluators? CEO Lee Jeong-hyeon's answer to this question was, "In the end, it's experience." It is recommended to select an evaluator with ‘experience’ in indications and evaluation methods.

Experience is important because clinical trial video evaluation is performed based on clinical trial regulations, design, and evaluation criteria through the evaluator participating in the clinical trial. In addition, the FDA also describes items such as 'participation as a researcher in a clinical trial' and 'professional knowledge required for image evaluation' to determine whether the evaluators are qualified within the guidelines.

CEO Lee Jeong-hyun's conclusion became solidified during a conversation with Kim Gyeong-won, CEO of the parent company 'Trial Informatics', who has extensive experience in independent video evaluation. CEO Kyung-won Kim has a history of performing numerous independent imaging evaluations after completing clinical fellowship at Dana-Farber Cancer Center under Harvard Medical School.

According to CEO Kim's opinion quoted by CEO Lee Jeong-hyeon, in conclusion, there is no difference in independent video evaluation methods domestically and internationally.

He said, "When selecting evaluators, I think there is a need to base the selection on their experience. Just because a foreign evaluator does not mean they are good at evaluating images, existing images, including the indications for clinical trials, evaluation methods, and the evaluator's clinical trials, are “It is recommended that an appropriate evaluator be selected based on evaluation experience,” he suggested.

He continued, “It is presumed that the reason why the sponsor has preferred foreign independent evaluators is because the history of independent imaging evaluation in Korea is relatively short, so it has been difficult to appoint an independent evaluator with abundant clinical trial experience in Korea.” He added, “There have been clinical trials conducted in Korea over the past few years.” “The number of evaluators who have accumulated independent evaluation experience is increasing, and domestic radiology is currently at the highest level in the world, so I think there is no reason not to prefer domestic evaluators,” he added.

“‘Video quality control’ and ‘video management system’ services for independent video evaluation”

As third parties who are not directly involved in the clinical trial, evaluators do not simply evaluate and interpret the images or resulting data. They require evaluation standards, image quality, and image management systems for independent image evaluation work, and the expert group that can help in this case is an imaging CRO such as TI Image. Let’s take a detailed look at independent video evaluation using TI Image’s service examples. 

1) Video quality control

There are procedures that must be followed before conducting an independent imaging evaluation. This is quality control of the video to be evaluated.

CEO Jeong-Hyeon Lee said, "As fewer people know the importance of image quality control than expected, there is a tendency to not prefer quality control when conducting independent image evaluations in actual clinical trials, but this is a wrong idea." He added, "The importance of image quality control is largely “These include ① consistency and standardization of video, ② regulatory agency requirements, and ③ prevention of video errors,” he said.

First, because independent video evaluation is conducted using only images, consistency and standardization of images are important. Because consistency and standardization of images can minimize the bias of evaluators or variations in evaluation results, quality control of all images taken by institution and subject is essential.

CEO Lee said, "Consistency and standardization of images are important in all clinical trials, but in clinical trials for specific indications, consistency and standardization must ensure that images are taken identical to those set before the start of the clinical trial." “This is because differences can have a significant impact on the actual evaluation,” he explained.

These problems can be solved through video quality control procedures. Errors such as video shooting method and video type suitability can be checked in a timely manner, making it possible to suggest solutions such as reshooting the video. If the results of an independent image evaluation do not undergo image quality control procedures, the quality may not be guaranteed.

Additionally, in the case of a global clinical trial, there may be differences in image quality as the images are taken at institutions in various countries. In order to prevent these problems in advance, it is necessary to have quality control performed by experts. The video expert here refers to a qualified person, meaning that they must shoot the video.

2) Image management system (ImageTrial)

When conducting independent video evaluation work, an end-to-end system is needed to manage video and evaluation results from beginning to end. This system, which collects, stores, and evaluates images of clinical trial subjects from domestic and foreign institutions, is called the ‘Clinical Trial Imaging Management System (CTIMS).’

CTIMS must be able to manage medical images in various formats such as DICOM (Digital Imaging and Communications in Medicine) and JPEG (Joint Photographic Experts Group) rather than plain text data, and the anonymization function required when collecting images is very important.

CEO Lee Jeong-hyun said, "In the past, these systems were developed by storing images in a web-based electronic case record (eCRF) and attaching a search function to it, but it was inconvenient due to poor user convenience and low data management efficiency." “As ‘Web-PACS’-based systems are developed, global imaging CRO companies are replacing existing eCRF-based systems,” he said.

The 'ImageTrial' system used by TI Image is a 100% web-based system developed by Kim Gyeong-won, CEO of Trial Informatics, based on many years of independent image evaluation experience and with IT experts with over 10 years of experience in the medical imaging field. It is a non-installation Web-PACS based system. Through this, it is possible to collect images from each institution to transmit the final evaluation data, and anonymization is also carried out automatically, so even if the institution uploads a non-anonymized video, the subject's information is unknown.

Representative Lee said, "Until such a system was developed, domestic and foreign institutions stored images on USB or CD and mailed them to independent image evaluation staff and evaluators. However, as regulatory agencies have recently no longer preferred this, they were stored on the web rather than external storage media. “Based on CTIMS, development has begun,” he said.

He continued, “The image trial system allows video upload and evaluation at the same time, and is designed to be user-friendly so that independent video evaluators can use it intuitively.” He added, “It was also developed considering the convenience of agency officials who must upload the subject’s video every time.” He added. Representative Lee explains that this is a measure that takes into account the characteristics of clinical trials in which various stakeholders are involved.

In addition, "FDA and EMA recommend that clinical trial data be submitted in the 'Clinical Data Interchange Standards Consortium (CDISC)' data format for standardization , and the Image Trial System complies with CDISC," he said. "Like this, the image management system is a simple medical system. “We need a system that considers recent trends, guidelines, and user convenience, rather than just video collection or management,” he emphasized.

He added, "Recently, personal information protection laws and medical data laws are becoming stricter by country, so the fact that anonymization can be performed in the system itself can be considered an important factor in selecting a system."

Meanwhile, these IT systems must comply with clinical trial regulations, including FDA 21 Part 11, and country-specific medical data laws, and their eligibility must be verified by a third party. Image Trial received a certificate from Zigzag Associate, an international QA (Quality Assurance) company, by meeting audit and computer system validation standards.