Comparison of the Diagnostic Accuracy of Semiautomated and Expert-Based Calculation of the Vertebral Heart Score in Cats and Dogs: First Stage Validation of the “Desktop Application for Automation of the VHS Calculation Based on Radiographs”

Introduction. Cardiac diseases are among the main causes of death in older cats and dogs [1]. Although echocardiography is the primary imaging method for assessing the morphological and functional status of the heart, diagnosing by means of radiography also plays a vital role in identifying pathologies. Since cardiac diseases are often associated with the enlargement of cardiac silhouette on radiographs, this inexpensive and widely accessible diagnostic method can be used primarily as a screening method in patients with ambiguous clinical symptoms (cough, fatigue, dyspnea, etc.) [2].

To estimate the size of the heart on radiographs reliably, Buchanan J.W. and Bücheler J. developed a method for determining the size of the cardiac silhouette correlated with the size of the vertebral bodies, known as the Vertebral Heart Score (VHS) [3]. In the original study, published in 1995, the long axis of the heart was measured from the ventral border of the left main bronchus to the most distal point of the apex of the heart. The short axis was measured along a line perpendicular to the long axis, at the widest part of the cardiac silhouette. The lengths of the two axes were compared to the length of the vertebrae, starting from the cranial edge of T4.

An alternative method described by Poad M.H. et al [4], suggests using the ventral edge of the vena cava as a reference point for measurement along short-axis, however, to our knowledge, there are no studies evaluating the effect of this modification on the quality of VHS calculation.

The main benefit of VHS calculation is providing a general practice veterinarian the objective measurements for identifying cardiomegaly. This method is currently recommended by the American College of Veterinary Internal Medicine (ACVIM) for classifying dogs with mitral valve disease [5]. Calculation of VHS is also useful for monitoring patients’ heart size over time for screening purposes, particularly among breeds susceptible to acquired heart diseases [6]. Furthermore, modern veterinary radiographic software includes specialized tools for VHS calculation on digital radiographs, making the method more convenient to use.

Advances in diagnostic imaging have provided access to powerful computer aided (CAD) analysis systems that have recently emerged in veterinary medicine. This means that CAD-based systems can be used for identifying key points and predicting their coordinates on the images. This, for example, can be used for determining the severity of osteoarthritis using radiographs of the coxofemoral joints and making a further prognosis [7]. Initially, significant results were shown in key point detection in facial recognition applications, but this technology is now being applied to medical imaging as it allows automated identification of key points on the image based on detection of anatomical landmarks [8].

Previous machine learning methods enabled the estimation of these indices. Nowadays, deep learning and convolutional neural networks (CNNs) are capable of surpassing human capacities. For example, deep learning algorithms were developed to calculate the cardiothoracic ratio (CTR) on human chest X-rays and were found to be more reliable, efficient, and less labor-consuming than manual calculations [9].

VHS widespreading into every-day clinical practices is hindered by the variability and subjectivity of manual measurements. Studies show that interobserver differences in VHS values can reach 0.8–1.2 vertebrae, equivalent to a transition from “norm” to “pathology” [10][11]. Even with strict adherence to the Buchanan & Bücheler method (detecting the landmarks: ventral border of the left bronchus/ tracheobronchial angle, apex of the heart, maximum width perpendicular to the long axis), the accuracy of measurement depends on the operator’s experience, physical condition (degree of fatigue), and cognitive biases.

In this regard, automating the calculation of the VHS using artificial intelligence (AI) algorithms, such as convolutional neural networks, is becoming particularly relevant. Modern AI-based systems are capable not only of detecting anatomical landmarks but also of minimizing human error, ensuring standardized, reproducible measurements [7]. The implementation of such tools into veterinary practice will make it possible to:

— increase the sensitivity and specificity of early diagnostics of cardiomegaly;

— ensure objective monitoring of the dynamics of the disease during repeated examinations;

— expand access to quality cardiological assessment in general veterinary practice settings where a cardiologist is not available.

Thus, the development and validation of algorithms for automatic calculation of VHS in small domestic animals is a vital scientific and practical objective aimed at improving the quality of cardiac pathology diagnostics, improving prognostication, and optimising veterinary medicine resources. To achieve the above objective, the authors have recently developed a “Desktop application for automation of the vertebral heart score (VHS) calculation based on radiographs” [12], trained to automatically and independently establish key points for VHS calculation in cats and dogs. The aim of the study is conducting pilot validation of the developed algorithm and comparing the efficiency and diagnostic accuracy of two VHS calculation methods (semi-automated and expert-based) in cats and dogs.

Materials and Methods. The right lateral chest radiographs of dogs and cats obtained using DR digital panels (wikiVET, China) at Kaluga and Moscow veterinary clinics were taken for the study, conducted in the period from November to December 2025. After anonymization, the images were converted to JPEG format. The exclusion criteria were the significant chest rotation, pleural effusion, neoplasms, and other artifacts that distort the cardiac silhouette.

A balanced group was formed by simple random sampling: 30 images of dogs and 30 of cats (n=60 in total). The sample of animals was determined based on: 1) recommendations for the minimum sample size for calculating the intraclass correlation coefficient (ICC), when upon the expected value of ICC >0.9 and a confidence interval of 95%, a sample of 30–50 observations proves to have sufficient statistical power to assess agreement [8]; 2) the practices described in the similar studies on validation of AI algorithms in veterinary radiology, where samples of 25–40 images per group were recognized as representative for the primary assessment of diagnostic accuracy [7][9]; 3) the principle of group balance to minimize bias error when comparing the methods. The formal a priori power analysis was not performed due to the lack of preliminary data on the variability of measurements obtained by this particular algorithm, which is a standard practice for first-stage validation studies of new AI tools.

The assessment was performed based on the original Buchanan & Bücheler method [3]: the long axis of the heart was measured from the ventral border of the main left bronchus to the apex of the heart, the short axis was drawn perpendicular to it in the widest part of the silhouette (Fig. 1). The calculations were made by two independent veterinary radiologists (having 4–5 years of experience in visual diagnostics, >8 years of experience in general clinical practice).

Such an approach adheres to the practices of primary AI algorithms validation common for the veterinary imaging that focus on estimating the bias and reproducibility rather than on multicenter consensus. To level out individual subjectivity, the arithmetic mean of the results obtained by both specialists was taken as the reference value, which provides a stable basis for comparing with the automated algorithm.

Automated calculation of VHS was performed using the authors’ proprietary desktop application based on a modified YOLOv8-seg model architecture, which enables cardiac silhouette segmentation, detection of vertebrae, and identification of key points. The model was trained using radiographs pre-labelled by the authors. The program interface provides semi-automated calculation with the possibility to manually adjust the axes to increase the accuracy (Fig. 2).

Statistical analysis was performed using MedCalc v.20 (MedCalc Software Ltd., Belgium). Prior to the main analysis, statistical assumptions for the use of parametric methods were verified. The normality of distribution of quantitative variables (long and short axis of the heart, VHS) was assessed using the Shapiro-Wilk test. The homogeneity of variances between measurement groups (two experts and the AI algorithm) was verified using the Brown-Forsythe test, which is robust to deviations from normality. Since no violations of assumptions were detected (p>0.05 for all tests), the use of the intraclass correlation coefficient with the absolute agreement model was considered methodologically justified.

To assess the agreement between the experts’ and AI algorithm results, the intraclass correlation coefficient (ICC, absolute agreement model) was calculated with 95% confidence intervals based on the F-distribution, which allows assessing the stability of the obtained agreement indicators. According to the generally accepted classification [8], ICC values <0.5 were interpreted as low, 0.5–0.75 as moderate, 0.75–0.9 as good, and >0.9 as excellent.

Additionally, Bland-Altman plots were constructed to visualize and estimate the bias between measurement methods (expert-based vs. the AI algorithm-based). In these plots, the mean value of the two compared measurements was plotted on the x-axis, and the difference between them (bias) was plotted on the y-axis. The mean bias and limits of agreement (LoA) were calculated as the mean ±1.96 standard deviations of the differences. A bias of no more than 5% of the mean value, as well as the absence of a systematic dependence of the difference from the measured value, were considered clinically acceptable.

To assess the productive capacity of the trained YOLOv8-seg model on a hold-out sample, the standard detection and segmentation metrics were calculated. For the task of detecting anatomical landmarks (borders of the heart, vertebral bodies), the following parameters were used: precision (Precision), completeness (Recall), F1-measure and mean accuracy with a threshold of IoU=0.5 (mAP@50). To assess the quality of segmentation of the cardiac silhouette, the following parameters were used: the Dice similarity coefficient (DSC) and the mAP@50 metric for masks. All metrics were calculated with a model confidence threshold of 0.5. Additionally, the mean inference time per image was recorded.

Research Results. Statistical analysis has revealed that the values of long and short axes, and VHS calculation values obtained using the developed AI-based algorithm and that received by two independent veterinarians specializing in visual diagnostics demonstrated a high degree of agreement both in the dog group (n=30) and the cat group (n=30). Statistical assumptions were verified before calculating the intraclass correlation coefficient (ICC). The normality of distribution of quantitative variables was assessed using the Shapiro-Wilk test, and the homogeneity of variances between measurement methods (two experts and the AI algorithm) — using the Brown-Forsythe test. As shown in Table 1, for all analysed parameters (long axis, short axis, and VHS) in both animal groups, the p values exceeded the significance threshold of 0.05, which doesn’t give the grounds for rejecting the null hypothesis of normal distribution and equality of variances. Thus, the use of parametric ICC with the absolute agreement model is methodologically justified.

For all tests, the significance level was α=0.05. P values >0.05 indicate the absense of reason to reject the null hypothesis (normal distribution/homogeneity of variances).

According to the generally accepted interpretation scale, ICC values ≥0.90 are classified as excellent agreement [8], which was the case for all measured parameters (Table 2).

The data in Table 2 demonstrate excellent ICC scores between calculation of key VHS parameters made by two veterinarians specialising in visual diagnostics and the desktop application. Furthermore, the calculation performed using the application takes less time: on average, a veterinarian familiar with the VHS method requires 3 to 5 minutes to calculate the value. The use of application reduces twice the examination time.

To further verify the absence of clinically significant bias, Bland-Altman plots were constructed to analyse the difference (bias) between the AI measurements and the mean value obtained by two veterinarians. The mean bias for VHS ranged from -0.34 to +0.15 vertebrae in dogs and from -0.17 to +0.23 vertebrae in cats, which corresponded to acceptable values. The limits of agreement intervals on the Bland-Altman plot were considered acceptable if the standard deviation of the bias was within 5% of the mean value (Fig. 3).

To assess the productive capacity of the developed algorithm, standard detection and segmentation quality metrics were analysed on the test sample. As shown in Table 3, the model demonstrated high performance indicators across all the parameters: the F1-measure exceeded 0.93 for both animal groups, and the Dice similarity coefficient for cardiac silhouette segmentation was 0.92–0.94, indicating precise detection of anatomical structures. The mean processing time for one image was ~1.4 seconds, consistent with the claimed reduction in time compared to manual calculations.

Study limitations and directions for further stages of validation. The current study has several limitations. Firstly, it was performed on a relatively small sample of 60 animal chest radiographs. This sample size is sufficient for a pilot assessment of ICC agreement and the detection of gross biases of the algorithm, but may be insufficient for evaluating its performance in narrow subgroups (e.g., specific breeds, severe cardiomegaly, or image artifacts). The representativeness of the general patients’ population sample requires confirmation in further multicenter consensus studies. For further validation stages, it is planned to calculate an a priori minimum sample size based on the variances obtained in this pilot study, which will ensure the required power (≥80%) to detect clinically significant differences.

Secondly, only two specialists were involved in the development of reference measurements, although high interobserver variability of VHS is well known. This choice was induced by the objectives of the pilot validation phase: the aim of the study was an initial assessment of the agreement between the algorithm and clinical practice results, rather than a large-scale interlaboratory comparison. The high variability of manual measurements was compensated for by using the intraclass correlation coefficient with the absolute agreement model and Bland-Altman plot analysis that allow for objective identification of discrepancies and confirmation of their clinical insignificance. For further external validation, it is planned to involve a multidisciplinary group of specialists from independent clinics, including veterinarians with varying levels of experience. This will enable assessment of the algorithm’s robustness in the real-life practice, where a significant factor of subjectivity is present.

Thirdly, the developed algorithm was validated on images without significant artifacts or anatomical anomalies. In real-life clinical settings, the presence of pleural effusion, significant chest rotation, or spondylosis may affect the accuracy of detecting the cardiac and vertebral silhouettes. Similar limitations are described in the studies on veterinary AI models validation, which note algorithm performance degradation in heterogeneous samples [13][14][15]. To overcome this problem, further studies are needed in multicenter cohorts including patients with varying degrees of cardiomegaly, comorbidities, and radiographs of varying quality. It would be interesting to assess the behavior of AI in such cases.

Finally, fourthly, the current study did not evaluate the diagnostic sensitivity and specificity of the algorithm in predicting specific cardiac diseases. Like traditional VHS calculation, the AI tool is designed solely for objective measurement of cardiac silhouette dimensions and not for making an etiological diagnosis. In compliance with the current recommendations for reporting in the field of medical AI (e.g., CLAIM guidelines [16]), the next step should be prospective clinical validation with assessment of the impact of the automated VHS calculation on medical decision-making and patient treatment outcomes.

Discussion and Conclusion. The results obtained in this pilot study demonstrate excellent agreement (ICC≥0.97) between the VHS calculations based on the YOLOv8-seg model and measurements of two veterinary radiologists. These data are consistent with the current trends for implementing deep learning algorithms in veterinary diagnostic imaging, where AI systems are successfully used for quantitative analysis of radiographs and minimization of subjective factors [7][10][13][14].

High interobserver variability in manual VHS calculation remains one of the main problems of this method. As demonstrated in [11], the difference between the results of calculation made by experienced specialists can equal to the size of more than one vertebra, which can lead to misdiagnosing the patient’s condition. Similar limitations have been noted in other studies referring to quantitative radiometric methods, where the subjectivity in selecting the anatomical landmarks directly affects the reproducibility of the results [1][4]. The implementation of computer vision algorithms makes it possible to standardize the measurement process to identify the key points with high precision and exclude the operator cognitive biases [7][14].

Thus, this desktop application can be an excellent aid in diagnosing patients’ cardiac status, and can be used even by general practice veterinarians lacking the knowledge and experience in VHS calculation. Furthermore, the reduction of VHS calculation time from 3–5 to ~1.5 minutes demonstrated in our study has direct clinical significance. In the context of the high workload of the general practice veterinarians, the automation of the every-day measurements facilitates quicker diagnostic decision-making and optimises the workflow [6][7]. Furthermore, standardized AI assessments create a reliable foundation for patient monitoring in dynamics, which is particularly important when following-up the progression of mitral valve disease, where even small changes in the cardiac silhouette have prognostic value [5][6].