Pulmonary diseases such as COVID-19, pneumonia, and tuberculosis continue to be among the world’s leading causes of death. Proper and on-time pulmonary condition classification is critical for early treatment, and this is particularly true considering the extreme heterogeneity of data in medical imaging. Although convolutional neural networks (CNNs), including EfficientNet, are powerful models for performing medical diagnosis, their single-size-fits-all approach of compound scaling is not able to accommodate variations in image complexity. In this work, we introduce Entropy Adaptive Compound Scaling (EACS), a new extension of the EfficientNet framework based on normalized Shannon entropy used as a dynamic per-image scaling cue. EACS adjusts the compound scaling factor on a per-image basis and allows the network to raise depth and width and to increase resolution for complex input instances while retaining computational speed for less complex input instances without adjusting network architecture. EACS is assessed on two imaging modalities: chest X-rays and computed tomography (CT) scans. Our results on experiments reveal that EACS enhances COVID-19 detection accuracy from 88.07% to 90.94% on chest X-rays and elevates average accuracy on CT scans from 88.66% to 91.66%, respectively. These results establish EACS as an effective, lightweight, and generalizable solution to improve diagnostic capabilities in challenging, everyday medical imaging scenarios.