Modelling Assistants Under the Lens: What the Current Landscape and Gaps Mean for Digital Health Software Development using Low-Code/No-Code

We conducted a systematic mapping [1] following the guidelines proposed by Petersen et al. [3] to review the state of the art and answer RQ1 and RQ2 (among other questions). Our study collected and analysed data from both the literature and practice as follows:

• To gather data from the literature, three reviewers selected 58 proposals from a set of 3.176 records from 1985 to 2024 using inclusion/exclusion criteria. Having 58 proposals selected with a substantial inclusion agreement [4]—i.e., having a 0.8 > K-statistic > 0.61. We clustered the data to answer the research questions.
• To gather data from the practice, we explored the 17 LCNC tools that were mentioned in the Gartner Magic Quadrant for enterprise low-code/no-code application platforms [2] and extracted quotes about goals, limitations, and evaluation metrics from modelling assistants.

After executing our study, we found that software development teams can benefit from modelling assistants in several key areas, such as software model evolution, vulnerability detection, consistency checking, and change propagation, among others.

However, many modelling assistants come with limitations:

• Accuracy: Recommendations may be inconsistent or irrelevant; automatic extractions may lack precision.
• Effort: Assistants may increase modelling overhead through additional steps or manual interventions.
• Generality: Some are narrowly tailored to specific domains, refactorings, or training datasets.
• Learnability: Users face steep learning curves or must adopt intermediate modelling languages.
• Scope: Key assistant features may be missing, limiting expected benefits.
• Usability: Poor user interaction design can hinder adoption by users.

Evaluation metrics, when provided, tend to focus on:

• Effectiveness: Fault detection rates, accuracy, recall, precision, and accepted suggestions.
• Efficiency: Modelling time, performance, computational cost, and recommendation latency.
• User perception: Perceived usefulness and potential for industrial adoption.

Despite their importance, this data is often missing.

In literature:

• 50% of proposals fail to report any limitations.
• 48.6% do not include any evaluation metrics.

In practice, the gap is wider:

• 75% of modelling assistants lack documented limitations.
• 68.8% provide no evaluation data.

This is especially problematic in domains as digital health software development: the risk of adopting an assistant that generates flawed recommendations/editions could directly affect the resulting digital health software and, in consequence, the patient and practitioner experience with it. The absence of clear information makes an already difficult decision even more uncertain—and riskier. What can we do, as a community, to bridge this gap? (See next section.)

1. For modelling assistant designers/developers/engineers: While technical features are essential, transparency is just as crucial. Report not only what your assistant can do, but also its limitations and how well it performs. Without this, LCNC development teams in quality-critical domains like digital health cannot make informed decisions. Fill the gaps—report fully.
2. For researchers working on modelling assistance: We play a key role in shaping trustworthy modelling assistants. This requires clear methods and frameworks that prevent underreporting and promote transparent designs from modelling assistant developers. Existing initiatives like the Framework for Designing Intelligent Modelling Assistance (FR-IMA) [5] and the Emerging Framework for Modelling Assistance [6] are good starting points. The next steps include proposing tailorable methods that help design and adopt modelling assistance in context.
3. And finally, don’t forget the users: In this knowledge capsule, we discuss goals, limitations, and evaluation metrics. But users are also important. Check our last post titled “It’s Not Just Developers”: Why Modelling Assistance Must Fit All Roles in Low-Code/No-Code Software Development for Digital Health [7]!