In this example, we will review and highlight the importance of using a validated QSAR model for the accurate prediction of properties (i.e. activity, solubility, clearance) and during molecule generation. While some model metrics will be reviewed, a detailed discussion regarding the interpretation of model scores can be located here: QSAR Model Scores Interpretation.
A critical component of building an accurate QSAR model is defining the target product profile (TPP). For a review of how to select a TPP, please refer to the following link: Creating a New Predictor by Defining a Target Product Profile (TPP).
When defining a TPP, it is necessary to provide a balance of compounds that meet and do not meet the desired threshold. A balance doesn't mean that the number of active and inactive compounds need to be perfectly equal, but of the same order of magnitude. This is denoted by “Molecules In” and “Molecules Out” within the TPP. In addition, note that “11 molecules are currently matching your target product profile”.
Balanced QSAR
Once the QSAR model has been trained, note the performances. For both metrics, pKI IC50 Pi3K and PkI IC50 mTor, we see reasonable results for AUC, Precision, and Recall. These metrics indicate the models ability to correctly distinguish between active and inactive molecules within the dataset.
Now we will intentionally select an imbalanced TPP to produce QSAR models that display poor performances. We can modify the TPP by adjusting the sliders. Note that “No molecule is currently matching your target product profile” and there is a significant difference between “Molecules In” and “Molecules Out” for the metrics of interest.
Imbalanced QSAR
Once the imbalanced QSAR model has been trained, note the performance. While we see what appears to be a reasonable AUC for pKI IC50 Pi3K and PkI IC50 mTor, note the significant reduction in Precision, and Recall. The low scores calculated for precision and recall indicate the model's inability to appropriately predict when a molecule is active.
We can now rescore molecules to compare predictions based on these QSAR models. For a review on scoring individual molecules with the QSAR module, please refer to the following link: Applying the QSAR Predictor on Custom Molecules
The compound above was pulled from the initial training data set and is active on both pKi IC50 Pi3K and pKi IC50 mTor. Once rescored, note the difference in predictions between in pKi IC50 mTor for the well trained QSAR model and the poor model. The imbalanced model does not accurately predict the compound to be active for pKi IC50 mTor.
Balanced QSAR Rescoring
Imbalanced QSAR Rescoring
It is critical to select an appropriately balanced TPP when defining the parameters to build QSAR models. Poorly trained QSAR models that are applied to subsequent generations will add noise to the system, resulting in molecules that no longer match the desired TPP.
Let's review the results from two Fine Tuning generators. One generator was run with balanced QSAR models that perform well, and one run with the imbalanced QSAR models that display poor recall and precision. For a review on the use of the fine tuning generator, please see the following use case: Lead Optimization with the Fine Tuning Generator.
Balanced Fine Tuning Generator
Imbalanced Fine Tuning Generator
Displayed above are the top six scoring compounds from their respective generators using the well trained and poorly trained QSAR models. Note the significant difference between Iktos Ranking scores and the structures of generated molecules. Introduction of the unbalanced QSAR models results in poorly scored molecules due to inefficient exploration of the chemical space and inaccurate predictions of molecular properties.
NOTE: when a Makya QSAR model is selected to guide generation, generated molecules are optimized with respect to the predicted scores, but not directly to the Tanimoto similarity to the training dataset. This can be done by selecting the training dataset (or a subset of it) as a chemical space.