Triplet Interaction Improves Graph Transformers: Accurate Molecular Graph Learning with Triplet Graph Transformers
Graph transformers typically lack direct pair-to-pair communication, instead forcing neighboring pairs to exchange information via a common node. We propose the Triplet Graph Transformer (TGT) that enables direct communication between two neighboring pairs in a graph via novel triplet attention and aggregation mechanisms. TGT is applied to molecular property prediction by first predicting interatomic distances from 2D graphs and then using these distances for downstream tasks. A novel three-stage training procedure and stochastic inference further improve training efficiency and model performance. Our model achieves new state-of-the-art (SOTA) results on open challenge benchmarks PCQM4Mv2 and OC20 IS2RE. We also obtain SOTA results on QM9, MOLPCBA, and LIT-PCBA molecular property prediction benchmarks via transfer learning. We also demonstrate the generality of TGT with SOTA results on the traveling salesman problem (TSP).
PDF AbstractTask | Dataset | Model | Metric Name | Metric Value | Global Rank | Uses Extra Training Data |
Benchmark |
---|---|---|---|---|---|---|---|
Drug Discovery | LIT-PCBA(ALDH1) | EGT+TGT-At-DP | AUC | 0.806 | # 1 | ||
Drug Discovery | LIT-PCBA(KAT2A) | EGT+TGT-At-DP | AUC | 0.746 | # 1 | ||
Drug Discovery | LIT-PCBA(MAPK1) | EGT+TGT-At-DP | AUC | 0.743 | # 1 | ||
Initial Structure to Relaxed Energy (IS2RE), Direct | OC20 | TGT-At (Mean Aggregation) | Validation Mean Energy MAE | 0.4022 | # 1 | ||
Test Mean Energy MAE | 0.4131 | # 1 | |||||
Test Mean EWT (%) | 8.00 | # 3 | |||||
Initial Structure to Relaxed Energy (IS2RE), Direct | OC20 | TGT-At (Median Aggregation) | Validation Mean Energy MAE | 0.4030 | # 2 | ||
Test Mean Energy MAE | 0.4147 | # 3 | |||||
Test Mean EWT (%) | 8.30 | # 1 | |||||
Graph Property Prediction | ogbg-molpcba | TGT-Ag+TGT-At-DP | Test AP | 0.3167 ± 0.0031 | # 3 | ||
Number of params | 47000000 | # 4 | |||||
Ext. data | Yes | # 1 | |||||
Graph Regression | PCQM4Mv2-LSC | TGT-At | Validation MAE | 0.0671 | # 1 | ||
Test MAE | 0.0683 | # 1 | |||||
Link Prediction | TSP/HCP Benchmark set | TGT-Agx4 | F1 | 0.871 | # 1 |