A new research poster finds that abbreviating common fluorinated groups in a chemical language model's vocabulary changes which molecules the model goes on to propose.
A research poster from the School of Emergent Priorities on what happens to molecular design when a chemical group is given a word of its own
Give a thing a shorter name and it tends to be said more often. A new research poster from the School of Emergent Priorities reports that the same holds for software that writes molecules.
Eight Tokens for One Group is by Ms Sunniva Balogun, Doctoral Candidate, with Ms Ylva Onyekwere, Doctoral Candidate, and Mr Eskil Nwabueze, Doctoral Candidate in the School of Continuous Improvement. Chemical structures are routinely handed to a model as a line of text, and a model reads that line in pieces; a common fluorine-bearing group can cost eight of them. The team trained two otherwise identical models on the same 1.4 million structures, giving one of them a short abbreviation for six such groups, and then compared what each proposed when asked for something new.
The abbreviation did what it was meant to, trimming the strings by about an eighth. It also moved the chemistry. Groups with an abbreviation turned up in the model’s suggestions more often than they appear in the material it learned from, and groups left spelled out turned up less.
We set out to make the strings shorter and found that we had also changed what the model was willing to suggest. The University has been patient with findings of that shape.
— Ms Sunniva Balogun, Doctoral Candidate
The poster reflects a growing confidence at the University in work that sits between two fields and settles in neither. “A vocabulary looks like a housekeeping decision right up until you look at what it produced,” said Mr Nwabueze, who led the evaluation.
The full poster is available from the University’s research repository under an open licence, doi:10.5555/slop.gz6viw.