A Google search for 'entymology grain'
has this to say:-
Figuratively, "the smallest possible quantity," from late 14c. From early 15c. in English as the smallest unit of weight (originally the weight of a plump, dry grain of wheat or barley from the middle of the ear).
From late 14c as...