What semantic notions underlie 人 (person) and 僅 (solely)?

Question

CUHK and Yellowbridge don't expound the semantic shift. I'm especially addled by the shift in lexical categories too, from a noun to an adverb.

Synopsis  : 從「人」，「堇」聲，用作副詞，表示才、只。

Elaboration: 從「人」，「堇」聲，用作副詞，表示才、只。《說文》：「材能也。从人，堇聲。」《國語‧周語中》：「今天降災於周室，余一人僅能守府。」

　　「僅」也指數量少。《公羊傳‧恆公三年》：「此其曰有年何？僅有年也。彼其曰大有年何?大豐年也。」何休注：「僅，猶劣也，謂五穀多少皆有，不能大成熟。」

Answer 1

Outlier

1 (orig.) ability, talent
2 ○ only

Here the ○ icon is an indicator of a phonetic loan. So, basically it is just a:

character that is "borrowed" to write another homophonous or near-homophonous morpheme