RFC3484のDestination Address SelectionのRule 9で選択してるんだろ。
正しい動作。DNSラウンドロビンによる負荷分散は時代遅れ。
Rule 9: Use longest matching prefix.
When DA and DB belong to the same address family (both are IPv6 or
both are IPv4): If CommonPrefixLen(DA, Source(DA)) >
CommonPrefixLen(DB, Source(DB)), then prefer DA. Similarly, if
CommonPrefixLen(DA, Source(DA)) < CommonPrefixLen(DB, Source(DB)),
then prefer DB.

Source(D)は
We write Source(D) to indicate the selected source address for a
destination D. For IPv6 addresses, the previous section specifies
the source address selection algorithm. Source address selection for
IPv4 addresses is not specified in this document.