All rights are only between an individual and their government

Different rights have different scopes. Some rights, like the right to live, are between everyone. Some others, like the right to receive a purchased product, are only between buyer and seller. Freedom of speech "just happens" to be between individual and government, in countries where it's even a right.