When you have Advanced Queue mode off, this is the expected behavior. No matter in which view you are, if you select a song it will be loaded into the player with the rest of the songs in that list following.
If you are working in Advanced Queue mode, only the selected song is added to the queue. So maybe you might take a look at how Advanced Queue mode works if you like to build your queue in realtime during a performance. See my tutorial on this: https://youtu.be/xL7eINiMrak