It’s not much different than if you were to go to a business that buys and sells gold. They’ll have the current market rate on display and for video games eBay is a decent place to look for the market rate. With retro games there may at least be a bit of lag if they have prices directly on the games and not do something shady like looking up current prices at checkout.
What you said is definitely a problem though. The person selling a game should look at recent sold copies over the past 6ish months and consider the physical quality of the game in question to determine a price better; I don’t think a lot of shops put much time or thought into this aspect of it.
I don’t really know how you could do it any other way when retro games are a finite resource that isn’t being produced any more.
A lot of that 30% goes to things other digital stores aren’t even close to offering. Also, if your game does well enough the cut Steam takes is lowered, albeit this only really happens for very successful games.