Believe it or not, I actually created an alternative to Houzeo called Falaya. When researching competitors before starting I noticed many other people like yourself dealing with the same issues you've mentioned here.
At Falaya we actually use in-house brokers and agents for everything. The reason for this is that while agents search on the MLS on behalf of their buyers many disregard the realtor remarks and just call the "listing agent" rather than the seller directly. We built our technology in a way where our admin can take that call for you and take notes about the conversation then route that "lead" directly to you via our mobile app.
Unfortunately, the outside agent/broker that Houzeo used to list your property has most likely received several calls from buyer agents that never made it to you.
​
As for non-agents seeing the realtor remarks, that will never happen because the realtor remarks only exist in the MLS and ONLY Realtors can access the MLS.
​
I'm just speaking from experience here as Falaya also lists seller contact info in the realtor remarks on the MLS, but buyer agents rarely ever contact the seller directly. We take a ton of calls from buyer agents with interested buyers that we have to route to the seller.