You shouldn't have to pay for sketches done only for the purpose of estimation. If you can submit those drawings for a building permit, they are plans, and should carry a price in line with the time spent creating and editing them to suit your specifications.
it all depends on the agreement ,which seems non existant....it is possible you DO pay for sketches too, not just permit sets.... often times I`m asked to do just sketches, for some projects.
I dont know if your state has "verbal " contracts as Texas does...but if not...the court ,if you were sued , would ask for the contract ,and as there is none....
"estimating drawing" COULD be asked for to determine if buyer could afford the house...but , again ,its all depending on what was contrcated for.