mountains should go from lines of 20, up to 60. There are examples given in the tutorial section I believe.
As for the boulders, there's an example in the colortable. It goes something like:
1 | 2
3 | 4
This is how you placed the numerically valued pixels. The pixel with 1 in the topleft corner, with 2 on the topright corner etc. You get my point, I hope.
Though I personally prefer not to use boulders in maps. I dont advise it unless you're making something special. There are much more different boulder graphics in UO that can be placed manually to fill up a small area which would look muchos better.
