New Town residents have been left bewildered by the decision to allow a modern home to be built on a site previously populated by trees.