As long as the distance to the wall is above the camera's default minimum distance (about 50 mm or more from the wall, then), and your maximum distance is less than 3 m (as RMS error becomes noticable past 3 m distance) then it should be a "good" distance. It is very subjective though, as there are a range of environmental factors - such as light-sources - that can affect a camera's results.
The quality of the camera image is partly dependent on how much 'texture' detail it can read in a scene. Flat plain surfaces such as walls and doors do not provide the camera with much texture. In these circumstances, the camera's IR Projector component can be used to project a semi-random pattern of dots onto the surface to increase its readable texture detail. Alternatively, you can point the camera at a print-out of the textured image supplied in the tuning guide.
I would speculate that as you zoom in close on a plain white wall, the amount of readable texture is being reduced, like focusing the camera in on a sheet of plain white paper and excluding everything outside of the plain white featureless paper from the camera's view.
In regard to the orange dot, it is hard to say what that is. It is in a 'bad' area of the image, where the far-left of the image breaks up into an "invalid depth band". Pages 58-59 of the current edition of the data sheet for the 400 Series cameras explains about this phenomenon.
https://www.intel.com/content/dam/support/us/en/documents/emerging-technologies/intel-realsense-technology/Intel-RealSense-D400-Series-Datasheet.pdf
