Displacement Maps (or Heightmaps) can be adjusted and modified to increase their quality. There are several problems you might encounter trying to use your displacement maps for tessellation or parallax occlusion mapping.
Disturbing issues like stretching, hard edges and noise can be solved by doing some very quick and easy Photoshop work.
White pixel = pushes geometry out.
Grey pixel = does not displace the geometry.
Black pixel = pushes the geometry in.
The values we can define height with are within 128 - 255.
The values we can define depth with are within 0 - 128.
In this example you can clearly see the stretching. Usually the problem is the displacement map having too sharp edges.
By smoothing the map using "Gaussian Blur" you can get some of this stretching out. A good smoothing value in this case is 4 - 5 pixel.
In this case we start with a very contrasty displacement map. This kind of map is very critical to use, because it has a lot of potential causing errors and unintended effects.
As we said we have a range of pixel values we use to define the heights and depths. Many spots in this map have a transition from white (255) to black (0), which provides almost no falloff in between.
Notice the bricks coming out way too strong causing many issues.
One thing you can do to make your map more decent, is to level it and shift all black values to grey. When you do so, you notice only the bricks coming and the rubble stays in place.
This works way better - but there is still some stretching going on which we can simply get rid of by smoothing again.