how to improve the homography accuracy?

how to improve the homography accuracy?

I used opencv's cv::findHomography API to caculate the homography matrix
of two planar images. The matched key points are extracted by Sift and
matched by BFMatcher. As I know, cv:findHomography use ransac iteration to
find out the best four corresponding points to get the homography matrix.
So I draw the selected four pairs of points with the calculated contour
using homograhy matrix of the edge of the object. The result are as the
links:
http://postimg.org/image/5igwvfrx9/
As we can see, the selected matched points by ransac are correct, but the
contour shows that the homography is not accurate.
But these test shows that, both the selected matched points and the
homography are correct:
http://postimg.org/image/dvjnvtm53/
My guess is that if the selected matched points are too close, the small
error of the pixel position will lead to the significant error of the
homography matrix. If the four points are in the corner of the image, then
the shift of the matched points by 4-6 pixels still got good homography
matrix. (According the homogenous coordinate, I think it is reasonable, as
the small error in the near plane will be amplified in the far away)
My question is:
1.Is my guess right? 2.Since the four matched points are generated by the
ransac iteration, the overall error of all the keypoints are minal. But
How to get the stable homography, at least making the contour's mapping is
correct? The theory proved that if the four corresponding points in a
plane are found, the homography matrix should be caculated, but is there
any trick in the engineer work?
Thank you.