that and also be aware that the direction vector's length is used as maximum distance, so if there is no intersection found within that distance, it will not hit
I've been toying with line intersections a lot lately and I would recommend debugging the values coming out of the intersect (or intersect_all) functions by printing or setting detail attributes and making sure they look good. You may be getting intersections you don't expect that are throwing off your results. You may also need to try intersect_all and tweak the optional tolerance values.