This paper proposes an algorithm for detecting objects representing potential hazards to drivers based on the combination of local information derived from optical flows and global information obtained from the host vehicle's status. The algorithm uses artificial neural networks to infer the degree of danger posed by moving objects in dynamic images taken with a vehicle-mounted camera. This approach allows more flexible adaptation of the algorithm to many drivers with dissimilar characteristics. Experiments were conducted with both miniature vehicles in a virtual environment and real vehicles in a real driving situation using video images of multiple moving objects. The results show that the algorithm can infer hazardous situations similar to the judgments made by human drivers. The proposed algorithm provides the foundation for constructing a practical driving assistance system