As autonomous vehicles operating on the urban roads, being conscious of the road context is a crucial prerequisite to safely negotiate with the other vehicles. This paper proposes a probabilistic approach to infer the road context from the vehicle behaviors. Specifically, the consistencies of the randomly-observed vehicle states are extracted first, thereafter the road context is inferred in a probabilistic manner by coupling these consistencies. The feasibility of the proposed road context inference approach has been validated by the case study of an urban road that includes roundabout and T-junction. The experiments demonstrate that the inferred road context can be successfully applied for the autonomous vehicles in various aspects.