Today we can observe that security in the transportation has become a very important issue due to number of incidences related to moving vehicles and the use of some type of vehicle in number of terrorist attacks throughout the world. In this paper we have presented automatic vehicle detection system for aerial surveillance. Also, we avoid the existing frameworks and stereotype system of vehicle detection in aerial surveillance, which are either sliding window or region based. A pixel wise classification method is designed for the vehicle detection. The novelty lies in the fact that, in spite of performing pixel wise classification, relations among neighboring pixels in a region are preserved in the feature extraction process.  The vehicle colours and local features are considered in this study. For vehicle colour extraction, we utilize a colour transform to separate vehicle colours and nonvehicle colours effectively. The regional local features are converted into quantitative observations when they are applying for pixel wise classification through DBN.