TNO has developed an extremely innovative method to be able to recognise human behaviour and intentions in video images. At the beginning of 2010 TNO was allowed to join a DARPA (Defense Advanced Research Projects Agency) research programme. This programme was set up to develop a state-of-the-art visual intelligence system that could enable unmanned vehicles to automatically recognise human activities. After the first year of the programme, it was clear that the system developed by TNO to automatically recognise activities on video images was the best of those tested.

Our QVI/TNO software is world leader in this area. Software currently on the market can detect people and vehicles in video images, as well as identify relatively simple activities such as: person crossing a line, person entering an area, a gathering of people. In addition, the software can be used to count people or vehicles.

Our QVI intelligent video analysis technology however, goes many steps further, and can identify intentions from behaviour to a high level of certainty. In general one can distinguish two factors that strongly influence the performance of an intelligent camera system. These are variability in the situation (for example daytime, evening, and changing weather circumstances) and the number of people in view. The QVI software is able to handle these variables without difficulty, and is therefore able to generate the correct warnings for an operator for example

The system consists of the following components:

1.   Visual processing: Detection of people and objects in the scene, and the extraction of the relevant low-level features.

2.   Event description: Translate low level features of the various entities to a higher level of abstraction. The properties are linked to “physical world  properties” (such as distance, speed, relative position with reference to other entities in the scene).

3.   Reasoning: Derivation of behaviour from the information received about the entities and the relevant relationships. This makes use of various state-of-the-art technology.

4.   Reporting: Conversion of the results from the ‘reasoning” component into a pre-defined format for each task. (for example recognition, description, gap-filling and detection of anomalies).