Title page for 91522065


[Back to Results | New Search]

Student Number 91522065
Author Chih-Chang Yu(§E°õ¹ü)
Author's Email Address No Public.
Statistics This thesis had been viewed 1517 times. Download 302 times.
Department Computer Science and Information Engineering
Year 2008
Semester 1
Degree Ph.D.
Type of Document Doctoral Dissertation
Language English
Title Human Behavior Analysis using Model-based and Model-free Approaches
Date of Defense 2009-01-06
Page Count 92
Keyword
  • Action recognition
  • Behavior analysis
  • Human body modeling
  • Abstract Recently, the development of video archives grows rapidly due to the advancement and popularization of multimedia internetworking technologies and high-capacity data storage devices. To efficiently summarize these multimedia contents, an automated video understanding system is highly required. When performing video understanding and summarization, researchers are most interested in analyzing human behaviors due to the high demanding of various applications. Hence, having a detailed description of human actions can provide rich information for these applications. In this dissertation, we make a broad study on human behavior analysis. Among them, we comprehensively study two main categories of approaches for human action recognition. Problems that may occur in both categories of approaches are fully addressed and solutions are proposed.
    The first category is the model-based approach. For this type of approach, several body parts including head, torso, arms and legs are extracted to build a human body model. A hierarchical system is designed starting with head extraction, torso extraction, and following by limb extraction. In terms of limb extraction, two methods are proposed including line-based and patch based methods. The line-based method is simpler and faster. However, it cannot deal with the partial occlusion problem. Thus, we further propose the patch based method which adopts a probabilistic framework to find the best configuration of limbs. By using the patch based method, we can successfully tackle the partial occlusion problem, which usually happens on the limbs.
    The second category is the model-free approach. This type of approach tries to recognize human actions via the overall video objects. In this dissertation, we propose a novel approach based on the human silhouettes. As we know, the L1-norm is a popular way to estimate the similarity between two patterns. However, the computation efficiency decreases because the L1-norm measurement is relevant to the dimension of feature. In our work, we convert the human action recognition problem to a histogram matching problem. By doing so, many characteristics of histogram matching can be employed to improve the recognition efficiency and accuracy. Moreover, a novel histogram matching method is proposed by creating multi-resolution histograms, whose bins at higher resolution levels are unevenly partitioned into its lower resolution levels. By utilizing this multi-resolution structure, the computation time will only be relevant to the partitioned histogram bins and the recognition time can be reduced to 9% of the original L1-norm measurement. Because of the reduced computational complexity, the proposed approach allows a real-time recognition system to be realized.
    To demonstrate the feasibility and validity of the proposed approaches, several generic human actions, such as walking, running, jumping, waving hands, falling were performed under a monocular camera. With the success of the experimental results, we believe that the development of this framework can eventually be applied to all kinds of human centric event detection and behavior understanding systems.
    Table of Content ABSTRACTii
    CONTENTSv
    LIST OF FIGURESvii
    LIST OF TABLESx
    CHAPTER 1 INTRODUCTION1
    1.1Motivation1
    1.2Model-based Human Behavior Analysis3
    1.3Model-free Human Behavior Analysis4
    1.4Organization of the Dissertation5
    CHAPTER 2 HUMAN BODY MODELING: HEAD AND TORSO EXTRACTION6
    2.1Related Works6
    2.2System Overview8
    2.3Background Extraction with Shadow Removal10
    2.4Human Body Parts Decomposition11
    2.5Head Region Detection15
    2.5.1Head Acquisition15
    2.5.2Kalman Filtering18
    2.5.3Head Region Tracking20
    2.6Torso Estimation23
    2.7Conclusion26
    CHAPTER 3 HUMAN BODY MODELING: HEURISTIC AND PROBABLISTIC LIMB EXTRACTION28
    3.1Related Works28
    3.2Limb Ends Extraction and Tracking30
    3.2.1Limb Ends Extraction31
    3.2.2Limb Ends Tracking32
    3.3Line-based Limb Modeling34
    3.4Patch-based Limb Modeling37
    3.5Experiments42
    3.5.1Effectiveness on Spline Interpolation Using Different Lengths42
    3.5.2Performance of the Line-based Approach43
    3.5.3Performance of the Patch-based Approach46
    3.5.4Human Body Modeling on Behavior Analysis Application48
    3.6Conclusions50
    CHAPTER 4 HUMAN ACTION RECOGNITION: A MODEL-FREE APPRAOCH52
    4.1Related Works53
    4.2Average Motion Energy (AME)54
    4.3Histogram Based Approach55
    4.3.1Basic Characteristics of Histogram55
    4.3.2Characteristic of Mutli-resolution Histogram56
    4.3.3Motion Energy Histogram (MEH)60
    4.3.4Multi-Resolution Motion Energy Histogram (MRMEH)62
    4.4Efficient Action Recognition Using MRMEH65
    4.5Time Complexity Analysis66
    4.6Experiments68
    4.6.1Dataset68
    4.6.2Recognition70
    4.6.3Recognition Accuracy Analysis71
    4.6.4Recognition Efficiency Analysis75
    4.6.5Real-time Action Recognition Application76
    4.7Conclusion78
    CHAPTER 5 CONCLUSIONS79
    5.1Concluding Remarks79
    5.2Future Works82
    REFERENCES84
    APPENDIX89
    Reference [1]D. M. Gavrila. ¡§The visual analysis of human movement: A survey,¡¨ Computer Vision and Image Understanding: CVIU, vol.73 (1), pp. 82¡V98, 1999.
    [2]P. Viola, M. Jones, and D. Snow. Detecting pedestrians using patterns of motion and appearance. In Proc. 9th Int¡¦l Conf. Computer Vision, pages 734¡V741, 2003.
    [3]J. Grahn and H. Kjellstron, ¡§Using SVM for efficient detection of human motion¡¨, IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance, pp.231- 238, 2005
    [4]B. Li, H. Holstein, "Recognition of Human Periodic Motion — A Frequency Domain Approach," 16th International Conference on Pattern Recognition, Vol. 1, pp.10311, 2002
    [5]Q. Meng, B. Li and H. Holstein, ¡§Recognition of human periodic movements from unstructured information using a motion-based frequency domain approach¡¨, Image and Vision Computing, vol. 24(8), pp. 795-809, 2006.
    [6]G. Mori and J. Malik, ¡§Estimating human body configurations using shape context matching.¡¨ In Proc. 7th European Conf. on Computer Vision, Vol.3, pp. 666-680, 2002.
    [7]S. X. Ju, M. J. Black, and Y. Yacoob, ¡§Cardboard people: A parameterized model of articulated image motion.¡¨ 2nd International Conference on Automatic Face and Gesture Recognition, 1996
    [8]X. Lan and D.Hutternlocher, ¡§A unified spatio-temporal articulated model for tracking.¡¨ Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol.1, pp 722-729, 2004.
    [9]G. Mori, X. Ren, A. Efros, and J. Malik. ¡§Recovering human body configurations: Combining segmentation and recognition,¡¨ Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 326-333, 2004
    [10]Javed, M. Shah, ¡§Tracking And Object Classification for Automated Surveillance,¡¨ Proc. 7th European Conf. Computer Vision, vol.2423, pp. 343-357, 2002
    [11]D. Ramanan, D.A. Forsyth and A. Zisserman, ¡§Tracking People by Learning Their Appearance,¡¨ IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no.1, pp. 65-81, 2007
    [12]D. Ramanan, D.A. Forsyth, and A. Zisserman, ¡§Strike a Pose: Tracking People by Finding Stylized Poses,¡¨ Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR 05), pp. 271-278, 2005.
    [13]Chen, D. Y., Shih, S. W. and Liao, H. Y. Mark Laio, ¡§Human Action Recognition Using 2-D Spatio-Temporal Templates,¡¨ International Conference on Multimedia and Expo, pp. 667-670 (2007)
    [14]I. Haritaoglu, D. Harwood, and L.S. Davis, ¡§Ghost: A human body part labeling system using silhouettes,¡¨ proc. of the 14th Intl. Conf. on Pattern Recognition, pp. 77 ¡V 82., 1998.
    [15]I. Haritaoglu, D. Harwood, and L. S. Davis, 2000. ¡§W4: Real-Time Surveillance of People and Their Activities,¡¨ IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 809-830.
    [16]F. Remondino and A. Roditakis, ¡§Human Figure Reconstruction and Modeling from Single Image or Monocular Video Sequence¡¨, 4th International Conf. on 3D Digital Imaging and Modeling, pp.116-123, 2003.
    [17]D. Hoffman and W. Richards, ¡§Parts of recognition,¡¨ Cognition 18: 65-96, 1984.
    [18]K. Siddiqi, and B. B. Kimia, ¡§Parts of visual form: computational aspects,¡¨ IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 17, pp. 239¡V251,1995
    [19]L. Piegl, W. Tiller, ¡§The NURBS Book, ¡¨ Springer, ISBN 3-540-61545-8, 1997.
    [20]R. E. Kalman, ¡§A New Approach to Linear Filtering and Prediction Problems,¡¨ Transactions of the ASME - Journal of Basic Engineering Vol. 82: pp. 35-45 , 1960
    [21]G. Welch and G. Bishop, ¡§An introduction to the Kalman filter,¡¨ SIGGRAPH 2001 course.
    [22]Javed and M. Shah, ¡§Tracking And Object Classification for Automated Surveillance,¡¨ Proc. of the 7th European Conf. Computer Vision, vol.2423, pp. 343-357, 2002
    [23]Fujiyoshi and A. J. Lipton. ¡§Real-Time Human Motion Analysis by Image Skeletonization,¡¨ Proc. of the 4th IEEE Workshop on Applications of Computer Vision, pp. 15-21, 1998.
    [24]D.Y. Chen, H.Y. Mark Liao, and S.W. Shih, ¡§Continuous Human Action Segmentation and Recognition Using a Spatio-Temporal Probabilistic Framework,¡¨ ism, 8th IEEE International Symposium on Multimedia, pp. 275-282,2006.
    [25]P. Peursum, H. H. Bui, S. Venkatesh, and G. West,¡¨ Robust Recognition and Segmentation of Human Actions Using HMMs with Missing Observations,¡¨ EURASIP Journal on Applied Signal Processing vol.13, pp.2110¡V2126,2005.
    [26]H.S. Chen, H. T. Chen, Y. W. Chen and S.Y. Lee, ¡§Human Action Recognition Using Star Skeleton,¡¨ ACM International Workshop on Video Surveillance & Sensor Networks, 2006.
    [27]J. Shotton, A. Blake, and R. Cipolla, ¡§Contour-Based Learning for Object Detection,¡¨ IEEE International Conference on Computer Vision, pp.503-510, 2005.
    [28]A. Rosenfeld and J. L. Pfaltz, ¡§Sequential operations in digital picture processing,¡¨ J. ACM, vol. 13, pp. 471¡V496, Oct. 1966.
    [29]U. Montanari, ¡§A method for obtaining skeletons using a quasieuclidean distance,¡¨ J. ACM, vol. 15, pp. 600¡V624, Oct. 1968.
    [30]G. Borgefors, ¡§Distance transformations in digital images,¡¨ Comput. Vision, Graphics, Image Processing, vol. 34, pp. 344¡V371, 1986.
    [31]S. Carbini, L. Delphin-Poulat, L. Perron and J.E. Viallet , ¡§From a Wizard of Oz experiment to a real time speech and gesture multimodal interface¡¨, Signal Processing, Vol. 86, Issue 12, pp. 3559-3577,2006
    [32]C. C. Chen, J. W. Hsieh, Y. T. Hsu and C. Y. Huang, ¡§Segmentation of Human Body Parts Using Deformable Triangulation,¡¨ Int¡¦l conf. on Pattern Recognition, vol.1, pp.355-358, 2006
    [33]Fujiyoshi and A. J. Lipton. ¡§Real-Time Human Motion Analysis by Image Skeletonization,¡¨ Proc. of the 4th IEEE Workshop on Applications of Computer Vision, pp. 15-21, 1998.
    [34]P.F. Felzenszwalb and D.P. Huttenlocher, ¡§Efficient Matching of Pictorial Structures,¡¨ Proc. of the IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 66-73, 2000.
    [35]L. Gorelick, M. Galun, E. Sharon, R. Basri, and A. Brandt, ¡§Shape Representation and Classification Using the Poisson Equation,¡¨ IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 12, pp. 1997-2005, 2006.
    [36]T. Horprasert, D. Harwood and L.S. Davis, ¡§A Statistical Approach for Real-time Robust Background Subtraction and Shadow Detection¡¨, IEEE ICCV'99 Frame-Rate Workshop, 1999
    [37]J.S. Hu and T.M. Su, ¡§Robust Background Subtraction with Shadow and Highlight Removal for Indoor Surveillance¡¨, EURASIP Journal on Advances in Signal Processing Vol. 2007 Article ID 82931, 14 pages, 2007.
    [38]N. Thome, D. Merad and S. Miguet. ¡§Human body part labeling and tracking using graph matching theory¡¨, IEEE Int¡¦l Conf. on Advanced Video and Signal based Surveillance, pp 38-46, 2006.
    [39]A. Elgammal, V.Shet, Y.Yacoob, and L.S. Davis, ¡§Gesture recognition using a probabilistic framework for pose matching¡¨, 7th International Conference on Control, Automation, Robotics and Vision (ICARCV 2002), vol. 2, pp. 763-769, 2002
    [40]J. D. Shutler, M. G. Grant, M. S. Nixon, and J. N. Carter, ¡§On a Large Sequence-Based Human Gait Database,¡¨ Proc. 4th International Conference on Recent Advances in Soft Computing, pp. 66-71, 2002
    [41]Tilley, The measure of man and woman, New York: John Wiley and Sons, 2002.
    [42]L. R. Rabiner. ¡§A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition,¡¨ Proceedings of the IEEE, pp. 257-286, 1989
    [43]Bobick and J. Davis. ¡§The recognition of human movement using temporal templates¡¨, PAMI, vol. 23, pp.257¡V267, 2001
    [44]H. W. Lam, S. T. Lee, and D. Zhang, ¡¨Human gait recognition by the fusion of motion and static spatio-temporal templates¡¨, Pattern Recognition, vol. 40, no. 9, pp. 2563-2573, Sep. 2007.
    [45]C. Schuldt, I. Laptev, and B. Caputo, ¡§Recognizing human actions: a local SVM approach¡¨, ICPR, vol.3, pp. 32¡V36, 2004.
    [46]S. Carlsson and J. Sullivan, ¡§Action recognition by shape matching to key frames¡¨, Workshop on Models Versus Exemplars in Computer Vision, 2001
    [47]A. Bobick and J. Davis. The recognition of human movement using temporal templates. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23(3) pp. 257¡V267, 2001
    [48]A. Veeraraghavan, A. Roy-Chowdhury, and R. Chellappa.Role of shape and kinematics in human movement analysis. CVPR (2004): 730-737
    [49]M. Blank, et al. Action as space-time shapes. ICCV (2005): 1395-1402
    [50]J. Han and B. Bhanu. Human activity recognition in thermal infrared imagery. Workshop on Object Tracking and Classification Beyond the Visible Spectrum (2005)
    [51]H. Lakany, ¡§Extracting a diagnostic gait signature¡¨, Pattern Recognition, Vol. 41(5), pp. 1644-1654, 2008.
    [52]J. Davis , ¡§Hierarchical Motion History Images for Recognizing Human Motion¡¨, IEEE Workshop on Detection and Recognition of Events in Video, 2001.
    [53]A. Bobick and J. Davis, ¡§The Representation and Recognition of Action Using Temporal Templates¡¨, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 23, No. 3, pp.257-267, 2001.
    [54]L. Wang  and D. Suter, ¡§Informative Shape Representations for Human Action Recognition¡¨, Int¡¦l Conf on Pattern Recognition, vol. 2, pp. 1266-1269, 2006.
    [55]B. C. Song, M. J. Kim, and J. B. Ra. A fast multiresolution feature matching algorithm for exhaustive search in large image databases. IEEE Trans. Circuits and Systems for Video Technology, vol. 11, no. 5, pp. 673 - 678, May 2001
    [56]F. D. Jou, K. C. Fan and Y. L. Chang. ¡§Efficient matching of large-size histograms.¡¨ Pattern Recognition Letters , vol. 25, issue: 3, pp. 277-286, Feb. 2004.
    [57]S. Sarkar, P. J. Phillips, Z. Liu, I. R. Vega, P. Grother, K. W. Bowyer, "The HumanID Gait Challenge Problem: Data Sets, Performance, and Analysis," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 2, pp. 162-177, Feb., 2005
    [58]R. Cutler, L.S. Davis, "Robust Real-Time Periodic Motion Detection, Analysis, and Applications," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 781-796, Aug., 2000
    [59]J. Han, B. Bhanu, "Individual Recognition Using Gait Energy Image," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 2, pp. 316-322, Feb., 2006
    [60]C.C. Yu, F.D. Jou, C.C. Lee, K.C. Fan and Thomas C. Chuang, "Efficient Multi-resolution Histogram Matching for Fast Video Retrieval", Pattern Recognition Letters, vol.29, pp.1858-1867, 2008
    [61]Xiaotao Zou, Bir Bhanu, "Human Activity Classification Based on Gait Energy Image and Coevolutionary Genetic Programming,", 18th International Conference on Pattern Recognition, Volume 3, pp.556-559, 2006
    [62]G. Salton and M. J. McGill. ¡§Introduction to modern information retrieval.¡¨ McGraw-Hill. ISBN 0070544840, 1983.
    [63]J.P. Eakins, J.M. Boardman, and K. Shields. Retrieval of trade mark images by shape feature-the ARTISAN project. IEE Colloquium on Intelligent Image Databases, May 1996.
    Advisor
  • Kuo-Chin Fan(­S°ê²M)
  • Files
  • 91522065.pdf
  • approve in 2 years
    Date of Submission 2009-01-12

    [Back to Results | New Search]


    Browse | Search All Available ETDs

    If you have dissertation-related questions, please contact with the NCU library extension service section.
    Our service phone is (03)422-7151 Ext. 57407,E-mail is also welcomed.