Semantic Image Scene Segmentation by Deep Machine Learning

Prof. Jiang Xudong, Nanyang Technological University, Singapore

Organized by IEEE Signal Processing Singapore Chapter & IEEE Circuits and Systems Singapore Chapter & Centre for Information Sciences and Systems (ICSS), School of EEE, NTU & Singapore University of Social Sciences (SUSS)

Abstract

Scene segmentation is a challenging task as it need classify every pixel in the image. It is crucial to exploit discriminative context and aggregate multi-scale features to achieve better segmentation. Context is essential for semantic segmentation. Due to the diverse shapes of objects and their complex layout in various scene images, the spatial scales and shapes of contexts for different objects have very large variation. It is thus ineffective or inefficient to aggregate various context information from a predefined fixed region. In this talk, I will first present a novel context contrasted local feature that not only leverages the informative context but also spotlights the local information in contrast to the context. The proposed context contrasted local feature greatly improves the parsing performance, especially for inconspicuous objects and background stuff. Furthermore, I will present a scheme of gated sum to selectively aggregate multi-scale features for each spatial position. The gates in this scheme control the information flow of different scale features. Their values are generated from the testing image by the proposed network learnt from the training data so that they are adaptive not only to the training data, but also to the specific testing image. Finally, I will present a scale- and shape-variant semantic mask for each pixel to confine its contextual region. To this end, a novel paired convolution is proposed to infer the semantic correlation of the pair and based on that to generate a shape mask. Using the inferred spatial scope of the contextual region, a shape-variant convolution is controlled by the shape mask that varies with the appearance of input. In this way, the proposed network aggregates the context information of a pixel from its semantic-correlated region instead of a predefined fixed region. In addition, this work also proposes a labeling denoising model to reduce wrong predictions caused by the noisy low-level features. This talk is based on two papers: H. Ding, X. Jiang, et al, “Context contrasted feature and gated multi-scale aggregation for scene segmentation,” CVPR’2018 Oral, and H. Ding, X. Jiang, et al, “Semantic Correlation Promoted Shape-Variant Context for Segmentation,” CVPR’2019 Oral.

Speaker Biography

Xudong Jiang received the B.Eng. and M.Eng. from the University of Electronic Science and Technology of China (UESTC), and the Ph.D. degree from Helmut Schmidt University, Hamburg, Germany, all in electrical engineering. From 1986 to 1993, he was a Lecturer with UESTC, where he received two Science and Technology Awards from the Ministry for Electronic Industry of China. From 1998 to 2004, he was with the Institute for Infocomm Research, A-Star, Singapore, as a Lead Scientist and the Head of the Biometrics Laboratory, where he developed a system that achieved the most efficiency and the second most accuracy at the International Fingerprint Verification Competition in 2000. He joined Nanyang Technological University (NTU), Singapore, as a Faculty Member, in 2004, and served as the Director of the Centre for Information Security from 2005 to 2011. Currently, he is a Tenured Associate Professor with the School of EEE, NTU. Dr Jiang holds 7 patents and has authored over 150 papers with 40 papers in the IEEE journals, including 11 papers in IEEE T-IP and 6 papers in IEEE T-PAMI. Two of his first authored papers have been listed as top 1% highly cited papers in the academic field of Engineering by Essential Science Indicators. He served as IFS Technical Committee Member of the IEEE Signal Processing Society from 2015 to 2017, Associate Editor for IEEE SPL for 2 terms from 2014 to 2018, Associate Editor for IEEE T-IP for 2 terms from 2016 to 2019 and the founding editorial board member for IET Biometrics form 2012 to 2019. Dr Jiang is currently a Senior Area Editor for IEEE T-IP and Editor-in-Chief for IET Biometrics. His current research interests include image processing, pattern recognition, computer vision, machine learning, and biometrics.

Back to Top

Leveraging Old Tricks in A New World - Efficient Generation of Labeled Data for Deep Learning

Prof. Gaurav Sharma, University of Rochester, New York, USA

Organized by IEEE Signal Processing Singapore Chapter & IEEE Circuits and Systems Singapore Chapter & Singapore University of Social Sciences

Technical sponsor: IEEE SPS Student Branch – Singapore University of Technology and Design 

Abstract

In the emerging world of artificial intelligence algorithms and analytics, curated and labeled data is often considered the “new oil.” However, extracting this “oil” is often expensive and fraught with difficulties. Specifically, generation of labeled data can be labor intensive and tedious. Crowd sourcing can distribute the labor and mitigate the cost and tedium for some application scenarios. However, for other applications, such as medicine, the requirement of specialized knowledge and skills can make crowd sourcing unviable and generation of labeled data is therefore both expensive and limited, in volume and accuracy, by the availability of physicians’ time. The problem is even more acute for tasks requiring sample level labeling of large, high-resolution spatio-temporal datasets, for example, for pixel-level labeling of images for medical image segmentation. In this talk, through case studies, we highlight examples where creative use of conventional machine learning, computer vision, and image processing techniques allows us to efficiently generate labeled data for new applications of deep learning. The examples particularly highlight that these conventional tools continue to be effective and useful and will therefore co-exist symbiotically with modern deep learning methodologies.

Speaker Biography

Gaurav Sharma is a professor in the Departments of Electrical and Computer Engineering, Computer Science, and Biostatistics and Computational Biology, and a Distinguished Researcher in Center of Excellence in Data Science (CoE) at the Goergen Institute for Data Science at the University of Rochester. He received the PhD degree in Electrical and Computer engineering from North Carolina State University, Raleigh in 1996. From 1993 through 2003, he was with the Xerox Innovation group in Webster, NY, most recently in the position of Principal Scientist and Project Leader. His research interests include data analytics, cyber physical systems, signal and image processing, computer vision, and media security; areas in which he has 53 patents and has authored over 220 journal and conference publications. He currently serves as the Editor-in-Chief for the IEEE Transactions on Image Processing. From 2011 through 2015, he served as the Editor-in-Chief for the Journal of Electronic Imaging and, in the past, has served as an associate editor for the Journal of Electronic Imaging, the IEEE Transactions on Image Processing, and for the IEEE Transactions on Information Forensics and Security. He is a member of the IEEE Publications, Products, and Services Board (PSPB) and chaired the IEEE Conference Publications Committee in 2017-18. He is the editor of the Digital Color Imaging Handbook published by CRC press in 2003. Dr. Sharma is a fellow of the IEEE, a fellow of SPIE, a fellow of the Society for Imaging Science and Technology (IS&T) and has been elected to Sigma Xi, Phi Kappa Phi, and Pi Mu Epsilon. In recognition of his research contributions, he received an IEEE Region I technical innovation award in 2008. Dr. Sharma is a 2020-2021 Distinguished Lecturer for the IEEE Signal Processing Society. 

Back to Top

Robust Cluster Enumeration and Object Labeling

Prof. Abdelhak M. Zoubir, Technische Universität Darmstadt, Germany

Organized by Temasek Labs @ NTU & IEEE Circuits and Systems Singapore Chapter & IEEE Signal Processing Singapore Chapter

Abstract

This talk concerns the area of cluster analysis by providing principled methods to determine the number of data clusters and cluster memberships, even in the presence of outliers. We develop Bayesian cluster enumeration based on modeling the data as a family of Gaussian and t distributions. Real-world applicability is demonstrated by considering advanced signal processing applications, such as distributed camera networks and radar-based person identification.

Speaker Biography

Abdelhak M. Zoubir is a Fellow of the IEEE and IEEE Distinguished Lecturer (Class 2010- 2011). He received his Dr.-Ing. from Ruhr-Universität Bochum, Germany in 1992. He was with Queensland University of Technology, Australia from 1992-1998 where he was Associate Professor. In 1999, he joined Curtin University of Technology, Australia as a Professor of Telecommunications. In 2003, he moved to Technische Universität Darmstadt, Germany as Professor of Signal Processing and Head of the Signal Processing Group. His research interest lies in statistical methods for signal processing with emphasis on bootstrap techniques, robust detection and estimation and array processing applied to telecommunications, radar, sonar, automotive monitoring and safety, and biomedicine. He published over 400 journal and conference papers on the above areas. Dr. Zoubir served as General Chair and Technical Chair of numerous international IEEE conferences and workshops; most notably he was the Technical Co-Chair of ICASSP-14 held in Florence, Italy. He also served on publication boards of various journals, notably as Editor-In-Chief of the IEEE Signal Processing Magazine (2012-2014). Dr. Zoubir was the Chair (2010-2011) of the IEEE Signal Processing Society (SPS) Technical Committee Signal Processing Theory and Methods (SPTM). He served on the Board of Governors of the IEEE SPS and was the president of the European Association of Signal Processing (EURASIP).

Back to Top