Virtual Meeting Link

Abstract: 

For the past few years, we have been intensively collaborating with mathematicians from Oxford and the University of Sydney. As a product of this collaboration, we have successfully demonstrated that analysing and interpreting the outputs of (graph) neural networks -- trained on carefully chosen problems of interest -- can directly assist mathematicians with proving difficult theorems and conjecturing new approaches to long-standing open problems. Specifically, using our method we have independently derived novel top-tier mathematical results in areas as diverse as representation theory and knot theory.

By doing so, we present AI as the mathematician's "pocket calculator of the 21st century". The significance of our result has been recognised by the journal Nature, where our work featured on the cover page.

In this talk, I aim to tell you all about our findings, from a personal perspective. Expect key details of our modelling work + an account of how it felt to interact with top mathematicians.

Speaker's Bio: Please see https://petar-v.com/ for a short bio.

In-Person Meeting Location:

Storey Innovation Center 1400

Live Meeting Link for Virtual Audience

Talk Abstract: Artificial intelligence and deep learning are increasingly transforming all scientific disciplines with their superior capability to learn to detect patterns from large amount of data and to learn predictive models from data without relying upon theory or deep mechanistic understanding, with their capability to build generative models for inverse design of materials and molecules and with the models to generate synthetic data. In this talk, we present our research focusing on using deep learning and machine learning to discover and model the patterns in and relationships of structures and functions in materials and molecules and how to exploit such learned dark/implicit knowledge in deep learning based generative design of novel materials, graph neural network based materials property prediction, and deep learning based crystal structure prediction of inorganic materials. Considering that the number of inorganic materials discovered so far (~250,000) by humanity is only a tiny portion of the almost infinite chemical design space, our AI based data-driven computational materials discovery has the potential to transform the conventional trial-and-error approaches in materials discovery.

Speaker's Bio: Dr. Jianjun Hu is currently a Full Professor of computer science at the Department of Computer Science and Engineering, University of South Carolina, Columbia SC. He was associate professor from 2013 to 2022 and assistant professor from 2007 to 2013 at the same department. Dr. Hu received his B.S. and M.S. degrees of Mechanical Engineering in 1995 and 1998 respectively from Wuhan University of Technology, China. He received the Ph.D. degree of Computer Science in 2004 from Michigan State University in the area of machine learning and evolutionary computation, under the supervision of Professor Erik Goodman. He then worked as Postdoctoral Fellow at Purdue University with Prof. Daisuke Kihara and University of Southern California with Prof. Xianghong Zhou from 2004 to 2007 in the area of bioinformatics. Dr. Hu’s main research has focused on machine learning, deep learning, evolutionary computation and their applications in materials informatics, bioinformatics, engineering design, and intelligent manufacturing. His works have been published in PNAS, Advanced Science, Nature npj Computational Materials, Patterns (Cell Press), Evolutionary Computation Journal, Journal of physical chemistry, Scientific Report, and so on with a total of more than 200 journal and conference papers (H-index 35 with > 4200 citations). Currently, his main research is focused on utilizing deep learning to discover the relationship of structures and functions in materials, molecules, and proteins and exploit the learned implicit knowledge for generative design of transformative new materials, drugs, and proteins. His work and research lab info can be found online at: http://mleg.cse.sc.edu/publication

This Friday (8/26), from 2:20 pm - 3:10 pm, at the Seminar in Advances in Computing, Dr. Forest Agostinelli from UofSC will give an in-person talk entitled “Explainable Artificial Intelligence and the Rubik’s Cube”.

In-Person Meeting Location:

Storey Innovation Center 1400

Live Meeting Link for Virtual Audience:

https://teams.microsoft.com/l/meetup-join/19%3ameeting_Nzc3MThiZTktZDY0Zi00MDQzLWE5YmMtMzhiYjlhZTBiMjE3%40thread.v2/0?context=%7b%22Tid%22%3a%224b2a4b19-d135-420e-8bb2-b1cd238998cc%22%2c%22Oid%22%3a%2218da07c8-8a10-4930-a982-b5863c90ddf4%22%7d 

Talk Abstract: The field of artificial intelligence (AI) has allowed computers to learn to synthesize chemical compounds, fold proteins, and write code. However, these AI algorithms cannot explain the thought processes behind their decisions. Not only is explainable AI important for us to be able to trust it with delicate tasks such as surgery and disaster relief, but it could also help us obtain new insights and discoveries. In this talk, I will present DeepCubeA, an AI algorithm that can solve the Rubik’s cube, and six other puzzles, without human guidance. Next, I will discuss how we are building on this work to create AI algorithms that can both solve puzzles and explain their solutions in a manner that we can understand. Finally, I will discuss how this work relates to problems in the natural sciences.

Speaker's Bio: Forest Agostinelli is an assistant professor at the University of South Carolina. He received his B.S. from the Ohio State University, his M.S. from the University of Michigan, and his Ph.D. from the University of California, Irvine under Professor Pierre Baldi. His group conducts research in the fields of deep learning, reinforcement learning, search, explainability, bioinformatics, and neuroscience. His homepage is located at https://cse.sc.edu/~foresta/.

THESIS DEFENSE

Author : Nishat Tabassum

Advisor : Dr. Casey Cole

Date : July 11, 2022

Time 3:00 pm

Place : Virtual (Teams link below)

 

Meeting Link:here

 

Abstract

 Concrete structures have been a major aspect of social infrastructure since the 1700s, so it has been used for centuries. Concrete is used for the durability and support it provides to buildings and bridges. Assessing the state of these structures is important in preserving the longevity of structures and the safety of the public. Detecting cracks in their early stage allows repairs to be made without the need to replace the whole structure, so it reduces the cost. Traditional methods are slowly falling behind as technology advances and an increase in demand for a practical method of crack detection. This study aims to review the practicality of CNN for evaluating damages from cracks autonomously. In addition, many previous methods of crack detection such as traditional manual techniques, image processing techniques, and machine learning methods are discussed. These methods will be investigated to assess the results and effectiveness of each method. Four primary cracks and sixteen secondary cracks of varying depths were chosen to train the CNN model for binary classification of whether a crack is present. A database of images of concrete without cracks was utilized to train the CNN model to recognize the features of images with and without cracks. Multiclass CNN was trained with a dataset of known depths of cracks to predict the severity of damages and cracks. Few studies have been done on depth prediction of cracks, so the aim of this study is to suggest XGBoost of a regression model as an effective method of in-depth prediction. vi Test results show that both the CNN models produced high accuracy in crack identification and damage zone classification. So, it is an effective method that can be used by civil engineers to monitor the well-being of the concrete to reduce labor and increase time efficiency. In addition, the XGBoost of a regression model produced exemplary accuracy in results for predicting the depths of cracks. This demonstrates the possibility of crack depth prediction. Predicting the depths of cracks is important in gaining insight into the health of the structure and can help determine the severity of the cracks and damage to the structure.

THESIS DEFENSE

Author : Joseph Lindsay

Advisor : Dr. Ramtin Zand

Date : July 6, 2022

Time 3:00 pm

Place : Virtual (Teams link below)

 

Meeting Link: click here

 

Abstract

 

While many of the most exciting quantum computing algorithms are currently impossible to be implemented until fault-tolerant quantum error correction is achieved, noisy intermediate-scale quantum (NISQ) devices allow for smaller scale applications that leverage the paradigm for speed-ups to be researched and realized. A currently popular application for these devices is quantum machine learning (QML). Recent works over the past few years indicate that QML algorithms can function just as well as their classical counterparts, and even outperform them in some cases. Many current QML models take advantage of variational quantum algorithm (VQA) circuits, given that their scale is typically small enough to be compatible with NISQ devices and the method of automatic differentiation for optimizing circuit parameters is familiar to machine learning. As with many skeptics on its benefits of quantum computing in general, there is some concern as to whether machine learning is the "best" use case for the advantages that NISQ devices make possible. To this end, the nature of this work is to investigate the utilization of stochastic methods inspired by QML in attempt to approach the reported successes in performance. Using the long short-term memory (LSTM) model as a case study and by analyzing the performance of classical, stochastic, and QML methods, this work aims to elucidate if it is possible to achieve QML's benefits on classical machines by incorporating aspects of its stochasticity.

DISSERTATION DEFENSE

Author : Xinyi Wu

Advisor : Dr. Song Wang

Date : June 7, 2022

Time 9:00 am

Place : Virtual (Zoom link below)

 

Meeting Link: https://zoom.us/j/98175586174?pwd=bEs2a1g0azBmUlhCNWluT2R2bDhidz09

 

Abstract

 

As a long-standing computer vision task, semantic segmentation is still extensively researched till now because of its importance to visual understanding and analysis. The goal of semantic segmentation is to classify each pixel of images based on the pre-defined classes. In the era of deep learning, convolutional neural networks largely improve the accuracy and efficiency of semantic segmentation. However, this success is achieved with two limitations: 1) a large-scale labeled dataset is required for training while the labeling process for this task is quite labor-intensive and tedious; 2) the trained deep networks can get promising results when testing on the same domain (i.e., intra-domain test) but might suffer from a large performance drop when testing on different domains (i.e.,  cross-domain test). Therefore, developing algorithms that can transfer knowledge from labeled source domains to unlabeled target domains is highly desirable to address these two limitations. 

In this research, we explore three settings of cross domain semantic segmentation conditioned on the use of different training data in the target domain: 1) the use of a sole unlabeled target image, 2) the use of multiple unlabeled target images, and 3) the use of unlabeled target videos, respectively. 

At the first part, we tackle the problem of one-shot unsupervised domain adaptation (OSUDA) for semantic segmentation where the segmentors only use one unlabeled target image during training. In this case, traditional unsupervised domain adaptation models usually fail since they cannot adapt to the target domain with over-fitting to one (or few) unlabeled target samples. To address this problem, existing OSUDA methods usually integrate a style-transfer module to perform domain randomization based on the unlabeled target sample, with which multiple domains around the target sample can be explored during training. However, such a style-transfer module relies on an additional set of images as style reference for pre-training and also increases the memory demand for domain adaptation. Here we propose a new OSUDA method that can effectively relieve such computational burden by making full use of the sole target image in two aspects: (1) implicitly stylizing the source domain in both image and feature levels; (2) softly selecting the source training pixels. Experimental results on two commonly-used synthetic-to-real scenarios demonstrate the effectiveness and efficiency of the proposed method.  

Secondly, we work on the problem of nighttime semantic segmentation which plays an equally important role as that of daytime images in autonomous driving but is much more challenging and less studied due to poor illuminations and arduous human annotations. Our proposed solution employs an adversarial training with a labeled daytime dataset and an unlabeled dataset that contains coarsely aligned day-night image pairs. The unlabeled daytime images from the target dataset serve as an intermediate domain to mitigate the difficulty in day-to-night adaption since they share similarities with the source in illumination pattern and contain the same static-category objects as the their nighttime counterparts. Extensive experiments on Dark Zurich and Nighttime Driving datasets show that our method achieves state-of-the-art performance for nighttime semantic segmentation. 

Finally, we propose a domain adaptation method for video semantic segmentation, i.e.,  the target is in video format. Before our work, other works were achieving this goal by transferring the knowledge from the source domain of self-labeled simulated videos to the target domain of unlabeled real-world videos. In our work, we argue that it is not necessary to use a labeled video dataset as the source since the temporal continuity of video segmentation in the target domain can be estimated and enforced without reference to videos in the source domain. This motivates a new framework of Image-to-Video Domain Adaptive Semantic Segmentation (I2VDA), where the source domain is a set of images without temporal information. Under this setting, we bridge the domain gap via adversarial training based on only the spatial knowledge, and develop a novel temporal augmentation strategy, through which the temporal consistency in the target domain is well-exploited and learned. In addition, we introduce a new training scheme by leveraging a proxy network to produce pseudo-labels on-the-fly, which is very effective to improve the stability of adversarial training. Experimental results on two synthetic-to-real scenarios show that the proposed I2VDA method can achieve even better performance on video semantic segmentation than existing state-of-the-art video-to-video domain adaption approaches. 

DISSERTATION DEFENSE

Author : Lan Fu

Advisor : Dr. Song Wang

Date : May 30, 2022

Time 9:00 am

Place : Virtual (Zoom link below)

Zoom link is : https://us05web.zoom.us/j/9860655563?pwd=Qld4ZUozUkFBSGFoa3lRZjNBN3ZVUT09 

 

Abstract

 

Many images are captured in sub-optimal environment, bringing about various kinds of degradations, such as noise, blur, and shadow. Adverse illumination is one of the most important factors resulting in image degradation with color and illumination distortion or even unidentified image content. Degradation caused by the adverse illumination makes the images suffer from worse visual quality, which might also lead to negative effects on high-level perception tasks, e.g., object detection.

 

Image restoration under adverse illumination is an effective way to remove such kind of degradations to obtain visual pleasing images. Existing state-of-the-art deep neural networks (DNNs) based image restoration methods have achieved impressive  performance for image visual quality improvement. However, different real-world applications require the image restoration under adverse illumination to achieve different goals. For example, in the computational photography field, visually pleasing image is desired in the smartphone photography. Nevertheless, for traffic surveillance and autonomous driving in the low light or nighttime scenario, high-level perception tasks, e.g., object detection, become more important to ensure safe and robust driving performance. Therefore, in this dissertation, we try to explore DNN-based image restoration solutions for images captured under adverse illumination in various applications: 1) image visual quality enhancement, 2) object detection improvement, and 3) enhanced image visual quality and better detection performance simultaneously.

 

First, in the computational photography field, a visually pleasing image is desired. We take shadow removal task as an example to fully explore image visual quality enhancement. Shadow removal is still a challenging task due to its inherent background-dependent and spatial-variant properties, leading to unknown and diverse shadow patterns. We propose a novel solution by formulating this task as an exposure fusion problem to address the challenges. We propose shadow-aware FusionNet to `smartly' fuse multiple over-exposure images with pixel-wise fusion weight maps, and boundary-aware RefineNet to eliminate the remaining shadow trace further. Experiment results show that our method outperforms other CNN-based methods in three datasets.

 

Second, we explore the application of CNN-based night-to-day image translation for vehicle detection improvement in the traffic surveillance field for safe and robust driving performance. We propose a detail-preserving method to implement the nighttime to daytime image translation and thus adapt daytime trained detection model to nighttime vehicle detection. We firstly utilize StyleMix method to acquire paired images of daytime and nighttime for following nighttime to daytime image translation training. The translation is implemented based on kernel prediction network to avoid texture corruption. Experimental results showed that the proposed method fit the nighttime vehicle detection task to reuse the daytime domain knowledge.

 

Third, we explore the image visual quality and facial landmark detection improvement simultaneously. For the portrait images captured in the wild, the facial landmark detection can be affected by the foreign shadow. We construct a novel benchmark SHAREL covering diverse face shadow patterns with different intensities, sizes, shapes, and locations to study the effects of shadow removal on facial landmark detection. Moreover, we propose a novel adversarial shadow attack to mine hard shadow patterns. We conduct extensive analysis on three shadow removal methods and three landmark detectors. Then, we design a novel landmark detection-aware shadow removal framework, which empowers shadow removal to achieve higher restoration quality and enhance the shadow robustness of deployed facial landmark detectors.

DISSERTATION DEFENSE

Author : Jun Zhou

Advisor : Dr. Song Wang

Date : May 24, 2022

Time 9:00 am

Place : Virtual (Teams link below)

The Teams invite link is here

Abstract

A challenging problem in modern archaeology is to identify and reconstruct full decorative curve pattern designs from fragmented heritage objects, such as the pottery sherds from southeastern North America. The difficulties of this problem lie in 1) these pottery sherds are usually fragmented so that each sherd only covers a small portion of its underlying full design; 2) these sherds can be highly degraded that curves may contain missing segments or become very shallow; and 3) curve patterns on sherd surfaces may overlap, resulting in composite patterns. Abstracted from this archaeological problem, two computer vision problems are studied: design identification for identifying underlying full design on a sherd by curve pattern matching and sherd identification for grouping unidentified sherds for new design discovery by curve pattern clustering. For design identification, two new curve pattern matching methods are proposed, a Chamfer matching based method for composite pattern matching and a patch-based matching method for noisy curve patterns and composite patterns by deep metric learning and region growing. For sherd identification, a new curve pattern clustering method is proposed involving curve pattern similarity matrix building by deep feature learning, graph partition and iterative cluster refinement. An archaeological computer-aided system, called Snowvision, is developed in this research. The proposed algorithms frame the core of Snowvision.

DISSERTATION DEFENSE 

Author : Yuhang Lu

Advisor : Dr. Song Wang

Date : May 23, 2022

Time 9:00 am

Place : Virtual (Zoom link below)

Meeting Link: https://zoom.us/j/7182193712

 

Abstract

Semantic segmentation that aims at grouping discrete pixels into connected regions is a fundamental step in many high-level computer vision tasks. In recent years, Convolutional Neural Networks (CNNs) have made breakthrough progresses in public semantic segmentation benchmarks. The ability of learning from large-scale labeled datasets empowers them to generalize to unseen images better than traditional non-learning-based methods. Nevertheless, the heavy dependency on labeled data also limits their applications in tasks where high-quality ground truth segmentation masks are scarce or difficult to acquire. In this dissertation, we study the problem of alleviating the data dependency for CNN-based segmentation with a focus on leveraging the shape prior knowledge of objects.

    Shape prior knowledge could provide rich learning-free information of object boundaries if properly utilized. However, this is not trivial for CNN-based segmentation because of its nature of pixel-wise classification. To address this problem, we propose novel methods to integrate three types of shape priors into CNN training, including implicit, explicit and class-agnostic priors. They cover from specific objects with strong prior to general objects with weak prior. To demonstrate the practical value of our methods, we present each of them within a challenging real-world image segmentation task. 1) We propose a weakly supervised segmentation method to extract curve structures stamped on cultural heritage objects, which implicitly takes advantage of the prior knowledge of their thin and elongated shape to relax the training label from pixel-wise curve mask to single-pixel curve skeleton, and outperforms fully supervised alternatives by at least 7.7% in F1 score. 2) We propose a one-shot segmentation method to learn to segment anatomical structure from X-ray images with only one labeled image, which is realized by explicitly model the shape and appearance prior knowledge of objects into the objective function of CNNs. It performs competitively compared to state-of-the-art fully supervised methods when using a single label, and could outperform them when a human-in-the-loop mechanism is incorporated. 3) Finally, we attempt to model shape priors in a universal form that is agnostic to object classes, where the knowledge can be distilled from a few labeled samples through a meta-learning strategy. Given a base model pretrained on existing large-scale dataset, our method could adapt it to any unseen domains with the help of a few labeled images and masks. Experimental results show that our method significantly improve the performance of base models in a variety of cross-domain segmentation tasks.

DISSERTATION DEFENSE 

Author : Zhenyao Wu

Advisor : Dr. Song Wang

Date : May 18, 2022

Time 9:00 am

Place : Virtual (Zoom link below)

 

Meeting Link: https://zoom.us/j/91863722659?pwd=KytCSmc3NGRRbHhPSmczM2EyUnpuQT09

 

Abstract

 

Estimating depth from images has become a very popular task in computer vision which aims to restore the 3D scene from 2D images and identify important geometric knowledge of the scene. Its performance has been significantly improved by convolutional neural networks in recent years, which surpass the traditional methods by a large margin. However, the natural scenes are usually complicated, and hard to build the correspondence between pixels across frames, such as the region containing moving objects, illumination changes, occlusions, and reflections. This research explores rich and comprehensive spatial correspondence across images and designs three new network architectures for depth estimation whose inputs can be a single image, stereo pairs, or monocular video. 

First,  we propose a novel semantic stereo network named SSPCV-Net, which includes newly designed pyramid cost volumes for describing semantic and spatial correspondence on multiple levels. The semantic features are inferred from a semantic segmentation subnetwork while the spatial features are constructed by hierarchical spatial pooling. In the end, we design a 3D multi-cost aggregation module to integrate the extracted multilevel correspondence and perform regression for accurate disparity maps. We conduct comprehensive experiments and comparisons with some recent stereo matching networks on Scene Flow, KITTI 2015 and 2012, and Cityscapes benchmark datasets, and the results show that the proposed SSPCV-Net significantly promotes the state-of-the-art stereo-matching performance. 

Second, we present a novel SC-GAN network with end-to-end adversarial training for depth estimation from monocular videos without estimating the camera pose and pose change over time. To exploit cross-frame relations, SC-GAN includes a spatial correspondence module that uses Smolyak sparse grids to efficiently match the features across adjacent frames and an attention mechanism to learn the importance of features in different directions. Furthermore, the generator in SC-GAN learns to estimate depth from the input frames, while the discriminator learns to distinguish between the ground-truth and estimated depth map for the reference frame. Experiments on the KITTI and Cityscapes datasets show that the proposed SC-GAN can achieve much more accurate depth maps than many existing state-of-the-art methods on monocular videos. 

Finally, we propose a new method for single image depth estimation which utilize the spatial correspondence from stereo matching. To achieve the goal, we incorporate a pre-trained stereo network as a teacher to provide depth cues for the features and output generated by the student network which is a monocular depth estimation network. To further leverage the depth cues, we developed a new depth-aware convolution operation that can adaptively choose subsets of relevant features for convolutions at each location. Specifically, we compute hierarchical depth features as the guidance, and then estimate the depth map using such depth-aware convolution which can leverage the guidance to adapt the filters.  Experimental results on the KITTI online benchmark and Eigen split datasets show that the proposed method achieves the state-of-the-art performance for single-image depth estimation.