Deep Learning with Machine Learning

Bayesian

Scalable Bayesian Optimization Using Deep Neural Networks

• intro: ICML 2015
• paper: http://jmlr.org/proceedings/papers/v37/snoek15.html
• arxiv: http://arxiv.org/abs/1502.05700
• github: https://github.com/bshahr/torch-dngo

Bayesian Dark Knowledge

• paper: http://arxiv.org/abs/1506.04416
• notes: Notes on Bayesian Dark Knowledge

Memory-based Bayesian Reasoning with Deep Learning

• intro: Google DeepMind
• slides: http://blog.shakirm.com/wp-content/uploads/2015/11/CSML_BayesDeep.pdf

Towards Bayesian Deep Learning: A Survey

• arxiv: http://arxiv.org/abs/1604.01662

Towards Bayesian Deep Learning: A Framework and Some Existing Methods

• intro: IEEE Transactions on Knowledge and Data Engineering (TKDE), 2016
• arxiv: http://arxiv.org/abs/1608.06884

Bayesian Deep Learning: Neural Networks in PyMC3 estimated with Variational Inference

• blog: http://blog.quantopian.com/bayesian-deep-learning/

Bayesian Deep Learning Part II: Bridging PyMC3 and Lasagne to build a Hierarchical Neural Network

• blog: http://twiecki.github.io/blog/2016/07/05/bayesian-deep-learning/

Deep Bayesian Active Learning with Image Data

• project page: http://mlg.eng.cam.ac.uk/yarin/publications.html#Gal2016Active
• arxiv: https://arxiv.org/abs/1703.02910

Deep Learning: A Bayesian Perspective

• intro: University of Chicago & George Mason University
• arxiv: https://arxiv.org/abs/1706.00473
• paper: https://projecteuclid.org/euclid.ba/1510801992

On Bayesian Deep Learning and Deep Bayesian Learning

• intro: University of Oxford & DeepMind
• lecture: http://csml.stats.ox.ac.uk/news/2017-12-08-ywteh-breiman-lecture/
• video: https://www.facebook.com/nipsfoundation/videos/1555493854541848/
• mirror: https://www.bilibili.com/video/av17121345/

Bayesian Neural Networks

• arxiv: https://arxiv.org/abs/1801.07710
• github: https://github.com/mullachv/MLExp

Bayesian Convolutional Neural Networks

• intro: NeuralSpace
• arxiv: https://arxiv.org/abs/1806.05978

Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adam

• intro: ICML 2018
• intro: RIKEN Center for Advanced Intelligence project & University of British Columbia & University of Oxford
• arxiv: https://arxiv.org/abs/1806.04854
• github: https://github.com/emtiyaz/vadam

Bag of Words (BoW)

Deep Learning Transcends the Bag of Words

• blog: http://www.kdnuggets.com/2015/12/deep-learning-outgrows-bag-words-recurrent-neural-networks.html

E2BoWs: An End-to-End Bag-of-Words Model via Deep Convolutional Neural Network

• intro: ChinaMM 2017, image retrieval
• arxiv: https://arxiv.org/abs/1709.05903

Boosting

Deep Boosting

• intro: ICML 2014
• paper: http://www.cs.princeton.edu/~usyed/CortesMohriSyedICML2014.pdf
• github: https://github.com/google/deepboost

Deep Incremental Boosting

https://arxiv.org/abs/1708.03704

Bootstrap

Training Deep Neural Networks on Noisy Labels with Bootstrapping

• arxiv: http://arxiv.org/abs/1412.6596

ConvCSNet: A Convolutional Compressive Sensing Framework Based on Deep Learning

Compressive Sensing

ConvCSNet: A Convolutional Compressive Sensing Framework Based on Deep Learning

https://arxiv.org/abs/1801.10342

Perceptual Compressive Sensing

https://arxiv.org/abs/1802.00176

Full Image Recover for Block-Based Compressive Sensing

https://arxiv.org/abs/1802.00179

Conditional Random Fields

Deep Markov Random Field for Image Modeling

• intro: ECCV 2016
• arxiv: http://arxiv.org/abs/1609.02036
• github: https://github.com/zhirongw/deep-mrf

Deep, Dense, and Low-Rank Gaussian Conditional Random Fields

• arxiv: https://arxiv.org/abs/1611.09051

DeepCRF: Neural Networks and CRFs for Sequence Labeling

• intro: A implementation of Conditional Random Fields (CRFs) with Deep Learning Method
• github: https://github.com/aonotas/deep-crf

Decision Tree

Deep Neural Decision Forests

• intro: ICCV 2015. Microsoft Research. ICCV’15 Marr Prize
• paper: http://research.microsoft.com/pubs/255952/ICCV15_DeepNDF_main.pdf
• slides: https://docs.google.com/presentation/d/1Ze7BAiWbMPyF0ax36D-aK00VfaGMGvvgD_XuANQW1gU/edit#slide=id.p
• github: https://github.com/chrischoy/fully-differentiable-deep-ndf-tf
• supplement: http://research.microsoft.com/pubs/255952/ICCV15_DeepNDF_suppl.pdf
• notes: http://pan.baidu.com/s/1jGRWem6

Neural Network and Decision Tree

• blog: http://rotationsymmetry.github.io/2015/07/18/neural-network-decision-tree/

Decision Forests, Convolutional Networks and the Models in-Between

• arxiv: http://arxiv.org/abs/1603.01250
• notes: http://blog.csdn.net/stdcoutzyx/article/details/50993124

Distilling a Neural Network Into a Soft Decision Tree

• intro: Google Brain
• arxiv: https://arxiv.org/abs/1711.09784
• github(PyTorch): https://github.com//kimhc6028/soft-decision-tree

End-to-end Learning of Deterministic Decision Trees

https://arxiv.org/abs/1712.02743

Deep Neural Decision Trees

• intro: presented at 2018 ICML Workshop on Human Interpretability in Machine Learning (WHI 2018), Stockholm, Sweden
• arxiv: https://arxiv.org/abs/1806.06988
• github: https://github.com/wOOL/DNDT

Adaptive Neural Trees

• intro: ICML 2019
• intro: University College London & Imperial College London & Microsoft Research
• arxiv: https://arxiv.org/abs/1807.06699
• paper: http://proceedings.mlr.press/v97/tanno19a.html
• github(official): https://github.com/rtanno21609/AdaptiveNeuralTrees

Visualizing the decision-making process in deep neural decision forest

• intro: CVPR 2019 workshops on explainable AI
• arxiv: https://arxiv.org/abs/1904.09201
• github: https://github.com/Nicholasli1995/VisualizingNDF

NBDT: Neural-Backed Decision Trees

• project page: http://nbdt.alvinwan.com/
• arxiv: https://arxiv.org/abs/2004.00221
• github: https://github.com/alvinwan/neural-backed-decision-trees

Dictionary Learning

Greedy Deep Dictionary Learning

• arxiv: http://arxiv.org/abs/1602.00203

Sparse Factorization Layers for Neural Networks with Limited Supervision

• arxiv: https://arxiv.org/abs/1612.04468

Online Convolutional Dictionary Learning

https://arxiv.org/abs/1709.00106

Deep Dictionary Learning: A PARametric NETwork Approach

https://arxiv.org/abs/1803.04022

Deep Micro-Dictionary Learning and Coding Network

• intro: WACV 2018
• arxiv: https://arxiv.org/abs/1809.04185

Fisher Vectors

Backpropagation Training for Fisher Vectors within Neural Networks

• arxiv: https://arxiv.org/abs/1702.02549

Semi-supervised Fisher vector network

https://arxiv.org/abs/1801.04438

Gaussian Processes

Questions on Deep Gaussian Processes

• blog: http://inverseprobability.com/2015/02/28/questions-on-deep-gaussian-processes/

Qs – Deep Gaussian Processes

• blog: https://www.opendatascience.com/blog/qs-deep-gaussian-processes/

Practical Learning of Deep Gaussian Processes via Random Fourier Features

• arxiv: https://arxiv.org/abs/1610.04386

Deep Learning with Gaussian Process

• blog: https://amundtveit.com/2016/12/02/deep-learning-with-gaussian-process/

Doubly Stochastic Variational Inference for Deep Gaussian Processes

• arxiv: https://arxiv.org/abs/1705.08933
• github: https://github.com/thangbui/deepGP_approxEP
• github: https://github.com/ICL-SML/Doubly-Stochastic-DGP

Deep Gaussian Mixture Models

https://arxiv.org/abs/1711.06929

How Deep Are Deep Gaussian Processes?

https://arxiv.org/abs/1711.11280

Variational inference for deep Gaussian processes

• intro: NIPS workshop on Advances in Approximate Bayesian Inference 2017
• slide: http://adamian.github.io/talks/Damianou_NIPS17.pdf

Deep Gaussian Processes with Decoupled Inducing Inputs

• intro: University of Cambridge & University of Seville
• arxiv: https://arxiv.org/abs/1801.02939

Gaussian Process Behaviour in Wide Deep Neural Networks

• intro: University of Cambridge
• arxiv: https://arxiv.org/abs/1804.11271
• github: https://github.com/widedeepnetworks/widedeepnetworks

Differentiable Compositional Kernel Learning for Gaussian Processes

• intro: ICML 2018. University of Toronto
• arxiv: https://arxiv.org/abs/1806.04326

Deep Convolutional Networks as shallow Gaussian Processes

• intro: University of Cambridge
• arxiv: https://arxiv.org/abs/1808.05587
• github: https://github.com/rhaps0dy/convnets-as-gps

Deep convolutional Gaussian processes

• intro: Aalto university
• arxiv: https://arxiv.org/abs/1810.03052
• github: https://github.com/kekeblom/DeepCGP

Graph Convolutional Gaussian Processes

• intro: ICML 2019
• arxiv: https://arxiv.org/abs/1905.05739

Wide Feedforward or Recurrent Neural Networks of Any Architecture are Gaussian Processes

• intro: NeurIPS 2019
• arxiv: https://arxiv.org/abs/1910.12478
• github: https://github.com/thegregyang/GP4A

Graphical Models

GibbsNet: Iterative Adversarial Inference for Deep Graphical Models

• intro: NIPS 2017
• arxiv: https://arxiv.org/abs/1712.04120

GMM

DeepGUM: Learning Deep Robust Regression with a Gaussian-Uniform Mixture Model

• intro: ECCV 2018
• arxiv: https://arxiv.org/abs/1808.09211

HMM

Unsupervised Neural Hidden Markov Models

• intro: EMNLP 2016
• paper: http://www.isi.edu/natural-language/mt/neural-hmm16.pdf
• github: https://github.com/ketranm/neuralHMM

Histogram

Learnable Histogram: Statistical Context Features for Deep Neural Networks

• intro: ECCV 2016
• arxiv: https://arxiv.org/abs/1804.09398

Kalman Filter

Deep Robust Kalman Filter

https://arxiv.org/abs/1703.02310

Kernel Methods

Kernel Methods for Deep Learning

• intro: NIPS 2009
• paper: https://papers.nips.cc/paper/3628-kernel-methods-for-deep-learning
• paper: http://cseweb.ucsd.edu/~saul/papers/nips09_kernel.pdf

Deep Kernel Learning

• arxiv: http://arxiv.org/abs/1511.02222

Stochastic Variational Deep Kernel Learning

• intro: NIPS 2016
• arxiv: https://arxiv.org/abs/1611.00336
• code: https://people.orie.cornell.edu/andrew/code/#SVDKL

A Deep Learning Approach To Multiple Kernel Fusion

• arxiv: https://arxiv.org/abs/1612.09007

Optimizing Kernel Machines using Deep Learning

• keywords: DKMO (Deep Kernel Machine Optimization)
• arxiv: https://arxiv.org/abs/1711.05374

Stacked Kernel Network

https://arxiv.org/abs/1711.09219

Deep Embedding Kernel

• intro: Kennesaw State University
• arxiv: https://arxiv.org/abs/1804.05806

Learning Explicit Deep Representations from Deep Kernel Networks

https://arxiv.org/abs/1804.11159

k-Nearest Neighbors

Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning

• intro: Pennsylvania State University
• arxiv: https://arxiv.org/abs/1803.04765
• arxiv-vanity: https://www.arxiv-vanity.com/papers/1803.04765/

LBP

Deep Local Binary Patterns

https://arxiv.org/abs/1711.06597

Local Binary Pattern Networks

• intro: local binary pattern networks or LBPNet
• arxiv: https://arxiv.org/abs/1803.07125

Neural Nearest Neighbors Networks

• intro: NIPS 2018
• arxiv: https://arxiv.org/abs/1810.12575
• github: https://github.com/visinf/n3net/

Deep Logistic Regression

Single-Label Multi-Class Image Classification by Deep Logistic Regression

• intro: AAAI 2019
• arxiv: https://arxiv.org/abs/1811.08400

Probabilistic Programming

Deep Probabilistic Programming with Edward

• intro: Columbia University & Adobe Research & Google
• poster: http://dustintran.com/papers/TranHoffmanMurphyBrevdoSaurousBlei2016_poster.pdf

SVM

Large-scale Learning with SVM and Convolutional for Generic Object Categorization

• paper: http://yann.lecun.com/exdb/publis/pdf/huang-lecun-06.pdf

Convolutional Neural Support Vector Machines:Hybrid Visual Pattern Classifiers for Multi-robot Systems

• paper: http://people.idsia.ch/~nagi/conferences/idsia/icmla2012.pdf

Deep Learning using Linear Support Vector Machines

• intro: Workshop on Representational Learning, ICML 2013
• arxiv: https://arxiv.org/abs/1306.0239
• paper: http://deeplearning.net/wp-content/uploads/2013/03/dlsvm.pdf
• github: https://github.com/momer/deep-learning-faces
• code: https://code.google.com/p/deeplearning-faces/

Deep Support Vector Machines

• video: http://videolectures.net/roks2013_wiering_vector/

• slides: http://www.esat.kuleuven.be/sista/ROKS2013/files/presentations/DSVM_ROKS_2013_WIERING.pdf Trusting SVM for Piecewise Linear CNNs

• intro: PL-CNNs
• arxiv: https://arxiv.org/abs/1611.02185

Random Forest

Towards the effectiveness of Deep Convolutional Neural Network based Fast Random Forest Classifier

• arxiv: http://arxiv.org/abs/1609.08864

Deep Forest: Towards An Alternative to Deep Neural Networks

• projetc: http://lamda.nju.edu.cn/code_gcForest.ashx
• arxiv: https://arxiv.org/abs/1702.08835
• github(official): https://github.com/kingfengji/gcForest

Forward Thinking: Building Deep Random Forests

• arxiv: https://arxiv.org/abs/1705.07366
• github: https://github.com/tkchris93/ForwardThinking

Deep Regression Forests for Age Estimation

https://arxiv.org/abs/1712.07195

Deep Differentiable Random Forests for Age Estimation

https://arxiv.org/abs/1907.10665

Template Matching

QATM: Quality-Aware Template Matching For Deep Learning

• intro: CVPR 2019
• arxiv: https://arxiv.org/abs/1903.07254
• github(official, Tensorflow): https://github.com/cplusx/QATM