Fun With Deep Learning

Painting

Artist Agent: A Reinforcement Learning Approach to Automatic Stroke Generation in Oriental Ink Painting

Emoji

Brewing EmojiNet

Image2Emoji: Zero-shot Emoji Prediction for Visual Media

Teaching Robots to Feel: Emoji & Deep Learning 👾 💭 💕

Text input with relevant emoji sorted with deeplearning

Sketch

Sketch-a-Net that Beats Humans

How Do Humans Sketch Objects?

Learning to Simplify: Fully Convolutional Networks for Rough Sketch Cleanup (SIGGRAPH 2016)

Convolutional Sketch Inversion

Sketch Me That Shoe (CVPR 2016)

Mastering Sketching: Adversarial Augmentation for Structured Prediction

SketchyGAN: Towards Diverse and Realistic Sketch to Image Synthesis

Image Colorization

Deep Colorization

Learning Large-Scale Automatic Image Colorization

Learning Representations for Automatic Colorization

Colorful Image Colorization

Colorising Black & White Photos using Deep Learning

https://hackernoon.com/colorising-black-white-photos-using-deep-learning-4da22a05f531

Automatic Colorization (Tensorflow + VGG)

colornet: Neural Network to colorize grayscale images

https://github.com/pavelgonchar/colornet

Let there be Color!: Joint End-to-end Learning of Global and Local Image Priors for Automatic Image Colorization with Simultaneous Classification (SIGGRAPH 2016)

Convolutional autoencoder to colorize greyscale images

Image-Color: A deep learning approach to colorizing images

Creating an artificial artist: Color your photos using Neural Networks

Paints Chainer: line drawing colorization using chainer

Unsupervised Diverse Colorization via Generative Adversarial Networks

(DE)^2 CO: Deep Depth Colorization

https://arxiv.org/abs/1703.10881

A Neural Representation of Sketch Drawings

Real-Time User-Guided Image Colorization with Learned Deep Priors

PixColor: Pixel Recursive Colorization

cGAN-based Manga Colorization Using a Single Training Image

Interactive Deep Colorization With Simultaneous Global and Local Inputs

https://arxiv.org/abs/1801.09083

Image Colorization with Generative Adversarial Networks

https://arxiv.org/abs/1803.05400

Learning to Color from Language

Deep Exemplar-based Colorization

Pixel-level Semantics Guided Image Colorization

https://arxiv.org/abs/1808.01597

User-Guided Deep Anime Line Art Colorization with Conditional Adversarial Networks

Pixelated Semantic Colorization

https://arxiv.org/abs/1901.10889

Colorization Transformer

Sounds

Visually Indicated Sounds

Music

GRUV: Algorithmic Music Generation using Recurrent Neural Networks

DeepHear - Composing and harmonizing music with neural networks

Using AutoHarp and a Character-Based RNN to Create MIDI Drum Loops

Musical Audio Synthesis Using Autoencoding Neural Nets

sound-rnn: Generating sound using recurrent neural networks

Using LSTM Recurrent Neural Networks for Music Generation (Project for AI Prac Fall 2015 at Cornell)

Visually Indicated Sounds (MIT. 2015)

Training a Recurrent Neural Network to Compose Music

LSTM Realbook

LSTMetallica: Generation drum tracks by learning the drum tracks of 60 Metallica songs

deepjazz: Deep learning driven jazz generation using Keras & Theano!

Magenta: Music and Art Generation with Machine Intelligence

Music Transcription with Convolutional Neural Networks

Long Short-Term Memory Recurrent Neural Network Architectures for Generating Music and Japanese Lyrics

BachBot: Use deep learning to generate and harmonize music in the style of Bach

Generate Music in TensorFlow

Generate new lyrics in the style of any artist using LSTMs and TensorFlow

sound-GAN: Generative Adversial Network for music composition

Analyzing Six Deep Learning Tools for Music Generation

WIMP2: Creating Music with AI: Highlights of Current Research

Song From PI: A Musically Plausible Network for Pop Music Generation

Grammar Argumented LSTM Neural Networks with Note-Level Encoding for Music Composition

用TensorFlow生成周杰伦歌词

Hip-Hop - Generating lyrics with RNNs

Metis Final Project: Music Composition with LSTMs

http://blog.naoya.io/metis-final-project-music-composition-with-lstms/

Neural Translation of Musical Style

Poetry

NeuralSnap: Generates poetry from images using convolutional and recurrent neural networks

Generating Chinese Classical Poems with RNN Encoder-Decoder

Chinese Poetry Generation with Planning based Neural Network

Weiqi (Go)

Teaching Deep Convolutional Neural Networks to Play Go

Move Evaluation in Go Using Deep Convolutional Neural Networks(Google DeepMind, Google Brain)

Training Deep Convolutional Neural Networks to Play Go

Computer Go Research - The Challenges Ahead (Martin Müller. IEEE CIG 2015)

GoCNN: Using CNN for Go (Weiqi/Baduk) board evaluation with tensorflow

DarkGo: Go in Darknet

BetaGo: Go bots for the people

Deep Learning and the Game of Go

DarkForest

Better Computer Go Player with Neural Network and Long-term Prediction (Facebook AI Research)

AlphaGo

Mastering the game of Go with deep neural networks and tree search

AlphaGo Teach

AlphaGo的分析

How Alphago Works

AlphaGo in Depth

Leela

  • intro: Leela is a strong Go playing program combining advances in Go programming and further original research into a small, easy to use graphical interface.
  • homepage: https://sjeng.org/leela.html

Mastering the game of Go without human knowledge

Computer Go & AlphaGo Zero

AlphaZero: Mastering Games without Human Knowledge - NIPS 2017

PhoenixGo

The future is here – AlphaZero learns chess

https://en.chessbase.com/post/the-future-is-here-alphazero-learns-chess

AlphaGo Zero Cheat Sheet

https://applied-data.science/static/main/res/alpha_go_zero_cheat_sheet.png

Chess

Giraffe: Using Deep Reinforcement Learning to Play Chess

Spawkfish: neural network based chess engine

Chess position evaluation with convolutional neural network in Julia

Deep Learning for … Chess

DeepChess: End-to-End Deep Neural Network for Automatic Learning in Chess

Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm

Game

Learning Game of Life with a Convolutional Neural Network

Reinforcement Learning using Tensor Flow: A deep Q learning demonstration using Google Tensorflow

Poker-CNN: A Pattern Learning Strategy for Making Draws and Bets in Poker Games Using Convolutional Networks

TorchCraft: a Library for Machine Learning Research on Real-Time Strategy Games

BlizzCon 2016 DeepMind and StarCraft II Deep Learning Panel Transcript

DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker

Gym StarCraft: StarCraft environment for OpenAI Gym, based on Facebook’s TorchCraft

  • intro: Gym StarCraft is an environment bundle for OpenAI Gym. It is based on Facebook’s TorchCraft, which is a bridge between Torch and StarCraft for AI research.
  • github: https://github.com/deepcraft/gym-starcraft

Multiagent Bidirectionally-Coordinated Nets for Learning to Play StarCraft Combat Games

https://arxiv.org/abs/1703.10069

Learning Macromanagement in StarCraft from Replays using Deep Learning

Multi-platform Version of StarCraft: Brood War in a Docker Container: Technical Report

Macro action selection with deep reinforcement learning in StarCraft

DeepLeague

DeepLeague: leveraging computer vision and deep learning on the League of Legends mini map + giving away a dataset of over 100,000 labeled images to further esports analytics research

DeepLeague (Part 2): The Technical Details

Courses

Learning Machines

http://www.patrickhebron.com/learning-machines/

Learning Bit by Bit

https://itp.nyu.edu/varwiki/Syllabus/LearningBitbyBitS10

MACHINE LEARNING FOR MUSICIANS AND ARTISTS (Course opens January 2016)

https://www.kadenze.com/courses/machine-learning-for-musicians-and-artists/info

Machine learning for artists @ ITP-NYU, Spring 2016

Machine Learning for Artists @ OpenDot, November 2016

The Neural Aesthetic @ SchoolOfMa, Summer 2016

http://ml4a.github.io/classes/neural-aesthetic/

Blogs

Review of machine / deep learning in an artistic context

https://medium.com/@memoakten/machine-deep-learning-in-an-artistic-context-441f28774bcc#.gegpq99ag

Apprentice Work

https://www.technologyreview.com/s/600762/apprentice-work/

Exploring the Intersection of Art and Machine Intelligence

http://googleresearch.blogspot.jp/2016/02/exploring-intersection-of-art-and.html

Using machine learning to generate music

http://www.datasciencecentral.com/profiles/blogs/using-machine-learning-to-generate-music

art in the age of machine intelligence

https://medium.com/artists-and-machine-intelligence/what-is-ami-ccd936394a83#.hyt4ei9a9

Understanding Aesthetics with Deep Learning

https://devblogs.nvidia.com/parallelforall/understanding-aesthetics-deep-learning/

Go, Marvin Minsky, and the Chasm that AI Hasn’t Yet Crossed

blog: https://medium.com/backchannel/has-deepmind-really-passed-go-adc85e256bec#.inx8nfid0

A Return to Machine Learning

Resources

Music, Art and Machine Intelligence Workshop 2016

Published: 09 Oct 2015

Face Recognition

Papers

Published: 09 Oct 2015

Deep Learning with Machine Learning

Bayesian

Published: 09 Oct 2015

Deep Learning Tutorials

Tutorials

Deep learning

Toward Theoretical Understanding of Deep Learning

VGG Convolutional Neural Networks Practical

Hacker’s guide to Neural Networks

http://karpathy.github.io/neuralnets/

Deep Learning Tutorials

Deep Learning in a Nutshell: Core Concepts

http://devblogs.nvidia.com/parallelforall/deep-learning-nutshell-core-concepts/

Deep Learning in a Nutshell: History and Training

http://devblogs.nvidia.com/parallelforall/deep-learning-nutshell-history-training/

A Deep Learning Tutorial: From Perceptrons to Deep Networks

http://www.toptal.com/machine-learning/an-introduction-to-deep-learning-from-perceptrons-to-deep-networks

Deep Neural Networks (with Python code)

Three Classes of Deep Learning Architectures and Their Applications: A Tutorial Survey

Stanford Unsupervised Feature Learning and Deep Learning Tutorial: UFLDL Tutorial

The Unreasonable Effectiveness of Deep Learning (LeCun)

Deep learning from the bottom up

Introduction to Deep Learning with Python (By Alec Radford. Theano)

New to deep learning? Here are 4 easy lessons from Google

Deep Learning 101

Neural Networks Demystified

Deep Learning SIMPLIFIED

A ‘Brief’ History of Neural Nets and Deep Learning

Deep Neural Networks — An Overview

https://medium.com/@asjad/deep-neural-networks-an-overview-480112b12a13#.i7apzmnso

A Tutorial on Deep Neural Networks for Intelligent Systems

Deep Learning for Computer Vision – Introduction to Convolution Neural Networks

BI Lab Deep Learning Tutorial

Deep Learning Tutorials

Neural Network Architectures

A Practical Introduction to Deep Learning with Caffe and Python

Notes on Convolutional Neural Networks

Feed Forward and Backward Run in Deep Convolution Neural Network

Convolutional Networks

http://deeplearning4j.org/convolutionalnets.html

Exploring convolutional neural networks with DL4J

Understanding Convolutional Neural Networks

Laws, Sausages and ConvNets

Convolutional Neural Networks (CNNs): An Illustrated Explanation

intro_deep: Introduction tutorials to deep learning with Theano and OpenDeep

Deep Learning on Java by Breandan Considine

Using Convolutional Neural Networks and TensorFlow for Image Classification (NYC TensorFlow meetup)

Neural networks with Theano and Lasagne

Introduction to Deep Learning

Introduction to Deep Learning for Image Recognition - SciPy US 2016

Deep learning tutorials (2nd ed.)

A Beginner’s Guide To Understanding Convolutional Neural Networks

A Beginner’s Guide To Understanding Convolutional Neural Networks Part 2

The 9 Deep Learning Papers You Need To Know About (Understanding CNNs Part 3)

Deep Learning Part 1: Comparison of Symbolic Deep Learning Frameworks

Deep Learning Part 2: Transfer Learning and Fine-tuning Deep Convolutional Neural Networks

Deep Learning Part 3: Combining Deep Convolutional Neural Network with Recurrent Neural Network

Introduction to Deep Learning for Image Processing

The best explanation of Convolutional Neural Networks on the Internet!

The Evolution and Core Concepts of Deep Learning & Neural Networks

An Intuitive Explanation of Convolutional Neural Networks

How Convolutional Neural Networks Work

Preliminary Note on the Complexity of a Neural Network

Deep Learning Tutorial

Jupyter notebooks and code for Intro to DL talk at Genesys

Learn Deep Learning the Hard Way

A Complete Guide on Getting Started with Deep Learning in Python

Deep learning for complete beginners: Recognising handwritten digits

Deep learning for complete beginners: Using convolutional nets to recognise images

Deep learning for complete beginners: neural network fine-tuning techniques

How do Convolutional Neural Networks work?

http://brohrer.github.io/how_convolutional_neural_networks_work.html

Creating a Neural Network That Can Tell if a Name Is Male or Female, in JavaScript

Softmax Classifiers Explained

The Softmax function and its derivative

How an algorithm behind Deep Learning works

The Neural Network Zoo

http://www.asimovinstitute.org/neural-network-zoo/

Recognising Beer with TensorFlow

Deep learning architecture diagrams

Getting Started with Deep Learning and Python

Deep Learning Practicals

A simple workflow for deep learning

A primer on universal function approximation with deep learning (in Torch and R)

An Introduction to Implementing Neural Networks using TensorFlow

https://www.analyticsvidhya.com/blog/2016/10/an-introduction-to-implementing-neural-networks-using-tensorflow/

A Gentle Introduction to Convolutional Neural Networks

Beginning Machine Learning with Keras and TensorFlow

Shortest Way to Deep Learning

Deep learning with Matlab

Convolutional neural networks for computer vision with Matlab

Neural Net Computing Explodes

Tutorial: Optimizing Neural Networks using Keras (with Image recognition case study)

https://www.analyticsvidhya.com/blog/2016/10/tutorial-optimizing-neural-networks-using-keras-with-image-recognition-case-study/

15 Deep Learning Tutorials

Deep Learning Episode 1: Optimizing DeepMind’s A3C on Torch

http://www.allinea.com/blog/201607/deep-learning-episode-1-optimizing-deepminds-a3c-torch

Deep Learning Episode 2: Scaling TensorFlow over multiple EC2 GPU nodes

http://www.allinea.com/blog/201608/deep-learning-episode-2-scaling-tensorflow-over-multiple-ec2-gpu-nodes

Deep Learning Episode 3: Supercomputer vs Pong

http://www.allinea.com/blog/201610/deep-learning-episode-3-supercomputer-vs-pong

Deep Learning Episode 4: Supercomputer vs Pong II

http://www.allinea.com/blog/201610/deep-learning-episode-4-supercomputer-vs-pong-ii

Nuts and Bolts of Applying Deep Learning — Summary

Intro to Deep Learning for Computer Vision

http://chaosmail.github.io/deeplearning/2016/10/22/intro-to-deep-learning-for-computer-vision/

If I Can Learn to Play Atari, I Can Learn TensorFlow

TensorFlow workshop materials

Some theorems on deep learning

Pokemon, Colors, and Deep Learning

Why Deep Learning is Radically Different from Machine Learning

Deep Learning: The Unreasonable Effectiveness of Randomness

Deep Meta-Learning : Machines now Bootstrap Themselves

Are Deep Neural Networks Creative?

Are Deep Neural Networks Creative? v2

Develop/Train A Convolutional Neural Netwok For MNIST Dataset

Rethinking Generalization in Deep Learning

https://medium.com/intuitionmachine/rethinking-generalization-in-deep-learning-ec66ed684ace#.tcnsqik5w

The hard thing about deep learning

The hard thing about deep learning

Introduction to Autoencoders

Two Days to a Demo

Deep Learning Tutorials for 10 Weeks

Deep Learning in Clojure With Cortex

A Guide to Deep Learning by YerevaNN

Learning to Learn, to Program, to Explore and to Seek Knowledge

Have Fun with Machine Learning: A Guide for Beginners

Deep Learning Cheat Sheet

How to train your Deep Neural Network

http://rishy.github.io//ml/2017/01/05/how-to-train-your-dnn/

A deep learning traffic light detector using dlib and a few images from Google street view

Recognizing Traffic Lights With Deep Learning

Tutorials for deep learning

The Holographic Principle: Why Deep Learning Works

https://medium.com/intuitionmachine/the-holographic-principle-and-deep-learning-52c2d6da8d9

Deep Neural Networks - A Brief History

Fundamental Deep Learning code in TFLearn, Keras, Theano and TensorFlow

Deep Neural Network from scratch

https://matrices.io/deep-neural-network-from-scratch/

Convolutional Neural Networks

https://github.com/Alfredvc/cnn_workshop

Exploring Optimizers

https://github.com//KeremTurgutlu/deeplearning/blob/master/Exploring%20Optimizers.ipynb

A Gentle Introduction to Exploding Gradients in Neural Networks

https://machinelearningmastery.com/exploding-gradients-in-neural-networks/

Only Numpy: (Why I do Manual Back Propagation) Implementing Multi Channel/Layer Convolution Neural Network on Numpy with Interactive Code

https://medium.com/swlh/only-numpy-why-i-do-manual-back-propagation-implementing-multi-channel-layer-convolution-neural-7d83242fcc24

92.45% on CIFAR-10 in Torch

Convolution

Understanding Convolutions

Note on the implementation of a convolutional neural networks

Convolution in Caffe: a memo

我对卷积的理解

An Analysis of Convolution for Inference

http://www.slideshare.net/nervanasys/an-analysis-of-convolution-for-inference

Understanding Convolution in Deep Learning

A guide to convolution arithmetic for deep learning

Going beyond full utilization: The inside scoop on Nervana’s Winograd kernels

Playing with convolutions in TensorFlow: From a short introduction to convolution to a complete model

How convolutional neural networks see the world: An exploration of convnet filters with Keras

One by One [ 1 x 1 ] Convolution - counter-intuitively useful

http://iamaaditya.github.io/2016/03/one-by-one-convolution/

Checkerboard artifact free sub-pixel convolution: A note on sub-pixel convolution, resize convolution and convolution resize

Receptive Field

A guide to receptive field arithmetic for Convolutional Neural Networks

https://medium.com/@nikasa1889/a-guide-to-receptive-field-arithmetic-for-convolutional-neural-networks-e0f514068807

Momentum

Why Momentum Really Works

maxDNN

maxDNN: An Efficient Convolution Kernel for Deep Learning with Maxwell GPUs

GEMM (General Matrix Matrix Multiply)

Why GEMM is at the heart of deep learning

A full walk through of the SGEMM implementation

Backpropagation

Learning representations by back-propagating errors

Learning Internal Representations by Error Propagating

Calculus on Computational Graphs: Backpropagation

Styles of Truncated Backpropagation

Is BackPropagation Necessary?

Backpropagation In Convolutional LSTMs

https://www.doc.ic.ac.uk/~ahanda/ConvLSTMs.pdf

Backward Pass on Conv Layer

Convolutional Neural Networks backpropagation: from intuition to derivation

Backpropagation In Convolutional Neural Networks

Why do we rotate weights when computing the gradients in a convolution layer of a convolution network?

http://soumith.ch/ex/pages/2014/08/07/why-rotate-weights-convolution-gradient/

Note on the implementation of a convolutional neural networks

http://cthorey.github.io./backprop_conv/

Attention

Attention in a Convolutional Neural Net

Attention-based Networks

Attention in Neural Networks and How to Use It

http://akosiorek.github.io/ml/2017/10/14/visual-attention.html

Softmax

Hierarchical softmax and negative sampling: short notes worth telling

https://towardsdatascience.com/hierarchical-softmax-and-negative-sampling-short-notes-worth-telling-2672010dbe08

Caffe

DIY Deep Learning for Vision: a Hands-On Tutorial with Caffe

Deep learning tutorial on Caffe technology : basic commands, Python and C++ code

http://christopher5106.github.io/deep/learning/2015/09/04/Deep-learning-tutorial-on-Caffe-Technology.html

Using Caffe with your own dataset

https://medium.com/@alexrachnog/using-caffe-with-your-own-dataset-b0ade5d71233

OpenCV 3.0.0-dev: Load Caffe framework models

http://docs.opencv.org/master/d5/de7/tutorial_dnn_googlenet.html#gsc.tab=0

Chainer

Chainer Info

https://github.com/hidetomasuoka/chainer-info

Keras

Keras tutorial

Keras Tutorial: The Ultimate Beginner’s Guide to Deep Learning in Python

https://elitedatascience.com/keras-tutorial-deep-learning-in-python

Deep Learning with Keras: Tutorial @ EuroScipy 2016

Transfer Learning and Fine Tuning for Cross Domain Image Classification with Keras

MXNet

10 Deep Learning projects based on Apache MXNet

https://medium.com/@julsimon/10-deep-learning-projects-based-on-apache-mxnet-8231109f3f64

Awesome MXNet(Beta)

https://github.com/chinakook/Awesome-MXNet

TVM

Optimize Deep Learning GPU Operators with TVM: A Depthwise Convolution Example

http://tvmlang.org/2017/08/22/Optimize-Deep-Learning-GPU-Operators-with-TVM-A-Depthwise-Convolution-Example.html

Theano

Theano Tutorial @ LTI, Carnegie Mellon University

An Introduction to MXNet/Gluon

TensorFlow

LearningTensorFlow.com: A beginners guide to a powerful framework.

TensorFlow Examples: TensorFlow tutorials and code examples for beginners

Awesome TensorFlow: A curated list of awesome TensorFlow experiments, libraries, and projects

The Good, Bad, & Ugly of TensorFlow: A survey of six months rapid evolution (+ tips/hacks and code to fix the ugly stuff)

Tensorflow Tutorials using Jupyter Notebook

TensorFlow Tutorial

FIRST CONTACT WITH TENSORFLOW

Introduction to TensorFlow

TensorFlow-Tutorials: Simple tutorials using Google’s TensorFlow Framework

Neural Network Toolbox on TensorFlow

Awesome Tensorflow Implementations

The Ultimate List of TensorFlow Resources: Books, Tutorials & More

Install TensorFlow: Slides and code from our TensorFlow Workshop

A Tour of TensorFlow

TensorFlow Tutorials

Shapes and dynamic dimensions in TensorFlow

TensorFlow saving/restoring and mixing multiple models

https://blog.metaflow.fr/tensorflow-saving-restoring-and-mixing-multiple-models-c4c94d5d7125#.242xy4d46

Getting to Know TensorFlow

Image Classification and Segmentation with Tensorflow and TF-Slim http://warmspringwinds.github.io/tensorflow/tf-slim/2016/10/30/image-classification-and-segmentation-using-tensorflow-and-tf-slim/

Not another MNIST tutorial with TensorFlow

Dive Into TensorFlow

TensorFlow Exercises - focusing on the comparison with NumPy.

A Gentle Guide to Using Batch Normalization in Tensorflow

Using TensorFlow in Windows with a GPU

Tensorflow and deep learning - without a PhD

4 Steps To Learn TensorFlow When You Already Know scikit-learn https://medium.com/@Zelros/4-steps-to-learn-tensorflow-when-you-already-know-scikit-learn-3cd0340456b5#.q206au7u9

Gentlest Introduction to Tensorflow

learn code with tensorflow

TensorFlow Machine Learning Cookbook

TensorFlow Image Recognition on a Raspberry Pi

http://svds.com/tensorflow-image-recognition-raspberry-pi/

TensorFlow For Machine Intelligence

Installing TensorFlow on Raspberry Pi 3 (and probably 2 as well)

CodinGame: Deep Learning - TensorFlow

A Practical Guide for Debugging Tensorflow Codes

Debugging Tips on TensorFlow

Tensorflow Projects: Deep learning using tensorflow

Machine Learning with TensorFlow

Convolutional Networks: from TensorFlow to iOS BNNS

Android TensorFlow Machine Learning Example

TensorFlow and Deep Learning Tutorials

https://github.com/wagamamaz/tensorflow-tutorial

Finetuning AlexNet with TensorFlow

Deep Learning examples using Tensorflow

https://github.com/aditya101993/Deep-Learning

How To Write Your Own Tensorflow in C++

https://oneraynyday.github.io/ml/2017/10/20/Tensorflow-C++/

Tensorflow on Android

A Guide to Running Tensorflow Models on Android

TensorFlow Android stand-alone demo

Torch

Torch Developer Guide

PyTorch

Practical PyTorch tutorials

The Incredible PyTorch

PyTorch quick start: Classifying an image

tutorial for researchers to learn deep learning with pytorch.

https://github.com/yunjey/pytorch-tutorial

Building a System for Fun!

Facial Recognition On A Jetson TX1 In Tensorflow

Build an AI Cat Chaser with Jetson TX1 and Caffe

Deep Learning in Aerial Systems Using Jetson

Cherry Autonomous Racecar (CAR): NCAT ECE Senior Design Project

Traffic Signs Classification

Traffic signs classification with Deep Learning.

Traffic Sign Recognition with TensorFlow

Traffic signs classification with a convolutional network

http://navoshta.com/traffic-signs-classification/

Convolutional Neural Network for Traffic Sign Classification — CarND

Talks

A Tour of Deep Learning With C++

Published: 09 Oct 2015

Deep Learning Tricks

Papers

Practical recommendations for gradient-based training of deep architectures

Bag of Tricks for Image Classification with Convolutional Neural Networks

Blogs

Efficient BackProp

Deep Learning for Vision: Tricks of the Trade

Optimizing RNN performance

  • intro: Silicon Valley AI Lab
  • keywords: Optimize GEMM, parallel GPU, GRU and LSTM…
  • blog: http://svail.github.io/

Must Know Tips/Tricks in Deep Neural Networks

Training Tricks from Deeplearning4j

http://deeplearning4j.org/trainingtricks.html

Suggestions for DL from Llya Sutskeve

Efficient Training Strategies for Deep Neural Network Language Models

Neural Networks Best Practice

Dark Knowledge from Hinton

Stochastic Gradient Descent Tricks(Leon Bottou)

http://leon.bottou.org/publications/pdf/tricks-2012.pdf

Advice for applying Machine Learning

https://jmetzen.github.io/2015-01-29/ml_advice.html

How to Debug Learning Algorithm for Regression Model

http://vitalflux.com/machine-learning-debug-learning-algorithm-regression-model/

Large-scale L-BFGS using MapReduce

Selecting good features

– Part I: univariate selection: http://blog.datadive.net/selecting-good-features-part-i-univariate-selection/ – Part II: linear models and regularization: http://blog.datadive.net/selecting-good-features-part-ii-linear-models-and-regularization/ – Part III: random forests: http://blog.datadive.net/selecting-good-features-part-iii-random-forests/ – Part IV: stability selection, RFE and everything side by side: http://blog.datadive.net/selecting-good-features-part-iv-stability-selection-rfe-and-everything-side-by-side/

机器学习代码心得之有监督学习的模块

http://www.weibo.com/p/1001603795687165852957

Stochastic Gradient Boosting: Choosing the Best Number of Iterations

Large-Scale High-Precision Topic Modeling on Twitter

H2O World - Top 10 Deep Learning Tips & Tricks - Arno Candel

http://www.slideshare.net/0xdata/h2o-world-top-10-deep-learning-tips-tricks-arno-candel

How To Improve Deep Learning Performance: 20 Tips, Tricks and Techniques That You Can Use To Fight Overfitting and Get Better Generalization

http://machinelearningmastery.com/improve-deep-learning-performance/

Neural Network Training Speed Trick

https://medium.com/machine-learning-at-petiteprogrammer/neural-network-training-speed-trick-92d6b22a7754#.4v6qukpn7

The Black Magic of Deep Learning - Tips and Tricks for the practitioner

http://nmarkou.blogspot.ru/2017/02/the-black-magic-of-deep-learning-tips.html

Published: 09 Oct 2015

Deep Learning Software and Hardware

Papers

Accelerating Deep Convolutional Neural Networks Using Specialized Hardware

Installation / Deploying

Setting up a Deep Learning Machine from Scratch (Software): Instructions for setting up the software on your deep learning machine

  • intro: A detailed guide to setting up your machine for deep learning research. Includes instructions to install drivers, tools and various deep learning frameworks. This was tested on a 64 bit machine with Nvidia Titan X, running Ubuntu 14.04
  • github: https://github.com/saiprashanths/dl-setup

How to install CUDA Toolkit and cuDNN for deep learning

Deploying Deep Learning: Guide to deploying deep-learning inference networks and realtime object detection with TensorRT and Jetson TX1.

Install Log

Lessons Learned from Deploying Deep Learning at Scale

Docker

All-in-one Docker image for Deep Learning

NVIDIA Docker: GPU Server Application Deployment Made Easy

Deep learning base image for Docker (Tensorflow, Caffe, MXNet, Torch, Openface, etc.)

https://github.com/dominiek/deep-base

Deepo: a Docker image with a full reproducible deep learning research environment

Cloud

SuperVessel Cloud for POWER/OpenPOWER LoginRegisterTutorials

http://www.ptopenlab.com/

Building Deep Neural Networks in the Cloud with Azure GPU VMs, MXNet and Microsoft R Server

https://blogs.technet.microsoft.com/machinelearning/2016/09/15/building-deep-neural-networks-in-the-cloud-with-azure-gpu-vms-mxnet-and-microsoft-r-server/

Microsoft open sources its next-gen cloud hardware design

Google Taps AMD For Accelerating Machine Learning In The Cloud

http://www.forbes.com/sites/aarontilley/2016/11/15/google-taps-amd-for-accelerating-machine-learning-in-the-cloud/#3549d8554181

Amazon EC2

Deep Learning AMI on AWS Marketplace

https://aws.amazon.com/marketplace/pp/B01M0AXXQB

We Have To Go Deeper: AWS p2.xlarge GPU optimized deep learning cluster-grenade

A GPU enabled AMI for Deep Learning

Keras with GPU on Amazon EC2 – a step-by-step instruction

https://medium.com/@mateuszsieniawski/keras-with-gpu-on-amazon-ec2-a-step-by-step-instruction-4f90364e49ac#.k27d0mqir

Microsoft R Server

Training Deep Neural Networks on ImageNet Using Microsoft R Server and Azure GPU VMs

Hardware System

I: Building a Deep Learning (Dream) Machine

II: Running a Deep Learning (Dream) Machine

A Full Hardware Guide to Deep Learning

Build your own Deep Learning Box

32-TFLOP Deep Learning GPU Box: A super-fast linux-based machine with multiple GPUs for training deep neural nets

https://hackaday.io/project/12070-32-tflop-deep-learning-gpu-box

Hands-on with the NVIDIA DIGITS DevBox for Deep Learning

Considerations when setting up deep learning hardware

Building a Workstation for Deep Learning

Deep Learning Machine: First build experience

Building a machine learning/deep learning workstation for under $5000

Hardware Guide: Neural Networks on GPUs (Updated 2016-1-30)

Building Your Own Deep Learning Box

https://medium.com/@bfortuner/building-your-own-deep-learning-box-47b918aea1eb#.4r5zchk4f

Setting up a Deep learning machine in a lazy yet quick way https://medium.com/@sravsatuluri/setting-up-a-deep-learning-machine-in-a-lazy-yet-quick-way-be2642318850#.jrxrkfxa2

Deep Confusion: Misadventures In Building A Deep Learning Machine

http://www.topbots.com/deep-confusion-misadventures-in-building-a-machine-learning-server/

DIY-Deep-Learning-Workstation

GPU

Which GPU(s) to Get for Deep Learning: My Experience and Advice for Using GPUs in Deep Learning

从深度学习选择什么样的gpu来谈谈gpu的硬件架构

GPU折腾手记——2015 (by 李沐)

HPC, Deep Learning and GPUs(2016 Stanford HPC Conference)

Modern GPU 2.0: Design patterns for GPU computing

CuMF: CUDA-Acclerated ALS on mulitple GPUs.

Basic Performance Analysis of NVIDIA GPU Accelerator Cards for Deep Learning Applications

CuPy : NumPy-like API accelerated with CUDA

NumPy GPU acceleration

Efficient Convolutional Neural Network Inference on Mobile GPUs (Embedded Vision Summit)

Deep Learning with Multiple GPUs on Rescale: Torch

GPU-accelerated Theano & Keras on Windows 10 native

NVIDIA Announces Quadro GP100 - Big Pascal Comes to Workstations

http://www.anandtech.com/show/11102/nvidia-announces-quadro-gp100

FPGA

Recurrent Neural Networks Hardware Implementation on FPGA

Is implementing deep learning on FPGAs a natural next step after the success with GPUs?

Efficient Implementation of Neural Network Systems Built on FPGAs, Programmed with OpenCL

Deep Learning on FPGAs: Past, Present, and Future

FPGAs Challenge GPUs as a Platform for Deep Learning

Convolution Neural Network CNN Implementation on Altera FPGA using OpenCL

Accelerating Deep Learning Using Altera FPGAs (Embedded Vision Summit)

Machine Learning on FPGAs: Neural Networks

Comprehensive Evaluation of OpenCL-based Convolutional Neural Network Accelerators in Xilinx and Altera FPGAs

Microsoft Goes All in for FPGAs to Build Out AI Cloud

Caffeinated FPGAs: FPGA Framework For Convolutional Neural Networks

Intel Unveils FPGA to Accelerate Neural Networks

http://datacenterfrontier.com/intel-unveils-fpga-to-accelerate-ai-neural-networks/

Deep Learning with FPGA

A General Neural Network Hardware Architecture on FPGA

Approximate FPGA-based LSTMs under Computation Time Constraints

ARM / Processor

‘Neural network’ spotted deep inside Samsung’s Galaxy S7 silicon brain: Secrets of Exynos M1 cores spilled

Intel will add deep-learning instructions to its processors

SRAM

ShiDianNao: Shifting Vision Processing Closer to the Sensor http://lap.epfl.ch/files/content/sites/lap/files/shared/publications/DuJun15_ShiDianNaoShiftingVisionProcessingCloserToTheSensor_ISCA15.pdf

Blogs

Emerging “Universal” FPGA, GPU Platform for Deep Learning

An Early Look at Startup Graphcore’s Deep Learning Chip

https://www.nextplatform.com/2017/03/09/early-look-startup-graphcores-deep-learning-chip/

Hardware for Deep Learning

https://medium.com/towards-data-science/hardware-for-deep-learning-8d9b03df41a

Videos

Energy-efficient Hardware for Embedded Vision and Deep Convolutional Neural Networks

Published: 09 Oct 2015

Deep Learning Resources

ImageNet

Published: 09 Oct 2015

Deep Learning Frameworks

Amazon DSSTNE

Amazon DSSTNE: Deep Scalable Sparse Tensor Network Engine

Apache SINGA

Blocks

Blocks: A Theano framework for building and training neural networks

Blocks and Fuel: Frameworks for deep learning

BrainCore

BrainCore: The iOS and OS X neural network framework

https://github.com/aleph7/BrainCore

Brainstorm

Brainstorm: Fast, flexible and fun neural networks

Caffe

Caffe: Convolutional Architecture for Fast Feature Embedding

OpenCL Caffe

Caffe on both Linux and Windows

ApolloCaffe: a fork of Caffe that supports dynamic networks

fb-caffe-exts: Some handy utility libraries and tools for the Caffe deep learning framework

Caffe-Android-Lib: Porting caffe to android platform

caffe-android-demo: An android caffe demo app exploiting caffe pre-trained ImageNet model for image classification

Caffe.js: Run Caffe models in the browser using ConvNetJS

Intel Caffe

  • intro: This fork of BVLC/Caffe is dedicated to improving performance of this deep learning framework when running on CPU, in particular Intel® Xeon processors (HSW+) and Intel® Xeon Phi processors
  • github https://github.com/intel/caffe

NVIDIA Caffe

https://github.com/NVIDIA/caffe

Mini-Caffe

Caffe on Mobile Devices

CaffeOnACL

  • intro: Using ARM Compute Library (NEON+GPU) to speed up caffe; Providing utilities to debug, profile and tune application performance
  • github: https://github.com/OAID/caffeOnACL

Multi-GPU / MPI Caffe

Caffe with OpenMPI-based Multi-GPU support

mpi-caffe: Model-distributed Deep Learning with Caffe and MPI

Caffe-MPI for Deep Learning

Caffe Utils

Caffe-model

Caffe2

Caffe2: A New Lightweight, Modular, and Scalable Deep Learning Framework

CDNN2

CDNN2 - CEVA Deep Neural Network Software Framework

Chainer

Chainer: a neural network framework

Introduction to Chainer: Neural Networks in Python

CNTK

CNTK: Computational Network Toolkit

An Introduction to Computational Networks and the Computational Network Toolkit

http://research.microsoft.com/apps/pubs/?id=226641

ConvNetJS

ConvNetJS: Deep Learning in Javascript. Train Convolutional Neural Networks (or ordinary ones) in your browser

DeepBeliefSDK

DeepBeliefSDK: The SDK for Jetpac’s iOS, Android, Linux, and OS X Deep Belief image recognition framework

DeepDetect

DeepDetect: Open Source API & Deep Learning Server

Deeplearning4j (DL4J)

Deeplearning4j: Deep Learning for Java

Deeplearning4j images for cuda and hadoop.

Deeplearning4J Examples

DeepLearningKit

DeepLearningKit: Open Source Deep Learning Framework for Apple’s tvOS, iOS and OS X

Tutorial — Using DeepLearningKit with iOS for iPhone and iPad

https://medium.com/@atveit/tutorial-using-deeplearningkit-with-ios-for-iphone-and-ipad-de727679bae4#.1bvnhxhjo

DeepSpark

DeepSpark: Deeplearning framework running on Spark

DIGITS

DIGITS: the Deep Learning GPU Training System

dp

dp: A deep learning library for streamlining research and development using the Torch7 distribution

Dragon

Dragon: A Computation Graph Virtual Machine Based Deep Learning Framework

DyNet

**DyNet: The Dynamic Neural Network Toolkit **

DyNet Benchmarks

IDLF

IDLF: The Intel® Deep Learning Framework

Keras

Keras: Deep Learning library for Theano and TensorFlow

MarcBS/keras fork

Hera: Train/evaluate a Keras model, get metrics streamed to a dashboard in your browser.

Installing Keras for deep learning

Keras Applications - deep learning models that are made available alongside pre-trained weights

https://keras.io/applications/

Keras resources: Directory of tutorials and open-source code repositories for working with Keras, the Python deep learning library

Keras.js: Run trained Keras models in the browser, with GPU support

keras2cpp

keras-cn: Chinese keras documents with more examples, explanations and tips.

Kerasify: Small library for running Keras models from a C++ application

https://github.com/moof2k/kerasify

Knet

Knet: Koç University deep learning framework

Lasagne

Lasagne: Lightweight library to build and train neural networks in Theano

Leaf

Leaf: The Hacker’s Machine Learning Engine

LightNet

LightNet: A Versatile, Standalone and Matlab-based Environment for Deep Learning

MatConvNet

MatConvNet: CNNs for MATLAB

Marvin

Marvin: A minimalist GPU-only N-dimensional ConvNet framework

MatConvNet: CNNs for MATLAB

Mocha.jl

Mocha.jl: Deep Learning for Julia

MXNet

MXNet

MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems

MXNet Model Gallery: Pre-trained Models of DMLC Project

a short introduction to mxnet design and implementation (chinese)

Deep learning for hackers with MXnet (1) GPU installation and MNIST

https://no2147483647.wordpress.com/2015/12/07/deep-learning-for-hackers-with-mxnet-1/

mxnet_Efficient, Flexible Deep Learning Framework

Use Caffe operator in MXNet

Deep Learning in a Single File for Smart Devices

https://mxnet.readthedocs.org/en/latest/tutorial/smart_device.html

MXNet Pascal Titan X benchmark

用MXnet实战深度学习之一:安装GPU版mxnet并跑一个MNIST手写数字识别

http://phunter.farbox.com/post/mxnet-tutorial1

用MXnet实战深度学习之二:Neural art

http://phunter.farbox.com/post/mxnet-tutorial2

Programming Models and Systems Design for Deep Learning

Awesome MXNet

Getting Started with MXNet

https://indico.io/blog/getting-started-with-mxnet/

gtc_tutorial: MXNet Tutorial for NVidia GTC 2016

MXNET Dependency Engine

MXNET是这样压榨深度学习的内存消耗的

WhatsThis-iOS: MXNet WhatThis Example for iOS

MXNET-MPI: Embedding MPI parallelism in Parameter Server Task Model for scaling Deep Learning

ncnn

neocortex.js

Run trained deep neural networks in the browser or node.js

Neon

Neon: Nervana’s Python-based deep learning library

Tools to convert Caffe models to neon’s serialization format

Nervana’s Deep Learning Course

NNabla

NNabla - Neural Network Libraries by Sony

  • intro: NNabla - Neural Network Libraries NNabla is a deep learning framework that is intended to be used for research, development and production. We aim it running everywhere like desktop PCs, HPC clusters, embedded devices and production servers.
  • homepage: https://nnabla.org/
  • github: https://github.com/sony/nnabla

OpenDeep

OpenDeep: a fully modular & extensible deep learning framework in Python

OpenNN

OpenNN - Open Neural Networks Library

Paddle

PaddlePaddle: PArallel Distributed Deep LEarning

基于Spark的异构分布式深度学习平台

http://geek.csdn.net/news/detail/58867

Petuum

Petuum: a distributed machine learning framework

PlaidML

PlaidML: A framework for making deep learning work everywhere

Platoon

Platoon: Multi-GPU mini-framework for Theano

Poseidon

Poseidon: Distributed Deep Learning Framework on Petuum

Purine

Purine: A bi-graph based deep learning framework

PyTorch

PyTorch

Datasets, Transforms and Models specific to Computer Vision

https://github.com/pytorch/vision/

Convert torch to pytorch

https://github.com/clcarwin/convert_torch_to_pytorch

TensorFlow

TensorFlow

Benchmarks

TensorDebugger (TDB)

TensorDebugger(TDB): Interactive, node-by-node debugging and visualization for TensorFlow

ofxMSATensorFlow: OpenFrameworks addon for Google’s data-flow graph based numerical computation / machine intelligence library TensorFlow.

TFLearn: Deep learning library featuring a higher-level API for TensorFlow

TensorFlow on Spark

TensorBoard

TensorFlow.jl: A Julia wrapper for the TensorFlow Python library

TensorLayer: Deep learning and Reinforcement learning library for TensorFlow

OpenCL support for TensorFlow

Pretty Tensor: Fluent Networks in TensorFlow

Rust language bindings for TensorFlow

TensorFlow Ecosystem: Integration of TensorFlow with other open-source frameworks

Caffe to TensorFlow

TensorFlow Mobile

https://www.tensorflow.org/mobile/

Papers

TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems

TensorFlow: A system for large-scale machine learning

TensorFlow Distributions

https://arxiv.org/abs/1711.10604

Tutorials

TensorFlow 官方文档中文版

Theano

Theano

Theano-Tutorials: Bare bones introduction to machine learning from linear regression to convolutional neural networks using Theano

Theano: A Python framework for fast computation of mathematical expressions

Configuring Theano For High Performance Deep Learning

http://www.johnwittenauer.net/configuring-theano-for-high-performance-deep-learning/

Theano: a short practical guide

Ian Goodfellow’s Tutorials on Theano

Plato: A library built on top of Theano

Theano Windows Install Guide

Theano-MPI: a Theano-based Distributed Training Framework

tiny-dnn (tiny-cnn)

tiny-dnn: A header only, dependency-free deep learning framework in C++11

Deep learning with C++ - an introduction to tiny-dnn

Torch

Torch

loadcaffe: Load Caffe networks in Torch7

Applied Deep Learning for Computer Vision with Torch

pytorch: Python wrappers for torch and lua

Torch Toolbox: A collection of snippets and libraries for Torch

cltorch: a Hardware-Agnostic Backend for the Torch Deep Neural Network Library, Based on OpenCL

Torchnet: An Open-Source Platform for (Deep) Learning Research

THFFmpeg: Torch bindings for FFmpeg (reading videos only)

caffegraph: Load Caffe networks in Torch7 using nngraph

Optimized-Torch: Intel Torch is dedicated to improving Torch performance when running on CPU

Torch Video Tutorials

Torch in Action

VELES

VELES: Distributed platform for rapid Deep learning application development

WebDNN

WebDNN: Fastest DNN Execution Framework on Web Browser

Yann

Yann: Yet Another Neural Network Toolbox

Benchmarks

Easy benchmarking of all publicly accessible implementations of convnets

https://github.com/soumith/convnet-benchmarks

Stanford DAWN Deep Learning Benchmark (DAWNBench) - An End-to-End Deep Learning Benchmark and Competition

http://dawn.cs.stanford.edu/benchmark/index.html

Tutorials

Deep Learning Implementations and Frameworks (DLIF)

Papers

Comparative Study of Deep Learning Software Frameworks

Benchmarking State-of-the-Art Deep Learning Software Tools

Projects

TensorFuse: Common interface for Theano, CGT, and TensorFlow

DeepRosetta: An universal deep learning models conversor

Deep Learning Model Convertors

https://github.com/ysh329/deep-learning-model-convertor

References

Frameworks and Libraries for Deep Learning

http://creative-punch.net/2015/07/frameworks-and-libraries-for-deep-learning/

TensorFlow vs. Theano vs. Torch

https://github.com/zer0n/deepframeworks/blob/master/README.md

Evaluation of Deep Learning Toolkits

https://github.com/zer0n/deepframeworks/blob/master/README.md

Deep Machine Learning libraries and frameworks

https://medium.com/@abduljaleel/deep-machine-learning-libraries-and-frameworks-5fdf2bb6bfbe#.q1mhj7c36

Torch vs Theano

Deep Learning Software: NVIDIA Deep Learning SDK

https://developer.nvidia.com/deep-learning-software

A comparison of deep learning frameworks

TensorFlow Meets Microsoft’s CNTK

Is there a case for still using Torch, Theano, Brainstorm, MXNET and not switching to TensorFlow?

  • reddit: [https://www.reddit.com/r/MachineLearning/comments/47qh90/is_there_a_case_for_still_using_torch_theano/][https://www.reddit.com/r/MachineLearning/comments/47qh90/is_there_a_case_for_still_using_torch_theano/]

DL4J vs. Torch vs. Theano vs. Caffe vs. TensorFlow

http://deeplearning4j.org/compare-dl4j-torch7-pylearn.html

Popular Deep Learning Libraries

The simple example of Theano and Lasagne super power

https://grzegorzgwardys.wordpress.com/2016/05/15/the-simple-example-of-theano-and-lasagne-super-power/

Comparison of deep learning software

A Look at Popular Machine Learning Frameworks

5 Deep Learning Projects You Can No Longer Overlook

Comparison of Deep Learning Libraries After Years of Use

Deep Learning Part 1: Comparison of Symbolic Deep Learning Frameworks

Deep Learning Frameworks Compared

DL4J vs. Torch vs. Theano vs. Caffe vs. TensorFlow

https://deeplearning4j.org/compare-dl4j-torch7-pylearn.html

Deep Learning frameworks: a review before finishing 2016

https://medium.com/@ricardo.guerrero/deep-learning-frameworks-a-review-before-finishing-2016-5b3ab4010b06#.a6fdrqssl

The Anatomy of Deep Learning Frameworks

https://medium.com/@gokul_uf/the-anatomy-of-deep-learning-frameworks-46e2a7af5e47

Python Deep Learning Frameworks Reviewed

https://indico.io/blog/python-deep-learning-frameworks-reviewed/

Apple’s deep learning frameworks: BNNS vs. Metal CNN

http://machinethink.net/blog/apple-deep-learning-bnns-versus-metal-cnn/

Published: 09 Oct 2015

About me

Hi world~

Recent Posts

Links

Hoangdacviet © 2021 Powered by Kami Designed by Kome Code by SvTre.com LLC Protected By DMCA