machine_learning.md

• Michael Jordan

  • Искусственный интеллект, большие данные и дезинформация технологий: интервью профессора Беркли
    • AMA: Michael I Jordan
    • Big Data, Hype, the Media and Other Provocative Words to Put in a Title
    • a goodreads list of books for ML
    • 5 free e-books for machine learning mastery

• fresh Deep Learning book MIT press

• SETTING UP EOS FREYA AND ANACONDA ON A HYBRID GRAPHICS LAPTOP FOR GPU ACCELERATED DEEP LEARNING

• Bayesian Machine Learning, Explained

• Deeplearning4J: Spark + GPU

  • Deeplearning4J is an Apache2 Licensed open-sourced, distributed neural net library written in Java and Scala.
    • Deeplearning4J: Neural Net Platform
    • spark-gpu-examples
    • Scientific Computing for the JVM (NDArrays)
      • ND4J: N-Dimensional Arrays for Java

• Recurrent Neural Networks

  • Supervised Sequence Labelling with Recurrent Neural Networks
  • Alex Graves
  • Recurrent Neural Networks (RNN) Explained — the ELI5 way

• Boltzmann Machines

• Deep Learning

  • MIT 6.S191: Introduction to Deep Learning
  • Tutorials, assignments, and competitions for MIT Deep Learning related courses
  • Architecture comparison of AlexNet, VGGNet, ResNet, Inception, DenseNet
  • Deep Learning Frameworks: Choose The Best Fit For You
  • Dive into Deep Learning
  • Deep Learning: Methods and Applications
  • The Anatomy of Deep Learning Frameworks
  • Getting Started with Deep Learning
  • Information Theory of Deep Learning. Naftali Tishby
  • DeepPavlov: An open-source library for end-to-end dialogue systems and chatbots
  • Facebook TransCoder
    • Facebook AI Open-Sourced ‘TransCoder’: A Deep Learning Based Self-Supervised Neural Transcompiler System
  • A new open source framework for automatic differentiation with graphs

• Nervana Systems

  • Nervana
    • github repo for Nervana Systems
  • Nervana's python based Deep Learning Framework http://neon.nervanasys.com/
  • NervanaSystems/nervanagpu: Nervana GPU library link is dead
    • rehosting the removed nervanagpu
  • custom form of cuda-convnet2
    • original cuda-convnet2
  • Assembler for NVIDIA Maxwell architecture
    • open source reverse engineering of Maxwell scheduler hints

• XGBoost : Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library, for Python, R, Java, Scala, C++ and more. Runs on single machine, Hadoop, Spark, Flink and DataFlow

  • XGBoost: A Scalable Tree Boosting System
  • Rust bindings for the XGBoost gradient boosting library

• Misc

  • cheat sheets

    • Cheat Sheets for AI, Neural Networks, Machine Learning, Deep Learning & Big Data

  • Regularization techniques for Neural Networks

  • Обзор материалов по машинному обучению

    • Обзор материалов по машинному обучению (13 — 20 февраля 2017 года)
    • Обзор материалов по машинному обучению № 2 (21 — 27 февраля 2017 года)
    • Обзор материалов по машинному обучению № 3 (16 — 23 апреля 2018 года)
    • Большая подборка материалов по машинному обучению: книги, видеокурсы, онлайн-курсы

  • Which GPU(s) to Get for Deep Learning: My Experience and Advice for Using GPUs in Deep Learning

  • The Brain vs Deep Learning Part I: Computational Complexity — Or Why the Singularity Is Nowhere Near

  • https://medium.com/deep-learning-101/how-to-create-a-mind-the-secret-of-human-thought-revealed-6211bbdb092a

  • An Intuitive Explanation of Convolutional Neural Networks

    • A Quick Introduction to Neural Networks

  • Convolutional Neural Networks backpropagation: from intuition to derivation

  • Sirajology videos

    • TensorFlow in 5 Minutes
    • Build a Recurrent Neural Net in 5 Min

  • colah's blog

    • https://christopherolah.wordpress.com/

    • Understanding LSTM Networks

      • Brandon Rohrer videos on ML
        • How convolutional neural networks work, in depth
        • How do Convolutional Neural Networks work? Data Science and Robots Blog
        • A Beginner's Guide to Convolutional Neural Networks

    • Calculus on Computational Graphs: Backpropagation

      • Stalingrad Home Page
        • dysvunctional-language

    • Deep Learning, NLP, and Representations

    • Neural Networks, Manifolds, and Topology

    • Understanding Convolutions

    • Neural Networks, Types, and Functional Programming

    • Visualizing Representations: Deep Learning and Human Beings

    • Fanfiction, Graphs, and PageRank

    • Visual Information Theory

    • How to Re-Invent the Cognitive Stack of Intelligence

  • Andrej Karpathy blog

    • Efficient Image Captioning code in Torch, runs on GPU
      • Watch a neural network describe what it sees on a stroll through Amsterdam
    • The Unreasonable Effectiveness of Recurrent Neural Networks
      • Multi-layer Recurrent Neural Networks (LSTM, GRU, RNN) for character-level language models in Torch

  • MindSpore

    • MindSpore is a new open source deep learning training/inference framework that could be used for mobile, edge and cloud scenarios
    • MindSpore github mirror
      • Myia is a new differentiable programming language. It aims to support large scale high performance computations (e.g. linear algebra) and their gradients. The main application Myia aims to support is research in artificial intelligence, in particular deep learning algorithms.

  • Torch is a scientific computing framework with wide support for machine learning algorithms

    • torch/torch7
      • Autograd automatically differentiates native Torch code. Ported to Lua from the original Python version.
      • Differentiable Programming: Computing source-code derivatives
    • Torch Cheatsheet
    • Learn Lua in 15 Minutes
    • cutorch install failure with ubuntu 15.10 & cuda 7.5.18
      • python-zmq is required for iTorch
    • Gaussian Processes for Torch7
    • Generating Faces with Torch
    • Net2Net: Accelerating Learning via Knowledge Transfer
    • Training and investigating Residual Nets

  • TensorFlow - Google’s latest machine learning system, open sourced for everyone

    • TensorFlow

      • TensorFlow: A primer
      • Learning Artificial Intelligence(AI) and Tensorflow Without a PhD by Google's Martin Görner
      • TensorFlow Tutorial: 10 minutes Practical TensorFlow lesson for quick learners
      • Awesome TensorFlow
      • TensorFlow white paper

    • Tensorflow 2.0

      • Google launches TensorFlow 2.0 alpha with fewer APIs
      • TensorFlow.jl: A Julia Front End to the TensorFlow World (TF Dev Summit '19)

    • Distributed TensorFlow training (Google I/O '18)

    • Setup_TensorFlow.md

      • TensorFlow-Examples

      ./configure

        + answer `Y` to CUDA
        + set cudnn version to *7*
        + downgrade `bazel` version to __0.5.4__ from __0.6__
        ```sh
        sudo pacman -U /var/cache/pacman/pkg/bazel-0.5.4-1-x86_64.pkg.tar.xz
        ```
            + a workaround for:
            + [no such package '@local_config_cuda//cuda'](https://github.com/tensorflow/tensorflow/issues/11859)
      

      bazel build --subcommands -c opt --copt=-mavx --copt=-mavx2 --copt=-mfma --copt=-mfpmath=both    --config=cuda //tensorflow/tools/pip_package:build_pip_package
      sudo pip3 install wheel
      bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
      sudo pip3 uninstall tensorflow
      sudo pip3 install /tmp/tensorflow_pkg/tensorflow-1.12.0-cp37-cp37m-linux_x86_64.whl

      • How do I install TensorFlow's tensorboard? tl;dr

      >>> import tensorflow as tf
      Limited tf.compat.v2.summary API due to missing TensorBoard installation

      • CUDA 9RC + cuDNN7
        • Upgrade to CuDNN 7 and CUDA 9
        • compilation error
      • How to find which version of TensorFlow is installed in my system? tl;dr

      python -c 'import tensorflow as tf; print(tf.__version__)'

      • Getting Started r0.12

      $ python
      ...
      >>> import tensorflow as tf
      >>> hello = tf.constant('Hello, TensorFlow!')
      >>> sess = tf.Session()
      >>> print(sess.run(hello))
      Hello, TensorFlow!
      >>> a = tf.constant(10)
      >>> b = tf.constant(32)
      >>> print(sess.run(a + b))
      42
      >>>

      • gpu-accelerated-applications-tensorflow-installation
      • How to compile Tensorflow with SSE4.2 and AVX instructions?
      • Feature request: Please provide AVX2/FMA capable builds

    • sonnet: TensorFlow-based neural network library

      • Google’s DeepMind Is Now Open Sourcing Sonnet

    • How to train a neural network to code by itself ?

      • Generate C code with Tensorflow

    • Deep Recurrent Q-Network

    • Open source software library for numerical computation using data flow graphs

    • A deep Q learning demonstration using Google Tensorflow

    • Simplified interface for TensorFlow (mimicking Scikit Learn)

    • Google’s Tensor2Tensor makes it easier to conduct deep learning experiments

      • T2T: Tensor2Tensor Transformers

    • Обзор примера применения обучения с подкреплением с использованием TensorFlow

    • gvnn: Neural Network Library for Geometric Computer Vision

    • tensorflow_vs_torch

  • Google open-sources natural language understanding tools

    • Announcing SyntaxNet: The World’s Most Accurate Parser Goes Open Source
    • SyntaxNet: Neural Models of Syntax
    • How to build Tensorflow lite shared library?
      • Сборка TensorFlow Lite с помощью CMake

  • Caffe2

    • Caffe2 github

  • Amazon's Deep Scalable Sparse Tensor Network Engine (DSSTNE)

    • Amazon open-sources its own deep learning software, DSSTNE

  • A curated list of speech and natural language processing resources

  • Keras - language modeling

  • Computer Vision

    • A Brief History of Computer Vision (and Convolutional Neural Networks)

      • A Basic Introduction to Separable Convolutions

    • Stanford Course: CS231n: Convolutional Neural Networks for Visual Recognition

      • CS231n: Convolutional Neural Networks for Visual Recognition: Spring 2019
      • Slides for CS231n circa 2017
      • github for CS231n
      • Stanford Computer Vision youtube videos

  • A Beginner's Guide to the Mathematics of Neural Networks

  • Convolutional Neural Networks

    • A Beginner's Guide To Understanding Convolutional Neural Networks
    • A Beginner's Guide To Understanding Convolutional Neural Networks Part 2
    • The 9 Deep Learning Papers You Need To Know About (Understanding CNNs Part 3)
    • Что такое свёрточная нейронная сеть
    • Convolutional Neural Networks cheatsheet
      • CS230 Deep Learning

  • Reinforcement Learning

    • Reinforcement Learning: a comprehensive introduction [Part 0]
    • Reinforcement Learning: a comprehensive introduction [Part 1]
    • Reinforcement Learning: a comprehensive introduction [Part 2]

  • RNN et. al

    • Optimizing RNN performance : Part I: Investigating performance of GPU BLAS Libraries

    • MemN2N for language modeling

    • Recurrent Neural Networks Tutorial, Part 1 – Introduction to RNNs

    • Recurrent Neural Networks Tutorial, Part 2 – Implementing a RNN with Python, Numpy and Theano

    • Recurrent Neural Networks Tutorial, Part 3 – Backpropagation Through Time and Vanishing Gradients

    • Recurrent Neural Network Tutorial, Part 4 – Implementing a GRU/LSTM RNN with Python and Theano

    • Understanding Convolutional Neural Networks for NLP | WildML

    • Top 5 arXiv Deep Learning Papers, Explained

    • thesis: Reinforcement Learning with Recurrent Neural Networks

    • Demystifying LSTM neural networks

    • A Critical Review of Recurrent Neural Networks for Sequence Learning

    • Minimal, clean example of lstm neural network training in python, for learning purposes

    • A Framework for Distributed Deep Learning Layer Design in Python

    • A DSL for deep neural networks, supporting Caffe and Torch http://ajtulloch.github.io/dnngraph

    • Tamp: A Library for Compact Deep Neural Networks with Structured Matrices

      • Tamp github repository
      • Tamp: A Library for Compact Deep Neural Networks with Structured Matrices : pdf

    • Recurrent Neural Networks (RNN) list

    • Convolutional Neural Networks (CNN)

    • Learning Simple Algorithms from Examples

      • Learning Simple Algorithms from Examples github

    • Human-level concept learning through probabilistic program induction Bayesian Program Learning (BPL) model for one-shot learning

      • Байесовская нейронная сеть — потому что а почему бы и нет, черт возьми (часть 1)

    • THE NEURAL NETWORK ZOO

      • Зоопарк архитектур нейронных сетей. Часть 1
      • Зоопарк архитектур нейронных сетей. Часть 2

  • misc.

    • Deep Learning – Introduction to Generative Adversarial Neural Networks (GANs)

    • Is Deep Learning Approaching a Wall?

    • Introducing the Facebook Field Guide to Machine Learning video series

    • Surprise! Neurons are Now More Complex than We Thought!!

    • Innovations of AlphaGo

    • Deep Learning links

    • VK: Библиотеки реализующие алгоритмы Deep Learning Nikolay Sergievsky

    • FB link on ML

    • Сравнение библиотек глубокого обучения на примере задачи классификации рукописных цифр

      • Torch vs Theano
      • Theano: A Python framework for fast computation of mathematical expressions

    • What is the intuition behind the concept of capsules in deep learning?

  • A Gentle Introduction to Bayes Theorem for Machine Learning

    • Which Model to Use for Cortical Spiking Neurons?

    • DeepTracking: Seeing Beyond Seeing Using Recurrent Neural Networks

    • Model-Based Machine Learning, Free Early Book Draft

    • Recent Advances in Face Recognition

    • An Introduction to Machine Learning Theory and Its Applications: A Visual Tutorial with Examples

      • An Introduction to Machine Learning Theory and Its Applications: A Visual Tutorial with Examples

    • An Algorithmic God -- Gregory Chaitin

    • Homomorphic machine learning; in Haskell

      • Mike Izbicki' blog

    • RM-SORN: a reward-modulated self-organizing recurrent neural network

    • Beginning studies on deep learning

    • Baidu Research: warp-ctc (https://github.com/baidu-research/warp-ctc)

    • tiny-dnn — анонс библиотеки

    • GeNN: a code generation framework for accelerated brain simulations

      • GeNN: a code generation framework for accelerated brain simulations - Scientific Reports 6, Article number: 18854
        • GeNN is a GPU enhanced Neuronal Network simulation environment based on NVIDIA CUDA technology.
          • gethub: GeNN is a GPU enhanced Neuronal Network simulation environment based on Nvidia CUDA technology.

    • Deep Visualization Toolbox video

      • Recurrent Neural Networks (RNN) and Long Short-Term Memory (LSTM)

    • DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs

    • A cognitive neural architecture able to learn and communicate through natural language

      • another paper link for cognitive neural architecture...
      • annabell arxiv link
      • golosio/annabell - github
      • A cognitive neural architecture able to learn and communicate through natural language - written in CUDA

    • reddit Yann LeCun

    • CS231n: Convolutional Neural Networks for Visual Recognition

      • notes for CS231n

    • An Introduction to Machine Learning Theory and Its Applications: A Visual Tutorial with Examples

    • Book:Machine Learning – The Complete Guide

    • Oxford Computer Science - Machine Learning 2015

      • Oxford Machine Learning: 2014-2015

    • Anyone Can Learn To Code an LSTM-RNN in Python (Part 1: RNN) Baby steps to your neural network's first memories.

    • Learning Simple Algorithms from Examples

    • Samsung DL framework

    • IBM's SystemML Machine Learning - Now Apache SystemML

      • Apache SystemML is a distributed and declarative machine learning platform
      • IBM open-sources machine learning SystemML

    • Deconvolutional Neural Network

    • Unitary Evolution Recurrent Neural Networks

    • On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models

      • How to Learn an Algorithm - Jürgen Schmidhuber

    • Fast Training of Convolutional Networks through FFTs

    • Artificial General Intelligence

    • Deep Reinforcement Learning Papers

    • UCLA researchers release open source code for powerful image detection algorithm

      • JalaliLabUCLA/Image-feature-detection-using-Phase-Stretch-Transform

    • Deep Visual Analogy-Making

      • visual-analogy-tensorflow

    • wiki on ML

    • Deep Learning Software : NVIDIA list

    • Gary Marcus interview

    • New neural network building block allows faster and more accurate text understanding

    • PathNet is a network of neural networks

  • Rust based projects

    • Open Machine Intelligence Framework written in Rust targeting CPU/GPU http://autumnai.github.io/leaf

      • Crate leaf
      • Blob: Machine Learning http://autumnai.github.io/phloem
        • Crate phloem
      • Examples for the Deep Learning Framework Leaf
      • Preprocessing: Machine Learning http://autumnai.github.io/cuticula
        • Crate cuticula
      • Collenchyma library for full Neural Network algorithm support
        • Collenchyma library for full Neural Network algorithm support
        • collenchyma-BLAS

    • rustlearn

    • cudnn

    • Evaluation of Deep Learning Toolkits

    • Deep Learning in Rust: baby steps

    • Deep Learning in Rust: a walk in the park

• Dlib

  • Dlib C++ Library
  • Dlib Blog
  • A Clean C++11 Deep Learning API

• NeoML is an end-to-end machine learning framework that allows you to build, train, and deploy ML models

  • ABBYY open sources NeoML

• FPGA for AI

  • quora: FPGAs-useful-in-neural-networks
    • Recurrent Neural Networks Hardware Implementation on FPGA

• bleeding rnn+yolo

  • You Only Look Once -- YOLO
    • YOLO: Real-Time Object Detection
      • a paper: You Only Look Once:Unified, Real-Time Object Detection
      • [YOLOv3: An Incremental Improvement](YOLO: Real-Time Object Detection)
  • Neural language notes
  • neural-network-papers
  • Awesome Recurrent Neural Networks
  • Attention and Augmented Recurrent Neural Networks
    • augmented-rnns github
    • The Wild Week in AI
  • 20+ hottest research papers on Computer Vision, Machine Learning
  • Whetstone
    • Spiking Software Streamlines AI on Neural Networks
      • Getting Started for Whetstone
      • Whetstone
      • Whetstone: A Method for Training Deep Artificial Neural Networks for Binary Communication

• Brain simulators

  • Neocortical Simulator: An extensible neural simulator for heterogeneous clusters
    • NCS site
    • GENESIS
    • simulators list

• The Hard Problem

  • Scientists Just Proved These Two Brain Networks Are Key to Consciousness
    • Обнаружены две нейронные сети, отвечающие за сознание
  • Нейроморфные вычисления и их успехи
    • NEURON 8.0 RELEASED
    • The Neural Simulation Tool - NEST
  • Philosophical pop science
    • Can Integrated Information Theory Explain Consciousness?
  • Cognitive Architectures

    • LMU - Legendre Memory Unit

      • Legendre Memory Units: Continuous-Time Representation in Recurrent Neural Network
        • [R] Legendre Memory Units: Continuous-Time Representation in Recurrent Neural Networks (NeurIPS2019 Spotlight)
        • Legendre Memory Units: Continuous-Time Representation in Recurrent Neural Network
        • KerasLMU: Recurrent neural networks using Legendre Memory Units
        • PyTorch Legendre Memory Unit

    • The MicroPsi Project

      • Python version of cognitive architecture MicroPsi

    • Numenta (HTM): creating applications

    • Spaun, the most realistic artificial human brain yet

      • Repository of Neural and Cognitive Models
      • The Nengo Neural Simulator
        • A Python library for creating and simulating large-scale brain models : github repo

    • gkassel/pyaixi

    • Joel Veness Research Scientist, Google DeepMind

    • How to build a General Intelligence: Circuits and Pathways

    • Using category theory to assess the relationship between consciousness and integrated information theory

    • From Computation to Consciousness: Can AI reveal the nature of our minds?

    • Why can’t the world’s greatest minds solve the mystery of consciousness?

    • Consciousness and AI

    • Brain Computation Is Organized via Power-of-Two-Based Permutation Logic

      • HTM Forum: Power-of-Two-Based Permutation Logic
      • ycombinator: Brain Computation Is Organized via Power-of-Two-Based Permutation Logic
      • Intelligence May Stem From a Basic Algorithm in the Human Brain
      • One Simple Algorithm Could Explain Human Intelligence

    • From the Phenomenology to the Mechanisms of Consciousness: Integrated Information Theory 3.0

    • Cliques of Neurons Bound into Cavities Provide a Missing Link between Structure and Function

      • The Human Brain Builds Structures in 11 Dimensions, Discover Scientists

    • Huggingface aka Hugging Face

      • Candle: A New Machine Learning Framework for Rust
      • Instruction-tuning Stable Diffusion with InstructPix2Pix

    • Books

      • Mathematics for Machine Learning
      • Deep Learning books
        • Mathematics for Machine Learning by Garrett Thomas terse, precise, and concise
        • MIT Deep Learning Book in PDF format
        • An Algebraic Perspective on Deep Learning
        • The Neural Ring: Using Algebraic Geometry to Analyze Neural Codes
          • The Neural Ring: Using Algebraic Geometry to Analyze Neural Codes
      • Artificial Cognitive Systems: A Primer
        • Artificial Cognitive Systems by Prof. David Vernon
      • The Geometry of Meaning: Semantics Based on Conceptual Spaces
      • Unifying the Mind: Cognitive Representations as Graphical Models
      • Brain Computation as Hierarchical Abstraction
      • Explaining the Computational Mind
      • The Cognitive-Emotional Brain: From Interactions to Integration
      • 10 Free Must-Read Books for Machine Learning and Data Science

    • Courses

      • The Neuronal Basis of Consciousness Course: A Free Online Course from Caltech
        • The Neuronal Basis of Consciousness Course - CNS/Bi/Psy 120

    • Misc

      • Noam Chomsky on the Future of Deep Learning

        • ycombinator: Noam Chomsky on the Future of Deep Learning
          • Merge (linguistics)

      • Abstraction and Reasoning Challenge

        • ARC Competition : EDA + PyTorch CNN

      • Top 20 Machine Learning Interview Questions & Answers

      • TensorFlow or Keras? Which one should I learn?

      • Xception: компактная глубокая нейронная сеть

        • Transfer learning with Xception architecture in Keras

      • Archlinux PKGBUILDs for Data Science, Machine Learning, Deep Learning, NLP and Computer Vision

      • Embedded Machine Learning Design for Dummies

      • Find The Most Updated and Free Artificial Intelligence, Machine Learning, Data Science, Deep Learning, Mathematics, Python, R Programming Resources. (Last Update: October 2, 2019)

        • Python code for ArtificialIntelligence: Foundations ofComputational Agents

      • GitHub Releases Dataset of Six Million Open-Source Methods for Code Search Research

        • CodeSearchNet
        • A new model and dataset for long-range memory

      • Boosting Machine Learning Performance With Rust

• AMD GPU deeplearning

  • Train neural networks using AMD GPU and Keras
  • AMD GPU support for AI

• A collection of research papers on decision, classification and regression trees with implementations

• Fourier Neural Operator for Parametric Partial Differential Equations

• 18.337J/6.338J: Parallel Computing and Scientific Machine Learning

• DeepFly3D is a PyTorch and PyQT5 implementation of 2D-3D tethered Drosophila pose estimation

• Posit-арифметика: победа над floating point на его собственном поле. Часть 1

• Posit-арифметика: победа над floating point на его собственном поле. Часть 2