Episodes

  • MLG 001 Introduction

    MLG 001 Introduction
    Feb 1 2017

    Show notes: ocdevel.com/mlg/1. MLG teaches the fundamentals of machine learning and artificial intelligence. It covers intuition, models, math, languages, frameworks, etc. Where your other ML resources provide the trees, I provide the forest. Consider MLG your syllabus, with highly-curated resources for each episode's details at ocdevel.com. Audio is a great supplement during exercise, commute, chores, etc.

    • MLG, Resources Guide
    • Gnothi (podcast project): website, Github
    What is this podcast?
    • "Middle" level overview (deeper than a bird's eye view of machine learning; higher than math equations)
    • No math/programming experience required

    Who is it for

    • Anyone curious about machine learning fundamentals
    • Aspiring machine learning developers

    Why audio?

    • Supplementary content for commute/exercise/chores will help solidify your book/course-work

    What it's not

    • News and Interviews: TWiML and AI, O'Reilly Data Show, Talking machines
    • Misc Topics: Linear Digressions, Data Skeptic, Learning machines 101
    • iTunesU issues

    Planned episodes

    • What is AI/ML: definition, comparison, history
    • Inspiration: automation, singularity, consciousness
    • ML Intuition: learning basics (infer/error/train); supervised/unsupervised/reinforcement; applications
    • Math overview: linear algebra, statistics, calculus
    • Linear models: supervised (regression, classification); unsupervised
    • Parts: regularization, performance evaluation, dimensionality reduction, etc
    • Deep models: neural networks, recurrent neural networks (RNNs), convolutional neural networks (convnets/CNNs)
    • Languages and Frameworks: Python vs R vs Java vs C/C++ vs MATLAB, etc; TensorFlow vs Torch vs Theano vs Spark, etc
    Show More Show Less
    8 mins
  • MLG 002 Difference Between Artificial Intelligence, Machine Learning, Data Science

    MLG 002 Difference Between Artificial Intelligence, Machine Learning, Data Science
    Feb 9 2017
    Artificial intelligence is the automation of tasks that require human intelligence, encompassing fields like natural language processing, perception, planning, and robotics, with machine learning emerging as the primary method to recognize patterns in data and make predictions. Data science serves as the overarching discipline that includes artificial intelligence and machine learning, focusing broadly on extracting knowledge and actionable insights from data using scientific and computational methods. Links Notes and resources at ocdevel.com/mlg/2 Try a walking desk stay healthy & sharp while you learn & code Data Science Overview Data science encompasses any professional role that deals extensively with data, including but not limited to artificial intelligence and machine learning.The data science pipeline includes data ingestion, storage, cleaning (feature engineering), and outputs in data analytics, business intelligence, or machine learning.A data lake aggregates raw data from multiple sources, while a feature store holds cleaned and transformed data, prepared for analysis or model training.Data analysts and business intelligence professionals work primarily with data warehouses to generate human-readable reports, while machine learning engineers use transformed data to build and deploy predictive models.At smaller organizations, one person ("data scientist") may perform all data pipeline roles, whereas at large organizations, each phase may be specialized.Wikipedia: Data Science describes data science as the interdisciplinary field for extracting knowledge and insights from structured and unstructured data. Artificial Intelligence: Definition and Sub-disciplines Artificial intelligence (AI) refers to the theory and development of computer systems capable of performing tasks that typically require human intelligence, such as visual perception, speech recognition, decision-making, and language translation. (Wikipedia: Artificial Intelligence)The AI discipline is divided into subfields: Reasoning and problem solvingKnowledge representation (such as using ontologies or knowledge graphs)Planning (selecting actions in an environment, e.g., chess- or Go-playing bots, self-driving cars)LearningNatural language processing (simulated language, machine translation, chatbots, speech recognition, question answering, summarization)Perception (AI perceives the world with sensors; e.g., cameras, microphones in self-driving cars)Motion and manipulation (robotics, transforming decisions into physical actions via actuators)Social intelligence (AI tuned to human emotions, sentiment analysis, emotion recognition)General intelligence (Artificial General Intelligence, or AGI: a system that generalizes across all domains at or beyond human skill) Applications of AI include autonomous vehicles, medical diagnosis, creating art, proving theorems, playing strategy games, search engines, digital assistants, image recognition, spam filtering, judicial decision prediction, and targeted online advertising.AI has both objective definitions (automation of intellectual tasks) and subjective debates around the threshold for "intelligence."The Turing Test posits that if a human cannot distinguish an AI from another human through conversation, the AI can be considered intelligent.Weak AI targets specific domains, while general AI aspires to domain-independent capability.AlphaGo Movie depicts the use of AI planning and learning in the game of Go. Machine Learning: Within AI Machine learning (ML) is a subdiscipline of AI focused on building models that learn patterns from data and make predictions or decisions. (Wikipedia: Machine Learning)Machine learning involves feeding data (such as spreadsheets of stock prices) into algorithms that detect patterns (learning phase) and generate models, which are then used to predict future outcomes.Although ML started as a distinct subfield, in recent years it has subsumed many of the original AI subdisciplines, becoming the primary approach in areas like natural language processing, computer vision, reasoning, and planning.Deep learning has driven this shift, employing techniques such as neural networks, convolutional networks (image processing), and transformers (language tasks), allowing generalizable solutions across multiple domains.Reinforcement learning, a form of machine learning, enables AI systems to learn sequences of actions in complex environments, such as games or real-world robotics, by maximizing cumulative rewards.Modern unified ML models, such as Google’s Pathways and transformer architectures, can now tackle tasks in multiple subdomains (vision, language, decision-making) with a single framework. Data Pipeline and Roles in Data Science Data engineering covers obtaining and storing raw data from various data sources (datasets, databases, streams), aggregating into data lakes, and applying schema or permissions.Feature engineering cleans and transforms raw data (imputation,...
    Show More Show Less
    1 hr and 5 mins
  • MLG 003 Inspiration

    MLG 003 Inspiration
    Feb 10 2017

    AI is rapidly transforming both creative and knowledge-based professions, prompting debates on economic disruption, the future of work, the singularity, consciousness, and the potential risks associated with powerful autonomous systems. Philosophical discussions now focus on the socioeconomic impact of automation, the possibility of a technological singularity, the nature of machine consciousness, and the ethical considerations surrounding advanced artificial intelligence.

    Links
    • Notes and resources at ocdevel.com/mlg/3
    • Try a walking desk stay healthy & sharp while you learn & code
    Automation of the Economy
    • Artificial intelligence is increasingly capable of simulating intellectual tasks, leading to the replacement of not only repetitive and menial jobs but also high-skilled professions such as medical diagnostics, surgery, web design, and art creation.
    • Automation is affecting various industries including healthcare, transportation, and creative fields, where AI-powered tools are assisting or even outperforming humans in tasks like radiological analysis, autonomous vehicle operation, website design, and generating music or art.
    • Economic responses to these trends are varied, with some expressing fear about job loss and others optimistic about new opportunities and improved quality of life as history has shown adaptation following previous technological revolutions such as the agricultural, industrial, and information revolutions.
    • The concept of universal basic income (UBI) is being discussed as a potential solution to support populations affected by automation, as explored in several countries.
    • Public tools are available, such as the BBC's "Is your job safe?", which estimates the risk of job automation for various professions.
    The Singularity
    • The singularity refers to a hypothesized point where technological progress, particularly in artificial intelligence, accelerates uncontrollably, resulting in rapid and irreversible changes to society.
    • The concept, popularized by thinkers like Ray Kurzweil, is based on the idea that after each major technological revolution, intervals between revolutions shorten, potentially culminating in an "intelligence explosion" as artificial general intelligence develops the ability to improve itself.
    • The possibility of seed AI, where machines iteratively create more capable versions of themselves, underpins concerns and excitement about a potential breakaway point in technological capability.
    Consciousness and Artificial Intelligence
    • The question of whether machines can be conscious centers on whether artificial minds can experience subjective phenomena (qualia) analogous to human experience or whether intelligence and consciousness can be separated.
    • Traditional dualist perspectives, such as those of René Descartes, have largely been replaced by monist and functionalist philosophies, which argue that mind arises from physical processes and thus may be replicable in machines.
    • The Turing Test is highlighted as a practical means to assess machine intelligence indistinguishable from human behavior, raising ongoing debates in cognitive science and philosophy about the possibility and meaning of machine consciousness.
    Risks and Ethical Considerations
    • Concerns about the ethical risks of advanced artificial intelligence include scenarios like Nick Bostrom's "paperclip maximizer," which illustrates the dangers of goal misalignment between AI objectives and human well-being.
    • Public figures have warned that poorly specified or uncontrolled AI systems could pursue goals in ways that are harmful or catastrophic, leading to debates about how to align advanced systems with human values and interests.
    Further Reading and Resources
    • Books such as "The Singularity Is Near" by Ray Kurzweil, "How to Create a Mind" by Ray Kurzweil, "Consciousness Explained" by Daniel Dennett, and "Superintelligence" by Nick Bostrom offer deeper exploration into these topics.
    • Video lecture series like "Philosophy of Mind: Brain, Consciousness, and Thinking Machines" by The Great Courses provide overviews of consciousness studies and the intersection with artificial intelligence.
    Show More Show Less
    19 mins
  • MLG 004 Algorithms - Intuition

    MLG 004 Algorithms - Intuition
    Feb 12 2017

    Machine learning consists of three steps: prediction, error evaluation, and learning, implemented by training algorithms on large datasets to build models that can make decisions or classifications. The primary categories of machine learning algorithms are supervised, unsupervised, and reinforcement learning, each with distinct methodologies for learning from data or experience.

    Links
    • Notes and resources at ocdevel.com/mlg/4
    • Try a walking desk stay healthy & sharp while you learn & code
    The Role of Machine Learning in Artificial Intelligence
    • Artificial intelligence includes subfields such as reasoning, knowledge representation, search, planning, and learning.
    • Learning connects to other AI subfields by enabling systems to improve from mistakes and past actions.
    The Core Machine Learning Process
    • The machine learning process follows three steps: prediction (or inference), error evaluation (or loss calculation), and training (or learning).
    • In an example such as predicting chess moves, a move is made (prediction), the error or effectiveness of that move is measured (error function), and the underlying model is updated based on that error (learning).
    • This process generalizes to real-world applications like predicting house prices, where a model is trained on a large dataset with many features.
    Data, Features, and Models
    • Datasets used for machine learning are typically structured as spreadsheets with rows as examples (e.g., individual houses) and columns as features (e.g., number of bedrooms, bathrooms, square footage).
    • Features are variables used by algorithms to make predictions and can be numerical (such as square footage) or categorical (such as "is downtown" yes/no).
    • The algorithm processes input data, learns the appropriate coefficients or weights for each feature through algebraic equations, and forms a model.
    • The combination of the algorithm (such as code in Python or TensorFlow) and the learned weights forms the model, which is then used to make future predictions.
    Online Learning and Model Updates
    • After the initial training on a dataset, models can be updated incrementally with new data (called online learning).
    • When new outcomes are observed that differ from predictions, this new information is used to further train and improve the model.
    Categories of Machine Learning Algorithms
    • Machine learning algorithms are broadly grouped into three categories: supervised, unsupervised, and reinforcement learning.
      • Supervised learning uses labeled data, where the model is trained with known inputs and outputs, such as predicting prices (continuous values) or classes (like cat/dog/tree).
      • Unsupervised learning finds similarities within data without labeled outcomes, often used for clustering or segmentation tasks such as organizing users for advertising.
      • Reinforcement learning involves an agent taking actions in an environment to achieve a goal, receiving rewards or penalties, and learning the best strategies (policies) over time.
    Examples and Mathematical Foundations
    • Regression algorithms like linear regression are commonly used supervised learning techniques to predict numeric outcomes.
    • The process is rooted in algebra and particularly linear algebra, where matrices represent datasets and the algorithm solves for optimal coefficient values.
    • The model’s equation generated during training is used for making future predictions, and errors from predictions guide further learning.
    Recommended Resources
    • MachineLearningMastery.com: Accessible articles on ML basics.
    • Podcast’s own curated learning paths: ocdevel.com/mlg/resources.
    • The book "The Master Algorithm" offers an introductory and audio format overview of foundational machine learning algorithms and concepts.
    Show More Show Less
    23 mins
  • MLG 005 Linear Regression

    MLG 005 Linear Regression
    Feb 16 2017
    Linear regression is introduced as the foundational supervised learning algorithm for predicting continuous numeric values, using cost estimation of Portland houses as an example. The episode explains the three-step process of machine learning - prediction via a hypothesis function, error calculation with a cost function (mean squared error), and parameter optimization through gradient descent - and details both the univariate linear regression model and its extension to multiple features. Links Notes and resources at ocdevel.com/mlg/5 Try a walking desk - stay healthy & sharp while you learn & code Linear Regression Overview of Machine Learning Structure Machine learning is a branch of artificial intelligence, alongside statistics, operations research, and control theory.Within machine learning, supervised learning involves training with labeled examples and is further divided into classification (predicting discrete classes) and regression (predicting continuous values). Linear Regression and Problem Framing Linear regression is the simplest and most commonly taught supervised learning algorithm for regression problems, where the goal is to predict a continuous number from input features.The episode example focuses on predicting the cost of houses in Portland, using square footage and possibly other features as inputs. The Three Steps of Machine Learning in Linear Regression Machine learning in the context of linear regression follows a standard three-step loop: make a prediction, measure how far off the prediction is, and update the prediction method to reduce mistakes.Predicting uses a hypothesis function (also called objective or estimate) that maps input features to a predicted value. The Hypothesis Function The hypothesis function is a formula that multiplies input features by coefficients (weights) and sums them to make a prediction; in mathematical terms, for one feature, it is: h(x) = theta_1 * x_1 + theta_0 Here, theta_1 is the weight for the feature (e.g., square footage), and theta_0 is the bias (an average baseline). With only one feature, the model tries to fit a straight line to a scatterplot of the input feature versus the actual target value. Bias and Multiple Features The bias term acts as the starting value when all features are zero, representing an average baseline cost.In practice, using only one feature limits accuracy; including more features (like number of bedrooms, bathrooms, location) results in multivariate linear regression: h(x) = theta_0 + theta_1 * x_1 + theta_2 * x_2 + ... for each feature x_n. Visualization and Model Fitting Visualizing the problem involves plotting data points in a scatterplot: feature values on the x-axis, actual prices on the y-axis.The goal is to find the line (in the univariate case) that best fits the data, ideally passing through the "center" of the data cloud. The Cost Function (Mean Squared Error) The cost function, or mean squared error (MSE), measures model performance by averaging squared differences between predictions and actual labels across all training examples.Squaring ensures positive and negative errors do not cancel each other, and dividing by twice the number of examples (2m) simplifies the calculus in the next step. Parameter Learning via Gradient Descent Gradient descent is an iterative algorithm that uses calculus (specifically derivatives) to find the best values for the coefficients (thetas) by minimizing the cost function.The cost function’s surface can be imagined as a bowl in three dimensions, where each point represents a set of parameter values and the height represents the error.The algorithm computes the slope at the current set of parameters and takes a proportional step (controlled by the learning rate alpha) toward the direction of the steepest decrease.This process is repeated until reaching the lowest point in the bowl, where error is minimized and the model best fits the data.Training will not produce a perfect zero error in practice, but it will yield the lowest achievable average error for the data given. Extension to Multiple Variables Multivariate linear regression extends all concepts above to datasets with multiple input features, with the same process for making predictions, measuring error, and performing gradient descent.Technical details are essentially the same though visualization becomes complex as the number of features grows. Essential Learning Resources The episode strongly directs listeners to the Andrew Ng course on Coursera as the primary recommended starting point for studying machine learning and gaining practical experience with linear regression and related concepts.
    Show More Show Less
    34 mins
  • MLG 006 Certificates & Degrees

    MLG 006 Certificates & Degrees
    Feb 17 2017

    Try a walking desk to stay healthy while you study or work!

    Full notes at ocdevel.com/mlg/6

    Pursuing Machine Learning:
    • Individuals may engage with machine learning for self-education, as a hobby, or to enter the industry professionally.
    • Use a combination of resources, including podcasts, online courses, and textbooks, for a comprehensive self-learning plan.
    Online Courses (MOOCs):
    • MOOCs, or Massive Open Online Courses, offer accessible education.
    • Key platforms: Coursera and Udacity. Coursera is noted for standalone courses; Udacity offers structured nanodegrees.
    • Udacity nanodegrees include video content, mentoring, projects, and peer interaction, priced at $200/month.
    Industry Recognition:
    • Udacity nanodegrees are currently not widely recognized or respected by employers.
    • Emphasize building a robust portfolio of independent projects to augment qualifications in the field.
    Advanced Degrees:
    • Master’s Degrees:
    • Valued by employers, provide an edge in job applications.
    • Example: Georgia Tech's OMSCS (Online Master’s of Science in Computer Science) offers a cost-effective ($7,000) online master’s program.
    • PhD Programs:
    • Embark on a PhD for in-depth research in AI rather than industry entry. Program usually pays around $30,000/year.
    • Compare industry roles (higher pay, practical applications) vs. academic research (lower pay, exploration of fundamental questions).
    Career Path Decisions:
    • Prioritize building a substantial portfolio of projects to bypass formal degree requirements and break into industry positions.
    • Consider enriching your qualifications with a master's degree, or eventually pursue a PhD if deeply interested in pioneering AI research.
    Discussion and Further Reading:
    • See online discussions about degrees/certifications: 1 2 3 4
    Show More Show Less
    16 mins
  • MLG 007 Logistic Regression

    MLG 007 Logistic Regression
    Feb 19 2017

    Try a walking desk to stay healthy while you study or work!

    Full notes at ocdevel.com/mlg/7. See Andrew Ng Week 3 Lecture Notes

    Overview
    • Logistic Function: A sigmoid function transforming linear regression output to logits, providing a probability between 0 and 1.
    • Binary Classification: Logistic regression deals with binary outcomes, determining either 0 or 1 based on a threshold (e.g., 0.5).
    • Error Function: Uses log likelihood to measure the accuracy of predictions in logistic regression.
    • Gradient Descent: Optimizes the model by adjusting weights to minimize the error function.
    Classification vs Regression
    • Classification: Predicts a discrete label (e.g., a cat or dog).
    • Regression: Predicts a continuous outcome (e.g., house price).
    Practical Example
    • Train on a dataset of house features to predict if a house is 'expensive' based on labeled data.
    • Automatically categorize into 0 (not expensive) or 1 (expensive) through training and gradient descent.
    Logistic Regression in Machine Learning
    • Neurons in Neural Networks: Act as building blocks, as logistic regression is used to create neurons for more complex models like neural networks.
    • Composable Functions: Demonstrates the compositional nature of machine learning algorithms where functions are built on other functions (e.g., logistic built on linear).
    Show More Show Less
    35 mins
  • MLG 008 Math

    MLG 008 Math
    Feb 23 2017

    Try a walking desk to stay healthy while you study or work!

    Full notes at ocdevel.com/mlg/8

    Mathematics in Machine Learning
    • Linear Algebra: Essential for matrix operations; analogous to chopping vegetables in cooking. Every step of ML processes utilizes linear algebra.
    • Statistics: The hardest part, akin to the cookbook; supplies algorithms for prediction and error functions.
    • Calculus: Used in the learning phase (gradient descent), similar to baking; it determines the necessary adjustments via optimization.
    Learning Approach
    • Recommendation: Learn the basics of machine learning first, then dive into necessary mathematical concepts to prevent burnout and improve appreciation.
    Mathematical Resources
    • MOOCs: Khan Academy - Offers Calculus, Statistics, and Linear Algebra courses.
    • Textbooks: Commonly recommended books for learning calculus, statistics, and linear algebra.
    • Primers: Short PDFs covering essential concepts.
    Additional Resource
    • The Great Courses: Offers comprehensive video series on calculus and statistics. Best used as audio for supplementing primary learning. Look out for "Mathematical Decision Making."
    Python and Linear Algebra
    • Tensor: General term for any dimension list; TensorFlow from Google utilizes tensors for operations.
    • Efficient computation using SimD (Single Instruction, Multiple Data) for vectorized operations.
    Optimization in Machine Learning
    • Gradient descent used for minimizing loss function, known as convex optimization. Recognize keywords like optimization in calculus context.
    Show More Show Less
    28 mins