Home Tech blog Machine Learning – How to Do It

Machine Learning – How to Do It

Michał Drożdż
11.04.2019

We have updated this text for you!
Update date: 31.12.2024
Author of the update: Tadeusz Sławik

Machine learning (ML) is widely and successfully used nowadays. For instance, ML models power recommendations on platforms like Netflix and Amazon, enable facial recognition in modern smartphones, and drive innovations in healthcare diagnostics. While it might seem that computers “know” things, they actually compute. All they do is operate on numbers. We teach them how to “know” by defining what to compute and how to process data.

Introduction

Are you wondering what machine learning really is? Machine learning is a system that includes an algorithm designed to transform (classify) given input into a corresponding label or class (output).

How does classification work internally? When an ML system receives an input sample, it extracts a feature vector (a vector of numbers) from the input and determines which of the defined classes (labels) this vector most likely belongs to.

This article provides an overview of the development process for a machine learning (ML) system, with a particular focus on supervised learning. The goal is to introduce concepts that can help you start exploring ML.

Definitions

Let’s begin by discussing some fundamental terms:

  • Learning (training) set: A collection of artifacts (e.g., images, sound samples, statistics) that include true positive and true negative examples. This set is used to train the system by providing the feature vector for each artifact and the expected output. The system learns which feature values correspond to which outputs.
  • Validation set: A set of artifacts used to fine-tune the hyperparameters of the trained model. This step is essential for many methods to avoid overfitting and ensure generalization.
  • Testing set: A set of artifacts used to evaluate how well the system has been trained. Each input is classified, and the system’s predictions are compared with the expected outputs to assess performance.
  • Feature vector: A numerical representation of an artifact’s characteristics. It captures the properties of the artifact in a structured format that the algorithm can process.
  • Feature descriptor (or extractor): A method for converting an artifact into a feature vector. This process translates the artifact into a numerical representation suitable for analysis.
  • Classification algorithm: An algorithm that determines which feature vector corresponds to which label or output class.
  • Overfitting: A situation where the model perfectly fits the training dataset but performs poorly on unseen data. Overfitting can be mitigated through techniques such as regularization, data augmentation, and cross-validation.

How to Build an ML System

Step 1: Gather Data

The first step is collecting suitable samples for training, validation, and testing sets. The quality and relevance of the data are critical. For example:

  • If you’re building a speech recognition system for telephone calls, gather samples from actual phone conversations.
  • If you’re developing a system to detect behaviors in groups, record real group interactions instead of staged scenarios.

It’s important to collect both positive examples (containing the behavior or object you want to detect) and negative examples (that do not contain it).

Step 2: Preprocess the Data

Before extracting features, preprocessing can improve data quality. This step might include:

  • Noise reduction for audio.
  • Grayscale conversion or normalization for images.
  • Tokenization and stemming for text.

Step 3: Extract Features

Feature extraction translates raw data into meaningful representations. For example, in text analysis, feature extraction might involve counting word frequencies, generating word embeddings like Word2Vec, or calculating sentiment scores. In audio processing, it could involve extracting features such as pitch, frequency spectra, or mel-frequency cepstral coefficients (MFCCs) commonly used in speech recognition. Simple features for images might include:

  • Number of black and white pixels.
  • Ratio of black to white pixels.
  • Number of corners or edges.
  • Maximum/minimum/average dimensions.

Advanced feature descriptors include:

  • HOG (Histogram of Oriented Gradients)
  • LBP (Local Binary Patterns)
  • GLCM (Gray Level Co-occurrence Matrix)

In modern ML systems, especially those using deep learning, feature extraction is often automated within neural network layers.

Step 4: Choose and Apply a Classification Algorithm

Select an algorithm based on your data and problem. Some common algorithms include:

  • Support Vector Machine (SVM)
  • K Nearest Neighbors (KNN)
  • Artificial Neural Networks (ANN)
  • Independent Component Analysis 
  • Ensemble Methods (e.g., Random Forest, Gradient Boosting)

For complex problems, deep learning techniques, such as convolutional neural networks (CNNs) or transformers, are increasingly popular.

Step 5: Train and Evaluate the Model

Train the model using the training set, fine-tune it using the validation set, and finally evaluate it with the testing set. Remember:

  • A perfect 100% accuracy on the test set is likely a sign of overfitting. Always aim for a balance between accuracy and generalization.
  • Use metrics such as precision, recall, F1 score, and confusion matrices to understand model performance.

Step 6: Iterate and Optimize

Experiment with different combinations of features, algorithms, and preprocessing techniques. Incorporate domain knowledge and research to refine your approach.

Modern Tools and Frameworks

Several frameworks can help you develop ML systems:

  • TensorFlow: A popular open-source framework supporting Python and C++ (first released in 2015). TensorBoard provides excellent visualization tools.
  • PyTorch: A widely used framework known for its flexibility and user-friendly dynamic computation graph (introduced by Facebook in 2016).
  • Scikit-learn: Ideal for beginners and classical ML algorithms.
  • JAX: A newer framework designed for high-performance ML research.

Emerging Topics in ML

  1. Transfer Learning: Reusing pre-trained models to solve new problems with less data. For example, a model trained on ImageNet can be fine-tuned to classify medical images.
  2. Explainable AI (XAI): Understanding model decisions using tools like SHAP (SHapley Additive exPlanations) or LIME (Local Interpretable Model-Agnostic Explanations). These tools can illustrate why a model predicted a certain output, improving trust and debugging.
  3. Ethical AI: Ensuring fairness, transparency, and accountability in ML systems. For instance, identifying and mitigating bias in recruitment algorithms.
  4. Self-Supervised Learning: Leveraging unlabeled data for training by generating pseudo-labels. Techniques like contrastive learning are commonly used in domains like computer vision and natural language processing.
  5. Generative AI: Applications such as GPT (for generating human-like text) and Stable Diffusion (for creating detailed images from text descriptions), showcasing creativity and innovation across industries.

Conclusion

Machine Learning is a powerful and evolving tool that can automate processes, enhance decision-making, and solve complex problems. Success depends on high-quality data, careful algorithm selection, and thoughtful evaluation. By staying updated on modern tools and methodologies, you can build effective ML systems that adapt to real-world challenges.

References

Meet the geek-tastic people, and allow us to amaze you with what it's like to work with j‑labs!

Contact us