D2L 3.2. Object-Oriented Design for Implementation

3.2. Object-Oriented Design for Implementation — Dive into Deep Learning 1.0.0-beta0 documentation (d2l.ai)

3.2. Object-Oriented Design for Implementation — Dive into Deep Learning 1.0.0-beta0 documentation

oo-design.ipynb - Colaboratory (google.com)

 

In our introduction to linear regression, we walked through various components including the data, the model, the loss function, and the optimization algorithm. Indeed, linear regression is one of the simplest machine learning models. Training it, however, uses many of the same components that other models in this book require. Therefore, before diving into the implementation details it is worth designing some of the APIs that we use throughout. Treating components in deep learning as objects, we can start by defining classes for these objects and their interactions. This object-oriented design for implementation will greatly streamline the presentation and you might even want to use it in your projects.

 

선형 회귀 소개에서는 데이터, 모델, 손실 함수 및 최적화 알고리즘을 포함한 다양한 구성 요소를 살펴보았습니다. 실제로 선형 회귀는 가장 간단한 기계 학습 모델 중 하나입니다. 그러나 이를 훈련하는 데는 이 책의 다른 모델에 필요한 것과 동일한 구성 요소가 많이 사용됩니다. 따라서 구현 세부 사항을 살펴보기 전에 우리가 전체적으로 사용하는 일부 API를 설계하는 것이 좋습니다. 딥 러닝의 구성 요소를 객체로 처리하려면 이러한 객체와 해당 상호 작용에 대한 클래스를 정의하는 것부터 시작할 수 있습니다. 구현을 위한 이 객체 지향 디자인은 프레젠테이션을 크게 간소화하며 프로젝트에서 이를 사용하고 싶을 수도 있습니다.

 

Inspired by open-source libraries such as PyTorch Lightning, at a high level we wish to have three classes: (i) Module contains models, losses, and optimization methods; (ii) DataModule provides data loaders for training and validation; (iii) both classes are combined using the Trainer class, which allows us to train models on a variety of hardware platforms. Most code in this book adapts Module and DataModule. We will touch upon the Trainer class only when we discuss GPUs, CPUs, parallel training, and optimization algorithms.

 

PyTorch Lightning과 같은 오픈 소스 라이브러리에서 영감을 받아 높은 수준에서 우리는 세 가지 클래스를 갖고 싶습니다. (i) 모듈에는 모델, 손실 및 최적화 방법이 포함되어 있습니다. (ii) DataModule은 교육 및 검증을 위한 데이터 로더를 제공합니다. (iii) 두 클래스 모두 Trainer 클래스를 사용하여 결합되어 다양한 하드웨어 플랫폼에서 모델을 교육할 수 있습니다. 이 책에 나오는 대부분의 코드는 Module과 DataModule을 채택합니다. GPU, CPU, 병렬 훈련 및 최적화 알고리즘을 논의할 때만 Trainer 클래스를 다룰 것입니다.

 

D2L 모듈 인스톨이 안 되서 실행 못함

import time
import numpy as np
import torch
from torch import nn
from d2l import torch as d2l

 

3.2.1. Utilities

We need a few utilities to simplify object-oriented programming in Jupyter notebooks. One of the challenges is that class definitions tend to be fairly long blocks of code. Notebook readability demands short code fragments, interspersed with explanations, a requirement incompatible with the style of programming common for Python libraries. The first utility function allows us to register functions as methods in a class after the class has been created. In fact, we can do so even after we have created instances of the class! It allows us to split the implementation of a class into multiple code blocks.

 

Jupyter 노트북에서 객체 지향 프로그래밍을 단순화하려면 몇 가지 유틸리티가 필요합니다. 과제 중 하나는 클래스 정의가 상당히 긴 코드 블록인 경향이 있다는 것입니다. 노트북 가독성을 위해서는 설명이 산재된 짧은 코드 조각이 필요합니다. 이는 Python 라이브러리에 일반적인 프로그래밍 스타일과 호환되지 않는 요구 사항입니다. 첫 번째 유틸리티 함수를 사용하면 클래스가 생성된 후 함수를 클래스의 메서드로 등록할 수 있습니다. 사실, 클래스의 인스턴스를 생성한 후에도 그렇게 할 수 있습니다! 이를 통해 클래스 구현을 여러 코드 블록으로 분할할 수 있습니다.

 

def add_to_class(Class):  #@save
    """Register functions as methods in created class."""
    def wrapper(obj):
        setattr(Class, obj.__name__, obj)
    return wrapper

 

Let’s have a quick look at how to use it. We plan to implement a class A with a method do. Instead of having code for both A and do in the same code block, we can first declare the class A and create an instance a.

 

사용법을 간단히 살펴보겠습니다. 우리는 do 메소드를 사용하여 클래스 A를 구현할 계획입니다. A와 do에 대한 코드를 동일한 코드 블록에 두는 대신 먼저 클래스 A를 선언하고 인스턴스 a를 만들 수 있습니다.

 

class A:
    def __init__(self):
        self.b = 1

a = A()

 

Next we define the method do as we normally would, but not in class A’s scope. Instead, we decorate this method by add_to_class with class A as its argument. In doing so, the method is able to access the member variables of A just as we would expect had it been included as part of A’s definition. Let’s see what happens when we invoke it for the instance a.

 

다음으로 우리는 평소처럼 do 메서드를 정의하지만 클래스 A의 범위에서는 그렇지 않습니다. 대신, 클래스 A를 인수로 사용하여 add_to_class로 이 메소드를 장식합니다. 그렇게 함으로써, 메소드는 A의 정의의 일부로 포함되었을 때 예상했던 것처럼 A의 멤버 변수에 액세스할 수 있습니다. 인스턴스 a에 대해 호출하면 어떤 일이 발생하는지 살펴보겠습니다.

 

@add_to_class(A)
def do(self):
    print('Class attribute "b" is', self.b)

a.do()

 

The second one is a utility class that saves all arguments in a class’s __init__ method as class attributes. This allows us to extend constructor call signatures implicitly without additional code.

 

두 번째는 클래스 속성으로 클래스의 __init__ 메서드에 있는 모든 인수를 저장하는 유틸리티 클래스입니다. 이를 통해 추가 코드 없이 생성자 호출 서명을 암시적으로 확장할 수 있습니다.

 

class HyperParameters:  #@save
    """The base class of hyperparameters."""
    def save_hyperparameters(self, ignore=[]):
        raise NotImplemented

 

 

We defer its implementation into Section 23.7. To use it, we define our class that inherits from HyperParameters and calls save_hyperparameters in the __init__ method.

 

우리는 그 구현을 섹션 23.7로 연기합니다. 이를 사용하려면 HyperParameters에서 상속하고 __init__ 메서드에서 save_hyperparameters를 호출하는 클래스를 정의합니다.

 

# Call the fully implemented HyperParameters class saved in d2l
class B(d2l.HyperParameters):
    def __init__(self, a, b, c):
        self.save_hyperparameters(ignore=['c'])
        print('self.a =', self.a, 'self.b =', self.b)
        print('There is no self.c =', not hasattr(self, 'c'))

b = B(a=1, b=2, c=3)

self.a = 1 self.b = 2
There is no self.c = True

 

The final utility allows us to plot experiment progress interactively while it is going on. In deference to the much more powerful (and complex) TensorBoard we name it ProgressBoard. The implementation is deferred to Section 23.7. For now, let’s simply see it in action.

 

최종 유틸리티를 사용하면 실험이 진행되는 동안 대화식으로 실험 진행 상황을 플롯할 수 있습니다. 훨씬 더 강력하고 복잡한 TensorBoard를 존중하여 우리는 이를 ProgressBoard라고 명명합니다. 구현은 섹션 23.7로 연기됩니다. 지금은 실제로 작동하는 모습을 살펴보겠습니다.

 

The draw method plots a point (x, y) in the figure, with label specified in the legend. The optional every_n smooths the line by only showing 1/n points in the figure. Their values are averaged from the n neighbor points in the original figure.

 

draw 메소드는 범례에 지정된 레이블을 사용하여 그림의 점(x, y)을 플롯합니다. 선택적 Every_n은 그림에 1/n 지점만 표시하여 선을 매끄럽게 만듭니다. 해당 값은 원래 그림의 n개 이웃 점에서 평균을 냅니다.

 

class ProgressBoard(d2l.HyperParameters):  #@save
    """The board that plots data points in animation."""
    def __init__(self, xlabel=None, ylabel=None, xlim=None,
                 ylim=None, xscale='linear', yscale='linear',
                 ls=['-', '--', '-.', ':'], colors=['C0', 'C1', 'C2', 'C3'],
                 fig=None, axes=None, figsize=(3.5, 2.5), display=True):
        self.save_hyperparameters()

    def draw(self, x, y, label, every_n=1):
        raise NotImplemented

 

In the following example, we draw sin and cos with a different smoothness. If you run this code block, you will see the lines grow in animation.

 

다음 예에서는 다른 부드러움으로 sin과 cos를 그립니다. 이 코드 블록을 실행하면 애니메이션에서 줄이 늘어나는 것을 볼 수 있습니다.

 

board = d2l.ProgressBoard('x')
for x in np.arange(0, 10, 0.1):
    board.draw(x, np.sin(x), 'sin', every_n=2)
    board.draw(x, np.cos(x), 'cos', every_n=10)

3.2.2. Models

 

The Module class is the base class of all models we will implement. At the very least we need three methods. The first, __init__, stores the learnable parameters, the training_step method accepts a data batch to return the loss value, and finally, configure_optimizers returns the optimization method, or a list of them, that is used to update the learnable parameters. Optionally we can define validation_step to report the evaluation measures. Sometimes we put the code for computing the output into a separate forward method to make it more reusable.

 

모듈 클래스는 우리가 구현할 모든 모델의 기본 클래스입니다. 최소한 세 가지 방법이 필요합니다. 첫 번째 __init__는 학습 가능한 매개변수를 저장하고, training_step 메서드는 데이터 배치를 받아 손실 값을 반환하며, 마지막으로configure_optimizers는 학습 가능한 매개변수를 업데이트하는 데 사용되는 최적화 메서드 또는 그 목록을 반환합니다. 선택적으로 평가 측정값을 보고하기 위해 유효성 검사 단계를 정의할 수 있습니다. 때때로 우리는 출력을 더 쉽게 재사용할 수 있도록 별도의 전달 메서드에 출력을 계산하는 코드를 넣습니다.

 

class Module(nn.Module, d2l.HyperParameters):  #@save
    """The base class of models."""
    def __init__(self, plot_train_per_epoch=2, plot_valid_per_epoch=1):
        super().__init__()
        self.save_hyperparameters()
        self.board = ProgressBoard()

    def loss(self, y_hat, y):
        raise NotImplementedError

    def forward(self, X):
        assert hasattr(self, 'net'), 'Neural network is defined'
        return self.net(X)

    def plot(self, key, value, train):
        """Plot a point in animation."""
        assert hasattr(self, 'trainer'), 'Trainer is not inited'
        self.board.xlabel = 'epoch'
        if train:
            x = self.trainer.train_batch_idx / \
                self.trainer.num_train_batches
            n = self.trainer.num_train_batches / \
                self.plot_train_per_epoch
        else:
            x = self.trainer.epoch + 1
            n = self.trainer.num_val_batches / \
                self.plot_valid_per_epoch
        self.board.draw(x, value.to(d2l.cpu()).detach().numpy(),
                        ('train_' if train else 'val_') + key,
                        every_n=int(n))

    def training_step(self, batch):
        l = self.loss(self(*batch[:-1]), batch[-1])
        self.plot('loss', l, train=True)
        return l

    def validation_step(self, batch):
        l = self.loss(self(*batch[:-1]), batch[-1])
        self.plot('loss', l, train=False)

    def configure_optimizers(self):
        raise NotImplementedError

 

 

You may notice that Module is a subclass of nn.Module, the base class of neural networks in PyTorch. It provides convenient features for handling neural networks. For example, if we define a forward method, such as forward(self, X), then for an instance a we can invoke this method by a(X). This works since it calls the forward method in the built-in __call__ method. You can find more details and examples about nn.Module in Section 6.1.

 

Module은 PyTorch 신경망의 기본 클래스인 nn.Module의 하위 클래스라는 것을 알 수 있습니다. 신경망 처리에 편리한 기능을 제공합니다. 예를 들어, 전달(self, X)와 같은 전달 메서드를 정의하면 인스턴스 a에 대해 이 메서드를 a(X)로 호출할 수 있습니다. 이는 내장된 __call__ 메서드에서 전달 메서드를 호출하기 때문에 작동합니다. 섹션 6.1에서 nn.Module에 대한 자세한 내용과 예제를 확인할 수 있습니다.

 

 

3.2.3. Data

The DataModule class is the base class for data. Quite frequently the __init__ method is used to prepare the data. This includes downloading and preprocessing if needed. The train_dataloader returns the data loader for the training dataset. A data loader is a (Python) generator that yields a data batch each time it is used. This batch is then fed into the training_step method of Module to compute the loss. There is an optional val_dataloader to return the validation dataset loader. It behaves in the same manner, except that it yields data batches for the validation_step method in Module.

 

DataModule 클래스는 데이터의 기본 클래스입니다. 데이터를 준비하는 데 __init__ 메서드가 자주 사용됩니다. 여기에는 필요한 경우 다운로드 및 전처리가 포함됩니다. train_dataloader는 훈련 데이터 세트에 대한 데이터 로더를 반환합니다. 데이터 로더는 사용될 때마다 데이터 배치를 생성하는 (Python) 생성기입니다. 그런 다음 이 배치는 모듈의 training_step 메서드에 입력되어 손실을 계산합니다. 유효성 검사 데이터 세트 로더를 반환하는 선택적 val_dataloader가 있습니다. Module의 유효성 검사 단계 메서드에 대한 데이터 배치를 생성한다는 점을 제외하면 동일한 방식으로 작동합니다.

 

class DataModule(d2l.HyperParameters):  #@save
    """The base class of data."""
    def __init__(self, root='../data', num_workers=4):
        self.save_hyperparameters()

    def get_dataloader(self, train):
        raise NotImplementedError

    def train_dataloader(self):
        return self.get_dataloader(train=True)

    def val_dataloader(self):
        return self.get_dataloader(train=False)

 

3.2.4. Training

The Trainer class trains the learnable parameters in the Module class with data specified in DataModule. The key method is fit, which accepts two arguments: model, an instance of Module, and data, an instance of DataModule. It then iterates over the entire dataset max_epochs times to train the model. As before, we will defer the implementation of this method to later chapters.

 

Trainer 클래스는 DataModule에 지정된 데이터를 사용하여 Module 클래스의 학습 가능한 매개변수를 교육합니다. 핵심 메소드는 fit입니다. 이는 Module의 인스턴스인 model과 DataModule의 인스턴스인 data라는 두 가지 인수를 허용합니다. 그런 다음 전체 데이터 세트 max_epochs 회를 반복하여 모델을 교육합니다. 이전과 마찬가지로 이 메서드의 구현은 이후 장으로 미루겠습니다.

 

class Trainer(d2l.HyperParameters):  #@save
    """The base class for training models with data."""
    def __init__(self, max_epochs, num_gpus=0, gradient_clip_val=0):
        self.save_hyperparameters()
        assert num_gpus == 0, 'No GPU support yet'

    def prepare_data(self, data):
        self.train_dataloader = data.train_dataloader()
        self.val_dataloader = data.val_dataloader()
        self.num_train_batches = len(self.train_dataloader)
        self.num_val_batches = (len(self.val_dataloader)
                                if self.val_dataloader is not None else 0)

    def prepare_model(self, model):
        model.trainer = self
        model.board.xlim = [0, self.max_epochs]
        self.model = model

    def fit(self, model, data):
        self.prepare_data(data)
        self.prepare_model(model)
        self.optim = model.configure_optimizers()
        self.epoch = 0
        self.train_batch_idx = 0
        self.val_batch_idx = 0
        for self.epoch in range(self.max_epochs):
            self.fit_epoch()

    def fit_epoch(self):
        raise NotImplementedError

 

 

3.2.5. Summary

To highlight the object-oriented design for our future deep learning implementation, the above classes simply show how their objects store data and interact with each other. We will keep enriching implementations of these classes, such as via @add_to_class, in the rest of the book. Moreover, these fully implemented classes are saved in the D2L library, a lightweight toolkit that makes structured modeling for deep learning easy. In particular, it facilitates reusing many components between projects without changing much at all. For instance, we can replace just the optimizer, just the model, just the dataset, etc.; this degree of modularity pays dividends throughout the book in terms of conciseness and simplicity (this is why we added it) and it can do the same for your own projects.

 

미래의 딥 러닝 구현을 위한 객체 지향 설계를 강조하기 위해 위 클래스에서는 객체가 데이터를 저장하고 서로 상호 작용하는 방법을 간단히 보여줍니다. 이 책의 나머지 부분에서는 @add_to_class를 통해 이러한 클래스의 구현을 계속 강화할 것입니다. 또한 완벽하게 구현된 이러한 클래스는 딥 러닝을 위한 구조화된 모델링을 쉽게 만들어주는 경량 툴킷인 D2L 라이브러리에 저장됩니다. 특히, 프로젝트 간에 많은 구성요소를 전혀 변경하지 않고도 재사용할 수 있습니다. 예를 들어, 옵티마이저만, 모델만, 데이터세트만 교체할 수 있습니다. 이러한 수준의 모듈화는 간결함과 단순성 측면에서 책 전반에 걸쳐 이점을 제공하며(이것이 우리가 이를 추가한 이유입니다) 귀하의 프로젝트에서도 동일한 작업을 수행할 수 있습니다.

 

3.2.6. Exercises

1. Locate full implementations of the above classes that are saved in the D2L library. We strongly recommend that you look at the implementation in detail once you have gained some more familiarity with deep learning modeling.
2. Remove the save_hyperparameters statement in the B class. Can you still print self.a and self.b? Optional: if you have dived into the full implementation of the HyperParameters class, can you explain why?