Defining Neural Networks
========================

In this part of the onboarding you will be introduced to a very simple neural network. You will find out how to use the network with your dataset.

Model
-----

Pytorch makes defining models simple.
The following class is defined in ``kitcar_ml/onboarding/model.py``.
It is just one fully connected layer and will perform poorly on the MNIST dataset.

.. literalinclude::
  ../../../kitcar_ml/onboarding/model.py
  :language: python
  :linenos:

Testing
-------

Before improving the actual model we need to take a closer look at how we can run the model on real data.
Therefore, you need to take a closer look at ``kitcar_ml/onboarding/script.py``.

The :py:func:`test` function is already defined and works out of the box.
It expects the path to your label file and will then load the model and run it.
Therefore you can just execute:

.. prompt:: bash

  python3 kitcar_ml/onboarding/script.py --test --test_label_file PATH/TO/YOUR/LABELS.yaml

You should receive an accuracy of around 10%.

**Why?** Because the model is not trained at all and just randomly guesses numbers. *Remember: The dataset contains the numbers 0-9.*


Training
--------

Now we can run the model and know how to load a dataset. The missing piece is to actually train the model to predict useful numbers.

.. admonition:: Your Task

  Complete the :py:func:`train` function.

.. tip::

  All of the parts you will need are already described in the comments within the function.
  There are some great tutorials on the internet that will help you figure out the basics
  of pytorch. E.g. https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html#define-a-loss-function-and-optimizer.

.. tip::

  You can use :py:func:`test` as a guideline for accessing the data and targets.

Similar to testing, you can run the training using the command:

.. prompt:: bash

  python3 kitcar_ml/onboarding/script.py --train --train_label_file PATH/TO/YOUR/LABELS.yaml

You should start to see the model improve. However, it will not be perfect.

Improvements
------------

The ``script.py`` and ``model.py`` are very basic so far. There are many improvements that can be made.
For the following tasks you will have to take a look at pytorch's documentation or other online material.
The goal is really that you get comfortable working with pytorch, datasets and neural networks in general.

.. admonition:: Further Tasks

  #. Improve your model to get >95% accuracy. First look for layers that pytorch already provides.
     If you can't improve your model, you can also find a lot of blog posts about mnist online.
  #. Save the model after training & Load the model before testing. Complete :py:func:`save` and :py:func:`load` in ``model.py``.
  #. Calculate an average loss in your :py:func:`test` function (optional)
  #. Use your GPU for training (optional)