{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "799724ca",
   "metadata": {},
   "source": [
    "# Exercise 5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5784c7b5",
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e6b14f4",
   "metadata": {},
   "source": [
    "## Task 1: Download and inspect a DatasetDict\n",
    "\n",
    "1. Download the `dair-ai/emotion` dataset\n",
    "2. Find out how many rows and columns it has\n",
    "3. Find the cache directory and convince yourself that you would know how to delete the dataset to free up space\n",
    "4. Create a pandas DataFrame containing the training split of the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c3a41c57",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "092cea71",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8f6e59bb",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fd93b4ce",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "a6a072a2",
   "metadata": {},
   "source": [
    "## Task 2: DatasetDict.map\n",
    "\n",
    "Use `DatasetDict.map` to add a new variable called `label_name` to the dataset. The translation is as follows: sadness (0), joy (1), love (2), anger (3), fear (4), surprise (5).\n",
    "\n",
    "**Test the function on one row before you call map**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a5e52133",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "da00521d",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "73e9ad91",
   "metadata": {},
   "source": [
    "## Task 3: Batched map\n",
    "\n",
    "Rewrite the function from the previous task such that it works if you set `batched=True` in map, i.e. if you do `emotions.map(my_func, batched=True)`\n",
    "\n",
    "Use the strategies you have just learned to find out how this works. Don't try out random things!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8a694516",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "13547beb",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "d7990fda",
   "metadata": {},
   "source": [
    "## Task 4: Write Tokenizers\n",
    "\n",
    "Write a function called `character_tokenizer` that takes a string and returns a list of tokens. Use all characters of the latin alphabet and distingish lowercase and uppercase characters. Don't forget puntcuation. Encode the following text:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "87a4a6a9",
   "metadata": {},
   "outputs": [],
   "source": [
    "text = \"Programming isn't about what you know; it's about what you can figure out.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1150ba95",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "c1578d64",
   "metadata": {},
   "source": [
    "Even this simple example shows you that a lot can go wrong when coding your own tokenizer. Always use pre-trained or pre-implemented tokenizers in practice!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e5dafe4d",
   "metadata": {},
   "source": [
    "## Task 5: Use a pretrained tokenizer\n",
    "\n",
    "Use a pre-trained tokenizer for the `\"distilbert-base-uncased\"` model to encode the `text` from above. Decode each token so you can see how words were split into tokens. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d80596cf",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "57a800cd",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "21e5aa5d",
   "metadata": {},
   "source": [
    "Now wrap the tokenizer into into a function called `tokenize` and tokenize the entire dataset using `DatasetDict.map`. \n",
    "\n",
    "For the tokenizer, the settings should be: \n",
    "    - padding=True\n",
    "    - truncate=True\n",
    "For `map` the settings should be:\n",
    "    - batched=True,\n",
    "    - batch_size=None,\n",
    "    \n",
    "Hint, if you write the function correctly, the following should work:\n",
    "\n",
    "```python\n",
    "tokenize(emotions[\"train\"][:3])\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d05b1530",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6c8580f1",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "2307ede5",
   "metadata": {},
   "source": [
    "## Important\n",
    "\n",
    "Setting `batched=True` and `batch_size=None` means that all tweets are processed in one batch. This is very important. If the dataset was processed in multiple batches, each batch might be padded to a different size (the number of tokens in the longest tweet of that batch)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b4ad4d8e",
   "metadata": {},
   "source": [
    "## Task 6: Redo numpy exercises in torch\n",
    "\n",
    "The following is a subset of the exercises you did in the second lecture using numpy. Repeat them using `torch.tensors` instead of `np.arrays`. This is mainly to show how similar numpy and pytorch is. \n",
    "\n",
    "Create the following tensors:\n",
    "\n",
    "1. A three-dimensional tensor of shape `(3, 3, 4)` containing zeros\n",
    "2. A two-dimensional tensor with 4 rows and 3 columns that contain that is equivalent to the list `[[0.1, 0.2, 0.3], [0.4, 0.5, 0.6], [0.7, 0.8, 0.9], [1.0,1.1,1.2]]`. Do not just type in the numbers.\n",
    "3. Select the bottom left 2 x 2 array from the array you just created\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "41f09872",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3e52e90e",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f99a10c0",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "bd68ba78",
   "metadata": {},
   "source": [
    "Now do the following calculations with tensors \n",
    "\n",
    "1. Do a matrix multiplication of the two tensors x and y\n",
    "2. Do an elementwise multiplication of the tensors x and y\n",
    "3. Do an elementwise addition x and z\n",
    "4. Do an elementwise addition of x and `z.reshape(-1, 1)`\n",
    "5. Sum the two rows in x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b34f5712",
   "metadata": {},
   "outputs": [],
   "source": [
    "x = torch.tensor([[0.5, 1.5], [2.5, 3.5]])\n",
    "y = torch.diag(torch.tensor([2.0, 3.0]))\n",
    "z = torch.tensor([2.0, 3.0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "11eeca30",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ff1d5110",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7385d4cd",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b82328e7",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f8a00b69",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b34514fc",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "bc1e69a9",
   "metadata": {},
   "source": [
    "## Task 7: Differences between torch and numpy\n",
    "\n",
    "The following exercises show a few differences between torch and numpy. \n",
    "\n",
    "1. Do a matrix multiplication of the tensors u and v\n",
    "2. Check the device of the tensor u\n",
    "3. Explicitly set the device to 'cpu'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2cedb4f7",
   "metadata": {},
   "outputs": [],
   "source": [
    "u = torch.ones(2, 2)\n",
    "v = torch.tensor([[1, 2], [3, 4]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8ff680e3",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d188ffe3",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5e51d46e",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}