{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Example: Pause and Continue\n", "This notebook shows how one can execute two functions for one experiment by setting the status to `paused`, and unpausing it later on with a different execution function. This example is heavily based on the `example_general_usage.ipynb` notebook.\n", "\n", "To execute this notebook you need to install:\n", "```\n", "pip install py_experimenter\n", "pip install scikit-learn\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Experiment Configuration File\n", "First we define an experiment configuraiton file. Note that in comparison to the basic example two resultfields `paused_at_seconds` and `resumed_at_seconds` were added." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import os\n", "\n", "content = \"\"\"\n", "PY_EXPERIMENTER:\n", " n_jobs: 1\n", " \n", " Database:\n", " provider: sqlite\n", " database: py_experimenter\n", " table: \n", " name: example_pause_and_continue\n", " keyfields:\n", " dataset:\n", " type: VARCHAR(50)\n", " values: [iris]\n", " cross_validation_splits:\n", " type: INT\n", " values: [5]\n", " seed:\n", " type: INT\n", " values:\n", " start: 2\n", " stop: 7\n", " step: 2\n", " kernel:\n", " type: VARCHAR(50)\n", " values: [linear, poly, rbf, sigmoid]\n", " resultfields:\n", " pipeline: LONGTEXT\n", " train_f1: DECIMAL\n", " train_accuracy: DECIMAL\n", " test_f1: DECIMAL\n", " test_accuracy: DECIMAL\n", " paused_at_seconds: DOUBLE\n", " resumed_at_seconds: DOUBLE\n", " \n", " CUSTOM:\n", " path: sample_data\n", "\n", " CODECARBON:\n", " offline_mode: False\n", " measure_power_secs: 25\n", " tracking_mode: process\n", " log_level: error\n", " save_to_file: True\n", " output_dir: output/CodeCarbon\n", "\"\"\"\n", "# Create config directory if it does not exist\n", "if not os.path.exists('config'):\n", " os.mkdir('config')\n", " \n", "# Create config file\n", "experiment_configuration_file_path = os.path.join('config', 'example_pause_and_continue.yml')\n", "with open(experiment_configuration_file_path, \"w\") as f: \n", " f.write(content)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Defining Pausing Execution Function\n", "Next we fill the table, define the execution function that gets paused after five seconds and run this execution function." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2024-03-11 08:18:43,065 | py-experimenter - INFO | Found 4 keyfields\n", "2024-03-11 08:18:43,066 | py-experimenter - INFO | Found 7 resultfields\n", "2024-03-11 08:18:43,067 | py-experimenter - WARNING | No logtables given\n", "2024-03-11 08:18:43,067 | py-experimenter - WARNING | No custom section defined in config\n", "2024-03-11 08:18:43,068 | py-experimenter - WARNING | No codecarbon section defined in config\n", "2024-03-11 08:18:43,069 | py-experimenter - INFO | Initialized and connected to database\n", "2024-03-11 08:18:43,105 | py-experimenter - INFO | 12 rows successfully added to database. 0 rows were skipped.\n", "[codecarbon INFO @ 08:18:43] [setup] RAM Tracking...\n", "[codecarbon INFO @ 08:18:43] [setup] GPU Tracking...\n", "[codecarbon INFO @ 08:18:43] No GPU found.\n", "[codecarbon INFO @ 08:18:43] [setup] CPU Tracking...\n", "[codecarbon WARNING @ 08:18:43] No CPU tracking mode found. Falling back on CPU constant mode.\n", "[codecarbon WARNING @ 08:18:44] We saw that you have a 12th Gen Intel(R) Core(TM) i7-1260P but we don't know it. Please contact us.\n", "[codecarbon INFO @ 08:18:44] CPU Model on constant consumption mode: 12th Gen Intel(R) Core(TM) i7-1260P\n", "[codecarbon INFO @ 08:18:44] >>> Tracker's metadata:\n", "[codecarbon INFO @ 08:18:44] Platform system: Linux-5.15.146.1-microsoft-standard-WSL2-x86_64-with-glibc2.35\n", "[codecarbon INFO @ 08:18:44] Python version: 3.9.0\n", "[codecarbon INFO @ 08:18:44] CodeCarbon version: 2.3.4\n", "[codecarbon INFO @ 08:18:44] Available RAM : 15.475 GB\n", "[codecarbon INFO @ 08:18:44] CPU count: 16\n", "[codecarbon INFO @ 08:18:44] CPU model: 12th Gen Intel(R) Core(TM) i7-1260P\n", "[codecarbon INFO @ 08:18:44] GPU count: None\n", "[codecarbon INFO @ 08:18:44] GPU model: None\n", "[codecarbon INFO @ 08:18:52] Energy consumed for RAM : 0.000008 kWh. RAM Power : 5.803128719329834 W\n", "[codecarbon INFO @ 08:18:52] Energy consumed for all CPUs : 0.000059 kWh. Total CPU Power : 42.5 W\n", "[codecarbon INFO @ 08:18:52] 0.000068 kWh of electricity used since the beginning.\n", "2024-03-11 08:18:52,537 | py-experimenter - INFO | All configured executions finished.\n" ] } ], "source": [ "import datetime\n", "\n", "from py_experimenter.experimenter import ExperimentStatus, PyExperimenter\n", "from py_experimenter.result_processor import ResultProcessor\n", "\n", "experimenter = PyExperimenter(experiment_configuration_file_path=experiment_configuration_file_path, name='example_notebook')\n", "\n", "experimenter.fill_table_from_config()\n", "\n", "def pause_after_5_seconds(parameters: dict, result_processor: ResultProcessor, custom_config: dict):\n", " import time\n", " time.sleep(5)\n", " result_processor.process_results({\n", " 'paused_at_seconds': datetime.datetime.now().timestamp()\n", " })\n", " return ExperimentStatus.PAUSED\n", "\n", "\n", "\n", "experimenter.execute(pause_after_5_seconds, max_experiments=1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Showcase Paused Execution\n", "Below we only show that the execution of the experiment with `id=1` has been paused." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
IDdatasetcross_validation_splitsseedkernelcreation_datestatusstart_datenamemachinepipelinetrain_f1train_accuracytest_f1test_accuracypaused_at_secondsresumed_at_secondsend_dateerror
01iris52linear2024-03-11 08:18:43paused2024-03-11 08:18:43example_notebookWorklaptopNoneNoneNoneNoneNone1.710142e+09None2024-03-11 08:18:52None
12iris54linear2024-03-11 08:18:43createdNoneNoneNoneNoneNoneNoneNoneNoneNaNNoneNoneNone
23iris56linear2024-03-11 08:18:43createdNoneNoneNoneNoneNoneNoneNoneNoneNaNNoneNoneNone
34iris52poly2024-03-11 08:18:43createdNoneNoneNoneNoneNoneNoneNoneNoneNaNNoneNoneNone
45iris54poly2024-03-11 08:18:43createdNoneNoneNoneNoneNoneNoneNoneNoneNaNNoneNoneNone
56iris56poly2024-03-11 08:18:43createdNoneNoneNoneNoneNoneNoneNoneNoneNaNNoneNoneNone
67iris52rbf2024-03-11 08:18:43createdNoneNoneNoneNoneNoneNoneNoneNoneNaNNoneNoneNone
78iris54rbf2024-03-11 08:18:43createdNoneNoneNoneNoneNoneNoneNoneNoneNaNNoneNoneNone
89iris56rbf2024-03-11 08:18:43createdNoneNoneNoneNoneNoneNoneNoneNoneNaNNoneNoneNone
910iris52sigmoid2024-03-11 08:18:43createdNoneNoneNoneNoneNoneNoneNoneNoneNaNNoneNoneNone
1011iris54sigmoid2024-03-11 08:18:43createdNoneNoneNoneNoneNoneNoneNoneNoneNaNNoneNoneNone
1112iris56sigmoid2024-03-11 08:18:43createdNoneNoneNoneNoneNoneNoneNoneNoneNaNNoneNoneNone
\n", "
" ], "text/plain": [ " ID dataset cross_validation_splits seed kernel creation_date \\\n", "0 1 iris 5 2 linear 2024-03-11 08:18:43 \n", "1 2 iris 5 4 linear 2024-03-11 08:18:43 \n", "2 3 iris 5 6 linear 2024-03-11 08:18:43 \n", "3 4 iris 5 2 poly 2024-03-11 08:18:43 \n", "4 5 iris 5 4 poly 2024-03-11 08:18:43 \n", "5 6 iris 5 6 poly 2024-03-11 08:18:43 \n", "6 7 iris 5 2 rbf 2024-03-11 08:18:43 \n", "7 8 iris 5 4 rbf 2024-03-11 08:18:43 \n", "8 9 iris 5 6 rbf 2024-03-11 08:18:43 \n", "9 10 iris 5 2 sigmoid 2024-03-11 08:18:43 \n", "10 11 iris 5 4 sigmoid 2024-03-11 08:18:43 \n", "11 12 iris 5 6 sigmoid 2024-03-11 08:18:43 \n", "\n", " status start_date name machine pipeline \\\n", "0 paused 2024-03-11 08:18:43 example_notebook Worklaptop None \n", "1 created None None None None \n", "2 created None None None None \n", "3 created None None None None \n", "4 created None None None None \n", "5 created None None None None \n", "6 created None None None None \n", "7 created None None None None \n", "8 created None None None None \n", "9 created None None None None \n", "10 created None None None None \n", "11 created None None None None \n", "\n", " train_f1 train_accuracy test_f1 test_accuracy paused_at_seconds \\\n", "0 None None None None 1.710142e+09 \n", "1 None None None None NaN \n", "2 None None None None NaN \n", "3 None None None None NaN \n", "4 None None None None NaN \n", "5 None None None None NaN \n", "6 None None None None NaN \n", "7 None None None None NaN \n", "8 None None None None NaN \n", "9 None None None None NaN \n", "10 None None None None NaN \n", "11 None None None None NaN \n", "\n", " resumed_at_seconds end_date error \n", "0 None 2024-03-11 08:18:52 None \n", "1 None None None \n", "2 None None None \n", "3 None None None \n", "4 None None None \n", "5 None None None \n", "6 None None None \n", "7 None None None \n", "8 None None None \n", "9 None None None \n", "10 None None None \n", "11 None None None " ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "experimenter.get_table()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Define resuming execution function" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Lastly, we can define a second execution function that resumes the paused execution function. After running this execution function, the experiment is finished and the resulting table is shown." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "[codecarbon INFO @ 08:18:52] [setup] RAM Tracking...\n", "[codecarbon INFO @ 08:18:52] [setup] GPU Tracking...\n", "[codecarbon INFO @ 08:18:52] No GPU found.\n", "[codecarbon INFO @ 08:18:52] [setup] CPU Tracking...\n", "[codecarbon WARNING @ 08:18:52] No CPU tracking mode found. Falling back on CPU constant mode.\n", "[codecarbon WARNING @ 08:18:54] We saw that you have a 12th Gen Intel(R) Core(TM) i7-1260P but we don't know it. Please contact us.\n", "[codecarbon INFO @ 08:18:54] CPU Model on constant consumption mode: 12th Gen Intel(R) Core(TM) i7-1260P\n", "[codecarbon INFO @ 08:18:54] >>> Tracker's metadata:\n", "[codecarbon INFO @ 08:18:54] Platform system: Linux-5.15.146.1-microsoft-standard-WSL2-x86_64-with-glibc2.35\n", "[codecarbon INFO @ 08:18:54] Python version: 3.9.0\n", "[codecarbon INFO @ 08:18:54] CodeCarbon version: 2.3.4\n", "[codecarbon INFO @ 08:18:54] Available RAM : 15.475 GB\n", "[codecarbon INFO @ 08:18:54] CPU count: 16\n", "[codecarbon INFO @ 08:18:54] CPU model: 12th Gen Intel(R) Core(TM) i7-1260P\n", "[codecarbon INFO @ 08:18:54] GPU count: None\n", "[codecarbon INFO @ 08:18:54] GPU model: None\n", "[codecarbon INFO @ 08:18:57] Energy consumed for RAM : 0.000000 kWh. RAM Power : 5.803128719329834 W\n", "[codecarbon INFO @ 08:18:57] Energy consumed for all CPUs : 0.000001 kWh. Total CPU Power : 42.5 W\n", "[codecarbon INFO @ 08:18:57] 0.000001 kWh of electricity used since the beginning.\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
IDdatasetcross_validation_splitsseedkernelcreation_datestatusstart_datenamemachinepipelinetrain_f1train_accuracytest_f1test_accuracypaused_at_secondsresumed_at_secondsend_dateerror
01iris52linear2024-03-11 08:18:43done2024-03-11 08:18:43example_notebookWorklaptopPipeline(steps=[('standardscaler', StandardSca...0.9716670.9716670.9666670.9666671.710142e+091.710142e+092024-03-11 08:18:57None
12iris54linear2024-03-11 08:18:43createdNoneNoneNoneNoneNaNNaNNaNNaNNaNNaNNoneNone
23iris56linear2024-03-11 08:18:43createdNoneNoneNoneNoneNaNNaNNaNNaNNaNNaNNoneNone
34iris52poly2024-03-11 08:18:43createdNoneNoneNoneNoneNaNNaNNaNNaNNaNNaNNoneNone
45iris54poly2024-03-11 08:18:43createdNoneNoneNoneNoneNaNNaNNaNNaNNaNNaNNoneNone
56iris56poly2024-03-11 08:18:43createdNoneNoneNoneNoneNaNNaNNaNNaNNaNNaNNoneNone
67iris52rbf2024-03-11 08:18:43createdNoneNoneNoneNoneNaNNaNNaNNaNNaNNaNNoneNone
78iris54rbf2024-03-11 08:18:43createdNoneNoneNoneNoneNaNNaNNaNNaNNaNNaNNoneNone
89iris56rbf2024-03-11 08:18:43createdNoneNoneNoneNoneNaNNaNNaNNaNNaNNaNNoneNone
910iris52sigmoid2024-03-11 08:18:43createdNoneNoneNoneNoneNaNNaNNaNNaNNaNNaNNoneNone
1011iris54sigmoid2024-03-11 08:18:43createdNoneNoneNoneNoneNaNNaNNaNNaNNaNNaNNoneNone
1112iris56sigmoid2024-03-11 08:18:43createdNoneNoneNoneNoneNaNNaNNaNNaNNaNNaNNoneNone
\n", "
" ], "text/plain": [ " ID dataset cross_validation_splits seed kernel creation_date \\\n", "0 1 iris 5 2 linear 2024-03-11 08:18:43 \n", "1 2 iris 5 4 linear 2024-03-11 08:18:43 \n", "2 3 iris 5 6 linear 2024-03-11 08:18:43 \n", "3 4 iris 5 2 poly 2024-03-11 08:18:43 \n", "4 5 iris 5 4 poly 2024-03-11 08:18:43 \n", "5 6 iris 5 6 poly 2024-03-11 08:18:43 \n", "6 7 iris 5 2 rbf 2024-03-11 08:18:43 \n", "7 8 iris 5 4 rbf 2024-03-11 08:18:43 \n", "8 9 iris 5 6 rbf 2024-03-11 08:18:43 \n", "9 10 iris 5 2 sigmoid 2024-03-11 08:18:43 \n", "10 11 iris 5 4 sigmoid 2024-03-11 08:18:43 \n", "11 12 iris 5 6 sigmoid 2024-03-11 08:18:43 \n", "\n", " status start_date name machine \\\n", "0 done 2024-03-11 08:18:43 example_notebook Worklaptop \n", "1 created None None None \n", "2 created None None None \n", "3 created None None None \n", "4 created None None None \n", "5 created None None None \n", "6 created None None None \n", "7 created None None None \n", "8 created None None None \n", "9 created None None None \n", "10 created None None None \n", "11 created None None None \n", "\n", " pipeline train_f1 \\\n", "0 Pipeline(steps=[('standardscaler', StandardSca... 0.971667 \n", "1 None NaN \n", "2 None NaN \n", "3 None NaN \n", "4 None NaN \n", "5 None NaN \n", "6 None NaN \n", "7 None NaN \n", "8 None NaN \n", "9 None NaN \n", "10 None NaN \n", "11 None NaN \n", "\n", " train_accuracy test_f1 test_accuracy paused_at_seconds \\\n", "0 0.971667 0.966667 0.966667 1.710142e+09 \n", "1 NaN NaN NaN NaN \n", "2 NaN NaN NaN NaN \n", "3 NaN NaN NaN NaN \n", "4 NaN NaN NaN NaN \n", "5 NaN NaN NaN NaN \n", "6 NaN NaN NaN NaN \n", "7 NaN NaN NaN NaN \n", "8 NaN NaN NaN NaN \n", "9 NaN NaN NaN NaN \n", "10 NaN NaN NaN NaN \n", "11 NaN NaN NaN NaN \n", "\n", " resumed_at_seconds end_date error \n", "0 1.710142e+09 2024-03-11 08:18:57 None \n", "1 NaN None None \n", "2 NaN None None \n", "3 NaN None None \n", "4 NaN None None \n", "5 NaN None None \n", "6 NaN None None \n", "7 NaN None None \n", "8 NaN None None \n", "9 NaN None None \n", "10 NaN None None \n", "11 NaN None None " ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import random\n", "\n", "import numpy as np\n", "from sklearn.datasets import load_iris\n", "from sklearn.model_selection import cross_validate\n", "from sklearn.pipeline import make_pipeline\n", "from sklearn.preprocessing import StandardScaler\n", "from sklearn.svm import SVC\n", "\n", "def resume_after_5_seconds(parameters: dict, result_processor: ResultProcessor, custom_config: dict):\n", " result_processor.process_results({\n", " 'resumed_at_seconds': datetime.datetime.now().timestamp()\n", " })\n", " seed = parameters['seed']\n", " random.seed(seed)\n", " np.random.seed(seed)\n", "\n", " data = load_iris()\n", " # In case you want to load a file from a path\n", " # path = os.path.join(custom_config['path'], parameters['dataset'])\n", " # data = pd.read_csv(path)\n", "\n", " X = data.data\n", " y = data.target\n", "\n", " model = make_pipeline(StandardScaler(), SVC(kernel=parameters['kernel'], gamma='auto'))\n", " result_processor.process_results({\n", " 'pipeline': str(model)\n", " })\n", "\n", " if parameters['dataset'] != 'iris':\n", " raise ValueError(\"Example error\")\n", "\n", " scores = cross_validate(model, X, y,\n", " cv=parameters['cross_validation_splits'],\n", " scoring=('accuracy', 'f1_micro'),\n", " return_train_score=True\n", " )\n", "\n", " result_processor.process_results({\n", " 'train_f1': np.mean(scores['train_f1_micro']),\n", " 'train_accuracy': np.mean(scores['train_accuracy'])\n", " })\n", "\n", " result_processor.process_results({\n", " 'test_f1': np.mean(scores['test_f1_micro']),\n", " 'test_accuracy': np.mean(scores['test_accuracy'])\n", " })\n", "\n", "\n", "experimenter.unpause_experiment(1, resume_after_5_seconds)\n", "\n", "experimenter.get_table()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## CodeCarbon Entries\n", "Note that for each execution a different `CodeCarbon` entry is created." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
IDexperiment_idcodecarbon_timestampproject_namerun_idduration_secondsemissions_kgemissions_rate_kg_seccpu_power_wattgpu_power_watt...cpu_modelgpu_countgpu_modellongitudelatituderam_total_sizetracking_modeon_cloudpower_usage_efficiencyoffline_mode
0112024-03-11T08:18:52codecarbona87e6dcb-aace-44b1-9d83-c9744312ce625.0960712.471057e-050.00000542.50.0...12th Gen Intel(R) Core(TM) i7-1260PNoneNone9.531252.477115.47501machineN1.00
1212024-03-11T08:18:57codecarbon5132d06c-9a59-4d0e-8dc5-0afa4984a5f80.1339383.813803e-070.00000342.50.0...12th Gen Intel(R) Core(TM) i7-1260PNoneNone9.531252.477115.47501machineN1.00
\n", "

2 rows × 34 columns

\n", "
" ], "text/plain": [ " ID experiment_id codecarbon_timestamp project_name \\\n", "0 1 1 2024-03-11T08:18:52 codecarbon \n", "1 2 1 2024-03-11T08:18:57 codecarbon \n", "\n", " run_id duration_seconds emissions_kg \\\n", "0 a87e6dcb-aace-44b1-9d83-c9744312ce62 5.096071 2.471057e-05 \n", "1 5132d06c-9a59-4d0e-8dc5-0afa4984a5f8 0.133938 3.813803e-07 \n", "\n", " emissions_rate_kg_sec cpu_power_watt gpu_power_watt ... \\\n", "0 0.000005 42.5 0.0 ... \n", "1 0.000003 42.5 0.0 ... \n", "\n", " cpu_model gpu_count gpu_model longitude \\\n", "0 12th Gen Intel(R) Core(TM) i7-1260P None None 9.5312 \n", "1 12th Gen Intel(R) Core(TM) i7-1260P None None 9.5312 \n", "\n", " latitude ram_total_size tracking_mode on_cloud power_usage_efficiency \\\n", "0 52.4771 15.47501 machine N 1.0 \n", "1 52.4771 15.47501 machine N 1.0 \n", "\n", " offline_mode \n", "0 0 \n", "1 0 \n", "\n", "[2 rows x 34 columns]" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "experimenter.get_codecarbon_table()" ] } ], "metadata": { "kernelspec": { "display_name": "py-experimenter-env", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.0" }, "orig_nbformat": 4 }, "nbformat": 4, "nbformat_minor": 2 }