From f38fb864d3636b8991ad3885660daa19ebaf4d2c Mon Sep 17 00:00:00 2001 From: Tom Schierenbeck Date: Thu, 28 Mar 2024 10:01:09 +0100 Subject: [PATCH] Completed tutorial on algebra --- examples/product_spaces.ipynb | 227 ++++++++++++++++++++-------------- 1 file changed, 134 insertions(+), 93 deletions(-) diff --git a/examples/product_spaces.ipynb b/examples/product_spaces.ipynb index a4ef131..a65034e 100644 --- a/examples/product_spaces.ipynb +++ b/examples/product_spaces.ipynb @@ -6,12 +6,30 @@ "# Product $\\sigma$-Algebra\n", "\n", "This tutorial is part of a bigger series on probabilistic machine learning and aids in understanding product $\\sigma$-algebras.\n", - "As mentioned in the introduction, research has shown that events that are described by independent constraints (rules) are most likely the only events where probability estimation is tractable.\n", - "Spaces that are constructed by independent constraints are called product spaces.\n", + "At the end of this notebook, you will understand...\n", + "- The Definition of the $\\sigma$-Algebra\n", + "- Why these concepts matter, especially for robotics\n", + "- How to apply them using the package \n", + "\n", + "\n", + "## Motivation\n", + "\n", + "Since studying concepts like the $\\sigma$-Algebra is dry, I want to motivate you on why this matters and you should take the time to think about the $\\sigma$-Algebra and specially the Product $\\sigma$-Algebra:\n", + "\n", + "- **Foundations of probability theory**: $\\sigma$-algebras are the building blocks for defining probability in a rigorous way. By understanding them, you gain a deeper understanding of how probabilities are assigned to events.\n", + "- **Working with complex events**: In real-world scenarios, events can be intricate. Sigma algebras allow you to describe not just simple events but also unions, intersections, and complements of these events, giving you a powerful tool to analyze probabilities of more complex situations.\n", + "- **Connection to advanced math**: Sigma algebras bridge the gap between set theory and advanced mathematical concepts like measure theory and integration. Studying them opens doors to these powerful tools used in various scientific fields.\n", "\n", - "Understanding the shape of such events is a key competence to building (new) tractable probabilistic models.\n", "\n", - "In this tutorial, we will work with some visualizations of said events, to get a better understanding of their shape and behavior." + "If you are interested in robotics, it is also important since:\n", + "- **Reasoning with uncertainty**: Robots often operate in environments with uncertainty. $\\sigma$ algebras provide a mathematical foundation to represent uncertain events and reason about the probability of different events happening (like sensor readings or obstacles appearing).\n", + "- **Decision making under probability**: Many robotic tasks involve making decisions based on probabilities. By understanding $\\sigma$-algebras, you can build algorithms that consider the chance of different outcomes and choose the action with the highest probability of success.\n", + "- **Planning and control under uncertainty**: Planning robot actions often requires considering various possibilities. $\\sigma$-algebras allow you to create probabilistic models of the environment, enabling robots to plan and control their movements while accounting for uncertainties.\n", + "\n", + "Research has shown that events that are described by independent constraints (rules) are most likely the only events where probability estimation is tractable.\n", + "Spaces that are constructed by independent constraints are called product spaces.\n", + "Understanding the shape of such events is a key competence to building (new) tractable probabilistic models.\n", + "In this tutorial, we will work with some visualizations of said events, to get a better understanding of their shape, behavior and math." ], "metadata": { "collapsed": false @@ -20,13 +38,13 @@ }, { "cell_type": "code", - "execution_count": 1, + "execution_count": 58, "id": "initial_id", "metadata": { "collapsed": true, "ExecuteTime": { - "end_time": "2024-03-27T11:41:11.325527Z", - "start_time": "2024-03-27T11:41:11.145238Z" + "end_time": "2024-03-28T09:00:49.252002Z", + "start_time": "2024-03-28T09:00:49.216907Z" } }, "outputs": [ @@ -53,7 +71,7 @@ "source": [ "We begin by reciting the definition of a $\\sigma$-algebra.\n", "\n", - "### $\\sigma$-Algebra\n", + "## $\\sigma$-Algebra\n", "\n", "A $\\sigma$ algebra is a set of sets that contains all set-differences that can be constructed by combining arbitrary subsets of said set. Furthermore, it contains all countable unions of sets and all infinite intersection of the set. More formally, according to Kolmogoroff: \n", "\n", @@ -77,9 +95,9 @@ "outputs": [ { "data": { - "text/plain": "[set(),\n {'c'},\n {'a'},\n {'b'},\n {'a', 'c'},\n {'b', 'c'},\n {'a', 'b'},\n {'a', 'b', 'c'}]" + "text/plain": "[set(),\n {'a'},\n {'b'},\n {'c'},\n {'a', 'b'},\n {'a', 'c'},\n {'b', 'c'},\n {'a', 'b', 'c'}]" }, - "execution_count": 2, + "execution_count": 59, "metadata": {}, "output_type": "execute_result" } @@ -96,10 +114,14 @@ "powerset_of_E" ], "metadata": { - "collapsed": false + "collapsed": false, + "ExecuteTime": { + "end_time": "2024-03-28T09:00:49.298182Z", + "start_time": "2024-03-28T09:00:49.294046Z" + } }, "id": "efd225f025a25755", - "execution_count": 2 + "execution_count": 59 }, { "cell_type": "markdown", @@ -118,7 +140,7 @@ "data": { "text/plain": "True" }, - "execution_count": 3, + "execution_count": 60, "metadata": {}, "output_type": "execute_result" } @@ -127,10 +149,14 @@ "E in powerset_of_E" ], "metadata": { - "collapsed": false + "collapsed": false, + "ExecuteTime": { + "end_time": "2024-03-28T09:00:49.304090Z", + "start_time": "2024-03-28T09:00:49.299383Z" + } }, "id": "4718cfb47e7aabf5", - "execution_count": 3 + "execution_count": 60 }, { "cell_type": "markdown", @@ -151,10 +177,14 @@ " print(f\"Set difference {A - B} not in powerset\")" ], "metadata": { - "collapsed": false + "collapsed": false, + "ExecuteTime": { + "end_time": "2024-03-28T09:00:49.309071Z", + "start_time": "2024-03-28T09:00:49.304784Z" + } }, "id": "dc64275c217072e", - "execution_count": 4 + "execution_count": 61 }, { "cell_type": "markdown", @@ -179,12 +209,12 @@ "metadata": { "collapsed": false, "ExecuteTime": { - "end_time": "2024-03-27T11:41:11.377945Z", - "start_time": "2024-03-27T11:41:11.373670Z" + "end_time": "2024-03-28T09:00:49.316136Z", + "start_time": "2024-03-28T09:00:49.310238Z" } }, "id": "e7dc354bb8a70a12", - "execution_count": 5 + "execution_count": 62 }, { "cell_type": "markdown", @@ -202,7 +232,7 @@ }, { "cell_type": "code", - "execution_count": 6, + "execution_count": 63, "outputs": [ { "name": "stdout", @@ -222,8 +252,8 @@ "metadata": { "collapsed": false, "ExecuteTime": { - "end_time": "2024-03-27T11:41:11.383495Z", - "start_time": "2024-03-27T11:41:11.378610Z" + "end_time": "2024-03-28T09:00:49.320810Z", + "start_time": "2024-03-28T09:00:49.316944Z" } }, "id": "27ea010bc8892833" @@ -245,7 +275,7 @@ { "cell_type": "markdown", "source": [ - "### Product $\\sigma$-Algebra\n", + "## Product $\\sigma$-Algebra\n", "\n", "Let $(E_1,\\Im_1)$ and $(E_2,\\Im_2)$ be measurable spaces.\n", "The product $\\sigma$-algebra of $\\Im_1$ and $\\Im_2$ is denoted $\\Im_1 \\otimes \\Im_2$, and defined as:\n", @@ -275,7 +305,7 @@ "data": { "text/plain": "[('bowl', 'blue'),\n ('bowl', 'green'),\n ('bowl', 'red'),\n ('cup', 'blue'),\n ('cup', 'green'),\n ('cup', 'red'),\n ('spoon', 'blue'),\n ('spoon', 'green'),\n ('spoon', 'red')]" }, - "execution_count": 7, + "execution_count": 64, "metadata": {}, "output_type": "execute_result" } @@ -287,12 +317,12 @@ "metadata": { "collapsed": false, "ExecuteTime": { - "end_time": "2024-03-27T11:41:11.389386Z", - "start_time": "2024-03-27T11:41:11.384286Z" + "end_time": "2024-03-28T09:00:49.326522Z", + "start_time": "2024-03-28T09:00:49.321490Z" } }, "id": "379cef1a50307c47", - "execution_count": 7 + "execution_count": 64 }, { "cell_type": "markdown", @@ -312,7 +342,7 @@ "data": { "text/plain": "[('bowl', 'blue'), ('cup', 'blue')]" }, - "execution_count": 8, + "execution_count": 65, "metadata": {}, "output_type": "execute_result" } @@ -324,12 +354,12 @@ "metadata": { "collapsed": false, "ExecuteTime": { - "end_time": "2024-03-27T11:41:11.394755Z", - "start_time": "2024-03-27T11:41:11.390099Z" + "end_time": "2024-03-28T09:00:49.332076Z", + "start_time": "2024-03-28T09:00:49.327154Z" } }, "id": "a53e718226f00339", - "execution_count": 8 + "execution_count": 65 }, { "cell_type": "markdown", @@ -351,12 +381,12 @@ "metadata": { "collapsed": false, "ExecuteTime": { - "end_time": "2024-03-27T11:41:11.399327Z", - "start_time": "2024-03-27T11:41:11.395848Z" + "end_time": "2024-03-28T09:00:49.337038Z", + "start_time": "2024-03-28T09:00:49.332889Z" } }, "id": "1bb140fb470e3cf7", - "execution_count": 9 + "execution_count": 66 }, { "cell_type": "markdown", @@ -375,7 +405,7 @@ "data": { "text/plain": "{Symbolic(item): ('bowl', 'cup'), Symbolic(color): ('blue', 'red')}" }, - "execution_count": 10, + "execution_count": 67, "metadata": {}, "output_type": "execute_result" } @@ -387,12 +417,12 @@ "metadata": { "collapsed": false, "ExecuteTime": { - "end_time": "2024-03-27T11:41:11.404742Z", - "start_time": "2024-03-27T11:41:11.399885Z" + "end_time": "2024-03-28T09:00:49.342190Z", + "start_time": "2024-03-28T09:00:49.338370Z" } }, "id": "b74c89bfbed07d6f", - "execution_count": 10 + "execution_count": 67 }, { "cell_type": "markdown", @@ -411,7 +441,7 @@ "data": { "text/plain": "\"{color: ('blue',), item: ('bowl',)} u {item: ('cup',), color: ('red',)}\"" }, - "execution_count": 11, + "execution_count": 68, "metadata": {}, "output_type": "execute_result" } @@ -423,17 +453,28 @@ "metadata": { "collapsed": false, "ExecuteTime": { - "end_time": "2024-03-27T11:41:11.410421Z", - "start_time": "2024-03-27T11:41:11.405470Z" + "end_time": "2024-03-28T09:00:49.349675Z", + "start_time": "2024-03-28T09:00:49.342868Z" } }, "id": "7fd789bf96187ca1", - "execution_count": 11 + "execution_count": 68 }, { "cell_type": "markdown", "source": [ - "The correct union is a complex event. A complex event is a union of disjoint events." + "The correct union is a complex event. A complex event is a union of disjoint events.\n", + "\n", + "## Connections to Logic\n", + "\n", + "Algebraic concepts are hard to grasp. Since you, the reader, are very likely a Computer Scientists I will re-explain a random event from the perspective of logic.\n", + "We can rewrite the assignment of a variable to a set as a boolean variable. For example,\n", + "$Item_{\\{\\text{bowl}, \\text{cup}\\}} = item \\in \\{\\text{bowl}, \\text{cup}\\}$\n", + "is a boolean variable that is true if the item is a bowl or a cup.\n", + "We can rewrite the statement of the union as logical statement.\n", + "$$ \\left( Item_{\\{\\text{bowl}\\}} \\land Color_{\\{\\text{blue}\\}} \\right) \\lor \\left( Item_{\\{\\text{cup}\\}} \\land Color_{\\{\\text{red}\\}} \\right) $$\n", + "This logical statement describes either a blue bowl or a red cup.\n", + "The complex random events can always be thought of as a disjunction of conjunctions, hence a logical statement in the [disjunctive normal form](https://en.wikipedia.org/wiki/Disjunctive_normal_form)." ], "metadata": { "collapsed": false @@ -455,7 +496,7 @@ }, { "cell_type": "code", - "execution_count": 20, + "execution_count": 69, "outputs": [ { "data": { @@ -1328,7 +1369,7 @@ "plotlyServerURL": "https://plot.ly" } }, - "text/html": "
" + "text/html": "
" }, "metadata": {}, "output_type": "display_data" @@ -1346,8 +1387,8 @@ "metadata": { "collapsed": false, "ExecuteTime": { - "end_time": "2024-03-27T14:18:43.878509Z", - "start_time": "2024-03-27T14:18:43.872359Z" + "end_time": "2024-03-28T09:00:49.377357Z", + "start_time": "2024-03-28T09:00:49.371423Z" } }, "id": "a86b223360f60725" @@ -1367,13 +1408,13 @@ }, { "cell_type": "code", - "execution_count": 13, + "execution_count": 70, "outputs": [ { "data": { "text/plain": "{Continuous(x): [2,3] | [4,5] | [6,7], Continuous(y): [10,15] | [25,27]}" }, - "execution_count": 13, + "execution_count": 70, "metadata": {}, "output_type": "execute_result" } @@ -1385,8 +1426,8 @@ "metadata": { "collapsed": false, "ExecuteTime": { - "end_time": "2024-03-27T11:41:11.425929Z", - "start_time": "2024-03-27T11:41:11.423090Z" + "end_time": "2024-03-28T09:00:49.382698Z", + "start_time": "2024-03-28T09:00:49.379835Z" } }, "id": "86d42ecd896bef10" @@ -1403,7 +1444,7 @@ }, { "cell_type": "code", - "execution_count": 14, + "execution_count": 71, "outputs": [ { "data": { @@ -2336,7 +2377,7 @@ "plotlyServerURL": "https://plot.ly" } }, - "text/html": "
" + "text/html": "
" }, "metadata": {}, "output_type": "display_data" @@ -2350,8 +2391,8 @@ "metadata": { "collapsed": false, "ExecuteTime": { - "end_time": "2024-03-27T11:41:11.433972Z", - "start_time": "2024-03-27T11:41:11.426820Z" + "end_time": "2024-03-28T09:00:49.430495Z", + "start_time": "2024-03-28T09:00:49.425129Z" } }, "id": "7d81de549d20f02b" @@ -2368,7 +2409,7 @@ }, { "cell_type": "code", - "execution_count": 15, + "execution_count": 72, "outputs": [ { "data": { @@ -4594,7 +4635,7 @@ "plotlyServerURL": "https://plot.ly" } }, - "text/html": "
" + "text/html": "
" }, "metadata": {}, "output_type": "display_data" @@ -4611,8 +4652,8 @@ "metadata": { "collapsed": false, "ExecuteTime": { - "end_time": "2024-03-27T11:41:11.515868Z", - "start_time": "2024-03-27T11:41:11.499928Z" + "end_time": "2024-03-28T09:00:49.511362Z", + "start_time": "2024-03-28T09:00:49.499268Z" } }, "id": "da2b51a986fa9d05" @@ -4648,7 +4689,7 @@ "data": [ { "fill": "toself", - "legendgroup": "140154644927104", + "legendgroup": "140005024932816", "line": { "color": "#636EFA" }, @@ -4687,7 +4728,7 @@ }, { "fill": "toself", - "legendgroup": "140154644927104", + "legendgroup": "140005024932816", "line": { "color": "#636EFA" }, @@ -5566,7 +5607,7 @@ "plotlyServerURL": "https://plot.ly" } }, - "text/html": "
" + "text/html": "
" }, "metadata": {}, "output_type": "display_data" @@ -5583,12 +5624,12 @@ "metadata": { "collapsed": false, "ExecuteTime": { - "end_time": "2024-03-27T11:41:11.530368Z", - "start_time": "2024-03-27T11:41:11.522022Z" + "end_time": "2024-03-28T09:00:49.521705Z", + "start_time": "2024-03-28T09:00:49.514608Z" } }, "id": "97f6f80a61c8abaf", - "execution_count": 16 + "execution_count": 73 }, { "cell_type": "markdown", @@ -5652,7 +5693,7 @@ 7, 6 ], - "legendgroup": "140154644926624", + "legendgroup": "140005024932096", "name": "Event", "showlegend": true, "x": [ @@ -5732,7 +5773,7 @@ 7, 6 ], - "legendgroup": "140154644926624", + "legendgroup": "140005024932096", "name": "Event", "showlegend": false, "x": [ @@ -5812,7 +5853,7 @@ 7, 6 ], - "legendgroup": "140154644926624", + "legendgroup": "140005024932096", "name": "Event", "showlegend": false, "x": [ @@ -5892,7 +5933,7 @@ 7, 6 ], - "legendgroup": "140154644926624", + "legendgroup": "140005024932096", "name": "Event", "showlegend": false, "x": [ @@ -5972,7 +6013,7 @@ 7, 6 ], - "legendgroup": "140154644926624", + "legendgroup": "140005024932096", "name": "Event", "showlegend": false, "x": [ @@ -6052,7 +6093,7 @@ 7, 6 ], - "legendgroup": "140154644926624", + "legendgroup": "140005024932096", "name": "Event", "showlegend": false, "x": [ @@ -6936,7 +6977,7 @@ "plotlyServerURL": "https://plot.ly" } }, - "text/html": "
" + "text/html": "
" }, "metadata": {}, "output_type": "display_data" @@ -6957,12 +6998,12 @@ "metadata": { "collapsed": false, "ExecuteTime": { - "end_time": "2024-03-27T11:41:11.620509Z", - "start_time": "2024-03-27T11:41:11.606657Z" + "end_time": "2024-03-28T09:00:49.533752Z", + "start_time": "2024-03-28T09:00:49.522521Z" } }, "id": "9e47b2f884e4c9dd", - "execution_count": 17 + "execution_count": 74 }, { "cell_type": "markdown", @@ -6987,7 +7028,7 @@ "data": [ { "fill": "toself", - "legendgroup": "140154644601440", + "legendgroup": "140005024696592", "line": { "color": "#636EFA" }, @@ -7014,7 +7055,7 @@ }, { "fill": "toself", - "legendgroup": "140154644601440", + "legendgroup": "140005024696592", "line": { "color": "#636EFA" }, @@ -7041,7 +7082,7 @@ }, { "fill": "toself", - "legendgroup": "140154644601440", + "legendgroup": "140005024696592", "line": { "color": "#636EFA" }, @@ -7068,7 +7109,7 @@ }, { "fill": "toself", - "legendgroup": "140154644601440", + "legendgroup": "140005024696592", "line": { "color": "#636EFA" }, @@ -7935,7 +7976,7 @@ "plotlyServerURL": "https://plot.ly" } }, - "text/html": "
" + "text/html": "
" }, "metadata": {}, "output_type": "display_data" @@ -7956,12 +7997,12 @@ "metadata": { "collapsed": false, "ExecuteTime": { - "end_time": "2024-03-27T11:41:11.637373Z", - "start_time": "2024-03-27T11:41:11.627497Z" + "end_time": "2024-03-28T09:00:49.544428Z", + "start_time": "2024-03-28T09:00:49.534748Z" } }, "id": "b8916038b2521a38", - "execution_count": 18 + "execution_count": 75 }, { "cell_type": "markdown", @@ -7982,7 +8023,7 @@ "data": [ { "fill": "toself", - "legendgroup": "140154644888496", + "legendgroup": "140005023904048", "line": { "color": "#636EFA" }, @@ -8009,7 +8050,7 @@ }, { "fill": "toself", - "legendgroup": "140154644888496", + "legendgroup": "140005023904048", "line": { "color": "#636EFA" }, @@ -8048,7 +8089,7 @@ }, { "fill": "toself", - "legendgroup": "140154644888496", + "legendgroup": "140005023904048", "line": { "color": "#636EFA" }, @@ -8075,7 +8116,7 @@ }, { "fill": "toself", - "legendgroup": "140154644888496", + "legendgroup": "140005023904048", "line": { "color": "#636EFA" }, @@ -8954,7 +8995,7 @@ "plotlyServerURL": "https://plot.ly" } }, - "text/html": "
" + "text/html": "
" }, "metadata": {}, "output_type": "display_data" @@ -8968,18 +9009,18 @@ "metadata": { "collapsed": false, "ExecuteTime": { - "end_time": "2024-03-27T11:41:11.662740Z", - "start_time": "2024-03-27T11:41:11.652118Z" + "end_time": "2024-03-28T09:00:49.559742Z", + "start_time": "2024-03-28T09:00:49.549873Z" } }, "id": "510b80164a41cfc", - "execution_count": 19 + "execution_count": 76 }, { "cell_type": "markdown", "source": [ "Now this already sounds somewhat useful. However, just the events are of limited use. The real power of the product algebra comes when you start to calculate probabilities of events.\n", - "For this, you can checkout this tutorial on [probability theory](https://probabilistic-model.readthedocs.io/en/latest/examples/probability_theory.html)." + "For this, you can check out this tutorial on [probability theory](https://probabilistic-model.readthedocs.io/en/latest/examples/probability_theory.html)." ], "metadata": { "collapsed": false