Updated README.md, few bug tweaks, adding regression_results.ipynb
This commit is contained in:
hpayer committed
1 parent
b7b1e38b4e
commit
ac4d2afed4
7 files changed
+878
-338
No files matched your search
@@ -1,30 +1,37 @@
|
|||||||
# Museum Analytics #
|
# Museum Analytics #
|
||||||
|
|
||||||
## ⚙️ Configuration Setup
|
## Installation and how to use
|
||||||
|
|
||||||
This project uses environment variables to secure sensitive information. Follow these steps to configure your local environment:
|
1. **Make sure docker is installed: see https://docs.docker.com/get-started/get-docker/ for instructions.**
|
||||||
|
|
||||||
1. **Duplicate the template file:**
|
2. **Clone the repo**
|
||||||
|
```Bash
|
||||||
|
git clone https://git.hanspayer.com/hpayer/museum_analytics.git
|
||||||
|
```
|
||||||
|
3. **Duplicate the template file:**
|
||||||
Copy the `.env.example` file and rename it to `.env` in the root directory.
|
Copy the `.env.example` file and rename it to `.env` in the root directory.
|
||||||
```bash
|
```bash
|
||||||
cp .env.example .env
|
cd museum_analytics
|
||||||
|
cp ./api/app/.env.example ./api/app/.env
|
||||||
```
|
```
|
||||||
2. **Update your keys:**
|
4. **Modify .env**
|
||||||
Open the newly created `.env` file in your text editor and replace the placeholder values with your actual credentials:
|
Open the newly created `.env` file in your text editor and replace the placeholder values with your actual credentials:
|
||||||
```text
|
```text
|
||||||
WIKIPEDIA_USER_AGENT="MuseumAnalytics/0.1 (your_email_address)"
|
WIKIPEDIA_USER_AGENT="MuseumAnalytics/0.1 (your_email_address)"
|
||||||
```
|
```
|
||||||
⚠️ **Important:** Never commit your `.env` file to Git. It is already added to `.gitignore` to protect your secrets.
|
⚠️ **Important:** Never commit your `.env` file to Git. It is already added to `.gitignore` to protect your secrets.
|
||||||
|
|
||||||
## Installation
|
|
||||||
|
|
||||||
1. **Make sure docker is installed: see https://docs.docker.com/get-started/get-docker/ for instructions.**
|
5. **Build and start the museum analytics and jupyter notebook servers with docker compose:**
|
||||||
2. **Clone the repo**
|
|
||||||
```Bash
|
```Bash
|
||||||
git clone https://git.hanspayer.com/hpayer/museum_analytics.git
|
# in the museum_analytics working directory
|
||||||
|
docker compose up --build
|
||||||
```
|
```
|
||||||
3. Build and start the museum analytics server with docker:
|
6. **Open the http://127.0.0.1:8888/ in your browser to access jupyter notebook.**
|
||||||
|
|
||||||
|
7. **Open the regression_results.ipynb file and Run the code in jupyter notebook.**
|
||||||
|
|
||||||
|
8. **To shutdown the servers, run this command in the terminal**
|
||||||
```Bash
|
```Bash
|
||||||
cd ./api
|
docker compose down
|
||||||
docker build --network=host --no-cache -t museum-analytics-server .
|
|
||||||
```
|
```
|
||||||
@@ -7,7 +7,7 @@ import pandas as pd
|
|||||||
|
|
||||||
load_dotenv()
|
load_dotenv()
|
||||||
|
|
||||||
WIKIPEDIA_USER_AGENT = os.getenv("WIKIPEDIA_USER_AGENT")
|
WIKIPEDIA_USER_AGENT: str = os.getenv("WIKIPEDIA_USER_AGENT", "MuseumAnalytics/0.1 (johndoe@gmail.com)")
|
||||||
|
|
||||||
|
|
||||||
class TableNotFound(Exception):
|
class TableNotFound(Exception):
|
||||||
|
|||||||
@@ -8,14 +8,6 @@ Run locally (from the repo root):
|
|||||||
or run this file directly (e.g. from the IDE):
|
or run this file directly (e.g. from the IDE):
|
||||||
|
|
||||||
python api/museum_api.py
|
python api/museum_api.py
|
||||||
|
|
||||||
Or with a DataFrame you already have in memory:
|
|
||||||
|
|
||||||
from api.museum_api import create_app
|
|
||||||
import uvicorn
|
|
||||||
uvicorn.run(create_app(df), port=8000)
|
|
||||||
|
|
||||||
Expected columns: Name, Visitors, City, Country, Population
|
|
||||||
"""
|
"""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
@@ -252,7 +244,7 @@ def create_app(
|
|||||||
return [Prediction(population=p, predicted_visitors=float(v))
|
return [Prediction(population=p, predicted_visitors=float(v))
|
||||||
for p, v in zip(req.populations, preds)]
|
for p, v in zip(req.populations, preds)]
|
||||||
|
|
||||||
@app.get("data")
|
@app.get("/data")
|
||||||
def data():
|
def data():
|
||||||
"""Every row the model was fitted on, with its prediction."""
|
"""Every row the model was fitted on, with its prediction."""
|
||||||
m = get_model()
|
m = get_model()
|
||||||
|
|||||||
+3
-2
@@ -15,8 +15,9 @@ services:
|
|||||||
ports:
|
ports:
|
||||||
- "8888:8888"
|
- "8888:8888"
|
||||||
environment:
|
environment:
|
||||||
- FASTAPI_URL=http://localhost:8000
|
- MUSEUM_ANALYTICS_URL=http://museum_analytics_app:8000
|
||||||
|
|
||||||
depends_on:
|
depends_on:
|
||||||
- museum_analytics_app
|
- museum_analytics_app
|
||||||
volumes:
|
volumes:
|
||||||
- ./notebooks:/home/jovyan/work
|
- ./notebooks:/workspace
|
||||||
@@ -23,4 +23,4 @@ COPY . /workspace
|
|||||||
|
|
||||||
EXPOSE 8888
|
EXPOSE 8888
|
||||||
|
|
||||||
CMD ["jupyter", "lab", "--ip=0.0.0.0", "--port=8888", "--no-browser", "--allow-root", "--NotebookApp.token=''"]
|
CMD ["jupyter", "lab", "--ip=0.0.0.0", "--port=8888", "--no-browser", "--allow-root", "--ServerApp.token=", "--ServerApp.password=", "--ServerApp.custom_display_url=http://localhost:8888"]
|
||||||
@@ -0,0 +1,853 @@
|
|||||||
|
{
|
||||||
|
"cells": [
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"id": "cell-00",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"# Museum regression results\n",
|
||||||
|
"\n",
|
||||||
|
"A read-only report on the regression model that the `museum_analytics` FastAPI server has loaded.\n",
|
||||||
|
"Everything here comes from the API over HTTP. The notebook never calls `/train`, so it shows the\n",
|
||||||
|
"model the server built from its own data at startup and leaves it unchanged.\n",
|
||||||
|
"\n",
|
||||||
|
"| Endpoint | Used for |\n",
|
||||||
|
"|---|---|\n",
|
||||||
|
"| `GET /health` | check the server is up and has a model |\n",
|
||||||
|
"| `GET /model` | fit statistics and the fitted equation |\n",
|
||||||
|
"| `GET /data` | every museum the model was fitted on, with its prediction |\n",
|
||||||
|
"| `POST /predict` | the fitted curve, and predictions for new cities |\n",
|
||||||
|
"| `GET /residuals` | museums that most beat or miss what their city size predicts |"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": 2,
|
||||||
|
"id": "cell-01",
|
||||||
|
"metadata": {},
|
||||||
|
"outputs": [],
|
||||||
|
"source": [
|
||||||
|
"import os\n",
|
||||||
|
"import requests\n",
|
||||||
|
"import numpy as np\n",
|
||||||
|
"import pandas as pd\n",
|
||||||
|
"import matplotlib.pyplot as plt\n",
|
||||||
|
"from matplotlib.ticker import FuncFormatter, LogLocator, NullFormatter\n",
|
||||||
|
"\n",
|
||||||
|
"# Inside docker-compose this is http://museum_analytics_app:8000; outside it, localhost.\n",
|
||||||
|
"API = os.getenv(\"MUSEUM_ANALYTICS_URL\", \"http://localhost:8000\")\n",
|
||||||
|
"\n",
|
||||||
|
"\n",
|
||||||
|
"def get(path, **params):\n",
|
||||||
|
" r = requests.get(f\"{API}{path}\", params=params, timeout=10)\n",
|
||||||
|
" r.raise_for_status()\n",
|
||||||
|
" return r.json()\n",
|
||||||
|
"\n",
|
||||||
|
"\n",
|
||||||
|
"def post(path, payload):\n",
|
||||||
|
" r = requests.post(f\"{API}{path}\", json=payload, timeout=10)\n",
|
||||||
|
" r.raise_for_status()\n",
|
||||||
|
" return r.json()\n",
|
||||||
|
"\n",
|
||||||
|
"\n",
|
||||||
|
"BLUE, ORANGE, INK, MUTED, GRID = \"#2a78d6\", \"#eb6834\", \"#0b0b0b\", \"#52514e\", \"#e6e5e0\"\n",
|
||||||
|
"plt.rcParams.update({\n",
|
||||||
|
" \"axes.spines.top\": False, \"axes.spines.right\": False,\n",
|
||||||
|
" \"axes.edgecolor\": MUTED, \"axes.labelcolor\": INK, \"axes.titlecolor\": INK,\n",
|
||||||
|
" \"xtick.color\": MUTED, \"ytick.color\": MUTED,\n",
|
||||||
|
" \"axes.grid\": True, \"axes.axisbelow\": True, \"grid.color\": GRID, \"grid.linewidth\": 0.8,\n",
|
||||||
|
" \"legend.frameon\": False, \"figure.dpi\": 110,\n",
|
||||||
|
"})\n",
|
||||||
|
"\n",
|
||||||
|
"\n",
|
||||||
|
"def human(x, _=None):\n",
|
||||||
|
" \"\"\"1_500_000 -> '1.5M'\"\"\"\n",
|
||||||
|
" for div, suf in ((1e9, \"B\"), (1e6, \"M\"), (1e3, \"K\")):\n",
|
||||||
|
" if abs(x) >= div:\n",
|
||||||
|
" return f\"{x / div:.3g}{suf}\"\n",
|
||||||
|
" return f\"{x:.3g}\"\n",
|
||||||
|
"\n",
|
||||||
|
"\n",
|
||||||
|
"def human_axes(ax, log=False):\n",
|
||||||
|
" \"\"\"Readable tick labels; on log axes tick at 1, 2, 5 x 10^k.\"\"\"\n",
|
||||||
|
" for axis in (ax.xaxis, ax.yaxis):\n",
|
||||||
|
" if log:\n",
|
||||||
|
" axis.set_major_locator(LogLocator(base=10, subs=(1, 2, 5)))\n",
|
||||||
|
" axis.set_minor_formatter(NullFormatter())\n",
|
||||||
|
" axis.set_major_formatter(FuncFormatter(human))"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"id": "cell-02",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"## 1. Check the server"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": 3,
|
||||||
|
"id": "cell-03",
|
||||||
|
"metadata": {},
|
||||||
|
"outputs": [
|
||||||
|
{
|
||||||
|
"data": {
|
||||||
|
"text/plain": [
|
||||||
|
"{'status': 'ok', 'model_loaded': True}"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"execution_count": 3,
|
||||||
|
"metadata": {},
|
||||||
|
"output_type": "execute_result"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"source": [
|
||||||
|
"try:\n",
|
||||||
|
" health = get(\"/health\")\n",
|
||||||
|
"except requests.ConnectionError:\n",
|
||||||
|
" raise SystemExit(f\"Can't reach the API at {API}. Start it with: docker compose up --build -d\")\n",
|
||||||
|
"\n",
|
||||||
|
"if not health[\"model_loaded\"]:\n",
|
||||||
|
" raise SystemExit(\"The server is up but has no model loaded. POST /train first.\")\n",
|
||||||
|
"health"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"id": "cell-04",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"## 2. Model summary\n",
|
||||||
|
"\n",
|
||||||
|
"`GET /model` returns the fit statistics. With the default `log` scale the model is\n",
|
||||||
|
"log10(visitors) ~ log10(population), so the slope is an elasticity: how visitors scale with city size."
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": 4,
|
||||||
|
"id": "cell-05",
|
||||||
|
"metadata": {},
|
||||||
|
"outputs": [
|
||||||
|
{
|
||||||
|
"name": "stdout",
|
||||||
|
"output_type": "stream",
|
||||||
|
"text": [
|
||||||
|
"visitors = 338,577.3 * population^0.128\n",
|
||||||
|
"A 10x larger city is associated with 1.34x the visitors.\n",
|
||||||
|
"\n",
|
||||||
|
"Caution: city population explains only about 2% of the variation in visitors, so treat predictions as rough baselines.\n"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"data": {
|
||||||
|
"text/html": [
|
||||||
|
"<div>\n",
|
||||||
|
"<style scoped>\n",
|
||||||
|
" .dataframe tbody tr th:only-of-type {\n",
|
||||||
|
" vertical-align: middle;\n",
|
||||||
|
" }\n",
|
||||||
|
"\n",
|
||||||
|
" .dataframe tbody tr th {\n",
|
||||||
|
" vertical-align: top;\n",
|
||||||
|
" }\n",
|
||||||
|
"\n",
|
||||||
|
" .dataframe thead th {\n",
|
||||||
|
" text-align: right;\n",
|
||||||
|
" }\n",
|
||||||
|
"</style>\n",
|
||||||
|
"<table border=\"1\" class=\"dataframe\">\n",
|
||||||
|
" <thead>\n",
|
||||||
|
" <tr style=\"text-align: right;\">\n",
|
||||||
|
" <th></th>\n",
|
||||||
|
" <th>value</th>\n",
|
||||||
|
" <th>meaning</th>\n",
|
||||||
|
" </tr>\n",
|
||||||
|
" </thead>\n",
|
||||||
|
" <tbody>\n",
|
||||||
|
" <tr>\n",
|
||||||
|
" <th>n</th>\n",
|
||||||
|
" <td>71</td>\n",
|
||||||
|
" <td>rows fitted</td>\n",
|
||||||
|
" </tr>\n",
|
||||||
|
" <tr>\n",
|
||||||
|
" <th>scale</th>\n",
|
||||||
|
" <td>log</td>\n",
|
||||||
|
" <td>axis transform</td>\n",
|
||||||
|
" </tr>\n",
|
||||||
|
" <tr>\n",
|
||||||
|
" <th>aggregate_by_city</th>\n",
|
||||||
|
" <td>False</td>\n",
|
||||||
|
" <td>one row per city?</td>\n",
|
||||||
|
" </tr>\n",
|
||||||
|
" <tr>\n",
|
||||||
|
" <th>slope</th>\n",
|
||||||
|
" <td>0.128263</td>\n",
|
||||||
|
" <td>change in (log) visitors per unit of (log) pop...</td>\n",
|
||||||
|
" </tr>\n",
|
||||||
|
" <tr>\n",
|
||||||
|
" <th>intercept</th>\n",
|
||||||
|
" <td>5.529658</td>\n",
|
||||||
|
" <td>fitted intercept</td>\n",
|
||||||
|
" </tr>\n",
|
||||||
|
" <tr>\n",
|
||||||
|
" <th>R²</th>\n",
|
||||||
|
" <td>0.084989</td>\n",
|
||||||
|
" <td>share of variance explained (training data)</td>\n",
|
||||||
|
" </tr>\n",
|
||||||
|
" <tr>\n",
|
||||||
|
" <th>CV R²</th>\n",
|
||||||
|
" <td>0.016122</td>\n",
|
||||||
|
" <td>R² on held-out folds (5-fold CV)</td>\n",
|
||||||
|
" </tr>\n",
|
||||||
|
" <tr>\n",
|
||||||
|
" <th>Pearson r</th>\n",
|
||||||
|
" <td>0.291528</td>\n",
|
||||||
|
" <td>linear correlation</td>\n",
|
||||||
|
" </tr>\n",
|
||||||
|
" <tr>\n",
|
||||||
|
" <th>Spearman ρ</th>\n",
|
||||||
|
" <td>0.331742</td>\n",
|
||||||
|
" <td>rank correlation</td>\n",
|
||||||
|
" </tr>\n",
|
||||||
|
" </tbody>\n",
|
||||||
|
"</table>\n",
|
||||||
|
"</div>"
|
||||||
|
],
|
||||||
|
"text/plain": [
|
||||||
|
" value meaning\n",
|
||||||
|
"n 71 rows fitted\n",
|
||||||
|
"scale log axis transform\n",
|
||||||
|
"aggregate_by_city False one row per city?\n",
|
||||||
|
"slope 0.128263 change in (log) visitors per unit of (log) pop...\n",
|
||||||
|
"intercept 5.529658 fitted intercept\n",
|
||||||
|
"R² 0.084989 share of variance explained (training data)\n",
|
||||||
|
"CV R² 0.016122 R² on held-out folds (5-fold CV)\n",
|
||||||
|
"Pearson r 0.291528 linear correlation\n",
|
||||||
|
"Spearman ρ 0.331742 rank correlation"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"execution_count": 4,
|
||||||
|
"metadata": {},
|
||||||
|
"output_type": "execute_result"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"source": [
|
||||||
|
"summary = get(\"/model\")\n",
|
||||||
|
"is_log = summary[\"scale\"] == \"log\"\n",
|
||||||
|
"\n",
|
||||||
|
"stats = pd.DataFrame({\n",
|
||||||
|
" \"value\": [summary[\"n_samples\"], summary[\"scale\"], summary[\"aggregate_by_city\"],\n",
|
||||||
|
" summary[\"slope\"], summary[\"intercept\"], summary[\"r2\"],\n",
|
||||||
|
" summary[\"cv_r2_5fold\"], summary[\"pearson_r\"], summary[\"spearman_rho\"]],\n",
|
||||||
|
" \"meaning\": [\"rows fitted\", \"axis transform\", \"one row per city?\",\n",
|
||||||
|
" \"change in (log) visitors per unit of (log) population\", \"fitted intercept\",\n",
|
||||||
|
" \"share of variance explained (training data)\",\n",
|
||||||
|
" \"R² on held-out folds (5-fold CV)\",\n",
|
||||||
|
" \"linear correlation\", \"rank correlation\"],\n",
|
||||||
|
"}, index=[\"n\", \"scale\", \"aggregate_by_city\", \"slope\", \"intercept\", \"R²\",\n",
|
||||||
|
" \"CV R²\", \"Pearson r\", \"Spearman ρ\"])\n",
|
||||||
|
"\n",
|
||||||
|
"print(summary[\"equation\"])\n",
|
||||||
|
"print(summary[\"interpretation\"])\n",
|
||||||
|
"fit = summary[\"cv_r2_5fold\"] if summary[\"cv_r2_5fold\"] is not None else summary[\"r2\"]\n",
|
||||||
|
"if fit < 0.3:\n",
|
||||||
|
" print(f\"\\nCaution: city population explains only about {max(fit, 0):.0%} of the variation in \"\n",
|
||||||
|
" \"visitors, so treat predictions as rough baselines.\")\n",
|
||||||
|
"stats"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"id": "cell-06",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"## 3. The fitted line\n",
|
||||||
|
"\n",
|
||||||
|
"Each dot is a museum. The line is drawn by sending a grid of populations to `POST /predict`, so it\n",
|
||||||
|
"is exactly what the server would answer. The five museums furthest from the line are labelled."
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": 5,
|
||||||
|
"id": "cell-07",
|
||||||
|
"metadata": {},
|
||||||
|
"outputs": [
|
||||||
|
{
|
||||||
|
"data": {
|
||||||
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA8gAAAKJCAYAAABnIQB/AAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjExLjIsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvgI3uAAAAAAlwSFlzAAAQ6wAAEOsBUJTofAAA0TNJREFUeJzs3XlcVOX+B/DPcIYZmBmBAURWQVZXBERz39PSVjWXtFzaS9MWK0u7rb+63bLlVnZtT9O0NDPLfclyDXHfUVFAUcEBgWGZOXN+fyCT4wwwwMAA83m/XrySc57znO85Z2biO88mkyRJAhEREREREZGLc3N2AERERERERESNARNkIiIiIiIiIjBBJiIiIiIiIgLABJmIiIiIiIgIABNkIiIiIiIiIgBMkImIiIiIiIgAMEEmIiIiIiIiAsAEmYiIiIiIiAgAE2QiIiIiIiIiAEyQiYiIiIiIiAAwQa6TRx55BN7e3g12HFWP97Z69XmPeP+pvnz88ccICwtDfn6+s0Npcl5++WVoNBrk5ORUua0yeXl50Gg0eO655+ozzFpr7PHZ66WXXkJ8fDzKysqcHQoRkUtjgnzN9OnTodFosHnz5irL9e7dG76+vnb9UVETEydOhJ+fn0PrpHK8t47lKvfzueeeg0ajMf94eXkhPDwco0aNwvbt263K5+fn47nnnkNkZCRat26NyZMnQ6fTOSHyf+KZOXMmYmNj4efnh8TERHz00UcwmUz1UkdKSgrGjh2LuLg4+Pv7o1OnTpg6dSrS09PrfC25ubn497//jalTp/ILmCao4r2Ul5fn7FAAAMXFxXjllVfQoUMH+Pr6omPHjnjzzTdrlJjaW8fZs2fxf//3f+jbty+Cg4MRERGBwYMH44cffrB6Hz311FO4dOkSPvvsM4dcJxER1ZJEkiRJ0p49eyS1Wi1Nnjy50jIHDx6U1Gq1NH78eIef//7775d8fX0dXi/x3t7o4Ycflry8vGp9vKvcz5kzZ0pqtdrmj7e3t/TXX39ZlP/mm2+kzz//XMrNzZWOHDkitWvXTpoxY4ZTYi8tLZX69u1rM/annnrK4XVs3LhR8vLyslk2NDRUOnv2bJ2u5/XXX5e0Wq2Um5tbp3pc1Zw5cyS1Wi1dvny5VsfrdDpJrVZLM2fOrNXxFe8lnU5Xq+OrU5P4TCaTdNddd9l8rd577712na8mdSQnJ1f6OfLAAw9Y1T1t2jQpPDxcKi0tte/iiYjI4diCfE1SUhI6duyIX3/9tdIufAsWLABQ3oJGRK5h06ZNKCwsRF5eHvbt24c777wTRqMR77//vkW5iRMn4sEHH4Svry/atm2LVq1awdfX1ykxf/XVV9izZw/i4+Oxc+dO5ObmYuXKlfD398fnn3+O/fv3O7SOTz/9FKIo4vHHH8fx48eRm5uLv//+G0OGDIFOp8PChQtrfS1GoxHffvsthg4d6rT7Sc3HihUrsH79ekRGRmLz5s3Izc3Fhg0bEB4ejl9++QXr1693aB1t2rTBrFmzsHXrVmRlZSE9PR3vvfce3N3d8cMPP+DQoUMWdY8bNw45OTlYsWKFoy+diIjsxAT5Ovfffz+Ki4vx448/Wu0zGAxYsmQJQkJCMGjQIAC2x1uePn0aDz74oLmbYWJiIp555hlcuHDBXObG4xITE7Fs2TKUlpZadOlcvXq1uczu3bsxYsQIhIaGws/PDzfddBPmzZsHSZIszj927FiEhISgqKgITz31FCIjI81d2+yJzZbHH38cXl5eyMrKstqXmZkJLy8vPPnkkzW6B7Vx9uxZPPHEE4iLizN393zttddw9epVc5ma3Nsnn3wSXl5eOHv2rM1ztWjRAs8880y1cVXc84KCAjzxxBNo3bo1WrVqhZEjR+L48eNW5Wv6LO2pt0+fPujRo4fVuQ4fPgyNRoMPP/ywymvIz8/HRx99hH79+iEoKAjBwcG488478ffff1uUq+61WtkY5Jpec0lJCWbOnImIiAi0bNkSt99+O06cOFHlNdQ3uVyO6OhozJs3DzKZDEePHq207KxZsyCKol2vn/qwdOlSAOVJbseOHaFUKjFw4EC8/fbbkCQJP/zwg0PrKCkpAQC8+uqrCAkJgVKpRLt27czXX7G/NrZv344LFy5g6NChNvfb87lQ1eeiKIr48MMPkZycDD8/P4SFhWH06NE2v0T46aefMHDgQISFhSE0NBQDBw7E999/D1EUa1TGlldeeQUajQb79u2z2ldQUICAgACMHTvWvE2v1+OLL77A4MGDERoaisDAQAwdOrTaYUIVKhuDvHPnTtx8883w9/dHVFQUXn75ZRgMBqvj7T3/Lbfcgk8//RQAEBoaav7M+Oabb8xlLl68iKeeegpt27aFVqtFTEwMnnrqKZtDFOyNrzJLliwBUD6mvWvXrlAqlejevTs+/vhjAMDixYsdWsfSpUvx0ksvISkpCd7e3vD398cjjzyCKVOmAADOnDljUXf37t2h1WqxfPlyu6+JiIgczKnt141Mbm6u5OvrK/Xr189q34oVKyS1Wi29+uqr5m03dlUtLi6W2rdvb7Mr1ZQpUyo9LiEhweYxv//+uyRJkrRp0yZJq9XaLDN16lSLOMeMGSMFBwdLd955p0W5rKwsu2Kz5c8//5TUarX07rvvWu37z3/+I6nVamnHjh01ugc1dfDgQSk0NNRmvR988EGt7u2BAwcktVotvfbaa1bne/XVVyW1Wi0dPny42tgq7vntt99udZ6wsDDp3Llz5rK1eZb21Nu7d2+pe/fuVrEdOnSo2nskSf90wbzxx9fXV9q7d69d97Oyumtzzffcc49V2bZt20p6vb7a5+EoFd1Cd+3aZbE9Ly9P0mg0UnJystUxZWVl0mOPPSYNHjxYunLlil3nmTdvXqVdMG/8SUhIqLY+o9Eo+fr6Sl26dLHaV1BQILVo0UK6+eabHVrH/PnzJbVaLb3wwgvS+fPnpdLSUun48ePSyJEjJY1GI/3999923Anb3nzzTUmtVksHDx602mfv50Jln4u5ubnS5MmTbR7v5+dn/lyTJElauXJlpc9l27ZtdpepzIkTJyS12nY34e+++05Sq9XSihUrzNs+/vhjm+fx8vKSNm7caHG8rS7Wtrbt3r1b8vX1tapzwoQJVrHZe/6hQ4faLPf1119LkiRJmZmZUkxMjM0y3bp1s3jP1yS+ysTExEhhYWGSyWSy2C6KohQcHCx17ty5Qep4+eWXJbVaLR07dsxq31133SWFhoZKoihWWw8RETkeW5Cv4+vri+HDhyMlJcWqdWjhwoWQyWS47777Kj3+4MGDOHv2LG6//XYcOHAAubm5OHjwID744AOEhIRUetzevXsxcuRIKJVKFBYWmn9uvfVWSJKEp59+GmVlZXjsscdw5MgRZGVl4euvv4ZWq8XXX39t1cqXn5+PEydOYPny5cjOzkZhYSGysrJqFRsA9OrVCxEREVi0aJHVvsWLFyMqKgrdu3ev0z2ozuOPPw6dToe7774b27dvx8WLF7F37168/PLL8PLyqvS4qu5tp06dcNNNN2HBggUWrTuiKGLhwoXo1asX2rdvb1d8Fff8559/RnZ2Nvbu3Ys77rgDV65cweuvvw4AdXqWVdXrCFqtFi+++CJ27tyJS5cu4dSpU/j3v/8NURTx9ttvm8tVdT9tqe01Hz9+HCtXrkR2djYOHjyI/v37IyMjA+vWrXPYNdeU0WhEWloaHn/8cUiShIEDB1rsv3TpEm677TaIoohVq1ZBq9U6Jc6cnByUlpYiOjraap9Go0GrVq1w/vx5h9bx0EMP4d1338Xy5csRExMDX19fJCUlISsrCz/++COSk5NrfT3Hjh0DAERGRlrtq8nngq3PxYMHD2Lp0qVo2bIlvv/+e5w/fx6HDx/GxIkTUVJSgqefftp8/G+//QaZTIbPP/8cmZmZyM7Oxh9//IH7778fcrnc7jKViYmJQbdu3fDjjz/CaDRa7Fu0aBF8fX0t3mdqtRrTp083X3d6ejrmzZsHpVKJN954o2Y3+ZoXX3wRpaWlePTRR3H06FFkZmbiq6++wqZNm6zK2nv+NWvW4PHHHwdQ3uOo4jNj0qRJAIBnn30W58+fxxNPPIH9+/eb/58xceJEHD58GJ9//nmt4rNFkiRcuHABUVFRkMlkFvvc3NzQpk0bmz2lHF3HxYsX8c033+Duu+9GXFyc1f42bdpAp9MhOzvbrusiIiIHc2Z23hht2LBBUqvV0osvvmjelp2dLXl7e0vDhg2zKHtja1laWpqkVqulr776qspz2Gplq2zioyNHjkhqtVq6/fbbrfZVtCrMmTPHvG3MmDGSWq22akGwN7bKvPHGG5JarZb27Nlj3paSkiKp1Wrprbfecth5bDl58qSkVqulwYMHW31jf6Oa3FtJkqRFixZJarVaWr16tXnb77//LqnVamnJkiV2xVdxzzds2GCxvaSkRGrXrp0UEhIimUymWj/L6uqVpLq3IFdmypQpUkhIiMW2qu7njXXX9ppTUlIsyu7bt09Sq9XSm2++aVfcjlDVJF3Dhw+X8vLyLMpXtHRe/zN06NAGi7fCmTNnJLVaLU2aNMnm/vj4eKlNmzYOrUOv10vTpk2T/Pz8LK4/LCxMev/992t9LZIkSbfeeqvN11tNPhcq+1x85plnrFpmJal8Eqb+/ftLarVaSk9PlyRJkl588UUpMDBQKiwsrPQ89pSpyueff27RI0OSJOncuXOSRqOxe8K3WbNmSS1atJCKiorM2+xpQc7JyZHUarXN3gUVPQTsaaG1df7KJum6evWq5OXlJY0ZM8aqHpPJJHXq1Em69dZbHRafXq+X1Gq1uc4bDR48WFKr1VW+nupaR05OjtS9e3cpOTm50knLXn/9dUmtVkv79u2r8nqIiKh+sAX5BgMGDEBYWBgWL15s/hZ/0aJFMBqN1U7OFRUVhdmzZ2PmzJkYMWIE3nnnHWzevLlOaxpWjI8Line truncated
|
||||||
|
"text/plain": [
|
||||||
|
"<Figure size 990x660 with 1 Axes>"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"metadata": {},
|
||||||
|
"output_type": "display_data"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"source": [
|
||||||
|
"points = pd.DataFrame(get(\"/data\"))\n",
|
||||||
|
"\n",
|
||||||
|
"grid = (np.geomspace if is_log else np.linspace)(points.Population.min(), points.Population.max(), 200)\n",
|
||||||
|
"curve = pd.DataFrame(post(\"/predict\", {\"populations\": grid.tolist()}))\n",
|
||||||
|
"\n",
|
||||||
|
"# Residual on the scale the model was fitted on\n",
|
||||||
|
"if is_log:\n",
|
||||||
|
" points[\"residual\"] = np.log10(points.Visitors / points.Predicted)\n",
|
||||||
|
"else:\n",
|
||||||
|
" points[\"residual\"] = points.Visitors - points.Predicted\n",
|
||||||
|
"\n",
|
||||||
|
"fig, ax = plt.subplots(figsize=(9, 6))\n",
|
||||||
|
"ax.scatter(points.Population, points.Visitors, s=40, color=BLUE, alpha=0.75,\n",
|
||||||
|
" edgecolor=\"white\", linewidth=1, label=\"Museum\", zorder=3)\n",
|
||||||
|
"ax.plot(curve.population, curve.predicted_visitors, color=ORANGE, lw=2,\n",
|
||||||
|
" label=f\"Fit: {summary['equation']}\", zorder=4)\n",
|
||||||
|
"\n",
|
||||||
|
"for _, r in points.loc[points.residual.abs().nlargest(5).index].iterrows():\n",
|
||||||
|
" ax.annotate(r.Name, (r.Population, r.Visitors), xytext=(6, 4),\n",
|
||||||
|
" textcoords=\"offset points\", fontsize=8, color=MUTED)\n",
|
||||||
|
"\n",
|
||||||
|
"if is_log:\n",
|
||||||
|
" ax.set_xscale(\"log\")\n",
|
||||||
|
" ax.set_yscale(\"log\")\n",
|
||||||
|
"human_axes(ax, is_log)\n",
|
||||||
|
"ax.set_xlabel(\"City population\")\n",
|
||||||
|
"ax.set_ylabel(\"Annual visitors\")\n",
|
||||||
|
"cv = summary[\"cv_r2_5fold\"]\n",
|
||||||
|
"ax.set_title(f\"Visitors vs. city population R² = {summary['r2']:.2f}\"\n",
|
||||||
|
" + (f\" (cross-validated {cv:.2f})\" if cv is not None else \"\"),\n",
|
||||||
|
" loc=\"left\", fontsize=12)\n",
|
||||||
|
"ax.legend(loc=\"upper left\")\n",
|
||||||
|
"plt.tight_layout()\n",
|
||||||
|
"plt.show()"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"id": "cell-08",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"## 4. How good are the predictions?\n",
|
||||||
|
"\n",
|
||||||
|
"Left: actual vs. predicted visitors. Points on the dashed line are predicted perfectly; above it the\n",
|
||||||
|
"museum beats its prediction. Right: the spread of residuals. A narrow, centred spread means city size\n",
|
||||||
|
"explains a lot; a wide one means other factors matter more."
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": 6,
|
||||||
|
"id": "cell-09",
|
||||||
|
"metadata": {},
|
||||||
|
"outputs": [
|
||||||
|
{
|
||||||
|
"data": {
|
||||||
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAABOEAAAIbCAYAAACt/IquAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjExLjIsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvgI3uAAAAAAlwSFlzAAAQ6wAAEOsBUJTofAAAvrVJREFUeJzs3Xl4VPX5/vF75mRmwswEMkkgQAIBISziAiEgiBuCS9UKRqpWRbQuX1tpLShSte67tloX3Nr+6i5uiBtarQIqVi1G64IgIomCbIEJMJmsZ87vD5qYnSwzmSXv13V5tXPm5Mwzk4XJnc/neWyWZVkCAAAAAAAAEDH2aBcAAAAAAAAAJDpCOAAAAAAAACDCCOEAAAAAAACACCOEAwAAAAAAACKMEA4AAAAAAACIMEI4AAAAAAAAIMII4QAAAAAAAIAII4QDAAAAAAAAIowQDgAAAAAAAIgwQjgAAAAAAAAgwgjhEtQVV1whr9er0tLSaJcSV2677TZ5vV6tW7eu7lhmZqZmzZoVxapa1ly9AABE29KlS+X1evXcc8+16fxDDz1U48aNi3BV7a+rJY3rDdd126q5x3vxxRfl9Xr1+uuvd0kNLdUBAABaRgjXgsrKSg0YMEBer1cPPvhgp641d+5ceb1eBQKBMFWHeNGrVy+de+650S4DANDNPPHEE/J6vXX/paSkKCsrS0cffbSef/75aJeH/3n99dfl9Xr14osvRruUFsVDjQAAxIukaBcQq15++WX5/X716NFDjz32mC688MJol4Qo2bJlS7RLAACgUyzL0s6dO/XBBx/ogw8+kN/v1/nnnx+xx5s8eXK3+uNjVz/fWHl9Y6UOAADiBSvhWvDYY4+pT58+mj9/vj7//HN99tln0S4JAACgXf7yl78oEAho9+7d+uabbzRnzhxJ0h133BHlygAAALofQrhmfP/991q2bJlOP/10zZw5U0lJSXr00UebPXfz5s2aO3euRo0apbS0NB1wwAG6/PLLtX37dknSUUcdpYcffliS1Ldv37ptIU888YQkafr06Ro2bFiT627ZskVer1dXX3113bGysjI9/PDDmjJlirKzs9W3b18de+yxWrZsWYef61VXXSWv16svv/yyyX07d+5URkaGzjjjjLpjP/74o2bPnq19991XGRkZOuCAA3TRRRepqKiowzVcdNFFddt1582bp0GDBql379467rjjmoSftecGg0FdddVVGjZsmFJSUup6or355ps6/vjj1b9/f2VkZOiQQw7Rs88+2+QxTdPULbfcopEjRyojI0OHHXaYli5d2mx9LfWEe+utt3TiiSfWfS6mTJmiF154QZZl6auvvpLX65VpmnrmmWfqPu/p6ekNrhGJegEAaMxms6l///664YYbNHjwYP3444/auXNng3Oqq6t11113afz48UpPT1d2drZOPfVUffXVVw3Oq6mp0R133KGxY8cqMzNTQ4YMUUFBgZYvX153Tku9wjZt2qSzzz5bWVlZ6t+/v04//XRt3ry5Sb0LFiyQ1+tt8tiSNHHiRE2cOLHutmVZ+uc//6lTTjlFubm5Sk9P17hx47RgwQJZltWh16u99Tb3fPf2Ot122236xS9+IUmaOXNm3XuF2vcc9Xu8Pf300xo3bpx8Pp8efPDBvfZie+SRRzR69Gilp6crPz9fTz31VIP72/r67q3GlurYsWOH5s2bp5EjR8rn8yk3N1ezZ89usrug/nN87rnnlJ+fr/T0dI0ePbpJzQAAJAK2ozbjsccek2VZOvPMM5WZmamjjz5azz33nG655RYlJyfXnVdUVKQpU6Y0eEPx3Xff6d5775XD4dD1118f1rr+9re/6corr2xw7P3339e0adP00ksv6Ygjjmj3NU8//XTdddddeuqpp3TzzTc3uG/RokWqqKjQ6aefLmnPm9yTTjqpwRu27777Tt999502btyoxYsXt/vx67vwwgsbXOPdd9/Vscceq3feeUf77rtvk3MXLVpUdzsUCumBBx7QvHnzGpz32Wef6Ve/+pW2bdumiy66qO74xRdfrEceeaTudmFhoU466SQde+yxbar1vvvu0x/+8IcGxz766CN99NFHys/Pb9M1urJeAADqs9vt6tGjR91t0zQ1Y8YMvf3223XHKisr9dprr2n58uV6++23NWrUKEnS1VdfrXvuuafuvLKyMr355pt68803W92auGvXLh1zzDH67rvv6o69/PLLWrNmjQzD6PBzWbVqlU4++eQGx77++mvNnz9f27Zt07XXXtuh63a23o6+To09/fTTTd7ztObJJ59s8H5q9erVuuCCC1ReXt4lfWoDgYCOPvporV69uu7Ypk2b9Mgjj+idd97Ru+++q4yMjAYf88ILL2jhwoV1t7/99ltdcMEFyszM1JQpUyJeMwAAXYWVcI2EQiE9+eSTGj9+vEaMGCFpz1/+SktL9fLLLzc4d86cOdqyZYuOPvpoLV++XFu2bNEXX3yhW2+9tW7F01tvvaULLrhA0p5Vc4FAQIFAQGeeeWa7a/N6vZozZ44++OADbdmyRevXr9eCBQvkdDp10003dej5jhw5Unl5eXr22WdlmmaD+5566illZGTo6KOPliStX79eX331lY444ggVFhZqx44dWrVqlR566CENHjy4Q49f3/vvv68nn3xSP/74o7788kvNnDlTgUBAV111VZNz33vvPT355JPauHGjAoGAXC6XrrjiCvXv31+PP/64ioqKtGXLFr3yyisaPHiwrrvuOvn9fknS559/rkceeUR9+vTR008/rR9//FFffPGFTjnlFL366qt7rfO7777TlVdeqR49eujGG2/UV199pc2bN2vp0qUqKCiQ3W7XqFGjFAgEZBiGTj311LrPe+0Kye+//77L6gUAwLIsbdq0SVdddZXWr1+vSZMmyel01t3/97//XW+//baOPPJILV26tO59xp/+9CdVVFToj3/8Y925r732mjIzM/X6669r69atKi4u1ssvv7zXPwbee++9+u677zRp0iT9+9//1pYtW/TWW2/JMAytWrWqw8/Nbrdr2rRpWrJkiYqKirRp0ya9+uqrys3N1b333qsdO3Z06LqdrXdvr9P8+fPrVpA9/vjjde8VGu++WLx4sa699lqtXbtWgUBAv/nNb1p93JdeeknXXHONvv32WxUVFelPf/qTnE6nrrrqqnb3b2trjfXdd999Wr16tUaPHq133nlHW7Zs0YoVKzRx4kR9//33uuWWW5p8zHPPPaebb75Z3377rTZu3Fj3vvZvf/tbu+oFACDmWWjgX//6l+XxeKy///3vdceqqqqsQYMGWccdd1zdse3bt1ter9caN26cVVVV1eo158yZY3k8Hmv37t1N7ps2bZqVm5vb5PjmzZstj8djXXXVVXuted68eVavXr2s8vLyumOXX3655fF4LL/fv9ePf+CBByyPx2O9+eabdcfWr19veb1e69JLL607VlJSYvXs2dO644479nrN9vjNb35jeTwe65lnnmlw3DRN66CDDrJ69epllZWVtXruXXfdZXk8Huudd95pcv1//vOflsfjsRYvXmxZlmXdfPPNlsfjsZ5//vkmjzd+/HjL4/FY3377bd3xPn36WGeddVbd7TvuuMPyeDzWvffeu9fn1rNnT+tXv/pVk+ORrBcAgMcff9zyeDzN/jdmzJgm/25MmTLFGjBggFVRUdHkWrNnz7ZSU1Pr7ps8ebJ19NFHt/r477zzjuXxeKxnn3227tjBBx9s9e3b19q2bVuDc9esWWOlpKRY+fn5dcfuu+8+y+PxWF9++WWTa0+YMMGaMGHCXl+DN9980/J4PNbrr7/eal0taU+9zV23La/TkiVLLI/HYy1atKjJfYsWLbI8Ho91/vnnN7mvucerPf+8885rcv4VV1xheTwe67XXXrMsq32vb2s1tvR5TktLs3744YcG527bts3q379/g/e9tTXXf79Za+zYsdYBBxzQ5DgAAPGMlXCNPProo+rRo4dmzJhRd8zhcOi0007Tu+++W9f77LvvvpNlWTrmmGPkcDi6pLZgMKibbrpJBx10kPr06VPXl+P+++9XTU2NSkpKOnTdX/ziF3I4HA16bzz11FOyLKtBP7j09HTdeuutuu2223TCCSfo5ptv1ptvvqny8vJOPzdJTf6Cbrfbdfjhh6umpkYbNmxo9dw1a9ZIkn7+85/XvS61/xUUFEiSNm7cKEl1n8PJkyc3eby2bOmt7T93wgkntOVpNasr6wUAoJZhGLrooos0ZMiQBsfXrFmjHTt2KD09vcm/S//4xz9UXV2trVu3SpJuvfVWfffddxo3bpwuv/xyPfPMM3X3taaoqEj7779/k62Iw4YN04ABAzr1vJ555hkdc8wxGjBggFJSUuT1enXSSSdJ2rMVsiM6W29HX6fG2vtvfeP3C/WPrV+/vt2P317FxcUaPny4srOzGxzPyMjQmDFj9OOPP6qysrLBfQcffHCT6+Tk5DB5FQCQcAjh6tmxY4dee+01lZeXq3///g3egN57772yLEuPPfaYJHW60W8tu93e7LWaC7ZmzpypW265RV999ZWCwWCT+xu/oWmr9PR0HXPMMXr11Ve1e/duSXvezI4cOVKjR49ucO6vf/1rff3115o5c6Z27dqlm2++Wbm5uXryySc79NgdlZaW1uB2Wz4f1dXVez23LdcJx+e+K+sFAHRftdNRd+zYoffee0/777+/Lr74Yi1ZsqTBeW3596SqqkqSNH78eH355Ze6+eab5fF49Mwzz2i//fbTxRdf3Gq/svb8e2az2Vr8mIqKiga3H3jgAZ177rlasWKF/H5/k4/p6Pujzv7729HXqTGfz9fmc9uqPa9ve1mWVXf9vT1+rfo9l+ufw/scAECiIYSrZ+HChXt9o/bkk08qFAppyJAhstlsevPNN+vCkpbY7Xte5sY916Q9fxXctm1bk9Dtgw8+aHB78+bN+ucLine truncated
|
||||||
|
"text/plain": [
|
||||||
|
"<Figure size 1320x550 with 2 Axes>"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"metadata": {},
|
||||||
|
"output_type": "display_data"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "stdout",
|
||||||
|
"output_type": "stream",
|
||||||
|
"text": [
|
||||||
|
"82% of museums are within 2× of their predicted visitors.\n"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"source": [
|
||||||
|
"fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 5))\n",
|
||||||
|
"\n",
|
||||||
|
"ax1.scatter(points.Predicted, points.Visitors, s=40, color=BLUE, alpha=0.75,\n",
|
||||||
|
" edgecolor=\"white\", linewidth=1, zorder=3)\n",
|
||||||
|
"lo = min(points.Predicted.min(), points.Visitors.min()) * 0.9\n",
|
||||||
|
"hi = max(points.Predicted.max(), points.Visitors.max()) * 1.1\n",
|
||||||
|
"ax1.plot([lo, hi], [lo, hi], color=MUTED, lw=1, ls=\"--\", label=\"Perfect prediction\")\n",
|
||||||
|
"if is_log:\n",
|
||||||
|
" ax1.set_xscale(\"log\")\n",
|
||||||
|
" ax1.set_yscale(\"log\")\n",
|
||||||
|
"ax1.set_xlim(lo, hi) # same range on both axes so the diagonal is a true 45° line\n",
|
||||||
|
"ax1.set_ylim(lo, hi)\n",
|
||||||
|
"ax1.set_aspect(\"equal\")\n",
|
||||||
|
"human_axes(ax1, is_log)\n",
|
||||||
|
"ax1.set_xlabel(\"Predicted visitors\")\n",
|
||||||
|
"ax1.set_ylabel(\"Actual visitors\")\n",
|
||||||
|
"ax1.set_title(\"Actual vs. predicted\", loc=\"left\", fontsize=12)\n",
|
||||||
|
"ax1.legend(loc=\"lower right\")\n",
|
||||||
|
"\n",
|
||||||
|
"ax2.hist(points.residual, bins=20, color=BLUE, edgecolor=\"white\", linewidth=2)\n",
|
||||||
|
"ax2.axvline(0, color=MUTED, lw=1)\n",
|
||||||
|
"if is_log:\n",
|
||||||
|
" ticks = np.array([0.1, 0.2, 0.5, 1, 2, 5, 10])\n",
|
||||||
|
" ticks = ticks[(np.log10(ticks) >= points.residual.min() - 0.1) &\n",
|
||||||
|
" (np.log10(ticks) <= points.residual.max() + 0.1)]\n",
|
||||||
|
" ax2.set_xticks(np.log10(ticks), [f\"{t:g}×\" for t in ticks])\n",
|
||||||
|
" ax2.set_xlabel(\"Actual ÷ predicted visitors (log scale)\")\n",
|
||||||
|
"else:\n",
|
||||||
|
" ax2.xaxis.set_major_formatter(FuncFormatter(human))\n",
|
||||||
|
" ax2.set_xlabel(\"Actual − predicted visitors\")\n",
|
||||||
|
"ax2.set_ylabel(\"Museums\")\n",
|
||||||
|
"ax2.set_title(\"Residual distribution\", loc=\"left\", fontsize=12)\n",
|
||||||
|
"ax2.grid(axis=\"x\", visible=False)\n",
|
||||||
|
"\n",
|
||||||
|
"plt.tight_layout()\n",
|
||||||
|
"plt.show()\n",
|
||||||
|
"\n",
|
||||||
|
"within_2x = (points.residual.abs() <= np.log10(2)).mean() if is_log else None\n",
|
||||||
|
"if within_2x is not None:\n",
|
||||||
|
" print(f\"{within_2x:.0%} of museums are within 2× of their predicted visitors.\")"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"id": "cell-10",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"## 5. Which museums beat (or miss) their city's size?\n",
|
||||||
|
"\n",
|
||||||
|
"`GET /residuals` ranks museums by actual ÷ predicted visitors. Above 1× (blue) the museum draws more\n",
|
||||||
|
"than its city's size alone would suggest; below 1× (orange) it draws fewer."
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": 7,
|
||||||
|
"id": "cell-11",
|
||||||
|
"metadata": {},
|
||||||
|
"outputs": [
|
||||||
|
{
|
||||||
|
"data": {
|
||||||
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA9MAAALhCAYAAABcyPdXAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjExLjIsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvgI3uAAAAAAlwSFlzAAAQ6wAAEOsBUJTofAABAABJREFUeJzs3XdUFFcbBvCHXfouvRcBRZpYAHvB3ntMM7GkGE30S0+s0dhrjNGYoib2HnvvAvaODSyoKNKl97bs9wdhZd2F3VEQMc/vHE7cmTv3vvfOSHz3ztzRkcvlchARERERERGR1kTVHQARERERERFRTcNkmoiIiIiIiEggJtNEREREREREAjGZJiIiIiIiIhKIyTQRERERERGRQEymiYiIiIiIiARiMk1EREREREQkEJNpIiIiIiIiIoGYTBMREREREREJxGSaiIiIiIiISCAm00REREREr6CEhARIpVL8+OOP1R2Kilc5tsp28OBBtG7dGjY2NpBKpThx4kR1h0SvCN3qDoCIiIjovywvLw/W1tZal4+Ojoa5uXnVBVTG48ePMX/+fAQHByMmJgaWlpZo1KgRvvvuO7Ro0eKlxEBUnWJiYjB48GDk5eVVdyj0CmIyTUREREQq4uLi0KZNGyQnJyu2xcbGIjY2FocOHcLmzZvRo0ePaoyQqOqFhIQgLy8Pc+fOxfDhw6Gvr1/dIdErhLd5ExEREVUjQ0NDZGVlKf0sWbIEALBw4UKVfS9rVnrDhg1ITk5Gx44dceHCBSQnJ+Pu3bv48ssvUVxcjD/++OOlxEFUneLi4gAAnTp1YiJNKphMExEREdUQ0dHR+Oyzz+Du7g4LCwv4+vpiwoQJyMjIUCq3aNEiSKVS3Lx5E/Pnz4e3tzesrKwQGBiIw4cPa9VW6W2t33zzDerVqwcDAwM4Ojpi2rRpEIvFWt/2qk3Ma9euhVQqxbZt21SOl8vlqF+/Ppo2baq0PTMzE1OmTIGfnx8sLS3h6uqKYcOG4fHjx+WOxeLFi9GwYUOYmZnhwIEDWsVfVukXHR07doSTkxMcHBzQs2dPtc/Qfvrpp5BKpSgsLMSPP/4Id3d3WFtbo1u3brh+/bpK+ZMnT6JTp06wtraGh4cHpk6dCplMpnVsvXv3Rr169VS2x8TEQCqVYurUqS8lttjYWHz55Zfw8vKChYUFvLy8MHr0aKSnp6sdn/z8fEyYMAEeHh4wMTHBo0ePyu1jYGAgmjZtioSEBAwePBiOjo5wdHTE0KFDERMTo1I+LCwMQ4YMgZubGywtLeHn54d58+ahqKhIbb1JSUkYPnw4XFxcYGdnB6lUismTJwMAmjRpAqlUipYtWyqOO3r0KLp37w57e3vY2Nigbdu22LRpU7lxl63f3t4ejx49glQqxcyZMxESEoJ27drBxsYG/v7+2LVrFwAgOTlZcYyDgwNGjhyJnJwctePzzz//oFOnTrCzs4OtrS26dOmCI0eOKJUp2+apU6fQpUsX2Nra4tNPPwUAbNq0Ce3bt4ezszNq1aqFzp07Y/PmzSguLi73vPynyYmIiIjolbJ27Vq5RCKR//XXX4ptsbGxcnd3d7lEIlH5ad26tTw3N1dRduHChXKJRCIfNGiQSllTU1P5wYMHNcZw7do1uampqbxfv37yW7duyfPz8+WxsbHyiRMnyiUSiXzx4sUa69A25szMTLmNjY38zTffVKnj1KlTcolEIl+wYIFiW2ZmprxFixZq63V3d5fHxcWpjMWQIUOUyu3du1dj/M+aP3++2jbNzMzkJ0+eVCo7YsQIuUQikQ8bNkylvKurqzwtLU2pjxYWFirlBg8eLJdIJPJJkyZpjK1Xr15yHx8fle3R0dFyiUQinzJlSpXHFhkZKa9Tp47aMWrTpo08Ly9PJYZnz8uDBw/K7WObNm3kfn5+8ubNm6vU7+vrK09NTVWUPXnypNza2lptLB988IHaelu2bKkoY2VlpfbYFi1ayOVyuXzTpk1yqVSqtszs2bM11m9tbS1/+PChXCKRyN99912VMTYzM5OfO3dObV9Hjx6tMjaTJk1SG4tUKpVv3bpVUa68Nj/55BP5li1b1NYhkUjkly9fLve8/JdxZpqIiIioBpg+fTri4+PRsWNHnDt3DgkJCTh06BC8vb1x9epVLFu2TOWYgwcPYtGiRXj06BEiIiIwfvx4yGQyjB49GnK5vML2GjZsiO3btyMpKQlNmjSBpaUlPDw8sGnTJsyZMwf/+9//Ki1mqVSKvn374ujRo0hMTFSqY+PGjRCJRBg4cKBi25w5c3Djxg28/fbbOHfuHJKSkhAREYGxY8ciPj4es2fPVonlwIEDWLRoESIjI5GVlYVevXppjP9ZJiYm+O6773D27FkkJCQgMjISixcvhlgsxsyZM9Uec+rUKWzZsgWxsbG4ffs2evXqhaSkJMXMIwCMGzcOBQUF+PLLL3H79m08fvwYy5Ytw9GjRwXHKERlx/btt98iISEBX3/9Na5fv47k5GRcu3YN7733HkJDQ7Fq1SqVY06fPo2NGzciNjYWWVlZqF27doUxR0REoKioCIcPH0ZCQgLOnTuHwMBAPHz4EAsXLgQAyGQyfPbZZyguLsbcuXMRERGBpKQknDlzBm3btsXWrVtx8uRJtfUeOnQIT548QVJSErKysjBp0iQAwOXLl5GVlYWzZ88iNzcXo0ePho6ODiZNmoSIiAg8evQIixYtgqGhIWbPno2oqKgK63/y5Ili3969e/HJJ5/gzp07iI2NxfTp01FUVIQ333wTIpEIx48fR2JiIo4fPw47OzusWrUK+fn5iuMvXbqEBQsWwNPTE9u3b0d0dDRiY2OxceNGWFlZYezYsSgsLFSKZ+/evXj77bdx7do1ZGRk4K+//sLevXshFouxYsUKxMTEID4+HsHBwXj//fchFosrPC//WdWdzRMRERGRMnUz07Vq1ZK7ubnJMzIylMrevn1bbmJiIu/cubNiW+ls7LRp01TqLp2tvnHjhsY4Fi9eLHd1dVWaobK0tJSPGjVKnp2drfF4ITEfP35cLpFI5L/99ptiW25urtzR0VHet29fpePr1q0rb968uby4uFilzb59+8q9vLwUn0vHYvr06RrjfV7ffPON3MLCQl5QUKDYVjrzeuzYMaWykZGRcolEIh8zZoxcLi+ZvZdIJPLevXur1Lt48eIqnZmuzNiSkpLkUqlUZdZXLpfLZTKZ3NPTU/7GG2+oxLBt2zaNfSvVpk0buUQikYeFhSltT05Oljs6OsobN24sl8uf3s0wd+5clTqSkpLkJiYm8h9++EGl3lu3bqmUnzNnjlwikchv376t2Hb06FG5RCKRf/bZZyrlZ8+eLZdIJPI///xTY/2ls8RdunRRqcfLy0sukUjk9+/fV9o+bdo0uUQikd+7d0+x7fvvv5dLJBL5nTt3VOpZsWKFXCKRyC9evKjUZrNmzeQymUyp7LfffiuvVauWPCcnR6UeUo8z00RERESvuPT0dKSkpKB58+YwMTFR2ufl5QU3NzdERkaqHNehQ4dytz18+LDCNleuXIlx48ahWbNmOH/+PJKSknDnzh188cUXWL16NT7//PNKjbldu3ZwcnLChg0bFNv27t2L9PR0DBo0SLEtIyMDcXFxuHnzJkxMTCCVSpV+jh07htjYWJWZ9/bt2yv+vGrVKpXjtmzZUmF/ACA7OxszZsxAs2bNYGtrqzh22bJlKCgoUFr5vFSrVq2UPru6uirqAqAYg44dO6ocq25bZarM2CIiIiCXy7F161aVsTU1NUVMTIza55rLnpcxY8aoHBsWFqZU3t7eXuXZcEtLSzRs2FBxTd+5cwcAMG3aNJX6XF1dUVxcrBKLvb09vL291Y7Ts0rb6dSpk8q+rl27KpXRpn51r5lzdnaGra0t6tSpo7IdgNLMdml/AwICVPr7xRdfAIBKfwMDAyESKaeC33//PZydndGoUSN88cUXWLFiBe7fv682ZirBZJqIiIjoFVeaGOro6Ly0NtetWwc9PT2sWrUKvr6+MDQ0hJOTE6ZMmYJWrVph69atFS5CJjTm0lu5r127pkigNm7cCBMTE/Tu3Vul3ooUFxerLJBlYWGhVRwVef/99zFnzhyEh4erXQSq7K23pQwNDZU+l45HaT8q6o82fS0lEonUls/NzS33mMqMTZtYCwoKVLZZWlpqPE6o54lFyPXxPH8fK6rfwMBAZZuOjo7K+SnbZtkFwSqrvw4ODjh16hTWrl0LNzc3BAcHo127dujfvz9SUlI0tvFfxGSaiIiI6BVnbm4OS0tLnD9/HllZWUr7IiIi8PDhQ7XPmgYFBZW7zc3NrcI2K/rHs1wuR3FxscoKzS8a8/vvvw+gJIlOTEzEsWPH0L9/fxgbGyvKmJmZwcbGBi1atFB5bVjZH11d3XJj+/DDD1XKv/322xWOR3R0NI4dO4Y2bdrg3LlziI+PR2ZmJrKysvDZZ59VeGxFSmcejx8/rrJP3fkrj7W1NRISElS+4Dhz5sxLic3d3R06Ojpqx7b0JzQ0tML25s2bp3KMr6+vUpn4+HiEh4crbUtJScH169cV13TdunXLra/0Z/369YLGoqzS6/bYsWMq+0qfJdf096sy1a1bFyKRCHfu3Cm3v5qu71IikQjNmzfHd999hzVr1iA0NBQhISGYM2dOFfeiZmIyTURERFQD9OrVC4mJiRgyZAjCw8ORnZ2Ns2fPYtCgQSguLlaavS21aNEirFixAsnJyYiPj8esWbOwc+dO1KlTRyVJeVaDBg1QWFiIDz/8EOHh4cjPz0dMTAymTJmCs2fPwt7eHra2tpUas5eXF5o0aYLNmzdj06ZNKCoqUiTYZb333ns4d+4cRowYgatXryIzMxOZmZm4du0aZsyYgfHjx2sxos/H0NAQUqkUBgYGiI+Pxx9//IGVK1c+d30ODg7w9/dHUFAQJkyYgJiYGKSlpWHjxo3lLmqmjqenJwoKCjB69GgLine truncated
|
||||||
|
"text/plain": [
|
||||||
|
"<Figure size 990x748 with 1 Axes>"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"metadata": {},
|
||||||
|
"output_type": "display_data"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"data": {
|
||||||
|
"text/html": [
|
||||||
|
"<div>\n",
|
||||||
|
"<style scoped>\n",
|
||||||
|
" .dataframe tbody tr th:only-of-type {\n",
|
||||||
|
" vertical-align: middle;\n",
|
||||||
|
" }\n",
|
||||||
|
"\n",
|
||||||
|
" .dataframe tbody tr th {\n",
|
||||||
|
" vertical-align: top;\n",
|
||||||
|
" }\n",
|
||||||
|
"\n",
|
||||||
|
" .dataframe thead th {\n",
|
||||||
|
" text-align: right;\n",
|
||||||
|
" }\n",
|
||||||
|
"</style>\n",
|
||||||
|
"<table border=\"1\" class=\"dataframe\">\n",
|
||||||
|
" <thead>\n",
|
||||||
|
" <tr style=\"text-align: right;\">\n",
|
||||||
|
" <th></th>\n",
|
||||||
|
" <th>Name</th>\n",
|
||||||
|
" <th>City</th>\n",
|
||||||
|
" <th>Country</th>\n",
|
||||||
|
" <th>Population</th>\n",
|
||||||
|
" <th>Visitors</th>\n",
|
||||||
|
" <th>Predicted</th>\n",
|
||||||
|
" <th>Ratio</th>\n",
|
||||||
|
" </tr>\n",
|
||||||
|
" </thead>\n",
|
||||||
|
" <tbody>\n",
|
||||||
|
" <tr>\n",
|
||||||
|
" <th>0</th>\n",
|
||||||
|
" <td>Louvre</td>\n",
|
||||||
|
" <td>Paris</td>\n",
|
||||||
|
" <td>France</td>\n",
|
||||||
|
" <td>11060000.0</td>\n",
|
||||||
|
" <td>9000000</td>\n",
|
||||||
|
" <td>2710895.23</td>\n",
|
||||||
|
" <td>3.32</td>\n",
|
||||||
|
" </tr>\n",
|
||||||
|
" <tr>\n",
|
||||||
|
" <th>1</th>\n",
|
||||||
|
" <td>Shenzhen Museum</td>\n",
|
||||||
|
" <td>Shenzhen</td>\n",
|
||||||
|
" <td>China</td>\n",
|
||||||
|
" <td>14678000.0</td>\n",
|
||||||
|
" <td>6805000</td>\n",
|
||||||
|
" <td>2811109.70</td>\n",
|
||||||
|
" <td>2.42</td>\n",
|
||||||
|
" </tr>\n",
|
||||||
|
" <tr>\n",
|
||||||
|
" <th>2</th>\n",
|
||||||
|
" <td>National Museum of China</td>\n",
|
||||||
|
" <td>Beijing</td>\n",
|
||||||
|
" <td>China</td>\n",
|
||||||
|
" <td>21893095.0</td>\n",
|
||||||
|
" <td>7031700</td>\n",
|
||||||
|
" <td>2959030.62</td>\n",
|
||||||
|
" <td>2.38</td>\n",
|
||||||
|
" </tr>\n",
|
||||||
|
" <tr>\n",
|
||||||
|
" <th>3</th>\n",
|
||||||
|
" <td>British Museum</td>\n",
|
||||||
|
" <td>London</td>\n",
|
||||||
|
" <td>United Kingdom</td>\n",
|
||||||
|
" <td>11262000.0</td>\n",
|
||||||
|
" <td>6440120</td>\n",
|
||||||
|
" <td>2717195.79</td>\n",
|
||||||
|
" <td>2.37</td>\n",
|
||||||
|
" </tr>\n",
|
||||||
|
" <tr>\n",
|
||||||
|
" <th>4</th>\n",
|
||||||
|
" <td>Natural History Museum, South Kensington</td>\n",
|
||||||
|
" <td>London</td>\n",
|
||||||
|
" <td>United Kingdom</td>\n",
|
||||||
|
" <td>11262000.0</td>\n",
|
||||||
|
" <td>6301972</td>\n",
|
||||||
|
" <td>2717195.79</td>\n",
|
||||||
|
" <td>2.32</td>\n",
|
||||||
|
" </tr>\n",
|
||||||
|
" <tr>\n",
|
||||||
|
" <th>5</th>\n",
|
||||||
|
" <td>China Science and Technology Museum</td>\n",
|
||||||
|
" <td>Beijing</td>\n",
|
||||||
|
" <td>China</td>\n",
|
||||||
|
" <td>21893095.0</td>\n",
|
||||||
|
" <td>6421000</td>\n",
|
||||||
|
" <td>2959030.62</td>\n",
|
||||||
|
" <td>2.17</td>\n",
|
||||||
|
" </tr>\n",
|
||||||
|
" <tr>\n",
|
||||||
|
" <th>6</th>\n",
|
||||||
|
" <td>Nanjing Museum</td>\n",
|
||||||
|
" <td>Nanjing</td>\n",
|
||||||
|
" <td>China</td>\n",
|
||||||
|
" <td>9341685.0</td>\n",
|
||||||
|
" <td>5680000</td>\n",
|
||||||
|
" <td>2652816.45</td>\n",
|
||||||
|
" <td>2.14</td>\n",
|
||||||
|
" </tr>\n",
|
||||||
|
" <tr>\n",
|
||||||
|
" <th>7</th>\n",
|
||||||
|
" <td>Metropolitan Museum of Art</td>\n",
|
||||||
|
" <td>New York City</td>\n",
|
||||||
|
" <td>United States</td>\n",
|
||||||
|
" <td>19268388.0</td>\n",
|
||||||
|
" <td>5984091</td>\n",
|
||||||
|
" <td>2910956.74</td>\n",
|
||||||
|
" <td>2.06</td>\n",
|
||||||
|
" </tr>\n",
|
||||||
|
" <tr>\n",
|
||||||
|
" <th>8</th>\n",
|
||||||
|
" <td>Museum of Science</td>\n",
|
||||||
|
" <td>Boston</td>\n",
|
||||||
|
" <td>United States</td>\n",
|
||||||
|
" <td>4453352.0</td>\n",
|
||||||
|
" <td>1324000</td>\n",
|
||||||
|
" <td>2412348.33</td>\n",
|
||||||
|
" <td>0.55</td>\n",
|
||||||
|
" </tr>\n",
|
||||||
|
" <tr>\n",
|
||||||
|
" <th>9</th>\n",
|
||||||
|
" <td>Palacio de Cristal del Retiro</td>\n",
|
||||||
|
" <td>Madrid</td>\n",
|
||||||
|
" <td>Spain</td>\n",
|
||||||
|
" <td>6211000.0</td>\n",
|
||||||
|
" <td>1318823</td>\n",
|
||||||
|
" <td>2517507.58</td>\n",
|
||||||
|
" <td>0.52</td>\n",
|
||||||
|
" </tr>\n",
|
||||||
|
" <tr>\n",
|
||||||
|
" <th>10</th>\n",
|
||||||
|
" <td>Smithsonian American Art Museum (with Renwick ...</td>\n",
|
||||||
|
" <td>Washington, D.C.</td>\n",
|
||||||
|
" <td>United States</td>\n",
|
||||||
|
" <td>5230370.0</td>\n",
|
||||||
|
" <td>1273450</td>\n",
|
||||||
|
" <td>2462626.87</td>\n",
|
||||||
|
" <td>0.52</td>\n",
|
||||||
|
" </tr>\n",
|
||||||
|
" <tr>\n",
|
||||||
|
" <th>11</th>\n",
|
||||||
|
" <td>Art Institute of Chicago</td>\n",
|
||||||
|
" <td>Chicago</td>\n",
|
||||||
|
" <td>United States</td>\n",
|
||||||
|
" <td>8609571.0</td>\n",
|
||||||
|
" <td>1324241</td>\n",
|
||||||
|
" <td>2625192.03</td>\n",
|
||||||
|
" <td>0.50</td>\n",
|
||||||
|
" </tr>\n",
|
||||||
|
" <tr>\n",
|
||||||
|
" <th>12</th>\n",
|
||||||
|
" <td>Getty Center</td>\n",
|
||||||
|
" <td>Los Angeles</td>\n",
|
||||||
|
" <td>United States</td>\n",
|
||||||
|
" <td>11984083.0</td>\n",
|
||||||
|
" <td>1301332</td>\n",
|
||||||
|
" <td>2738940.93</td>\n",
|
||||||
|
" <td>0.48</td>\n",
|
||||||
|
" </tr>\n",
|
||||||
|
" <tr>\n",
|
||||||
|
" <th>13</th>\n",
|
||||||
|
" <td>Centro Cultural Banco do Brasil</td>\n",
|
||||||
|
" <td>São Paulo</td>\n",
|
||||||
|
" <td>Brazil</td>\n",
|
||||||
|
" <td>23086000.0</td>\n",
|
||||||
|
" <td>1364208</td>\n",
|
||||||
|
" <td>2979235.54</td>\n",
|
||||||
|
" <td>0.46</td>\n",
|
||||||
|
" </tr>\n",
|
||||||
|
" <tr>\n",
|
||||||
|
" <th>14</th>\n",
|
||||||
|
" <td>Chinese Aviation Museum</td>\n",
|
||||||
|
" <td>Beijing</td>\n",
|
||||||
|
" <td>China</td>\n",
|
||||||
|
" <td>21893095.0</td>\n",
|
||||||
|
" <td>1292278</td>\n",
|
||||||
|
" <td>2959030.62</td>\n",
|
||||||
|
" <td>0.44</td>\n",
|
||||||
|
" </tr>\n",
|
||||||
|
" <tr>\n",
|
||||||
|
" <th>15</th>\n",
|
||||||
|
" <td>Moscow Kremlin Museum</td>\n",
|
||||||
|
" <td>Moscow</td>\n",
|
||||||
|
" <td>Russia</td>\n",
|
||||||
|
" <td>19100000.0</td>\n",
|
||||||
|
" <td>1240113</td>\n",
|
||||||
|
" <td>2907681.34</td>\n",
|
||||||
|
" <td>0.43</td>\n",
|
||||||
|
" </tr>\n",
|
||||||
|
" </tbody>\n",
|
||||||
|
"</table>\n",
|
||||||
|
"</div>"
|
||||||
|
],
|
||||||
|
"text/plain": [
|
||||||
|
" Name City \\\n",
|
||||||
|
"0 Louvre Paris \n",
|
||||||
|
"1 Shenzhen Museum Shenzhen \n",
|
||||||
|
"2 National Museum of China Beijing \n",
|
||||||
|
"3 British Museum London \n",
|
||||||
|
"4 Natural History Museum, South Kensington London \n",
|
||||||
|
"5 China Science and Technology Museum Beijing \n",
|
||||||
|
"6 Nanjing Museum Nanjing \n",
|
||||||
|
"7 Metropolitan Museum of Art New York City \n",
|
||||||
|
"8 Museum of Science Boston \n",
|
||||||
|
"9 Palacio de Cristal del Retiro Madrid \n",
|
||||||
|
"10 Smithsonian American Art Museum (with Renwick ... Washington, D.C. \n",
|
||||||
|
"11 Art Institute of Chicago Chicago \n",
|
||||||
|
"12 Getty Center Los Angeles \n",
|
||||||
|
"13 Centro Cultural Banco do Brasil São Paulo \n",
|
||||||
|
"14 Chinese Aviation Museum Beijing \n",
|
||||||
|
"15 Moscow Kremlin Museum Moscow \n",
|
||||||
|
"\n",
|
||||||
|
" Country Population Visitors Predicted Ratio \n",
|
||||||
|
"0 France 11060000.0 9000000 2710895.23 3.32 \n",
|
||||||
|
"1 China 14678000.0 6805000 2811109.70 2.42 \n",
|
||||||
|
"2 China 21893095.0 7031700 2959030.62 2.38 \n",
|
||||||
|
"3 United Kingdom 11262000.0 6440120 2717195.79 2.37 \n",
|
||||||
|
"4 United Kingdom 11262000.0 6301972 2717195.79 2.32 \n",
|
||||||
|
"5 China 21893095.0 6421000 2959030.62 2.17 \n",
|
||||||
|
"6 China 9341685.0 5680000 2652816.45 2.14 \n",
|
||||||
|
"7 United States 19268388.0 5984091 2910956.74 2.06 \n",
|
||||||
|
"8 United States 4453352.0 1324000 2412348.33 0.55 \n",
|
||||||
|
"9 Spain 6211000.0 1318823 2517507.58 0.52 \n",
|
||||||
|
"10 United States 5230370.0 1273450 2462626.87 0.52 \n",
|
||||||
|
"11 United States 8609571.0 1324241 2625192.03 0.50 \n",
|
||||||
|
"12 United States 11984083.0 1301332 2738940.93 0.48 \n",
|
||||||
|
"13 Brazil 23086000.0 1364208 2979235.54 0.46 \n",
|
||||||
|
"14 China 21893095.0 1292278 2959030.62 0.44 \n",
|
||||||
|
"15 Russia 19100000.0 1240113 2907681.34 0.43 "
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"execution_count": 7,
|
||||||
|
"metadata": {},
|
||||||
|
"output_type": "execute_result"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"source": [
|
||||||
|
"N = 8\n",
|
||||||
|
"over = pd.DataFrame(get(\"/residuals\", top=N, order=\"over\"))\n",
|
||||||
|
"under = pd.DataFrame(get(\"/residuals\", top=N, order=\"under\"))\n",
|
||||||
|
"res = (pd.concat([over, under.iloc[::-1]])\n",
|
||||||
|
" .drop_duplicates(\"Name\")\n",
|
||||||
|
" .sort_values(\"Ratio\"))\n",
|
||||||
|
"\n",
|
||||||
|
"fig, ax = plt.subplots(figsize=(9, 0.35 * len(res) + 1.2))\n",
|
||||||
|
"colors = [BLUE if r >= 1 else ORANGE for r in res.Ratio]\n",
|
||||||
|
"ax.barh(res.Name + \" (\" + res.City + \")\", np.log10(res.Ratio), color=colors, height=0.7)\n",
|
||||||
|
"ax.axvline(0, color=MUTED, lw=1)\n",
|
||||||
|
"ticks = np.array([0.05, 0.1, 0.2, 0.5, 1, 2, 5, 10, 20])\n",
|
||||||
|
"ticks = ticks[(np.log10(ticks) >= np.log10(res.Ratio).min() - 0.1) &\n",
|
||||||
|
" (np.log10(ticks) <= np.log10(res.Ratio).max() + 0.1)]\n",
|
||||||
|
"ax.set_xticks(np.log10(ticks), [f\"{t:g}×\" for t in ticks])\n",
|
||||||
|
"ax.set_xlabel(\"Actual ÷ predicted visitors (log scale)\")\n",
|
||||||
|
"ax.set_title(f\"Top {N} over- and under-performers\", loc=\"left\", fontsize=12)\n",
|
||||||
|
"ax.grid(axis=\"y\", visible=False)\n",
|
||||||
|
"plt.tight_layout()\n",
|
||||||
|
"plt.show()\n",
|
||||||
|
"\n",
|
||||||
|
"res[[\"Name\", \"City\", \"Country\", \"Population\", \"Visitors\", \"Predicted\", \"Ratio\"]] \\\n",
|
||||||
|
" .sort_values(\"Ratio\", ascending=False).round(2).reset_index(drop=True)"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"id": "cell-12",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"## 6. Predict for new cities\n",
|
||||||
|
"\n",
|
||||||
|
"Edit `cities` and re-run. A single city uses `GET /predict?population=...`; a batch uses `POST /predict`.\n",
|
||||||
|
"The predictions are drawn on the fitted line from section 3."
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": 8,
|
||||||
|
"id": "cell-13",
|
||||||
|
"metadata": {},
|
||||||
|
"outputs": [
|
||||||
|
{
|
||||||
|
"name": "stdout",
|
||||||
|
"output_type": "stream",
|
||||||
|
"text": [
|
||||||
|
"Montréal (single call): 2.14M visitors\n"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"data": {
|
||||||
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA9IAAAKJCAYAAABXrRFeAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjExLjIsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvgI3uAAAAAAlwSFlzAAAQ6wAAEOsBUJTofAAA4BtJREFUeJzs3XlYlOX6B/DvzDAMMsOIrK6g4DoDuKSWlimRibmvqbn00/S0HFvMFpM0TlqnRT1mtp86mhkqeszKVAy10+LWIjKjpiCuKaLiAAMD877v7w9kdAKEGQZmgO/nurySd72HeSHvee7nfmSSJEkgIiIiIiIiomqRuzsAIiIiIiIiovqEiTQRERERERGRA5hIExERERERETmAiTQRERERERGRA5hIExERERERETmAiTQRERERERGRA5hIExERERERETmAiTQRERERERGRA5hIExERERERETmAiTQRERERERGRA5hIe6jXX38dGo0GGRkZtm2hoaGYNm2aG6OqXEXxuoKzr9mTv1e38s4776Bbt27w9/eHRqNBUVGRu0Oqlxx9/+vr80JERERE7sFE+ibHjh2DRqOx+xMaGoq77roLH3zwAURRdHeITmvatClmzJjh7jDcytO/B9u2bcMLL7yAEydOwGq1ujucBsfT338iIiIiqj+83B2ApysoKMDvv/+O33//HX/88QeWLFnitlguXrzotnu7i7OvuT5+r7Zt2wYASElJwe233w65nJ9zOcvR978+Pi9ERERE5D78l3oFHnzwQeTn5yM/Px9ZWVlYsmQJFAoFPvroI/6Dm2rNhQsXoNVq0adPHybRREREREQejP9ar0JQUBD+9re/YeTIkRBFEenp6QCAxx9/HBqNBmazGS+99BI6duwIPz8/2xzhHTt2YMiQIWjZsiWCgoJw1113Yf369eWuLwgCXnvtNXTp0gVBQUG4++67sWvXrgpjqWweZ0pKCoYPH47WrVujefPmiIuLw8aNGyFJEgwGAzQaDQRBwLp162wl64GBgXbXqI14/6qkpATh4eGIi4urcP/KlSuh0WjsrlfRa/72228xaNAghIWFoUWLFrj77rvx0Ucfobi4uMLzqvM9kCQJH330Efr06YOgoCC0atUKI0aMwE8//WR3b6vVCo1Gg8cffxzp6ekYPnw4mjdvjqFDh1Y7tr/6/vvvodFo8PXXX8NkMtnie+KJJ1waW0VuPufo0aMYPnw4QkNDER4ejjlz5sBisZQ7pzrPSmxsLLp37263bfXq1dBoNIiNjbXb/s4770Cj0dh+tqpyq+e9jKPvf2U/W9X9uXDmfSciIiKi+oul3Q6SyWR2Xz/yyCPYtGmT7WtRFPHee+/h2WeftTvu999/x/Tp03Hp0iU8/vjjtu1PPvkk/vOf/9i+/vXXXzFq1CjEx8dXK5533nkHL7zwgt22ffv2Yd++fejZs2e1rlFX8SqVSowbNw7vv/8+MjIyEBkZabf/iy++QKtWrdC/f/9Kr/HDDz/ggQcesJuv/uuvv+LXX39F8+bNMWzYsCrjqMhzzz2H9957z/Z1UVERvvvuO+zZswfr1q3DoEGD7I4/e/YsBg0ahGvXrgEofd89ObaqnD9/HoMGDcLly5cBlE5p+PDDDyGKIv71r3/Zjqvus9K/f3+89dZbOH/+PFq2bAkA2LNnD+RyOX799Vdcu3YNTZs2tW0PDAyEXq+vMs6qnvfw8PAqr1Fd1X2ttfW+ExEREZEHk8jm6NGjklqtlmbNmmXblpOTI3300UeSVquV/Pz8pAsXLkiSJEmPPfaYpFarpbZt20qbN2+WcnNzJUmSpFOnTkn+/v5Shw4dpE2bNkmXLl2S8vPzpdTUVCkqKkoKDg6Wrly5IkmSJB06dEhSq9VSu3btpC1btkjXrl2TMjMzpZkzZ0pqtVpSq9XSiRMnbLGEhIRIU6dOtX2dkZEhabVaKSgoSFq2bJmUlZUl5eXlSfv375emTJkinT592nasVquVpk+fXu4112a8Ffn1118ltVot/eMf/7DbbjQaJbVaLS1YsMBu+19f84svviip1Wrp3XfflXJyciSz2SwdOnRIeuaZZ6Tt27dXet6tvgeHDx+W1Gq11KpVK+mLL76Qrly5Ip05c0aaN2+epFarpS5dukhWq1WSJEkqKSmxvdaxY8dKRqNRKikpcSi2yowZM0Zq2bJlrcRWmZvPefDBB6UjR45IBQUF0u7du6U2bdpIQUFBUlFRkSRJjj0rqampklqtltauXWu7V/v27aWHH35YUqvV0tdffy1JkiRZrVapZcuW0uTJk6v8/jjyvDvy/ld0vCOvtabvOxERERHVP0ykb1KWSFf2Z86cObZjyxLpdevW2V1j2bJlklqtllJTU8tdf/v27ZJarZY2b94sSZIkvfrqq5JarZaSk5PtjhMEQerdu3eVifSbb74pqdVqacWKFVW+tsqSiNqMtzI9e/aU9Hq9JIqibVtCQoKkVqulI0eO2B3719e8ZMkSyc/PT8rJybnlPRxJpMpe17///e9y+0aPHi2p1Wrpl19+kSTpRuIZGRkpFRQU2B1b3dgqU1Ei7arYKlN2TnR0tFRcXGy3b+HChZJarZYMBoMkSY49K2azWQoICJD+9re/SZJ042dr165dUkxMjDR37lxJkiTpwIEDklqtlj766KMqY3Xkea9pIu3Ia63p+05ERERE9Q9Lu6vg6+uLDh06YMqUKZg1a1a5/QMGDLD7+tixYwBwy3LOc+fOAQCysrIAoNycUblcjgEDBsBgMNwytrL52LeaA1uVuoy3zKRJk/DSSy/hxx9/xF133QVRFLFu3Trcdttt6Ny58y3PnTlzJnbs2IEePXogPj4e3bt3R79+/apVFlyZU6dOAQDuvffecvsGDhyI7du34+TJk+jRo4dte69eveDr61tvYqtKr169oFQq7baVlUkXFBQAcOxZadKkCXr16oU9e/YAKC3f9vHxwR133IG7777bbjuAW5bzl3HF815djrzW2njfiYiIiMizsdlYBW7u2p2dnY0ff/wRjzzySIWdlAMCAuy+lm5qeFSZkpKSKo+tznWqc4wrruGqeMtMmDABCoUCX3zxBQBg9+7dOH/+PCZOnFjluX5+fti2bRu2bNmCqKgoHDx4EMOGDUNsbCxOnz5d7Rgqiv2v899v5a/vu6fHVhUfH59y28ruWRaDI88KUJocnzlzBpmZmdizZw969+4NHx8f9O/fH0ajERcvXsSePXvQsmVLdOjQocpru+J5ry5HXmttvO9ERERE5NmYSLtY+/btAZQmh2XJ+F//PPnkkwCAdu3aAUC5rteiKNpG6qpzr2+++abKY+VyOQRBcGu8ZVq0aIHY2Fj897//RVFREdauXQulUomxY8dW+xpdu3bF7Nmz8fHHH+Pw4cM4deoU5s2bd8tzKvsetG3bFgCwc+fOcvu+++47ADdee23FVhlXx1YTjjwrwI1R5l27duF///ufrXqjf//+kMlk+O6777B3795qjUbffP/qPO8Vqez9v9W9qvtaAde+70RERETk2ZhIu9jYsWOhVCoxdepUbNy4ERcvXkRxcTGysrKwadMmDB482NYZeciQIQCAZ599Fl9//TXy8vJw8uRJPPLII9Uqkx49ejQUCgUSExPx9ttv48yZMygoKMCBAwcwbdo0nDlzxnasv78/fv/9d/z5559ui/dmkyZNgslkQlJSEr766isMGjQIQUFBVZ731FNP4bXXXkNaWhry8/ORl5eHlJQU5Obm2sqgK1PZ96CsfHfBggVYv349cnNzce7cObz00kv49ttvERYWhq5du9ZqbJVxVWyu4MizApSWi6vVarz77ru4cuWKLWEODg6GTqfD0qVLYTabq51IO/K8V6Sy97+mr7U23nciIiIi8nBumZntoSrq2l2ZsmZjFXVF/uijjySNRlNp07JLly7Zjn388cfL7W/atKn0wAMPVNlsTJIk6V//+lel98nKyrIdN3HiRLt9AQEBtR7vrZjNZqlFixZSixYtJLVaLX355ZcVHvfX11x2n4r+LF269Jbfq1t9D+bOnVvhNZs2bSpt27bNdlxZc67HHnusXKzVja0yFTUbc1VslbnVOZ9++qmkVqulffv22bY58qxIkiSNGDFCUqvVUvPmze1+Vp599lnbOTd3265KdZ93R9//io6v7mut6ftORERERPUPR6RrwcMPP4ytW7fi/vvvR0BAAFQqFSIiIjB27Fh8++23diOvy5cvx/z589G6dWuoVCp069YNGzduRPfu3at1ryeffBLr169Hv3794OfnBz8/P9xxxx1YtWoVwsLCbMe98cYbiI+Ph1ardWu8ZZo0aYLRo0fDZDIhICCg2utm/+tf/8L8+fPRtWtXqNVqBAQEoE+fPvj444/x9NNP3/LcW30P3njjDSxduhRRUVFQqVTw8/PDPffcg61bt5Zbp7k2Yqsq7prG5iqOPCvAjWZ8ffv2hZeXV7ntERERaNOmTbXvX93nvSK3ev9r8lpr630nIiIiIs8lk6Q67OBDREREREREVM9xRJqIiIiIiIjIAUykiYiIiIiIiBzARJqIiIiIiIjIAUykiYiIiIiIiBzARJqIiIiIiIjIAUykiYiIiIiIiBzARJqIiIiIiIjIAUykAQiCgMs5ORAEwd2hEBERERERkYdjIg0g9+pVzJo1A7lXr7o7lEZBEKz48/wpCILV3aGQm/FZ8Cx8PzwH3wu6GZ8Hz8L3w3PwvaCb1fXzwESaiIiIiIiIyAFMpImIiIiIiIgcwESaiIiIiIiIyAFMpImIiIiIiIgcwESaiIiIiIiIyAFMpImIiIiIiIgcwESaiIiIiIiIyAFMpImIiIiIiIgcwESaiIiIiIiIyAFMpImIiIiIiIgcwESaiIiIiIiIyAFMpImIiIiIiIgcwESaiIiIiIiIyAFe7g6A6tbbb78Ns9lcbvuwYcPQtm1brFixAhMnTkR4eHidxrVp0yZLine truncated
|
||||||
|
"text/plain": [
|
||||||
|
"<Figure size 990x660 with 1 Axes>"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"metadata": {},
|
||||||
|
"output_type": "display_data"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"data": {
|
||||||
|
"text/html": [
|
||||||
|
"<div>\n",
|
||||||
|
"<style scoped>\n",
|
||||||
|
" .dataframe tbody tr th:only-of-type {\n",
|
||||||
|
" vertical-align: middle;\n",
|
||||||
|
" }\n",
|
||||||
|
"\n",
|
||||||
|
" .dataframe tbody tr th {\n",
|
||||||
|
" vertical-align: top;\n",
|
||||||
|
" }\n",
|
||||||
|
"\n",
|
||||||
|
" .dataframe thead th {\n",
|
||||||
|
" text-align: right;\n",
|
||||||
|
" }\n",
|
||||||
|
"</style>\n",
|
||||||
|
"<table border=\"1\" class=\"dataframe\">\n",
|
||||||
|
" <thead>\n",
|
||||||
|
" <tr style=\"text-align: right;\">\n",
|
||||||
|
" <th></th>\n",
|
||||||
|
" <th>Population</th>\n",
|
||||||
|
" <th>Predicted visitors</th>\n",
|
||||||
|
" </tr>\n",
|
||||||
|
" </thead>\n",
|
||||||
|
" <tbody>\n",
|
||||||
|
" <tr>\n",
|
||||||
|
" <th>Reykjavík</th>\n",
|
||||||
|
" <td>140000.0</td>\n",
|
||||||
|
" <td>1547807.0</td>\n",
|
||||||
|
" </tr>\n",
|
||||||
|
" <tr>\n",
|
||||||
|
" <th>Lyon</th>\n",
|
||||||
|
" <td>520000.0</td>\n",
|
||||||
|
" <td>1831517.0</td>\n",
|
||||||
|
" </tr>\n",
|
||||||
|
" <tr>\n",
|
||||||
|
" <th>Montréal</th>\n",
|
||||||
|
" <td>1780000.0</td>\n",
|
||||||
|
" <td>2144653.0</td>\n",
|
||||||
|
" </tr>\n",
|
||||||
|
" <tr>\n",
|
||||||
|
" <th>Mexico City</th>\n",
|
||||||
|
" <td>9200000.0</td>\n",
|
||||||
|
" <td>2647621.0</td>\n",
|
||||||
|
" </tr>\n",
|
||||||
|
" <tr>\n",
|
||||||
|
" <th>Tokyo</th>\n",
|
||||||
|
" <td>14000000.0</td>\n",
|
||||||
|
" <td>2794109.0</td>\n",
|
||||||
|
" </tr>\n",
|
||||||
|
" </tbody>\n",
|
||||||
|
"</table>\n",
|
||||||
|
"</div>"
|
||||||
|
],
|
||||||
|
"text/plain": [
|
||||||
|
" Population Predicted visitors\n",
|
||||||
|
"Reykjavík 140000.0 1547807.0\n",
|
||||||
|
"Lyon 520000.0 1831517.0\n",
|
||||||
|
"Montréal 1780000.0 2144653.0\n",
|
||||||
|
"Mexico City 9200000.0 2647621.0\n",
|
||||||
|
"Tokyo 14000000.0 2794109.0"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"execution_count": 8,
|
||||||
|
"metadata": {},
|
||||||
|
"output_type": "execute_result"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"source": [
|
||||||
|
"cities = {\"Reykjavík\": 140_000, \"Lyon\": 520_000, \"Montréal\": 1_780_000,\n",
|
||||||
|
" \"Mexico City\": 9_200_000, \"Tokyo\": 14_000_000}\n",
|
||||||
|
"\n",
|
||||||
|
"single = get(\"/predict\", population=cities[\"Montréal\"])\n",
|
||||||
|
"print(f\"Montréal (single call): {human(single['predicted_visitors'])} visitors\")\n",
|
||||||
|
"\n",
|
||||||
|
"pred = pd.DataFrame(post(\"/predict\", {\"populations\": list(cities.values())}), index=list(cities))\n",
|
||||||
|
"\n",
|
||||||
|
"fig, ax = plt.subplots(figsize=(9, 6))\n",
|
||||||
|
"ax.scatter(points.Population, points.Visitors, s=30, color=GRID, edgecolor=MUTED,\n",
|
||||||
|
" linewidth=0.5, label=\"Fitted museums\", zorder=2)\n",
|
||||||
|
"ax.plot(curve.population, curve.predicted_visitors, color=ORANGE, lw=2, label=\"Fit\", zorder=3)\n",
|
||||||
|
"ax.scatter(pred.population, pred.predicted_visitors, s=70, color=BLUE,\n",
|
||||||
|
" edgecolor=\"white\", linewidth=2, label=\"Predicted city\", zorder=4)\n",
|
||||||
|
"for city, r in pred.iterrows():\n",
|
||||||
|
" ax.annotate(f\"{city}: {human(r.predicted_visitors)}\", (r.population, r.predicted_visitors),\n",
|
||||||
|
" xytext=(8, -12), textcoords=\"offset points\", fontsize=9, color=INK)\n",
|
||||||
|
"if is_log:\n",
|
||||||
|
" ax.set_xscale(\"log\")\n",
|
||||||
|
" ax.set_yscale(\"log\")\n",
|
||||||
|
"human_axes(ax, is_log)\n",
|
||||||
|
"ax.set_xlabel(\"City population\")\n",
|
||||||
|
"ax.set_ylabel(\"Predicted annual visitors per museum\")\n",
|
||||||
|
"ax.set_title(\"Predicted visitors for new cities\", loc=\"left\", fontsize=12)\n",
|
||||||
|
"ax.legend(loc=\"upper left\")\n",
|
||||||
|
"plt.tight_layout()\n",
|
||||||
|
"plt.show()\n",
|
||||||
|
"\n",
|
||||||
|
"pred.rename(columns={\"population\": \"Population\", \"predicted_visitors\": \"Predicted visitors\"}).round(0)"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"metadata": {
|
||||||
|
"kernelspec": {
|
||||||
|
"display_name": "Python 3 (ipykernel)",
|
||||||
|
"language": "python",
|
||||||
|
"name": "python3"
|
||||||
|
},
|
||||||
|
"language_info": {
|
||||||
|
"codemirror_mode": {
|
||||||
|
"name": "ipython",
|
||||||
|
"version": 3
|
||||||
|
},
|
||||||
|
"file_extension": ".py",
|
||||||
|
"mimetype": "text/x-python",
|
||||||
|
"name": "python",
|
||||||
|
"nbconvert_exporter": "python",
|
||||||
|
"pygments_lexer": "ipython3",
|
||||||
|
"version": "3.12.15"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"nbformat": 4,
|
||||||
|
"nbformat_minor": 5
|
||||||
|
}
|
||||||
@@ -1,313 +0,0 @@
|
|||||||
{
|
|
||||||
"cells": [
|
|
||||||
{
|
|
||||||
"cell_type": "markdown",
|
|
||||||
"id": "0af7e8f5",
|
|
||||||
"metadata": {},
|
|
||||||
"source": [
|
|
||||||
"# Museum visitors vs. city population\n",
|
|
||||||
"\n",
|
|
||||||
"This notebook talks to the `museum_api` FastAPI server over HTTP: it starts the server (or uses one you already run), sends it the data, and plots the regression it returns.\n",
|
|
||||||
"\n",
|
|
||||||
"**Setup:** put `../../api/app/museum_api.py` next to this notebook, then `pip install scikit-learn pandas fastapi uvicorn requests matplotlib`."
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "code",
|
|
||||||
"id": "4c50f329",
|
|
||||||
"metadata": {
|
|
||||||
"ExecuteTime": {
|
|
||||||
"end_time": "2026-10-07T21:11:58.303386100Z",
|
|
||||||
"start_time": "2026-10-07T21:11:57.570566Z"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"source": [
|
|
||||||
"import threading, time\n",
|
|
||||||
"import requests\n",
|
|
||||||
"import numpy as np\n",
|
|
||||||
"import pandas as pd\n",
|
|
||||||
"import matplotlib.pyplot as plt\n",
|
|
||||||
"from matplotlib.ticker import FuncFormatter\n",
|
|
||||||
"\n",
|
|
||||||
"API = \"http://127.0.0.1:8000\" # change if your server runs elsewhere"
|
|
||||||
],
|
|
||||||
"outputs": [
|
|
||||||
{
|
|
||||||
"ename": "ModuleNotFoundError",
|
|
||||||
"evalue": "No module named 'matplotlib'",
|
|
||||||
"output_type": "error",
|
|
||||||
"traceback": [
|
|
||||||
"\u001B[31m---------------------------------------------------------------------------\u001B[39m",
|
|
||||||
"\u001B[31mModuleNotFoundError\u001B[39m Traceback (most recent call last)",
|
|
||||||
"\u001B[36mCell\u001B[39m\u001B[36m \u001B[39m\u001B[32mIn[1]\u001B[39m\u001B[32m, line 5\u001B[39m\n\u001B[32m 1\u001B[39m \u001B[38;5;28;01mimport\u001B[39;00m threading, time\n\u001B[32m 2\u001B[39m \u001B[38;5;28;01mimport\u001B[39;00m requests\n\u001B[32m 3\u001B[39m \u001B[38;5;28;01mimport\u001B[39;00m numpy \u001B[38;5;28;01mas\u001B[39;00m np\n\u001B[32m 4\u001B[39m \u001B[38;5;28;01mimport\u001B[39;00m pandas \u001B[38;5;28;01mas\u001B[39;00m pd\n\u001B[32m----> \u001B[39m\u001B[32m5\u001B[39m \u001B[38;5;28;01mimport\u001B[39;00m matplotlib.pyplot \u001B[38;5;28;01mas\u001B[39;00m plt\n\u001B[32m 6\u001B[39m \u001B[38;5;28;01mfrom\u001B[39;00m matplotlib.ticker \u001B[38;5;28;01mimport\u001B[39;00m FuncFormatter\n\u001B[32m 7\u001B[39m \n\u001B[32m 8\u001B[39m API = \u001B[33m\"http://127.0.0.1:8000\"\u001B[39m \u001B[38;5;66;03m# change if your server runs elsewhere\u001B[39;00m\n",
|
|
||||||
"\u001B[31mModuleNotFoundError\u001B[39m: No module named 'matplotlib'"
|
|
||||||
]
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"execution_count": 1
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "markdown",
|
|
||||||
"id": "69524967",
|
|
||||||
"metadata": {},
|
|
||||||
"source": [
|
|
||||||
"## 1. Start the server\n",
|
|
||||||
"\n",
|
|
||||||
"If a server is already answering at `API` (e.g. you started it with `uvicorn museum_api:app` in a terminal), this cell just uses it. Otherwise it launches one in a background thread inside this kernel. Restart the kernel to stop it."
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "code",
|
|
||||||
"execution_count": 2,
|
|
||||||
"id": "b9d41502",
|
|
||||||
"metadata": {},
|
|
||||||
"outputs": [
|
|
||||||
{
|
|
||||||
"ename": "NameError",
|
|
||||||
"evalue": "name 'API' is not defined",
|
|
||||||
"output_type": "error",
|
|
||||||
"traceback": [
|
|
||||||
"\u001B[31m---------------------------------------------------------------------------\u001B[39m",
|
|
||||||
"\u001B[31mNameError\u001B[39m Traceback (most recent call last)",
|
|
||||||
"\u001B[36mCell\u001B[39m\u001B[36m \u001B[39m\u001B[32mIn[2]\u001B[39m\u001B[32m, line 7\u001B[39m\n\u001B[32m 3\u001B[39m \u001B[38;5;28;01mreturn\u001B[39;00m requests.get(f\"{API}/health\", timeout=\u001B[32m1\u001B[39m).ok\n\u001B[32m 4\u001B[39m \u001B[38;5;28;01mexcept\u001B[39;00m requests.ConnectionError:\n\u001B[32m 5\u001B[39m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[38;5;28;01mFalse\u001B[39;00m\n\u001B[32m 6\u001B[39m \n\u001B[32m----> \u001B[39m\u001B[32m7\u001B[39m \u001B[38;5;28;01mif\u001B[39;00m \u001B[38;5;28;01mnot\u001B[39;00m server_up():\n\u001B[32m 8\u001B[39m \u001B[38;5;28;01mimport\u001B[39;00m uvicorn\n\u001B[32m 9\u001B[39m \u001B[38;5;28;01mfrom\u001B[39;00m museum_api \u001B[38;5;28;01mimport\u001B[39;00m create_app\n\u001B[32m 10\u001B[39m server = uvicorn.Server(uvicorn.Config(create_app(), host=\u001B[33m\"127.0.0.1\"\u001B[39m, port=\u001B[32m8000\u001B[39m, log_level=\u001B[33m\"warning\"\u001B[39m))\n",
|
|
||||||
"\u001B[36mCell\u001B[39m\u001B[36m \u001B[39m\u001B[32mIn[2]\u001B[39m\u001B[32m, line 4\u001B[39m, in \u001B[36mserver_up\u001B[39m\u001B[34m()\u001B[39m\n\u001B[32m 1\u001B[39m \u001B[38;5;28;01mdef\u001B[39;00m server_up():\n\u001B[32m 2\u001B[39m \u001B[38;5;28;01mtry\u001B[39;00m:\n\u001B[32m 3\u001B[39m \u001B[38;5;28;01mreturn\u001B[39;00m requests.get(f\"{API}/health\", timeout=\u001B[32m1\u001B[39m).ok\n\u001B[32m----> \u001B[39m\u001B[32m4\u001B[39m \u001B[38;5;28;01mexcept\u001B[39;00m requests.ConnectionError:\n\u001B[32m 5\u001B[39m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[38;5;28;01mFalse\u001B[39;00m\n",
|
|
||||||
"\u001B[31mNameError\u001B[39m: name 'API' is not defined"
|
|
||||||
]
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"source": [
|
|
||||||
"def server_up():\n",
|
|
||||||
" try:\n",
|
|
||||||
" return requests.get(f\"{API}/health\", timeout=1).ok\n",
|
|
||||||
" except requests.ConnectionError:\n",
|
|
||||||
" return False\n",
|
|
||||||
"\n",
|
|
||||||
"if not server_up():\n",
|
|
||||||
" import uvicorn\n",
|
|
||||||
" from api.app.museum_api import create_app\n",
|
|
||||||
" server = uvicorn.Server(uvicorn.Config(create_app(), host=\"127.0.0.1\", port=8000, log_level=\"warning\"))\n",
|
|
||||||
" threading.Thread(target=server.run, daemon=True).start()\n",
|
|
||||||
" for _ in range(50):\n",
|
|
||||||
" if server_up(): break\n",
|
|
||||||
" time.sleep(0.1)\n",
|
|
||||||
"\n",
|
|
||||||
"requests.get(f\"{API}/health\").json()"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "markdown",
|
|
||||||
"id": "742501f7",
|
|
||||||
"metadata": {},
|
|
||||||
"source": [
|
|
||||||
"## 2. Load your data\n",
|
|
||||||
"\n",
|
|
||||||
"Replace this with however you build your DataFrame. It needs the columns `Name, Visitors, City, Country, Population`."
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "code",
|
|
||||||
"execution_count": null,
|
|
||||||
"id": "cb58b0f5",
|
|
||||||
"metadata": {},
|
|
||||||
"outputs": [],
|
|
||||||
"source": [
|
|
||||||
"df = pd.read_csv(\"museums.csv\")\n",
|
|
||||||
"df.head()"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "markdown",
|
|
||||||
"id": "362d6127",
|
|
||||||
"metadata": {},
|
|
||||||
"source": [
|
|
||||||
"## 3. Train the model through the API\n",
|
|
||||||
"\n",
|
|
||||||
"`scale=\"log\"` fits log(visitors) ~ log(population). Set `aggregate_by_city=True` to count each city once instead of once per museum."
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "code",
|
|
||||||
"execution_count": null,
|
|
||||||
"id": "ab229859",
|
|
||||||
"metadata": {},
|
|
||||||
"outputs": [],
|
|
||||||
"source": [
|
|
||||||
"def train(df, scale=\"log\", aggregate_by_city=False):\n",
|
|
||||||
" records = df[[\"Name\", \"Visitors\", \"City\", \"Country\", \"Population\"]].dropna().to_dict(\"records\")\n",
|
|
||||||
" r = requests.post(f\"{API}/train\", json={\"records\": records, \"scale\": scale,\n",
|
|
||||||
" \"aggregate_by_city\": aggregate_by_city})\n",
|
|
||||||
" r.raise_for_status()\n",
|
|
||||||
" return r.json()\n",
|
|
||||||
"\n",
|
|
||||||
"summary = train(df, scale=\"log\")\n",
|
|
||||||
"pd.Series(summary).to_frame(\"value\")"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "markdown",
|
|
||||||
"id": "f655bbcf",
|
|
||||||
"metadata": {},
|
|
||||||
"source": [
|
|
||||||
"## 4. Plot the regression"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "code",
|
|
||||||
"execution_count": null,
|
|
||||||
"id": "1a23269f",
|
|
||||||
"metadata": {},
|
|
||||||
"outputs": [],
|
|
||||||
"source": [
|
|
||||||
"BLUE, ORANGE, INK, MUTED = \"#2a78d6\", \"#eb6834\", \"#0b0b0b\", \"#52514e\"\n",
|
|
||||||
"plt.rcParams.update({\"axes.spines.top\": False, \"axes.spines.right\": False,\n",
|
|
||||||
" \"axes.edgecolor\": MUTED, \"axes.labelcolor\": INK,\n",
|
|
||||||
" \"xtick.color\": MUTED, \"ytick.color\": MUTED,\n",
|
|
||||||
" \"axes.grid\": True, \"axes.axisbelow\": True, \"grid.color\": \"#e6e5e0\", \"grid.linewidth\": 0.8,\n",
|
|
||||||
" \"figure.dpi\": 110})\n",
|
|
||||||
"\n",
|
|
||||||
"def human(x, _=None):\n",
|
|
||||||
" for div, suf in ((1e9, \"B\"), (1e6, \"M\"), (1e3, \"K\")):\n",
|
|
||||||
" if abs(x) >= div: return f\"{x/div:g}{suf}\"\n",
|
|
||||||
" return f\"{x:g}\"\n",
|
|
||||||
"\n",
|
|
||||||
"points = pd.DataFrame(requests.get(f\"{API}/data\").json())\n",
|
|
||||||
"\n",
|
|
||||||
"# Fitted curve: ask the API to predict across the population range\n",
|
|
||||||
"grid = np.geomspace(points.Population.min(), points.Population.max(), 100)\n",
|
|
||||||
"curve = pd.DataFrame(requests.post(f\"{API}/predict\", json={\"populations\": grid.tolist()}).json())\n",
|
|
||||||
"\n",
|
|
||||||
"fig, ax = plt.subplots(figsize=(9, 6))\n",
|
|
||||||
"ax.scatter(points.Population, points.Visitors, s=40, color=BLUE, alpha=0.75,\n",
|
|
||||||
" edgecolor=\"white\", linewidth=1, label=\"Museums\", zorder=3)\n",
|
|
||||||
"ax.plot(curve.population, curve.predicted_visitors, color=ORANGE, lw=2,\n",
|
|
||||||
" label=f\"Fit: {summary['equation']}\", zorder=4)\n",
|
|
||||||
"\n",
|
|
||||||
"# Label the 5 biggest outliers (furthest from the line on the log scale)\n",
|
|
||||||
"points[\"logres\"] = np.log10(points.Visitors / points.Predicted)\n",
|
|
||||||
"for _, r in points.reindex(points.logres.abs().sort_values(ascending=False).index).head(5).iterrows():\n",
|
|
||||||
" ax.annotate(r.Name, (r.Population, r.Visitors), xytext=(6, 4),\n",
|
|
||||||
" textcoords=\"offset points\", fontsize=8, color=MUTED)\n",
|
|
||||||
"\n",
|
|
||||||
"if summary[\"scale\"] == \"log\":\n",
|
|
||||||
" ax.set_xscale(\"log\"); ax.set_yscale(\"log\")\n",
|
|
||||||
"ax.xaxis.set_major_formatter(FuncFormatter(human))\n",
|
|
||||||
"ax.yaxis.set_major_formatter(FuncFormatter(human))\n",
|
|
||||||
"ax.set_xlabel(\"City population\"); ax.set_ylabel(\"Annual visitors\")\n",
|
|
||||||
"cv = summary[\"cv_r2_5fold\"]\n",
|
|
||||||
"ax.set_title(f\"Visitors vs. city population R² = {summary['r2']:.2f}\"\n",
|
|
||||||
" + (f\" (cross-validated {cv:.2f})\" if cv is not None else \"\"),\n",
|
|
||||||
" loc=\"left\", fontsize=12, color=INK)\n",
|
|
||||||
"ax.legend(frameon=False, loc=\"upper left\")\n",
|
|
||||||
"plt.tight_layout(); plt.show()\n",
|
|
||||||
"\n",
|
|
||||||
"print(summary[\"interpretation\"])"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "markdown",
|
|
||||||
"id": "a4eafc4a",
|
|
||||||
"metadata": {},
|
|
||||||
"source": [
|
|
||||||
"## 5. Which museums beat (or miss) their city's size?\n",
|
|
||||||
"\n",
|
|
||||||
"Ratio = actual ÷ predicted visitors. Above 1× means the museum draws more than its city's size alone would suggest."
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "code",
|
|
||||||
"execution_count": null,
|
|
||||||
"id": "9a823747",
|
|
||||||
"metadata": {},
|
|
||||||
"outputs": [],
|
|
||||||
"source": [
|
|
||||||
"N = 8\n",
|
|
||||||
"over = pd.DataFrame(requests.get(f\"{API}/residuals\", params={\"top\": N, \"order\": \"over\"}).json())\n",
|
|
||||||
"under = pd.DataFrame(requests.get(f\"{API}/residuals\", params={\"top\": N, \"order\": \"under\"}).json())\n",
|
|
||||||
"res = pd.concat([over, under.iloc[::-1]]).drop_duplicates(\"Name\").iloc[::-1]\n",
|
|
||||||
"\n",
|
|
||||||
"fig, ax = plt.subplots(figsize=(9, 0.35 * len(res) + 1))\n",
|
|
||||||
"colors = [BLUE if r >= 1 else ORANGE for r in res.Ratio]\n",
|
|
||||||
"ax.barh(res.Name + \" (\" + res.City + \")\", np.log10(res.Ratio), color=colors, height=0.7)\n",
|
|
||||||
"ax.axvline(0, color=MUTED, lw=1)\n",
|
|
||||||
"ticks = np.array([0.1, 0.2, 0.5, 1, 2, 5, 10])\n",
|
|
||||||
"ticks = ticks[(np.log10(ticks) >= np.log10(res.Ratio).min() - 0.1) &\n",
|
|
||||||
" (np.log10(ticks) <= np.log10(res.Ratio).max() + 0.1)]\n",
|
|
||||||
"ax.set_xticks(np.log10(ticks), [f\"{t:g}×\" for t in ticks])\n",
|
|
||||||
"ax.set_xlabel(\"Actual ÷ predicted visitors (log scale)\")\n",
|
|
||||||
"ax.set_title(\"Over-performers (blue) and under-performers (orange)\", loc=\"left\", fontsize=12, color=INK)\n",
|
|
||||||
"ax.grid(axis=\"y\", visible=False)\n",
|
|
||||||
"plt.tight_layout(); plt.show()\n",
|
|
||||||
"\n",
|
|
||||||
"res[[\"Name\", \"City\", \"Country\", \"Population\", \"Visitors\", \"Predicted\", \"Ratio\"]].round(2)"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "markdown",
|
|
||||||
"id": "ce7a12d7",
|
|
||||||
"metadata": {},
|
|
||||||
"source": [
|
|
||||||
"## 6. Compare model variants\n",
|
|
||||||
"\n",
|
|
||||||
"Retrains the server with each setting and collects the fit statistics. The last setting tried stays loaded, so the final call re-trains the default."
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "code",
|
|
||||||
"execution_count": null,
|
|
||||||
"id": "9adca34e",
|
|
||||||
"metadata": {},
|
|
||||||
"outputs": [],
|
|
||||||
"source": [
|
|
||||||
"rows = []\n",
|
|
||||||
"for scale in (\"linear\", \"log\"):\n",
|
|
||||||
" for agg in (False, True):\n",
|
|
||||||
" s = train(df, scale=scale, aggregate_by_city=agg)\n",
|
|
||||||
" rows.append({k: s[k] for k in (\"scale\", \"aggregate_by_city\", \"n_samples\",\n",
|
|
||||||
" \"r2\", \"cv_r2_5fold\", \"pearson_r\", \"spearman_rho\")})\n",
|
|
||||||
"train(df, scale=\"log\") # restore the default model\n",
|
|
||||||
"pd.DataFrame(rows).round(3)"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "markdown",
|
|
||||||
"id": "1c0bcd2f",
|
|
||||||
"metadata": {},
|
|
||||||
"source": [
|
|
||||||
"## 7. Predict for any city"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "code",
|
|
||||||
"execution_count": null,
|
|
||||||
"id": "6ff399ee",
|
|
||||||
"metadata": {},
|
|
||||||
"outputs": [],
|
|
||||||
"source": [
|
|
||||||
"cities = {\"Montréal\": 1_780_000, \"Lyon\": 520_000, \"Tokyo\": 14_000_000}\n",
|
|
||||||
"pred = requests.post(f\"{API}/predict\", json={\"populations\": list(cities.values())}).json()\n",
|
|
||||||
"pd.DataFrame(pred, index=cities.keys()).round(0)"
|
|
||||||
]
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"metadata": {
|
|
||||||
"kernelspec": {
|
|
||||||
"display_name": "Python 3 (ipykernel)",
|
|
||||||
"language": "python",
|
|
||||||
"name": "python3"
|
|
||||||
},
|
|
||||||
"language_info": {
|
|
||||||
"codemirror_mode": {
|
|
||||||
"name": "ipython",
|
|
||||||
"version": 3
|
|
||||||
},
|
|
||||||
"file_extension": ".py",
|
|
||||||
"mimetype": "text/x-python",
|
|
||||||
"name": "python",
|
|
||||||
"nbconvert_exporter": "python",
|
|
||||||
"pygments_lexer": "ipython3",
|
|
||||||
"version": "3.12.10"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"nbformat": 4,
|
|
||||||
"nbformat_minor": 5
|
|
||||||
}
|
|
||||||
Reference in new issue
Block a user