{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_block_id": "7f45dc2cdc9a4104ac8ab395e3332a98",
        "cloud_content_sha256": "sha256:8abaa74371c9b1023b30eefc8b672a3b3ceab6b94be7302cc595b473b1d96aea"
      },
      "source": "# LAB08｜生成器与判别器怎样在反馈中改变\n\n",
      "id": "lab08-000"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_block_id": "4e42058b87f148acaacac625c9f2bff5",
        "cloud_content_sha256": "sha256:982a860dd33d71539403d795fda92128cdc34cdc4f46018320ad39b02ad2cb1f"
      },
      "source": "## 先看实验回答了什么\n\n同一组随机起点，经过训练可以被映射到八个数据簇附近；但让判别器每轮多学几次，并不保证生成器覆盖得更完整。本次固定预算里，D:G=3:1 的点更集中，两个模式却明显不足。这个结果正好要求我们分开看**单个样本像不像**和**一批样本有没有覆盖不同可能性**。\n\n本册是真实训练的二维小 MLP GAN。它不是预写坐标动画，也不是 pix2pix、CycleGAN 或真实图像 GAN 的复现。全部图和数值来自同一次规定预算的 NumPy 运行；无需下载权重或数据。\n\n![同一组2048个潜变量在训练前、1000步与4000步的生成点。灰点是独立真实样本，橙点是生成样本，金色叉是八个模式中心，蓝色背景是当时判别器给出的真实概率。上排D:G=1:1，下排3:1。](https://codingai-lec04.pages.dev/assets/experiments/learned/lab08-training.png)\n\n先横着读上排：初始化的橙点聚在中央，后来沿环移动，最后到达八个簇附近，但簇间仍有许多点。再读下排：末态六个簇更紧密，上、下两个模式几乎没有样本。最后才看蓝色背景：这是当前 D 的判断，并非真实数据密度，也不是一个固定不动的教师答案。\n\n",
      "id": "lab08-001"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_block_id": "5566371e0b1640c9bbcb726554122153",
        "cloud_content_sha256": "sha256:cb65ea577a69feae1f4976cdf39c45b33a1640349e42f6a3902aacc3efb6c850"
      },
      "source": "## 与 LAB09 共用的世界：八个可能区域\n\n真实数据先均匀选一个编号 k∈{0,…,7}，再从中心 μₖ 周围取二维高斯噪声。中心位于半径 2 的圆上，单坐标标准差为 0.12：\n\n\\[\n\\mu_k=2(\\cos(2\\pi k/8),\\sin(2\\pi k/8)),\\quad\nx=\\mu_k+0.12\\,\\epsilon,\\quad \\epsilon\\sim\\mathcal N(0,I).\n\\]\n\nG 的输入是二维标准正态 z；输出是二维坐标。G 没有得到“这个 z 应去第几个模式”的配对标签。D 得到真实坐标和 G 产生的坐标，并学习分辨二者。两方都有两个 64 单元的 tanh 隐藏层；同一套参数处理所有样本。\n\n覆盖读法提前固定：以每个中心为圆心，半径 0.36（单坐标标准差的三倍）作为邻域；2048 个生成点中，至少 1% 落入该邻域才计为覆盖一个模式。近模式比例是落入八个邻域的点占全部生成点的比例。这是二维圆邻域，不应套用一维“三倍标准差覆盖99.7%”的说法。两项都要看：全部点坍塌到一个中心时近模式比例可以接近100%，覆盖却只有1/8。\n\n",
      "id": "lab08-002"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_block_id": "02f96aaaa1f24c178aec518b2b676bec",
        "cloud_content_sha256": "sha256:b2e3077ac1dcef56450e79f185fab274b403ca8491bd3216e6b60d7f55764e06"
      },
      "source": "## 一次 D 更新：谁得到纠正\n\n令 l 为 D 输出的 logit，D(x)=sigmoid(l)。D 在一个 256 真样本＋256 生成样本的拼接批次上最小化二元交叉熵的平均值。真实标签为1，生成标签为0。对每个 logit 的梯度是 (D(x)−y)/批次总数。\n\n```python\nfake, _ = G.forward(z)      # 只保留数值，丢弃 G 的中间缓存\nlogits, cacheD = D.forward(concatenate([real, fake]))\n_, gradsD = D.backward(cacheD, (sigmoid(logits) - labels) / len(labels))\nD.update(gradsD, lr)\n# 没有 G.backward，也没有 G.update\n```\n\nNumPy 不会隐式构建自动求导图，所以“切断”是可检查的：D 步只有 fake 的数值流入 D，代码根本没有计算或施加 G 的参数梯度。PyTorch 中对生成结果使用 detach，承担的是相同的边界职责。\n\n独立的32＋32样本实际梯度探针里，第一真实点为 (0.0734, −2.3492)，D给出0.48681；它对logit的梯度为−0.008019，梯度下降会把真实分数推高。第一生成点为 (−0.2576, 0.04149)，D给出0.47562；梯度为＋0.007432，会把生成分数推低。本次D步使D参数的变化范数为0.01994，G参数变化**严格为0**。这些是运行记录，不是为公式手编的数值。\n\n",
      "id": "lab08-003"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_block_id": "3e06c15a86ac45cbada06240f2f34994",
        "cloud_content_sha256": "sha256:60b07093c59b826b3d2542c297835b6fb7469b602925132d85f25c0f9f993916"
      },
      "source": "## 一次 G 更新：固定 D 不等于绕过 D\n\nG 使用非饱和目标 −mean(log D(G(z)))。它希望生成结果被判为真，所以此时用于计算 G 损失的目标是1。目标改变不代表我们谎称生成点是真实数据：这是两方不同的优化目标。\n\n```python\nfake, cacheG = G.forward(z)\nlogits, cacheD = D.forward(fake)\nd_fake, unused_D_grads = D.backward(cacheD, (sigmoid(logits)-1)/len(z))\n_, gradsG = G.backward(cacheG, d_fake)\nG.update(gradsG, lr)\n# D 的导数参与链式法则，但没有 D.update\n```\n\n此刻 D 像一张暂时固定的地形：它告诉 G 输出坐标往哪个方向挪会改变评分。反向计算必须经过 D 对输入的导数，才能继续到 G 的参数；如果连这条导数也切断，G 就得不到当前评分如何依赖自己输出的信息。\n\n在紧接前一D步的实际探针中，第一生成点的 D 概率为0.47713，G损失对logit的梯度为−0.01634；传到二维生成坐标上的梯度为 (−0.005727, −0.001027)。再沿G反传后，G参数改变0.02008，D参数变化**严格为0**。这里的参数变化是一次Adam更新后的范数，不应误读成坐标立即移动了同样距离。\n\n**学生常问：既然 D 固定了，梯度怎么还能经过它？** 固定的是参数取值和是否更新；函数在当前输入处仍有导数。就像固定函数 f(x)=2x 不会让 df/dx 消失。这段解释补上了“交替优化”一词往往掩盖的机制。\n\n",
      "id": "lab08-004"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_block_id": "a458dff9c56444d59ef60330f2babde4",
        "cloud_content_sha256": "sha256:22787b8addfd58c02ad7dbff4f81eaf9f3bd54da5bb8d7b3c754beae8c0d6618"
      },
      "source": "## 一个设置的对照：更多反馈改变了什么\n\n两臂从相同 G、D 初始化出发；每个 G 迭代使用相同真实批次、同一组 zD 与 zG。3:1 对同一 D 批次重复三次优化，1:1只做一次。两臂各做4000次G更新，因此**G预算相同，D计算量不同**；这不是等总算力对照。学习率均为0.0003，Adam β₁=0.5、β₂=0.99。\n\n| 设置／时点 | 覆盖模式 | 近模式比例 | 最近中心距离中位数 |\n|---|---:|---:|---:|\n| 相同初始化 | 0/8 | 0.00% | 1.6639 |\n| 1:1，1000次G更新 | 7/8 | 21.68% | 0.6077 |\n| 1:1，4000次G更新 | 8/8 | 54.25% | 0.3248 |\n| 3:1，1000次G更新 | 8/8 | 47.02% | 0.3760 |\n| 3:1，4000次G更新 | 6/8 | 85.64% | 0.1623 |\n| 独立真实样本参照 | 8/8 | 99.17% | 0.1392 |\n\n3:1末态的八个邻域计数依次为279、290、10、282、259、285、16、333。第2和第6号模式未达到每模至少21个点的覆盖阈值；它们并非数学上完全不可能被生成，而是在本次样本与规定读法中明显欠覆盖。1:1覆盖更完整，却仍有约46%的点不在近模式邻域。\n\n这个固定种子的对照支持“本次更多D更新带来更密集但不完整的结果”，不支持“多训D必然导致坍塌”。我们没有挑选失败图，也没有尝试别的参数直到出现想要的故事。距离更近也不自动证明分布更准确：把每个模式都压成一个点，会丢掉模式内部的真实方差。\n\n![两臂的D损失与非饱和G损失曲线。每100次G更新记录一次当前批次损失；两方目标随对手变化，不能把曲线当作统一质量刻度。](https://codingai-lec04.pages.dev/assets/experiments/learned/lab08-losses.png)\n\nD损失和G损失回答“当前对手下这一步有多难”，而覆盖回答“最终样本去了哪里”。两方同时变化，损失不必单调下降；比较图像、覆盖和分布应在明确冻结的采样条件下进行。\n\n",
      "id": "lab08-005"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_block_id": "7f18e8aa91734df9af94a831ce23cc63",
        "cloud_content_sha256": "sha256:8590fc885967b6ff3b9fe3a8b4b758abdefa947db1241f54056bb59234871436"
      },
      "source": "## 自己运行与只改一个变量\n\n本册的实现已在独立 Python 进程实际运行；两臂合计约24.05秒，环境为 Python 3.12.14、NumPy 2.3.5、单线程BLAS。不同机器时间会变化。依赖为 NumPy、Matplotlib 与 threadpoolctl；运行后在 outputs 目录保存图、完整样本、判别场、权重和结果JSON。Notebook与 .py 包含相同完整实现，不依赖其他册的变量。\n\n默认直接运行全部单元即可重建基线。建议第一次只观察已有的1:1／3:1对照，不需要继续搜索设置。若要再做一次学习任务，可固定全部设置，仅把 G 的训练步数改为2000，并同步改最后的checkpoint。先写下预期，再读取近模式比例、每模式计数与真实图，解释是否出现了“更密集却遗漏模式”。不要只保留更好看的那一张。\n\n**自检。** 如果G步误用了fake.detach，会怎样？答案：D仍能算分数，但关于G参数的梯度路径被切断，生成器无法从此损失学习。若D步也更新G，两方优化边界就变了，不再是本册规定的交替训练算法。\n\n\n\n重算时，`run()` 用 threadpoolctl 将这一次实验的 BLAS 计算限为单线程，并在结果 JSON 保存实际线程信息。Notebook 内核可能已加载 NumPy，只在代码中设置环境变量不能可靠改变已启动的线程池。这个执行控制不改变数据、随机种子、训练预算或采样规则；运行时间仍按当前机器实测，不能将本地秒数当作云端承诺。\n\n\n**云端复核（2026-09-30）。** 本册在 Deepnote 的独立内核按自身步骤完整运行。两臂各完成4000次 G 更新；D:G=1:1 的近模式比例为54.25%、覆盖8/8，3:1为85.64%、覆盖6/8。D步只改变D，G步梯度经过D但只改变G的参数，实际参数差验证了这两条路径。上方原始保存图继续标为本地基线，两次测量分别记录；大图在插件的运行快照预览中可能因尺寸被省略，这不代替学生在同册查看自己的输出。\n",
      "id": "lab08-006"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_block_id": "76208f8f422848ed82d4e290bd26c2cb",
        "cloud_content_sha256": "sha256:8b999516287eacb783a84c2373663302050f92845757b5e737d5d0c2b9b04587"
      },
      "source": "## 从二维反馈回到图像方法\n\n这里看见的是学习反馈怎样塑造分布。图像翻译还要说明输入条件怎样进入G和D、配对监督从哪里来、循环或潜变量重建约束解决什么问题。小GAN不会替这些设计作证明。继续读第四章与CLS04，再回到LAB03的循环一致反例，可以区分“会学习逼真反馈”和“能保证语义对应”两个问题。\n\n来源：Goodfellow 等，[Generative Adversarial Nets](https://arxiv.org/abs/1406.2661)，重点看交替更新算法与非饱和生成器目标。本文的二维数据、预算和对照是课程实现，不是原论文结果。\n",
      "id": "lab08-007"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_block_id": "59584570310a49b08371d9bf917fc307",
        "cloud_content_sha256": "sha256:66d3835ab16346ac367aa99dd92d15e7589be388a89d44a33786ad40ae8e718f"
      },
      "source": "## 完整可运行实现\n\n下面代码与下载版 `.py` 同源。先读上面的已保存实测结果；从新内核按顺序运行以下代码，可重新训练并保存自己的图、权重、数组与指标。这里显示的既有图来自独立 Python 进程，不冒称已在当前 Deepnote 计算实例中运行。\n",
      "id": "lab08-008"
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_block_id": "f9b21400f34b426eba709efb5fd12600",
        "output_provenance": {
          "origin": "Deepnote executed snapshot",
          "run_id": "8802fa19-cf89-4ccd-965c-0685e830b443",
          "content_hash": "sha256:ee2ff65985392bd5e82c9d9ed92d22c5f2916638c251ef2322d9ce3bb170fcab"
        },
        "verified_cloud_execution": {
          "run_id": "8802fa19-cf89-4ccd-965c-0685e830b443",
          "content_hash": "sha256:ee2ff65985392bd5e82c9d9ed92d22c5f2916638c251ef2322d9ce3bb170fcab",
          "output_preview_omitted": false,
          "completed_at": "2026-09-30T02:24:39.432Z"
        },
        "cloud_content_sha256": "sha256:ee2ff65985392bd5e82c9d9ed92d22c5f2916638c251ef2322d9ce3bb170fcab"
      },
      "execution_count": null,
      "source": "# Dependencies: Python 3.10+, numpy, matplotlib, threadpoolctl. No downloads or hidden notebook state.\nimport os\nos.environ.setdefault('OPENBLAS_NUM_THREADS','1')\nos.environ.setdefault('OMP_NUM_THREADS','1')\nimport json, time, math, hashlib, platform\nfrom pathlib import Path\nimport numpy as np\nfrom threadpoolctl import threadpool_limits, threadpool_info\nimport matplotlib\nmatplotlib.use('Agg')\nimport matplotlib.pyplot as plt\n\nCENTERS = 2.0*np.stack([np.cos(np.arange(8)*2*np.pi/8),np.sin(np.arange(8)*2*np.pi/8)],axis=1)\nDATA_STD = .12\n\ndef draw_data(rng,n):\n    k=rng.integers(0,8,n)\n    return CENTERS[k]+DATA_STD*rng.normal(size=(n,2))\n\ndef coverage(x):\n    dist=np.linalg.norm(x[:,None,:]-CENTERS[None,:,:],axis=2)\n    nearest=dist.argmin(1); inside=dist.min(1)<=3*DATA_STD\n    counts=np.bincount(nearest[inside],minlength=8)\n    return {'n':len(x),'covered_modes':int(np.sum(counts>=.01*len(x))),\n            'mode_counts_within_3sigma':counts.tolist(),'on_mode_fraction':float(inside.mean()),\n            'median_distance_to_center':float(np.median(dist.min(1))),\n            'off_plot_fraction':float(np.mean(np.any(abs(x)>3.1,axis=1)))}\n\nclass MLP:\n    def __init__(self,sizes,seed,final_scale=1.):\n        rng=np.random.default_rng(seed); self.p=[]\n        for k,(a,b) in enumerate(zip(sizes[:-1],sizes[1:])):\n            scale=final_scale if k==len(sizes)-2 else 1.\n            self.p.extend([rng.normal(size=(a,b))*np.sqrt(2/(a+b))*scale,np.zeros(b)])\n        self.m=[np.zeros_like(p) for p in self.p]; self.v=[np.zeros_like(p) for p in self.p]; self.step=0\n    def forward(self,x):\n        acts=[x]\n        for k in range(0,len(self.p),2):\n            x=x@self.p[k]+self.p[k+1]\n            if k<len(self.p)-2: x=np.tanh(x)\n            acts.append(x)\n        return x,acts\n    def backward(self,acts,grad):\n        grads=[None]*len(self.p)\n        for layer in range(len(acts)-2,-1,-1):\n            if layer<len(acts)-2: grad=grad*(1-acts[layer+1]**2)\n            grads[2*layer]=acts[layer].T@grad; grads[2*layer+1]=grad.sum(0)\n            grad=grad@self.p[2*layer].T\n        return grad,grads\n    def update(self,grads,lr,b1=.9,b2=.999):\n        self.step+=1\n        for k,(p,g) in enumerate(zip(self.p,grads)):\n            self.m[k]=b1*self.m[k]+(1-b1)*g; self.v[k]=b2*self.v[k]+(1-b2)*g*g\n            p-=lr*(self.m[k]/(1-b1**self.step))/(np.sqrt(self.v[k]/(1-b2**self.step))+1e-8)\n    def flat(self): return np.concatenate([p.ravel() for p in self.p])\n    def state(self): return {f'p{i}':p.copy() for i,p in enumerate(self.p)}\n\ndef sigmoid(x): return 1/(1+np.exp(-np.clip(x,-50,50)))\ndef bce_logits(logit,label): return np.mean(np.logaddexp(0,logit)-label*logit)\ndef dump_json(path,data): path.write_text(json.dumps(data,indent=2,ensure_ascii=False),encoding='utf8')\ndef check_gradient():\n    m=MLP([2,5,2],980); x=np.random.default_rng(981).normal(size=(3,2)); target=np.ones((3,2))\n    y,a=m.forward(x); _,g=m.backward(a,(y-target)/y.size)\n    errors=[]\n    for k,ij in [(0,(0,1)),(2,(1,0)),(3,(0,))]:\n        old=m.p[k][ij]; h=1e-5\n        m.p[k][ij]=old+h; plus=.5*np.mean((m.forward(x)[0]-target)**2)\n        m.p[k][ij]=old-h; minus=.5*np.mean((m.forward(x)[0]-target)**2)\n        m.p[k][ij]=old; errors.append(abs((plus-minus)/(2*h)-g[k][ij]))\n    assert max(errors)<1e-7,errors\n    return max(errors)\n\ndef scatter_base(ax,real):\n    ax.scatter(real[:,0],real[:,1],s=4,c='#bcc8cf',alpha=.35,rasterized=True)\n    ax.scatter(CENTERS[:,0],CENTERS[:,1],s=35,c='#e5ac33',marker='x',zorder=10)\n    ax.set(xlim=(-3.1,3.1),ylim=(-3.1,3.1),aspect='equal',xlabel='x₁',ylabel='x₂')\n\ndef setup_style():\n    plt.rcParams.update({'font.family':'DejaVu Sans','font.size':10,'axes.spines.top':False,'axes.spines.right':False,'figure.facecolor':'white','savefig.facecolor':'white'})\n",
      "outputs": [],
      "id": "lab08-009"
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_block_id": "e8faf3e5f9e8429d8f1620a41fc7fc32",
        "output_provenance": {
          "origin": "Deepnote executed snapshot",
          "run_id": "8802fa19-cf89-4ccd-965c-0685e830b443",
          "content_hash": "sha256:097974c44d25374dcc251e0b589fde6f105332b7e2d143ed290f8ceb4044c63f"
        },
        "verified_cloud_execution": {
          "run_id": "8802fa19-cf89-4ccd-965c-0685e830b443",
          "content_hash": "sha256:097974c44d25374dcc251e0b589fde6f105332b7e2d143ed290f8ceb4044c63f",
          "output_preview_omitted": false,
          "completed_at": "2026-09-30T02:24:39.432Z"
        },
        "cloud_content_sha256": "sha256:097974c44d25374dcc251e0b589fde6f105332b7e2d143ed290f8ceb4044c63f"
      },
      "execution_count": null,
      "source": "# Frozen settings: the comparison changes only D updates per G update.\nCONFIG={'seed_model':700,'seed_training':701,'seed_evaluation':702,'steps':4000,'batch':256,\n        'learning_rate_G':.0003,'learning_rate_D':.0003,'D_updates':[1,3],\n        'architecture_G':[2,64,64,2],'architecture_D':[2,64,64,1],\n        'checkpoints':[0,1000,4000],'data':'8 equal Gaussian modes; radius=2; sigma=.12',\n        'coverage':'At least 1% of all 2048 generated points within radius .36 of a center',\n        'seed_policy':'same initialization, same real/zD/zG batch at each G iteration; extra D updates reuse that batch',\n        'budget_note':'Equal G updates, unequal D computation. No hyperparameter search.'}\n",
      "outputs": [],
      "id": "lab08-010"
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_block_id": "394bcae81be8438fac5aadb5aa94a675",
        "output_provenance": {
          "origin": "Deepnote executed snapshot",
          "run_id": "8802fa19-cf89-4ccd-965c-0685e830b443",
          "content_hash": "sha256:d5fb5734cebe581b3fb4879503b2e55560146f7a1d6b822c38df0c17c8a39b77"
        },
        "verified_cloud_execution": {
          "run_id": "8802fa19-cf89-4ccd-965c-0685e830b443",
          "content_hash": "sha256:d5fb5734cebe581b3fb4879503b2e55560146f7a1d6b822c38df0c17c8a39b77",
          "output_preview_omitted": false,
          "completed_at": "2026-09-30T02:24:39.432Z"
        },
        "cloud_content_sha256": "sha256:d5fb5734cebe581b3fb4879503b2e55560146f7a1d6b822c38df0c17c8a39b77"
      },
      "execution_count": null,
      "source": "# A real D step and a real G step. NumPy has no implicit graph.\ndef d_step(G,D,real,z,lr):\n    fake,_=G.forward(z)  # values only: G cache is discarded; no G backward or optimizer call\n    inp=np.concatenate([real,fake]); target=np.concatenate([np.ones((len(real),1)),np.zeros((len(fake),1))])\n    logits,cacheD=D.forward(inp)\n    _,gradsD=D.backward(cacheD,(sigmoid(logits)-target)/len(inp))\n    loss=float(bce_logits(logits,target)); D.update(gradsD,lr,b1=.5,b2=.99)\n    return loss\n\ndef g_step(G,D,z,lr):\n    fake,cacheG=G.forward(z); logits,cacheD=D.forward(fake)\n    # Non-saturating G loss is -mean(log D(G(z))). D backward supplies dL/dfake.\n    d_fake,unused_D_grads=D.backward(cacheD,(sigmoid(logits)-1)/len(z))\n    _,gradsG=G.backward(cacheG,d_fake)\n    loss=float(bce_logits(logits,1)); G.update(gradsG,lr,b1=.5,b2=.99)\n    # D.update is deliberately absent: its Jacobian is used, its weights are fixed.\n    return loss,{'dL_dfake_norm':float(np.linalg.norm(d_fake)),\n                 'dL_dGparams_norm':float(np.linalg.norm(np.concatenate([g.ravel() for g in gradsG])))}\n\ndef gradient_probe(G,D,rng):\n    real=draw_data(rng,32); z=rng.normal(size=(32,2))\n    beforeG=G.flat().copy(); beforeD=D.flat().copy()\n    fake=G.forward(z)[0]; logit_real=D.forward(real)[0]; logit_fake=D.forward(fake)[0]\n    lossD=d_step(G,D,real,z,CONFIG['learning_rate_D'])\n    record={'D_step':{'loss':lossD,'G_parameter_change':float(np.linalg.norm(G.flat()-beforeG)),\n                     'D_parameter_change':float(np.linalg.norm(D.flat()-beforeD)),\n                     'first_real':real[0].tolist(),'first_generated':fake[0].tolist(),\n                     'first_real_logit':float(logit_real[0,0]),'first_real_probability':float(sigmoid(logit_real)[0,0]),\n                     'first_fake_logit':float(logit_fake[0,0]),'first_fake_probability':float(sigmoid(logit_fake)[0,0]),\n                     'first_real_dL_dlogit':float((sigmoid(logit_real)[0,0]-1)/64),\n                     'first_fake_dL_dlogit':float(sigmoid(logit_fake)[0,0]/64)}}\n    beforeG=G.flat().copy(); beforeD=D.flat().copy()\n    fake,cacheG=G.forward(z); lg,cd=D.forward(fake); dx,_=D.backward(cd,(sigmoid(lg)-1)/len(z))\n    lossG,details=g_step(G,D,z,CONFIG['learning_rate_G'])\n    record['G_step']={'loss':lossG,'G_parameter_change':float(np.linalg.norm(G.flat()-beforeG)),\n                      'D_parameter_change':float(np.linalg.norm(D.flat()-beforeD)),\n                      'first_fake_logit':float(lg[0,0]),'first_fake_probability':float(sigmoid(lg)[0,0]),\n                      'first_dL_dlogit':float((sigmoid(lg)[0,0]-1)/len(z)),\n                      'first_dL_dfake':dx[0].tolist(),**details}\n    return record\n",
      "outputs": [],
      "id": "lab08-011"
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_block_id": "6ba30faa9eb640099a9b44ddf476ef5d",
        "output_provenance": {
          "origin": "Deepnote executed snapshot",
          "run_id": "8802fa19-cf89-4ccd-965c-0685e830b443",
          "content_hash": "sha256:564d19b0c3ac07a7572fc72ed36db1da110e23f31ce9f45978cddf2cfb35796c"
        },
        "verified_cloud_execution": {
          "run_id": "8802fa19-cf89-4ccd-965c-0685e830b443",
          "content_hash": "sha256:564d19b0c3ac07a7572fc72ed36db1da110e23f31ce9f45978cddf2cfb35796c",
          "output_preview_omitted": false,
          "completed_at": "2026-09-30T02:24:39.432Z"
        },
        "cloud_content_sha256": "sha256:564d19b0c3ac07a7572fc72ed36db1da110e23f31ce9f45978cddf2cfb35796c"
      },
      "execution_count": null,
      "source": "# Train both frozen arms and save every stated result.\n@threadpool_limits.wrap(limits=1, user_api='blas')\ndef run(output_dir='outputs'):\n    begin=time.time(); out=Path(output_dir); out.mkdir(parents=True,exist_ok=True); setup_style()\n    max_grad_error=check_gradient()\n    erng=np.random.default_rng(CONFIG['seed_evaluation']); fixed_z=erng.normal(size=(2048,2)); real=draw_data(erng,2048)\n    grid_axis=np.linspace(-3.1,3.1,101); xx,yy=np.meshgrid(grid_axis,grid_axis); grid=np.c_[xx.ravel(),yy.ravel()]\n    records={}; saved={'centers':CENTERS,'fixed_z':fixed_z,'real_evaluation':real,'grid':grid}\n    fig,axes=plt.subplots(2,3,figsize=(14,8),constrained_layout=True)\n    lossfig,lossaxes=plt.subplots(1,2,figsize=(11,3.6),constrained_layout=True)\n    for row,ratio in enumerate(CONFIG['D_updates']):\n        G=MLP(CONFIG['architecture_G'],CONFIG['seed_model']); D=MLP(CONFIG['architecture_D'],CONFIG['seed_model']+1)\n        rng=np.random.default_rng(CONFIG['seed_training']); snaps={}; curve=[]; start=time.time()\n        for step in range(CONFIG['steps']+1):\n            if step in CONFIG['checkpoints']:\n                generated=G.forward(fixed_z)[0]; field=sigmoid(D.forward(grid)[0]).reshape(xx.shape)\n                metrics=coverage(generated); snaps[str(step)]=metrics\n                saved[f'd{ratio}_step{step}_samples']=generated; saved[f'd{ratio}_step{step}_Dfield']=field\n                ax=axes[row,CONFIG['checkpoints'].index(step)]\n                mesh=ax.contourf(xx,yy,field,levels=np.linspace(0,1,11),cmap='Blues',alpha=.6)\n                scatter_base(ax,real); ax.scatter(generated[:,0],generated[:,1],s=4,c='#e06b38',alpha=.5,rasterized=True)\n                ax.set_title(f'D:G={ratio}:1 · G step {step}\\ncoverage {metrics[\"covered_modes\"]}/8 · near modes {metrics[\"on_mode_fraction\"]:.1%}')\n            if step==CONFIG['steps']: break\n            real_batch=draw_data(rng,CONFIG['batch']); zD=rng.normal(size=(CONFIG['batch'],2)); zG=rng.normal(size=(CONFIG['batch'],2))\n            for _ in range(ratio): lossD=d_step(G,D,real_batch,zD,CONFIG['learning_rate_D'])\n            lossG,_=g_step(G,D,zG,CONFIG['learning_rate_G'])\n            if step%100==0: curve.append([step+1,lossD,lossG])\n        np.savez_compressed(out/f'lab08-d{ratio}-weights.npz',**{f'G_{k}':v for k,v in G.state().items()},**{f'D_{k}':v for k,v in D.state().items()})\n        curve=np.array(curve); saved[f'd{ratio}_losses']=curve\n        lossaxes[0].plot(curve[:,0],curve[:,1],label=f'D:G={ratio}:1');lossaxes[1].plot(curve[:,0],curve[:,2],label=f'D:G={ratio}:1')\n        records[f'D_updates_{ratio}']={'snapshots':snaps,'elapsed_seconds':time.time()-start}\n    fig.colorbar(mesh,ax=axes,label='D(x): estimated real probability',shrink=.6)\n    fig.suptitle('Same latent points throughout · gray: real · orange: generated · ×: mode centers',fontsize=13)\n    fig.savefig(out/'lab08-training.png',dpi=170);plt.close(fig)\n    for ax,title in zip(lossaxes,['Discriminator loss','Generator non-saturating loss']): ax.set(xlabel='G updates',ylabel=title);ax.legend()\n    lossfig.savefig(out/'lab08-losses.png',dpi=170);plt.close(lossfig)\n    probe=gradient_probe(MLP(CONFIG['architecture_G'],CONFIG['seed_model']),MLP(CONFIG['architecture_D'],CONFIG['seed_model']+1),np.random.default_rng(910))\n    np.savez_compressed(out/'lab08-data.npz',**saved)\n    report={'config':CONFIG,'results':records,'gradient_probe':probe,'gradient_check_max_abs_error':max_grad_error,\n            'runtime_seconds':time.time()-begin,'python':platform.python_version(),'numpy':np.__version__,\n            'blas_threadpools':threadpool_info(),\n            'run_utc':time.strftime('%Y-%m-%dT%H:%M:%SZ',time.gmtime()),'real_reference_coverage':coverage(real)}\n    dump_json(out/'lab08-metrics.json',report);print(json.dumps(report,ensure_ascii=False,indent=2));return report\n",
      "outputs": [],
      "id": "lab08-012"
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_block_id": "96d0204df39f47b4b6e4ed6e931424c6",
        "output_provenance": {
          "origin": "Deepnote executed snapshot",
          "run_id": "8802fa19-cf89-4ccd-965c-0685e830b443",
          "content_hash": "sha256:409c877c813182d86579c3db92e2bcb6040861770755bad887117da62f2ba875"
        },
        "verified_cloud_execution": {
          "run_id": "8802fa19-cf89-4ccd-965c-0685e830b443",
          "content_hash": "sha256:409c877c813182d86579c3db92e2bcb6040861770755bad887117da62f2ba875",
          "output_preview_omitted": false,
          "completed_at": "2026-09-30T02:24:39.432Z"
        },
        "cloud_content_sha256": "sha256:409c877c813182d86579c3db92e2bcb6040861770755bad887117da62f2ba875"
      },
      "execution_count": null,
      "source": "report = run('outputs')\n",
      "outputs": [
        {
          "name": "stdout",
          "text": "{\n  \"config\": {\n    \"seed_model\": 700,\n    \"seed_training\": 701,\n    \"seed_evaluation\": 702,\n    \"steps\": 4000,\n    \"batch\": 256,\n    \"learning_rate_G\": 0.0003,\n    \"learning_rate_D\": 0.0003,\n    \"D_updates\": [\n      1,\n      3\n    ],\n    \"architecture_G\": [\n      2,\n      64,\n      64,\n      2\n    ],\n    \"architecture_D\": [\n      2,\n      64,\n      64,\n      1\n    ],\n    \"checkpoints\": [\n      0,\n      1000,\n      4000\n    ],\n    \"data\": \"8 equal Gaussian modes; radius=2; sigma=.12\",\n    \"coverage\": \"At least 1% of all 2048 generated points within radius .36 of a center\",\n    \"seed_policy\": \"same initialization, same real/zD/zG batch at each G iteration; extra D updates reuse that batch\",\n    \"budget_note\": \"Equal G updates, unequal D computation. No hyperparameter search.\"\n  },\n  \"results\": {\n    \"D_updates_1\": {\n      \"snapshots\": {\n        \"0\": {\n          \"n\": 2048,\n          \"covered_modes\": 0,\n          \"mode_counts_within_3sigma\": [\n            0,\n            0,\n            0,\n            0,\n            0,\n            0,\n            0,\n            0\n          ],\n          \"on_mode_fraction\": 0.0,\n          \"median_distance_to_center\": 1.6638549925234734,\n          \"off_plot_fraction\": 0.0\n        },\n        \"1000\": {\n          \"n\": 2048,\n          \"covered_modes\": 7,\n          \"mode_counts_within_3sigma\": [\n            103,\n            35,\n            55,\n            55,\n            75,\n            32,\n            19,\n            70\n          ],\n          \"on_mode_fraction\": 0.216796875,\n          \"median_distance_to_center\": 0.6076510860265074,\n          \"off_plot_fraction\": 0.0\n        },\n        \"4000\": {\n          \"n\": 2048,\n          \"covered_modes\": 8,\n          \"mode_counts_within_3sigma\": [\n            128,\n            143,\n            164,\n            148,\n            124,\n            101,\n            163,\n            140\n          ],\n          \"on_mode_fraction\": 0.54248046875,\n          \"median_distance_to_center\": 0.32482193534371595,\n          \"off_plot_fraction\": 0.0\n        }\n      },\n      \"elapsed_seconds\": 15.850052118301392\n    },\n    \"D_updates_3\": {\n      \"snapshots\": {\n        \"0\": {\n          \"n\": 2048,\n          \"covered_modes\": 0,\n          \"mode_counts_within_3sigma\": [\n            0,\n            0,\n            0,\n            0,\n            0,\n            0,\n            0,\n            0\n          ],\n          \"on_mode_fraction\": 0.0,\n          \"median_distance_to_center\": 1.6638549925234734,\n          \"off_plot_fraction\": 0.0\n        },\n        \"1000\": {\n          \"n\": 2048,\n          \"covered_modes\": 8,\n          \"mode_counts_within_3sigma\": [\n            127,\n            179,\n            40,\n            128,\n            128,\n            157,\n            54,\n            150\n          ],\n          \"on_mode_fraction\": 0.47021484375,\n          \"median_distance_to_center\": 0.3759880884700424,\n          \"off_plot_fraction\": 0.0\n        },\n        \"4000\": {\n          \"n\": 2048,\n          \"covered_modes\": 6,\n          \"mode_counts_within_3sigma\": [\n            279,\n            290,\n            10,\n            282,\n            259,\n            285,\n            16,\n            333\n          ],\n          \"on_mode_fraction\": 0.8564453125,\n          \"median_distance_to_center\": 0.1623375488908803,\n          \"off_plot_fraction\": 0.0\n        }\n      },\n      \"elapsed_seconds\": 30.7358877658844\n    }\n  },\n  \"gradient_probe\": {\n    \"D_step\": {\n      \"loss\": 0.7374548193319083,\n      \"G_parameter_change\": 0.0,\n      \"D_parameter_change\": 0.01993765754756485,\n      \"first_real\": [\n        0.07340117214257046,\n        -2.3491666345894755\n      ],\n      \"first_generated\": [\n        -0.2576279886478896,\n        0.04148777982071576\n      ],\n      \"first_real_logit\": -0.05276182833296705,\n      \"first_real_probability\": 0.48681260204293425,\n      \"first_fake_logit\": -0.09760943391934174,\n      \"first_fake_probability\": 0.47561699774099214,\n      \"first_real_dL_dlogit\": -0.008018553093079153,\n      \"first_fake_dL_dlogit\": 0.007431515589703002\n    },\n    \"G_step\": {\n      \"loss\": 0.6882525396619139,\n      \"G_parameter_change\": 0.02007749873901901,\n      \"D_parameter_change\": 0.0,\n      \"first_fake_logit\": -0.09155189697681232,\n      \"first_fake_probability\": 0.4771279991403109,\n      \"first_dL_dlogit\": -0.016339750026865284,\n      \"first_dL_dfake\": [\n        -0.005727314823879403,\n        -0.0010267512323057764\n      ],\n      \"dL_dfake_norm\": 0.03087642494617375,\n      \"dL_dGparams_norm\": 0.3919912784600375\n    }\n  },\n  \"gradient_check_max_abs_error\": 1.4114598378967003e-11,\n  \"runtime_seconds\": 55.833566665649414,\n  \"python\": \"3.13.12\",\n  \"numpy\": \"2.4.6\",\n  \"blas_threadpools\": [\n    {\n      \"user_api\": \"blas\",\n      \"internal_api\": \"openblas\",\n      \"num_threads\": 1,\n      \"prefix\": \"libscipy_openblas\",\n      \"filepath\": \"/root/venv/lib/python3.13/site-packages/numpy.libs/libscipy_openblas64_-32a4b2a6.so\",\n      \"version\": \"0.3.31.188.0\",\n      \"threading_layer\": \"pthreads\",\n      \"architecture\": \"Haswell\"\n    }\n  ],\n  \"run_utc\": \"2026-09-30T02:24:33Z\",\n  \"real_reference_coverage\": {\n    \"n\": 2048,\n    \"covered_modes\": 8,\n    \"mode_counts_within_3sigma\": [\n      239,\n      253,\n      275,\n      259,\n      251,\n      257,\n      240,\n      257\n    ],\n    \"on_mode_fraction\": 0.99169921875,\n    \"median_distance_to_center\": 0.139217189657945,\n    \"off_plot_fraction\": 0.0\n  }\n}\n",
          "output_type": "stream"
        }
      ],
      "id": "lab08-013"
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_block_id": "edecbc164b914345ba5544355bc0c9f8",
        "verified_cloud_execution": {
          "run_id": "8802fa19-cf89-4ccd-965c-0685e830b443",
          "content_hash": "sha256:3db74ab83e671adf866066eefe4d7ec3abead7cceb88ada0bae6da5162b899b5",
          "output_preview_omitted": true,
          "completed_at": "2026-09-30T02:24:39.432Z"
        },
        "cloud_content_sha256": "sha256:3db74ab83e671adf866066eefe4d7ec3abead7cceb88ada0bae6da5162b899b5"
      },
      "execution_count": null,
      "source": "from IPython.display import display, Image\ndisplay(Image(filename='outputs/lab08-training.png'))\ndisplay(Image(filename='outputs/lab08-losses.png'))\n",
      "outputs": [],
      "id": "lab08-014"
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "name": "python",
      "version": "3.12"
    },
    "course_execution": {
      "origin": "independent Python process",
      "saved_results": "2026-09-29",
      "script": "lab08_gan.py",
      "script_sha256": "aaf2fff25340feb5c21da74d27e9695069b6156a969e0d6b32576668de81d858",
      "deepnote_execution_claimed": false
    },
    "lecture04": {
      "source_notebook_id": "f16324b960a446e2aa03168c17e9fc0b",
      "saved_results_origin": "Static baseline figures retained; code outputs carry per-cell run provenance."
    },
    "deepnote_readback": {
      "notebook_id": "f16324b960a446e2aa03168c17e9fc0b",
      "updated_at": "2026-09-30T02:27:57.450Z",
      "block_count": 15,
      "all_source_blocks_match": true
    },
    "execution_verification": "Every code block matches a successful Deepnote run snapshot by ID and hash. Per-cell metadata distinguishes imported cloud outputs from retained local baseline outputs."
  },
  "nbformat": 4,
  "nbformat_minor": 5
}