From ee7e8641f89075936fcfd1bb859757be5f5adc5a Mon Sep 17 00:00:00 2001 From: crosstyan Date: Tue, 9 Dec 2025 15:54:50 +0800 Subject: [PATCH] init --- .DS_Store | Bin 0 -> 10244 bytes README.md | 106 ++++++++++++++++++++++++++++++++++++++++++++++++++ build.md | 1 + fig.mermaid | 93 +++++++++++++++++++++++++++++++++++++++++++ fig/.DS_Store | Bin 0 -> 6148 bytes fig/fig.svg | 1 + 6 files changed, 201 insertions(+) create mode 100644 .DS_Store create mode 100644 README.md create mode 100644 build.md create mode 100644 fig.mermaid create mode 100644 fig/.DS_Store create mode 100644 fig/fig.svg diff --git a/.DS_Store b/.DS_Store new file mode 100644 index 0000000000000000000000000000000000000000..5496de1ef07efcbf687e3e4d035f557d0b79a6f0 GIT binary patch literal 10244 zcmeHMTWl0n7(V}Ofinz+p|;#)+?5q5P-NReMXRuFx5}+hx-GqMS$21ZcEaq;vNO9& zZEEV17m&mkiHaIuh?fL43K5=6l%R?6fgn*{G%?Y5c~S9_m>B;vXSVH9@J&O~Imw*= zod0r8|8Kv4Cg+q80+W3RJrd z$1)qB40$r@$uTa`f(>{mN*)T{5d(NQ>O-Di)RSXe@^Av);RC##!8@UVyF0EQa@`3; zx%7J<0Uv>>2(Ytz1-X|D5}i2Z=l4SwCfhPrES55?6zb@SH%~VS&ZU-eyZW#8?9imU z0bZ2>atL=FOcm{{xAOT^*3fMy61fUOpkl_%S#%ba=>GVSRg61HCvUeVb9>pW+p>&Q zX`D>$(Y0biu5PnTC#joS&JEC24PEy2>ZWQHTdlmQ+HP!|8>mRdguHlUq@l6CF4WM} zI9eAPsc+i2p)Rz(v1xQvq&4An&0UGZM#i*`VXD}`d;5?|J|FLo=48%b5cato3 zp^YaRh?f$*R!UTyElHJ1Z(skw-hByK&f8j-W+WZGpv97oW+&u%Hs-xGZJC`pt$_LV zrY$3%G0`$Voix&Ubk?!K1GcH0-%rQr6UAK(gD$l)XPQQZI_K-F+DO zz(b`CO;nWC9wV933{UkyGpe@t7I?;i)W@XDhXxm>eU| zkyprj7=!bflsF2Pm!3ciOQ z;77O)zri2yrw|b42$jNIAtbC4Rtsx{O~Phji?CZ55E8-=&p;@Z;QsQ=gkhYQO3*HN zZxrFo%?mB~$d;|!w(q!Y3+>CfthDZ^u34}!w0!lt^&1~*87Cm;Yl)(hsqeDAtVUe6 zyNtMzscsEQcLjT}9JHeoMaYY?REqK}R<`sC^S)aO2jv-9Fnqc~stw9Bu~_(YO>k{c z7O+P7G!kr(~e?l&k zFOcIuA;!V z@HjjT$Ke?`0Vm;kWcn$133>h+ybf=`yKolH!TWIDW&Ee`Ib4HpZs2f7Ifvo;1cxVo zZ(lvxRX>A&Z#{-*tckDfRBPF{Hqgm7@!|^coM~9*AZki=#H@bv5%3Z45%3Z45%3Y1 zE(ip8q9Jzw-+t@=|4$b<{5*XGrX>PU5sSxKG4)#=3D{kG2%m%au*UqxxO79XbFbr3 ty4UeIVb}4s_%L7ZD#9H{J>3|WZaDh5;VA#7e+E?g=YRkFKfTZY{{VC2-|+wd literal 0 HcmV?d00001 diff --git a/README.md b/README.md new file mode 100644 index 0000000..c107498 --- /dev/null +++ b/README.md @@ -0,0 +1,106 @@ +# Homework 01: Survey & Design of 3D Multi-view Human Pose Estimation System + +调研现有的多视角单人 (multiple view of single person) 多视角多人 (multi-view multi-person) +人体姿态估计系统/管线 + +## 参考答案 + +AI 生成的架构图, 不代表我此刻的实际想法, which changes every moment. + +```mermaid +flowchart TD + %% ========================= + %% Multi-view 2D stage + %% ========================= + subgraph VIEWS["Per-view input (cameras 1..N)"] + direction LR + C1["Cam 1\n2D detections: 133×2 (+conf)"] --> T1["2D latest tracking cache\n(view 1)"] + C2["Cam 2\n2D detections: 133×2 (+conf)"] --> T2["2D latest tracking cache\n(view 2)"] + C3["Cam 3\n2D detections: 133×2 (+conf)"] --> T3["2D latest tracking cache\n(view 3)"] + C4["Cam 4\n2D detections: 133×2 (+conf)"] --> T4["2D latest tracking cache\n(view 4)"] + end + + %% ========================= + %% Cross-view association + %% ========================= + subgraph ASSOC["Cross-view data association (epipolar)"] + direction TB + EPI["Epipolar constraint\n(Sampson / point-to-epiline)"]:::core + CYCLE["Cycle consistency / view-graph pruning"]:::core + GROUP["Assemble per-target multi-view observation set\n{view_id → 133×2}"]:::core + EPI --> CYCLE --> GROUP + end + + T1 --> EPI + T2 --> EPI + T3 --> EPI + T4 --> EPI + + %% ========================= + %% Geometry / lifting + %% ========================= + subgraph GEOM["3D measurement construction"] + direction TB + RT["Camera models\nK, [R|t], SO(3)/SE(3)"]:::meta + DLT["DLT / triangulation (init)"]:::core + NN["Optional NN lifting / completion"]:::core + BA["Optional reprojection refinement\n(1–5 iters)"]:::core + Y["3D measurement y(t)\nJ×3 positions (+quality / R / cov)"]:::out + RT --> DLT --> NN --> BA --> Y + end + + GROUP --> DLT + + %% ========================= + %% Tracking filter + lifecycle + %% ========================= + subgraph FILTER["Tracking filter (per target)"] + direction TB + GATE["Gating\n(Mahalanobis / per-joint + global)"]:::core + IMM["IMM (motion model bank)\n(CV/CA or low/med/high Q)"]:::core + PRED["Predict\nΔt, self-propagate"]:::core + UPD["Update\nKF (linear)\nstate: [p(3J), v(3J)]"]:::core + MISS["Miss handling & track lifecycle\n(tentative → confirmed → deleted)"]:::meta + + GATE --> IMM --> PRED --> UPD --> MISS + end + + Y --> GATE + + %% Optional inertial fusion + IMU["IMU (optional)"]:::meta --> INERT["EKF/UKF branch (optional)\nwhen augmenting state with orientation"]:::meta --> IMM + + %% ========================= + %% IK + optional feedback + %% ========================= + subgraph IKSTAGE["IK stage (constraint / anatomy)"] + direction TB + IK["IK optimization target\n(minimize joint position error,\nadd bone length / joint limits)"]:::core + FB["Optional feedback to filter\npseudo-measurement z_IK with large R"]:::meta + IK --> FB + end + + UPD --> IK + FB -.-> GATE + + %% ========================= + %% SMPL / mesh fitting + %% ========================= + subgraph SMPLSTAGE["SMPL / SMPL-X fitting"] + direction TB + VP["VPoser / pose prior"]:::core + SMPL["SMPL(θ, β, root)\nfit to joints / reprojection"]:::core + JR["JR: Joint Regressor\nmesh → joints (loop closure)"]:::core + OUT["Outputs\nmesh + joints + pose params"]:::out + VP --> SMPL --> JR --> OUT + JR -. residual / reproject .-> SMPL + end + + IK --> SMPL + + classDef core fill:#0b1020,stroke:#5eead4,color:#e5e7eb,stroke-width:1.2px; + classDef meta fill:#111827,stroke:#93c5fd,color:#e5e7eb,stroke-dasharray: 4 3; + classDef out fill:#052e2b,stroke:#34d399,color:#ecfeff,stroke-width:1.4px; +``` + +如果图片不能正确预览, 见 [fig/fig.svg](fig/fig.svg) diff --git a/build.md b/build.md new file mode 100644 index 0000000..70758f0 --- /dev/null +++ b/build.md @@ -0,0 +1 @@ +mmdc -i fig.mermaid -o fig.svg \ No newline at end of file diff --git a/fig.mermaid b/fig.mermaid new file mode 100644 index 0000000..8783fd8 --- /dev/null +++ b/fig.mermaid @@ -0,0 +1,93 @@ +flowchart TD + %% ========================= + %% Multi-view 2D stage + %% ========================= + subgraph VIEWS["Per-view input (cameras 1..N)"] + direction LR + C1["Cam 1\n2D detections: 133×2 (+conf)"] --> T1["2D latest tracking cache\n(view 1)"] + C2["Cam 2\n2D detections: 133×2 (+conf)"] --> T2["2D latest tracking cache\n(view 2)"] + C3["Cam 3\n2D detections: 133×2 (+conf)"] --> T3["2D latest tracking cache\n(view 3)"] + C4["Cam 4\n2D detections: 133×2 (+conf)"] --> T4["2D latest tracking cache\n(view 4)"] + end + + %% ========================= + %% Cross-view association + %% ========================= + subgraph ASSOC["Cross-view data association (epipolar)"] + direction TB + EPI["Epipolar constraint\n(Sampson / point-to-epiline)"]:::core + CYCLE["Cycle consistency / view-graph pruning"]:::core + GROUP["Assemble per-target multi-view observation set\n{view_id → 133×2}"]:::core + EPI --> CYCLE --> GROUP + end + + T1 --> EPI + T2 --> EPI + T3 --> EPI + T4 --> EPI + + %% ========================= + %% Geometry / lifting + %% ========================= + subgraph GEOM["3D measurement construction"] + direction TB + RT["Camera models\nK, [R|t], SO(3)/SE(3)"]:::meta + DLT["DLT / triangulation (init)"]:::core + NN["Optional NN lifting / completion"]:::core + BA["Optional reprojection refinement\n(1–5 iters)"]:::core + Y["3D measurement y(t)\nJ×3 positions (+quality / R / cov)"]:::out + RT --> DLT --> NN --> BA --> Y + end + + GROUP --> DLT + + %% ========================= + %% Tracking filter + lifecycle + %% ========================= + subgraph FILTER["Tracking filter (per target)"] + direction TB + GATE["Gating\n(Mahalanobis / per-joint + global)"]:::core + IMM["IMM (motion model bank)\n(CV/CA or low/med/high Q)"]:::core + PRED["Predict\nΔt, self-propagate"]:::core + UPD["Update\nKF (linear)\nstate: [p(3J), v(3J)]"]:::core + MISS["Miss handling & track lifecycle\n(tentative → confirmed → deleted)"]:::meta + + GATE --> IMM --> PRED --> UPD --> MISS + end + + Y --> GATE + + %% Optional inertial fusion + IMU["IMU (optional)"]:::meta --> INERT["EKF/UKF branch (optional)\nwhen augmenting state with orientation"]:::meta --> IMM + + %% ========================= + %% IK + optional feedback + %% ========================= + subgraph IKSTAGE["IK stage (constraint / anatomy)"] + direction TB + IK["IK optimization target\n(minimize joint position error,\nadd bone length / joint limits)"]:::core + FB["Optional feedback to filter\npseudo-measurement z_IK with large R"]:::meta + IK --> FB + end + + UPD --> IK + FB -.-> GATE + + %% ========================= + %% SMPL / mesh fitting + %% ========================= + subgraph SMPLSTAGE["SMPL / SMPL-X fitting"] + direction TB + VP["VPoser / pose prior"]:::core + SMPL["SMPL(θ, β, root)\nfit to joints / reprojection"]:::core + JR["JR: Joint Regressor\nmesh → joints (loop closure)"]:::core + OUT["Outputs\nmesh + joints + pose params"]:::out + VP --> SMPL --> JR --> OUT + JR -. residual / reproject .-> SMPL + end + + IK --> SMPL + + classDef core fill:#0b1020,stroke:#5eead4,color:#e5e7eb,stroke-width:1.2px; + classDef meta fill:#111827,stroke:#93c5fd,color:#e5e7eb,stroke-dasharray: 4 3; + classDef out fill:#052e2b,stroke:#34d399,color:#ecfeff,stroke-width:1.4px; \ No newline at end of file diff --git a/fig/.DS_Store b/fig/.DS_Store new file mode 100644 index 0000000000000000000000000000000000000000..5008ddfcf53c02e82d7eee2e57c38e5672ef89f6 GIT binary patch literal 6148 zcmeH~Jr2S!425mzP>H1@V-^m;4Wg<&0T*E43hX&L&p$$qDprKhvt+--jT7}7np#A3 zem<@ulZcFPQ@L2!n>{z**++&mCkOWA81W14cNZlEfg7;MkzE(HCqgga^y>{tEnwC%0;vJ&^%eQ zLs35+`xjp>T0

SMPL / SMPL-X fitting

IK stage (constraint / anatomy)

Tracking filter (per target)

3D measurement construction

Cross-view data association (epipolar)

Per-view input (cameras 1..N)

residual / reproject

Cam 1\n2D detections: 133×2 (+conf)

2D latest tracking cache\n(view 1)

Cam 2\n2D detections: 133×2 (+conf)

2D latest tracking cache\n(view 2)

Cam 3\n2D detections: 133×2 (+conf)

2D latest tracking cache\n(view 3)

Cam 4\n2D detections: 133×2 (+conf)

2D latest tracking cache\n(view 4)

Epipolar constraint\n(Sampson / point-to-epiline)

Cycle consistency / view-graph pruning

Assemble per-target multi-view observation set\n{view_id → 133×2}

Camera models\nK, [R|t], SO(3)/SE(3)

DLT / triangulation (init)

Optional NN lifting / completion

Optional reprojection refinement\n(1–5 iters)

3D measurement y(t)\nJ×3 positions (+quality / R / cov)

Gating\n(Mahalanobis / per-joint + global)

IMM (motion model bank)\n(CV/CA or low/med/high Q)

Predict\nΔt, self-propagate

Update\nKF (linear)\nstate: [p(3J), v(3J)]

Miss handling & track lifecycle\n(tentative → confirmed → deleted)

IMU (optional)

EKF/UKF branch (optional)\nwhen augmenting state with orientation

IK optimization target\n(minimize joint position error,\nadd bone length / joint limits)

Optional feedback to filter\npseudo-measurement z_IK with large R

VPoser / pose prior

SMPL(θ, β, root)\nfit to joints / reprojection

JR: Joint Regressor\nmesh → joints (loop closure)

Outputs\nmesh + joints + pose params

 \ No newline at end of file