# Policy Transfer by Imagining Flow

#### Ben Eisner

<p style="font-size: .7em;"><a href=https://beisner.me/talks/flow>https://beisner.me/talks/flow</a></p>

-----

## A Motivating Example

#### you might remember this one!

##### (but I promise it's a bit different this time...)

---

How does this object work?

---

<div class="r-stack">
  <div class="fragment current-visible" data-fragment-index="1">For now, let's just ask:<p><b>How does this object move?</b></p></div>
  <div>
    <p class="fragment" data-fragment-index="2">Maybe we notice individual parts...</p>
    <p class="fragment" data-fragment-index="3">...and imagine <u>what kinds of motions are possible.</u></p>
    <p class="fragment" data-fragment-index="4">(we can answer this because we've seen lighters before)</p>
  </div>
</div>

---

<div class="r-stack">
    <div class="fragment current-visible" data-fragment-index="1">
        Informally, let's call this visual affordance <mark>"imagined flow"</mark>.
    </div>
    <div class="current-visible" >
        <p class="fragment" data-fragment-index="2">
            With <mark>imagined flow</mark> flow alone,
        </p>
        <p class="fragment" data-fragment-index="3">
            we can derive a reasonably good manipulation policy.
        </p>
    </div>
    
    
</div>

---

Two assumptions we'd like to relax:

1. We have an object model we can segment into parts.

2. <u>We already have an understanding of how we want this object to move.</u>

---

Relaxation 1: No need for object model or segmentation.

<div class="fragment">
<b>Geometry seems relevant to this task</b>, so we choose to operate on <mark>unlabeled point clouds</mark>
</div>

(colors here are for visualization, not provided)

---

Relaxation 2: No need to understand the task completely - we can use a motion demonstration from another object!

<div class="fragment">
In other words, <mark>observe flow on a source object</mark> during a demonstration, and try and <u>transfer that characteristic flow</u> to another object.
</div>
---

## <u>Problem Statement</u>

### Given

<div class="container">
    <div class="col given fragment">
        <u>Source Object $A$:</u><br>
        <ul>
            <li>a source point cloud $P_A$</li>
            <li>a ground-truth flow $F_A$</li>
        </ul>
    </div>
    <div class="col given fragment">
        <u>Destination object $B$:</u><br>
        <ul>
            <li>a destination point cloud $P_B$</li>
        </ul>
    </div>
</div>

<br>

### Task

<div class="fragment">
<mark>Transfer</mark> flow $F_A$ to object $B$ to produce flow $F_B$
so that the object deforms in a <u>characteristically similar way</u>.
</div>

---

In other words, we want to learn a function

`$$\tilde{F_B} = f_{\theta}(P_A, P_B, F_A)$$`

where, for some ground truth flow `$F_{B|A}$` we minimize:

`$$\mathbb{E}\left[D(F_{B|A}, \tilde{F_B})\right]$$`

for some similarity function `$D$`.

-----

### Related Work

---

<u>Predicting articulation structure.</u>

Screw-Net

Category-Level Articulated Object Pose Estimation

Visual Identification of Articulated Object Parts

---
### <u>Screw-Net</u>

**Key idea: unify articulations under screw theory, and estimate screw parameters from video.**

<div style="font-size: 0.3em; line-height: 1.2em;">
    <u>ScrewNet: Category-Independent Articulation Model Estimation From Depth Images Using Screw Theory</u> <br>
    Ajinkya Jain, Rudolf Lioutikov, Caleb Chuck, Scott Niekum <br>
    <a href="https://arxiv.org/abs/2008.10518">https://arxiv.org/abs/2008.10518</a>
</div>
---

### <u>Category-Level Articulated Object Pose Estimation</u>

**Key idea: Consider each object into a cannonical space, segment, and predict articulations w.r.t. a part template.**

<div style="font-size: 0.3em; line-height: 1.2em;">
    <u>Category-Level Articulated Object Pose Estimation</u> <br>
    Xiaolong Li, He Wang, Li Yi, Leonidas Guibas, A. Lynn Abbott, Shuran Song <br>
    <a href="https://arxiv.org/abs/1912.11913">https://arxiv.org/abs/1912.11913</a>
</div>

---

### Visual Identification of Articulated Object Parts

**Key idea: Densely predict relative displacement of segmented parts, and use that to predict articulation type.**

<div style="font-size: 0.3em; line-height: 1.2em;">
    <u>Visual Identification of Articulated Object Parts</u> <br>
    Vicky Zeng, Timothy E. Lee, Jacky Liang, Oliver Kroemer <br>
    <a href="https://arxiv.org/abs/2012.00284">https://arxiv.org/abs/2012.00284</a>
</div>

---

<u>Learning manipulation affordances for articulated objects.</u>

Act the Part: Learning Interaction Strategies for Articulated Object Part Discovery

Where2Act: From Pixels to Actions for Articulated 3D Objects

---

### Act the Part: Learning Interaction Strategies for Articulated Object Part Discovery

**Key idea: Learn visual affordances to achieve articulation, and use motion to segment novel object parts.**

<div style="font-size: 0.3em; line-height: 1.2em;">
    <u>Act the Part: Learning Interaction Strategies for Articulated Object Part Discovery
    </u> <br>
    Samir Yitzhak Gadre, Kiana Ehsani, Shuran Song <br>
    <a href="https://arxiv.org/abs/2105.01047">https://arxiv.org/abs/2105.01047</a>
</div>

---

### Where2Act: From Pixels to Actions for Articulated 3D Objects

**Key idea: Learn visual affordances for distinct motion primitives from simulated interactions.**

<div style="font-size: 0.3em; line-height: 1.2em;">
    <u>Where2Act: From Pixels to Actions for Articulated 3D Objects
    </u> <br>
    Kaichun Mo, Leonidas Guibas, Mustafa Mukadam, Abhinav Gupta, Shubham Tulsiani<br>
    <a href="https://arxiv.org/abs/2101.02692">https://arxiv.org/abs/2101.02692</a>
</div>

-----

### Method

---

### Architecture

---

### Architecture

<ul>
    <li class="fragment"><mark>FlowNet</mark> encodes the source object geometry and the object flow (i.e. from a demonstration)</li>
    <li class="fragment">
        <mark>TransferNet</mark>
        <ul>
            <li>first, encodes the geometry of the destination object</li>
            <li>second, injects the FlowNet embedding at the lowest layer</li>
            <li>third, reconstructs the dstination flow</li>
        </ul>
    </li>
</ul>

---

### Dataset: PartNet-Mobility

<ul>
    <li class="fragment">50+ object categories, 2000+ objects</li>
    <li class="fragment">3D models for each object</li>
    <li class="fragment"><u>Models annotated with articulations, so demonstrations can be generated.</u></li>
</ul>

---

### Training

<div style="font-size: 0.7em;">
Choose a single training class and a single articulation type (i.e. faucets/handles)

<u>For each training step:</u>
<ul>
    <li class="fragment">Select a pair of objects $A,B$ which each have 1+ instance of the articulation type in question</li>
    <li class="fragment">On each object, generate flows $F_A$, $F_{B|A}$ that correspond to articulation motions in the same direction and with the same magnitude </li>
    <li class="fragment">Construct tuple ($P_A, F_A, P_B, F_{A|B}$)</li>
    <li class="fragment">Compute flow: $$\tilde{F}_{B|A} = f_{\theta, \phi}(P_A, F_A, P_B)$$</li>
    <li class="fragment">Compute loss: $$L_{\text{MSE}} = \frac{1}{N}\sum_{f_{B,i}, \tilde{f}_{B,i} \in (F_{B|A}, \tilde{F}_{B|A})}{|| f_{B,i} - \tilde{f}_{B,i} ||^2}$$</li>
</ul>
</div>

---

PSA: Don't give up on your training curves too early.

(2hrs in, loss starts to drop)

-----

### Results (preliminary)

---

## Quantitative

This table shows the <mark>RMSE of the predicted flow vectors</mark>, averaged across the
object.

<table>
    <tr>
        <th></th>
        <th>Door</th>
        <th>Faucet</th>
        <th>Laptop</th>
        <th>Stapler</th>
    </tr>
    <tr>
        <td>Ours (train/val)</td>
        <td>0.05/0.32</td>
        <td>0.01/0.08</td>
        <td>0.01/0.12</td>
        <td>0.01/0.2</td>
    </tr>
</table>

Takeaway: the model has the capacity to represent sophisticated flows, and can generalize pretty well to novel instances.

Current TODO: Gather better results for baselines (our trivial ones don't work)

---

### Qualitative

---

### Laptop, train

---

---

---

### Faucet, train

---

---

---

### Faucet, unseen

---

---

-----

### Summary

<u><b>Problem Statement:</b></u>

Transfer flow from one object to another.

<u>**Method**:</u>

Use a dual-headed encoder, where we inject the flow embedding of a source object directly into the decoder for the destination flow.

<u>**Results**:</u>

They look ok!

-----

### Discussion Questions

* What are some ways we can use flow in downstream manipulation tasks?

* Is flow the "right" representation for this task? Why not articulation parameters directly?

* How might this model extend to objects with many articulations?