-
Notifications
You must be signed in to change notification settings - Fork 2
Expand file tree
/
Copy pathE3NeRF.html
More file actions
202 lines (187 loc) · 9.02 KB
/
Copy pathE3NeRF.html
File metadata and controls
202 lines (187 loc) · 9.02 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>E3NeRF</title>
<link rel="stylesheet" type="text/css" href="assets/scripts/bulma.min.css">
<link rel="stylesheet" type="text/css" href="assets/scripts/theme.css">
<link rel="stylesheet" type="text/css" href="https://cdn.bootcdn.net/ajax/libs/font-awesome/4.7.0/css/font-awesome.min.css">
</head>
<body>
<section class="hero is-light" style="">
<div class="hero-body" style="padding-top: 50px;">
<div class="container" style="text-align: center;margin-bottom:5px;">
<h1 class="title">
E<sup>3</sup>NeRF: Efficient Event-Enhanced Neural Radiance Fields from Blurry Images
</h1>
<div class="author"><p style="font-size: 22px">Yunshan Qi<sup>1</sup>       Jia Li<sup>1 *</sup>       Yifan Zhao<sup>1</sup>       Yu Zhang<sup>3</sup>       Lin Zhu<sup>2 *</sup></p></div>
<div class="aff">
<p style="font-size: 20px"><sup>1</sup>Beihang University    <sup>2</sup>Beijing Institute of Technology    <sup>3</sup>SenseTime and Tetras.AI    </p>
<p> </p>
</div>
<div class="group"><p style="font-size: 22px"><a href="http://cvteam.buaa.edu.cn/"><sup>*</sup>CVTEAM</a></p></div>
<div class="con">
<p style="font-size: 24px; margin-top:5px; margin-bottom: 15px;">
<!--b>Accepted by International Conference on Computer Vision (ICCV) 2023</b-->
</p>
</div>
<div class="columns">
<div class="column"></div>
<div class="column"></div>
<!--div class="column">
<a href="assets/E2NeRF/Papers.pdf" target="_blank">
<p class="link" style="font-size: 24px; margin-top:5px; margin-bottom: 15px;">[Paper]</p>
</a>
</-->
<div class="column">
<a href="https://arxiv.org/html/2408.01840v2" target="_blank">
<p class="link" style="font-size: 24px; margin-top:5px; margin-bottom: 15px;">[Arxiv]</p>
</a>
</div>
<div class="column">
<a href="https://github.com/iCVTEAM/E3NeRF" target="_blank">
<p class="link" style="font-size: 24px; margin-top:5px; margin-bottom: 15px;">[Code]</p>
</a>
</div>
<div class="column">
<a href="https://github.com/iCVTEAM/E3NeRF" target="_blank">
<p class="link" style="font-size: 24px; margin-top:5px; margin-bottom: 15px;">[Dataset]</p>
</a>
</div>
<div class="column"></div>
<div class="column"></div>
</div>
</div>
</div>
</section>
<div style="text-align: center;">
<div class="head_cap">
<p style="font-size: 20px">
<b>The Motivation of E<sup>3</sup>NeRF</b>
</p>
</div>
<div class="container" style="max-width:850px">
<div style="text-align: center;">
<img src="./assets/E3NeRF/motivation.png" class="centerImage">
</div>
</div>
</div>
<section class="hero">
<div class="hero-body">
<div class="container" style="max-width: 800px" >
<h1 style="">Abstract</h1>
<p style="text-align: justify; font-size: 16px;">
Neural Radiance Fields (NeRF) achieves impressive novel view rendering performance by learning implicit 3D representation from sparse view images.
However, it is difficult to reconstruct a sharp NeRF from blurry input that often occurs in the wild.
To solve this problem, we propose a novel Efficient Event-Enhanced NeRF (E<sup>3</sup>NeRF), reconstructing sharp NeRF by utilizing both blurry images and corresponding event streams.
A blur rendering loss and an event rendering loss are introduced, which guide the NeRF training via modeling the physical image motion blur process and event generation process, respectively.
To improve the efficiency of the framework, we further leverage the latent spatial-temporal blur information in the event stream to evenly distribute training over temporal blur and focus training on spatial blur.
Moreover, a camera pose estimation framework for real-world data is built with the guidance of the events, generalizing the method to more practical applications.
Compared to previous image-based and event-based NeRF works, our framework makes more profound use of the internal relationship between events and images.
Extensive experiments on both synthetic data and real-world data demonstrate that E<sup>3</sup>NeRF can effectively learn a sharp NeRF from blurry images, especially for high-speed non-uniform motion and low-light scenes.
</p>
</div>
</div>
</section>
<div style="text-align: center;">
<div class="head_cap">
<p style="font-size: 20px">
<b>The Framework of E<sup>3</sup>NeRF</b>
</p>
</div>
<div class="container" style="max-width:850px">
<div style="text-align: center;">
<img src="./assets/E3NeRF/framework.png" class="centerImage">
</div>
</div>
</div>
<div style="text-align: center;">
<div class="head_cap">
<p style="font-size: 20px">
<b>The Results of E<sup>3</sup>NeRF</b>
</p>
</div>
<div class="container" style="max-width:850px">
<div style="text-align: center;">
<img src="./assets/E3NeRF/gif.gif" class="centerImage">
</div>
</div>
</div>
<div style="text-align: center;">
<div class="head_cap">
<p style="font-size: 26px">
<b>Video Results of Real-World-Challenge Dataset
</p>
</div>
<div class="container" style="max-width:850px">
<div style="text-align: center;">
<iframe width="800" height="350" src="https://www.youtube.com/embed/yz4xdOx90VY?si=v0H0oMSTsvlh3Rb1" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
</div>
</div>
<div class="head_cap">
<p style="font-size: 26px">
<b>Video Results of Synthetic Severely Shaking Dataset
</p>
</div>
<div class="container" style="max-width:850px">
<div style="text-align: center;">
<iframe width="800" height="450" src="https://www.youtube.com/embed/GWOp_ENikMw?si=upDrYUbEtji-khdj" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>
</div>
</div>
</div>
<!--section class="hero is-light" style="background-color:#FFFFFF;">
<div class="hero-body">
<div class="container" style="max-width:800px;margin-bottom:20px;">
<h1>
Qualitative Comparison on Synthetic Data
</h1>
</div>
<div class="container" style="max-width:800px">
<div style="text-align: center;">
<img src="./assets/E2NeRF/qualitative1.png" class="centerImage">
</div>
</div>
<div class="container" style="max-width:800px;margin-bottom:20px;">
<h1>
Qualitative Comparison on Real Data
</h1>
</div>
<div class="container" style="max-width:800px">
<div style="text-align: center;">
<img src="./assets/E2NeRF/qualitative2.png" class="centerImage">
</div>
</div>
</div>
</section>
<section class="hero" style="padding-top:0px;">
<div class="hero-body">
<div class="container" style="max-width:800px;">
<div class="card"-->
<!--header class="card-header">
<p class="card-header-title">
BibTex Citation
</p>
<a class="card-header-icon button-clipboard" style="border:0px; background: inherit;" data-clipboard-target="#bibtex-info" >
<i class="fa fa-copy" height="20px"></i>
</a>
</header>
<div class="card-content">
<pre style="background-color:inherit;padding: 0px;" id="bibtex-info">@article{Ma_Xia_Li_2021,
title={Pyramidal Feature Shrinking for Salient Object Detection},
volume={35},
url={https://ojs.aaai.org/index.php/AAAI/article/view/16331},
number={3},
journal={Proceedings of the AAAI Conference on Artificial Intelligence},
author={Ma, Mingcan and Xia, Changqun and Li, Jia},
year={2021},
month={May},
pages={2311-2318}
}</pre-->
</div>
</section>
<script type="text/javascript" src="assets/scripts/clipboard.min.js"></script>
<script>
new ClipboardJS('.button-clipboard');
</script>
</body>
</html>