-
Notifications
You must be signed in to change notification settings - Fork 2
Expand file tree
/
Copy pathE2NeRF.html
More file actions
188 lines (166 loc) · 7.96 KB
/
Copy pathE2NeRF.html
File metadata and controls
188 lines (166 loc) · 7.96 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>E2NeRF</title>
<link rel="stylesheet" type="text/css" href="assets/scripts/bulma.min.css">
<link rel="stylesheet" type="text/css" href="assets/scripts/theme.css">
<link rel="stylesheet" type="text/css" href="https://cdn.bootcdn.net/ajax/libs/font-awesome/4.7.0/css/font-awesome.min.css">
</head>
<body>
<section class="hero is-light" style="">
<div class="hero-body" style="padding-top: 50px;">
<div class="container" style="text-align: center;margin-bottom:5px;">
<h1 class="title">
E<sup>2</sup>NeRF: Event Enhanced Neural Radiance Fields from Blurry Images
</h1>
<div class="author"><p style="font-size: 22px">Yunshan Qi<sup>1</sup>       Lin Zhu<sup>2 *</sup>       Yu Zhang<sup>3</sup>       Jia Li<sup>1,4 *</sup></p></div>
<div class="aff">
<p style="font-size: 20px"><sup>1</sup>Beihang University    <sup>2</sup>Beijing Institute of Technology    <sup>3</sup>SenseTime and Tetras.AI    <sup>4</sup>Peng Cheng Laboratory</p>
<p> </p>
</div>
<div class="group"><p style="font-size: 22px"><a href="http://cvteam.net/"><sup>*</sup>CVTEAM</a></p></div>
<div class="con">
<p style="font-size: 24px; margin-top:5px; margin-bottom: 15px;">
<b>Accepted by International Conference on Computer Vision (ICCV) 2023</b>
</p>
</div>
<div class="columns">
<div class="column"></div>
<div class="column"></div>
<div class="column">
<a href="assets/E2NeRF/Papers.pdf" target="_blank">
<p class="link" style="font-size: 24px; margin-top:5px; margin-bottom: 15px;">[Paper]</p>
</a>
</div>
<div class="column">
<a href="https://github.com/iCVTEAM/E2NeRF" target="_blank">
<p class="link" style="font-size: 24px; margin-top:5px; margin-bottom: 15px;">[Code]</p>
</a>
</div>
<div class="column">
<a href="https://github.com/iCVTEAM/E2NeRF" target="_blank">
<p class="link" style="font-size: 24px; margin-top:5px; margin-bottom: 15px;">[Dataset]</p>
</a>
</div>
<div class="column">
<a href="https://cvteam.buaa.edu.cn/papers/2023-ICCV-Sup.pdf" target="_blank">
<p class="link" style="font-size: 24px; margin-top:5px; margin-bottom: 15px;">[Sup]</p>
</a>
</div>
<div class="column"></div>
<div class="column"></div>
</div>
</div>
</div>
</section>
<div style="text-align: center;">
<div class="head_cap">
<p style="font-size: 20px">
<b>The Framework of E<sup>2</sup>NeRF</b>
</p>
</div>
<div class="container" style="max-width:850px">
<div style="text-align: center;">
<img src="./assets/E2NeRF/method.png" class="centerImage">
</div>
</div>
</div>
<div style="text-align: center;">
<div class="head_cap">
<p style="font-size: 20px">
<b>The Results of E<sup>2</sup>NeRF</b>
</p>
</div>
<div class="container" style="max-width:850px">
<div style="text-align: center;">
<img src="./assets/E2NeRF/gif4.gif" class="centerImage">
</div>
</div>
</div>
<section class="hero">
<div class="hero-body">
<div class="container" style="max-width: 800px" >
<h1 style="">Abstract</h1>
<p style="text-align: justify; font-size: 16px;">
Neural Radiance Fields (NeRF) achieves impressive rendering performance by learning volumetric 3D representation from several images of different views.
However, it is difficult to reconstruct a sharp NeRF from blurry input as often occurred in the wild.
To solve this problem, we propose a novel Event-Enhanced NeRF (E<sup>2</sup>NeRF) by utilizing the combination data of a bio-inspired event camera and a standard RGB camera.
To effectively introduce event stream into the learning process of neural volumetric representation, we propose a blur rendering loss and an event rendering loss, which guide the network via modelling real blur process and event generation process, respectively.
Moreover, a camera pose estimation framework for real-world data is built with the guidance of event stream to generalize the method to practical applications.
In contrast to previous image-based or event-based NeRF, our framework effectively utilizes the internal relationship between events and images.
As a result, E<sup>2</sup>NeRF not only achieves image deblurring but also achieves high-quality novel view image generation.
Extensive experiments on both synthetic data and real-world data demonstrate that E<sup>2</sup>NeRF can effectively learn a sharp NeRF from blurry images, especially in complex and low-light scenes.
</p>
</div>
</div>
</section>
<section class="hero">
<div class="hero-body">
<div class="container" style="max-width:800px;margin-bottom:20px;">
<h1>
Supplementary Video
</h1>
</div>
<div class="container" style="max-width: 800px" >
<iframe width="800" height="450" src="https://www.youtube.com/embed/EvTHcLFX8yY?si=3bXlSkxSNiIYt_Vz" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>
</div>
</div>
</section>
<section class="hero is-light" style="background-color:#FFFFFF;">
<div class="hero-body">
<div class="container" style="max-width:800px;margin-bottom:20px;">
<h1>
Qualitative Comparison on Synthetic Data
</h1>
</div>
<div class="container" style="max-width:800px">
<div style="text-align: center;">
<img src="./assets/E2NeRF/qualitative1.png" class="centerImage">
</div>
</div>
<div class="container" style="max-width:800px;margin-bottom:20px;">
<h1>
Qualitative Comparison on Real Data
</h1>
</div>
<div class="container" style="max-width:800px">
<div style="text-align: center;">
<img src="./assets/E2NeRF/qualitative2.png" class="centerImage">
</div>
</div>
</div>
</section>
<section class="hero" style="padding-top:0px;">
<div class="hero-body">
<div class="container" style="max-width:800px;">
<div class="card">
<header class="card-header">
<p class="card-header-title">
BibTex Citation
</p>
<a class="card-header-icon button-clipboard" style="border:0px; background: inherit;" data-clipboard-target="#bibtex-info" >
<i class="fa fa-copy" height="20px"></i>
</a>
</header>
<div class="card-content">
<pre style="background-color:inherit;padding: 0px;" id="bibtex-info">
@inproceedings{qi2023e2nerf,
title={E2nerf: Event enhanced neural radiance fields from blurry images},
author={Qi, Yunshan and Zhu, Lin and Zhang, Yu and Li, Jia},
booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
pages={13254--13264},
year={2023}
}
</pre>
</div>
</div>
</div>
</div>
</section>
<script type="text/javascript" src="assets/scripts/clipboard.min.js"></script>
<script>
new ClipboardJS('.button-clipboard');
</script>
</body>
</html>