The first workshop on

SEAI: Simulation Technology for Embodied AI

Saturday, October 16, ICCV 2021 Virtual


Recently, we have witnessed a surge in interest in embodied agents that could explore and interact with complex and unstructured environments. The design and control of such intelligent agents raise a large number of interesting research questions.It is worth noting that, recent embodied AI learning works are mostly first trained in simulated environments (e.g., AI Habitat, AI2THOR, RoboSuite, SAPIEN) and then transferred to the real world. However, the simulation for embodied AI landscape is quite fragmented and immature so far. It is yet an open question how to build a robotic-ready simulation environment, which is typically multi-disciplinary involving challenges from both computer vision, computer graphics, machine learning and robotics.

The key goal of this workshop is to assemble experts in simulation technology and embodied AI learning to synchronize and coordinate the efforts. First of all, this workshop would help simulation researchers to understand embodied AI learning and tailor their techniques for more effective and convenient learning. More importantly, embodied AI researchers will improve their understanding to the limitation (e.g., domain gap) of current simulation techniques, and use these simulators in a proper way. Finally, the workshop has also invited learning-based simulation experts, and we hope to take the chance to discuss the possible role of learning-based techniques in improving embodied AI learning environments.

The workshop will be held virtually in conjunction with ICCV 2021. We invited keynote speakers to cover a wide range of topics from physics-based modeling and simulation, learning-based simulation, human body simulation to embodied vision and robot learning. In addition, the workshop will feature a spotlight section for various existing simulation platforms, a spotlight section for accepted papers, a poster session and a live panel discussion.

Call for Papers

Topics: We invite both 4-page short papers and 2-page abstracts for works falling onto the simulation for embodied AI landscape. Paper topics may include but are not limited to:

Submission: All submissions will be handled electronically via the workshop CMT website. Submissions should use the ICCV template. (If the download link does not work, please find the author kit at the ICCV 2021 website Submissions could be either 4-page short papers or 2-page abstracts in the ICCV format, excluding references, acknowledgements, and supplementary materials. All the accepted submissions will be published separately from the main conference in the post-proceedings by default, though authors could indicate explicitly if they want to opt out the post-proceedings. Dual submission is allowed, but must be explicitly stated at the time of submission and will not be included in the post-proceedings. Reviewing will be double-blind. Each submission will be reviewed for originality, significance, clarity, soundness, relevance and technical contents. Please contact for any concerns.

Timeline (11:59 PM Pacific Standard Time)

May 4Announcement and call for submissions
July 26Paper submission deadline
Aug. 9Review decisions announced
Aug. 16Camera ready deadline

Workshop Schedule (EDT)

Start End Event Speaker Video
8:50am9:00amOpening Remarkrecording
9:00am9:30amFast and Differentiable Physical Simulations using the Taichi Programming LanguageYuanming Hurecording
9:30am10:00amDifferentiable Physics Simulations for Deep LearningNils Thuereyrecording
10:00am10:40amSpotlight Section: Simulation Environments for Embodied AIrecording
10:40am11:10amReinforcement Learning Based Control for Underactuated Task in Fluid SimulatorsBo Renrecording
11:10am11:40amEfficient Nonlinear FEM with Deep NetworksYin Yangrecording
11:40am12:10pmFrictional Contact with Guarantees and SmoothnessChenfanfu Jiangrecording
12:10pm1:30pmLunch Break
1:30pm2:00pmSelf-Simulating SystemsHod Lipsonrecording
2:00pm2:30pmBetter Visual Representations for Better Simulation to Real TransferLerrel Pintorecording
2:30pm3:00pmOn the Use of Simulation in Robotics and Autonomous VehiclesDan Negrutrecording
3:00pm3:40pmPaper Spotlight Presentationrecording
3:40pm4:10pmAI-augmented Physics ModelsKaren Liurecording
4:10pm4:40pmDifferentiable Physics for Learning and ControlMing Linrecording
4:40pm5:10pmPanel Discussion and Community Buildingrecording

Spotlight Presentation

3:00pm3:08pmA Bayesian Perspective on the Sim-To-Real GapFabio Ramos
3:08pm3:16pmSPACE: A Simulator for Physical Interactions and Causal Learning in 3D EnvironmentsJiafei Duan
3:16pm3:24pmV2X-Sim: A Virtual Collaborative Perception Dataset for Autonomous DrivingYiming Li
3:24pm3:32pmGes-THOR: Communicative Learning with Natural Gestures for Embodied Navigation Agents with Human-in-the-SceneQi Wu
3:32pm3:40pmPlan2Scene: Converting Floorplans to 3D ScenesQirui Wu

Invited Speakers



© 2021 SEAI: Simulation Technology for Embodied AI