humanoid_lab.html

---
layout: default
---

{% capture crumbs %}{{ site.data[site.active_lang].hl.title }}{% endcapture %}
{% capture crumbs_title %}{{ site.data[site.active_lang].hl.title_short }}{% endcapture %}
{% capture crumbs_subtitle %}
<br />
<a href="{{ site.data[site.active_lang].hl.link }}" target="_blank">
  {{ site.data[site.active_lang].hl.link-name }} <i class="bi bi-globe"></i>
</a>
{% endcapture %}

{% include breadcrumbs.html breadcrumbs=crumbs title=crumbs_title subtitle=crumbs_subtitle %}

<!-- Page Content -->
<div class="container py-5">

  <div class="row mb-5">
    <div class="col-lg-4">
      <img src="{{site.baseurl_root}}/assets/humanoidlab/four_humanoids.jpg" class="img-fluid rounded mb-4">
      <img src="{{site.baseurl_root}}/assets/humanoidlab/combined_logo.png" class="img-fluid rounded">
    </div>

    <div class="col-lg-8">
      <div class="d-flex flex-column align-items-center text-center float-end ms-3 mb-3">
        <a href="{{site.baseurl}}/members/member-kanehiro.html">
          <img src="{{site.baseurl_root}}/assets/members/kanehiro.jpg" class="img-thumbnail" width="120"
            alt="KANEHIRO Fumio">
        </a>
        <div class="text-center">
          <strong>KANEHIRO Fumio</strong><br>
          <b>金広 文男</b><br>
          <span class="text-muted small">f-kanehiro_*_aist.go.jp</span><br>
          <span class="badge bg-info text-dark">{{ site.data[site.active_lang].hl.professor }}</span>
        </div>
      </div>

      <!-- Now the rest of the intro text follows -->
      {% include {{ site.active_lang }}/hl-introduction.html %}
      {% include {{ site.active_lang }}/hl-introduction-2.html %}
    </div>
  </div>

  <!-- Research Section -->
  <div class="row">
    <div class="col-12">
      <h3 class="mb-4">{{ site.data[site.active_lang].hl.research-content }}</h3>
    </div>

    <!-- Research Card 1 -->
    <div class="col-12 mb-4">
      <div class="p-4 bg-primary-subtle rounded">
        <div class="row">
          <div class="col-md-9">
            <h5><strong>Vision-based Belt Manipulation by Humanoid Robot</strong></h5>
            <p>Deformable objects are very common around us in our daily life. Because they have infinitely many degrees
              of freedom, they present a challenging problem in robotics. Inspired by practical industrial applications,
              we present our research on using a humanoid robot to take a long, thin and flexible belt out of a bobbin
              and pick up the bending part of the belt from the ground. By proposing a novel non-prehensile manipulation
              strategy “scraping” which utilizes the friction between the gripper and the surface of the belt, efficient
              manipulation can be achieved. In addition, a 3D shape detection algorithm for deformable objects is used
              during manipulation process. By integrating the novel “scraping” motion and the shape detection algorithm
              into our multi-objective QP-based controller, we show experimentally humanoid robots can complete this
              complex task.</p>
          </div>
          <div class="col-md-3">
            <img src="{{site.baseurl_root}}/assets/humanoidlab/hl_03.jpg" class="img-fluid rounded">
          </div>
        </div>
      </div>
    </div>

    <!-- Research Card 2 -->
    <div class="col-12 mb-4">
      <div class="p-4 bg-primary-subtle rounded">
        <div class="row">
          <div class="col-md-9">
            <h5><strong>sim2real: Learning Humanoids Locomotion using RL</strong></h5>
            <p>Recent advances in deep reinforcement learning (RL) based techniques combined with training in simulation
              have offered a new approach to developing control policies for legged robots. However, the application of
              such approaches to real hardware has largely been limited to quadrupedal robots with direct-drive
              actuators and light-weight bipedal robots with low gear-ratio transmission systems. Application to
              life-sized humanoid robots has been elusive due to the large sim2real gap arising from their large size,
              heavier limbs, and a high gear-ratio transmission systems.</p>
            <p>In this work, we investigate methods for effectively overcoming the sim2real gap issue for large-humanoid
              robots for the goal of deploying RL policies trained in simulation to the real hardware.</p>
            <p>Link to YouTube video: <a href="https://youtu.be/IeUaSsBRbNY">here</a>.</p>
          </div>
          <div class="col-md-3">
            <img src="{{site.baseurl_root}}/assets/humanoidlab/sim2real.png" class="img-fluid rounded">
          </div>
        </div>
      </div>
    </div>

    <!-- Research Card 3 -->
    <div class="col-12 mb-4">
      <div class="p-4 bg-primary-subtle rounded">
        <div class="row">
          <div class="col-md-9">
            <h5><strong>Enhanced Visual Feedback with Decoupled Viewpoint Control in Immersive Teleoperation using
                SLAM</strong></h5>
            <p>During humanoid robot teleoperation, there is a noticeable delay between the motion of the operator’s and
              robot’s head. This latency could cause the lag in visual feedback, which decreases the immersion of the
              system, may cause some dizziness and reduce the efficiency of interaction in teleoperation since operator
              needs to wait for the real-time visual feedback. To solve this problem, we developed a decoupled viewpoint
              control solution which allows the operator to obtain the visual feedback changes with low-latency in VR
              and to increase the reachable visibility range. Besides, we propose a complementary SLAM solution which
              uses the reconstructed mesh to complement the blank area that is not covered by the real-time robot’s
              point cloud visual feedback. The operator could sense the robot head’s real-time orientation by observing
              the pose of the point cloud.</p>
          </div>
          <div class="col-md-3">
            <img src="{{site.baseurl_root}}/assets/humanoidlab/hl_cslam.png" class="img-fluid rounded">
          </div>
        </div>
      </div>
    </div>

  <!-- Research Card 4 -->
  <div class="col-12 mb-4">
      <div class="p-4 bg-primary-subtle rounded">
        <div class="row">
          <div class="col-md-9">
            <h5><strong>Multi-Contact Motions Using the Elbow or Knee of a Humanoid Robot via External Force Estimation</strong></h5>
            <p>Humanoid robots are expected to work like humans. However, there are still some limitations that make this goal difficult.
               We focus on the limitation of using intermediate control points such as elbows or knees. While humans use their intermediate 
               body parts, humanoid robots cannot do so. One reason is that humanoid robots lack force sensors there and typically have force 
               sensors only on the end-effectors. To perform control, it is necessary to measure the output. One solution to this problem is to 
               distribute sensors over the whole body, but this is unrealistic due to mechanical and electrical complexity. We adopt external force
               estimation based on the “Kinetics Observer.” The Kinetics Observer uses IMUs, joint encoders, and force sensors on the end-effectors 
               as measurements, which are standard equipment in many humanoid robots. By applying the external force estimation from the observer, 
               it becomes possible to perform force control without direct force measurement. The picture shows simulation and experimental scene
               demonstrating multi-contact motion without direct force measurement.</p>
          </div>
          <div class="col-md-3">
            <img src="{{site.baseurl_root}}/assets/humanoidlab/hl_estimation_based_force_control.png" class="img-fluid rounded">
          </div>
        </div>
      </div>
    </div>

  <!-- Research Card 5 -->
  <div class="col-12 mb-4">
      <div class="p-4 bg-primary-subtle rounded">
        <div class="row">
          <div class="col-md-9">
            <h5><strong>Monocular Visual SLAM that works with Pre-built Point Cloud Maps using Point and Line Features</strong></h5>
            <p>
              Monocular Visual SLAM is attractive for its low cost and portability, but it suffers from scale ambiguity and typically cannot localize
              directly in heterogeneous prior maps such as dense LiDAR/3D-reconstruction point clouds. This project enables a monocular
              point-line SLAM pipeline to perform absolute localization in a pre-built dense point cloud map. We first extract 3D structural line
              segments from the dense point cloud offline, and then detect 2D line segments in incoming images online. By establishing robust
              2D-3D line correspondences and solving a Perspective-n-Line (PnL) optimization, the system estimates metric absolute camera poses
              during initialization, recovering real-world scale without requiring additional sensors. After monocular initialization,
              multiple PnL poses are accumulated and aligned with the SLAM trajectory to compute a fixed Sim(3) transformation, which is then
              propagated throughout tracking and mapping to maintain globally consistent poses in the dense map coordinate frame.
            </p>
          </div>
          <div class="col-md-3">
            <img src="{{site.baseurl_root}}/assets/humanoidlab/MonocularVisualSLAMthatworkswithPre-builtPointCloudMapsusingPointandLineFeatures.png" class="img-fluid rounded">
          </div>
        </div>
      </div>
    </div>
    
  </div>

  <!-- Past Research Section -->
  <div class="row">
    <div class="col-12">
      <h3 class="mb-4">{{ site.data[site.active_lang].hl.past-research }}</h3>
      <button class="btn btn-outline-primary mb-3" type="button" data-bs-toggle="collapse"
        data-bs-target="#pastResearchCollapse">
        Toggle Past Research
      </button>

      <div class="collapse" id="pastResearchCollapse">
        <!-- Bipedal Walking With Footstep Plans via Reinforcement Learning -->
        <div class="col-12 mb-4">
          <div class="p-4 bg-primary-subtle rounded">
            <div class="row">
              <div class="col-md-9">
                <h5><strong>Bipedal Walking With Footstep Plans via Reinforcement Learning</strong></h5>
                <p>To enable application of RL policy controller humanoid robots in real-world settings, it is crucial
                  to build a system that can achieve robust walking in any direction, on 2D and 3D terrains, and be
                  controllable by a user-command. In this paper, we tackle this problem by learning a policy to follow
                  a given step sequence. The policy is trained with the help of a set of procedurally generated step
                  sequences (also called footstep plans).</p>
                <p>We show that simply feeding the upcoming 2 steps to the policy is sufficient to achieve
                  omnidirectional walking, turning in place, standing, and climbing stairs. Our method employs
                  curriculum learning on the objective function and on sample complexity, and circumvents the need for
                  reference motions or pre-trained weights. We demonstrate the application of our proposed method to
                  learn RL policies for 3 notably distinct robot platforms - HRP5P, JVRC-1, and Cassie, in the MuJoCo.
                </p>
              </div>
              <div class="col-md-3">
                <div class="ratio ratio-16x9">
                  <iframe src="https://www.youtube.com/embed/-mxaQ-f9Ee4" title="Bipedal Walking Video"
                    allowfullscreen></iframe>
                </div>
              </div>
            </div>
          </div>
        </div>

        <!-- Simultaneous localization and mapping(SLAM) in dynamic environment -->
        <div class="col-12 mb-4">
          <div class="p-4 bg-primary-subtle rounded">
            <div class="row">
              <div class="col-md-9">
                <h5><strong>Simultaneous Localization and Mapping (SLAM) in Dynamic Environment</strong></h5>
                <p>SLAM in dynamic environments has become a popular topic. This problem is called dynamic SLAM, where
                  many solutions have been proposed to segment out the dynamic objects that introduce errors to camera
                  tracking and 3D reconstruction.</p>
                <p>However, state-of-the-art methods face challenges in balancing accuracy and speed. We propose a
                  multi-purpose dynamic SLAM framework that allows users to select among various segmentation
                  strategies, each suited for different scenes. If semantic segmentation is used, it supports
                  object-oriented semantic mapping which is highly beneficial for high-level robotic tasks.</p>
              </div>
              <div class="col-md-3">
                <img src="{{site.baseurl_root}}/assets/humanoidlab/hl_01_slam.jpg" class="img-fluid rounded" alt="SLAM">
              </div>
            </div>
          </div>
        </div>

        <!-- 6-DoF Object Pose Estimation -->
        <div class="col-12 mb-4">
          <div class="p-4 bg-primary-subtle rounded">
            <div class="row">
              <div class="col-md-9">
                <h5><strong>6-DoF Object Pose Estimation</strong></h5>
                <p>For a humanoid robot to interact with objects, it must estimate the object’s 6-DoF pose—its 3D
                  position and orientation (roll, pitch, yaw). This is usually done using the robot’s vision sensors.
                </p>
                <p>Accurate estimation is critical for precise grasping and manipulation, especially under challenging
                  lighting, occlusion, or with incomplete object models. This work explores Deep Learning-based
                  techniques to enable robust, accurate pose estimation in such conditions.</p>
              </div>
              <div class="col-md-3">
                <img src="{{site.baseurl_root}}/assets/humanoidlab/hl_02.png" class="img-fluid rounded"
                  alt="6DoF Pose Estimation">
              </div>
            </div>
          </div>
        </div>
      </div>
    </div>
  </div>


  <!-- Publications -->
  <div class="row mt-5">
    <div class="col-12">
      <h3 class="mb-4">{{ site.data[site.active_lang].pages.publications }}</h3>
      {% include publications_table.html project_id="humanoid_lab" %}
    </div>
  </div>

  <!-- Students -->
  <div class="row mt-5">
    <div class="col-12">
      <h3 class="mb-4">{{ site.data[site.active_lang].hl.student-members }}</h3>
      <div class="table-responsive">
        <table class="table table-bordered table-striped">
          <thead class="table-light">
            <tr>
              <th>{{ site.data[site.active_lang].hl.student-name }}</th>
              <th>{{ site.data[site.active_lang].hl.student-grade }}</th>
              <th>{{ site.data[site.active_lang].hl.student-email }}</th>
            </tr>
          </thead>
          <tbody>
            {% for student in site.data[site.active_lang].hl.students %}
            <tr>
              <td>
                {% if student.website %}
                <a href="{{ student.website }}">{{ student.name }}</a>
                {% elsif student.id %}
                <a href="{{ site.baseurl }}/members/member-{{ student.id }}.html">{{ student.name }}</a>
                {% else %}
                {{ student.name }}
                {% endif %}
              </td>
              <td>{{ student.grade }}</td>
              <td>{{ student.email }}</td>
            </tr>
            {% endfor %}
          </tbody>
        </table>
      </div>
    </div>
  </div>

  <!-- Location -->
  <div class="row mt-5">
    <div class="col-12">
      <h3 class="mb-4">{{ site.data[site.active_lang].hl.location }}</h3>
      <div class="ratio ratio-16x9">
        <iframe
          src="https://www.google.com/maps/embed?pb=!1m28!1m12!1m3!1d25795.692830871016!2d140.1074103178154!3d36.082234076560354!2m3!1f0!2f0!3f0!3m2!1i1024!2i768!4f13.1!4m13!3e6!4m5!1s0x60220bff99f57b0b%3A0x1cad40e7632fb4b8!2zVW5pdmVyc2l0eSBvZiBUc3VrdWJhIOetkeazouWkpw!3m2!1d36.103866599999996!2d140.1020979!4m5!1s0x60220cc567b824f5%3A0xecc14922713a4044!2z44CSMzA1LTg1NjAgSWJhcmFraSwgVHN1a3ViYSwgVW1lem9ubywgMSBDaG9tZeKIkjEtMSDkuK3lpK7nrKwx44Gk44GP44Gw5pys6YOo5oOF5aCx5oqA6KGT5YWx5ZCM56CU56m25qOfIFRzdWt1YmEgQ2VudGVyLCBBSVNUOiBOYXRpb25hbCBJbnN0aXR1dGUgb2YgQWR2YW5jZWQgSW5kdXN0cmlhbCBTY2llbmNlIGFuZCBUZWNobm9sb2d5!3m2!1d36.0624307!2d140.1356783!5e0!3m2!1sen!2sjp!4v1650429789876!5m2!1sen!2sjp"
          style="border:0;" allowfullscreen loading="lazy"></iframe>
      </div>
    </div>
  </div>

  <!-- Visitor Counter -->
  <div class="row mt-5">
    <div class="col text-end">
      <img
        src="https://hits.seeyoufarm.com/api/count/incr/badge.svg?url=https%3A%2F%2Funit.aist.go.jp%2Fjrl-22022%2Fen%2Fhumanoid_lab.html&count_bg=%2379C83D&title_bg=%23555555&icon=&icon_color=%23E7E7E7&title=Hits&edge_flat=false" />
      <div>(since 03/2023)</div>
    </div>
  </div>

</div>