I want to use Hololens2's spatial recognition to detect what I put on my desk.

Asked 2 years ago, Updated 2 years ago, 72 views

What do you want to do

·If you have a mug, watch, or cell phone on your desk, attach a bounding box only to the watch as seen from Hololens 2.
·The bounding box is projected in real time, and the size changes depending on the size of the clock
·In addition, the bounding box will follow you even if you put the clock on your wrist.

Problems

"·I was able to recognize ""desk"" and ""floor"" using the ""space awareness"" function of Hololens, but
"  "I think it is difficult to recognize small things like ""something"" (for example, a clock) on the desk."
·Experimentally, I skip Ray from Head and try to infer the shape of the object using collision determination with the spatial recognition layer.
 Space-aware mesh is rough (even high mesh) and can slip through or out of alignment.

If detailed recognition becomes possible, it is assumed that a bounding box can be attached to the target. Do you know any improvement measures or breakthrough measures?Also, I would appreciate it if you could let me know if you have any sample sources for reference.

Device:Hololens2
development environment:Unity

c# unity3d

2022-09-30 20:14

1 Answers

Are you trying to recognize objects from spatial mesh information?

Currently, if it's fine object recognition, I think it's more realistic to make object recognition based on RGB camera image information...

For example, this article contains a sample code: Labeling Toy Aircraft in 3D space using an ONNX model and Windows ML on a HoloLens


2022-09-30 20:14

If you have any answers or tips


© 2024 OneMinuteCode. All rights reserved.